2024-05-09T06:16:00.855 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T06:16:00.855 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/orig.config.yaml 2024-05-09T06:16:00.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:16:00.991 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698532' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeKuSuvCSTl2NOkCWajvBdgPOXctLAGTsGHL9ZQm//uM7SwdJMUl9tPGxm5hv398tEOGQdsNRqE2tarekbC8dU= smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAE21YDipqInL/6q0E+gLCZIjnPIPuvSv5t8AujAjJbR26SK46h59tFXCDS+N+rI8ZF+tFcukC7mUQiL5sKICJM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/fs/functional/overrides/pg_health.yaml meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T06:16:00.992 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T06:16:00.993 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T06:16:00.993 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T06:16:00.994 INFO:teuthology.task.internal:Checking packages... 2024-05-09T06:16:01.021 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T06:16:01.021 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:16:01.021 INFO:teuthology.packaging:ref: None 2024-05-09T06:16:01.021 INFO:teuthology.packaging:tag: None 2024-05-09T06:16:01.021 INFO:teuthology.packaging:branch: squid 2024-05-09T06:16:01.021 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:16:01.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T06:16:01.228 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T06:16:01.230 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T06:16:01.237 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T06:16:01.237 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T06:16:01.243 INFO:teuthology.task.internal:Saving configuration 2024-05-09T06:16:01.260 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T06:16:01.266 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:16:01.288 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:20.359972', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeKuSuvCSTl2NOkCWajvBdgPOXctLAGTsGHL9ZQm//uM7SwdJMUl9tPGxm5hv398tEOGQdsNRqE2tarekbC8dU='} 2024-05-09T06:16:01.309 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:20.361390', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAE21YDipqInL/6q0E+gLCZIjnPIPuvSv5t8AujAjJbR26SK46h59tFXCDS+N+rI8ZF+tFcukC7mUQiL5sKICJM='} 2024-05-09T06:16:01.309 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T06:16:01.316 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-09T06:16:01.316 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-09T06:16:01.316 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T06:16:01.385 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe99ffaa160>, signals=[15]) 2024-05-09T06:16:01.386 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T06:16:01.392 INFO:teuthology.task.internal:Opening connections... 2024-05-09T06:16:01.392 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:16:01.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:16:01.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:16:01.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:16:01.547 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T06:16:01.554 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-05-09T06:16:01.572 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-05-09T06:16:01.573 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:16:01.631 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:16:01.632 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:16:01.633 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-05-09T06:16:01.652 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-05-09T06:16:01.670 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-05-09T06:16:01.670 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-05-09T06:16:01.728 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-05-09T06:16:01.728 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-05-09T06:16:01.728 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-05-09T06:16:01.728 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:16:01.729 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:16:01.729 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-05-09T06:16:01.755 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T06:16:01.764 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T06:16:01.770 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T06:16:01.770 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:16:01.773 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:16:01.791 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T06:16:01.797 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T06:16:01.797 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:16:01.831 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:16:01.865 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T06:16:01.916 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T06:16:01.931 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-09T06:16:01.931 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-09T06:16:01.931 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:16:01.932 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:16:01.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:16:01.949 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-05-09T06:16:01.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:16:01.950 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-09T06:16:02.007 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:16:02.007 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.14.0-437.el9.x86_64 2024-05-09T06:16:02.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2024-05-09T06:16:02.009 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:16:02.009 INFO:teuthology.task.kernel:Running kernel on smithi006: 5.14.0-437.el9.x86_64 2024-05-09T06:16:02.009 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2024-05-09T06:16:04.228 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-09T06:16:04.410 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-09T06:16:06.127 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-05-09T06:16:12.486 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-05-09T06:16:21.345 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 2.1 kB/s | 16 kB 00:07 2024-05-09T06:16:22.987 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-09T06:16:23.520 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-05-09T06:16:28.304 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T06:16:28.954 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-09T06:16:30.394 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:16:30.394 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:16:30.495 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:16:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:Install 4 Packages 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 78 M 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 123 M 2024-05-09T06:16:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-05-09T06:16:30.521 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-09T06:16:31.964 INFO:teuthology.orchestra.run.smithi001.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 387 kB/s | 484 kB 00:01 2024-05-09T06:16:32.948 INFO:teuthology.orchestra.run.smithi001.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 6.9 MB/s | 15 MB 00:02 2024-05-09T06:16:33.798 INFO:teuthology.orchestra.run.smithi001.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 11 MB/s | 34 MB 00:03 2024-05-09T06:16:34.706 INFO:teuthology.orchestra.run.smithi001.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-09T06:16:34.706 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:16:34.707 INFO:teuthology.orchestra.run.smithi001.stdout:Total 19 MB/s | 78 MB 00:04 2024-05-09T06:16:35.416 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-05-09T06:16:35.492 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-05-09T06:16:35.839 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-05-09T06:16:35.840 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-05-09T06:16:36.820 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-05-09T06:16:36.820 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-05-09T06:16:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-05-09T06:16:40.014 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:16:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:16:40.814 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-09T06:16:42.895 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:16:42.895 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:16:42.909 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:16:43.042 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:16:43.115 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-05-09T06:16:43.124 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-05-09T06:16:43.124 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-05-09T06:16:43.124 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-05-09T06:16:43.124 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout:Install 4 Packages 2024-05-09T06:16:43.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:16:43.126 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 78 M 2024-05-09T06:16:43.126 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 123 M 2024-05-09T06:16:43.126 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-05-09T06:16:43.432 INFO:teuthology.orchestra.run.smithi006.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 4.7 MB/s | 484 kB 00:00 2024-05-09T06:16:43.966 INFO:teuthology.orchestra.run.smithi006.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 24 MB/s | 15 MB 00:00 2024-05-09T06:16:44.541 INFO:teuthology.orchestra.run.smithi006.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 28 MB/s | 34 MB 00:01 2024-05-09T06:16:44.991 INFO:teuthology.orchestra.run.smithi006.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 18 MB/s | 28 MB 00:01 2024-05-09T06:16:44.991 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:16:44.991 INFO:teuthology.orchestra.run.smithi006.stdout:Total 42 MB/s | 78 MB 00:01 2024-05-09T06:16:45.702 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-05-09T06:16:46.105 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-05-09T06:16:46.105 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-05-09T06:16:47.052 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-05-09T06:16:47.053 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-05-09T06:16:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:16:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:16:49.615 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-05-09T06:16:50.257 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:16:50.350 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:16:53.154 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:16:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:16:53.296 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:16:58.376 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:16:58.424 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:03.518 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:17:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:17:25.432 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-09T06:17:25.819 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:25.897 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.898 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:25.898 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:17:25.899 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:17:25.899 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:17:25.899 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-09T06:17:25.899 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:17:25.899 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:25.949 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:25.950 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:25.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 68.el9 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8264254 2024-05-09T06:17:25.981 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:17:25.982 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:17:25.983 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:17:25.983 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:17:25.983 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-05-09T06:17:25.983 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:17:25.983 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:17:25.984 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:17:25.984 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:17:25.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:17:26.633 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-05-09T06:17:27.935 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:17:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-05-09T06:17:27.963 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-05-09T06:17:27.993 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.wj8kyNhKnd 2024-05-09T06:17:27.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wj8kyNhKnd 2024-05-09T06:17:28.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.wj8kyNhKnd 2024-05-09T06:17:28.193 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.wj8kyNhKnd is 6KB 2024-05-09T06:17:28.240 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.wj8kyNhKnd 2024-05-09T06:17:28.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:17:28.316 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:17:28.316 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:17:28.316 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:17:28.316 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:17:28.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:17:28.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-05-09T06:17:34.613 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:34.654 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:35.035 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:35.035 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:17:35.035 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:35.035 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:35.980 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:17:35.981 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-05-09T06:17:36.318 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:36.442 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:36.443 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:36.443 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:17:36.443 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:17:36.443 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:17:36.443 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-09T06:17:36.443 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:17:36.443 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:36.495 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:36.495 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:36.495 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2024-05-09T06:17:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:17:36.528 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-05-09T06:17:36.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:17:36.529 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:17:36.529 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:17:36.530 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:17:36.530 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:17:37.219 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-05-09T06:17:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:17:38.647 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-05-09T06:17:38.650 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-09T06:17:38.667 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.x8c9QpxxOG 2024-05-09T06:17:38.667 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.x8c9QpxxOG 2024-05-09T06:17:38.734 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.x8c9QpxxOG 2024-05-09T06:17:38.870 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.x8c9QpxxOG is 6KB 2024-05-09T06:17:38.919 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.x8c9QpxxOG 2024-05-09T06:17:38.933 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:17:38.996 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:17:38.996 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:17:38.996 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:17:38.996 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:17:38.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:17:39.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-05-09T06:17:58.478 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T06:17:58.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:17:58.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:09.151 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T06:18:09.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:18:09.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:16.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-05-09T06:18:25.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:18:25.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:27.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-05-09T06:18:36.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:18:36.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:26.051 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T06:19:36.509 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T06:19:38.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:19:38.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:38.424 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-05-09T06:19:38.820 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:19:38.820 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:19:38.820 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-05-09T06:19:38.837 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:19:38.837 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:19:38.837 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:19:38.837 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:19:39.838 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T06:19:39.838 DEBUG:teuthology.orchestra.run.smithi001:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T06:19:39.921 INFO:teuthology.orchestra.run.smithi001.stdout:ttyS1 2024-05-09T06:19:39.952 DEBUG:teuthology.parallel:result is None 2024-05-09T06:19:48.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:19:48.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:48.869 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-09T06:19:49.227 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:19:49.228 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:19:49.228 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-09T06:19:49.244 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:19:49.244 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:19:49.245 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:19:49.245 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:19:50.246 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T06:19:50.246 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T06:19:50.318 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2024-05-09T06:19:50.356 DEBUG:teuthology.parallel:result is None 2024-05-09T06:19:50.356 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T06:19:50.363 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T06:19:50.363 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:19:50.366 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:19:50.386 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T06:19:50.392 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T06:19:50.400 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T06:19:50.400 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:19:50.426 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:19:50.470 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T06:19:50.477 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T06:19:50.477 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:19:50.507 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:19:50.539 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:19:50.542 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:19:50.551 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:19:50.554 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:19:50.556 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T06:19:50.563 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T06:19:50.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:19:50.596 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:19:50.626 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T06:19:50.635 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T06:19:50.635 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:19:50.664 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:19:50.683 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:19:50.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:19:50.813 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:19:50.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:19:50.939 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:19:50.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:19:51.006 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:19:51.049 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:19:51.103 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:19:51.187 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:19:51.233 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:19:51.234 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:19:51.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-05-09T06:19:51.307 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-05-09T06:19:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:19:51.392 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:19:51.873 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T06:19:51.882 INFO:teuthology.task.internal:Starting timer... 2024-05-09T06:19:51.882 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T06:19:51.891 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T06:19:51.901 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-09T06:19:51.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-05-09T06:19:51.974 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-05-09T06:19:51.977 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-05-09T06:19:52.066 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-05-09T06:19:52.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T06:19:52.068 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-05-09T06:19:52.092 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-05-09T06:19:52.092 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-05-09T06:19:52.171 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-05-09T06:19:52.172 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi006.front.sepia.ceph.com': 'permissive'} 2024-05-09T06:19:52.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:19:52.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:19:52.202 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:19:52.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:19:52.232 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T06:19:52.232 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:19:52.271 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:19:52.301 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T06:19:52.311 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T06:19:52.327 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T06:19:52.328 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi006.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T06:24:16.611 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi006.front.sepia.ceph.com')] 2024-05-09T06:24:16.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:24:16.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:24:16.694 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-05-09T06:24:16.771 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T06:24:16.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:24:16.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:24:16.853 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-09T06:24:16.931 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T06:24:16.931 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T06:24:16.941 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T06:24:16.941 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:24:16.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:24:16.944 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:24:16.944 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:24:16.974 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:24:16.990 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:24:17.003 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:24:17.015 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-05-09T06:24:17.018 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:24:17.027 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-05-09T06:24:17.042 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:24:17.044 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-05-09T06:24:17.055 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-05-09T06:24:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:24:17.070 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:24:17.085 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:24:17.111 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.140 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.144 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:17.145 INFO:teuthology.run_tasks:Running task install... 2024-05-09T06:24:17.153 DEBUG:teuthology.task.install:project ceph 2024-05-09T06:24:17.154 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T06:24:17.154 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T06:24:17.154 INFO:teuthology.task.install:Using flavor: default 2024-05-09T06:24:17.160 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T06:24:17.160 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T06:24:17.160 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:24:17.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:17.163 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:24:17.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:17.319 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T06:24:17.319 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T06:24:17.387 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T06:24:17.387 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T06:24:17.575 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:24:17.575 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:24:17.575 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:24:17.604 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T06:24:17.604 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:24:17.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:24:17.698 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:24:17.698 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:24:17.698 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:24:17.729 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T06:24:17.730 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:24:17.752 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:24:17.801 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:24:17.818 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-05-09T06:24:17.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-05-09T06:24:17.883 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:24:17.954 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-05-09T06:24:17.956 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-05-09T06:24:18.061 INFO:teuthology.orchestra.run.smithi001.stdout:54 files removed 2024-05-09T06:24:18.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T06:24:18.204 INFO:teuthology.orchestra.run.smithi006.stdout:54 files removed 2024-05-09T06:24:18.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T06:24:18.962 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 142 kB/s | 83 kB 00:00 2024-05-09T06:24:19.127 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 137 kB/s | 83 kB 00:00 2024-05-09T06:24:19.495 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-09T06:24:19.536 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-09T06:24:19.996 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-09T06:24:20.062 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-09T06:24:20.737 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-09T06:24:21.647 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.1 MB 00:01 2024-05-09T06:24:23.839 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-05-09T06:24:23.955 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-05-09T06:24:28.131 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-09T06:24:28.734 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-05-09T06:24:29.548 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 16 kB 00:00 2024-05-09T06:24:30.291 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-05-09T06:24:31.275 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-09T06:24:32.226 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-09T06:24:36.575 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T06:24:37.510 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-05-09T06:24:39.003 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.004 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.015 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T06:24:39.016 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.017 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T06:24:39.018 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T06:24:39.019 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T06:24:39.022 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T06:24:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T06:24:39.088 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T06:24:39.102 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T06:24:39.103 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T06:24:39.104 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T06:24:39.105 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:24:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T06:24:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T06:24:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:24:39.109 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:24:39.110 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T06:24:39.111 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:24:39.112 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T06:24:39.113 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout:Install 148 Packages 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 8 Packages 2024-05-09T06:24:39.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:24:39.115 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 225 M 2024-05-09T06:24:39.115 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-05-09T06:24:39.979 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.980 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.992 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T06:24:39.994 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T06:24:39.994 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T06:24:39.994 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T06:24:39.996 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T06:24:39.998 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T06:24:39.999 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T06:24:40.063 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T06:24:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T06:24:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T06:24:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:24:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T06:24:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:24:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T06:24:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T06:24:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:24:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:24:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:24:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T06:24:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 8 Packages 2024-05-09T06:24:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:24:40.096 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 225 M 2024-05-09T06:24:40.096 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-05-09T06:24:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-09T06:24:41.070 INFO:teuthology.orchestra.run.smithi001.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-09T06:24:41.161 INFO:teuthology.orchestra.run.smithi006.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 847 kB/s | 889 kB 00:01 2024-05-09T06:24:41.328 INFO:teuthology.orchestra.run.smithi006.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 870 kB/s | 145 kB 00:00 2024-05-09T06:24:41.895 INFO:teuthology.orchestra.run.smithi001.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.1 MB/s | 889 kB 00:00 2024-05-09T06:24:42.037 INFO:teuthology.orchestra.run.smithi001.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.0 MB/s | 145 kB 00:00 2024-05-09T06:24:42.853 INFO:teuthology.orchestra.run.smithi006.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-09T06:24:42.995 INFO:teuthology.orchestra.run.smithi001.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-05-09T06:24:43.228 INFO:teuthology.orchestra.run.smithi006.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-05-09T06:24:43.454 INFO:teuthology.orchestra.run.smithi001.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-05-09T06:24:43.612 INFO:teuthology.orchestra.run.smithi001.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-05-09T06:24:43.678 INFO:teuthology.orchestra.run.smithi006.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-05-09T06:24:44.589 INFO:teuthology.orchestra.run.smithi001.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-05-09T06:24:46.612 INFO:teuthology.orchestra.run.smithi006.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 1.4 MB/s | 4.7 MB 00:03 2024-05-09T06:24:46.946 INFO:teuthology.orchestra.run.smithi001.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 4.3 MB/s | 10 MB 00:02 2024-05-09T06:24:46.997 INFO:teuthology.orchestra.run.smithi001.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 507 kB/s | 25 kB 00:00 2024-05-09T06:24:50.062 INFO:teuthology.orchestra.run.smithi006.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 2.0 MB/s | 21 MB 00:10 2024-05-09T06:24:50.162 INFO:teuthology.orchestra.run.smithi006.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 252 kB/s | 25 kB 00:00 2024-05-09T06:24:51.063 INFO:teuthology.orchestra.run.smithi001.stdout:(10/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 2.3 MB/s | 17 MB 00:07 2024-05-09T06:24:51.246 INFO:teuthology.orchestra.run.smithi001.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 1.2 MB/s | 222 kB 00:00 2024-05-09T06:24:51.313 INFO:teuthology.orchestra.run.smithi001.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 406 kB/s | 27 kB 00:00 2024-05-09T06:24:51.389 INFO:teuthology.orchestra.run.smithi001.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 423 kB/s | 32 kB 00:00 2024-05-09T06:24:52.131 INFO:teuthology.orchestra.run.smithi001.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 1.0 MB/s | 746 kB 00:00 2024-05-09T06:24:52.314 INFO:teuthology.orchestra.run.smithi001.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 892 kB/s | 163 kB 00:00 2024-05-09T06:24:52.481 INFO:teuthology.orchestra.run.smithi001.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 761 kB/s | 127 kB 00:00 2024-05-09T06:24:52.679 INFO:teuthology.orchestra.run.smithi006.stdout:(10/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 1.9 MB/s | 17 MB 00:09 2024-05-09T06:24:52.929 INFO:teuthology.orchestra.run.smithi006.stdout:(11/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 1.6 MB/s | 10 MB 00:06 2024-05-09T06:24:52.954 INFO:teuthology.orchestra.run.smithi006.stdout:(12/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 807 kB/s | 222 kB 00:00 2024-05-09T06:24:52.956 INFO:teuthology.orchestra.run.smithi001.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 1.0 MB/s | 500 kB 00:00 2024-05-09T06:24:53.013 INFO:teuthology.orchestra.run.smithi006.stdout:(13/156): cephfs-shell-19.0.0-2515.g96805dcf.el 324 kB/s | 27 kB 00:00 2024-05-09T06:24:53.030 INFO:teuthology.orchestra.run.smithi006.stdout:(14/156): libcephfs-devel-19.0.0-2515.g96805dcf 421 kB/s | 32 kB 00:00 2024-05-09T06:24:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 980 kB/s | 163 kB 00:00 2024-05-09T06:24:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.0 MB/s | 127 kB 00:00 2024-05-09T06:24:53.647 INFO:teuthology.orchestra.run.smithi006.stdout:(17/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 1.1 MB/s | 746 kB 00:00 2024-05-09T06:24:53.672 INFO:teuthology.orchestra.run.smithi006.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 1.4 MB/s | 500 kB 00:00 2024-05-09T06:24:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 493 kB/s | 45 kB 00:00 2024-05-09T06:24:53.915 INFO:teuthology.orchestra.run.smithi006.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 900 kB/s | 135 kB 00:00 2024-05-09T06:24:54.074 INFO:teuthology.orchestra.run.smithi006.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.0 MB/s | 162 kB 00:00 2024-05-09T06:24:54.390 INFO:teuthology.orchestra.run.smithi006.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 1.0 MB/s | 321 kB 00:00 2024-05-09T06:24:54.616 INFO:teuthology.orchestra.run.smithi006.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 1.3 MB/s | 298 kB 00:00 2024-05-09T06:24:54.708 INFO:teuthology.orchestra.run.smithi006.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.1 MB/s | 99 kB 00:00 2024-05-09T06:24:54.816 INFO:teuthology.orchestra.run.smithi006.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 788 kB/s | 85 kB 00:00 2024-05-09T06:24:56.923 INFO:teuthology.orchestra.run.smithi001.stdout:(18/156): ceph-common-19.0.0-2515.g96805dcf.el9 1.3 MB/s | 21 MB 00:16 2024-05-09T06:24:57.040 INFO:teuthology.orchestra.run.smithi001.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 388 kB/s | 45 kB 00:00 2024-05-09T06:24:57.140 INFO:teuthology.orchestra.run.smithi001.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.3 MB/s | 135 kB 00:00 2024-05-09T06:24:57.215 INFO:teuthology.orchestra.run.smithi001.stdout:(21/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 1.2 MB/s | 5.3 MB 00:04 2024-05-09T06:24:57.241 INFO:teuthology.orchestra.run.smithi001.stdout:(22/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.6 MB/s | 162 kB 00:00 2024-05-09T06:24:57.316 INFO:teuthology.orchestra.run.smithi001.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 298 kB 00:00 2024-05-09T06:24:57.417 INFO:teuthology.orchestra.run.smithi001.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 995 kB/s | 99 kB 00:00 2024-05-09T06:24:57.492 INFO:teuthology.orchestra.run.smithi001.stdout:(25/156): python3-rados-19.0.0-2515.g96805dcf.e 1.1 MB/s | 321 kB 00:00 2024-05-09T06:24:57.508 INFO:teuthology.orchestra.run.smithi006.stdout:(26/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 1.1 MB/s | 3.1 MB 00:02 2024-05-09T06:24:57.525 INFO:teuthology.orchestra.run.smithi001.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 785 kB/s | 85 kB 00:00 2024-05-09T06:24:57.809 INFO:teuthology.orchestra.run.smithi006.stdout:(27/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 1.3 MB/s | 5.3 MB 00:04 2024-05-09T06:24:57.834 INFO:teuthology.orchestra.run.smithi006.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 527 kB/s | 171 kB 00:00 2024-05-09T06:24:57.901 INFO:teuthology.orchestra.run.smithi006.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 346 kB/s | 32 kB 00:00 2024-05-09T06:24:58.051 INFO:teuthology.orchestra.run.smithi006.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 654 kB/s | 142 kB 00:00 2024-05-09T06:24:58.609 INFO:teuthology.orchestra.run.smithi001.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 158 kB/s | 171 kB 00:01 2024-05-09T06:24:58.893 INFO:teuthology.orchestra.run.smithi001.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 112 kB/s | 32 kB 00:00 2024-05-09T06:24:59.168 INFO:teuthology.orchestra.run.smithi001.stdout:(29/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 1.8 MB/s | 3.1 MB 00:01 2024-05-09T06:24:59.243 INFO:teuthology.orchestra.run.smithi001.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 405 kB/s | 142 kB 00:00 2024-05-09T06:25:00.044 INFO:teuthology.orchestra.run.smithi001.stdout:(31/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 3.7 MB/s | 48 MB 00:13 2024-05-09T06:25:00.227 INFO:teuthology.orchestra.run.smithi001.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 1.3 MB/s | 249 kB 00:00 2024-05-09T06:25:00.294 INFO:teuthology.orchestra.run.smithi001.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 740 kB/s | 49 kB 00:00 2024-05-09T06:25:00.369 INFO:teuthology.orchestra.run.smithi001.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 219 kB/s | 16 kB 00:00 2024-05-09T06:25:00.495 INFO:teuthology.orchestra.run.smithi001.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.1 MB/s | 264 kB 00:00 2024-05-09T06:25:00.576 INFO:teuthology.orchestra.run.smithi006.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 1.3 MB/s | 3.5 MB 00:02 2024-05-09T06:25:00.753 INFO:teuthology.orchestra.run.smithi001.stdout:(36/156): ceph-mgr-dashboard-19.0.0-2515.g96805 2.2 MB/s | 3.5 MB 00:01 2024-05-09T06:25:00.820 INFO:teuthology.orchestra.run.smithi006.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 1.0 MB/s | 249 kB 00:00 2024-05-09T06:25:00.845 INFO:teuthology.orchestra.run.smithi001.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 2.1 MB/s | 763 kB 00:00 2024-05-09T06:25:00.871 INFO:teuthology.orchestra.run.smithi001.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 156 kB/s | 18 kB 00:00 2024-05-09T06:25:00.902 INFO:teuthology.orchestra.run.smithi006.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 593 kB/s | 49 kB 00:00 2024-05-09T06:25:00.946 INFO:teuthology.orchestra.run.smithi001.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 802 kB/s | 80 kB 00:00 2024-05-09T06:25:00.985 INFO:teuthology.orchestra.run.smithi006.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 197 kB/s | 16 kB 00:00 2024-05-09T06:25:01.037 INFO:teuthology.orchestra.run.smithi001.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-09T06:25:01.163 INFO:teuthology.orchestra.run.smithi001.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-09T06:25:01.211 INFO:teuthology.orchestra.run.smithi006.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 1.1 MB/s | 264 kB 00:00 2024-05-09T06:25:01.347 INFO:teuthology.orchestra.run.smithi001.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-09T06:25:01.480 INFO:teuthology.orchestra.run.smithi001.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-05-09T06:25:01.522 INFO:teuthology.orchestra.run.smithi001.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 608 kB/s | 106 kB 00:00 2024-05-09T06:25:01.544 INFO:teuthology.orchestra.run.smithi006.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-05-09T06:25:01.581 INFO:teuthology.orchestra.run.smithi001.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-09T06:25:01.594 INFO:teuthology.orchestra.run.smithi006.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 366 kB/s | 18 kB 00:00 2024-05-09T06:25:01.614 INFO:teuthology.orchestra.run.smithi001.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-09T06:25:01.678 INFO:teuthology.orchestra.run.smithi006.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 965 kB/s | 80 kB 00:00 2024-05-09T06:25:01.715 INFO:teuthology.orchestra.run.smithi001.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-05-09T06:25:01.778 INFO:teuthology.orchestra.run.smithi006.stdout:(39/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 1.3 MB/s | 763 kB 00:00 2024-05-09T06:25:01.812 INFO:teuthology.orchestra.run.smithi006.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2024-05-09T06:25:01.879 INFO:teuthology.orchestra.run.smithi006.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-09T06:25:01.904 INFO:teuthology.orchestra.run.smithi006.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-09T06:25:01.971 INFO:teuthology.orchestra.run.smithi006.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-09T06:25:02.046 INFO:teuthology.orchestra.run.smithi006.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-05-09T06:25:02.072 INFO:teuthology.orchestra.run.smithi006.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-09T06:25:02.130 INFO:teuthology.orchestra.run.smithi006.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-09T06:25:02.156 INFO:teuthology.orchestra.run.smithi006.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-09T06:25:02.198 INFO:teuthology.orchestra.run.smithi001.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-05-09T06:25:02.256 INFO:teuthology.orchestra.run.smithi006.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-05-09T06:25:02.265 INFO:teuthology.orchestra.run.smithi001.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-09T06:25:02.306 INFO:teuthology.orchestra.run.smithi006.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-05-09T06:25:02.332 INFO:teuthology.orchestra.run.smithi001.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-09T06:25:02.348 INFO:teuthology.orchestra.run.smithi006.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-09T06:25:02.374 INFO:teuthology.orchestra.run.smithi006.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-09T06:25:02.424 INFO:teuthology.orchestra.run.smithi006.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-09T06:25:02.424 INFO:teuthology.orchestra.run.smithi001.stdout:(51/156): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-05-09T06:25:02.449 INFO:teuthology.orchestra.run.smithi001.stdout:(52/156): automake-1.16.2-8.el9.noarch.rpm 825 kB/s | 689 kB 00:00 2024-05-09T06:25:02.475 INFO:teuthology.orchestra.run.smithi001.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-05-09T06:25:02.583 INFO:teuthology.orchestra.run.smithi006.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-05-09T06:25:02.609 INFO:teuthology.orchestra.run.smithi001.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 178 kB/s | 33 kB 00:00 2024-05-09T06:25:02.625 INFO:teuthology.orchestra.run.smithi006.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-05-09T06:25:02.658 INFO:teuthology.orchestra.run.smithi006.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-05-09T06:25:02.675 INFO:teuthology.orchestra.run.smithi001.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 18 kB 00:00 2024-05-09T06:25:02.692 INFO:teuthology.orchestra.run.smithi001.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-09T06:25:02.734 INFO:teuthology.orchestra.run.smithi006.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-09T06:25:02.767 INFO:teuthology.orchestra.run.smithi001.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-05-09T06:25:02.784 INFO:teuthology.orchestra.run.smithi006.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.3 MB/s | 3.0 MB 00:00 2024-05-09T06:25:02.826 INFO:teuthology.orchestra.run.smithi006.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-05-09T06:25:02.876 INFO:teuthology.orchestra.run.smithi001.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 800 kB/s | 160 kB 00:00 2024-05-09T06:25:02.893 INFO:teuthology.orchestra.run.smithi006.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-09T06:25:02.935 INFO:teuthology.orchestra.run.smithi006.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-05-09T06:25:02.952 INFO:teuthology.orchestra.run.smithi006.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-09T06:25:02.994 INFO:teuthology.orchestra.run.smithi006.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-05-09T06:25:03.036 INFO:teuthology.orchestra.run.smithi006.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-09T06:25:03.043 INFO:teuthology.orchestra.run.smithi001.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-05-09T06:25:03.077 INFO:teuthology.orchestra.run.smithi001.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-05-09T06:25:03.078 INFO:teuthology.orchestra.run.smithi006.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-09T06:25:03.095 INFO:teuthology.orchestra.run.smithi006.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-05-09T06:25:03.110 INFO:teuthology.orchestra.run.smithi001.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-09T06:25:03.152 INFO:teuthology.orchestra.run.smithi001.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-05-09T06:25:03.236 INFO:teuthology.orchestra.run.smithi001.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-09T06:25:03.261 INFO:teuthology.orchestra.run.smithi001.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-09T06:25:03.328 INFO:teuthology.orchestra.run.smithi001.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 35 kB 00:00 2024-05-09T06:25:03.378 INFO:teuthology.orchestra.run.smithi006.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-09T06:25:03.445 INFO:teuthology.orchestra.run.smithi006.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-09T06:25:03.562 INFO:teuthology.orchestra.run.smithi006.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-09T06:25:03.604 INFO:teuthology.orchestra.run.smithi006.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-05-09T06:25:03.646 INFO:teuthology.orchestra.run.smithi006.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-09T06:25:03.763 INFO:teuthology.orchestra.run.smithi006.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-05-09T06:25:03.830 INFO:teuthology.orchestra.run.smithi006.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T06:25:03.955 INFO:teuthology.orchestra.run.smithi006.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-09T06:25:04.031 INFO:teuthology.orchestra.run.smithi006.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-09T06:25:04.106 INFO:teuthology.orchestra.run.smithi006.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-09T06:25:04.223 INFO:teuthology.orchestra.run.smithi006.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-09T06:25:04.265 INFO:teuthology.orchestra.run.smithi006.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 26 kB 00:00 2024-05-09T06:25:04.307 INFO:teuthology.orchestra.run.smithi006.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-05-09T06:25:04.357 INFO:teuthology.orchestra.run.smithi006.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-09T06:25:04.382 INFO:teuthology.orchestra.run.smithi006.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-05-09T06:25:04.458 INFO:teuthology.orchestra.run.smithi006.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 250 kB 00:00 2024-05-09T06:25:04.479 INFO:teuthology.orchestra.run.smithi001.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.5 MB/s | 3.0 MB 00:02 2024-05-09T06:25:04.533 INFO:teuthology.orchestra.run.smithi006.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-09T06:25:04.592 INFO:teuthology.orchestra.run.smithi006.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-09T06:25:04.651 INFO:teuthology.orchestra.run.smithi006.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-05-09T06:25:04.726 INFO:teuthology.orchestra.run.smithi006.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-09T06:25:04.762 INFO:teuthology.orchestra.run.smithi001.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.2 MB/s | 4.7 MB 00:01 2024-05-09T06:25:04.801 INFO:teuthology.orchestra.run.smithi006.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-09T06:25:04.854 INFO:teuthology.orchestra.run.smithi001.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-05-09T06:25:04.876 INFO:teuthology.orchestra.run.smithi006.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-09T06:25:04.927 INFO:teuthology.orchestra.run.smithi006.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T06:25:04.985 INFO:teuthology.orchestra.run.smithi006.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 918 kB/s | 54 kB 00:00 2024-05-09T06:25:04.996 INFO:teuthology.orchestra.run.smithi001.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-05-09T06:25:05.054 INFO:teuthology.orchestra.run.smithi001.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 666 kB/s | 133 kB 00:00 2024-05-09T06:25:05.211 INFO:teuthology.orchestra.run.smithi006.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-05-09T06:25:05.269 INFO:teuthology.orchestra.run.smithi006.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-05-09T06:25:05.271 INFO:teuthology.orchestra.run.smithi001.stdout:(71/156): postgresql-13.14-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-05-09T06:25:05.328 INFO:teuthology.orchestra.run.smithi006.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-05-09T06:25:05.355 INFO:teuthology.orchestra.run.smithi001.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-09T06:25:05.412 INFO:teuthology.orchestra.run.smithi006.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T06:25:05.505 INFO:teuthology.orchestra.run.smithi001.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-05-09T06:25:05.604 INFO:teuthology.orchestra.run.smithi006.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-09T06:25:05.629 INFO:teuthology.orchestra.run.smithi006.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 165 kB/s | 36 kB 00:00 2024-05-09T06:25:05.680 INFO:teuthology.orchestra.run.smithi001.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-05-09T06:25:05.687 INFO:teuthology.orchestra.run.smithi006.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-05-09T06:25:05.738 INFO:teuthology.orchestra.run.smithi006.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-09T06:25:05.763 INFO:teuthology.orchestra.run.smithi006.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-05-09T06:25:05.805 INFO:teuthology.orchestra.run.smithi006.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T06:25:05.830 INFO:teuthology.orchestra.run.smithi006.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 948 kB/s | 64 kB 00:00 2024-05-09T06:25:05.831 INFO:teuthology.orchestra.run.smithi001.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-05-09T06:25:05.881 INFO:teuthology.orchestra.run.smithi006.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-05-09T06:25:05.914 INFO:teuthology.orchestra.run.smithi006.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-09T06:25:05.915 INFO:teuthology.orchestra.run.smithi001.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 308 kB/s | 26 kB 00:00 2024-05-09T06:25:05.998 INFO:teuthology.orchestra.run.smithi001.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-05-09T06:25:06.090 INFO:teuthology.orchestra.run.smithi006.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-05-09T06:25:06.098 INFO:teuthology.orchestra.run.smithi001.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-09T06:25:06.149 INFO:teuthology.orchestra.run.smithi001.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-05-09T06:25:06.265 INFO:teuthology.orchestra.run.smithi006.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-09T06:25:06.315 INFO:teuthology.orchestra.run.smithi006.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-05-09T06:25:06.340 INFO:teuthology.orchestra.run.smithi006.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-09T06:25:06.366 INFO:teuthology.orchestra.run.smithi001.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-05-09T06:25:06.391 INFO:teuthology.orchestra.run.smithi006.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-05-09T06:25:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-05-09T06:25:06.500 INFO:teuthology.orchestra.run.smithi001.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-05-09T06:25:06.525 INFO:teuthology.orchestra.run.smithi006.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-09T06:25:06.550 INFO:teuthology.orchestra.run.smithi006.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-05-09T06:25:06.583 INFO:teuthology.orchestra.run.smithi001.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-09T06:25:06.583 INFO:teuthology.orchestra.run.smithi006.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-09T06:25:06.633 INFO:teuthology.orchestra.run.smithi001.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T06:25:06.642 INFO:teuthology.orchestra.run.smithi006.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-05-09T06:25:06.676 INFO:teuthology.orchestra.run.smithi006.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-05-09T06:25:06.700 INFO:teuthology.orchestra.run.smithi001.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-05-09T06:25:06.701 INFO:teuthology.orchestra.run.smithi006.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-05-09T06:25:06.727 INFO:teuthology.orchestra.run.smithi006.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-05-09T06:25:06.752 INFO:teuthology.orchestra.run.smithi006.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-09T06:25:06.786 INFO:teuthology.orchestra.run.smithi006.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T06:25:06.792 INFO:teuthology.orchestra.run.smithi001.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-09T06:25:06.819 INFO:teuthology.orchestra.run.smithi006.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-09T06:25:06.844 INFO:teuthology.orchestra.run.smithi006.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-05-09T06:25:06.868 INFO:teuthology.orchestra.run.smithi001.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-09T06:25:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-05-09T06:25:06.903 INFO:teuthology.orchestra.run.smithi006.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-09T06:25:06.954 INFO:teuthology.orchestra.run.smithi006.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-05-09T06:25:06.963 INFO:teuthology.orchestra.run.smithi001.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-09T06:25:06.979 INFO:teuthology.orchestra.run.smithi006.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-09T06:25:06.996 INFO:teuthology.orchestra.run.smithi006.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-05-09T06:25:07.021 INFO:teuthology.orchestra.run.smithi006.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-05-09T06:25:07.026 INFO:teuthology.orchestra.run.smithi001.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-05-09T06:25:07.038 INFO:teuthology.orchestra.run.smithi006.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2024-05-09T06:25:07.052 INFO:teuthology.orchestra.run.smithi001.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 554 kB/s | 51 kB 00:00 2024-05-09T06:25:07.063 INFO:teuthology.orchestra.run.smithi006.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 462 kB/s | 19 kB 00:00 2024-05-09T06:25:07.077 INFO:teuthology.orchestra.run.smithi001.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T06:25:07.080 INFO:teuthology.orchestra.run.smithi006.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-05-09T06:25:07.127 INFO:teuthology.orchestra.run.smithi001.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-05-09T06:25:07.130 INFO:teuthology.orchestra.run.smithi006.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-05-09T06:25:07.164 INFO:teuthology.orchestra.run.smithi006.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-05-09T06:25:07.186 INFO:teuthology.orchestra.run.smithi001.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 761 kB/s | 44 kB 00:00 2024-05-09T06:25:07.190 INFO:teuthology.orchestra.run.smithi006.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T06:25:07.215 INFO:teuthology.orchestra.run.smithi006.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-09T06:25:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-09T06:25:07.248 INFO:teuthology.orchestra.run.smithi006.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-09T06:25:07.274 INFO:teuthology.orchestra.run.smithi006.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-05-09T06:25:07.307 INFO:teuthology.orchestra.run.smithi006.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T06:25:07.333 INFO:teuthology.orchestra.run.smithi006.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-05-09T06:25:07.367 INFO:teuthology.orchestra.run.smithi006.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-05-09T06:25:07.392 INFO:teuthology.orchestra.run.smithi006.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-05-09T06:25:07.395 INFO:teuthology.orchestra.run.smithi001.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 239 kB/s | 36 kB 00:00 2024-05-09T06:25:07.417 INFO:teuthology.orchestra.run.smithi006.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-09T06:25:07.442 INFO:teuthology.orchestra.run.smithi006.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-05-09T06:25:07.468 INFO:teuthology.orchestra.run.smithi006.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-09T06:25:07.478 INFO:teuthology.orchestra.run.smithi001.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-05-09T06:25:07.510 INFO:teuthology.orchestra.run.smithi006.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-09T06:25:07.535 INFO:teuthology.orchestra.run.smithi006.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-09T06:25:07.612 INFO:teuthology.orchestra.run.smithi001.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-09T06:25:07.644 INFO:teuthology.orchestra.run.smithi006.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-09T06:25:07.671 INFO:teuthology.orchestra.run.smithi001.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-05-09T06:25:07.730 INFO:teuthology.orchestra.run.smithi001.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-09T06:25:07.788 INFO:teuthology.orchestra.run.smithi001.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-09T06:25:07.836 INFO:teuthology.orchestra.run.smithi006.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 66 kB/s | 20 kB 00:00 2024-05-09T06:25:07.847 INFO:teuthology.orchestra.run.smithi001.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-05-09T06:25:07.861 INFO:teuthology.orchestra.run.smithi006.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 883 kB/s | 192 kB 00:00 2024-05-09T06:25:07.911 INFO:teuthology.orchestra.run.smithi006.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-09T06:25:07.981 INFO:teuthology.orchestra.run.smithi001.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-05-09T06:25:08.231 INFO:teuthology.orchestra.run.smithi001.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-05-09T06:25:08.373 INFO:teuthology.orchestra.run.smithi001.stdout:(103/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.3 MB/s | 5.1 MB 00:02 2024-05-09T06:25:08.673 INFO:teuthology.orchestra.run.smithi001.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-09T06:25:08.698 INFO:teuthology.orchestra.run.smithi001.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 950 kB/s | 309 kB 00:00 2024-05-09T06:25:08.757 INFO:teuthology.orchestra.run.smithi001.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-09T06:25:08.840 INFO:teuthology.orchestra.run.smithi001.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-05-09T06:25:08.915 INFO:teuthology.orchestra.run.smithi001.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-05-09T06:25:08.995 INFO:teuthology.orchestra.run.smithi006.stdout:(148/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-05-09T06:25:09.145 INFO:teuthology.orchestra.run.smithi006.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 15 MB/s | 2.3 MB 00:00 2024-05-09T06:25:09.158 INFO:teuthology.orchestra.run.smithi001.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-05-09T06:25:09.217 INFO:teuthology.orchestra.run.smithi001.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 224 kB/s | 67 kB 00:00 2024-05-09T06:25:09.242 INFO:teuthology.orchestra.run.smithi001.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T06:25:09.262 INFO:teuthology.orchestra.run.smithi006.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-05-09T06:25:09.321 INFO:teuthology.orchestra.run.smithi006.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 716 kB/s | 42 kB 00:00 2024-05-09T06:25:09.396 INFO:teuthology.orchestra.run.smithi006.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-05-09T06:25:09.459 INFO:teuthology.orchestra.run.smithi001.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-09T06:25:09.496 INFO:teuthology.orchestra.run.smithi006.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-05-09T06:25:09.509 INFO:teuthology.orchestra.run.smithi001.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-05-09T06:25:09.543 INFO:teuthology.orchestra.run.smithi001.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-05-09T06:25:09.618 INFO:teuthology.orchestra.run.smithi001.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-05-09T06:25:09.630 INFO:teuthology.orchestra.run.smithi006.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-05-09T06:25:09.643 INFO:teuthology.orchestra.run.smithi001.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-09T06:25:09.727 INFO:teuthology.orchestra.run.smithi001.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-09T06:25:09.777 INFO:teuthology.orchestra.run.smithi001.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-09T06:25:09.939 INFO:teuthology.orchestra.run.smithi006.stdout:(155/156): librados2-19.0.0-2515.g96805dcf.el9. 1.6 MB/s | 3.4 MB 00:02 2024-05-09T06:25:10.028 INFO:teuthology.orchestra.run.smithi001.stdout:(119/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-05-09T06:25:10.044 INFO:teuthology.orchestra.run.smithi001.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-05-09T06:25:10.070 INFO:teuthology.orchestra.run.smithi001.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 886 kB/s | 303 kB 00:00 2024-05-09T06:25:10.212 INFO:teuthology.orchestra.run.smithi001.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-09T06:25:10.237 INFO:teuthology.orchestra.run.smithi001.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 865 kB/s | 144 kB 00:00 2024-05-09T06:25:10.287 INFO:teuthology.orchestra.run.smithi001.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-09T06:25:10.321 INFO:teuthology.orchestra.run.smithi001.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-05-09T06:25:10.371 INFO:teuthology.orchestra.run.smithi001.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-09T06:25:10.396 INFO:teuthology.orchestra.run.smithi001.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T06:25:10.422 INFO:teuthology.orchestra.run.smithi001.stdout:(128/156): python3-google-auth-2.29.0-1.el9.noa 522 kB/s | 206 kB 00:00 2024-05-09T06:25:10.447 INFO:teuthology.orchestra.run.smithi001.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-09T06:25:10.506 INFO:teuthology.orchestra.run.smithi001.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-05-09T06:25:10.531 INFO:teuthology.orchestra.run.smithi001.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-05-09T06:25:10.590 INFO:teuthology.orchestra.run.smithi001.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-09T06:25:10.623 INFO:teuthology.orchestra.run.smithi001.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T06:25:10.657 INFO:teuthology.orchestra.run.smithi001.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-05-09T06:25:10.699 INFO:teuthology.orchestra.run.smithi001.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T06:25:10.741 INFO:teuthology.orchestra.run.smithi001.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-05-09T06:25:10.766 INFO:teuthology.orchestra.run.smithi001.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-09T06:25:10.791 INFO:teuthology.orchestra.run.smithi001.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 332 kB/s | 31 kB 00:00 2024-05-09T06:25:10.817 INFO:teuthology.orchestra.run.smithi001.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-09T06:25:10.842 INFO:teuthology.orchestra.run.smithi001.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-09T06:25:10.867 INFO:teuthology.orchestra.run.smithi001.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-09T06:25:10.893 INFO:teuthology.orchestra.run.smithi001.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-09T06:25:10.951 INFO:teuthology.orchestra.run.smithi001.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-09T06:25:10.976 INFO:teuthology.orchestra.run.smithi001.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-09T06:25:11.027 INFO:teuthology.orchestra.run.smithi001.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-09T06:25:11.043 INFO:teuthology.orchestra.run.smithi001.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-09T06:25:11.077 INFO:teuthology.orchestra.run.smithi001.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-09T06:25:11.352 INFO:teuthology.orchestra.run.smithi001.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-09T06:25:11.703 INFO:teuthology.orchestra.run.smithi001.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2024-05-09T06:25:12.036 INFO:teuthology.orchestra.run.smithi001.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.5 MB/s | 1.8 MB 00:00 2024-05-09T06:25:12.103 INFO:teuthology.orchestra.run.smithi001.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 629 kB/s | 42 kB 00:00 2024-05-09T06:25:12.212 INFO:teuthology.orchestra.run.smithi001.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-09T06:25:12.487 INFO:teuthology.orchestra.run.smithi001.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-05-09T06:25:12.570 INFO:teuthology.orchestra.run.smithi001.stdout:(154/156): librados2-19.0.0-2515.g96805dcf.el9. 2.2 MB/s | 3.4 MB 00:01 2024-05-09T06:25:12.763 INFO:teuthology.orchestra.run.smithi001.stdout:(155/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-05-09T06:25:12.896 INFO:teuthology.orchestra.run.smithi001.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.9 MB/s | 2.4 MB 00:00 2024-05-09T06:25:12.899 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:12.899 INFO:teuthology.orchestra.run.smithi001.stdout:Total 6.8 MB/s | 225 MB 00:32 2024-05-09T06:25:13.139 INFO:teuthology.orchestra.run.smithi006.stdout:(156/156): ceph-test-19.0.0-2515.g96805dcf.el9. 2.1 MB/s | 48 MB 00:22 2024-05-09T06:25:13.143 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:13.143 INFO:teuthology.orchestra.run.smithi006.stdout:Total 6.6 MB/s | 225 MB 00:34 2024-05-09T06:25:14.091 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-05-09T06:25:14.259 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-05-09T06:25:14.259 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-05-09T06:25:14.386 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-05-09T06:25:14.553 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-05-09T06:25:14.554 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-05-09T06:25:16.528 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-05-09T06:25:16.528 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-05-09T06:25:16.856 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-05-09T06:25:16.856 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-05-09T06:25:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-05-09T06:25:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T06:25:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T06:25:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T06:25:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T06:25:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T06:25:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T06:25:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-05-09T06:25:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T06:25:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T06:25:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T06:25:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T06:25:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T06:25:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T06:25:23.474 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T06:25:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:25:23.719 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:25:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:25:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T06:25:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:25:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:25:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:25:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:25:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T06:25:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:25:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T06:25:24.148 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:25:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:25:24.177 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:25:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:25:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:25:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:25:24.365 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T06:25:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:25:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:25:24.424 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T06:25:24.459 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:25:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:25:24.549 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:25:24.577 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:25:24.624 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:25:24.653 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:25:24.681 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:25:24.705 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:25:24.842 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:25:24.873 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:25:24.901 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:25:25.023 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.041 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:25:25.077 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:25:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:25:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T06:25:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T06:25:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T06:25:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T06:25:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:25:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T06:25:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T06:25:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T06:25:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T06:25:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T06:25:25.812 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T06:25:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T06:25:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T06:25:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T06:25:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T06:25:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T06:25:25.965 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T06:25:26.001 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T06:25:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T06:25:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T06:25:26.080 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T06:25:26.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T06:25:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T06:25:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T06:25:26.155 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:25:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T06:25:26.183 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T06:25:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T06:25:26.256 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T06:25:26.298 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T06:25:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T06:25:26.342 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T06:25:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T06:25:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T06:25:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T06:25:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T06:25:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T06:25:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:25:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T06:25:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T06:25:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T06:25:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T06:25:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T06:25:26.911 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T06:25:26.963 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T06:25:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T06:25:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T06:25:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T06:25:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T06:25:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T06:25:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T06:25:27.531 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T06:25:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T06:25:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T06:25:27.620 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T06:25:27.653 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T06:25:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T06:25:27.697 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T06:25:27.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T06:25:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T06:25:27.777 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T06:25:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T06:25:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T06:25:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T06:25:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T06:25:27.941 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T06:25:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T06:25:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T06:25:27.995 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T06:25:28.052 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T06:25:28.127 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T06:25:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T06:25:28.162 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T06:25:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T06:25:28.199 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T06:25:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T06:25:28.248 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T06:25:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T06:25:28.282 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T06:25:28.326 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T06:25:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T06:25:28.376 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T06:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T06:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T06:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T06:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T06:25:28.522 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T06:25:28.565 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T06:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T06:25:28.594 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T06:25:28.637 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T06:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T06:25:28.735 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T06:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T06:25:28.770 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T06:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T06:25:28.818 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T06:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T06:25:28.852 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T06:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T06:25:28.876 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T06:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T06:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T06:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T06:25:28.974 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T06:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T06:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T06:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T06:25:29.110 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T06:25:29.150 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T06:25:29.179 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T06:25:29.214 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T06:25:29.252 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T06:25:29.292 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T06:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T06:25:29.321 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T06:25:29.356 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T06:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T06:25:29.389 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T06:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T06:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T06:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T06:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T06:25:29.733 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T06:25:29.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T06:25:29.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T06:25:29.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T06:25:29.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T06:25:29.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T06:25:29.804 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T06:25:29.863 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T06:25:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T06:25:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T06:25:30.498 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T06:25:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T06:25:30.656 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T06:25:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T06:25:30.701 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T06:25:30.735 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T06:25:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T06:25:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T06:25:30.842 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T06:25:30.893 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T06:25:30.975 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T06:25:30.986 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T06:25:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T06:25:31.028 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T06:25:31.090 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T06:25:31.214 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T06:25:31.242 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T06:25:31.325 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T06:25:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T06:25:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T06:25:32.033 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T06:25:32.164 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T06:25:33.287 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T06:25:33.358 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T06:25:33.386 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T06:25:33.420 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T06:25:33.521 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T06:25:33.597 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T06:25:33.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T06:25:33.658 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T06:25:33.753 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T06:25:33.790 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:25:33.891 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:25:33.920 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T06:25:33.957 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T06:25:33.995 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T06:25:34.037 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:25:34.129 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:25:34.159 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T06:25:34.196 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T06:25:34.460 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T06:25:34.494 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:25:34.565 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:25:34.595 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T06:25:34.692 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T06:25:34.723 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:25:34.795 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:25:34.825 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T06:25:36.923 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:37.183 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:37.221 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:25:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:25:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:25:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:25:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:25:37.797 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:25:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:25:37.933 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:25:37.933 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:25:37.933 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:25:37.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:37.976 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:25:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:25:49.295 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-05-09T06:25:49.295 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-05-09T06:25:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:25:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:25:49.486 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:49.486 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T06:25:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-05-09T06:25:50.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:50.668 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:25:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:25:50.685 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:50.685 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T06:25:50.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:51.022 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:25:51.050 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:25:51.051 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:51.051 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:25:51.051 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:51.051 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:51.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:51.281 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:51.306 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:51.782 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:51.807 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:51.921 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T06:25:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T06:25:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T06:25:52.213 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:25:52.238 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:52.242 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:52.283 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T06:25:52.365 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T06:25:52.462 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T06:25:52.480 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:25:52.505 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:25:52.505 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:52.505 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:25:52.505 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:52.506 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:52.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:52.606 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T06:25:52.649 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T06:25:52.687 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T06:25:52.717 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T06:25:52.981 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:25:53.006 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:25:53.006 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:53.006 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:25:53.006 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:53.007 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:53.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:53.112 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T06:25:53.170 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T06:25:53.178 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:25:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:25:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T06:25:53.257 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T06:25:53.305 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T06:25:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:25:53.464 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T06:25:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T06:25:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T06:25:53.695 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T06:25:53.848 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T06:25:53.891 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T06:25:53.929 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T06:25:53.958 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T06:25:54.378 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:25:54.407 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:25:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:25:54.439 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T06:25:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:25:54.504 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T06:25:54.530 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:25:55.481 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:25:55.510 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:25:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:25:55.624 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:25:55.665 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:25:55.786 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T06:25:55.818 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:55.843 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:55.893 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:25:55.905 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:25:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T06:25:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T06:25:56.682 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:25:56.707 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:25:56.818 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:25:56.986 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T06:25:56.996 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T06:25:57.017 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:25:57.029 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:25:57.043 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:25:57.043 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:57.043 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:25:57.044 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:57.044 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:57.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:57.054 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:57.093 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:25:57.096 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:25:57.106 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:25:57.120 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:25:57.120 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:57.120 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:25:57.121 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:57.172 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:25:57.197 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:57.206 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T06:25:57.268 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T06:25:57.514 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:25:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:25:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:25:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:25:57.543 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:25:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:25:58.216 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T06:25:58.254 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:25:58.279 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:25:58.280 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:58.280 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:25:58.280 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:58.280 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:58.280 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:58.322 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:25:58.346 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:25:58.346 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:58.347 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:25:58.347 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:58.397 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:25:58.422 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:25:58.739 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:25:58.766 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:25:58.767 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:58.767 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:25:58.767 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:25:58.767 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:25:58.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T06:26:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T06:26:02.393 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T06:26:02.424 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T06:26:02.451 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T06:26:02.663 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T06:26:02.714 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T06:26:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T06:26:02.774 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T06:26:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T06:26:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:03.430 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:03.544 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T06:26:03.614 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T06:26:03.660 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T06:26:03.691 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T06:26:03.718 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T06:26:03.923 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T06:26:03.965 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T06:26:03.995 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T06:26:04.025 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T06:26:04.079 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T06:26:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T06:26:04.439 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T06:26:04.487 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T06:26:04.574 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T06:26:04.609 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T06:26:04.635 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T06:26:04.635 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:26:04.661 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:04.702 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:04.723 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:26:04.727 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:26:04.760 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T06:26:04.761 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:26:04.837 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:26:04.862 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T06:26:04.867 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:04.867 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:05.055 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:05.055 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:26:05.139 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:26:05.207 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:05.626 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T06:26:05.715 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T06:26:05.769 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T06:26:05.870 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T06:26:05.902 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T06:26:05.931 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T06:26:05.931 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:26:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:26:06.045 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T06:26:06.045 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:26:06.130 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:26:06.154 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T06:26:06.160 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:06.160 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:26:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:26:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:26:06.526 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:10.612 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:10.612 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T06:26:10.612 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T06:26:10.612 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T06:26:10.613 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T06:26:10.614 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T06:26:10.627 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T06:26:10.627 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T06:26:10.627 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T06:26:10.627 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T06:26:10.627 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T06:26:10.628 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T06:26:10.629 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T06:26:10.630 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T06:26:10.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T06:26:10.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T06:26:10.633 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T06:26:10.634 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T06:26:10.635 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T06:26:10.636 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T06:26:10.637 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T06:26:12.093 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T06:26:12.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T06:26:12.095 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T06:26:12.109 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T06:26:12.110 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T06:26:12.111 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T06:26:12.112 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T06:26:12.113 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T06:26:12.114 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T06:26:12.115 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T06:26:12.116 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T06:26:12.117 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T06:26:12.118 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T06:26:12.119 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T06:26:12.119 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T06:26:12.119 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T06:26:12.119 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T06:26:12.119 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T06:26:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:13.568 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T06:26:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T06:26:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T06:26:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:26:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:26:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T06:26:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T06:26:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T06:26:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:26:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:26:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T06:26:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T06:26:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:26:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T06:26:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:13.586 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-09T06:26:13.995 DEBUG:teuthology.parallel:result is None 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T06:26:14.973 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.974 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.975 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T06:26:14.976 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.977 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T06:26:14.978 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:26:14.979 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:26:14.980 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:26:14.981 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:26:14.982 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:26:14.983 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T06:26:14.984 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:26:14.985 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T06:26:14.985 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:14.985 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-05-09T06:26:15.425 DEBUG:teuthology.parallel:result is None 2024-05-09T06:26:15.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:26:15.645 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:26:15.672 INFO:teuthology.orchestra.run.smithi001.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T06:26:15.673 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T06:26:15.673 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T06:26:15.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:26:15.897 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:26:15.922 INFO:teuthology.orchestra.run.smithi006.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T06:26:15.923 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T06:26:15.923 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T06:26:15.924 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T06:26:15.925 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:15.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:26:15.957 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:15.958 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:26:15.996 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T06:26:15.996 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:15.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:26:16.028 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:26:16.098 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:16.099 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:26:16.127 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:26:16.196 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T06:26:16.197 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:16.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:26:16.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:26:16.299 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:16.299 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:26:16.328 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:26:16.396 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T06:26:16.396 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:16.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:26:16.427 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:26:16.498 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:16.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:26:16.526 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:26:16.594 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T06:26:16.685 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T06:26:16.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /var/log/ceph 2024-05-09T06:26:16.688 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /var/log/ceph 2024-05-09T06:26:16.716 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T06:26:16.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T06:26:16.758 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T06:26:16.786 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T06:26:16.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T06:26:16.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T06:26:16.863 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T06:26:16.864 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T06:26:16.864 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T06:26:16.864 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:16.864 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T06:26:16.917 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:17.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:17.068 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:17.068 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T06:26:17.086 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:17.206 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:17.240 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T06:26:17.240 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T06:26:17.240 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7698532', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2334, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '359/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi001.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeKuSuvCSTl2NOkCWajvBdgPOXctLAGTsGHL9ZQm//uM7SwdJMUl9tPGxm5hv398tEOGQdsNRqE2tarekbC8dU=', 'smithi006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAE21YDipqInL/6q0E+gLCZIjnPIPuvSv5t8AujAjJbR26SK46h59tFXCDS+N+rI8ZF+tFcukC7mUQiL5sKICJM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': ['/home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/fs/functional/overrides/pg_health.yaml'], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_21:24:04', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T06:26:17.241 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T06:26:17.243 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T06:26:17.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T06:26:17.302 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T06:26:17.359 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:17.359 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:26:17.390 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:26:17.390 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 06:26:05.576973884 +0000 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 06:23:57.790987198 +0000 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 06:23:57.790987198 +0000 2024-05-09T06:26:17.449 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 06:23:57.790987198 +0000 2024-05-09T06:26:17.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:26:17.517 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T06:26:17.518 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T06:26:17.518 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000122107 s, 4.2 MB/s 2024-05-09T06:26:17.519 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:26:17.580 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 06:26:05.577973872 +0000 2024-05-09T06:26:17.639 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 06:23:58.120983153 +0000 2024-05-09T06:26:17.640 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 06:23:58.120983153 +0000 2024-05-09T06:26:17.640 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 06:23:58.120983153 +0000 2024-05-09T06:26:17.640 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:26:17.708 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T06:26:17.708 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T06:26:17.708 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000110937 s, 4.6 MB/s 2024-05-09T06:26:17.709 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:26:17.769 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 06:26:05.577973872 +0000 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 06:23:58.473978827 +0000 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 06:23:58.473978827 +0000 2024-05-09T06:26:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 06:23:58.473978827 +0000 2024-05-09T06:26:17.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:26:17.899 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T06:26:17.899 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T06:26:17.899 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000123015 s, 4.2 MB/s 2024-05-09T06:26:17.900 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:26:17.959 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 06:26:05.577973872 +0000 2024-05-09T06:26:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 06:23:58.804974770 +0000 2024-05-09T06:26:18.020 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 06:23:58.804974770 +0000 2024-05-09T06:26:18.020 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 06:23:58.804974770 +0000 2024-05-09T06:26:18.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:26:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T06:26:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T06:26:18.088 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000122959 s, 4.2 MB/s 2024-05-09T06:26:18.090 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:26:18.150 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:18.150 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:18.150 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:26:18.170 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:26:18.170 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-05-09T06:26:18.230 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:26:18.230 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:18.230 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T06:26:18.230 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:18.231 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:18.231 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 06:26:06.889591312 +0000 2024-05-09T06:26:18.231 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 06:23:38.570169634 +0000 2024-05-09T06:26:18.231 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 06:23:38.570169634 +0000 2024-05-09T06:26:18.231 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 06:23:38.570169634 +0000 2024-05-09T06:26:18.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:26:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T06:26:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T06:26:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000249773 s, 2.0 MB/s 2024-05-09T06:26:18.301 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:26:18.360 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 06:26:06.889591312 +0000 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 06:23:38.917166122 +0000 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 06:23:38.917166122 +0000 2024-05-09T06:26:18.419 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 06:23:38.917166122 +0000 2024-05-09T06:26:18.419 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:26:18.486 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T06:26:18.486 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T06:26:18.486 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000209694 s, 2.4 MB/s 2024-05-09T06:26:18.487 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:26:18.547 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-05-09T06:26:18.604 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:26:18.604 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:18.604 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T06:26:18.604 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:18.605 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:18.605 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 06:26:06.890591302 +0000 2024-05-09T06:26:18.605 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 06:23:39.251162741 +0000 2024-05-09T06:26:18.605 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 06:23:39.251162741 +0000 2024-05-09T06:26:18.605 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 06:23:39.251162741 +0000 2024-05-09T06:26:18.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:26:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T06:26:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T06:26:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000193721 s, 2.6 MB/s 2024-05-09T06:26:18.671 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:26:18.728 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-05-09T06:26:18.785 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:26:18.785 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:26:18.785 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T06:26:18.785 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:26:18.786 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:26:18.786 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 06:26:06.890591302 +0000 2024-05-09T06:26:18.786 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 06:23:39.593159279 +0000 2024-05-09T06:26:18.786 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 06:23:39.593159279 +0000 2024-05-09T06:26:18.786 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 06:23:39.593159279 +0000 2024-05-09T06:26:18.786 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:26:18.851 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T06:26:18.851 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T06:26:18.851 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000206359 s, 2.5 MB/s 2024-05-09T06:26:18.853 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:26:18.910 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:18.910 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T06:26:18.910 INFO:tasks.ceph:Generating config... 2024-05-09T06:26:18.911 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T06:26:18.912 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T06:26:18.912 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T06:26:18.912 INFO:tasks.ceph:[client] pid file = 2024-05-09T06:26:18.912 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T06:26:18.912 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T06:26:18.912 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T06:26:18.913 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T06:26:18.913 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T06:26:18.914 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T06:26:18.914 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T06:26:18.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T06:26:18.968 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T06:26:18.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T06:26:19.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:26:19.047 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.1'), ('mon.b', '172.21.15.6'), ('mon.c', '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]')] 2024-05-09T06:26:19.048 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T06:26:19.048 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:19.048 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T06:26:19.105 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.1 --add b 172.21.15.6 --addv c '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:19.194 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:19.194 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: generated fsid 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 0 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:last_changed 2024-05-09T06:26:19.213455+0000 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-05-09T06:26:19.213455+0000 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:min_mon_release 17 (quincy) 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:election_strategy: 1 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:1: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.b 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-09T06:26:19.195 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T06:26:19.196 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T06:26:19.212 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 91cbcc67-ea38-4460-ad67-94260dad3bad... 2024-05-09T06:26:19.213 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:26:19.255 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T06:26:19.300 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T06:26:19.301 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:26:19.302 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T06:26:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T06:26:19.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:26:19.305 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:26:19.306 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T06:26:19.307 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T06:26:19.308 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = "" 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:26:19.309 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = True 2024-05-09T06:26:19.310 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T06:26:19.311 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = True 2024-05-09T06:26:19.311 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:26:19.311 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:26:19.311 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:26:19.311 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.312 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:26:19.313 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-09T06:26:19.314 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-09T06:26:19.315 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-09T06:26:19.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.317 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:26:19.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:26:19.319 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:26:19.320 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: mon down mkfs grace = 300 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:26:19.321 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:26:19.322 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = True 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = True 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:26:19.323 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:26:19.325 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T06:26:19.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T06:26:19.421 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T06:26:19.422 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:19.422 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T06:26:19.437 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:19.437 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T06:26:19.493 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:26:19.493 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:19.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T06:26:19.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:26:19.570 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:19.570 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:19.626 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:26:19.626 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:19.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T06:26:19.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:26:19.664 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:19.665 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:19.721 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T06:26:19.721 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T06:26:19.722 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T06:26:19.774 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T06:26:19.777 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T06:26:19.842 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T06:26:19.844 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T06:26:19.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T06:26:19.897 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T06:26:19.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T06:26:19.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T06:26:20.019 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T06:26:20.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T06:26:20.047 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T06:26:20.108 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T06:26:20.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T06:26:20.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T06:26:20.232 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T06:26:20.235 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T06:26:20.261 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T06:26:20.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T06:26:20.303 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T06:26:20.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-09T06:26:20.399 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-09T06:26:20.413 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T06:26:20.456 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T06:26:20.471 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-09T06:26:20.513 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-09T06:26:20.528 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T06:26:20.528 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T06:26:20.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T06:26:20.554 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:20.554 INFO:tasks.ceph:role: osd.0 2024-05-09T06:26:20.554 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:26:20.554 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:20.620 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:20.620 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:20.620 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T06:26:20.653 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T06:26:20.653 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T06:26:20.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T06:26:20.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T06:26:20.866 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:20.866 INFO:tasks.ceph:role: osd.1 2024-05-09T06:26:20.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:26:20.867 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T06:26:20.932 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:20.932 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:20.933 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:20.933 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:20.933 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:20.964 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T06:26:20.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T06:26:20.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T06:26:21.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T06:26:21.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T06:26:21.168 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:21.168 INFO:tasks.ceph:role: osd.2 2024-05-09T06:26:21.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:26:21.169 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T06:26:21.234 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:21.235 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:21.266 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T06:26:21.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T06:26:21.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T06:26:21.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T06:26:21.408 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T06:26:21.474 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:21.474 INFO:tasks.ceph:role: osd.3 2024-05-09T06:26:21.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:26:21.475 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T06:26:21.539 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:21.540 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:21.572 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T06:26:21.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T06:26:21.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T06:26:21.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T06:26:21.714 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:26:21.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:21.826+0000 7f675120f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T06:26:24.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:26:24.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.793+0000 7f8b0c1a3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T06:26:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.794+0000 7f8b0c1a3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T06:26:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.794+0000 7f8b0c1a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:26:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.794+0000 7f8b0c1a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:26:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.794+0000 7f8b0c1a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:26:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:24.794+0000 7f8b0c1a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T06:26:27.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:26:27.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.770+0000 7f8bc26fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.770+0000 7f8bc26fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.770+0000 7f8bc26fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.770+0000 7f8bc26fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.770+0000 7f8bc26fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:26:27.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:27.771+0000 7f8bc26fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T06:26:30.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:26:30.629 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T06:26:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T06:26:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:26:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:26:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:26:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:30.736+0000 7fb06ca3f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T06:26:33.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:26:33.612 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T06:26:33.644 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:33.644 INFO:tasks.ceph:role: osd.4 2024-05-09T06:26:33.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:26:33.645 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T06:26:33.721 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:33.722 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:34.221 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-09T06:26:34.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-09T06:26:34.224 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T06:26:34.335 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T06:26:34.371 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T06:26:34.441 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:34.441 INFO:tasks.ceph:role: osd.5 2024-05-09T06:26:34.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:26:34.441 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T06:26:34.516 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:34.516 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:34.516 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:34.517 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:34.966 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-09T06:26:34.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-09T06:26:34.969 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T06:26:35.062 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T06:26:35.097 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T06:26:35.165 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:35.165 INFO:tasks.ceph:role: osd.6 2024-05-09T06:26:35.165 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:26:35.165 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:35.240 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:35.718 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-09T06:26:35.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-09T06:26:35.720 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T06:26:35.874 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T06:26:35.909 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T06:26:35.979 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:26:35.979 INFO:tasks.ceph:role: osd.7 2024-05-09T06:26:35.979 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:26:35.979 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:26:36.054 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:26:36.055 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:26:36.055 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:26:36.055 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:26:36.055 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:26:36.570 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-09T06:26:36.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-09T06:26:36.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T06:26:36.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T06:26:36.736 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.856+0000 7f9c9060d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.857+0000 7f9c9060d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.857+0000 7f9c9060d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.857+0000 7f9c9060d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.857+0000 7f9c9060d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:26:36.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:36.857+0000 7f9c9060d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T06:26:39.699 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:26:39.730 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.842+0000 7f72dbef0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.842+0000 7f72dbef0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.842+0000 7f72dbef0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.842+0000 7f72dbef0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.842+0000 7f72dbef0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:26:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:39.843+0000 7f72dbef0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T06:26:42.678 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:26:42.709 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.819+0000 7f16c9284640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.820+0000 7f16c9284640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.820+0000 7f16c9284640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.820+0000 7f16c9284640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.820+0000 7f16c9284640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:26:42.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:42.820+0000 7f16c9284640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T06:26:45.653 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T06:26:45.685 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.796+0000 7f84cde11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.796+0000 7f84cde11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.796+0000 7f84cde11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.796+0000 7f84cde11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.796+0000 7f84cde11640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:26:45.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:45.797+0000 7f84cde11640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T06:26:47.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:47.245 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:26:48.633 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T06:26:48.664 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T06:26:48.664 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.664 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T06:26:48.694 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T06:26:48.765 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T06:26:48.834 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T06:26:48.902 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T06:26:48.970 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:48.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T06:26:49.038 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:49.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T06:26:49.105 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.105 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T06:26:49.137 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T06:26:49.207 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.207 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T06:26:49.275 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.275 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T06:26:49.343 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.344 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T06:26:49.412 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T06:26:49.480 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.480 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T06:26:49.548 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:49.548 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T06:26:49.567 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:49.568 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-09T06:26:49.628 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.628 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T06:26:49.645 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:49.645 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-09T06:26:49.701 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T06:26:49.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T06:26:49.704 DEBUG:teuthology.orchestra.run.smithi006:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T06:26:49.745 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.y] 2024-05-09T06:26:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q== 2024-05-09T06:26:49.745 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.0] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.1] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.2] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.3] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.x] 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw== 2024-05-09T06:26:49.746 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.4] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.5] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.6] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.7] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:26:49.747 INFO:teuthology.orchestra.run.smithi001.stdout:[client.2] 2024-05-09T06:26:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw== 2024-05-09T06:26:49.748 INFO:teuthology.orchestra.run.smithi001.stdout:[client.1] 2024-05-09T06:26:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ== 2024-05-09T06:26:49.748 INFO:teuthology.orchestra.run.smithi001.stdout:[client.3] 2024-05-09T06:26:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw== 2024-05-09T06:26:49.767 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.y] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.0] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.1] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.2] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.3] 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ== 2024-05-09T06:26:49.768 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.x] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.4] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.5] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.6] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg== 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.7] 2024-05-09T06:26:49.769 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q== 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout:[client.2] 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw== 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout:[client.1] 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ== 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout:[client.3] 2024-05-09T06:26:49.770 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw== 2024-05-09T06:26:49.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:26:49.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:26:49.866 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:49.869 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:49.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:49.927 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:49.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:49.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.039 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.094 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.149 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:26:50.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:26:50.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:50.259 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:50.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:50.317 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:26:50.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.431 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.486 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:26:50.597 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.599 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.658 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.717 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.775 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.778 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:26:50.833 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T06:26:50.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T06:26:50.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:26:50.957 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T06:26:50.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T06:26:51.011 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:26:51.109 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T06:26:51.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T06:26:51.202 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:26:51.300 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T06:26:51.326 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:51.329 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:26:51.382 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T06:26:51.383 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T06:26:51.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T06:26:51.386 INFO:tasks.ceph.mon.a:Started 2024-05-09T06:26:51.386 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T06:26:51.386 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T06:26:51.425 INFO:tasks.ceph.mon.b:Started 2024-05-09T06:26:51.425 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T06:26:51.425 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T06:26:51.427 INFO:tasks.ceph.mon.c:Started 2024-05-09T06:26:51.427 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T06:26:51.428 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T06:26:51.428 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T06:26:51.430 INFO:tasks.ceph.mgr.y:Started 2024-05-09T06:26:51.430 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T06:26:51.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T06:26:51.433 INFO:tasks.ceph.mgr.x:Started 2024-05-09T06:26:51.433 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T06:26:51.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T06:26:51.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.558+0000 7f3b412d4640 1 Processor -- start 2024-05-09T06:26:51.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.558+0000 7f3b412d4640 1 -- start start 2024-05-09T06:26:51.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b412d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:51.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b412d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:51.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b412d4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3c1629f0 con 0x7f3b3c15ea80 2024-05-09T06:26:51.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b412d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3c15f4c0 con 0x7f3b3c0d2870 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b412d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3c162020 con 0x7f3b3c06dfc0 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3b3c15ea80 0x7f3b3c09aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48552/0 (socket says 172.21.15.1:48552) 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55112/0 (socket says 172.21.15.1:55112) 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3b7fe640 1 -- 172.21.15.1:0/646753519 learned_addr learned my addr 172.21.15.1:0/646753519 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:51.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.559+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:51.760+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.161+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:52.962+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T06:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15e540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b3c15e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 msgr2=0x7f3b3c1624b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.1:3300/0 conn(0x7f3b3c0d2870 0x7f3b3c1624b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 -- 172.21.15.1:0/646753519 >> v1:172.21.15.1:6789/0 conn(0x7f3b3c15ea80 legacy=0x7f3b3c09aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 -- 172.21.15.1:0/646753519 --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3c15ef00 con 0x7f3b1c0036c0 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b38ff9640 1 -- 172.21.15.1:0/646753519 --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3c15f4c0 con 0x7f3b1c007350 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.560+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:26:54.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.761+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:54.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.761+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:54.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.761+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:54.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.761+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:54.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:54.761+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:26:55.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.162+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:55.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.162+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:55.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.162+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:55.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.162+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:55.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.162+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T06:26:55.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.963+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:55.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.963+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:26:55.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.963+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:26:55.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.963+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:26:55.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:55.963+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T06:26:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.734+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3229556339 0 0) 0x7f3b3c15ef00 con 0x7f3b1c0036c0 2024-05-09T06:26:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.734+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b28003520 con 0x7f3b1c0036c0 2024-05-09T06:26:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4245496390 0 0) 0x7f3b28003520 con 0x7f3b1c0036c0 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 msgr2=0x7f3b1c009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 --2- 172.21.15.1:0/646753519 >> v2:172.21.15.6:3300/0 conn(0x7f3b1c007350 0x7f3b1c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c00af20 con 0x7f3b1c0036c0 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3b34009a00 con 0x7f3b1c0036c0 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 >> v1:172.21.15.6:6789/0 conn(0x7f3b1c0036c0 legacy=0x7f3b1c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3b7fe640 1 -- 172.21.15.1:0/646753519 reap_dead start 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 0x7f3b28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28004e60 0x7f3b2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c15f4c0 con 0x7f3b28003a00 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28005620 con 0x7f3b28004e60 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2800bf00 con 0x7f3b3c06dfc0 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 0x7f3b28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3b7fe640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28004e60 0x7f3b2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28004e60 msgr2=0x7f3b2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28004e60 0x7f3b2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.736+0000 7f3b3affd640 1 -- 172.21.15.1:0/646753519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b28003520 con 0x7f3b28003a00 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.737+0000 7f3b3affd640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 0x7f3b28003e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009610 tx=0x7f3b28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96991203dc79a72d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.739+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34031070 con 0x7f3b28003a00 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.739+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34035400 con 0x7f3b28003a00 2024-05-09T06:26:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.739+0000 7f3b39ffb640 1 -- 172.21.15.1:0/646753519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34003c60 con 0x7f3b28003a00 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 -- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 msgr2=0x7f3b28003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 0x7f3b28003e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009610 tx=0x7f3b28005830 comp rx=0 tx=0).stop 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 -- 172.21.15.1:0/646753519 shutdown_connections 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28004e60 0x7f3b2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b3c06dfc0 0x7f3b28004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 --2- 172.21.15.1:0/646753519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b28003a00 0x7f3b28003e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 -- 172.21.15.1:0/646753519 >> 172.21.15.1:0/646753519 conn(0x7f3b3c0b7020 msgr2=0x7f3b3c0b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 -- 172.21.15.1:0/646753519 shutdown_connections 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.740+0000 7f3b412d4640 1 -- 172.21.15.1:0/646753519 wait complete. 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.742+0000 7f3b412d4640 1 Processor -- start 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.742+0000 7f3b412d4640 1 -- start start 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 0x7f3b3c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3c06dfc0 0x7f3b3c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1624b0 con 0x7f3b3c06dfc0 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c161cb0 con 0x7f3b28003a00 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b412d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3c162630 con 0x7f3b28004e60 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3affd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 0x7f3b3c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:54506/0 (socket says 172.21.15.1:54506) 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/593900619 learned_addr learned my addr 172.21.15.1:0/593900619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:26:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3b7fe640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3c06dfc0 0x7f3b3c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 msgr2=0x7f3b3c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 0x7f3b3c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/593900619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3c06dfc0 msgr2=0x7f3b3c15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3c06dfc0 0x7f3b3c15d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.743+0000 7f3b3a7fc640 1 -- 172.21.15.1:0/593900619 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c153cd0 con 0x7f3b28004e60 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b3affd640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 0x7f3b3c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b3a7fc640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b24017a60 tx=0x7f3b24017e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24016c80 con 0x7f3b28004e60 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24029070 con 0x7f3b28004e60 2024-05-09T06:26:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c158160 con 0x7f3b28004e60 2024-05-09T06:26:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24024460 con 0x7f3b28004e60 2024-05-09T06:26:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c158720 con 0x7f3b28004e60 2024-05-09T06:26:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b24022030 con 0x7f3b28004e60 2024-05-09T06:26:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.744+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b24024d70 con 0x7f3b28004e60 2024-05-09T06:26:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.745+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3c0752b0 con 0x7f3b28004e60 2024-05-09T06:26:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.750+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3b24033020 con 0x7f3b28004e60 2024-05-09T06:26:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.818+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3b3c0ceb50 con 0x7f3b28004e60 2024-05-09T06:26:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.822+0000 7f3b337fe640 1 -- 172.21.15.1:0/593900619 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3b24020040 con 0x7f3b28004e60 2024-05-09T06:26:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-05-09T06:26:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 msgr2=0x7f3b3c159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b24017a60 tx=0x7f3b24017e60 comp rx=0 tx=0).stop 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 shutdown_connections 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3c06dfc0 0x7f3b3c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b28004e60 0x7f3b3c159200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 --2- 172.21.15.1:0/593900619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b28003a00 0x7f3b3c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 >> 172.21.15.1:0/593900619 conn(0x7f3b3c0b7020 msgr2=0x7f3b3c15b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 shutdown_connections 2024-05-09T06:26:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:56.823+0000 7f3b412d4640 1 -- 172.21.15.1:0/593900619 wait complete. 2024-05-09T06:26:56.817 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T06:26:56.817 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T06:26:56.817 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:56.817 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T06:26:56.836 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:56.854+0000 7f525db10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T06:26:56.841 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:56.859+0000 7f618540e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T06:26:56.845 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:56.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T06:26:56.912 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:56.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T06:26:56.979 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:26:56.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T06:26:56.993 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:57.011+0000 7f525db10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:26:57.035 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:57.053+0000 7f618540e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:26:57.044 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:57.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T06:26:57.048 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:57.067+0000 7f525db10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T06:26:57.070 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:57.071 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T06:26:57.092 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:57.110+0000 7f618540e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T06:26:57.138 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:57.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T06:26:57.203 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:26:57.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T06:26:57.269 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 936044ae-137f-4baa-92c2-b88931dba8a7 0 2024-05-09T06:26:57.310 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:57.328+0000 7f525db10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T06:26:57.347 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:57.365+0000 7f618540e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T06:26:57.359 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:57.377+0000 7f525db10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T06:26:57.395 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:57.413+0000 7f618540e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T06:26:57.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.431+0000 7f4a142ca640 1 Processor -- start 2024-05-09T06:26:57.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.431+0000 7f4a142ca640 1 -- start start 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.432+0000 7f4a142ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a142ca640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 0x7f4a0c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a142ca640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0b4650 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a142ca640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0b14b0 con 0x7f4a0c0d04e0 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a142ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0b11d0 con 0x7f4a0c15be30 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12867640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 0x7f4a0c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40338/0 (socket says 172.21.15.6:40338) 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 -- 172.21.15.6:0/147435782 learned_addr learned my addr 172.21.15.6:0/147435782 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 -- 172.21.15.6:0/147435782 >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 msgr2=0x7f4a0c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 --2- 172.21.15.6:0/147435782 >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 0x7f4a0c0cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 -- 172.21.15.6:0/147435782 >> v1:172.21.15.1:6789/0 conn(0x7f4a0c0b0be0 legacy=0x7f4a0c0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12066640 1 -- 172.21.15.6:0/147435782 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c0cf070 con 0x7f4a0c15be30 2024-05-09T06:26:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.433+0000 7f4a12867640 1 --2- 172.21.15.6:0/147435782 >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 0x7f4a0c0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a12066640 1 --2- 172.21.15.6:0/147435782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009870 tx=0x7f4a0802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94a630ac0325f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a11064640 1 -- 172.21.15.6:0/147435782 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08037070 con 0x7f4a0c15be30 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a11064640 1 -- 172.21.15.6:0/147435782 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08032440 con 0x7f4a0c15be30 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a11064640 1 -- 172.21.15.6:0/147435782 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08032810 con 0x7f4a0c15be30 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a142ca640 1 -- 172.21.15.6:0/147435782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 msgr2=0x7f4a0c0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.434+0000 7f4a142ca640 1 --2- 172.21.15.6:0/147435782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009870 tx=0x7f4a0802a130 comp rx=0 tx=0).stop 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 -- 172.21.15.6:0/147435782 shutdown_connections 2024-05-09T06:26:57.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 --2- 172.21.15.6:0/147435782 >> v2:172.21.15.6:3300/0 conn(0x7f4a0c0d04e0 0x7f4a0c0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 --2- 172.21.15.6:0/147435782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cf810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 -- 172.21.15.6:0/147435782 >> 172.21.15.6:0/147435782 conn(0x7f4a0c0b3750 msgr2=0x7f4a0c0b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 -- 172.21.15.6:0/147435782 shutdown_connections 2024-05-09T06:26:57.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.435+0000 7f4a142ca640 1 -- 172.21.15.6:0/147435782 wait complete. 2024-05-09T06:26:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.436+0000 7f4a142ca640 1 Processor -- start 2024-05-09T06:26:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.436+0000 7f4a142ca640 1 -- start start 2024-05-09T06:26:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0c0d04e0 0x7f4a0c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c15c390 con 0x7f4a0c0d04e0 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c15c210 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c15c510 con 0x7f4a0c15be30 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12867640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12867640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40346/0 (socket says 172.21.15.6:40346) 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55204/0 (socket says 172.21.15.6:55204) 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12867640 1 -- 172.21.15.6:0/161575393 learned_addr learned my addr 172.21.15.6:0/161575393 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a11865640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0c0d04e0 0x7f4a0c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 -- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 msgr2=0x7f4a0c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 -- 172.21.15.6:0/161575393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0c0d04e0 msgr2=0x7f4a0c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0c0d04e0 0x7f4a0c0cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 -- 172.21.15.6:0/161575393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1f27c0 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12867640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a12066640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a08031000 tx=0x7f4a080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08037040 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08042070 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c0ca600 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.437+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08030900 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.438+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c0cabc0 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.438+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a08030aa0 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.438+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4a08047c70 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.439+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c0752b0 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.442+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4a08030d90 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.516+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "936044ae-137f-4baa-92c2-b88931dba8a7", "id": 0} v 0) -- 0x7f4a0c0b7bc0 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.520+0000 7f4a02ffd640 1 -- 172.21.15.6:0/161575393 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "936044ae-137f-4baa-92c2-b88931dba8a7", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4a08047460 con 0x7f4a0c0b0be0 2024-05-09T06:26:57.502 INFO:teuthology.orchestra.run.smithi006.stdout:0 2024-05-09T06:26:57.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 msgr2=0x7f4a0c0c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a08031000 tx=0x7f4a080046c0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 shutdown_connections 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a0c15be30 0x7f4a0c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0c0d04e0 0x7f4a0c0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 --2- 172.21.15.6:0/161575393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a0c0b0be0 0x7f4a0c0c55a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 >> 172.21.15.6:0/161575393 conn(0x7f4a0c0b3750 msgr2=0x7f4a0c06ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 shutdown_connections 2024-05-09T06:26:57.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.521+0000 7f4a142ca640 1 -- 172.21.15.6:0/161575393 wait complete. 2024-05-09T06:26:57.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 4d973a54-6715-4cb9-9841-1bd1cb6da31d 1 2024-05-09T06:26:57.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.633+0000 7f5047c22640 1 Processor -- start 2024-05-09T06:26:57.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 -- start start 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5040156580 0x7f50400708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f504009aa20 0x7f504009ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50400cf3b0 con 0x7f504015cf40 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f504009b3d0 con 0x7f5040156580 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.634+0000 7f5047c22640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f50400cfb50 con 0x7f504009aa20 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f504009aa20 0x7f504009ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f504009aa20 0x7f504009ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55216/0 (socket says 172.21.15.6:55216) 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 -- 172.21.15.6:0/2320512772 learned_addr learned my addr 172.21.15.6:0/2320512772 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50459be640 1 --2- 172.21.15.6:0/2320512772 >> v2:172.21.15.1:3300/0 conn(0x7f5040156580 0x7f50400708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 -- 172.21.15.6:0/2320512772 >> v2:172.21.15.1:3300/0 conn(0x7f5040156580 msgr2=0x7f50400708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 --2- 172.21.15.6:0/2320512772 >> v2:172.21.15.1:3300/0 conn(0x7f5040156580 0x7f50400708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 -- 172.21.15.6:0/2320512772 >> v1:172.21.15.1:6789/0 conn(0x7f504015cf40 legacy=0x7f50400d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.635+0000 7f50461bf640 1 -- 172.21.15.6:0/2320512772 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50400cf690 con 0x7f504009aa20 2024-05-09T06:26:57.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.636+0000 7f50461bf640 1 --2- 172.21.15.6:0/2320512772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504009aa20 0x7f504009ae90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f504006c080 tx=0x7f503802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a5c1b9000e0d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.636+0000 7f50449bc640 1 -- 172.21.15.6:0/2320512772 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038036070 con 0x7f504009aa20 2024-05-09T06:26:57.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.636+0000 7f50449bc640 1 -- 172.21.15.6:0/2320512772 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038031600 con 0x7f504009aa20 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.636+0000 7f50449bc640 1 -- 172.21.15.6:0/2320512772 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380319d0 con 0x7f504009aa20 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 -- 172.21.15.6:0/2320512772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504009aa20 msgr2=0x7f504009ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 --2- 172.21.15.6:0/2320512772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504009aa20 0x7f504009ae90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f504006c080 tx=0x7f503802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 -- 172.21.15.6:0/2320512772 shutdown_connections 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 --2- 172.21.15.6:0/2320512772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504009aa20 0x7f504009ae90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 --2- 172.21.15.6:0/2320512772 >> v2:172.21.15.1:3300/0 conn(0x7f5040156580 0x7f50400708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 -- 172.21.15.6:0/2320512772 >> 172.21.15.6:0/2320512772 conn(0x7f50400b7020 msgr2=0x7f50400b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 -- 172.21.15.6:0/2320512772 shutdown_connections 2024-05-09T06:26:57.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.637+0000 7f5047c22640 1 -- 172.21.15.6:0/2320512772 wait complete. 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.638+0000 7f5047c22640 1 Processor -- start 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.638+0000 7f5047c22640 1 -- start start 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f504009aa20 0x7f5040151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5040070e20 con 0x7f504009aa20 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f50400d0080 con 0x7f504015cf40 2024-05-09T06:26:57.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f5047c22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f50400d05a0 con 0x7f5040156580 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50451bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50451bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40354/0 (socket says 172.21.15.6:40354) 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55230/0 (socket says 172.21.15.6:55230) 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50451bd640 1 -- 172.21.15.6:0/930056130 learned_addr learned my addr 172.21.15.6:0/930056130 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50459be640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f504009aa20 0x7f5040151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 -- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 msgr2=0x7f504009dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 -- 172.21.15.6:0/930056130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f504009aa20 msgr2=0x7f5040151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f504009aa20 0x7f5040151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50461bf640 1 -- 172.21.15.6:0/930056130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401e6da0 con 0x7f504015cf40 2024-05-09T06:26:57.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.639+0000 7f50451bd640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50461bf640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5038030000 tx=0x7f5038004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038036050 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038042070 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401e7080 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038009510 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401f20d0 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5038007bd0 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.640+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f503803d450 con 0x7f504015cf40 2024-05-09T06:26:57.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.641+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50400752b0 con 0x7f504015cf40 2024-05-09T06:26:57.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.645+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5038047460 con 0x7f504015cf40 2024-05-09T06:26:57.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.719+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4d973a54-6715-4cb9-9841-1bd1cb6da31d", "id": 1} v 0) -- 0x7f5040153f20 con 0x7f504015cf40 2024-05-09T06:26:57.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.721+0000 7f50327fc640 1 -- 172.21.15.6:0/930056130 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d973a54-6715-4cb9-9841-1bd1cb6da31d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f5038047640 con 0x7f504015cf40 2024-05-09T06:26:57.703 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 msgr2=0x7f504009e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5038030000 tx=0x7f5038004a40 comp rx=0 tx=0).stop 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 shutdown_connections 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f504015cf40 0x7f504009e020 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5040156580 0x7f504009dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 --2- 172.21.15.6:0/930056130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f504009aa20 0x7f5040151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.722+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 >> 172.21.15.6:0/930056130 conn(0x7f50400b7020 msgr2=0x7f50400b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.723+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 shutdown_connections 2024-05-09T06:26:57.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.723+0000 7f5047c22640 1 -- 172.21.15.6:0/930056130 wait complete. 2024-05-09T06:26:57.715 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 75cf529c-4960-4054-8bb5-c4a1cf3b3a45 2 2024-05-09T06:26:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.838+0000 7f8a60977640 1 Processor -- start 2024-05-09T06:26:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.838+0000 7f8a60977640 1 -- start start 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a60977640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a5c0d1c70 0x7f8a5c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a60977640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5c161fe0 con 0x7f8a5c0d45f0 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a60977640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5c1622c0 con 0x7f8a5c15e190 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a60977640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c15f580 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a5a59c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8a5c15e190 0x7f8a5c0d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:41496/0 (socket says 172.21.15.6:41496) 2024-05-09T06:26:57.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a5a59c640 1 -- 172.21.15.6:0/244121378 learned_addr learned my addr 172.21.15.6:0/244121378 (peer_addr_for_me v1:172.21.15.6:0/0) 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a53fff640 1 --2- 172.21.15.6:0/244121378 >> v2:172.21.15.1:3300/0 conn(0x7f8a5c0d1c70 0x7f8a5c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231652473 0 0) 0x7f8a5c1622c0 con 0x7f8a5c15e190 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a48003520 con 0x7f8a5c15e190 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052383855 0 0) 0x7f8a5c161fe0 con 0x7f8a5c0d45f0 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a5c1622c0 con 0x7f8a5c0d45f0 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a53fff640 1 -- 172.21.15.6:0/244121378 >> v1:172.21.15.6:6789/0 conn(0x7f8a5c15e190 legacy=0x7f8a5c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a53fff640 1 -- 172.21.15.6:0/244121378 >> v1:172.21.15.1:6789/0 conn(0x7f8a5c0d45f0 legacy=0x7f8a5c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.839+0000 7f8a53fff640 1 -- 172.21.15.6:0/244121378 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c1625a0 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.840+0000 7f8a53fff640 1 --2- 172.21.15.6:0/244121378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c15f780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a440097a0 tx=0x7f8a4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54a6945db63b553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.840+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44037070 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.840+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a44032440 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.840+0000 7f8a59d9b640 1 -- 172.21.15.6:0/244121378 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44032810 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 -- 172.21.15.6:0/244121378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c0d1c70 msgr2=0x7f8a5c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 --2- 172.21.15.6:0/244121378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c15f780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a440097a0 tx=0x7f8a4402a230 comp rx=0 tx=0).stop 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 -- 172.21.15.6:0/244121378 shutdown_connections 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 --2- 172.21.15.6:0/244121378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c15f780 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 -- 172.21.15.6:0/244121378 >> 172.21.15.6:0/244121378 conn(0x7f8a5c073340 msgr2=0x7f8a5c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 -- 172.21.15.6:0/244121378 shutdown_connections 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.841+0000 7f8a60977640 1 -- 172.21.15.6:0/244121378 wait complete. 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.842+0000 7f8a60977640 1 Processor -- start 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.842+0000 7f8a60977640 1 -- start start 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 0x7f8a5c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 0x7f8a5c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c0b56d0 con 0x7f8a5c15e190 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c161c60 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a60977640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c0b5850 con 0x7f8a5c0d45f0 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 0x7f8a5c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55238/0 (socket says 172.21.15.6:55238) 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 -- 172.21.15.6:0/1766604944 learned_addr learned my addr 172.21.15.6:0/1766604944 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5ad9d640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 0x7f8a5c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 -- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 msgr2=0x7f8a5c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 0x7f8a5c0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 -- 172.21.15.6:0/1766604944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 msgr2=0x7f8a5c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 0x7f8a5c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5a59c640 1 -- 172.21.15.6:0/1766604944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c0cba50 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a53fff640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 0x7f8a5c0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.843+0000 7f8a5ad9d640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 0x7f8a5c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a5a59c640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c470 tx=0x7f8a4c00c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c017070 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4c005ce0 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c007290 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c0cbd30 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c0cc2f0 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.844+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8a4c012820 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.845+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c0752b0 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.847+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8a4c021070 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.923+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "75cf529c-4960-4054-8bb5-c4a1cf3b3a45", "id": 2} v 0) -- 0x7f8a5c08e1c0 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.926+0000 7f8a52ffd640 1 -- 172.21.15.6:0/1766604944 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75cf529c-4960-4054-8bb5-c4a1cf3b3a45", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8a4c012400 con 0x7f8a5c0d1c70 2024-05-09T06:26:57.908 INFO:teuthology.orchestra.run.smithi006.stdout:2 2024-05-09T06:26:57.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 msgr2=0x7f8a5c159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:57.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c470 tx=0x7f8a4c00c930 comp rx=0 tx=0).stop 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 shutdown_connections 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a5c15e190 0x7f8a5c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5c0d45f0 0x7f8a5c0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 --2- 172.21.15.6:0/1766604944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a5c0d1c70 0x7f8a5c159440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 >> 172.21.15.6:0/1766604944 conn(0x7f8a5c073340 msgr2=0x7f8a5c15d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 shutdown_connections 2024-05-09T06:26:57.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:57.927+0000 7f8a60977640 1 -- 172.21.15.6:0/1766604944 wait complete. 2024-05-09T06:26:57.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 37bd81bc-61f1-4708-8a99-57fff41b8bd5 3 2024-05-09T06:26:58.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.042+0000 7fe7180c4640 1 Processor -- start 2024-05-09T06:26:58.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.042+0000 7fe7180c4640 1 -- start start 2024-05-09T06:26:58.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 0x7fe710071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe71006e6d0 0x7fe71006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe710158e30 con 0x7fe71006e6d0 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe710072480 con 0x7fe71015c4a0 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe7180c4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe71015b420 con 0x7fe710158a50 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe715e60640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe71565f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 0x7fe710071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe715e60640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40380/0 (socket says 172.21.15.6:40380) 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe715e60640 1 -- 172.21.15.6:0/186038933 learned_addr learned my addr 172.21.15.6:0/186038933 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.043+0000 7fe716661640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.1:3300/0 conn(0x7fe71006e6d0 0x7fe71006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 -- 172.21.15.6:0/186038933 >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 msgr2=0x7fe710071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 0x7fe710071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 -- 172.21.15.6:0/186038933 >> v2:172.21.15.1:3300/0 conn(0x7fe71006e6d0 msgr2=0x7fe71006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.1:3300/0 conn(0x7fe71006e6d0 0x7fe71006eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 -- 172.21.15.6:0/186038933 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71015b6e0 con 0x7fe710158a50 2024-05-09T06:26:58.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe71565f640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 0x7fe710071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe715e60640 1 --2- 172.21.15.6:0/186038933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7100b2770 tx=0x7fe70c02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38229d577085feac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe714e5e640 1 -- 172.21.15.6:0/186038933 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c037070 con 0x7fe710158a50 2024-05-09T06:26:58.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.044+0000 7fe714e5e640 1 -- 172.21.15.6:0/186038933 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c032440 con 0x7fe710158a50 2024-05-09T06:26:58.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe714e5e640 1 -- 172.21.15.6:0/186038933 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c032810 con 0x7fe710158a50 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 -- 172.21.15.6:0/186038933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 msgr2=0x7fe7100718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 --2- 172.21.15.6:0/186038933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7100b2770 tx=0x7fe70c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 -- 172.21.15.6:0/186038933 shutdown_connections 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.1:3300/0 conn(0x7fe71006e6d0 0x7fe71006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 --2- 172.21.15.6:0/186038933 >> v2:172.21.15.6:3300/0 conn(0x7fe71015c4a0 0x7fe710071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 --2- 172.21.15.6:0/186038933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe7100718c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 -- 172.21.15.6:0/186038933 >> 172.21.15.6:0/186038933 conn(0x7fe7100cc4d0 msgr2=0x7fe7100cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 -- 172.21.15.6:0/186038933 shutdown_connections 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.045+0000 7fe7180c4640 1 -- 172.21.15.6:0/186038933 wait complete. 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.046+0000 7fe7180c4640 1 Processor -- start 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.046+0000 7fe7180c4640 1 -- start start 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe710146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe71015c4a0 0x7fe710150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe71015bf80 con 0x7fe71015c4a0 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe71015bca0 con 0x7fe71006e6d0 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe7180c4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe710070ea0 con 0x7fe710158a50 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe715e60640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe71565f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe710146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe715e60640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55250/0 (socket says 172.21.15.6:55250) 2024-05-09T06:26:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe715e60640 1 -- 172.21.15.6:0/2479656157 learned_addr learned my addr 172.21.15.6:0/2479656157 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.047+0000 7fe716661640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe71015c4a0 0x7fe710150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 -- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 msgr2=0x7fe710146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe710146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 -- 172.21.15.6:0/2479656157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe71015c4a0 msgr2=0x7fe710150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe71015c4a0 0x7fe710150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 -- 172.21.15.6:0/2479656157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100afed0 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe71565f640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe710146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe715e60640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000c00 tx=0x7fe70c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c041070 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c032e50 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c037040 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7100b01b0 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7100b0770 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe70c044070 con 0x7fe71006e6d0 2024-05-09T06:26:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.048+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe70c046cf0 con 0x7fe71006e6d0 2024-05-09T06:26:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.049+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7100b0f70 con 0x7fe71006e6d0 2024-05-09T06:26:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.051+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe70c046450 con 0x7fe71006e6d0 2024-05-09T06:26:58.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.125+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37bd81bc-61f1-4708-8a99-57fff41b8bd5", "id": 3} v 0) -- 0x7fe7100752b0 con 0x7fe71006e6d0 2024-05-09T06:26:58.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.128+0000 7fe7067fc640 1 -- 172.21.15.6:0/2479656157 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37bd81bc-61f1-4708-8a99-57fff41b8bd5", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe70c046860 con 0x7fe71006e6d0 2024-05-09T06:26:58.110 INFO:teuthology.orchestra.run.smithi006.stdout:3 2024-05-09T06:26:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 msgr2=0x7fe7100af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000c00 tx=0x7fe70c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 shutdown_connections 2024-05-09T06:26:58.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe71015c4a0 0x7fe710150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe710158a50 0x7fe710146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 --2- 172.21.15.6:0/2479656157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe71006e6d0 0x7fe7100af5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 >> 172.21.15.6:0/2479656157 conn(0x7fe7100cc4d0 msgr2=0x7fe710159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 shutdown_connections 2024-05-09T06:26:58.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.129+0000 7fe7180c4640 1 -- 172.21.15.6:0/2479656157 wait complete. 2024-05-09T06:26:58.121 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 62d462ba-310a-43bf-8f74-53e0341b086f 4 2024-05-09T06:26:58.149 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.167+0000 7f525db10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T06:26:58.161 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.179+0000 7f618540e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T06:26:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.242+0000 7feb34ee7640 1 Processor -- start 2024-05-09T06:26:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.242+0000 7feb34ee7640 1 -- start start 2024-05-09T06:26:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb34ee7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feb3006c2f0 0x7feb300ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb34ee7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb3015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb34ee7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb300cd2f0 con 0x7feb300b4a50 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb34ee7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7feb300cd5d0 con 0x7feb3006c2f0 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb34ee7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7feb300cd8b0 con 0x7feb3006cde0 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb2e59c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feb3006c2f0 0x7feb300ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb2e59c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feb3006c2f0 0x7feb300ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55254/0 (socket says 172.21.15.6:55254) 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb2e59c640 1 -- 172.21.15.6:0/1595460866 learned_addr learned my addr 172.21.15.6:0/1595460866 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb27fff640 1 --1- 172.21.15.6:0/1595460866 >> v1:172.21.15.6:6789/0 conn(0x7feb300b4a50 0x7feb3006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:41500/0 (socket says 172.21.15.6:41500) 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.243+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb3015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2e59c640 1 -- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 msgr2=0x7feb3015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2e59c640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb3015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2e59c640 1 -- 172.21.15.6:0/1595460866 >> v1:172.21.15.6:6789/0 conn(0x7feb300b4a50 legacy=0x7feb3006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2e59c640 1 -- 172.21.15.6:0/1595460866 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb300cdb90 con 0x7feb3006c2f0 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2dd9b640 1 -- 172.21.15.6:0/1595460866 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384978822 0 0) 0x7feb300cd2f0 con 0x7feb300b4a50 2024-05-09T06:26:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2ed9d640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb3015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2e59c640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb3006c2f0 0x7feb300ccc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb20009870 tx=0x7feb2002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83e2260cd910157 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.244+0000 7feb2dd9b640 1 -- 172.21.15.6:0/1595460866 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20037070 con 0x7feb3006c2f0 2024-05-09T06:26:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb2dd9b640 1 -- 172.21.15.6:0/1595460866 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2002ae50 con 0x7feb3006c2f0 2024-05-09T06:26:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb2dd9b640 1 -- 172.21.15.6:0/1595460866 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20032860 con 0x7feb3006c2f0 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 -- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb3006c2f0 msgr2=0x7feb300ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb3006c2f0 0x7feb300ccc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb20009870 tx=0x7feb2002a300 comp rx=0 tx=0).stop 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 -- 172.21.15.6:0/1595460866 shutdown_connections 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb3015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 --2- 172.21.15.6:0/1595460866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb3006c2f0 0x7feb300ccc30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 -- 172.21.15.6:0/1595460866 >> 172.21.15.6:0/1595460866 conn(0x7feb300b3730 msgr2=0x7feb300b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 -- 172.21.15.6:0/1595460866 shutdown_connections 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.245+0000 7feb34ee7640 1 -- 172.21.15.6:0/1595460866 wait complete. 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 Processor -- start 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 -- start start 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3006c2f0 0x7feb30155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb301557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b7a60 con 0x7feb3006c2f0 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7feb300d1250 con 0x7feb300b4a50 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb34ee7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7feb3015ddb0 con 0x7feb3006cde0 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb27fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb301557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55258/0 (socket says 172.21.15.6:55258) 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.247+0000 7feb27fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb301557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40422/0 (socket says 172.21.15.6:40422) 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 -- 172.21.15.6:0/412931233 learned_addr learned my addr 172.21.15.6:0/412931233 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2e59c640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3006c2f0 0x7feb30155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 -- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 msgr2=0x7feb301557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb301557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 -- 172.21.15.6:0/412931233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3006c2f0 msgr2=0x7feb30155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3006c2f0 0x7feb30155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 -- 172.21.15.6:0/412931233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301564d0 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb2ed9d640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb2800efc0 tx=0x7feb28009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28018030 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2800cde0 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301567b0 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28003bb0 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30156d70 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb2801b070 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.248+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7feb2801da50 con 0x7feb300b4a50 2024-05-09T06:26:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.250+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb300752b0 con 0x7feb300b4a50 2024-05-09T06:26:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.252+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feb28010040 con 0x7feb300b4a50 2024-05-09T06:26:58.245 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.263+0000 7f525db10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T06:26:58.252 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.270+0000 7f618540e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T06:26:58.290 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.308+0000 7f525db10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T06:26:58.297 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.315+0000 7f618540e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T06:26:58.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.326+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62d462ba-310a-43bf-8f74-53e0341b086f", "id": 4} v 0) -- 0x7feb300cef30 con 0x7feb300b4a50 2024-05-09T06:26:58.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.329+0000 7feb26ffd640 1 -- 172.21.15.6:0/412931233 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62d462ba-310a-43bf-8f74-53e0341b086f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7feb2801d500 con 0x7feb300b4a50 2024-05-09T06:26:58.311 INFO:teuthology.orchestra.run.smithi006.stdout:4 2024-05-09T06:26:58.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 msgr2=0x7feb300c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb2800efc0 tx=0x7feb28009f90 comp rx=0 tx=0).stop 2024-05-09T06:26:58.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 shutdown_connections 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb300b4a50 0x7feb300c7a70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb3006cde0 0x7feb301557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 --2- 172.21.15.6:0/412931233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3006c2f0 0x7feb30155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 >> 172.21.15.6:0/412931233 conn(0x7feb300b3730 msgr2=0x7feb300b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 shutdown_connections 2024-05-09T06:26:58.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.330+0000 7feb34ee7640 1 -- 172.21.15.6:0/412931233 wait complete. 2024-05-09T06:26:58.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 1d54b45c-4b3c-4a15-9fac-c84f3d8d235d 5 2024-05-09T06:26:58.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.484+0000 7f40bd370640 1 Processor -- start 2024-05-09T06:26:58.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.484+0000 7f40bd370640 1 -- start start 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40bd370640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 0x7f40b815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40bd370640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b8161fe0 con 0x7f40b815e190 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40bd370640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b81622c0 con 0x7f40b80d45f0 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40bd370640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f40b815f580 con 0x7f40b80d1c70 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40affff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 0x7f40b815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40b77fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f40b80d45f0 0x7f40b80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:41502/0 (socket says 172.21.15.6:41502) 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.485+0000 7f40b77fe640 1 -- 172.21.15.6:0/4166443665 learned_addr learned my addr 172.21.15.6:0/4166443665 (peer_addr_for_me v1:172.21.15.6:0/0) 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664311471 0 0) 0x7f40b81622c0 con 0x7f40b80d45f0 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40a4003520 con 0x7f40b80d45f0 2024-05-09T06:26:58.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536682239 0 0) 0x7f40b8161fe0 con 0x7f40b815e190 2024-05-09T06:26:58.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40affff640 1 -- 172.21.15.6:0/4166443665 >> v1:172.21.15.6:6789/0 conn(0x7f40b80d45f0 legacy=0x7f40b80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40affff640 1 -- 172.21.15.6:0/4166443665 >> v1:172.21.15.1:6789/0 conn(0x7f40b815e190 legacy=0x7f40b80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.486+0000 7f40affff640 1 -- 172.21.15.6:0/4166443665 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81625a0 con 0x7f40b80d1c70 2024-05-09T06:26:58.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.487+0000 7f40affff640 1 --2- 172.21.15.6:0/4166443665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 0x7f40b815f780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f40a00097d0 tx=0x7f40a002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8757a3d9f8304163 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.487+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0037070 con 0x7f40b80d1c70 2024-05-09T06:26:58.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.487+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a0032440 con 0x7f40b80d1c70 2024-05-09T06:26:58.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.487+0000 7f40b67fc640 1 -- 172.21.15.6:0/4166443665 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0032810 con 0x7f40b80d1c70 2024-05-09T06:26:58.470 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.487+0000 7f525db10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 -- 172.21.15.6:0/4166443665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 msgr2=0x7f40b815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 --2- 172.21.15.6:0/4166443665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 0x7f40b815f780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f40a00097d0 tx=0x7f40a002a210 comp rx=0 tx=0).stop 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 -- 172.21.15.6:0/4166443665 shutdown_connections 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 --2- 172.21.15.6:0/4166443665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d1c70 0x7f40b815f780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 -- 172.21.15.6:0/4166443665 >> 172.21.15.6:0/4166443665 conn(0x7f40b8073340 msgr2=0x7f40b809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 -- 172.21.15.6:0/4166443665 shutdown_connections 2024-05-09T06:26:58.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.488+0000 7f40bd370640 1 -- 172.21.15.6:0/4166443665 wait complete. 2024-05-09T06:26:58.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.489+0000 7f40bd370640 1 Processor -- start 2024-05-09T06:26:58.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.489+0000 7f40bd370640 1 -- start start 2024-05-09T06:26:58.472 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.488+0000 7f618540e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T06:26:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40b80d1c70 0x7f40b80ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 0x7f40b80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80b56d0 con 0x7f40b80d1c70 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8161c60 con 0x7f40b815e190 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40bd370640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f40b80b5850 con 0x7f40b80d45f0 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40affff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 0x7f40b80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40affff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40438/0 (socket says 172.21.15.6:40438) 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40affff640 1 -- 172.21.15.6:0/1884113676 learned_addr learned my addr 172.21.15.6:0/1884113676 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b6ffd640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40b80d1c70 0x7f40b80ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 -- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 msgr2=0x7f40b806d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 -- 172.21.15.6:0/1884113676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40b80d1c70 msgr2=0x7f40b80ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40b80d1c70 0x7f40b80ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40b77fe640 1 -- 172.21.15.6:0/1884113676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80ceda0 con 0x7f40b815e190 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.490+0000 7f40affff640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40b77fe640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 0x7f40b80cd580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f40b000ca50 tx=0x7f40b000cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0017070 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b000ab50 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b80cf080 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0012400 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b80cf640 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40b001a070 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.491+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f40b001cce0 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.492+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b80752b0 con 0x7f40b815e190 2024-05-09T06:26:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.494+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f40b001c760 con 0x7f40b815e190 2024-05-09T06:26:58.526 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.544+0000 7f618540e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T06:26:58.528 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.546+0000 7f525db10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T06:26:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.579+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d54b45c-4b3c-4a15-9fac-c84f3d8d235d", "id": 5} v 0) -- 0x7f40b808e1c0 con 0x7f40b815e190 2024-05-09T06:26:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.582+0000 7f40af7fe640 1 -- 172.21.15.6:0/1884113676 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d54b45c-4b3c-4a15-9fac-c84f3d8d235d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f40b001c940 con 0x7f40b815e190 2024-05-09T06:26:58.567 INFO:teuthology.orchestra.run.smithi006.stdout:5 2024-05-09T06:26:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 msgr2=0x7f40b80cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 0x7f40b80cd580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f40b000ca50 tx=0x7f40b000cf10 comp rx=0 tx=0).stop 2024-05-09T06:26:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 shutdown_connections 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40b815e190 0x7f40b80cd580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40b80d45f0 0x7f40b806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 --2- 172.21.15.6:0/1884113676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40b80d1c70 0x7f40b80ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 >> 172.21.15.6:0/1884113676 conn(0x7f40b8073340 msgr2=0x7f40b815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 shutdown_connections 2024-05-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.583+0000 7f40bd370640 1 -- 172.21.15.6:0/1884113676 wait complete. 2024-05-09T06:26:58.576 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 09b68656-4d19-45d1-853b-7a886409ed0c 6 2024-05-09T06:26:58.581 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.599+0000 7f618540e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T06:26:58.585 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.604+0000 7f525db10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T06:26:58.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.696+0000 7f7b5dc41640 1 Processor -- start 2024-05-09T06:26:58.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b5dc41640 1 -- start start 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b5dc41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7b5815e190 0x7f7b5815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b5dc41640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b58161fe0 con 0x7f7b580d45f0 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b5dc41640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b581622c0 con 0x7f7b580d1c70 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b5dc41640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7b5815f580 con 0x7f7b5815e190 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.697+0000 7f7b4effd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7b580d1c70 0x7f7b580d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:41508/0 (socket says 172.21.15.6:41508) 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b4effd640 1 -- 172.21.15.6:0/1415499660 learned_addr learned my addr 172.21.15.6:0/1415499660 (peer_addr_for_me v1:172.21.15.6:0/0) 2024-05-09T06:26:58.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b577fe640 1 --2- 172.21.15.6:0/1415499660 >> v2:172.21.15.1:3300/0 conn(0x7f7b5815e190 0x7f7b5815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1097613870 0 0) 0x7f7b581622c0 con 0x7f7b580d1c70 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b44003520 con 0x7f7b580d1c70 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b577fe640 1 -- 172.21.15.6:0/1415499660 >> v1:172.21.15.6:6789/0 conn(0x7f7b580d1c70 legacy=0x7f7b580d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b577fe640 1 -- 172.21.15.6:0/1415499660 >> v1:172.21.15.1:6789/0 conn(0x7f7b580d45f0 legacy=0x7f7b580b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909684735 0 0) 0x7f7b58161fe0 con 0x7f7b580d45f0 2024-05-09T06:26:58.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.698+0000 7f7b577fe640 1 -- 172.21.15.6:0/1415499660 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b581625a0 con 0x7f7b5815e190 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.699+0000 7f7b577fe640 1 --2- 172.21.15.6:0/1415499660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b5815f780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7b480097a0 tx=0x7f7b4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1640f383ab9b71b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.699+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48037070 con 0x7f7b5815e190 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.699+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b48032440 con 0x7f7b5815e190 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.699+0000 7f7b56ffd640 1 -- 172.21.15.6:0/1415499660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48032810 con 0x7f7b5815e190 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 -- 172.21.15.6:0/1415499660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 msgr2=0x7f7b5815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/1415499660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b5815f780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7b480097a0 tx=0x7f7b4802a230 comp rx=0 tx=0).stop 2024-05-09T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 -- 172.21.15.6:0/1415499660 shutdown_connections 2024-05-09T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/1415499660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b5815f780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 -- 172.21.15.6:0/1415499660 >> 172.21.15.6:0/1415499660 conn(0x7f7b58073340 msgr2=0x7f7b5809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 -- 172.21.15.6:0/1415499660 shutdown_connections 2024-05-09T06:26:58.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.700+0000 7f7b5dc41640 1 -- 172.21.15.6:0/1415499660 wait complete. 2024-05-09T06:26:58.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.701+0000 7f7b5dc41640 1 Processor -- start 2024-05-09T06:26:58.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.701+0000 7f7b5dc41640 1 -- start start 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 0x7f7b580b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b581f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580b56d0 con 0x7f7b5815e190 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58161c60 con 0x7f7b580d1c70 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b5dc41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b580b5850 con 0x7f7b580d45f0 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b577fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 0x7f7b580b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b577fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55266/0 (socket says 172.21.15.6:55266) 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b577fe640 1 -- 172.21.15.6:0/274079168 learned_addr learned my addr 172.21.15.6:0/274079168 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b57fff640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b581f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 -- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 msgr2=0x7f7b580b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 -- 172.21.15.6:0/274079168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 msgr2=0x7f7b581f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b581f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 -- 172.21.15.6:0/274079168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580cb550 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b577fe640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.702+0000 7f7b4effd640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 0x7f7b580b2200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b580d20f0 tx=0x7f7b4000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40017070 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b40005ce0 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b580cb830 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400072f0 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b580cbdf0 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b4001a070 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.703+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f7b400129f0 con 0x7f7b580d45f0 2024-05-09T06:26:58.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.704+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b580752b0 con 0x7f7b580d45f0 2024-05-09T06:26:58.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.707+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7b40021020 con 0x7f7b580d45f0 2024-05-09T06:26:58.699 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:58.717+0000 7f618540e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:26:58.709 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:58.727+0000 7f525db10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:26:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.782+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "09b68656-4d19-45d1-853b-7a886409ed0c", "id": 6} v 0) -- 0x7f7b5808e1c0 con 0x7f7b580d45f0 2024-05-09T06:26:58.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.788+0000 7f7b54ff9640 1 -- 172.21.15.6:0/274079168 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09b68656-4d19-45d1-853b-7a886409ed0c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f7b40007650 con 0x7f7b580d45f0 2024-05-09T06:26:58.770 INFO:teuthology.orchestra.run.smithi006.stdout:6 2024-05-09T06:26:58.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 msgr2=0x7f7b580b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 0x7f7b580b2200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b580d20f0 tx=0x7f7b4000c890 comp rx=0 tx=0).stop 2024-05-09T06:26:58.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 shutdown_connections 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b5815e190 0x7f7b581f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b580d45f0 0x7f7b580b2200 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 --2- 172.21.15.6:0/274079168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b580d1c70 0x7f7b580b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 >> 172.21.15.6:0/274079168 conn(0x7f7b58073340 msgr2=0x7f7b5806cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 shutdown_connections 2024-05-09T06:26:58.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.789+0000 7f7b5dc41640 1 -- 172.21.15.6:0/274079168 wait complete. 2024-05-09T06:26:58.782 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 86cb3022-da70-45da-8eeb-4186d6bc51f6 7 2024-05-09T06:26:58.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.944+0000 7f053e130640 1 Processor -- start 2024-05-09T06:26:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.944+0000 7f053e130640 1 -- start start 2024-05-09T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f053815ea80 0x7f053809aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f053815f4c0 con 0x7f053815ea80 2024-05-09T06:26:58.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0538162020 con 0x7f053806dfc0 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f053e130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05381629f0 con 0x7f05380d2870 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f0536ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f0536ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55282/0 (socket says 172.21.15.6:55282) 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f0536ffd640 1 -- 172.21.15.6:0/2298919949 learned_addr learned my addr 172.21.15.6:0/2298919949 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f0537fff640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.1:3300/0 conn(0x7f053815ea80 0x7f053809aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 -- 172.21.15.6:0/2298919949 >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 msgr2=0x7f053815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 -- 172.21.15.6:0/2298919949 >> v2:172.21.15.1:3300/0 conn(0x7f053815ea80 msgr2=0x7f053809aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.1:3300/0 conn(0x7f053815ea80 0x7f053809aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f05377fe640 1 -- 172.21.15.6:0/2298919949 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053815ef00 con 0x7f05380d2870 2024-05-09T06:26:58.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.945+0000 7f0536ffd640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.946+0000 7f05377fe640 1 --2- 172.21.15.6:0/2298919949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381624b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05380b47d0 tx=0x7f052802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d32b57f4389c97e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.946+0000 7f05367fc640 1 -- 172.21.15.6:0/2298919949 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528037070 con 0x7f05380d2870 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.946+0000 7f05367fc640 1 -- 172.21.15.6:0/2298919949 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052802add0 con 0x7f05380d2870 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.946+0000 7f05367fc640 1 -- 172.21.15.6:0/2298919949 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528032660 con 0x7f05380d2870 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 -- 172.21.15.6:0/2298919949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 msgr2=0x7f05381624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 --2- 172.21.15.6:0/2298919949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381624b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05380b47d0 tx=0x7f052802a080 comp rx=0 tx=0).stop 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 -- 172.21.15.6:0/2298919949 shutdown_connections 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.1:3300/0 conn(0x7f053815ea80 0x7f053809aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 --2- 172.21.15.6:0/2298919949 >> v2:172.21.15.6:3300/0 conn(0x7f053806dfc0 0x7f053815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 --2- 172.21.15.6:0/2298919949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381624b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 -- 172.21.15.6:0/2298919949 >> 172.21.15.6:0/2298919949 conn(0x7f05380b7020 msgr2=0x7f05380b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:58.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 -- 172.21.15.6:0/2298919949 shutdown_connections 2024-05-09T06:26:58.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.947+0000 7f053e130640 1 -- 172.21.15.6:0/2298919949 wait complete. 2024-05-09T06:26:58.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.948+0000 7f053e130640 1 Processor -- start 2024-05-09T06:26:58.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.948+0000 7f053e130640 1 -- start start 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 0x7f05381f1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f053815ea80 0x7f05380d1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f053809af60 con 0x7f053806dfc0 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0538161cb0 con 0x7f053815ea80 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f053e130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f053809b0e0 con 0x7f05380d2870 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0537fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f053815ea80 0x7f05380d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:40456/0 (socket says 172.21.15.6:40456) 2024-05-09T06:26:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 -- 172.21.15.6:0/1069271610 learned_addr learned my addr 172.21.15.6:0/1069271610 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f05377fe640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 0x7f05381f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 -- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f053815ea80 msgr2=0x7f05380d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f053815ea80 0x7f05380d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 -- 172.21.15.6:0/1069271610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 msgr2=0x7f05381f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 0x7f05381f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f0536ffd640 1 -- 172.21.15.6:0/1069271610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05380aff70 con 0x7f05380d2870 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.949+0000 7f05377fe640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 0x7f05381f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f0536ffd640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e9d0 tx=0x7f052c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c017070 con 0x7f05380d2870 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c00ca60 con 0x7f05380d2870 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05381f4ba0 con 0x7f05380d2870 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c00ce10 con 0x7f05380d2870 2024-05-09T06:26:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05380d0590 con 0x7f05380d2870 2024-05-09T06:26:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f052c01a070 con 0x7f05380d2870 2024-05-09T06:26:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.950+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f052c006520 con 0x7f05380d2870 2024-05-09T06:26:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.951+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05380752b0 con 0x7f05380d2870 2024-05-09T06:26:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:58.955+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f05381f4ba0 con 0x7f05380d2870 2024-05-09T06:26:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.028+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "86cb3022-da70-45da-8eeb-4186d6bc51f6", "id": 7} v 0) -- 0x7f05380b0db0 con 0x7f05380d2870 2024-05-09T06:26:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.036+0000 7f051ffff640 1 -- 172.21.15.6:0/1069271610 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "86cb3022-da70-45da-8eeb-4186d6bc51f6", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f052c005990 con 0x7f05380d2870 2024-05-09T06:26:59.018 INFO:teuthology.orchestra.run.smithi006.stdout:7 2024-05-09T06:26:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 msgr2=0x7f05381f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f052c00e9d0 tx=0x7f052c00ee90 comp rx=0 tx=0).stop 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 shutdown_connections 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f053815ea80 0x7f05380d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05380d2870 0x7f05381f4660 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 --2- 172.21.15.6:0/1069271610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053806dfc0 0x7f05381f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.037+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 >> 172.21.15.6:0/1069271610 conn(0x7f05380b7020 msgr2=0x7f05380b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.038+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 shutdown_connections 2024-05-09T06:26:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:26:59.038+0000 7f053e130640 1 -- 172.21.15.6:0/1069271610 wait complete. 2024-05-09T06:26:59.030 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T06:26:59.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T06:26:59.032 INFO:tasks.ceph.osd.0:Started 2024-05-09T06:26:59.032 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T06:26:59.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T06:26:59.034 INFO:tasks.ceph.osd.1:Started 2024-05-09T06:26:59.035 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T06:26:59.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T06:26:59.037 INFO:tasks.ceph.osd.2:Started 2024-05-09T06:26:59.037 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T06:26:59.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T06:26:59.039 INFO:tasks.ceph.osd.3:Started 2024-05-09T06:26:59.039 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T06:26:59.040 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T06:26:59.042 INFO:tasks.ceph.osd.4:Started 2024-05-09T06:26:59.042 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T06:26:59.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T06:26:59.044 INFO:tasks.ceph.osd.5:Started 2024-05-09T06:26:59.044 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T06:26:59.044 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T06:26:59.046 INFO:tasks.ceph.osd.6:Started 2024-05-09T06:26:59.046 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T06:26:59.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T06:26:59.050 INFO:tasks.ceph.osd.7:Started 2024-05-09T06:26:59.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:26:59.116 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.134+0000 7f618540e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T06:26:59.144 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.162+0000 7f525db10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T06:26:59.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.193+0000 7fae829c1640 1 Processor -- start 2024-05-09T06:26:59.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.193+0000 7fae829c1640 1 -- start start 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae829c1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae7c155250 0x7fae7c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae829c1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c155630 con 0x7fae7c06f7a0 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae829c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c06ff80 con 0x7fae7c0723f0 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae829c1640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fae7c0cb480 con 0x7fae7c155250 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae7bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae7c155250 0x7fae7c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae7b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fae7c06f7a0 0x7fae7c071ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49398/0 (socket says 172.21.15.1:49398) 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae7bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae7c155250 0x7fae7c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47772/0 (socket says 172.21.15.1:47772) 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.194+0000 7fae7b7fe640 1 -- 172.21.15.1:0/1951917239 learned_addr learned my addr 172.21.15.1:0/1951917239 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.195+0000 7fae7bfff640 1 -- 172.21.15.1:0/1951917239 >> v1:172.21.15.6:6789/0 conn(0x7fae7c0723f0 legacy=0x7fae7c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.195+0000 7fae7bfff640 1 -- 172.21.15.1:0/1951917239 >> v1:172.21.15.1:6789/0 conn(0x7fae7c06f7a0 legacy=0x7fae7c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.195+0000 7fae7bfff640 1 -- 172.21.15.1:0/1951917239 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c070260 con 0x7fae7c155250 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.195+0000 7fae7bfff640 1 --2- 172.21.15.1:0/1951917239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c155250 0x7fae7c15bd80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009720 tx=0x7fae6c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62359813d08f28c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.195+0000 7fae7affd640 1 -- 172.21.15.1:0/1951917239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037070 con 0x7fae7c155250 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae7affd640 1 -- 172.21.15.1:0/1951917239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c02ade0 con 0x7fae7c155250 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae7affd640 1 -- 172.21.15.1:0/1951917239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c032660 con 0x7fae7c155250 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 -- 172.21.15.1:0/1951917239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c155250 msgr2=0x7fae7c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 --2- 172.21.15.1:0/1951917239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c155250 0x7fae7c15bd80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009720 tx=0x7fae6c02a0d0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 -- 172.21.15.1:0/1951917239 shutdown_connections 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 --2- 172.21.15.1:0/1951917239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c155250 0x7fae7c15bd80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 -- 172.21.15.1:0/1951917239 >> 172.21.15.1:0/1951917239 conn(0x7fae7c0d0b50 msgr2=0x7fae7c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 -- 172.21.15.1:0/1951917239 shutdown_connections 2024-05-09T06:26:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.196+0000 7fae829c1640 1 -- 172.21.15.1:0/1951917239 wait complete. 2024-05-09T06:26:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.197+0000 7fae829c1640 1 Processor -- start 2024-05-09T06:26:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 -- start start 2024-05-09T06:26:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae7c0723f0 0x7fae7c145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae7c155250 0x7fae7c146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c06fc20 con 0x7fae7c06f7a0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0cb1a0 con 0x7fae7c155250 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae829c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c0cc020 con 0x7fae7c0723f0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47780/0 (socket says 172.21.15.1:47780) 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae7bfff640 1 -- 172.21.15.1:0/1586996321 learned_addr learned my addr 172.21.15.1:0/1586996321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.198+0000 7fae80f5e640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae7c155250 0x7fae7c146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7b7fe640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae7c0723f0 0x7fae7c145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 -- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae7c0723f0 msgr2=0x7fae7c145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae7c0723f0 0x7fae7c145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 -- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae7c155250 msgr2=0x7fae7c146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae7c155250 0x7fae7c146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 -- 172.21.15.1:0/1586996321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c0c3ab0 con 0x7fae7c06f7a0 2024-05-09T06:26:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae7bfff640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae6c000c00 tx=0x7fae6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037040 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c168d70 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c004030 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c004420 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.199+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c145690 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.200+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c0752b0 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.200+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae6c03a070 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.200+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fae6c03cdf0 con 0x7fae7c06f7a0 2024-05-09T06:26:59.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.202+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fae6c041020 con 0x7fae7c06f7a0 2024-05-09T06:26:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.264+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae7c1689d0 con 0x7fae7c06f7a0 2024-05-09T06:26:59.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.265+0000 7fae78ff9640 1 -- 172.21.15.1:0/1586996321 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fae6c03c670 con 0x7fae7c06f7a0 2024-05-09T06:26:59.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:26:59.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:26:59.030580+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 msgr2=0x7fae7c145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae6c000c00 tx=0x7fae6c004730 comp rx=0 tx=0).stop 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 shutdown_connections 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae7c155250 0x7fae7c146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae7c0723f0 0x7fae7c145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 --2- 172.21.15.1:0/1586996321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7c06f7a0 0x7fae7c145150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 >> 172.21.15.1:0/1586996321 conn(0x7fae7c0d0b50 msgr2=0x7fae7c158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:26:59.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 shutdown_connections 2024-05-09T06:26:59.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:26:59.266+0000 7fae829c1640 1 -- 172.21.15.1:0/1586996321 wait complete. 2024-05-09T06:26:59.259 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T06:26:59.259 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T06:26:59.403 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.421+0000 7f618540e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T06:26:59.419 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.438+0000 7f525db10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T06:26:59.458 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.476+0000 7f618540e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T06:26:59.474 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.492+0000 7f525db10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T06:26:59.579 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.597+0000 7f618540e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T06:26:59.596 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.614+0000 7f525db10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T06:26:59.643 INFO:tasks.ceph.osd.4.smithi006.stderr:2024-05-09T06:26:59.661+0000 7fcf22416640 -1 Falling back to public interface 2024-05-09T06:26:59.644 INFO:tasks.ceph.osd.5.smithi006.stderr:2024-05-09T06:26:59.661+0000 7fe2c4955640 -1 Falling back to public interface 2024-05-09T06:26:59.644 INFO:tasks.ceph.osd.5.smithi006.stderr:2024-05-09T06:26:59.661+0000 7fe2c3904640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-05-09T06:26:59.656 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T06:26:59.674+0000 7fbda19b1640 -1 Falling back to public interface 2024-05-09T06:26:59.659 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T06:26:59.677+0000 7f0d52bcb640 -1 Falling back to public interface 2024-05-09T06:26:59.666 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T06:26:59.684+0000 7fcccc416640 -1 Falling back to public interface 2024-05-09T06:26:59.667 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T06:26:59.685+0000 7f504fbfc640 -1 Falling back to public interface 2024-05-09T06:26:59.675 INFO:tasks.ceph.osd.7.smithi006.stderr:2024-05-09T06:26:59.693+0000 7f60fe22d640 -1 Falling back to public interface 2024-05-09T06:26:59.675 INFO:tasks.ceph.osd.6.smithi006.stderr:2024-05-09T06:26:59.694+0000 7f13c580d640 -1 Falling back to public interface 2024-05-09T06:26:59.678 INFO:tasks.ceph.osd.7.smithi006.stderr:2024-05-09T06:26:59.696+0000 7f60fc791640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6827/0: (98) Address already in use 2024-05-09T06:26:59.883 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.901+0000 7f618540e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T06:26:59.907 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.925+0000 7f525db10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T06:26:59.933 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:26:59.951+0000 7f618540e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T06:26:59.957 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:26:59.975+0000 7f525db10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T06:27:00.065 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:00.097 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.115+0000 7f618540e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T06:27:00.121 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.140+0000 7f525db10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T06:27:00.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.177+0000 7f0ff6fb7640 1 Processor -- start 2024-05-09T06:27:00.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.177+0000 7f0ff6fb7640 1 -- start start 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff015e440 0x7f0ff0072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ff00b1760 0x7f0ff0072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0ff0073390 0x7f0ff00ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff00b0ea0 con 0x7f0ff00b1760 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff00b1330 con 0x7f0ff0073390 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff6fb7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff015e820 con 0x7f0ff015e440 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ff00b1760 0x7f0ff0072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ff00b1760 0x7f0ff0072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47788/0 (socket says 172.21.15.1:47788) 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 -- 172.21.15.1:0/478213505 learned_addr learned my addr 172.21.15.1:0/478213505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff4d53640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff015e440 0x7f0ff0072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0ff5554640 1 --2- 172.21.15.1:0/478213505 >> v2:172.21.15.6:3300/0 conn(0x7f0ff0073390 0x7f0ff00ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 -- 172.21.15.1:0/478213505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff015e440 msgr2=0x7f0ff0072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff015e440 0x7f0ff0072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 -- 172.21.15.1:0/478213505 >> v2:172.21.15.6:3300/0 conn(0x7f0ff0073390 msgr2=0x7f0ff00ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:00.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 --2- 172.21.15.1:0/478213505 >> v2:172.21.15.6:3300/0 conn(0x7f0ff0073390 0x7f0ff00ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.178+0000 7f0feffff640 1 -- 172.21.15.1:0/478213505 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff00b02b0 con 0x7f0ff00b1760 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.179+0000 7f0feffff640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff00b1760 0x7f0ff0072e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b6470 tx=0x7f0fdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc8d3aef69d14b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.179+0000 7f0fef7fe640 1 -- 172.21.15.1:0/478213505 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc037070 con 0x7f0ff00b1760 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.179+0000 7f0fef7fe640 1 -- 172.21.15.1:0/478213505 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc032440 con 0x7f0ff00b1760 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.179+0000 7f0fef7fe640 1 -- 172.21.15.1:0/478213505 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc032810 con 0x7f0ff00b1760 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/478213505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff00b1760 msgr2=0x7f0ff0072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:00.163 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff00b1760 0x7f0ff0072e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b6470 tx=0x7f0fdc02a270 comp rx=0 tx=0).stop 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/478213505 shutdown_connections 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/478213505 >> v2:172.21.15.6:3300/0 conn(0x7f0ff0073390 0x7f0ff00ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff00b1760 0x7f0ff0072e50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/478213505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff015e440 0x7f0ff0072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/478213505 >> 172.21.15.1:0/478213505 conn(0x7f0ff00b1e70 msgr2=0x7f0ff00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/478213505 shutdown_connections 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.180+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/478213505 wait complete. 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.181+0000 7f0ff6fb7640 1 Processor -- start 2024-05-09T06:27:00.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.181+0000 7f0ff6fb7640 1 -- start start 2024-05-09T06:27:00.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff00b1760 0x7f0ff01f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ff015e440 0x7f0ff01f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:00.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00b0b30 con 0x7f0ff0073390 2024-05-09T06:27:00.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00b0850 con 0x7f0ff015e440 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff6fb7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff00afef0 con 0x7f0ff00b1760 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47802/0 (socket says 172.21.15.1:47802) 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 -- 172.21.15.1:0/1600980247 learned_addr learned my addr 172.21.15.1:0/1600980247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0feffff640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff00b1760 0x7f0ff01f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff5554640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ff015e440 0x7f0ff01f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 -- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff00b1760 msgr2=0x7f0ff01f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff00b1760 0x7f0ff01f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 -- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ff015e440 msgr2=0x7f0ff01f1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ff015e440 0x7f0ff01f1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.182+0000 7f0ff4d53640 1 -- 172.21.15.1:0/1600980247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff00ccc40 con 0x7f0ff0073390 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0ff4d53640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e9e0 tx=0x7f0fe400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4017070 con 0x7f0ff0073390 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe400c9d0 con 0x7f0ff0073390 2024-05-09T06:27:00.167 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400cdc0 con 0x7f0ff0073390 2024-05-09T06:27:00.168 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff00ccf20 con 0x7f0ff0073390 2024-05-09T06:27:00.168 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff00cd4e0 con 0x7f0ff0073390 2024-05-09T06:27:00.168 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0fe401a070 con 0x7f0ff0073390 2024-05-09T06:27:00.168 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.183+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0fe4006660 con 0x7f0ff0073390 2024-05-09T06:27:00.168 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.184+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff004f5f0 con 0x7f0ff0073390 2024-05-09T06:27:00.169 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.187+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0fe4005990 con 0x7f0ff0073390 2024-05-09T06:27:00.232 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.250+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ff00b62e0 con 0x7f0ff0073390 2024-05-09T06:27:00.233 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.251+0000 7f0fed7fa640 1 -- 172.21.15.1:0/1600980247 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0fe4005b70 con 0x7f0ff0073390 2024-05-09T06:27:00.233 INFO:teuthology.misc.health.smithi001.stdout: 2024-05-09T06:27:00.233 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":10,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:26:59.030580+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:00.234 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 msgr2=0x7f0ff00cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:00.234 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e9e0 tx=0x7f0fe400eea0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 shutdown_connections 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ff015e440 0x7f0ff01f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ff00b1760 0x7f0ff01f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 --2- 172.21.15.1:0/1600980247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff0073390 0x7f0ff00cc5b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 >> 172.21.15.1:0/1600980247 conn(0x7f0ff00b1e70 msgr2=0x7f0ff00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 shutdown_connections 2024-05-09T06:27:00.235 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:00.252+0000 7f0ff6fb7640 1 -- 172.21.15.1:0/1600980247 wait complete. 2024-05-09T06:27:00.243 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T06:27:00.246 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.264+0000 7f618540e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T06:27:00.266 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.284+0000 7f525db10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T06:27:00.298 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.316+0000 7f618540e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T06:27:00.318 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.336+0000 7f525db10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T06:27:00.372 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.390+0000 7f618540e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T06:27:00.392 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.410+0000 7f525db10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T06:27:00.419 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.437+0000 7f618540e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T06:27:00.439 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.457+0000 7f525db10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T06:27:00.919 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:00.937+0000 7f618540e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T06:27:00.944 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:00.962+0000 7f525db10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T06:27:01.040 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:01.058+0000 7f618540e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T06:27:01.066 INFO:tasks.ceph.mgr.x.smithi006.stderr:2024-05-09T06:27:01.085+0000 7f525db10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T06:27:02.054 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:02.071+0000 7f6125abc640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T06:27:02.054 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T06:27:02.073+0000 7f6125abc640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T06:27:03.109 INFO:tasks.ceph.osd.4.smithi006.stderr:2024-05-09T06:27:03.127+0000 7fcf22416640 -1 osd.4 0 log_to_monitors true 2024-05-09T06:27:03.136 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T06:27:03.154+0000 7f504fbfc640 -1 osd.0 0 log_to_monitors true 2024-05-09T06:27:03.159 INFO:tasks.ceph.osd.5.smithi006.stderr:2024-05-09T06:27:03.178+0000 7fe2c4955640 -1 osd.5 0 log_to_monitors true 2024-05-09T06:27:03.168 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T06:27:03.187+0000 7f0d52bcb640 -1 osd.2 0 log_to_monitors true 2024-05-09T06:27:03.170 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T06:27:03.188+0000 7fcccc416640 -1 osd.1 0 log_to_monitors true 2024-05-09T06:27:03.176 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T06:27:03.195+0000 7fbda19b1640 -1 osd.3 0 log_to_monitors true 2024-05-09T06:27:03.179 INFO:tasks.ceph.osd.6.smithi006.stderr:2024-05-09T06:27:03.197+0000 7f13c580d640 -1 osd.6 0 log_to_monitors true 2024-05-09T06:27:03.184 INFO:tasks.ceph.osd.7.smithi006.stderr:2024-05-09T06:27:03.202+0000 7f60fe22d640 -1 osd.7 0 log_to_monitors true 2024-05-09T06:27:06.487 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T06:27:06.505+0000 7f504bb92640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T06:27:06.493 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T06:27:06.511+0000 7fccc7e2a640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T06:27:06.503 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T06:27:06.521+0000 7f50439b2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:06.506 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T06:27:06.525+0000 7fccbfc4a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:06.545 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T06:27:06.563+0000 7f0d4eb61640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T06:27:06.546 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T06:27:06.564+0000 7fbd9d947640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T06:27:06.555 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T06:27:06.573+0000 7f0d46981640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:06.558 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T06:27:06.577+0000 7fbd95767640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:07.049 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.175+0000 7fed42a93640 1 Processor -- start 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.175+0000 7fed42a93640 1 -- start start 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed42a93640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fed3c0d2c60 0x7fed3c0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed42a93640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed3c0cea60 con 0x7fed3c15e310 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed42a93640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed3c0ced00 con 0x7fed3c0d4680 2024-05-09T06:27:07.159 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed42a93640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fed3c0d2390 con 0x7fed3c0d2c60 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed41030640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fed3c0d2c60 0x7fed3c0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed4082f640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fed3c15e310 0x7fed3c0d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49404/0 (socket says 172.21.15.1:49404) 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed4082f640 1 -- 172.21.15.1:0/3689084682 learned_addr learned my addr 172.21.15.1:0/3689084682 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed41030640 1 -- 172.21.15.1:0/3689084682 >> v1:172.21.15.6:6789/0 conn(0x7fed3c0d4680 legacy=0x7fed3c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed41030640 1 -- 172.21.15.1:0/3689084682 >> v1:172.21.15.1:6789/0 conn(0x7fed3c15e310 legacy=0x7fed3c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.176+0000 7fed41030640 1 -- 172.21.15.1:0/3689084682 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c0cefe0 con 0x7fed3c0d2c60 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed41030640 1 --2- 172.21.15.1:0/3689084682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c0d2c60 0x7fed3c0cfac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d437ac25d66d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed3b7fe640 1 -- 172.21.15.1:0/3689084682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34007740 con 0x7fed3c0d2c60 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed3b7fe640 1 -- 172.21.15.1:0/3689084682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed340078e0 con 0x7fed3c0d2c60 2024-05-09T06:27:07.160 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed3b7fe640 1 -- 172.21.15.1:0/3689084682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34007cb0 con 0x7fed3c0d2c60 2024-05-09T06:27:07.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed42a93640 1 -- 172.21.15.1:0/3689084682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c0d2c60 msgr2=0x7fed3c0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.177+0000 7fed42a93640 1 --2- 172.21.15.1:0/3689084682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c0d2c60 0x7fed3c0cfac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3402a660 comp rx=0 tx=0).stop 2024-05-09T06:27:07.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.178+0000 7fed42a93640 1 -- 172.21.15.1:0/3689084682 shutdown_connections 2024-05-09T06:27:07.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.178+0000 7fed42a93640 1 --2- 172.21.15.1:0/3689084682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c0d2c60 0x7fed3c0cfac0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.161 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.178+0000 7fed42a93640 1 -- 172.21.15.1:0/3689084682 >> 172.21.15.1:0/3689084682 conn(0x7fed3c09d200 msgr2=0x7fed3c0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:07.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.178+0000 7fed42a93640 1 -- 172.21.15.1:0/3689084682 shutdown_connections 2024-05-09T06:27:07.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.178+0000 7fed42a93640 1 -- 172.21.15.1:0/3689084682 wait complete. 2024-05-09T06:27:07.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.179+0000 7fed42a93640 1 Processor -- start 2024-05-09T06:27:07.162 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.179+0000 7fed42a93640 1 -- start start 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed3c0d2c60 0x7fed3c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0d4680 0x7fed3c1f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c06bc00 con 0x7fed3c15e310 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c0cf630 con 0x7fed3c0d4680 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed42a93640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fed3c06bd80 con 0x7fed3c0d2c60 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47892/0 (socket says 172.21.15.1:47892) 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed3bfff640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0d4680 0x7fed3c1f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:07.164 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 -- 172.21.15.1:0/983836227 learned_addr learned my addr 172.21.15.1:0/983836227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed4082f640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed3c0d2c60 0x7fed3c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 -- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed3c0d2c60 msgr2=0x7fed3c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed3c0d2c60 0x7fed3c15f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 -- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0d4680 msgr2=0x7fed3c1f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0d4680 0x7fed3c1f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.180+0000 7fed41030640 1 -- 172.21.15.1:0/983836227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c160190 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed41030640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed3402ac50 tx=0x7fed34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34040070 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed34031c90 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34045680 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c15e820 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.181+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c155d40 con 0x7fed3c15e310 2024-05-09T06:27:07.165 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.182+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c04f5f0 con 0x7fed3c15e310 2024-05-09T06:27:07.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.182+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fed34039040 con 0x7fed3c15e310 2024-05-09T06:27:07.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.183+0000 7fed397fa640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 0x7fed2006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:07.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.183+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fed340ab7a0 con 0x7fed3c15e310 2024-05-09T06:27:07.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.183+0000 7fed4082f640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 0x7fed2006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:07.166 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.184+0000 7fed4082f640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 0x7fed2006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed30005880 tx=0x7fed30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:07.170 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.188+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed3407abe0 con 0x7fed3c15e310 2024-05-09T06:27:07.180 INFO:tasks.ceph.osd.5.smithi006.stderr:2024-05-09T06:27:07.195+0000 7fe2c10ff640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T06:27:07.181 INFO:tasks.ceph.osd.6.smithi006.stderr:2024-05-09T06:27:07.196+0000 7f13c10f3640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T06:27:07.194 INFO:tasks.ceph.osd.6.smithi006.stderr:2024-05-09T06:27:07.211+0000 7f13b8f13640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:07.194 INFO:tasks.ceph.osd.5.smithi006.stderr:2024-05-09T06:27:07.212+0000 7fe2b870b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:07.197 INFO:tasks.ceph.osd.4.smithi006.stderr:2024-05-09T06:27:07.214+0000 7fcf1e63e640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T06:27:07.203 INFO:tasks.ceph.osd.7.smithi006.stderr:2024-05-09T06:27:07.220+0000 7f60f9f8c640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T06:27:07.209 INFO:tasks.ceph.osd.4.smithi006.stderr:2024-05-09T06:27:07.227+0000 7fcf15c4a640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:07.218 INFO:tasks.ceph.osd.7.smithi006.stderr:2024-05-09T06:27:07.235+0000 7f60f1598640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:27:07.357 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.375+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed3c0d1740 con 0x7fed3c15e310 2024-05-09T06:27:07.358 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.376+0000 7fed397fa640 1 -- 172.21.15.1:0/983836227 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fed3407eee0 con 0x7fed3c15e310 2024-05-09T06:27:07.358 INFO:teuthology.misc.health.smithi001.stdout: 2024-05-09T06:27:07.359 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":12,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:05.080837+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:07.360 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 msgr2=0x7fed2006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.360 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 0x7fed2006e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed30005880 tx=0x7fed30009040 comp rx=0 tx=0).stop 2024-05-09T06:27:07.360 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 msgr2=0x7fed3c160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:07.360 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed3402ac50 tx=0x7fed34004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:07.360 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 shutdown_connections 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed2006bda0 0x7fed2006e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed3c15e310 0x7fed3c160f90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed3c0d4680 0x7fed3c1f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 --2- 172.21.15.1:0/983836227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed3c0d2c60 0x7fed3c15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 >> 172.21.15.1:0/983836227 conn(0x7fed3c09d200 msgr2=0x7fed3c0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 shutdown_connections 2024-05-09T06:27:07.361 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:07.378+0000 7fed42a93640 1 -- 172.21.15.1:0/983836227 wait complete. 2024-05-09T06:27:07.370 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T06:27:14.177 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:14.283 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.301+0000 7fe625d70640 1 Processor -- start 2024-05-09T06:27:14.284 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.301+0000 7fe625d70640 1 -- start start 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe625d70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe625d70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 0x7fe6200d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe625d70640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6200cea60 con 0x7fe6200d2550 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe625d70640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200d2390 con 0x7fe6200d4680 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe625d70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200cf630 con 0x7fe62015e310 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.302+0000 7fe61f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 0x7fe6200d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:53306/0 (socket says 172.21.15.1:53306) 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 -- 172.21.15.1:0/3489631277 learned_addr learned my addr 172.21.15.1:0/3489631277 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:14.285 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 -- 172.21.15.1:0/3489631277 >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 msgr2=0x7fe6200d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:14.286 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 --2- 172.21.15.1:0/3489631277 >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 0x7fe6200d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.286 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 -- 172.21.15.1:0/3489631277 >> v1:172.21.15.6:6789/0 conn(0x7fe6200d2550 legacy=0x7fe6200d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.286 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61f7fe640 1 -- 172.21.15.1:0/3489631277 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200ced00 con 0x7fe62015e310 2024-05-09T06:27:14.286 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.303+0000 7fe61effd640 1 --2- 172.21.15.1:0/3489631277 >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 0x7fe6200d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe61f7fe640 1 --2- 172.21.15.1:0/3489631277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe610009870 tx=0x7fe61002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ae0a264982628f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe61e7fc640 1 -- 172.21.15.1:0/3489631277 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610037070 con 0x7fe62015e310 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe61e7fc640 1 -- 172.21.15.1:0/3489631277 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610032440 con 0x7fe62015e310 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe61e7fc640 1 -- 172.21.15.1:0/3489631277 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610032810 con 0x7fe62015e310 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 -- 172.21.15.1:0/3489631277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 msgr2=0x7fe6200cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 --2- 172.21.15.1:0/3489631277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe610009870 tx=0x7fe61002a270 comp rx=0 tx=0).stop 2024-05-09T06:27:14.287 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 -- 172.21.15.1:0/3489631277 shutdown_connections 2024-05-09T06:27:14.288 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 --2- 172.21.15.1:0/3489631277 >> v2:172.21.15.6:3300/0 conn(0x7fe6200d4680 0x7fe6200d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.288 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 --2- 172.21.15.1:0/3489631277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe62015e310 0x7fe6200cfac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.288 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 -- 172.21.15.1:0/3489631277 >> 172.21.15.1:0/3489631277 conn(0x7fe62009d200 msgr2=0x7fe6200d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:14.288 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 -- 172.21.15.1:0/3489631277 shutdown_connections 2024-05-09T06:27:14.288 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.305+0000 7fe625d70640 1 -- 172.21.15.1:0/3489631277 wait complete. 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.306+0000 7fe625d70640 1 Processor -- start 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.306+0000 7fe625d70640 1 -- start start 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6200d2550 0x7fe6201557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe62015e310 0x7fe62015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200d2080 con 0x7fe6200d4680 2024-05-09T06:27:14.290 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cf2c0 con 0x7fe62015e310 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe625d70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe62006bc00 con 0x7fe6200d2550 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60060/0 (socket says 172.21.15.1:60060) 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 -- 172.21.15.1:0/2860848470 learned_addr learned my addr 172.21.15.1:0/2860848470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61f7fe640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6200d2550 0x7fe6201557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61ffff640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe62015e310 0x7fe62015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 -- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6200d2550 msgr2=0x7fe6201557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6200d2550 0x7fe6201557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 -- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe62015e310 msgr2=0x7fe62015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.291 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe62015e310 0x7fe62015f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 -- 172.21.15.1:0/2860848470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200c8e60 con 0x7fe6200d4680 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.307+0000 7fe61effd640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe61400ebe0 tx=0x7fe61400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.308+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614017070 con 0x7fe6200d4680 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.308+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61400cc30 con 0x7fe6200d4680 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.308+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6200c9100 con 0x7fe6200d4680 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.308+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614013470 con 0x7fe6200d4680 2024-05-09T06:27:14.292 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.308+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62015f850 con 0x7fe6200d4680 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.309+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62004f5f0 con 0x7fe6200d4680 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.309+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe61401a070 con 0x7fe6200d4680 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.310+0000 7fe607fff640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 0x7fe60c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.310+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe61408a9d0 con 0x7fe6200d4680 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.310+0000 7fe61f7fe640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 0x7fe60c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.293 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.310+0000 7fe61f7fe640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 0x7fe60c06e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe610034000 tx=0x7fe61002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.296 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.315+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe614059080 con 0x7fe6200d4680 2024-05-09T06:27:14.479 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.497+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6200d1740 con 0x7fe6200d4680 2024-05-09T06:27:14.480 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.498+0000 7fe607fff640 1 -- 172.21.15.1:0/2860848470 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12752 (secure 0 0 0) 0x7fe61405d380 con 0x7fe6200d4680 2024-05-09T06:27:14.480 INFO:teuthology.misc.health.smithi001.stdout: 2024-05-09T06:27:14.480 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":16,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:11.205914+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:14.482 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.499+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 msgr2=0x7fe60c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.482 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.499+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 0x7fe60c06e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe610034000 tx=0x7fe61002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:14.482 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 msgr2=0x7fe6201f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.482 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe61400ebe0 tx=0x7fe61400c520 comp rx=0 tx=0).stop 2024-05-09T06:27:14.482 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 shutdown_connections 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60c06be30 0x7fe60c06e2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe62015e310 0x7fe62015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6200d4680 0x7fe6201f5060 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 --2- 172.21.15.1:0/2860848470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6200d2550 0x7fe6201557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 >> 172.21.15.1:0/2860848470 conn(0x7fe62009d200 msgr2=0x7fe6200b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 shutdown_connections 2024-05-09T06:27:14.483 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T06:27:14.500+0000 7fe625d70640 1 -- 172.21.15.1:0/2860848470 wait complete. 2024-05-09T06:27:14.491 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T06:27:14.492 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T06:27:14.492 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:27:14.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:27:14.494 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:27:14.494 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:27:14.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:27:14.496 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:27:14.496 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:27:14.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:27:14.499 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:27:14.499 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:27:14.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:27:14.501 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:27:14.501 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T06:27:14.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:14.573 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:27:14.573 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:27:14.573 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:27:14.591+0000 7f9c3419b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-09T06:27:14.590 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:27:14.590 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:27:14.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.633+0000 7f73509a2640 1 Processor -- start 2024-05-09T06:27:14.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.633+0000 7f73509a2640 1 -- start start 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f73509a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c161950 0x7f734c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f73509a2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f734c0d0f80 0x7f734c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f73509a2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734c0b33c0 con 0x7f734c0d4940 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f73509a2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0d4dd0 con 0x7f734c0d0f80 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f73509a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0b3660 con 0x7f734c161950 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f734ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f734c0d0f80 0x7f734c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f734ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f734c0d0f80 0x7f734c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60100/0 (socket says 172.21.15.1:60100) 2024-05-09T06:27:14.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f734ad9d640 1 -- 172.21.15.1:0/1143230859 learned_addr learned my addr 172.21.15.1:0/1143230859 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.634+0000 7f734a59c640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c161950 0x7f734c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734ad9d640 1 -- 172.21.15.1:0/1143230859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c161950 msgr2=0x7f734c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734ad9d640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c161950 0x7f734c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734ad9d640 1 -- 172.21.15.1:0/1143230859 >> v1:172.21.15.6:6789/0 conn(0x7f734c0d4940 legacy=0x7f734c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734ad9d640 1 -- 172.21.15.1:0/1143230859 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c0b3940 con 0x7f734c0d0f80 2024-05-09T06:27:14.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734ad9d640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0d2940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f734c0b3590 tx=0x7f734402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57d703386d5f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.635+0000 7f734959a640 1 -- 172.21.15.1:0/1143230859 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344036070 con 0x7f734c0d0f80 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f734959a640 1 -- 172.21.15.1:0/1143230859 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7344031600 con 0x7f734c0d0f80 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f734959a640 1 -- 172.21.15.1:0/1143230859 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440319d0 con 0x7f734c0d0f80 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 -- 172.21.15.1:0/1143230859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 msgr2=0x7f734c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0d2940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f734c0b3590 tx=0x7f734402a710 comp rx=0 tx=0).stop 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 -- 172.21.15.1:0/1143230859 shutdown_connections 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0d2940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 --2- 172.21.15.1:0/1143230859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c161950 0x7f734c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 -- 172.21.15.1:0/1143230859 >> 172.21.15.1:0/1143230859 conn(0x7f734c0d3e20 msgr2=0x7f734c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 -- 172.21.15.1:0/1143230859 shutdown_connections 2024-05-09T06:27:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.636+0000 7f73509a2640 1 -- 172.21.15.1:0/1143230859 wait complete. 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.637+0000 7f73509a2640 1 Processor -- start 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- start start 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f734c161950 0x7f734c14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c15e0b0 0x7f734c1e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0b52d0 con 0x7f734c0d0f80 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0b5150 con 0x7f734c161950 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0b5450 con 0x7f734c15e0b0 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60114/0 (socket says 172.21.15.1:60114) 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 -- 172.21.15.1:0/3298032682 learned_addr learned my addr 172.21.15.1:0/3298032682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734ad9d640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c15e0b0 0x7f734c1e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f7349d9b640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f734c161950 0x7f734c14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 -- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c15e0b0 msgr2=0x7f734c1e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c15e0b0 0x7f734c1e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 -- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f734c161950 msgr2=0x7f734c14b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f734c161950 0x7f734c14b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 -- 172.21.15.1:0/3298032682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c14b9f0 con 0x7f734c0d0f80 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f734a59c640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f733c00e8a0 tx=0x7f733c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c017070 con 0x7f734c0d0f80 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c00c940 con 0x7f734c0d0f80 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c14bcd0 con 0x7f734c0d0f80 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c00cd30 con 0x7f734c0d0f80 2024-05-09T06:27:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.639+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c14c290 con 0x7f734c0d0f80 2024-05-09T06:27:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.640+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c0cfb40 con 0x7f734c0d0f80 2024-05-09T06:27:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.641+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f733c01a070 con 0x7f734c0d0f80 2024-05-09T06:27:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.641+0000 7f733affd640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 0x7f732c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.641+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f733c08ac70 con 0x7f734c0d0f80 2024-05-09T06:27:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.641+0000 7f7349d9b640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 0x7f732c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.642+0000 7f7349d9b640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 0x7f732c06e320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f734c0cdc30 tx=0x7f7340009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.646+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f734c0cfb40 con 0x7f734c0d0f80 2024-05-09T06:27:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.830+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f734c0752b0 con 0x7f734c0d0f80 2024-05-09T06:27:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.831+0000 7f733affd640 1 -- 172.21.15.1:0/3298032682 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12752 (secure 0 0 0) 0x7f733c059320 con 0x7f734c0d0f80 2024-05-09T06:27:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:14.814 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":16,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:11.205914+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 msgr2=0x7f732c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 0x7f732c06e320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f734c0cdc30 tx=0x7f7340009040 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 msgr2=0x7f734c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f733c00e8a0 tx=0x7f733c00ed60 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 shutdown_connections 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f732c06be60 0x7f732c06e320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f734c15e0b0 0x7f734c1e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f734c161950 0x7f734c14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 --2- 172.21.15.1:0/3298032682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f734c0d0f80 0x7f734c0b6d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 >> 172.21.15.1:0/3298032682 conn(0x7f734c0d3e20 msgr2=0x7f734c0b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 shutdown_connections 2024-05-09T06:27:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.833+0000 7f73509a2640 1 -- 172.21.15.1:0/3298032682 wait complete. 2024-05-09T06:27:14.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:27:14.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:14.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.951+0000 7fef5426d640 1 Processor -- start 2024-05-09T06:27:14.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.951+0000 7fef5426d640 1 -- start start 2024-05-09T06:27:14.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef5426d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 0x7fef4c1436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef5426d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 0x7fef4c06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef5426d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4c143fe0 con 0x7fef4c141020 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef5426d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c162ae0 con 0x7fef4c140c40 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef5426d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c06bb70 con 0x7fef4c06b1e0 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef51808640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fef4c141020 0x7fef4c1623d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35454/0 (socket says 172.21.15.1:35454) 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.952+0000 7fef51808640 1 -- 172.21.15.1:0/2950414720 learned_addr learned my addr 172.21.15.1:0/2950414720 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef5280a640 1 --2- 172.21.15.1:0/2950414720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 0x7fef4c06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef52009640 1 --2- 172.21.15.1:0/2950414720 >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 0x7fef4c1436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791441986 0 0) 0x7fef4c143fe0 con 0x7fef4c141020 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef34003520 con 0x7fef4c141020 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2776589418 0 0) 0x7fef34003520 con 0x7fef4c141020 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef5280a640 1 -- 172.21.15.1:0/2950414720 >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 msgr2=0x7fef4c1436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef5280a640 1 --2- 172.21.15.1:0/2950414720 >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 0x7fef4c1436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef5280a640 1 -- 172.21.15.1:0/2950414720 >> v1:172.21.15.1:6789/0 conn(0x7fef4c141020 legacy=0x7fef4c1623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef5280a640 1 -- 172.21.15.1:0/2950414720 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1442c0 con 0x7fef4c06b1e0 2024-05-09T06:27:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.953+0000 7fef52009640 1 --2- 172.21.15.1:0/2950414720 >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 0x7fef4c1436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef5280a640 1 --2- 172.21.15.1:0/2950414720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 0x7fef4c06b630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009d90 tx=0x7fef3c02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5077f6baa619e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40001670 con 0x7fef4c06b1e0 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c007540 con 0x7fef4c06b1e0 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef51007640 1 -- 172.21.15.1:0/2950414720 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0078f0 con 0x7fef4c06b1e0 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef5426d640 1 -- 172.21.15.1:0/2950414720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 msgr2=0x7fef4c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.954+0000 7fef5426d640 1 --2- 172.21.15.1:0/2950414720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 0x7fef4c06b630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009d90 tx=0x7fef3c02a560 comp rx=0 tx=0).stop 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 -- 172.21.15.1:0/2950414720 shutdown_connections 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 --2- 172.21.15.1:0/2950414720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c06b1e0 0x7fef4c06b630 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 --2- 172.21.15.1:0/2950414720 >> v2:172.21.15.6:3300/0 conn(0x7fef4c140c40 0x7fef4c1436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 -- 172.21.15.1:0/2950414720 >> 172.21.15.1:0/2950414720 conn(0x7fef4c0d0b00 msgr2=0x7fef4c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:14.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 -- 172.21.15.1:0/2950414720 shutdown_connections 2024-05-09T06:27:14.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.955+0000 7fef5426d640 1 -- 172.21.15.1:0/2950414720 wait complete. 2024-05-09T06:27:14.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.957+0000 7fef5426d640 1 Processor -- start 2024-05-09T06:27:14.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.957+0000 7fef5426d640 1 -- start start 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c06b1e0 0x7fef4c1391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c140c40 0x7fef4c1396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c161f40 con 0x7fef4c141020 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1445a0 con 0x7fef4c06b1e0 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c143c20 con 0x7fef4c140c40 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60118/0 (socket says 172.21.15.1:60118) 2024-05-09T06:27:14.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 -- 172.21.15.1:0/149735817 learned_addr learned my addr 172.21.15.1:0/149735817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef51808640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c140c40 0x7fef4c1396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef52009640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c06b1e0 0x7fef4c1391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 -- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c140c40 msgr2=0x7fef4c1396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c140c40 0x7fef4c1396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 -- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c06b1e0 msgr2=0x7fef4c1391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c06b1e0 0x7fef4c1391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 -- 172.21.15.1:0/149735817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c15ff90 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5280a640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009eb0 tx=0x7fef3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c040070 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c031c40 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c160270 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009450 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c160830 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.958+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c06b630 con 0x7fef4c141020 2024-05-09T06:27:14.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.959+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef3c039040 con 0x7fef4c141020 2024-05-09T06:27:14.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.959+0000 7fef4affd640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 0x7fef3806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:14.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.959+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fef3c0abf80 con 0x7fef4c141020 2024-05-09T06:27:14.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.959+0000 7fef52009640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 0x7fef3806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:14.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.960+0000 7fef52009640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 0x7fef3806e320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef44009560 tx=0x7fef44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:14.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:14.964+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef4c06b630 con 0x7fef4c141020 2024-05-09T06:27:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.148+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef4c0752b0 con 0x7fef4c141020 2024-05-09T06:27:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.149+0000 7fef4affd640 1 -- 172.21.15.1:0/149735817 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fef3c0303a0 con 0x7fef4c141020 2024-05-09T06:27:15.131 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 msgr2=0x7fef3806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 0x7fef3806e320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef44009560 tx=0x7fef44008040 comp rx=0 tx=0).stop 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 msgr2=0x7fef4c15e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009eb0 tx=0x7fef3c004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 shutdown_connections 2024-05-09T06:27:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be60 0x7fef3806e320 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c141020 0x7fef4c15e3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c140c40 0x7fef4c1396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 --2- 172.21.15.1:0/149735817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c06b1e0 0x7fef4c1391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 >> 172.21.15.1:0/149735817 conn(0x7fef4c0d0b00 msgr2=0x7fef4c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 shutdown_connections 2024-05-09T06:27:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.151+0000 7fef5426d640 1 -- 172.21.15.1:0/149735817 wait complete. 2024-05-09T06:27:15.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:15.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 Processor -- start 2024-05-09T06:27:15.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 -- start start 2024-05-09T06:27:15.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46bc0cdab0 0x7f46bc0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46bc0ce0e0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f46bc0ce3c0 con 0x7f46bc0cdab0 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46c0e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc0ce6a0 con 0x7f46bc0cf180 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46bad9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46bc0cdab0 0x7f46bc0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46b3fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46bad9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46bc0cdab0 0x7f46bc0b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:52692/0 (socket says 172.21.15.1:52692) 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46b3fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:53382/0 (socket says 172.21.15.1:53382) 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.266+0000 7f46bad9d640 1 -- 172.21.15.1:0/4170797085 learned_addr learned my addr 172.21.15.1:0/4170797085 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46bad9d640 1 -- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 msgr2=0x7f46bc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46bad9d640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46bad9d640 1 -- 172.21.15.1:0/4170797085 >> v1:172.21.15.6:6789/0 conn(0x7f46bc06e6d0 legacy=0x7f46bc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46bad9d640 1 -- 172.21.15.1:0/4170797085 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc0ce980 con 0x7f46bc0cdab0 2024-05-09T06:27:15.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46b3fff640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46bad9d640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cdab0 0x7f46bc0b3300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009e50 tx=0x7f46b402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=268b769181fd6565 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.267+0000 7f46b9d9b640 1 -- 172.21.15.1:0/4170797085 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4036070 con 0x7f46bc0cdab0 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46b9d9b640 1 -- 172.21.15.1:0/4170797085 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b4031600 con 0x7f46bc0cdab0 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46b9d9b640 1 -- 172.21.15.1:0/4170797085 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40319d0 con 0x7f46bc0cdab0 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 -- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cdab0 msgr2=0x7f46bc0b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cdab0 0x7f46bc0b3300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009e50 tx=0x7f46b402a710 comp rx=0 tx=0).stop 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 -- 172.21.15.1:0/4170797085 shutdown_connections 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cdab0 0x7f46bc0b3300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4170797085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cf180 0x7f46bc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 -- 172.21.15.1:0/4170797085 >> 172.21.15.1:0/4170797085 conn(0x7f46bc0cc4b0 msgr2=0x7f46bc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:15.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 -- 172.21.15.1:0/4170797085 shutdown_connections 2024-05-09T06:27:15.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.268+0000 7f46c0e38640 1 -- 172.21.15.1:0/4170797085 wait complete. 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.269+0000 7f46c0e38640 1 Processor -- start 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.269+0000 7f46c0e38640 1 -- start start 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cdab0 0x7f46bc13f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cf180 0x7f46bc13fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1447c0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc070ed0 con 0x7f46bc0cf180 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46c0e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc144940 con 0x7f46bc0cdab0 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60130/0 (socket says 172.21.15.1:60130) 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46ba59c640 1 -- 172.21.15.1:0/4149567909 learned_addr learned my addr 172.21.15.1:0/4149567909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46bad9d640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cf180 0x7f46bc13fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.270+0000 7f46b3fff640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cdab0 0x7f46bc13f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 -- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cdab0 msgr2=0x7f46bc13f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cdab0 0x7f46bc13f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 -- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cf180 msgr2=0x7f46bc13fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cf180 0x7f46bc13fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 -- 172.21.15.1:0/4149567909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc1340c0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46ba59c640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c430 tx=0x7f46ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac017070 con 0x7f46bc06e6d0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac0052d0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003830 con 0x7f46bc06e6d0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc1343a0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.271+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1317a0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.272+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc0752b0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.273+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46ac01a070 con 0x7f46bc06e6d0 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.273+0000 7f46b2ffd640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 0x7f469c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.273+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f46ac089dd0 con 0x7f46bc06e6d0 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.273+0000 7f46b3fff640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 0x7f469c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:15.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.273+0000 7f46b3fff640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 0x7f469c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009560 tx=0x7f46a4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:15.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.277+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46ac058480 con 0x7f46bc06e6d0 2024-05-09T06:27:15.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.498+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46bc06cf50 con 0x7f46bc06e6d0 2024-05-09T06:27:15.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.499+0000 7f46b2ffd640 1 -- 172.21.15.1:0/4149567909 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f46ac08b090 con 0x7f46bc06e6d0 2024-05-09T06:27:15.480 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:15.480 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T06:27:15.480 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2024-05-09T06:27:15.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 msgr2=0x7f469c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 0x7f469c06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009560 tx=0x7f46a4009190 comp rx=0 tx=0).stop 2024-05-09T06:27:15.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 msgr2=0x7f46bc15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:15.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c430 tx=0x7f46ac00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 shutdown_connections 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f469c06be30 0x7f469c06e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46bc0cf180 0x7f46bc13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46bc0cdab0 0x7f46bc13f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 --2- 172.21.15.1:0/4149567909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc06e6d0 0x7f46bc15e490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 >> 172.21.15.1:0/4149567909 conn(0x7f46bc0cc4b0 msgr2=0x7f46bc06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 shutdown_connections 2024-05-09T06:27:15.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:15.501+0000 7f46c0e38640 1 -- 172.21.15.1:0/4149567909 wait complete. 2024-05-09T06:27:17.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:27:17.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:27:17.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:17.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.631+0000 7fa7ed1c1640 1 Processor -- start 2024-05-09T06:27:17.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.631+0000 7fa7ed1c1640 1 -- start start 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7ed1c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e1d0 0x7fa7e806f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7ed1c1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa7e80d2890 0x7fa7e80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7ed1c1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e806c2f0 con 0x7fa7e815e5b0 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7ed1c1640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e80cc3c0 con 0x7fa7e80d2890 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7ed1c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80ccc60 con 0x7fa7e815e1d0 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e759e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa7e80d2890 0x7fa7e80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa7e815e5b0 0x7fa7e80cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35470/0 (socket says 172.21.15.1:35470) 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e659c640 1 -- 172.21.15.1:0/1351739903 learned_addr learned my addr 172.21.15.1:0/1351739903 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:17.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e6d9d640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e1d0 0x7fa7e806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e759e640 1 -- 172.21.15.1:0/1351739903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e1d0 msgr2=0x7fa7e806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:17.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.632+0000 7fa7e759e640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e1d0 0x7fa7e806f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e759e640 1 -- 172.21.15.1:0/1351739903 >> v1:172.21.15.1:6789/0 conn(0x7fa7e815e5b0 legacy=0x7fa7e80cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e759e640 1 -- 172.21.15.1:0/1351739903 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e806f500 con 0x7fa7e80d2890 2024-05-09T06:27:17.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e5d9b640 1 -- 172.21.15.1:0/1351739903 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807371122 0 0) 0x7fa7d0001110 con 0x7fa7e815e5b0 2024-05-09T06:27:17.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e759e640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e80d2890 0x7fa7e80d0fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009dc0 tx=0x7fa7e002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748ea0e9ad93bfcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e5d9b640 1 -- 172.21.15.1:0/1351739903 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0007740 con 0x7fa7e80d2890 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e5d9b640 1 -- 172.21.15.1:0/1351739903 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e00078e0 con 0x7fa7e80d2890 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.633+0000 7fa7e5d9b640 1 -- 172.21.15.1:0/1351739903 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0007cb0 con 0x7fa7e80d2890 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/1351739903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e80d2890 msgr2=0x7fa7e80d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e80d2890 0x7fa7e80d0fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009dc0 tx=0x7fa7e002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/1351739903 shutdown_connections 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e80d2890 0x7fa7e80d0fc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/1351739903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e1d0 0x7fa7e806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/1351739903 >> 172.21.15.1:0/1351739903 conn(0x7fa7e80d1d70 msgr2=0x7fa7e80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/1351739903 shutdown_connections 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.634+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/1351739903 wait complete. 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 Processor -- start 2024-05-09T06:27:17.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 -- start start 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7e80d2890 0x7fa7e80c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e5b0 0x7fa7e80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e806f1b0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80cc8f0 con 0x7fa7e80d2890 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7ed1c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e806fdd0 con 0x7fa7e815e5b0 2024-05-09T06:27:17.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.636+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60150/0 (socket says 172.21.15.1:60150) 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 -- 172.21.15.1:0/899768418 learned_addr learned my addr 172.21.15.1:0/899768418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e759e640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e5b0 0x7fa7e80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e6d9d640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7e80d2890 0x7fa7e80c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 -- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e5b0 msgr2=0x7fa7e80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e5b0 0x7fa7e80caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 -- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7e80d2890 msgr2=0x7fa7e80c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7e80d2890 0x7fa7e80c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 -- 172.21.15.1:0/899768418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e815fcd0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7e659c640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c430 tx=0x7fa7d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00074e0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d00047e0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e815ffb0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004bd0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.637+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e8160570 con 0x7fa7e815e1d0 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.638+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e8158b00 con 0x7fa7e815e1d0 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.639+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7d0011070 con 0x7fa7e815e1d0 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.639+0000 7fa7df7fe640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 0x7fa7c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.639+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa7d008a5e0 con 0x7fa7e815e1d0 2024-05-09T06:27:17.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.639+0000 7fa7e6d9d640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 0x7fa7c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.640+0000 7fa7e6d9d640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 0x7fa7c806e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.644+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7e8158b00 con 0x7fa7e815e1d0 2024-05-09T06:27:17.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.863+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7e8160770 con 0x7fa7e815e1d0 2024-05-09T06:27:17.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.863+0000 7fa7df7fe640 1 -- 172.21.15.1:0/899768418 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3843 (secure 0 0 0) 0x7fa7e8160770 con 0x7fa7e815e1d0 2024-05-09T06:27:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:17.845 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T06:27:17.845 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 msgr2=0x7fa7c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 0x7fa7c806e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 msgr2=0x7fa7e80caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c430 tx=0x7fa7d000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 shutdown_connections 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7c806be30 0x7fa7c806e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7e815e5b0 0x7fa7e80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7e815e1d0 0x7fa7e80caa00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 --2- 172.21.15.1:0/899768418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7e80d2890 0x7fa7e80c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 >> 172.21.15.1:0/899768418 conn(0x7fa7e80d1d70 msgr2=0x7fa7e80aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 shutdown_connections 2024-05-09T06:27:17.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.865+0000 7fa7ed1c1640 1 -- 172.21.15.1:0/899768418 wait complete. 2024-05-09T06:27:17.857 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:17.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.979+0000 7f7837d4f640 1 Processor -- start 2024-05-09T06:27:17.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 -- start start 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78300b33c0 con 0x7f78300d4940 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f78300d4dd0 con 0x7f78300d0f80 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.980+0000 7f7837d4f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f78300b3660 con 0x7f7830161950 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f78362ec640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f78362ec640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:57936/0 (socket says 172.21.15.1:57936) 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f78352ea640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f78300d4940 0x7f78300cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:33186/0 (socket says 172.21.15.1:33186) 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43328/0 (socket says 172.21.15.1:43328) 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f78362ec640 1 -- 172.21.15.1:0/2289977644 learned_addr learned my addr 172.21.15.1:0/2289977644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 -- 172.21.15.1:0/2289977644 >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 msgr2=0x7f78300d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 --2- 172.21.15.1:0/2289977644 >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 -- 172.21.15.1:0/2289977644 >> v1:172.21.15.6:6789/0 conn(0x7f78300d4940 legacy=0x7f78300cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f7835aeb640 1 -- 172.21.15.1:0/2289977644 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78300b3940 con 0x7f7830161950 2024-05-09T06:27:17.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.981+0000 7f78362ec640 1 --2- 172.21.15.1:0/2289977644 >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7835aeb640 1 --2- 172.21.15.1:0/2289977644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78300b3590 tx=0x7f782802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49f8a8cda06fa39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7834ae9640 1 -- 172.21.15.1:0/2289977644 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828037070 con 0x7f7830161950 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7834ae9640 1 -- 172.21.15.1:0/2289977644 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828032440 con 0x7f7830161950 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7834ae9640 1 -- 172.21.15.1:0/2289977644 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828032810 con 0x7f7830161950 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 -- 172.21.15.1:0/2289977644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 msgr2=0x7f78300d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2289977644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78300b3590 tx=0x7f782802a180 comp rx=0 tx=0).stop 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 -- 172.21.15.1:0/2289977644 shutdown_connections 2024-05-09T06:27:17.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2289977644 >> v2:172.21.15.6:3300/0 conn(0x7f78300d0f80 0x7f78300d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2289977644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7830161950 0x7f78300d2400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.982+0000 7f7837d4f640 1 -- 172.21.15.1:0/2289977644 >> 172.21.15.1:0/2289977644 conn(0x7f78300d3e20 msgr2=0x7f78300d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:17.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.983+0000 7f7837d4f640 1 -- 172.21.15.1:0/2289977644 shutdown_connections 2024-05-09T06:27:17.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.983+0000 7f7837d4f640 1 -- 172.21.15.1:0/2289977644 wait complete. 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.984+0000 7f7837d4f640 1 Processor -- start 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.984+0000 7f7837d4f640 1 -- start start 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.984+0000 7f7837d4f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 0x7f78301f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830161950 0x7f78301f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 0x7f783015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78300b52d0 con 0x7f7830156c70 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f78300b5150 con 0x7f7830161950 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f78300b5450 con 0x7f78300d0f80 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f78362ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 0x7f783015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 0x7f78301f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f78362ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 0x7f783015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49012/0 (socket says 172.21.15.1:49012) 2024-05-09T06:27:17.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f78362ec640 1 -- 172.21.15.1:0/2533951819 learned_addr learned my addr 172.21.15.1:0/2533951819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f78352ea640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830161950 0x7f78301f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 -- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830161950 msgr2=0x7f78301f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830161950 0x7f78301f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 -- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 msgr2=0x7f783015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 0x7f783015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 -- 172.21.15.1:0/2533951819 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78301f7740 con 0x7f78300d0f80 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7835aeb640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 0x7f78301f4560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7828031000 tx=0x7f78280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828041020 con 0x7f78300d0f80 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828030700 con 0x7f78300d0f80 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828037040 con 0x7f78300d0f80 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.985+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78301f7a20 con 0x7f78300d0f80 2024-05-09T06:27:17.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.986+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78300b6bb0 con 0x7f78300d0f80 2024-05-09T06:27:17.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.986+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78301f7f40 con 0x7f78300d0f80 2024-05-09T06:27:17.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.987+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7828044020 con 0x7f78300d0f80 2024-05-09T06:27:17.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.987+0000 7f782e7fc640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:17.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.988+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f78280ab170 con 0x7f78300d0f80 2024-05-09T06:27:17.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.988+0000 7f78352ea640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:17.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.988+0000 7f78352ea640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 0x7f781c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f783015e4e0 tx=0x7f7824007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:17.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:17.993+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78301f7f40 con 0x7f78300d0f80 2024-05-09T06:27:18.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.180+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78300752b0 con 0x7f78300d0f80 2024-05-09T06:27:18.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.181+0000 7f782e7fc640 1 -- 172.21.15.1:0/2533951819 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12752 (secure 0 0 0) 0x7f782804b030 con 0x7f78300d0f80 2024-05-09T06:27:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:18.162 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":16,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:11.205914+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:18.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 msgr2=0x7f781c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 0x7f781c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f783015e4e0 tx=0x7f7824007e80 comp rx=0 tx=0).stop 2024-05-09T06:27:18.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 msgr2=0x7f78301f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 0x7f78301f4560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7828031000 tx=0x7f78280046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 shutdown_connections 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7830156c70 0x7f783015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7830161950 0x7f78301f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 --2- 172.21.15.1:0/2533951819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78300d0f80 0x7f78301f4560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 >> 172.21.15.1:0/2533951819 conn(0x7f78300d3e20 msgr2=0x7f78300ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 shutdown_connections 2024-05-09T06:27:18.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.183+0000 7f7837d4f640 1 -- 172.21.15.1:0/2533951819 wait complete. 2024-05-09T06:27:18.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:27:18.175 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:18.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.301+0000 7f34b2fd5640 1 Processor -- start 2024-05-09T06:27:18.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.301+0000 7f34b2fd5640 1 -- start start 2024-05-09T06:27:18.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34b2fd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f34ac15e0f0 0x7f34ac0d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34b2fd5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34ac0d2760 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34b2fd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34ac0d2a40 con 0x7f34ac0b85d0 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34b2fd5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac0b6e50 con 0x7f34ac15e0f0 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f34ac0d0a10 0x7f34ac0ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36832/0 (socket says 172.21.15.1:36832) 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a3fff640 1 -- 172.21.15.1:0/1613532348 learned_addr learned my addr 172.21.15.1:0/1613532348 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34b0d71640 1 --2- 172.21.15.1:0/1613532348 >> v2:172.21.15.6:3300/0 conn(0x7f34ac15e0f0 0x7f34ac0d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769088172 0 0) 0x7f34ac0d2760 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3494003520 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2509952985 0 0) 0x7f3494003520 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 >> v2:172.21.15.6:3300/0 conn(0x7f34ac15e0f0 msgr2=0x7f34ac0d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 --2- 172.21.15.1:0/1613532348 >> v2:172.21.15.6:3300/0 conn(0x7f34ac15e0f0 0x7f34ac0d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 >> v1:172.21.15.6:6789/0 conn(0x7f34ac0b85d0 legacy=0x7f34ac0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0d2d20 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f349c001670 con 0x7f34ac0d0a10 2024-05-09T06:27:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 >> v1:172.21.15.1:6789/0 conn(0x7f34ac0d0a10 legacy=0x7f34ac0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34940039b0 0x7f3494003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.302+0000 7f34a37fe640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f3494004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a37fe640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3494005080 0x7f349400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac0b6e50 con 0x7f34940043b0 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f349400bfe0 con 0x7f34940039b0 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f349400c1e0 con 0x7f3494005080 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f3494004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34b0d71640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34940039b0 0x7f3494003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34b1572640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3494005080 0x7f349400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 -- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3494005080 msgr2=0x7f349400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3494005080 0x7f349400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 -- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34940039b0 msgr2=0x7f3494003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34940039b0 0x7f3494003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34b1572640 1 -- 172.21.15.1:0/1613532348 reap_dead start 2024-05-09T06:27:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.303+0000 7f34a3fff640 1 -- 172.21.15.1:0/1613532348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3494003520 con 0x7f34940043b0 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34a3fff640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f3494004880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f349c007fd0 tx=0x7f3494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a0e3c80b13f648 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c007ca0 con 0x7f34940043b0 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c005ad0 con 0x7f34940043b0 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34a37fe640 1 -- 172.21.15.1:0/1613532348 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c007070 con 0x7f34940043b0 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 -- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 msgr2=0x7f3494004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f3494004880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f349c007fd0 tx=0x7f3494005830 comp rx=0 tx=0).stop 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 -- 172.21.15.1:0/1613532348 shutdown_connections 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/1613532348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f3494004880 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 -- 172.21.15.1:0/1613532348 >> 172.21.15.1:0/1613532348 conn(0x7f34ac09d3b0 msgr2=0x7f34ac0d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 -- 172.21.15.1:0/1613532348 shutdown_connections 2024-05-09T06:27:18.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.304+0000 7f34b2fd5640 1 -- 172.21.15.1:0/1613532348 wait complete. 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 Processor -- start 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 -- start start 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 0x7f34ac0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34ac09fbb0 0x7f34ac0a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac15e650 con 0x7f34940043b0 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac15e4d0 con 0x7f34ac0cd0e0 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.306+0000 7f34b2fd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0cfd70 con 0x7f34ac09fbb0 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49018/0 (socket says 172.21.15.1:49018) 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 -- 172.21.15.1:0/3702746530 learned_addr learned my addr 172.21.15.1:0/3702746530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b1572640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34ac09fbb0 0x7f34ac0a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34a3fff640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 0x7f34ac0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 -- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34ac09fbb0 msgr2=0x7f34ac0a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34ac09fbb0 0x7f34ac0a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 -- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 msgr2=0x7f34ac0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 0x7f34ac0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 -- 172.21.15.1:0/3702746530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0a11e0 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34a3fff640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 0x7f34ac0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b0d71640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34a80118f0 tx=0x7f34a8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8019070 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8004240 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac1f3760 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800f820 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.307+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac0a0c20 con 0x7f34940043b0 2024-05-09T06:27:18.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.308+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac0752b0 con 0x7f34940043b0 2024-05-09T06:27:18.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.309+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34a801c070 con 0x7f34940043b0 2024-05-09T06:27:18.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.309+0000 7f34a17fa640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 0x7f349806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.309+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f34a808add0 con 0x7f34940043b0 2024-05-09T06:27:18.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.309+0000 7f34a3fff640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 0x7f349806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.310+0000 7f34a3fff640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 0x7f349806e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f349c032000 tx=0x7f349c034040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.313+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34a8059510 con 0x7f34940043b0 2024-05-09T06:27:18.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.497+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34ac0b9670 con 0x7f34940043b0 2024-05-09T06:27:18.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.497+0000 7f34a17fa640 1 -- 172.21.15.1:0/3702746530 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f34a805d810 con 0x7f34940043b0 2024-05-09T06:27:18.479 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 msgr2=0x7f349806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 0x7f349806e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f349c032000 tx=0x7f349c034040 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 msgr2=0x7f34ac1f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34a80118f0 tx=0x7f34a8011db0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 shutdown_connections 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349806be30 0x7f349806e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34ac09fbb0 0x7f34ac0a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34ac0cd0e0 0x7f34ac0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 --2- 172.21.15.1:0/3702746530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34940043b0 0x7f34ac1f3220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 >> 172.21.15.1:0/3702746530 conn(0x7f34ac09d3b0 msgr2=0x7f34ac06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.499+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 shutdown_connections 2024-05-09T06:27:18.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.500+0000 7f34b2fd5640 1 -- 172.21.15.1:0/3702746530 wait complete. 2024-05-09T06:27:18.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:27:18.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.614+0000 7f6fac5f4640 1 Processor -- start 2024-05-09T06:27:18.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.614+0000 7f6fac5f4640 1 -- start start 2024-05-09T06:27:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fa40cf1e0 0x7f6fa406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6fa406f550 0x7f6fa406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa40ce740 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa415d390 con 0x7f6fa406f550 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fac5f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40ce9e0 con 0x7f6fa415c030 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fa9b8f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fa40cf1e0 0x7f6fa406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fa9b8f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fa40cf1e0 0x7f6fa406ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49024/0 (socket says 172.21.15.1:49024) 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6fa9b8f640 1 -- 172.21.15.1:0/473695514 learned_addr learned my addr 172.21.15.1:0/473695514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6faab91640 1 --2- 172.21.15.1:0/473695514 >> v2:172.21.15.6:3300/0 conn(0x7f6fa406f550 0x7f6fa406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.615+0000 7f6faa390640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 -- 172.21.15.1:0/473695514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 msgr2=0x7f6fa40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa40b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 -- 172.21.15.1:0/473695514 >> v2:172.21.15.6:3300/0 conn(0x7f6fa406f550 msgr2=0x7f6fa406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/473695514 >> v2:172.21.15.6:3300/0 conn(0x7f6fa406f550 0x7f6fa406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 -- 172.21.15.1:0/473695514 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa40cecc0 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.616+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa406ee90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b7be0 tx=0x7f6f9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acef45d06e093cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fa938e640 1 -- 172.21.15.1:0/473695514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94037070 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fa938e640 1 -- 172.21.15.1:0/473695514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f94032440 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fa938e640 1 -- 172.21.15.1:0/473695514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94032810 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 -- 172.21.15.1:0/473695514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40cf1e0 msgr2=0x7f6fa406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa406ee90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b7be0 tx=0x7f6f9402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 -- 172.21.15.1:0/473695514 shutdown_connections 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/473695514 >> v2:172.21.15.6:3300/0 conn(0x7f6fa406f550 0x7f6fa406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa406ee90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/473695514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa40b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 -- 172.21.15.1:0/473695514 >> 172.21.15.1:0/473695514 conn(0x7f6fa40b37c0 msgr2=0x7f6fa40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 -- 172.21.15.1:0/473695514 shutdown_connections 2024-05-09T06:27:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.617+0000 7f6fac5f4640 1 -- 172.21.15.1:0/473695514 wait complete. 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.618+0000 7f6fac5f4640 1 Processor -- start 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.618+0000 7f6fac5f4640 1 -- start start 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa40c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa4150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa415c590 con 0x7f6fa406f550 2024-05-09T06:27:18.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa415c410 con 0x7f6fa40cf1e0 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6fac5f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40cc850 con 0x7f6fa415c030 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6faa390640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6faa390640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49036/0 (socket says 172.21.15.1:49036) 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6faa390640 1 -- 172.21.15.1:0/3699484156 learned_addr learned my addr 172.21.15.1:0/3699484156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.619+0000 7f6faab91640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa4150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa40c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 -- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 msgr2=0x7f6fa4150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa4150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 -- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 msgr2=0x7f6fa40c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa40c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 -- 172.21.15.1:0/3699484156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa406c420 con 0x7f6fa406f550 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa40c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6faa390640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000e870 tx=0x7f6fa000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0017070 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa000c940 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000cd30 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa406c700 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.620+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa40c9df0 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.621+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa406e300 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.622+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fa001a070 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.622+0000 7f6f9affd640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 0x7f6f8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.622+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6fa008a8d0 con 0x7f6fa406f550 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.622+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 0x7f6f8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.623+0000 7f6fa9b8f640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 0x7f6f8806e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f94034000 tx=0x7f6f9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.626+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fa0058ee0 con 0x7f6fa406f550 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.812+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6fa40752b0 con 0x7f6fa406f550 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.812+0000 7f6f9affd640 1 -- 172.21.15.1:0/3699484156 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f6fa005d1e0 con 0x7f6fa406f550 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:27:18.793 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:27:18.794 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:27:18.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:27:18.794 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:27:18.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 msgr2=0x7f6f8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 0x7f6f8806e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f94034000 tx=0x7f6f9402e000 comp rx=0 tx=0).stop 2024-05-09T06:27:18.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 msgr2=0x7f6fa4150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000e870 tx=0x7f6fa000ed30 comp rx=0 tx=0).stop 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 shutdown_connections 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f8806be60 0x7f6f8806e320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fa415c030 0x7f6fa4150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fa40cf1e0 0x7f6fa40c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 --2- 172.21.15.1:0/3699484156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa406f550 0x7f6fa4150a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 >> 172.21.15.1:0/3699484156 conn(0x7f6fa40b37c0 msgr2=0x7f6fa40b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 shutdown_connections 2024-05-09T06:27:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.814+0000 7f6fac5f4640 1 -- 172.21.15.1:0/3699484156 wait complete. 2024-05-09T06:27:18.806 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:27:18.806 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:27:18.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.933+0000 7f6d98714640 1 Processor -- start 2024-05-09T06:27:18.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.933+0000 7f6d98714640 1 -- start start 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d98714640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d9015c290 0x7f6d900cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d98714640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900cc7d0 0x7f6d900ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d98714640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900cf9c0 con 0x7f6d900cf350 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d98714640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6d900cfca0 con 0x7f6d9015c290 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d98714640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900cff80 con 0x7f6d900cc7d0 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d9015c290 0x7f6d900cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d9015c290 0x7f6d900cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49044/0 (socket says 172.21.15.1:49044) 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 -- 172.21.15.1:0/3015459458 learned_addr learned my addr 172.21.15.1:0/3015459458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d96cb1640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900cc7d0 0x7f6d900ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 -- 172.21.15.1:0/3015459458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900cc7d0 msgr2=0x7f6d900ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900cc7d0 0x7f6d900ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 -- 172.21.15.1:0/3015459458 >> v1:172.21.15.6:6789/0 conn(0x7f6d900cf350 legacy=0x7f6d9015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.934+0000 7f6d964b0640 1 -- 172.21.15.1:0/3015459458 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d900d0260 con 0x7f6d9015c290 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d964b0640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d9015c290 0x7f6d900cee10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009870 tx=0x7f6d8c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f766b1dd10d0b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d954ae640 1 -- 172.21.15.1:0/3015459458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c037070 con 0x7f6d9015c290 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d954ae640 1 -- 172.21.15.1:0/3015459458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c032440 con 0x7f6d9015c290 2024-05-09T06:27:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d954ae640 1 -- 172.21.15.1:0/3015459458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c032810 con 0x7f6d9015c290 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d98714640 1 -- 172.21.15.1:0/3015459458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d9015c290 msgr2=0x7f6d900cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.935+0000 7f6d98714640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d9015c290 0x7f6d900cee10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009870 tx=0x7f6d8c02a1c0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 -- 172.21.15.1:0/3015459458 shutdown_connections 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900cc7d0 0x7f6d900ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 --2- 172.21.15.1:0/3015459458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d9015c290 0x7f6d900cee10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 -- 172.21.15.1:0/3015459458 >> 172.21.15.1:0/3015459458 conn(0x7f6d900d1cc0 msgr2=0x7f6d900d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 -- 172.21.15.1:0/3015459458 shutdown_connections 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.936+0000 7f6d98714640 1 -- 172.21.15.1:0/3015459458 wait complete. 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.937+0000 7f6d98714640 1 Processor -- start 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.937+0000 7f6d98714640 1 -- start start 2024-05-09T06:27:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 0x7f6d900cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d9015c290 0x7f6d900c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9015cd80 con 0x7f6d900cf350 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9015d260 con 0x7f6d900cc7d0 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900d0a60 con 0x7f6d9015c290 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49046/0 (socket says 172.21.15.1:49046) 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 -- 172.21.15.1:0/4282233656 learned_addr learned my addr 172.21.15.1:0/4282233656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d96cb1640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d9015c290 0x7f6d900c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d964b0640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 0x7f6d900cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 -- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d9015c290 msgr2=0x7f6d900c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d9015c290 0x7f6d900c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 -- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 msgr2=0x7f6d900cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 0x7f6d900cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 -- 172.21.15.1:0/4282233656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d900ca120 con 0x7f6d900cf350 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d964b0640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 0x7f6d900cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d95caf640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000c400 tx=0x7f6d8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d800052d0 con 0x7f6d900cf350 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80005490 con 0x7f6d900cf350 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d900ca400 con 0x7f6d900cf350 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80007260 con 0x7f6d900cf350 2024-05-09T06:27:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.938+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d900ca9c0 con 0x7f6d900cf350 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.939+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d90075320 con 0x7f6d900cf350 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.940+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d80007400 con 0x7f6d900cf350 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.940+0000 7f6d86ffd640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 0x7f6d7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.940+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6d80089f20 con 0x7f6d900cf350 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.940+0000 7f6d964b0640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 0x7f6d7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.941+0000 7f6d964b0640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 0x7f6d7406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c034000 tx=0x7f6d8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:18.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:18.945+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d80058660 con 0x7f6d900cf350 2024-05-09T06:27:19.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.134+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d900730b0 con 0x7f6d900cf350 2024-05-09T06:27:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.615+0000 7f6d86ffd640 1 -- 172.21.15.1:0/4282233656 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f6d8005c960 con 0x7f6d900cf350 2024-05-09T06:27:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:27:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 msgr2=0x7f6d7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 0x7f6d7406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c034000 tx=0x7f6d8c02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 msgr2=0x7f6d90150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000c400 tx=0x7f6d8000c8c0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 shutdown_connections 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d7406be30 0x7f6d7406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d9015c290 0x7f6d900c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900cf350 0x7f6d90150010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 --2- 172.21.15.1:0/4282233656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900cc7d0 0x7f6d900cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 >> 172.21.15.1:0/4282233656 conn(0x7f6d900d1cc0 msgr2=0x7f6d900aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 shutdown_connections 2024-05-09T06:27:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.618+0000 7f6d98714640 1 -- 172.21.15.1:0/4282233656 wait complete. 2024-05-09T06:27:19.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T06:27:19.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.748+0000 7f45ddbb7640 1 Processor -- start 2024-05-09T06:27:19.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.748+0000 7f45ddbb7640 1 -- start start 2024-05-09T06:27:19.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45ddbb7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f45d806c7a0 0x7f45d806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45ddbb7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d80cefc0 con 0x7f45d80b34e0 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45ddbb7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d80cf2a0 con 0x7f45d815e120 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45ddbb7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f45d80d02b0 con 0x7f45d806c7a0 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f45d806c7a0 0x7f45d806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d6ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f45d80b34e0 0x7f45d80ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36848/0 (socket says 172.21.15.1:36848) 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f45d806c7a0 0x7f45d806cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49060/0 (socket says 172.21.15.1:49060) 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d6ffd640 1 -- 172.21.15.1:0/41324104 learned_addr learned my addr 172.21.15.1:0/41324104 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d7fff640 1 -- 172.21.15.1:0/41324104 >> v1:172.21.15.6:6789/0 conn(0x7f45d815e120 legacy=0x7f45d80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d67fc640 1 -- 172.21.15.1:0/41324104 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152749398 0 0) 0x7f45d80cefc0 con 0x7f45d80b34e0 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d7fff640 1 -- 172.21.15.1:0/41324104 >> v1:172.21.15.1:6789/0 conn(0x7f45d80b34e0 legacy=0x7f45d80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.749+0000 7f45d7fff640 1 -- 172.21.15.1:0/41324104 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d80cf580 con 0x7f45d806c7a0 2024-05-09T06:27:19.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.750+0000 7f45d7fff640 1 --2- 172.21.15.1:0/41324104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d806c7a0 0x7f45d806cc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d90 tx=0x7f45d002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac5c182ef32a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:19.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.750+0000 7f45d67fc640 1 -- 172.21.15.1:0/41324104 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0007740 con 0x7f45d806c7a0 2024-05-09T06:27:19.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.750+0000 7f45d67fc640 1 -- 172.21.15.1:0/41324104 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d00078e0 con 0x7f45d806c7a0 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.750+0000 7f45d67fc640 1 -- 172.21.15.1:0/41324104 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0007cb0 con 0x7f45d806c7a0 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 -- 172.21.15.1:0/41324104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d806c7a0 msgr2=0x7f45d806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/41324104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d806c7a0 0x7f45d806cc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d90 tx=0x7f45d002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 -- 172.21.15.1:0/41324104 shutdown_connections 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/41324104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d806c7a0 0x7f45d806cc30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 -- 172.21.15.1:0/41324104 >> 172.21.15.1:0/41324104 conn(0x7f45d809aa20 msgr2=0x7f45d809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 -- 172.21.15.1:0/41324104 shutdown_connections 2024-05-09T06:27:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.751+0000 7f45ddbb7640 1 -- 172.21.15.1:0/41324104 wait complete. 2024-05-09T06:27:19.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.752+0000 7f45ddbb7640 1 Processor -- start 2024-05-09T06:27:19.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.752+0000 7f45ddbb7640 1 -- start start 2024-05-09T06:27:19.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 0x7f45d814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d80b34e0 0x7f45d80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80d08c0 con 0x7f45d815e120 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80d0740 con 0x7f45d806c7a0 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45ddbb7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f45d80d0a40 con 0x7f45d80b34e0 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49072/0 (socket says 172.21.15.1:49072) 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 -- 172.21.15.1:0/727710386 learned_addr learned my addr 172.21.15.1:0/727710386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d6ffd640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d80b34e0 0x7f45d80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d77fe640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 0x7f45d814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 -- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d80b34e0 msgr2=0x7f45d80b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d80b34e0 0x7f45d80b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 -- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 msgr2=0x7f45d814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 0x7f45d814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d7fff640 1 -- 172.21.15.1:0/727710386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d80b6790 con 0x7f45d815e120 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.753+0000 7f45d77fe640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 0x7f45d814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45d7fff640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d90 tx=0x7f45d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0040070 con 0x7f45d815e120 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d0031c90 con 0x7f45d815e120 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d80b51b0 con 0x7f45d815e120 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d003c640 con 0x7f45d815e120 2024-05-09T06:27:19.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.754+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d80b5770 con 0x7f45d815e120 2024-05-09T06:27:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.755+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d80752b0 con 0x7f45d815e120 2024-05-09T06:27:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.755+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45d0039040 con 0x7f45d815e120 2024-05-09T06:27:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.756+0000 7f45bffff640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 0x7f45c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.756+0000 7f45d77fe640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 0x7f45c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.756+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f45d00ac2b0 con 0x7f45d815e120 2024-05-09T06:27:19.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.756+0000 7f45d77fe640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 0x7f45c406e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45c800a830 tx=0x7f45c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.760+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45d004b340 con 0x7f45d815e120 2024-05-09T06:27:19.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:19.943+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f45d80d1bf0 con 0x7f45d815e120 2024-05-09T06:27:20.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.617+0000 7f45bffff640 1 -- 172.21.15.1:0/727710386 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f45d007aef0 con 0x7f45d815e120 2024-05-09T06:27:20.599 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 msgr2=0x7f45c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 0x7f45c406e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45c800a830 tx=0x7f45c8008040 comp rx=0 tx=0).stop 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 msgr2=0x7f45d815c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d90 tx=0x7f45d0004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 shutdown_connections 2024-05-09T06:27:20.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45c406be60 0x7f45c406e320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d815e120 0x7f45d815c2a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d80b34e0 0x7f45d80b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 --2- 172.21.15.1:0/727710386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d806c7a0 0x7f45d814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 >> 172.21.15.1:0/727710386 conn(0x7f45d809aa20 msgr2=0x7f45d806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:20.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.621+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 shutdown_connections 2024-05-09T06:27:20.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.622+0000 7f45ddbb7640 1 -- 172.21.15.1:0/727710386 wait complete. 2024-05-09T06:27:20.633 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:27:20.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.764+0000 7f3a83377640 1 Processor -- start 2024-05-09T06:27:20.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.764+0000 7f3a83377640 1 -- start start 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a83377640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a83377640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b4de0 0x7f3a7c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a83377640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7c06cf40 con 0x7f3a7c0b5260 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a83377640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7c156980 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a83377640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c06d870 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81914640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3a7c0b5260 0x7f3a7c070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36864/0 (socket says 172.21.15.1:36864) 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49082/0 (socket says 172.21.15.1:49082) 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81914640 1 -- 172.21.15.1:0/4270135899 learned_addr learned my addr 172.21.15.1:0/4270135899 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a80912640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b4de0 0x7f3a7c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 -- 172.21.15.1:0/4270135899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b4de0 msgr2=0x7f3a7c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b4de0 0x7f3a7c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 -- 172.21.15.1:0/4270135899 >> v1:172.21.15.1:6789/0 conn(0x7f3a7c0b5260 legacy=0x7f3a7c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a73fff640 1 -- 172.21.15.1:0/4270135899 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231404019 0 0) 0x7f3a7c06cf40 con 0x7f3a7c0b5260 2024-05-09T06:27:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.765+0000 7f3a81113640 1 -- 172.21.15.1:0/4270135899 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c06d220 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.766+0000 7f3a81113640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097a0 tx=0x7f3a7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604be98c2e75d17d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.766+0000 7f3a73fff640 1 -- 172.21.15.1:0/4270135899 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78037070 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.766+0000 7f3a73fff640 1 -- 172.21.15.1:0/4270135899 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78032440 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.766+0000 7f3a73fff640 1 -- 172.21.15.1:0/4270135899 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78032810 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 -- 172.21.15.1:0/4270135899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c15c4c0 msgr2=0x7f3a7c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097a0 tx=0x7f3a7802a180 comp rx=0 tx=0).stop 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 -- 172.21.15.1:0/4270135899 shutdown_connections 2024-05-09T06:27:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b4de0 0x7f3a7c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 --2- 172.21.15.1:0/4270135899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c06dd00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 -- 172.21.15.1:0/4270135899 >> 172.21.15.1:0/4270135899 conn(0x7f3a7c0cc4d0 msgr2=0x7f3a7c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 -- 172.21.15.1:0/4270135899 shutdown_connections 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.767+0000 7f3a83377640 1 -- 172.21.15.1:0/4270135899 wait complete. 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.768+0000 7f3a83377640 1 Processor -- start 2024-05-09T06:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.768+0000 7f3a83377640 1 -- start start 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b5260 0x7f3a7c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c0b2dc0 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c0b2c40 con 0x7f3a7c15c4c0 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c15cde0 con 0x7f3a7c0b5260 2024-05-09T06:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49094/0 (socket says 172.21.15.1:49094) 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 -- 172.21.15.1:0/3679088308 learned_addr learned my addr 172.21.15.1:0/3679088308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a80912640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b5260 0x7f3a7c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 -- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b5260 msgr2=0x7f3a7c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b5260 0x7f3a7c15af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 -- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c4c0 msgr2=0x7f3a7c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 -- 172.21.15.1:0/3679088308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c0b1280 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a81113640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a78000c00 tx=0x7f3a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78037040 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78042070 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c0cd710 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.769+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78030bb0 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.770+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c0cdcd0 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.771+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c0752b0 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.771+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a78047460 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.772+0000 7f3a71ffb640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 0x7f3a6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.772+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f3a780abaa0 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.772+0000 7f3a80912640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 0x7f3a6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.772+0000 7f3a80912640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 0x7f3a6006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c1f7260 tx=0x7f3a6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:20.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.776+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a78032e20 con 0x7f3a7c0b4de0 2024-05-09T06:27:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:20.997+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3a7c0cd4f0 con 0x7f3a7c0b4de0 2024-05-09T06:27:21.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.636+0000 7f3a71ffb640 1 -- 172.21.15.1:0/3679088308 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f3a78045360 con 0x7f3a7c0b4de0 2024-05-09T06:27:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.646+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 msgr2=0x7f3a6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.646+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 0x7f3a6006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c1f7260 tx=0x7f3a6c008040 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 msgr2=0x7f3a7c1557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a78000c00 tx=0x7f3a780046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 shutdown_connections 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3a6006be30 0x7f3a6006e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c4c0 0x7f3a7c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a7c0b5260 0x7f3a7c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 --2- 172.21.15.1:0/3679088308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a7c0b4de0 0x7f3a7c1557f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.647+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 >> 172.21.15.1:0/3679088308 conn(0x7f3a7c0cc4d0 msgr2=0x7f3a7c06dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.648+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 shutdown_connections 2024-05-09T06:27:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.648+0000 7f3a83377640 1 -- 172.21.15.1:0/3679088308 wait complete. 2024-05-09T06:27:21.650 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T06:27:21.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T06:27:21.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.784+0000 7f6970815640 1 Processor -- start 2024-05-09T06:27:21.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.784+0000 7f6970815640 1 -- start start 2024-05-09T06:27:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6970815640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f696c15e5f0 0x7f696c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6970815640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6970815640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696c0cb0e0 con 0x7f696c15e210 2024-05-09T06:27:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6970815640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0cca50 con 0x7f696c15e5f0 2024-05-09T06:27:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6970815640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0b17c0 con 0x7f696c1628f0 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f696c15e5f0 0x7f696c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f696c15e5f0 0x7f696c0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49104/0 (socket says 172.21.15.1:49104) 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 -- 172.21.15.1:0/1546878284 learned_addr learned my addr 172.21.15.1:0/1546878284 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f696ad9d640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 -- 172.21.15.1:0/1546878284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 msgr2=0x7f696c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 -- 172.21.15.1:0/1546878284 >> v1:172.21.15.6:6789/0 conn(0x7f696c15e210 legacy=0x7f696c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.785+0000 7f6969d9b640 1 -- 172.21.15.1:0/1546878284 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c0b1aa0 con 0x7f696c15e5f0 2024-05-09T06:27:21.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6969d9b640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c15e5f0 0x7f696c0b1170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6954009e50 tx=0x7f695402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92146c8d2d75e924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:21.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f696959a640 1 -- 172.21.15.1:0/1546878284 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954036070 con 0x7f696c15e5f0 2024-05-09T06:27:21.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f696959a640 1 -- 172.21.15.1:0/1546878284 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954031600 con 0x7f696c15e5f0 2024-05-09T06:27:21.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f696959a640 1 -- 172.21.15.1:0/1546878284 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69540319d0 con 0x7f696c15e5f0 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 -- 172.21.15.1:0/1546878284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c15e5f0 msgr2=0x7f696c0b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c15e5f0 0x7f696c0b1170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6954009e50 tx=0x7f695402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 -- 172.21.15.1:0/1546878284 shutdown_connections 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 --2- 172.21.15.1:0/1546878284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c15e5f0 0x7f696c0b1170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.786+0000 7f6970815640 1 -- 172.21.15.1:0/1546878284 >> 172.21.15.1:0/1546878284 conn(0x7f696c071a50 msgr2=0x7f696c0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.787+0000 7f6970815640 1 -- 172.21.15.1:0/1546878284 shutdown_connections 2024-05-09T06:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.787+0000 7f6970815640 1 -- 172.21.15.1:0/1546878284 wait complete. 2024-05-09T06:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.788+0000 7f6970815640 1 Processor -- start 2024-05-09T06:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.788+0000 7f6970815640 1 -- start start 2024-05-09T06:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c15e210 0x7f696c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0cd420 con 0x7f696c0afcf0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f696c162d70 con 0x7f696c15e210 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0cd5a0 con 0x7f696c1628f0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49118/0 (socket says 172.21.15.1:49118) 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 -- 172.21.15.1:0/859623749 learned_addr learned my addr 172.21.15.1:0/859623749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6969d9b640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696a59c640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c15e210 0x7f696c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 -- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 msgr2=0x7f696c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 -- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c15e210 msgr2=0x7f696c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c15e210 0x7f696c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 -- 172.21.15.1:0/859623749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c1578e0 con 0x7f696c0afcf0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f696ad9d640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c580 tx=0x7f695c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c015070 con 0x7f696c0afcf0 2024-05-09T06:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c004220 con 0x7f696c0afcf0 2024-05-09T06:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c0adf10 con 0x7f696c0afcf0 2024-05-09T06:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.789+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c003610 con 0x7f696c0afcf0 2024-05-09T06:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.790+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c0ae4d0 con 0x7f696c0afcf0 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.791+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c0aef90 con 0x7f696c0afcf0 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.791+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f695c0037b0 con 0x7f696c0afcf0 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.791+0000 7f6962ffd640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.792+0000 7f696a59c640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.792+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f695c08aad0 con 0x7f696c0afcf0 2024-05-09T06:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.792+0000 7f696a59c640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 0x7f694c06e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f696c1628c0 tx=0x7f6964009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.796+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f696c0aef90 con 0x7f696c0afcf0 2024-05-09T06:27:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:21.988+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f696c075320 con 0x7f696c0afcf0 2024-05-09T06:27:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.639+0000 7f6962ffd640 1 -- 172.21.15.1:0/859623749 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f696c1578e0 con 0x7f696c0afcf0 2024-05-09T06:27:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 msgr2=0x7f694c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 0x7f694c06e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f696c1628c0 tx=0x7f6964009040 comp rx=0 tx=0).stop 2024-05-09T06:27:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 msgr2=0x7f696c15ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c580 tx=0x7f695c00ca40 comp rx=0 tx=0).stop 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 shutdown_connections 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0afcf0 0x7f696c15ea50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f696c1628f0 0x7f696c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 --2- 172.21.15.1:0/859623749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c15e210 0x7f696c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 >> 172.21.15.1:0/859623749 conn(0x7f696c071a50 msgr2=0x7f696c06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 shutdown_connections 2024-05-09T06:27:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.643+0000 7f6970815640 1 -- 172.21.15.1:0/859623749 wait complete. 2024-05-09T06:27:22.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:27:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.775+0000 7f581889a640 1 Processor -- start 2024-05-09T06:27:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.775+0000 7f581889a640 1 -- start start 2024-05-09T06:27:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581889a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58140d2c70 0x7f58140cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581889a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140cf2c0 con 0x7f581415e070 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581889a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140cf5a0 con 0x7f581415e450 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581889a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f58140cfed0 con 0x7f58140d2c70 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f581415e070 0x7f5814162800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36866/0 (socket says 172.21.15.1:36866) 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f5812d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58140d2c70 0x7f58140cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.776+0000 7f581259c640 1 -- 172.21.15.1:0/2510183988 learned_addr learned my addr 172.21.15.1:0/2510183988 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f5812d9d640 1 -- 172.21.15.1:0/2510183988 >> v1:172.21.15.6:6789/0 conn(0x7f581415e450 legacy=0x7f58140d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f5812d9d640 1 -- 172.21.15.1:0/2510183988 >> v1:172.21.15.1:6789/0 conn(0x7f581415e070 legacy=0x7f5814162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f5812d9d640 1 -- 172.21.15.1:0/2510183988 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140cf880 con 0x7f58140d2c70 2024-05-09T06:27:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f581159a640 1 -- 172.21.15.1:0/2510183988 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637939157 0 0) 0x7f58140cf2c0 con 0x7f581415e070 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f5812d9d640 1 --2- 172.21.15.1:0/2510183988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58140d2c70 0x7f58140cec40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58140d0050 tx=0x7f580c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d124973c2ed8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f581159a640 1 -- 172.21.15.1:0/2510183988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c007740 con 0x7f58140d2c70 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f581159a640 1 -- 172.21.15.1:0/2510183988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c0078e0 con 0x7f58140d2c70 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.777+0000 7f581159a640 1 -- 172.21.15.1:0/2510183988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c007cb0 con 0x7f58140d2c70 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 -- 172.21.15.1:0/2510183988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58140d2c70 msgr2=0x7f58140cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 --2- 172.21.15.1:0/2510183988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58140d2c70 0x7f58140cec40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58140d0050 tx=0x7f580c02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 -- 172.21.15.1:0/2510183988 shutdown_connections 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 --2- 172.21.15.1:0/2510183988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58140d2c70 0x7f58140cec40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 -- 172.21.15.1:0/2510183988 >> 172.21.15.1:0/2510183988 conn(0x7f581409d1f0 msgr2=0x7f58140d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 -- 172.21.15.1:0/2510183988 shutdown_connections 2024-05-09T06:27:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.778+0000 7f581889a640 1 -- 172.21.15.1:0/2510183988 wait complete. 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.779+0000 7f581889a640 1 Processor -- start 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.779+0000 7f581889a640 1 -- start start 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58140d2c70 0x7f58141583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f581415e070 0x7f58141e8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58140b8060 con 0x7f581415e450 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f58140b7ee0 con 0x7f58140d2c70 2024-05-09T06:27:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581889a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f58140b81e0 con 0x7f581415e070 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f5812d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f5812d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49138/0 (socket says 172.21.15.1:49138) 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f5812d9d640 1 -- 172.21.15.1:0/4195433744 learned_addr learned my addr 172.21.15.1:0/4195433744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f581259c640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58140d2c70 0x7f58141583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.780+0000 7f5811d9b640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f581415e070 0x7f58141e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 -- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f581415e070 msgr2=0x7f58141e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f581415e070 0x7f58141e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 -- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58140d2c70 msgr2=0x7f58141583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58140d2c70 0x7f58141583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 -- 172.21.15.1:0/4195433744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814159130 con 0x7f581415e450 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5812d9d640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f580c009e50 tx=0x7f580c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c040070 con 0x7f581415e450 2024-05-09T06:27:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c031cc0 con 0x7f581415e450 2024-05-09T06:27:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c045680 con 0x7f581415e450 2024-05-09T06:27:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58140cca60 con 0x7f581415e450 2024-05-09T06:27:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.781+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581415abe0 con 0x7f581415e450 2024-05-09T06:27:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.782+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814068250 con 0x7f581415e450 2024-05-09T06:27:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.783+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f580c039040 con 0x7f581415e450 2024-05-09T06:27:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.783+0000 7f5802ffd640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 0x7f57f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.783+0000 7f581259c640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 0x7f57f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.783+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f580c0ac9c0 con 0x7f581415e450 2024-05-09T06:27:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.784+0000 7f581259c640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 0x7f57f406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5804000f60 tx=0x7f5804007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.787+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f580c0aee20 con 0x7f581415e450 2024-05-09T06:27:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:22.969+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f581406cd50 con 0x7f581415e450 2024-05-09T06:27:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.646+0000 7f5802ffd640 1 -- 172.21.15.1:0/4195433744 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f5814159130 con 0x7f581415e450 2024-05-09T06:27:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' created 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 msgr2=0x7f57f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 0x7f57f406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5804000f60 tx=0x7f5804007b50 comp rx=0 tx=0).stop 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 msgr2=0x7f58140cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f580c009e50 tx=0x7f580c004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 shutdown_connections 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f57f406be30 0x7f57f406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f581415e450 0x7f58140cc320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f581415e070 0x7f58141e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 --2- 172.21.15.1:0/4195433744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58140d2c70 0x7f58141583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.650+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 >> 172.21.15.1:0/4195433744 conn(0x7f581409d1f0 msgr2=0x7f581415ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.651+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 shutdown_connections 2024-05-09T06:27:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.651+0000 7f581889a640 1 -- 172.21.15.1:0/4195433744 wait complete. 2024-05-09T06:27:24.652 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T06:27:24.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.784+0000 7f2d6079e640 1 Processor -- start 2024-05-09T06:27:24.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.784+0000 7f2d6079e640 1 -- start start 2024-05-09T06:27:24.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.785+0000 7f2d6079e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d580cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:24.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.785+0000 7f2d6079e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d580d1cc0 con 0x7f2d580b2ed0 2024-05-09T06:27:24.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.785+0000 7f2d6079e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d580d4d10 con 0x7f2d580d2630 2024-05-09T06:27:24.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.785+0000 7f2d6079e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d580d1f80 con 0x7f2d580b2110 2024-05-09T06:27:24.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5dd39640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2d580b2ed0 0x7f2d580ceba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36882/0 (socket says 172.21.15.1:36882) 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5dd39640 1 -- 172.21.15.1:0/2539121053 learned_addr learned my addr 172.21.15.1:0/2539121053 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5ed3b640 1 --2- 172.21.15.1:0/2539121053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d580cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051981751 0 0) 0x7f2d580d1cc0 con 0x7f2d580b2ed0 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d40003520 con 0x7f2d580b2ed0 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144404160 0 0) 0x7f2d580d4d10 con 0x7f2d580d2630 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d580d1cc0 con 0x7f2d580d2630 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5ed3b640 1 -- 172.21.15.1:0/2539121053 >> v1:172.21.15.6:6789/0 conn(0x7f2d580d2630 legacy=0x7f2d5815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5ed3b640 1 -- 172.21.15.1:0/2539121053 >> v1:172.21.15.1:6789/0 conn(0x7f2d580b2ed0 legacy=0x7f2d580ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.786+0000 7f2d5ed3b640 1 -- 172.21.15.1:0/2539121053 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580b0c00 con 0x7f2d580b2110 2024-05-09T06:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d5ed3b640 1 --2- 172.21.15.1:0/2539121053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d580cf2b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009dc0 tx=0x7f2d4802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b874758cec8efa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48007740 con 0x7f2d580b2110 2024-05-09T06:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d480078e0 con 0x7f2d580b2110 2024-05-09T06:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d5d538640 1 -- 172.21.15.1:0/2539121053 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48007cb0 con 0x7f2d580b2110 2024-05-09T06:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 -- 172.21.15.1:0/2539121053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 msgr2=0x7f2d580cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 --2- 172.21.15.1:0/2539121053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d580cf2b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009dc0 tx=0x7f2d4802a6d0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 -- 172.21.15.1:0/2539121053 shutdown_connections 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 --2- 172.21.15.1:0/2539121053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d580cf2b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 -- 172.21.15.1:0/2539121053 >> 172.21.15.1:0/2539121053 conn(0x7f2d580d3ee0 msgr2=0x7f2d580d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 -- 172.21.15.1:0/2539121053 shutdown_connections 2024-05-09T06:27:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.787+0000 7f2d6079e640 1 -- 172.21.15.1:0/2539121053 wait complete. 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.788+0000 7f2d6079e640 1 Processor -- start 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.788+0000 7f2d6079e640 1 -- start start 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d581581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d580d2630 0x7f2d580cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580d2b90 con 0x7f2d580b2ed0 2024-05-09T06:27:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580d2a10 con 0x7f2d580d2630 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d6079e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d580d2d10 con 0x7f2d580b2110 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49148/0 (socket says 172.21.15.1:49148) 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 -- 172.21.15.1:0/118154088 learned_addr learned my addr 172.21.15.1:0/118154088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5e53a640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d581581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5ed3b640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d580d2630 0x7f2d580cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 -- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 msgr2=0x7f2d581581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d581581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 -- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d580d2630 msgr2=0x7f2d580cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d580d2630 0x7f2d580cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 -- 172.21.15.1:0/118154088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580cb570 con 0x7f2d580b2ed0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d5dd39640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c500 tx=0x7f2d4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.789+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c017070 con 0x7f2d580b2ed0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.790+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c005ce0 con 0x7f2d580b2ed0 2024-05-09T06:27:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.790+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d580cb850 con 0x7f2d580b2ed0 2024-05-09T06:27:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.790+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0072f0 con 0x7f2d580b2ed0 2024-05-09T06:27:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.790+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d580cbe10 con 0x7f2d580b2ed0 2024-05-09T06:27:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.791+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d58075320 con 0x7f2d580b2ed0 2024-05-09T06:27:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.791+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d4c01a070 con 0x7f2d580b2ed0 2024-05-09T06:27:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.792+0000 7f2d56ffd640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 0x7f2d4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.792+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f2d4c08a960 con 0x7f2d580b2ed0 2024-05-09T06:27:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.792+0000 7f2d5e53a640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 0x7f2d4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.792+0000 7f2d5e53a640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 0x7f2d4406e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007fd0 tx=0x7f2d50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.796+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d4c058a30 con 0x7f2d580b2ed0 2024-05-09T06:27:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:24.978+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f2d580b2cb0 con 0x7f2d580b2ed0 2024-05-09T06:27:25.595 INFO:tasks.ceph.mds.b.smithi006.stderr:2024-05-09T06:27:25.613+0000 7fd8a620e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:27:25.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.647+0000 7f2d56ffd640 1 -- 172.21.15.1:0/118154088 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f2d4c05cd30 con 0x7f2d580b2ed0 2024-05-09T06:27:25.629 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T06:27:25.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.651+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 msgr2=0x7f2d4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.651+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 0x7f2d4406e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007fd0 tx=0x7f2d50009040 comp rx=0 tx=0).stop 2024-05-09T06:27:25.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.651+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 msgr2=0x7f2d5814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.651+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c500 tx=0x7f2d4c00c9c0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 shutdown_connections 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d4406be60 0x7f2d4406e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d580d2630 0x7f2d580cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d580b2ed0 0x7f2d5814f330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 --2- 172.21.15.1:0/118154088 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d580b2110 0x7f2d581581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.653+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 >> 172.21.15.1:0/118154088 conn(0x7f2d580d3ee0 msgr2=0x7f2d580d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.663+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 shutdown_connections 2024-05-09T06:27:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.663+0000 7f2d6079e640 1 -- 172.21.15.1:0/118154088 wait complete. 2024-05-09T06:27:25.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T06:27:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.795+0000 7f408bff9640 1 Processor -- start 2024-05-09T06:27:25.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 -- start start 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415e0f0 0x7f40840d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f40840b75c0 0x7f40840b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40840d2760 con 0x7f40840d0a10 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f40840b6e50 con 0x7f40840b75c0 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.796+0000 7f408bff9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f40840d2a40 con 0x7f408415e0f0 2024-05-09T06:27:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f40840b75c0 0x7f40840b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f40840b75c0 0x7f40840b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49150/0 (socket says 172.21.15.1:49150) 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 -- 172.21.15.1:0/2936519711 learned_addr learned my addr 172.21.15.1:0/2936519711 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f4089d95640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415e0f0 0x7f40840d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 -- 172.21.15.1:0/2936519711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415e0f0 msgr2=0x7f40840d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415e0f0 0x7f40840d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 -- 172.21.15.1:0/2936519711 >> v1:172.21.15.6:6789/0 conn(0x7f40840d0a10 legacy=0x7f40840d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.797+0000 7f408a596640 1 -- 172.21.15.1:0/2936519711 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40840d2d20 con 0x7f40840b75c0 2024-05-09T06:27:25.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408a596640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40840b75c0 0x7f40840b7a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4074009e50 tx=0x7f407402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb4f76d8504673 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:25.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f4088d93640 1 -- 172.21.15.1:0/2936519711 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074036070 con 0x7f40840b75c0 2024-05-09T06:27:25.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f4088d93640 1 -- 172.21.15.1:0/2936519711 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4074031600 con 0x7f40840b75c0 2024-05-09T06:27:25.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f4088d93640 1 -- 172.21.15.1:0/2936519711 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40740319d0 con 0x7f40840b75c0 2024-05-09T06:27:25.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 -- 172.21.15.1:0/2936519711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40840b75c0 msgr2=0x7f40840b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40840b75c0 0x7f40840b7a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4074009e50 tx=0x7f407402a710 comp rx=0 tx=0).stop 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 -- 172.21.15.1:0/2936519711 shutdown_connections 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40840b75c0 0x7f40840b7a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 --2- 172.21.15.1:0/2936519711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415e0f0 0x7f40840d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 -- 172.21.15.1:0/2936519711 >> 172.21.15.1:0/2936519711 conn(0x7f408409d3b0 msgr2=0x7f40840d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 -- 172.21.15.1:0/2936519711 shutdown_connections 2024-05-09T06:27:25.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.798+0000 7f408bff9640 1 -- 172.21.15.1:0/2936519711 wait complete. 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 Processor -- start 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 -- start start 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840b75c0 0x7f4084148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d0a10 0x7f40840ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f408415e650 con 0x7f408415e0f0 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f408415e4d0 con 0x7f40840d0a10 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.800+0000 7f408bff9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f40840cfd70 con 0x7f40840b75c0 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49158/0 (socket says 172.21.15.1:49158) 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 -- 172.21.15.1:0/633415330 learned_addr learned my addr 172.21.15.1:0/633415330 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f4089d95640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840b75c0 0x7f4084148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f4089594640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d0a10 0x7f40840ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 -- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840b75c0 msgr2=0x7f4084148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840b75c0 0x7f4084148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 -- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d0a10 msgr2=0x7f40840ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d0a10 0x7f40840ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 -- 172.21.15.1:0/633415330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4084145bd0 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408a596640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4074034040 tx=0x7f4074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074036050 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4074042070 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074009510 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40840cdda0 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.801+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4084146170 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.802+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40840752b0 con 0x7f408415e0f0 2024-05-09T06:27:25.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.803+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4074007bf0 con 0x7f408415e0f0 2024-05-09T06:27:25.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.804+0000 7f40827fc640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 0x7f407006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:25.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.804+0000 7f4089d95640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 0x7f407006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:25.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.804+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f40740abf20 con 0x7f408415e0f0 2024-05-09T06:27:25.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.804+0000 7f4089d95640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 0x7f407006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f407c009aa0 tx=0x7f407c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:25.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:25.808+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f407407a5b0 con 0x7f408415e0f0 2024-05-09T06:27:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.034+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f40840d12d0 con 0x7f408415e0f0 2024-05-09T06:27:26.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.662+0000 7f40827fc640 1 -- 172.21.15.1:0/633415330 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f4074079eb0 con 0x7f408415e0f0 2024-05-09T06:27:26.644 INFO:teuthology.orchestra.run.smithi001.stderr:added data pool 4 to fsmap 2024-05-09T06:27:26.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 msgr2=0x7f407006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 0x7f407006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f407c009aa0 tx=0x7f407c009190 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 msgr2=0x7f40840cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4074034040 tx=0x7f4074004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 shutdown_connections 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f407006be30 0x7f407006e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f408415e0f0 0x7f40840cd520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d0a10 0x7f40840ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 --2- 172.21.15.1:0/633415330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840b75c0 0x7f4084148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 >> 172.21.15.1:0/633415330 conn(0x7f408409d3b0 msgr2=0x7f40840cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 shutdown_connections 2024-05-09T06:27:26.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.665+0000 7f408bff9640 1 -- 172.21.15.1:0/633415330 wait complete. 2024-05-09T06:27:26.661 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.789+0000 7f91e60d4640 1 Processor -- start 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.789+0000 7f91e60d4640 1 -- start start 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 0x7f91e00b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f91e00b2dd0 0x7f91e00d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f91e00d2710 0x7f91e0161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f91e00d2b80 con 0x7f91e00d2710 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f91e00d1480 con 0x7f91e00b2dd0 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91e60d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f91e00d0870 con 0x7f91e01617a0 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f91e00d2710 0x7f91e0161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f91e00d2710 0x7f91e0161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49164/0 (socket says 172.21.15.1:49164) 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 -- 172.21.15.1:0/3868880724 learned_addr learned my addr 172.21.15.1:0/3868880724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91df7fe640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 0x7f91e00b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 -- 172.21.15.1:0/3868880724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 msgr2=0x7f91e00b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91deffd640 1 --2- 172.21.15.1:0/3868880724 >> v2:172.21.15.6:3300/0 conn(0x7f91e00b2dd0 0x7f91e00d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 0x7f91e00b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 -- 172.21.15.1:0/3868880724 >> v2:172.21.15.6:3300/0 conn(0x7f91e00b2dd0 msgr2=0x7f91e00d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 --2- 172.21.15.1:0/3868880724 >> v2:172.21.15.6:3300/0 conn(0x7f91e00b2dd0 0x7f91e00d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91dffff640 1 -- 172.21.15.1:0/3868880724 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e00d0b50 con 0x7f91e00d2710 2024-05-09T06:27:26.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.790+0000 7f91df7fe640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 0x7f91e00b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.791+0000 7f91dffff640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00d2710 0x7f91e0161f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91e008c7f0 tx=0x7f91d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5d33b6706b18ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.791+0000 7f91de7fc640 1 -- 172.21.15.1:0/3868880724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8036070 con 0x7f91e00d2710 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.791+0000 7f91de7fc640 1 -- 172.21.15.1:0/3868880724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8031600 con 0x7f91e00d2710 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.791+0000 7f91de7fc640 1 -- 172.21.15.1:0/3868880724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80319d0 con 0x7f91e00d2710 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 -- 172.21.15.1:0/3868880724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00d2710 msgr2=0x7f91e0161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00d2710 0x7f91e0161f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91e008c7f0 tx=0x7f91d802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 -- 172.21.15.1:0/3868880724 shutdown_connections 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00d2710 0x7f91e0161f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3868880724 >> v2:172.21.15.6:3300/0 conn(0x7f91e00b2dd0 0x7f91e00d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3868880724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e01617a0 0x7f91e00b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 -- 172.21.15.1:0/3868880724 >> 172.21.15.1:0/3868880724 conn(0x7f91e009d180 msgr2=0x7f91e00d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 -- 172.21.15.1:0/3868880724 shutdown_connections 2024-05-09T06:27:26.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.792+0000 7f91e60d4640 1 -- 172.21.15.1:0/3868880724 wait complete. 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.793+0000 7f91e60d4640 1 Processor -- start 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.793+0000 7f91e60d4640 1 -- start start 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e00d2710 0x7f91e00a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91e01617a0 0x7f91e015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0161b80 con 0x7f91e00b2dd0 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f91e00d1110 con 0x7f91e01617a0 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91e60d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f91e0161d00 con 0x7f91e00d2710 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49170/0 (socket says 172.21.15.1:49170) 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.794+0000 7f91df7fe640 1 -- 172.21.15.1:0/3745065066 learned_addr learned my addr 172.21.15.1:0/3745065066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 -- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e00d2710 msgr2=0x7f91e00a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e00d2710 0x7f91e00a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 -- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91e01617a0 msgr2=0x7f91e015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:26.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91dffff640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91e01617a0 0x7f91e015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91e01617a0 0x7f91e015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 -- 172.21.15.1:0/3745065066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e01f4390 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91df7fe640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c820 tx=0x7f91d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0015070 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0007a30 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01f4670 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0007e00 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.795+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01f4c30 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.796+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e01f4e70 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.796+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91d0018070 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.797+0000 7f91c7fff640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.797+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f91d008b190 con 0x7f91e00b2dd0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.797+0000 7f91deffd640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:26.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.797+0000 7f91deffd640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 0x7f91cc06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91e01f7490 tx=0x7f91d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:26.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:26.801+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91d0059800 con 0x7f91e00b2dd0 2024-05-09T06:27:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.026+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91e00752b0 con 0x7f91e00b2dd0 2024-05-09T06:27:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.027+0000 7f91c7fff640 1 -- 172.21.15.1:0/3745065066 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5156 (secure 0 0 0) 0x7f91d0059100 con 0x7f91e00b2dd0 2024-05-09T06:27:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:27.009 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:26.036056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2024-05-09T06:27:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 msgr2=0x7f91cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 0x7f91cc06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91e01f7490 tx=0x7f91d4008040 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 msgr2=0x7f91e01f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c820 tx=0x7f91d000cc20 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 shutdown_connections 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f91cc06be30 0x7f91cc06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91e01617a0 0x7f91e015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91e00d2710 0x7f91e00a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 --2- 172.21.15.1:0/3745065066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91e00b2dd0 0x7f91e01f15b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 >> 172.21.15.1:0/3745065066 conn(0x7f91e009d180 msgr2=0x7f91e00b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 shutdown_connections 2024-05-09T06:27:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.029+0000 7f91e60d4640 1 -- 172.21.15.1:0/3745065066 wait complete. 2024-05-09T06:27:27.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.144+0000 7fdddfb41640 1 Processor -- start 2024-05-09T06:27:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.144+0000 7fdddfb41640 1 -- start start 2024-05-09T06:27:27.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddfb41640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fddd806ffa0 0x7fddd80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddfb41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddd80d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddfb41640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd806c2f0 con 0x7fddd815e1d0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddfb41640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fddd80cc3c0 con 0x7fddd806ffa0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddfb41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80ccc60 con 0x7fddd815e5b0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddd8dd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fddd815e1d0 0x7fddd806f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36892/0 (socket says 172.21.15.1:36892) 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddd8dd640 1 -- 172.21.15.1:0/258560532 learned_addr learned my addr 172.21.15.1:0/258560532 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fddde0de640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddd80d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/258560532 >> v2:172.21.15.6:3300/0 conn(0x7fddd806ffa0 0x7fddd80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797135853 0 0) 0x7fddd4001110 con 0x7fddd815e1d0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fddd815e1d0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 963743221 0 0) 0x7fddc4003520 con 0x7fddd815e1d0 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 msgr2=0x7fddd80d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddd80d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 >> v2:172.21.15.6:3300/0 conn(0x7fddd806ffa0 msgr2=0x7fddd80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 --2- 172.21.15.1:0/258560532 >> v2:172.21.15.6:3300/0 conn(0x7fddd806ffa0 0x7fddd80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd806f500 con 0x7fddd815e1d0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fddd4001840 con 0x7fddd815e1d0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 >> v1:172.21.15.1:6789/0 conn(0x7fddd815e1d0 legacy=0x7fddd806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddc40039b0 0x7fddc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddc4004800 0x7fddc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.145+0000 7fdddc8db640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80cc3c0 con 0x7fddc4004800 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ccc60 con 0x7fddc40039b0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fddc400b9c0 con 0x7fddd815e5b0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddc4004800 0x7fddc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd8dd640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddc40039b0 0x7fddc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fddde0de640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 -- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 msgr2=0x7fddc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddd815e5b0 0x7fddc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 -- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddc40039b0 msgr2=0x7fddc4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddc40039b0 0x7fddc4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.146+0000 7fdddd0dc640 1 -- 172.21.15.1:0/258560532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4003520 con 0x7fddc4004800 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddc4004800 0x7fddc4004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009af0 tx=0x7fddc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8858e278e6f020 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8036070 con 0x7fddc4004800 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8031400 con 0x7fddc4004800 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddc8db640 1 -- 172.21.15.1:0/258560532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8009690 con 0x7fddc4004800 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 -- 172.21.15.1:0/258560532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddc4004800 msgr2=0x7fddc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 --2- 172.21.15.1:0/258560532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddc4004800 0x7fddc4004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009af0 tx=0x7fddc4005830 comp rx=0 tx=0).stop 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fddde0de640 1 -- 172.21.15.1:0/258560532 reap_dead start 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 -- 172.21.15.1:0/258560532 shutdown_connections 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 -- 172.21.15.1:0/258560532 >> 172.21.15.1:0/258560532 conn(0x7fddd80d1d70 msgr2=0x7fddd80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 -- 172.21.15.1:0/258560532 shutdown_connections 2024-05-09T06:27:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.147+0000 7fdddfb41640 1 -- 172.21.15.1:0/258560532 wait complete. 2024-05-09T06:27:27.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 Processor -- start 2024-05-09T06:27:27.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 -- start start 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 0x7fddd81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddd81599f0 0x7fddd8159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fddd806f1b0 con 0x7fddd80c8e40 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80cc8f0 con 0x7fddd81599f0 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.149+0000 7fdddfb41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80ce100 con 0x7fddc4004800 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49196/0 (socket says 172.21.15.1:49196) 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fdddd8dd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 0x7fddd81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 -- 172.21.15.1:0/2814819759 learned_addr learned my addr 172.21.15.1:0/2814819759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fdddd0dc640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddd81599f0 0x7fddd8159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 -- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 msgr2=0x7fddd81500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 0x7fddd81500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 -- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddd81599f0 msgr2=0x7fddd8159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddd81599f0 0x7fddd8159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 -- 172.21.15.1:0/2814819759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd80c57d0 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fdddd8dd640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 0x7fddd81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddde0de640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddd0011c80 tx=0x7fddd000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0015430 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd001d070 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd80c5a70 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0015830 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.150+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd80c6030 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.151+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd80752b0 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.152+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddd00159d0 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.152+0000 7fddce7fc640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.152+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fddd008b880 con 0x7fddd80c8e40 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.153+0000 7fdddd8dd640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.153+0000 7fdddd8dd640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddd4001eb0 tx=0x7fddd4007b00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.157+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fddd0010040 con 0x7fddd80c8e40 2024-05-09T06:27:27.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.340+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddd80d26a0 con 0x7fddd80c8e40 2024-05-09T06:27:27.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.341+0000 7fddce7fc640 1 -- 172.21.15.1:0/2814819759 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fddd0059800 con 0x7fddd80c8e40 2024-05-09T06:27:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:27.324 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:27.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 msgr2=0x7fddbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fddd4001eb0 tx=0x7fddd4007b00 comp rx=0 tx=0).stop 2024-05-09T06:27:27.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 msgr2=0x7fddd80c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddd0011c80 tx=0x7fddd000f580 comp rx=0 tx=0).stop 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 shutdown_connections 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddd80c8e40 0x7fddd80c92c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddd81599f0 0x7fddd8159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 --2- 172.21.15.1:0/2814819759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fddc4004800 0x7fddd81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 >> 172.21.15.1:0/2814819759 conn(0x7fddd80d1d70 msgr2=0x7fddd80af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 shutdown_connections 2024-05-09T06:27:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.343+0000 7fdddfb41640 1 -- 172.21.15.1:0/2814819759 wait complete. 2024-05-09T06:27:27.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:27.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.458+0000 7fcb0fabd640 1 Processor -- start 2024-05-09T06:27:27.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.458+0000 7fcb0fabd640 1 -- start start 2024-05-09T06:27:27.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0fabd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcb080cf180 0x7fcb080cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0fabd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcb080cdab0 0x7fcb080b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0fabd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb080ce0e0 con 0x7fcb0806e6d0 2024-05-09T06:27:27.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0fabd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcb080cdec0 con 0x7fcb080cf180 2024-05-09T06:27:27.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0fabd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcb080ce3c0 con 0x7fcb080cdab0 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d058640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcb080cf180 0x7fcb080cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d859640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcb0806e6d0 0x7fcb0806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36908/0 (socket says 172.21.15.1:36908) 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d859640 1 -- 172.21.15.1:0/4101674141 learned_addr learned my addr 172.21.15.1:0/4101674141 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0e05a640 1 --2- 172.21.15.1:0/4101674141 >> v2:172.21.15.6:3300/0 conn(0x7fcb080cdab0 0x7fcb080b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d058640 1 -- 172.21.15.1:0/4101674141 >> v2:172.21.15.6:3300/0 conn(0x7fcb080cdab0 msgr2=0x7fcb080b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d058640 1 --2- 172.21.15.1:0/4101674141 >> v2:172.21.15.6:3300/0 conn(0x7fcb080cdab0 0x7fcb080b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d058640 1 -- 172.21.15.1:0/4101674141 >> v1:172.21.15.1:6789/0 conn(0x7fcb0806e6d0 legacy=0x7fcb0806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.459+0000 7fcb0d058640 1 -- 172.21.15.1:0/4101674141 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb080ce6a0 con 0x7fcb080cf180 2024-05-09T06:27:27.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0d058640 1 --2- 172.21.15.1:0/4101674141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb080cd570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b16f0 tx=0x7fcafc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65fa2e8c8056cad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0c857640 1 -- 172.21.15.1:0/4101674141 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc037070 con 0x7fcb080cf180 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0c857640 1 -- 172.21.15.1:0/4101674141 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc032440 con 0x7fcb080cf180 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0c857640 1 -- 172.21.15.1:0/4101674141 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc032810 con 0x7fcb080cf180 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 -- 172.21.15.1:0/4101674141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 msgr2=0x7fcb080cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/4101674141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb080cd570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b16f0 tx=0x7fcafc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 -- 172.21.15.1:0/4101674141 shutdown_connections 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/4101674141 >> v2:172.21.15.6:3300/0 conn(0x7fcb080cdab0 0x7fcb080b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/4101674141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb080cd570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.460+0000 7fcb0fabd640 1 -- 172.21.15.1:0/4101674141 >> 172.21.15.1:0/4101674141 conn(0x7fcb080cc4b0 msgr2=0x7fcb080cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.461+0000 7fcb0fabd640 1 -- 172.21.15.1:0/4101674141 shutdown_connections 2024-05-09T06:27:27.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.461+0000 7fcb0fabd640 1 -- 172.21.15.1:0/4101674141 wait complete. 2024-05-09T06:27:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 Processor -- start 2024-05-09T06:27:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 -- start start 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0806e6d0 0x7fcb0815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb080cdab0 0x7fcb081f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08070ed0 con 0x7fcb080cf180 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb080cec60 con 0x7fcb0806e6d0 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.462+0000 7fcb0fabd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcb081447c0 con 0x7fcb080cdab0 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49206/0 (socket says 172.21.15.1:49206) 2024-05-09T06:27:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 -- 172.21.15.1:0/732778241 learned_addr learned my addr 172.21.15.1:0/732778241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0d859640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0806e6d0 0x7fcb0815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0d058640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb080cdab0 0x7fcb081f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 -- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb080cdab0 msgr2=0x7fcb081f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb080cdab0 0x7fcb081f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 -- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0806e6d0 msgr2=0x7fcb0815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0806e6d0 0x7fcb0815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 -- 172.21.15.1:0/732778241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0815d880 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0e05a640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8007e40 tx=0x7fcaf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8014620 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf8019070 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0815db60 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8014a20 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.463+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0815e120 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.464+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb080752b0 con 0x7fcb080cf180 2024-05-09T06:27:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.465+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcaf8014bc0 con 0x7fcb080cf180 2024-05-09T06:27:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.466+0000 7fcb067fc640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 0x7fcaf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.466+0000 7fcb0d859640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 0x7fcaf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.466+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcaf808b9d0 con 0x7fcb080cf180 2024-05-09T06:27:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.466+0000 7fcb0d859640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 0x7fcaf406e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb08071250 tx=0x7fcb00007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.470+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcaf8059950 con 0x7fcb080cf180 2024-05-09T06:27:27.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.653+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb081e5cb0 con 0x7fcb080cf180 2024-05-09T06:27:27.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.654+0000 7fcb067fc640 1 -- 172.21.15.1:0/732778241 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fcaf805dc50 con 0x7fcb080cf180 2024-05-09T06:27:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:27.636 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 msgr2=0x7fcaf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 0x7fcaf406e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb08071250 tx=0x7fcb00007e80 comp rx=0 tx=0).stop 2024-05-09T06:27:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 msgr2=0x7fcb081f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8007e40 tx=0x7fcaf800e4b0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 shutdown_connections 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcaf406be30 0x7fcaf406e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb080cf180 0x7fcb081f2630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb080cdab0 0x7fcb081f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 --2- 172.21.15.1:0/732778241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0806e6d0 0x7fcb0815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 >> 172.21.15.1:0/732778241 conn(0x7fcb080cc4b0 msgr2=0x7fcb0813de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 shutdown_connections 2024-05-09T06:27:27.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.656+0000 7fcb0fabd640 1 -- 172.21.15.1:0/732778241 wait complete. 2024-05-09T06:27:27.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.787+0000 7f1548f3b640 1 Processor -- start 2024-05-09T06:27:27.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.788+0000 7f1548f3b640 1 -- start start 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.788+0000 7f1548f3b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1544161950 0x7f15440d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.788+0000 7f1548f3b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15440b33c0 con 0x7f15440d0f80 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.788+0000 7f1548f3b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15440b3660 con 0x7f15440d4940 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.788+0000 7f1548f3b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f15440d4dd0 con 0x7f1544161950 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f154259c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1544161950 0x7f15440d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f1542d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f15440d0f80 0x7f15440cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50738/0 (socket says 172.21.15.1:50738) 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f1542d9d640 1 -- 172.21.15.1:0/1619410956 learned_addr learned my addr 172.21.15.1:0/1619410956 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:27.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f154259c640 1 -- 172.21.15.1:0/1619410956 >> v1:172.21.15.6:6789/0 conn(0x7f15440d4940 legacy=0x7f15440cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f154259c640 1 -- 172.21.15.1:0/1619410956 >> v1:172.21.15.1:6789/0 conn(0x7f15440d0f80 legacy=0x7f15440cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.789+0000 7f154259c640 1 -- 172.21.15.1:0/1619410956 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15440b3940 con 0x7f1544161950 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f154259c640 1 --2- 172.21.15.1:0/1619410956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440d2400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f153c0097a0 tx=0x7f153c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee8f3d68420a43b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f154159a640 1 -- 172.21.15.1:0/1619410956 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c037070 con 0x7f1544161950 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f154159a640 1 -- 172.21.15.1:0/1619410956 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c02ad60 con 0x7f1544161950 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f154159a640 1 -- 172.21.15.1:0/1619410956 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c032660 con 0x7f1544161950 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f1548f3b640 1 -- 172.21.15.1:0/1619410956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 msgr2=0x7f15440d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1619410956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440d2400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f153c0097a0 tx=0x7f153c02a0d0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f1548f3b640 1 -- 172.21.15.1:0/1619410956 shutdown_connections 2024-05-09T06:27:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1619410956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440d2400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.790+0000 7f1548f3b640 1 -- 172.21.15.1:0/1619410956 >> 172.21.15.1:0/1619410956 conn(0x7f15440d3e20 msgr2=0x7f15440d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.791+0000 7f1548f3b640 1 -- 172.21.15.1:0/1619410956 shutdown_connections 2024-05-09T06:27:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.791+0000 7f1548f3b640 1 -- 172.21.15.1:0/1619410956 wait complete. 2024-05-09T06:27:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.792+0000 7f1548f3b640 1 Processor -- start 2024-05-09T06:27:27.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.792+0000 7f1548f3b640 1 -- start start 2024-05-09T06:27:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15440d0f80 0x7f15440c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15440b6f80 0x7f15440cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15440b5150 con 0x7f1544161950 2024-05-09T06:27:27.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15440cf850 con 0x7f15440d0f80 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f15440b52d0 con 0x7f15440b6f80 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44076/0 (socket says 172.21.15.1:44076) 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 -- 172.21.15.1:0/1901187727 learned_addr learned my addr 172.21.15.1:0/1901187727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f154259c640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15440d0f80 0x7f15440c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1542d9d640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15440b6f80 0x7f15440cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 -- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15440b6f80 msgr2=0x7f15440cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15440b6f80 0x7f15440cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 -- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15440d0f80 msgr2=0x7f15440c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15440d0f80 0x7f15440c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 -- 172.21.15.1:0/1901187727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15440b7d80 con 0x7f1544161950 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1541d9b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c430 tx=0x7f152c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c011790 con 0x7f1544161950 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152c017070 con 0x7f1544161950 2024-05-09T06:27:27.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15440b74e0 con 0x7f1544161950 2024-05-09T06:27:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.793+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c011bb0 con 0x7f1544161950 2024-05-09T06:27:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.794+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15440b7aa0 con 0x7f1544161950 2024-05-09T06:27:27.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.795+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15440752b0 con 0x7f1544161950 2024-05-09T06:27:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.795+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f152c01c460 con 0x7f1544161950 2024-05-09T06:27:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.796+0000 7f153affd640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 0x7f152406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.796+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f152c08af40 con 0x7f1544161950 2024-05-09T06:27:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.796+0000 7f154259c640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 0x7f152406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:27.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.796+0000 7f154259c640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 0x7f152406e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f153c034000 tx=0x7f153c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:27.800+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f152c058f60 con 0x7f1544161950 2024-05-09T06:27:28.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.023+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15440d2db0 con 0x7f1544161950 2024-05-09T06:27:28.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.023+0000 7f153affd640 1 -- 172.21.15.1:0/1901187727 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5156 (secure 0 0 0) 0x7f152c05d260 con 0x7f1544161950 2024-05-09T06:27:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:28.006 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:26.036056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:28.006 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2024-05-09T06:27:28.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 msgr2=0x7f152406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 0x7f152406e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f153c034000 tx=0x7f153c02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 msgr2=0x7f15440b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c430 tx=0x7f152c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 shutdown_connections 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f152406be60 0x7f152406e320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15440b6f80 0x7f15440cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1544161950 0x7f15440b6a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 --2- 172.21.15.1:0/1901187727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15440d0f80 0x7f15440c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 >> 172.21.15.1:0/1901187727 conn(0x7f15440d3e20 msgr2=0x7f154415ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 shutdown_connections 2024-05-09T06:27:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.025+0000 7f1548f3b640 1 -- 172.21.15.1:0/1901187727 wait complete. 2024-05-09T06:27:28.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:28.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.140+0000 7f9ad2da2640 1 Processor -- start 2024-05-09T06:27:28.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.140+0000 7f9ad2da2640 1 -- start start 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.140+0000 7f9ad2da2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad2da2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9acc0d4940 0x7f9acc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad2da2640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9acc0b33c0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad2da2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc0d4dd0 con 0x7f9acc0d4940 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad2da2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0b3660 con 0x7f9acc161950 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad133f640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9acc0d0f80 0x7f9acc0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50746/0 (socket says 172.21.15.1:50746) 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad133f640 1 -- 172.21.15.1:0/3138666334 learned_addr learned my addr 172.21.15.1:0/3138666334 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acbfff640 1 --2- 172.21.15.1:0/3138666334 >> v2:172.21.15.6:3300/0 conn(0x7f9acc0d4940 0x7f9acc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9ad0b3e640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acb7fe640 1 -- 172.21.15.1:0/3138666334 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668471980 0 0) 0x7f9acc0b33c0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acb7fe640 1 -- 172.21.15.1:0/3138666334 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9abc003520 con 0x7f9acc0d0f80 2024-05-09T06:27:28.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acbfff640 1 -- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 msgr2=0x7f9acc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acbfff640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acbfff640 1 -- 172.21.15.1:0/3138666334 >> v1:172.21.15.1:6789/0 conn(0x7f9acc0d0f80 legacy=0x7f9acc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.141+0000 7f9acbfff640 1 -- 172.21.15.1:0/3138666334 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc0b3940 con 0x7f9acc0d4940 2024-05-09T06:27:28.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.142+0000 7f9acbfff640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0d4940 0x7f9acc0d2940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b3590 tx=0x7f9ab802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d586710785fff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.142+0000 7f9acb7fe640 1 -- 172.21.15.1:0/3138666334 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8037070 con 0x7f9acc0d4940 2024-05-09T06:27:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.142+0000 7f9acb7fe640 1 -- 172.21.15.1:0/3138666334 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab8032440 con 0x7f9acc0d4940 2024-05-09T06:27:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.142+0000 7f9acb7fe640 1 -- 172.21.15.1:0/3138666334 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80327f0 con 0x7f9acc0d4940 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 -- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0d4940 msgr2=0x7f9acc0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0d4940 0x7f9acc0d2940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b3590 tx=0x7f9ab802a180 comp rx=0 tx=0).stop 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 -- 172.21.15.1:0/3138666334 shutdown_connections 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0d4940 0x7f9acc0d2940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/3138666334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 -- 172.21.15.1:0/3138666334 >> 172.21.15.1:0/3138666334 conn(0x7f9acc0d3e20 msgr2=0x7f9acc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 -- 172.21.15.1:0/3138666334 shutdown_connections 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.143+0000 7f9ad2da2640 1 -- 172.21.15.1:0/3138666334 wait complete. 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.144+0000 7f9ad2da2640 1 Processor -- start 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.144+0000 7f9ad2da2640 1 -- start start 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 0x7f9acc0b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0b52d0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0b5150 con 0x7f9acc154a00 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad2da2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0b5450 con 0x7f9acc161950 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44080/0 (socket says 172.21.15.1:44080) 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 -- 172.21.15.1:0/543560818 learned_addr learned my addr 172.21.15.1:0/543560818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9acbfff640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad133f640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 0x7f9acc0b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 -- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 msgr2=0x7f9acc0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 -- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 msgr2=0x7f9acc0b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 0x7f9acc0b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad0b3e640 1 -- 172.21.15.1:0/543560818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc09f840 con 0x7f9acc0d0f80 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.145+0000 7f9ad133f640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 0x7f9acc0b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ad0b3e640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e8a0 tx=0x7f9ac000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0017070 con 0x7f9acc0d0f80 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac000c940 con 0x7f9acc0d0f80 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000cd30 con 0x7f9acc0d0f80 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc09fb20 con 0x7f9acc0d0f80 2024-05-09T06:27:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.146+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc0a00e0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.147+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc0752b0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.147+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ac001a070 con 0x7f9acc0d0f80 2024-05-09T06:27:28.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.148+0000 7f9ac97fa640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 0x7f9ab006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.148+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ac008b300 con 0x7f9acc0d0f80 2024-05-09T06:27:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.148+0000 7f9acbfff640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 0x7f9ab006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.148+0000 7f9acbfff640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 0x7f9ab006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8034000 tx=0x7f9ab802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.152+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9ac0059320 con 0x7f9acc0d0f80 2024-05-09T06:27:28.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.342+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9acc0d21e0 con 0x7f9acc0d0f80 2024-05-09T06:27:28.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.343+0000 7f9ac97fa640 1 -- 172.21.15.1:0/543560818 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f9ac005d620 con 0x7f9acc0d0f80 2024-05-09T06:27:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:28.328 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 msgr2=0x7f9ab006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 0x7f9ab006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8034000 tx=0x7f9ab802e000 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 msgr2=0x7f9acc09daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e8a0 tx=0x7f9ac000ed60 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 shutdown_connections 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ab006be30 0x7f9ab006e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc154a00 0x7f9acc0b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9acc161950 0x7f9acc0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 --2- 172.21.15.1:0/543560818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9acc0d0f80 0x7f9acc09daa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 >> 172.21.15.1:0/543560818 conn(0x7f9acc0d3e20 msgr2=0x7f9acc15e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 shutdown_connections 2024-05-09T06:27:28.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.346+0000 7f9ad2da2640 1 -- 172.21.15.1:0/543560818 wait complete. 2024-05-09T06:27:28.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:27:20.997818+0000', 'modified': '2024-05-09T06:27:26.036056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6834/278895687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6832', 'nonce': 278895687}, {'type': 'v1', 'addr': '172.21.15.6:6834', 'nonce': 278895687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-05-09T06:27:28.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:27:28.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T06:27:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 Processor -- start 2024-05-09T06:27:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 -- start start 2024-05-09T06:27:28.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2504161950 0x7f25040d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 0x7f25040d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 0x7f25040cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f25040d4dd0 con 0x7f2504161950 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f25040b33c0 con 0x7f25040d0f80 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.871+0000 7f250a7a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f25040b3660 con 0x7f25040d4940 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2504161950 0x7f25040d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2504161950 0x7f25040d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44088/0 (socket says 172.21.15.1:44088) 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 -- 172.21.15.1:0/1642385401 learned_addr learned my addr 172.21.15.1:0/1642385401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2508d46640 1 --2- 172.21.15.1:0/1642385401 >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 0x7f25040cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f24fbfff640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 0x7f25040d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 -- 172.21.15.1:0/1642385401 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 msgr2=0x7f25040d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 0x7f25040d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 -- 172.21.15.1:0/1642385401 >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 msgr2=0x7f25040cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 --2- 172.21.15.1:0/1642385401 >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 0x7f25040cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2503fff640 1 -- 172.21.15.1:0/1642385401 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040b3940 con 0x7f2504161950 2024-05-09T06:27:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f2508d46640 1 --2- 172.21.15.1:0/1642385401 >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 0x7f25040cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.872+0000 7f24fbfff640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 0x7f25040d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f2503fff640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2504161950 0x7f25040d2400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f250408c710 tx=0x7f24f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9bb93a3eccda6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f25037fe640 1 -- 172.21.15.1:0/1642385401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4037070 con 0x7f2504161950 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f25037fe640 1 -- 172.21.15.1:0/1642385401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4032440 con 0x7f2504161950 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f25037fe640 1 -- 172.21.15.1:0/1642385401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4032810 con 0x7f2504161950 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 -- 172.21.15.1:0/1642385401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2504161950 msgr2=0x7f25040d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2504161950 0x7f25040d2400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f250408c710 tx=0x7f24f402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 -- 172.21.15.1:0/1642385401 shutdown_connections 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 --2- 172.21.15.1:0/1642385401 >> v2:172.21.15.6:3300/0 conn(0x7f25040d0f80 0x7f25040cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d4940 0x7f25040d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 --2- 172.21.15.1:0/1642385401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2504161950 0x7f25040d2400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 -- 172.21.15.1:0/1642385401 >> 172.21.15.1:0/1642385401 conn(0x7f25040d3e20 msgr2=0x7f25040d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 -- 172.21.15.1:0/1642385401 shutdown_connections 2024-05-09T06:27:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.873+0000 7f250a7a9640 1 -- 172.21.15.1:0/1642385401 wait complete. 2024-05-09T06:27:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 Processor -- start 2024-05-09T06:27:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 -- start start 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 0x7f2504153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 0x7f25040b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d1d80 con 0x7f250409de60 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d1c00 con 0x7f2504161950 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.875+0000 7f250a7a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f25040d1f00 con 0x7f25040d0f80 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44100/0 (socket says 172.21.15.1:44100) 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 -- 172.21.15.1:0/3296324327 learned_addr learned my addr 172.21.15.1:0/3296324327 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2503fff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 0x7f2504153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f24fbfff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 0x7f25040b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 -- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 msgr2=0x7f2504153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 0x7f2504153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 -- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 msgr2=0x7f25040b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 0x7f25040b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 -- 172.21.15.1:0/3296324327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040b7610 con 0x7f250409de60 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2503fff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 0x7f2504153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f24fbfff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 0x7f25040b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f2508d46640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ceb0 tx=0x7f24fc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f25017fa640 1 -- 172.21.15.1:0/3296324327 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc01a070 con 0x7f250409de60 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f25017fa640 1 -- 172.21.15.1:0/3296324327 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc00d840 con 0x7f250409de60 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25040b78f0 con 0x7f250409de60 2024-05-09T06:27:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f25017fa640 1 -- 172.21.15.1:0/3296324327 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00dc30 con 0x7f250409de60 2024-05-09T06:27:28.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.876+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25040b7eb0 con 0x7f250409de60 2024-05-09T06:27:28.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.878+0000 7f25017fa640 1 -- 172.21.15.1:0/3296324327 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24fc01d070 con 0x7f250409de60 2024-05-09T06:27:28.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.878+0000 7f25017fa640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 0x7f24e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.878+0000 7f25017fa640 1 -- 172.21.15.1:0/3296324327 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24fc08ba10 con 0x7f250409de60 2024-05-09T06:27:28.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.878+0000 7f2503fff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 0x7f24e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.879+0000 7f2503fff640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 0x7f24e406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24f4034000 tx=0x7f24f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.879+0000 7f250d59a740 1 Processor -- start 2024-05-09T06:27:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.879+0000 7f250d59a740 1 -- start start 2024-05-09T06:27:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51be2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51b9a350 0x55da51be05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da51afd4d0 con 0x55da51bc4670 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55da51aa53e0 con 0x55da51b9a350 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55da51af9490 con 0x55da51be1f10 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44104/0 (socket says 172.21.15.1:44104) 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 -- 172.21.15.1:0/299243386 learned_addr learned my addr 172.21.15.1:0/299243386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 -- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 msgr2=0x55da51be2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f24fbfff640 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51be2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51be2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2508d46640 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51b9a350 0x55da51be05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 -- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51b9a350 msgr2=0x55da51be05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51b9a350 0x55da51be05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 -- 172.21.15.1:0/299243386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da518112d0 con 0x55da51bc4670 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.880+0000 7f2503fff640 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f24f4035040 tx=0x7f24f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f429a847e52c91b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f24f9ffb640 1 -- 172.21.15.1:0/299243386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4037040 con 0x55da51bc4670 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f24f9ffb640 1 -- 172.21.15.1:0/299243386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4004030 con 0x55da51bc4670 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f24f9ffb640 1 -- 172.21.15.1:0/299243386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4004420 con 0x55da51bc4670 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 -- 172.21.15.1:0/299243386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 msgr2=0x55da51bc4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f24f4035040 tx=0x7f24f402c000 comp rx=0 tx=0).stop 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 -- 172.21.15.1:0/299243386 shutdown_connections 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51b9a350 0x55da51be05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51be2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 --2- 172.21.15.1:0/299243386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bc4a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 -- 172.21.15.1:0/299243386 >> 172.21.15.1:0/299243386 conn(0x55da51b96c80 msgr2=0x55da51bdf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 -- 172.21.15.1:0/299243386 shutdown_connections 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.881+0000 7f250d59a740 1 -- 172.21.15.1:0/299243386 wait complete. 2024-05-09T06:27:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.882+0000 7f250d59a740 1 -- start start 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.882+0000 7f250d59a740 1 Processor -- start 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51bef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 0x55da51bdf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da51b9b0f0 con 0x55da51bc7a90 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55da51ab88e0 con 0x55da51be1f10 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55da51b94bb0 con 0x55da51bc4670 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 10 client.0 _ll_register_callbacks cb 0x55da51a859c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44108/0 (socket says 172.21.15.1:44108) 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 -- 172.21.15.1:0/3301132218 learned_addr learned my addr 172.21.15.1:0/3301132218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 -- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 msgr2=0x55da51bef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2503fff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51bef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f24fbfff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 0x55da51bdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51bef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 -- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 msgr2=0x55da51bdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 0x55da51bdf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 -- 172.21.15.1:0/3301132218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da518db6b0 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2503fff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51bef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f24fbfff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 0x55da51bdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f2508d46640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24fc089c00 tx=0x7f24fc059a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d038678f0d58bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f24e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc059be0 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24fc059d80 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.883+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f4004030 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f24fc071be0 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 10 client.4431 fetch_fsmap learned FSMap version 7 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55da51bd7070 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4004030 con 0x55da51bc7a90 2024-05-09T06:27:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f24fc0620d0 con 0x55da51bc7a90 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 10 client.4431 fetch_fsmap finished waiting for FSMap version 7 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 10 client.4431 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 20 client.4431 populate_metadata read hostname 'smithi001' 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51bf3250 0x55da51bf3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 4 client.4431 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 --> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da51b0ac00 con 0x55da51bf3250 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.884+0000 7f2503fff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51bf3250 0x55da51bf3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.885+0000 7f2503fff640 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51bf3250 0x55da51bf3630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe1306e52e3f12a server_cookie=db449218789835fc in_seq=0 out_seq=0 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.885+0000 7f24e2ffd640 10 client.4431 ms_handle_connect on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.885+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3301132218 <== mds.0 v2:172.21.15.6:6832/278895687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da51b0ac00 con 0x55da51bf3250 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.885+0000 7f24e2ffd640 10 client.4431 handle_command_reply: tid=0 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.885+0000 7f250d59a740 1 client.4431 shutdown 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 msgr2=0x55da51bd6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24fc089c00 tx=0x7f24fc059a40 comp rx=0 tx=0).stop 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 shutdown_connections 2024-05-09T06:27:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51bf3250 0x55da51bf3630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc7a90 0x55da51bd6530 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51be1f10 0x55da51bdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 --2- 172.21.15.1:0/3301132218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51bef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f24e2ffd640 0 client.4431 ms_handle_reset on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 >> 172.21.15.1:0/3301132218 conn(0x55da51b9a350 msgr2=0x55da51be3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 shutdown_connections 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 1 -- 172.21.15.1:0/3301132218 wait complete. 2024-05-09T06:27:28.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.886+0000 7f250d59a740 20 client.4431 trim_cache size 0 max 16384 2024-05-09T06:27:28.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.904+0000 7f250d59a740 1 Processor -- start 2024-05-09T06:27:28.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.904+0000 7f250d59a740 1 -- start start 2024-05-09T06:27:28.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51bd88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51b99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25042007c0 con 0x55da51bc4670 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25041ffb20 con 0x55da51bc7a90 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2504200170 con 0x55da51be1f10 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44120/0 (socket says 172.21.15.1:44120) 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 -- 172.21.15.1:0/2411690288 learned_addr learned my addr 172.21.15.1:0/2411690288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 -- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 msgr2=0x55da51b99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51b99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 -- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 msgr2=0x55da51bd88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51bd88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 -- 172.21.15.1:0/2411690288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da51bf39a0 con 0x55da51bc4670 2024-05-09T06:27:28.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f2503fff640 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009930 tx=0x7f24f403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6999ad8a2da841 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f24f97fa640 1 -- 172.21.15.1:0/2411690288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4037040 con 0x55da51bc4670 2024-05-09T06:27:28.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f24f97fa640 1 -- 172.21.15.1:0/2411690288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4004400 con 0x55da51bc4670 2024-05-09T06:27:28.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.905+0000 7f24f97fa640 1 -- 172.21.15.1:0/2411690288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4055120 con 0x55da51bc4670 2024-05-09T06:27:28.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- 172.21.15.1:0/2411690288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 msgr2=0x55da51bd5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009930 tx=0x7f24f403a000 comp rx=0 tx=0).stop 2024-05-09T06:27:28.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- 172.21.15.1:0/2411690288 shutdown_connections 2024-05-09T06:27:28.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51be1f10 0x55da51b99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51bd88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 --2- 172.21.15.1:0/2411690288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51bc4670 0x55da51bd5ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- 172.21.15.1:0/2411690288 >> 172.21.15.1:0/2411690288 conn(0x55da51b9a350 msgr2=0x55da51b9a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- 172.21.15.1:0/2411690288 shutdown_connections 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- 172.21.15.1:0/2411690288 wait complete. 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.906+0000 7f250d59a740 1 -- start start 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 Processor -- start 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51b950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51c49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da51bc6470 con 0x55da51be1f10 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55da51bd98b0 con 0x55da51bc7a90 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.907+0000 7f250d59a740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55da51bd8f70 con 0x55da51bc4670 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f250d59a740 10 client.0 _ll_register_callbacks cb 0x7f24fc055c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:27:28.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2503fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51b950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44136/0 (socket says 172.21.15.1:44136) 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 -- 172.21.15.1:0/3838012707 learned_addr learned my addr 172.21.15.1:0/3838012707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24fbfff640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51c49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 -- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 msgr2=0x55da51b950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51b950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 -- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 msgr2=0x55da51c49500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51c49500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 -- 172.21.15.1:0/3838012707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da51bd4eb0 con 0x55da51be1f10 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f2508d46640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24fc056300 tx=0x7f24fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377cfd2cffaff239 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0590f0 con 0x55da51be1f10 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24fc058840 con 0x55da51be1f10 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f4004400 con 0x55da51be1f10 2024-05-09T06:27:28.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f24fc058a60 con 0x55da51be1f10 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f250d59a740 10 client.4437 fetch_fsmap learned FSMap version 7 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55da51bd5a50 con 0x55da51be1f10 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.908+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4004400 con 0x55da51be1f10 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f24fc083170 con 0x55da51be1f10 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 10 client.4437 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 20 client.4437 populate_metadata read hostname 'smithi001' 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51c518f0 0x55da51bc2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 4 client.4437 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 --> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] -- command(tid 0: {"prefix": "status"}) -- 0x55da51b04f50 con 0x55da51c518f0 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f2503fff640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51c518f0 0x55da51bc2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f2503fff640 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51c518f0 0x55da51bc2f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af29691052be9122 server_cookie=58103eb6c4cb26d4 in_seq=0 out_seq=0 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f24e2ffd640 10 client.4437 ms_handle_connect on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:28.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f24e2ffd640 1 -- 172.21.15.1:0/3838012707 <== mds.0 v2:172.21.15.6:6832/278895687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x55da51b04f50 con 0x55da51c518f0 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f24e2ffd640 10 client.4437 handle_command_reply: tid=0 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.909+0000 7f250d59a740 1 client.4437 shutdown 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 msgr2=0x55da51c020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24fc056300 tx=0x7f24fc004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 shutdown_connections 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55da51c518f0 0x55da51bc2f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da51be1f10 0x55da51c020c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da51bc7a90 0x55da51c49500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 --2- 172.21.15.1:0/3838012707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55da51bc4670 0x55da51b950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f24e2ffd640 0 client.4437 ms_handle_reset on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 >> 172.21.15.1:0/3838012707 conn(0x55da51b9a350 msgr2=0x55da51c503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 shutdown_connections 2024-05-09T06:27:28.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 1 -- 172.21.15.1:0/3838012707 wait complete. 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250d59a740 20 client.4437 trim_cache size 0 max 16384 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4290, 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 7.274003853, 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 7, 2024-05-09T06:27:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 24, 2024-05-09T06:27:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 23, 2024-05-09T06:27:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 14.299775961 2024-05-09T06:27:28.899 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:27:28.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 msgr2=0x7f24e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 0x7f24e406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24f4034000 tx=0x7f24f402e000 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 msgr2=0x7f250409f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ceb0 tx=0x7f24fc00a710 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 shutdown_connections 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24e406be30 0x7f24e406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f250409de60 0x7f250409f590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2504161950 0x7f25040b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 --2- 172.21.15.1:0/3296324327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25040d0f80 0x7f2504153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 >> 172.21.15.1:0/3296324327 conn(0x7f25040d3e20 msgr2=0x7f25040b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 shutdown_connections 2024-05-09T06:27:28.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:28.910+0000 7f250a7a9640 1 -- 172.21.15.1:0/3296324327 wait complete. 2024-05-09T06:27:28.903 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:27:28.903 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:28.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:27:29.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.025+0000 7f822f212640 1 Processor -- start 2024-05-09T06:27:29.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.025+0000 7f822f212640 1 -- start start 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822f212640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f822815e340 0x7f82280cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822f212640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 0x7f82280d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822f212640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822806d070 con 0x7f82280ce4d0 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822f212640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f822806d350 con 0x7f822815e340 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822f212640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f82280cc340 con 0x7f82280cdb50 2024-05-09T06:27:29.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f8227fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f822815e340 0x7f82280cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f8227fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f822815e340 0x7f82280cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44152/0 (socket says 172.21.15.1:44152) 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f8227fff640 1 -- 172.21.15.1:0/81787697 learned_addr learned my addr 172.21.15.1:0/81787697 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.026+0000 7f822d7af640 1 --2- 172.21.15.1:0/81787697 >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 0x7f82280d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f8227fff640 1 -- 172.21.15.1:0/81787697 >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 msgr2=0x7f82280d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f8227fff640 1 --2- 172.21.15.1:0/81787697 >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 0x7f82280d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f8227fff640 1 -- 172.21.15.1:0/81787697 >> v1:172.21.15.6:6789/0 conn(0x7f82280ce4d0 legacy=0x7f82280ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f8227fff640 1 -- 172.21.15.1:0/81787697 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280cc620 con 0x7f822815e340 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f822d7af640 1 --2- 172.21.15.1:0/81787697 >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 0x7f82280d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f8227fff640 1 --2- 172.21.15.1:0/81787697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f822815e340 0x7f82280cfe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f822806e300 tx=0x7f821c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4b9d419354ac38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f82277fe640 1 -- 172.21.15.1:0/81787697 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c037070 con 0x7f822815e340 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f82277fe640 1 -- 172.21.15.1:0/81787697 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c032440 con 0x7f822815e340 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.027+0000 7f82277fe640 1 -- 172.21.15.1:0/81787697 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c032810 con 0x7f822815e340 2024-05-09T06:27:29.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 -- 172.21.15.1:0/81787697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f822815e340 msgr2=0x7f82280cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 --2- 172.21.15.1:0/81787697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f822815e340 0x7f82280cfe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f822806e300 tx=0x7f821c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 -- 172.21.15.1:0/81787697 shutdown_connections 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 --2- 172.21.15.1:0/81787697 >> v2:172.21.15.6:3300/0 conn(0x7f82280cdb50 0x7f82280d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 --2- 172.21.15.1:0/81787697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f822815e340 0x7f82280cfe90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 -- 172.21.15.1:0/81787697 >> 172.21.15.1:0/81787697 conn(0x7f82280d1c50 msgr2=0x7f82280d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 -- 172.21.15.1:0/81787697 shutdown_connections 2024-05-09T06:27:29.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.028+0000 7f822f212640 1 -- 172.21.15.1:0/81787697 wait complete. 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.029+0000 7f822f212640 1 Processor -- start 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.029+0000 7f822f212640 1 -- start start 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 0x7f8228156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 0x7f82280c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82280cefc0 con 0x7f8228157550 2024-05-09T06:27:29.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f822806cd00 con 0x7f82280ce4d0 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822f212640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82280cf140 con 0x7f82280cdb50 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822d7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822d7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44158/0 (socket says 172.21.15.1:44158) 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822d7af640 1 -- 172.21.15.1:0/2979997413 learned_addr learned my addr 172.21.15.1:0/2979997413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f822cfae640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 0x7f8228156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.030+0000 7f8227fff640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 0x7f82280c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 -- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 msgr2=0x7f8228156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 0x7f8228156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 -- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 msgr2=0x7f82280c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 0x7f82280c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 -- 172.21.15.1:0/2979997413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280c9700 con 0x7f8228157550 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822cfae640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 0x7f8228156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:29.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f8227fff640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 0x7f82280c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822d7af640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f822000cd80 tx=0x7f822000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000f680 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822000f840 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000fc30 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82280c99e0 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.031+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82280c9fa0 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.032+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82280752b0 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.033+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82200213f0 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.033+0000 7f82257fa640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 0x7f821406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.033+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f822008bf60 con 0x7f8228157550 2024-05-09T06:27:29.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.033+0000 7f822cfae640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 0x7f821406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:29.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.034+0000 7f822cfae640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 0x7f821406e320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:29.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.038+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8220014030 con 0x7f8228157550 2024-05-09T06:27:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.256+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f82280cbf80 con 0x7f8228157550 2024-05-09T06:27:29.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.257+0000 7f82257fa640 1 -- 172.21.15.1:0/2979997413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f8220059f80 con 0x7f8228157550 2024-05-09T06:27:29.239 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 msgr2=0x7f821406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 0x7f821406e320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).stop 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 msgr2=0x7f82280caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f822000cd80 tx=0x7f822000a650 comp rx=0 tx=0).stop 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 shutdown_connections 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f821406be60 0x7f821406e320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8228157550 0x7f82280caf70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82280ce4d0 0x7f82280c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 --2- 172.21.15.1:0/2979997413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82280cdb50 0x7f8228156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 >> 172.21.15.1:0/2979997413 conn(0x7f82280d1c50 msgr2=0x7f82280cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 shutdown_connections 2024-05-09T06:27:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29.259+0000 7f822f212640 1 -- 172.21.15.1:0/2979997413 wait complete. 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:27:29.252 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T06:27:29.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:29.253 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:29.269 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42693sec preferred_lft 42693sec 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:29.270 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:27:29.271 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:27:29.271 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:29.271 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:29.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:27:29.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-05-09T06:27:29.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-05-09T06:27:29.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:27:29.272 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:29.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:29.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:29.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:29.462 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:27:29.488 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:27:29.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:29.489 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:27:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:27:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:27:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:27:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:27:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:27:29.557 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:29.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:29.905 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:27:29.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:29.922 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-09T06:27:29.978 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:29.978 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:29.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:29.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T06:27:29.978 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:30.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:30.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:30.101 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T06:27:30.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T06:27:30.102 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:30.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:30.234 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:30.234 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:30.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:27:30.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:27:30.235 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:30.318 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T06:27:30.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:30.319 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:30.335 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T06:27:30.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:30.336 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:30.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:30.460 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:30.517 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:30.519 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:30.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:30.519 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:30.575 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:27:30.575 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T06:27:30.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:30.643 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:30.701 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:30.702 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:30.703 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:30.704 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:30.704 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:30.704 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:30.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:30.704 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:30.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:2024-05-09T06:27:30.727+0000 7fb1f3b29480 -1 init, newargv = 0x55d8fc83e730 newargc=15 2024-05-09T06:27:30.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[42366]: starting ceph client 2024-05-09T06:27:30.716 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[42366]: starting fuse 2024-05-09T06:27:30.726 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:27:30.726 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:27:30.727 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.admin")) 2024-05-09T06:27:30.728 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:27:30.849 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.admin.42366.asok 2024-05-09T06:27:30.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:30.859 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42366.asok 2024-05-09T06:27:30.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:30.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42366.asok status 2024-05-09T06:27:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "admin", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "42366", 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4452, 2024-05-09T06:27:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 4452 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1781548902 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1781548902 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.4452 192.168.0.1:0/1781548902", 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1781548902", 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 7, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 24, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:27:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:27:31.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:31.017 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:27:31.017 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:31.034 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:27:31.034 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:31.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:31.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:31.102 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T06:27:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:27:31.210 INFO:teuthology.orchestra.run.smithi001.stdout:# file: . 2024-05-09T06:27:31.210 INFO:teuthology.orchestra.run.smithi001.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T06:27:31.210 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:31.216 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:27:31.216 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:27:31.232 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:31.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.395+0000 7f72a731d640 1 Processor -- start 2024-05-09T06:27:31.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.395+0000 7f72a731d640 1 -- start start 2024-05-09T06:27:31.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a731d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f72a01670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a731d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a0071c20 con 0x7f72a01454b0 2024-05-09T06:27:31.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a731d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a006eb40 con 0x7f72a00b2a60 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a731d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f72a006ee20 con 0x7f72a0164ce0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a50b9640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f72a01454b0 0x7f72a00b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50760/0 (socket says 172.21.15.1:50760) 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a50b9640 1 -- 172.21.15.1:0/79171676 learned_addr learned my addr 172.21.15.1:0/79171676 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f72a58ba640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f72a01670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371269937 0 0) 0x7f72a0071c20 con 0x7f72a01454b0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f728c003520 con 0x7f72a01454b0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4000662950 0 0) 0x7f728c003520 con 0x7f72a01454b0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 msgr2=0x7f72a01670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f72a01670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 >> v1:172.21.15.6:6789/0 conn(0x7f72a00b2a60 legacy=0x7f72a00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a006d8e0 con 0x7f72a01454b0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.396+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f729c001670 con 0x7f72a01454b0 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 >> v1:172.21.15.1:6789/0 conn(0x7f72a01454b0 legacy=0x7f72a00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f728c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f728c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f728c004d40 0x7f728c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72a006ee20 con 0x7f728c0039e0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f728c00ba70 con 0x7f728c004d40 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f728c00bc70 con 0x7f72a0164ce0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f728c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a48b8640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f728c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a58ba640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f728c004d40 0x7f728c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 -- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 msgr2=0x7f728c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a0164ce0 0x7f728c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 -- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f728c004d40 msgr2=0x7f728c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f728c004d40 0x7f728c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.397+0000 7f72a50b9640 1 -- 172.21.15.1:0/79171676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c003520 con 0x7f728c0039e0 2024-05-09T06:27:31.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a50b9640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f728c003e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c210 tx=0x7f728c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4326908a251db15e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c007d10 con 0x7f728c0039e0 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729c0058d0 con 0x7f728c0039e0 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f7297fff640 1 -- 172.21.15.1:0/79171676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c005d00 con 0x7f728c0039e0 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 -- 172.21.15.1:0/79171676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 msgr2=0x7f728c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 --2- 172.21.15.1:0/79171676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f728c003e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c210 tx=0x7f728c005830 comp rx=0 tx=0).stop 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a58ba640 1 -- 172.21.15.1:0/79171676 reap_dead start 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 -- 172.21.15.1:0/79171676 shutdown_connections 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 -- 172.21.15.1:0/79171676 >> 172.21.15.1:0/79171676 conn(0x7f72a00cc4b0 msgr2=0x7f72a00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 -- 172.21.15.1:0/79171676 shutdown_connections 2024-05-09T06:27:31.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.398+0000 7f72a731d640 1 -- 172.21.15.1:0/79171676 wait complete. 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 Processor -- start 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 -- start start 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 0x7f72a013db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a013e090 0x7f72a01464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00b5140 con 0x7f728c0039e0 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0145890 con 0x7f72a015b6c0 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.400+0000 7f72a731d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f72a006e0e0 con 0x7f72a013e090 2024-05-09T06:27:31.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44184/0 (socket says 172.21.15.1:44184) 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 -- 172.21.15.1:0/470446767 learned_addr learned my addr 172.21.15.1:0/470446767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a58ba640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a013e090 0x7f72a01464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a48b8640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 0x7f72a013db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 -- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a013e090 msgr2=0x7f72a01464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a013e090 0x7f72a01464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 -- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 msgr2=0x7f72a013db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 0x7f72a013db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 -- 172.21.15.1:0/470446767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a00ae6d0 con 0x7f728c0039e0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a48b8640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 0x7f72a013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a50b9640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c210 tx=0x7f72a015c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c040020 con 0x7f728c0039e0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729c000ed0 con 0x7f728c0039e0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c007d10 con 0x7f728c0039e0 2024-05-09T06:27:31.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a0149940 con 0x7f728c0039e0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.401+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a0149f00 con 0x7f728c0039e0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.402+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a00752b0 con 0x7f728c0039e0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.403+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f729c043020 con 0x7f728c0039e0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.403+0000 7f7295ffb640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.404+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f729c0aaee0 con 0x7f728c0039e0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.404+0000 7f72a48b8640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.404+0000 7f72a48b8640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 0x7f728406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72a006e9f0 tx=0x7f7290009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.408+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f729c078f00 con 0x7f728c0039e0 2024-05-09T06:27:31.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.593+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72a0072fa0 con 0x7f728c0039e0 2024-05-09T06:27:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.594+0000 7f7295ffb640 1 -- 172.21.15.1:0/470446767 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f729c07d200 con 0x7f728c0039e0 2024-05-09T06:27:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:31.576 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 msgr2=0x7f728406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 0x7f728406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72a006e9f0 tx=0x7f7290009040 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 msgr2=0x7f72a0149010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c210 tx=0x7f72a015c370 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 shutdown_connections 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f72a013e090 0x7f72a01464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a015b6c0 0x7f72a013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 --2- 172.21.15.1:0/470446767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f728c0039e0 0x7f72a0149010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 >> 172.21.15.1:0/470446767 conn(0x7f72a00cc4b0 msgr2=0x7f72a0143180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 shutdown_connections 2024-05-09T06:27:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.596+0000 7f72a731d640 1 -- 172.21.15.1:0/470446767 wait complete. 2024-05-09T06:27:31.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:27:31.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:31.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.714+0000 7f53d5b09640 1 Processor -- start 2024-05-09T06:27:31.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 -- start start 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53d00d2870 0x7f53d01624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f53d009aa20 0x7f53d009ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d01629f0 con 0x7f53d006dfc0 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f53d009b3d0 con 0x7f53d00d2870 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53d5b09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f53d015f4c0 con 0x7f53d009aa20 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53cf7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53d00d2870 0x7f53d01624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53cf7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53d00d2870 0x7f53d01624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44194/0 (socket says 172.21.15.1:44194) 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.715+0000 7f53ceffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f53d006dfc0 0x7f53d015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50770/0 (socket says 172.21.15.1:50770) 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3754520400 learned_addr learned my addr 172.21.15.1:0/3754520400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cffff640 1 --2- 172.21.15.1:0/3754520400 >> v2:172.21.15.6:3300/0 conn(0x7f53d009aa20 0x7f53d009ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3754520400 >> v2:172.21.15.6:3300/0 conn(0x7f53d009aa20 msgr2=0x7f53d009ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cf7fe640 1 --2- 172.21.15.1:0/3754520400 >> v2:172.21.15.6:3300/0 conn(0x7f53d009aa20 0x7f53d009ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3754520400 >> v1:172.21.15.1:6789/0 conn(0x7f53d006dfc0 legacy=0x7f53d015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53ce7fc640 1 -- 172.21.15.1:0/3754520400 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067514848 0 0) 0x7f53d01629f0 con 0x7f53d006dfc0 2024-05-09T06:27:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.716+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3754520400 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d015ec50 con 0x7f53d00d2870 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53cf7fe640 1 --2- 172.21.15.1:0/3754520400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d00d2870 0x7f53d01624b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53d006cbb0 tx=0x7f53c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2690fb7b56c2553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53ce7fc640 1 -- 172.21.15.1:0/3754520400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0037070 con 0x7f53d00d2870 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53ce7fc640 1 -- 172.21.15.1:0/3754520400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c0032440 con 0x7f53d00d2870 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53ce7fc640 1 -- 172.21.15.1:0/3754520400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0032810 con 0x7f53d00d2870 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 -- 172.21.15.1:0/3754520400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d00d2870 msgr2=0x7f53d01624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3754520400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d00d2870 0x7f53d01624b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53d006cbb0 tx=0x7f53c002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 -- 172.21.15.1:0/3754520400 shutdown_connections 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3754520400 >> v2:172.21.15.6:3300/0 conn(0x7f53d009aa20 0x7f53d009ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3754520400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d00d2870 0x7f53d01624b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 -- 172.21.15.1:0/3754520400 >> 172.21.15.1:0/3754520400 conn(0x7f53d00b7020 msgr2=0x7f53d00b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 -- 172.21.15.1:0/3754520400 shutdown_connections 2024-05-09T06:27:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.717+0000 7f53d5b09640 1 -- 172.21.15.1:0/3754520400 wait complete. 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.718+0000 7f53d5b09640 1 Processor -- start 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 -- start start 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53d009aa20 0x7f53d015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53d00d2870 0x7f53d015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53d015f1e0 con 0x7f53d006dfc0 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0162020 con 0x7f53d00d2870 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.719+0000 7f53d5b09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f53d0161cb0 con 0x7f53d009aa20 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44210/0 (socket says 172.21.15.1:44210) 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3687939900 learned_addr learned my addr 172.21.15.1:0/3687939900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53ceffd640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53d009aa20 0x7f53d015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cffff640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53d00d2870 0x7f53d015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53d009aa20 msgr2=0x7f53d015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53d009aa20 0x7f53d015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53d00d2870 msgr2=0x7f53d015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53d00d2870 0x7f53d015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 -- 172.21.15.1:0/3687939900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d00cba70 con 0x7f53d006dfc0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53cf7fe640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0037040 con 0x7f53d006dfc0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c0004230 con 0x7f53d006dfc0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d00b0250 con 0x7f53d006dfc0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0003610 con 0x7f53d006dfc0 2024-05-09T06:27:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.720+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d00b0810 con 0x7f53d006dfc0 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.721+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d00752b0 con 0x7f53d006dfc0 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.722+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53c00037b0 con 0x7f53d006dfc0 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.722+0000 7f53b7fff640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 0x7f53bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.722+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f53c00abf00 con 0x7f53d006dfc0 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.722+0000 7f53ceffd640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 0x7f53bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.723+0000 7f53ceffd640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 0x7f53bc06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f53d015b060 tx=0x7f53c4007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:31.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.727+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53c0079e80 con 0x7f53d006dfc0 2024-05-09T06:27:31.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.913+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53d00ceb50 con 0x7f53d006dfc0 2024-05-09T06:27:31.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.914+0000 7f53b7fff640 1 -- 172.21.15.1:0/3687939900 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f53c007e180 con 0x7f53d006dfc0 2024-05-09T06:27:31.896 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 msgr2=0x7f53bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 0x7f53bc06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f53d015b060 tx=0x7f53c4007de0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 msgr2=0x7f53d0150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c00046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 shutdown_connections 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53bc06be30 0x7f53bc06e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53d00d2870 0x7f53d015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53d009aa20 0x7f53d015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.915+0000 7f53d5b09640 1 --2- 172.21.15.1:0/3687939900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53d006dfc0 0x7f53d0150a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:31.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.916+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 >> 172.21.15.1:0/3687939900 conn(0x7f53d00b7020 msgr2=0x7f53d00d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:31.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.916+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 shutdown_connections 2024-05-09T06:27:31.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:31.916+0000 7f53d5b09640 1 -- 172.21.15.1:0/3687939900 wait complete. 2024-05-09T06:27:31.909 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:27:32.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.033+0000 7f6d3fef5640 1 Processor -- start 2024-05-09T06:27:32.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.034+0000 7f6d3fef5640 1 -- start start 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.034+0000 7f6d3fef5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6d380b1680 0x7f6d380ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.034+0000 7f6d3fef5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3815e900 con 0x7f6d380b11f0 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.034+0000 7f6d3fef5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3815ebe0 con 0x7f6d3815e390 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.034+0000 7f6d3fef5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3815eec0 con 0x7f6d380b1680 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3d490640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6d380b11f0 0x7f6d380cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50776/0 (socket says 172.21.15.1:50776) 2024-05-09T06:27:32.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3d490640 1 -- 172.21.15.1:0/1835006329 learned_addr learned my addr 172.21.15.1:0/1835006329 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3e492640 1 --2- 172.21.15.1:0/1835006329 >> v2:172.21.15.6:3300/0 conn(0x7f6d380b1680 0x7f6d380ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445731960 0 0) 0x7f6d3815e900 con 0x7f6d380b11f0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d24003520 con 0x7f6d380b11f0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1698180826 0 0) 0x7f6d24003520 con 0x7f6d380b11f0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 >> v2:172.21.15.6:3300/0 conn(0x7f6d380b1680 msgr2=0x7f6d380ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 --2- 172.21.15.1:0/1835006329 >> v2:172.21.15.6:3300/0 conn(0x7f6d380b1680 0x7f6d380ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 >> v1:172.21.15.6:6789/0 conn(0x7f6d3815e390 legacy=0x7f6d380cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3815f1a0 con 0x7f6d380b11f0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6d28001670 con 0x7f6d380b11f0 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 >> v1:172.21.15.1:6789/0 conn(0x7f6d380b11f0 legacy=0x7f6d380cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d240039b0 0x7f6d24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d24004800 0x7f6d24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3815eec0 con 0x7f6d240051c0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2400c060 con 0x7f6d240039b0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2400c260 con 0x7f6d24004800 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3e492640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.035+0000 7f6d3dc91640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d240039b0 0x7f6d24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3d490640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d24004800 0x7f6d24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 -- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d24004800 msgr2=0x7f6d24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d24004800 0x7f6d24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 -- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d240039b0 msgr2=0x7f6d24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d240039b0 0x7f6d24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 -- 172.21.15.1:0/1835006329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d24003520 con 0x7f6d240051c0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 -- 172.21.15.1:0/1835006329 reap_dead start 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.036+0000 7f6d3e492640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d2400b9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fa0 tx=0x7f6d24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b95e3bf8ce9a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30009bf0 con 0x7f6d240051c0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d30030460 con 0x7f6d240051c0 2024-05-09T06:27:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3cc8f640 1 -- 172.21.15.1:0/1835006329 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30030830 con 0x7f6d240051c0 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 -- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 msgr2=0x7f6d2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d2400b9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fa0 tx=0x7f6d24005830 comp rx=0 tx=0).stop 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 -- 172.21.15.1:0/1835006329 shutdown_connections 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/1835006329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d2400b9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 -- 172.21.15.1:0/1835006329 >> 172.21.15.1:0/1835006329 conn(0x7f6d380b1da0 msgr2=0x7f6d380b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 -- 172.21.15.1:0/1835006329 shutdown_connections 2024-05-09T06:27:32.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.037+0000 7f6d3fef5640 1 -- 172.21.15.1:0/1835006329 wait complete. 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.038+0000 7f6d3fef5640 1 Processor -- start 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.038+0000 7f6d3fef5640 1 -- start start 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 0x7f6d380af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 0x7f6d3815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d38162ba0 con 0x7f6d240051c0 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d380b1a80 con 0x7f6d3815c9e0 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3fef5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d380cffb0 con 0x7f6d3815cf10 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3dc91640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3dc91640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44236/0 (socket says 172.21.15.1:44236) 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3dc91640 1 -- 172.21.15.1:0/2404946132 learned_addr learned my addr 172.21.15.1:0/2404946132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3e492640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 0x7f6d3815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.039+0000 7f6d3d490640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 0x7f6d380af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 -- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 msgr2=0x7f6d3815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 0x7f6d3815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 -- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 msgr2=0x7f6d380af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 0x7f6d380af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 -- 172.21.15.1:0/2404946132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d380b4e80 con 0x7f6d240051c0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3e492640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 0x7f6d3815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3d490640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 0x7f6d380af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3dc91640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400f5f0 tx=0x7f6d3400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d340052e0 con 0x7f6d240051c0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34018070 con 0x7f6d240051c0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d381e83d0 con 0x7f6d240051c0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003810 con 0x7f6d240051c0 2024-05-09T06:27:32.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.040+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d381e8990 con 0x7f6d240051c0 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.041+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d380752b0 con 0x7f6d240051c0 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.041+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d340039b0 con 0x7f6d240051c0 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.042+0000 7f6d2e7fc640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 0x7f6d1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.042+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d3408aef0 con 0x7f6d240051c0 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.042+0000 7f6d3d490640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 0x7f6d1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.042+0000 7f6d3d490640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 0x7f6d1c06e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d28007cb0 tx=0x7f6d28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.046+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d340595f0 con 0x7f6d240051c0 2024-05-09T06:27:32.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.229+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d380b2ae0 con 0x7f6d240051c0 2024-05-09T06:27:32.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.229+0000 7f6d2e7fc640 1 -- 172.21.15.1:0/2404946132 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f6d34058ef0 con 0x7f6d240051c0 2024-05-09T06:27:32.211 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 msgr2=0x7f6d1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 0x7f6d1c06e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d28007cb0 tx=0x7f6d28009040 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 msgr2=0x7f6d380b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400f5f0 tx=0x7f6d3400fab0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 shutdown_connections 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d1c06be60 0x7f6d1c06e320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d3815cf10 0x7f6d3815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d3815c9e0 0x7f6d380af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 --2- 172.21.15.1:0/2404946132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d240051c0 0x7f6d380b4940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.231+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 >> 172.21.15.1:0/2404946132 conn(0x7f6d380b1da0 msgr2=0x7f6d38160540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.232+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 shutdown_connections 2024-05-09T06:27:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.232+0000 7f6d3fef5640 1 -- 172.21.15.1:0/2404946132 wait complete. 2024-05-09T06:27:32.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:27:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.348+0000 7f25730ee640 1 Processor -- start 2024-05-09T06:27:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.348+0000 7f25730ee640 1 -- start start 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.349+0000 7f25730ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0d1c70 0x7f256c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.349+0000 7f25730ee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256c161fe0 con 0x7f256c15e190 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.349+0000 7f25730ee640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256c1622c0 con 0x7f256c0d45f0 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.349+0000 7f25730ee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f256c15f580 con 0x7f256c0d1c70 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.349+0000 7f256bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0d1c70 0x7f256c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0d1c70 0x7f256c15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44242/0 (socket says 172.21.15.1:44242) 2024-05-09T06:27:32.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f2570e8a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f256c15e190 0x7f256c0d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50790/0 (socket says 172.21.15.1:50790) 2024-05-09T06:27:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256bfff640 1 -- 172.21.15.1:0/1643021745 learned_addr learned my addr 172.21.15.1:0/1643021745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256bfff640 1 -- 172.21.15.1:0/1643021745 >> v1:172.21.15.6:6789/0 conn(0x7f256c0d45f0 legacy=0x7f256c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256bfff640 1 -- 172.21.15.1:0/1643021745 >> v1:172.21.15.1:6789/0 conn(0x7f256c15e190 legacy=0x7f256c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256bfff640 1 -- 172.21.15.1:0/1643021745 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c1625a0 con 0x7f256c0d1c70 2024-05-09T06:27:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.350+0000 7f256b7fe640 1 -- 172.21.15.1:0/1643021745 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627116636 0 0) 0x7f256c161fe0 con 0x7f256c15e190 2024-05-09T06:27:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f256bfff640 1 --2- 172.21.15.1:0/1643021745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d1c70 0x7f256c15f780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2560009870 tx=0x7f256002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47aa738a706695a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f256b7fe640 1 -- 172.21.15.1:0/1643021745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560037070 con 0x7f256c0d1c70 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f256b7fe640 1 -- 172.21.15.1:0/1643021745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256002ae50 con 0x7f256c0d1c70 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f256b7fe640 1 -- 172.21.15.1:0/1643021745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560032860 con 0x7f256c0d1c70 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 -- 172.21.15.1:0/1643021745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d1c70 msgr2=0x7f256c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 --2- 172.21.15.1:0/1643021745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d1c70 0x7f256c15f780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2560009870 tx=0x7f256002a300 comp rx=0 tx=0).stop 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 -- 172.21.15.1:0/1643021745 shutdown_connections 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 --2- 172.21.15.1:0/1643021745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d1c70 0x7f256c15f780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 -- 172.21.15.1:0/1643021745 >> 172.21.15.1:0/1643021745 conn(0x7f256c073340 msgr2=0x7f256c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 -- 172.21.15.1:0/1643021745 shutdown_connections 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.351+0000 7f25730ee640 1 -- 172.21.15.1:0/1643021745 wait complete. 2024-05-09T06:27:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.352+0000 7f25730ee640 1 Processor -- start 2024-05-09T06:27:32.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 -- start start 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 0x7f256c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f256c15e190 0x7f256c0d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0b56d0 con 0x7f256c0d45f0 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f256c161c60 con 0x7f256c0d1c70 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.353+0000 7f25730ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0b5850 con 0x7f256c15e190 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44256/0 (socket says 172.21.15.1:44256) 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 -- 172.21.15.1:0/3586492887 learned_addr learned my addr 172.21.15.1:0/3586492887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f2570e8a640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 0x7f256c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 -- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f256c15e190 msgr2=0x7f256c0d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f257168b640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f256c15e190 0x7f256c0d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f256c15e190 0x7f256c0d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 -- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 msgr2=0x7f256c151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 0x7f256c151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 -- 172.21.15.1:0/3586492887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c0d0d40 con 0x7f256c0d45f0 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f2570e8a640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 0x7f256c151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f256bfff640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2560009870 tx=0x7f25600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560041070 con 0x7f256c0d45f0 2024-05-09T06:27:32.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256003c870 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c0d1020 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560037040 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.354+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c0d15e0 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.355+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c0752b0 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.356+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2560044070 con 0x7f256c0d45f0 2024-05-09T06:27:32.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.357+0000 7f25697fa640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 0x7f255806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.357+0000 7f2570e8a640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 0x7f255806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.357+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f25600abd00 con 0x7f256c0d45f0 2024-05-09T06:27:32.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.357+0000 7f2570e8a640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 0x7f255806e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25640012e0 tx=0x7f2564009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.361+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f256007a480 con 0x7f256c0d45f0 2024-05-09T06:27:32.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.547+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f256c08e1c0 con 0x7f256c0d45f0 2024-05-09T06:27:32.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.547+0000 7f25697fa640 1 -- 172.21.15.1:0/3586492887 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2560079d80 con 0x7f256c0d45f0 2024-05-09T06:27:32.529 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 msgr2=0x7f255806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 0x7f255806e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25640012e0 tx=0x7f2564009040 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 msgr2=0x7f256c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2560009870 tx=0x7f25600046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 shutdown_connections 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f255806be60 0x7f255806e320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f256c15e190 0x7f256c0d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0d45f0 0x7f256c0cffe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 --2- 172.21.15.1:0/3586492887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f256c0d1c70 0x7f256c151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.549+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 >> 172.21.15.1:0/3586492887 conn(0x7f256c073340 msgr2=0x7f256c15e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.550+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 shutdown_connections 2024-05-09T06:27:32.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.550+0000 7f25730ee640 1 -- 172.21.15.1:0/3586492887 wait complete. 2024-05-09T06:27:32.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:27:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.667+0000 7f828c190640 1 Processor -- start 2024-05-09T06:27:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.667+0000 7f828c190640 1 -- start start 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828c190640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f828415e270 0x7f82840cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828c190640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82840ceba0 0x7f82840ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828c190640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82840b1340 con 0x7f82840d08b0 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828c190640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f82840cc260 con 0x7f828415e270 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828c190640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82840d1020 con 0x7f82840ceba0 2024-05-09T06:27:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f828415e270 0x7f82840cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f828415e270 0x7f82840cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44268/0 (socket says 172.21.15.1:44268) 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828972b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f82840d08b0 0x7f82840ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50792/0 (socket says 172.21.15.1:50792) 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 -- 172.21.15.1:0/2876756899 learned_addr learned my addr 172.21.15.1:0/2876756899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f828a72d640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82840ceba0 0x7f82840ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 -- 172.21.15.1:0/2876756899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82840ceba0 msgr2=0x7f82840ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82840ceba0 0x7f82840ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 -- 172.21.15.1:0/2876756899 >> v1:172.21.15.1:6789/0 conn(0x7f82840d08b0 legacy=0x7f82840ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8289f2c640 1 -- 172.21.15.1:0/2876756899 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82840cc4b0 con 0x7f828415e270 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.668+0000 7f8288f2a640 1 -- 172.21.15.1:0/2876756899 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2014625845 0 0) 0x7f8274001110 con 0x7f82840d08b0 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.669+0000 7f8289f2c640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f828415e270 0x7f82840cf180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f65e9b8da34dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.669+0000 7f8288f2a640 1 -- 172.21.15.1:0/2876756899 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280037070 con 0x7f828415e270 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.669+0000 7f8288f2a640 1 -- 172.21.15.1:0/2876756899 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828002ae50 con 0x7f828415e270 2024-05-09T06:27:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.669+0000 7f8288f2a640 1 -- 172.21.15.1:0/2876756899 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280032860 con 0x7f828415e270 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 -- 172.21.15.1:0/2876756899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f828415e270 msgr2=0x7f82840cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f828415e270 0x7f82840cf180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828002a300 comp rx=0 tx=0).stop 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 -- 172.21.15.1:0/2876756899 shutdown_connections 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82840ceba0 0x7f82840ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 --2- 172.21.15.1:0/2876756899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f828415e270 0x7f82840cf180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 -- 172.21.15.1:0/2876756899 >> 172.21.15.1:0/2876756899 conn(0x7f82840d1c10 msgr2=0x7f82840d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 -- 172.21.15.1:0/2876756899 shutdown_connections 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.670+0000 7f828c190640 1 -- 172.21.15.1:0/2876756899 wait complete. 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.671+0000 7f828c190640 1 Processor -- start 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.671+0000 7f828c190640 1 -- start start 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840ceba0 0x7f8284161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 0x7f82840cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82840d0d20 con 0x7f82840d08b0 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82840cca70 con 0x7f82840ceba0 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f828415e650 con 0x7f828415e270 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44280/0 (socket says 172.21.15.1:44280) 2024-05-09T06:27:32.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 -- 172.21.15.1:0/25872306 learned_addr learned my addr 172.21.15.1:0/25872306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f8289f2c640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840ceba0 0x7f8284161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828a72d640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 0x7f82840cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 -- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 msgr2=0x7f82840cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 0x7f82840cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 -- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840ceba0 msgr2=0x7f8284161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840ceba0 0x7f8284161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 -- 172.21.15.1:0/25872306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82841605e0 con 0x7f82840d08b0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828a72d640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 0x7f82840cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828972b640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f827400c430 tx=0x7f827400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740074e0 con 0x7f82840d08b0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82740047e0 con 0x7f82840d08b0 2024-05-09T06:27:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274004bd0 con 0x7f82840d08b0 2024-05-09T06:27:32.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.672+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82841608a0 con 0x7f82840d08b0 2024-05-09T06:27:32.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.673+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8284160e60 con 0x7f82840d08b0 2024-05-09T06:27:32.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.674+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82840752b0 con 0x7f82840d08b0 2024-05-09T06:27:32.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.674+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8274011070 con 0x7f82840d08b0 2024-05-09T06:27:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.675+0000 7f827a7fc640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.675+0000 7f8289f2c640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.675+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f827408ad70 con 0x7f82840d08b0 2024-05-09T06:27:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.675+0000 7f8289f2c640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 0x7f826806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8280034000 tx=0x7f828002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.679+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82840b1340 con 0x7f82840d08b0 2024-05-09T06:27:32.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.864+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82840d29e0 con 0x7f82840d08b0 2024-05-09T06:27:32.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.865+0000 7f827a7fc640 1 -- 172.21.15.1:0/25872306 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f827400e280 con 0x7f82840d08b0 2024-05-09T06:27:32.847 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:32.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 msgr2=0x7f826806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 0x7f826806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8280034000 tx=0x7f828002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:32.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 msgr2=0x7f8284151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f827400c430 tx=0x7f827400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 shutdown_connections 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f828415e270 0x7f82840cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82840d08b0 0x7f8284151c60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 --2- 172.21.15.1:0/25872306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840ceba0 0x7f8284161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 >> 172.21.15.1:0/25872306 conn(0x7f82840d1c10 msgr2=0x7f82840702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 shutdown_connections 2024-05-09T06:27:32.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.867+0000 7f828c190640 1 -- 172.21.15.1:0/25872306 wait complete. 2024-05-09T06:27:32.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:27:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.983+0000 7fdfd3fc1640 1 Processor -- start 2024-05-09T06:27:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.983+0000 7fdfd3fc1640 1 -- start start 2024-05-09T06:27:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd3fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 0x7fdfcc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd3fc1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfcc0b4f70 0x7fdfcc070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd3fc1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc06eb40 con 0x7fdfcc1454b0 2024-05-09T06:27:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd3fc1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc06ee20 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd3fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc06d8e0 con 0x7fdfcc0b2a60 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd155c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 0x7fdfcc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd255e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfcc0b4f70 0x7fdfcc070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd155c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 0x7fdfcc0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:47042/0 (socket says 172.21.15.1:47042) 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd255e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfcc0b4f70 0x7fdfcc070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:36038/0 (socket says 172.21.15.1:36038) 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.984+0000 7fdfd155c640 1 -- 172.21.15.1:0/3729869293 learned_addr learned my addr 172.21.15.1:0/3729869293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.985+0000 7fdfd255e640 1 -- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 msgr2=0x7fdfcc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.985+0000 7fdfd255e640 1 --2- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 0x7fdfcc0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.985+0000 7fdfd255e640 1 -- 172.21.15.1:0/3729869293 >> v1:172.21.15.6:6789/0 conn(0x7fdfcc1454b0 legacy=0x7fdfcc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.985+0000 7fdfd255e640 1 -- 172.21.15.1:0/3729869293 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc06dbc0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd255e640 1 --2- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc070c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009d90 tx=0x7fdfc402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96462a4558230f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd0d5b640 1 -- 172.21.15.1:0/3729869293 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007740 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd0d5b640 1 -- 172.21.15.1:0/3729869293 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40078e0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd0d5b640 1 -- 172.21.15.1:0/3729869293 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007cb0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 msgr2=0x7fdfcc070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc070c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009d90 tx=0x7fdfc402a660 comp rx=0 tx=0).stop 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3729869293 shutdown_connections 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc070c10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3729869293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc0b2a60 0x7fdfcc0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3729869293 >> 172.21.15.1:0/3729869293 conn(0x7fdfcc0cc4b0 msgr2=0x7fdfcc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3729869293 shutdown_connections 2024-05-09T06:27:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.986+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3729869293 wait complete. 2024-05-09T06:27:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 Processor -- start 2024-05-09T06:27:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 -- start start 2024-05-09T06:27:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc0ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 0x7fdfcc1e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc06e260 con 0x7fdfcc0b2a60 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc06e0e0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd3fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc071150 con 0x7fdfcc1454b0 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.988+0000 7fdfd1d5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd255e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 0x7fdfcc1e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc0ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd1d5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44294/0 (socket says 172.21.15.1:44294) 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd1d5d640 1 -- 172.21.15.1:0/3898460058 learned_addr learned my addr 172.21.15.1:0/3898460058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 -- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 msgr2=0x7fdfcc1e43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 0x7fdfcc1e43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 -- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 msgr2=0x7fdfcc141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 -- 172.21.15.1:0/3898460058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1e4910 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd1d5d640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd255e640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 0x7fdfcc1e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd155c640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc0ac900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e850 tx=0x7fdfbc00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc017070 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00c940 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00ccf0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1e4be0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.989+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1e1400 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.990+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc0752b0 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.991+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfbc01a070 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.991+0000 7fdfc27fc640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 0x7fdfb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.991+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdfbc08b370 con 0x7fdfcc0b4f70 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.991+0000 7fdfd1d5d640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 0x7fdfb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.992+0000 7fdfd1d5d640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 0x7fdfb006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001b90 tx=0x7fdfc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:32.997+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdfbc059420 con 0x7fdfcc0b4f70 2024-05-09T06:27:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.195+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfcc072fa0 con 0x7fdfcc0b4f70 2024-05-09T06:27:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.195+0000 7fdfc27fc640 1 -- 172.21.15.1:0/3898460058 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fdfbc05d720 con 0x7fdfcc0b4f70 2024-05-09T06:27:33.177 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 msgr2=0x7fdfb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 0x7fdfb006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8001b90 tx=0x7fdfc8009040 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 msgr2=0x7fdfcc0ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc0ac900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e850 tx=0x7fdfbc00ed10 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 shutdown_connections 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdfb006be30 0x7fdfb006e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdfcc1454b0 0x7fdfcc1e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfcc0b4f70 0x7fdfcc0ac900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 --2- 172.21.15.1:0/3898460058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfcc0b2a60 0x7fdfcc141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 >> 172.21.15.1:0/3898460058 conn(0x7fdfcc0cc4b0 msgr2=0x7fdfcc0b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 shutdown_connections 2024-05-09T06:27:33.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.197+0000 7fdfd3fc1640 1 -- 172.21.15.1:0/3898460058 wait complete. 2024-05-09T06:27:33.190 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:27:33.190 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:27:33.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 Processor -- start 2024-05-09T06:27:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 -- start start 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ef80d4940 0x7f1ef80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef80d2b10 0x7f1ef80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef80b33c0 con 0x7f1ef8161950 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef80d4dd0 con 0x7f1ef80d4940 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.369+0000 7f1efcbd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef80b3660 con 0x7f1ef80d2b10 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ef80d4940 0x7f1ef80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ef80d4940 0x7f1ef80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44306/0 (socket says 172.21.15.1:44306) 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 -- 172.21.15.1:0/2753999 learned_addr learned my addr 172.21.15.1:0/2753999 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1ef6d9d640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef80d2b10 0x7f1ef80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 -- 172.21.15.1:0/2753999 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef80d2b10 msgr2=0x7f1ef80b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef80d2b10 0x7f1ef80b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 -- 172.21.15.1:0/2753999 >> v1:172.21.15.6:6789/0 conn(0x7f1ef8161950 legacy=0x7f1ef80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.370+0000 7f1eeffff640 1 -- 172.21.15.1:0/2753999 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80b3940 con 0x7f1ef80d4940 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1eeffff640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d4940 0x7f1ef80b4140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009870 tx=0x7f1ee002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339fa53b1db3ac4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1ef5d9b640 1 -- 172.21.15.1:0/2753999 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0037070 con 0x7f1ef80d4940 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1ef5d9b640 1 -- 172.21.15.1:0/2753999 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee0032440 con 0x7f1ef80d4940 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1ef5d9b640 1 -- 172.21.15.1:0/2753999 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0032810 con 0x7f1ef80d4940 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 -- 172.21.15.1:0/2753999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d4940 msgr2=0x7f1ef80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d4940 0x7f1ef80b4140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009870 tx=0x7f1ee002a270 comp rx=0 tx=0).stop 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 -- 172.21.15.1:0/2753999 shutdown_connections 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef80d2b10 0x7f1ef80b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/2753999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d4940 0x7f1ef80b4140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.371+0000 7f1efcbd5640 1 -- 172.21.15.1:0/2753999 >> 172.21.15.1:0/2753999 conn(0x7f1ef80d3e20 msgr2=0x7f1ef80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.372+0000 7f1efcbd5640 1 -- 172.21.15.1:0/2753999 shutdown_connections 2024-05-09T06:27:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.372+0000 7f1efcbd5640 1 -- 172.21.15.1:0/2753999 wait complete. 2024-05-09T06:27:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.373+0000 7f1efcbd5640 1 Processor -- start 2024-05-09T06:27:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.373+0000 7f1efcbd5640 1 -- start start 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ef8161950 0x7f1ef80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef814cf90 0x7f1ef80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80d1100 con 0x7f1ef80d2b10 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80d0f80 con 0x7f1ef8161950 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef80d1c00 con 0x7f1ef814cf90 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44312/0 (socket says 172.21.15.1:44312) 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 -- 172.21.15.1:0/3223956866 learned_addr learned my addr 172.21.15.1:0/3223956866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 -- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef814cf90 msgr2=0x7f1ef80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef814cf90 0x7f1ef80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 -- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ef8161950 msgr2=0x7f1ef80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1eeffff640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ef8161950 0x7f1ef80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ef8161950 0x7f1ef80cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 -- 172.21.15.1:0/3223956866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80cdc20 con 0x7f1ef80d2b10 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1ef659c640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c370 tx=0x7f1ee800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8017070 con 0x7f1ef80d2b10 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee80052f0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef80b8fd0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8003850 con 0x7f1ef80d2b10 2024-05-09T06:27:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.374+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef80b9590 con 0x7f1ef80d2b10 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.375+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef80752b0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.376+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ee801a070 con 0x7f1ef80d2b10 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.376+0000 7f1eeeffd640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 0x7f1ed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.376+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ee808a3b0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.377+0000 7f1eeffff640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 0x7f1ed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.377+0000 7f1eeffff640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 0x7f1ed806e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0034000 tx=0x7f1ee002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.381+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ee8058ab0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.569+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef80d21e0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.571+0000 7f1eeeffd640 1 -- 172.21.15.1:0/3223956866 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f1ee80583b0 con 0x7f1ef80d2b10 2024-05-09T06:27:33.553 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:33.553 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 msgr2=0x7f1ed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 0x7f1ed806e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0034000 tx=0x7f1ee002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 msgr2=0x7f1ef80b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c370 tx=0x7f1ee800c830 comp rx=0 tx=0).stop 2024-05-09T06:27:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 shutdown_connections 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ed806be30 0x7f1ed806e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ef814cf90 0x7f1ef80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ef8161950 0x7f1ef80cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 --2- 172.21.15.1:0/3223956866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ef80d2b10 0x7f1ef80b6a10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 >> 172.21.15.1:0/3223956866 conn(0x7f1ef80d3e20 msgr2=0x7f1ef80b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 shutdown_connections 2024-05-09T06:27:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.573+0000 7f1efcbd5640 1 -- 172.21.15.1:0/3223956866 wait complete. 2024-05-09T06:27:33.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:27:33.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.693+0000 7f6f567b7640 1 Processor -- start 2024-05-09T06:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.693+0000 7f6f567b7640 1 -- start start 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f567b7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 0x7f6f500d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f567b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f500b9ae0 0x7f6f500d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f567b7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f500d2120 con 0x7f6f500b4a10 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f567b7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6f500d1e40 con 0x7f6f500b9ae0 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f567b7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6f500d2400 con 0x7f6f500b9700 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f4f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f500b9ae0 0x7f6f500d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f4f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f500b9ae0 0x7f6f500d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44316/0 (socket says 172.21.15.1:44316) 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/3340086978 learned_addr learned my addr 172.21.15.1:0/3340086978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.694+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/3340086978 >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 0x7f6f500d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/3340086978 >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 msgr2=0x7f6f500d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4f7fe640 1 --2- 172.21.15.1:0/3340086978 >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 0x7f6f500d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/3340086978 >> v1:172.21.15.6:6789/0 conn(0x7f6f500b4a10 legacy=0x7f6f500ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/3340086978 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f500d26e0 con 0x7f6f500b9ae0 2024-05-09T06:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/3340086978 >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 0x7f6f500d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4f7fe640 1 --2- 172.21.15.1:0/3340086978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9ae0 0x7f6f500d0790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f5008e220 tx=0x7f6f4402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43190362b1f7ad65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4effd640 1 -- 172.21.15.1:0/3340086978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44037070 con 0x7f6f500b9ae0 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4effd640 1 -- 172.21.15.1:0/3340086978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f44032440 con 0x7f6f500b9ae0 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.695+0000 7f6f4effd640 1 -- 172.21.15.1:0/3340086978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44032810 con 0x7f6f500b9ae0 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 -- 172.21.15.1:0/3340086978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9ae0 msgr2=0x7f6f500d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 --2- 172.21.15.1:0/3340086978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9ae0 0x7f6f500d0790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f5008e220 tx=0x7f6f4402a2c0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 -- 172.21.15.1:0/3340086978 shutdown_connections 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 --2- 172.21.15.1:0/3340086978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9ae0 0x7f6f500d0790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 --2- 172.21.15.1:0/3340086978 >> v2:172.21.15.6:3300/0 conn(0x7f6f500b9700 0x7f6f500d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 -- 172.21.15.1:0/3340086978 >> 172.21.15.1:0/3340086978 conn(0x7f6f500d3ea0 msgr2=0x7f6f500d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 -- 172.21.15.1:0/3340086978 shutdown_connections 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.696+0000 7f6f567b7640 1 -- 172.21.15.1:0/3340086978 wait complete. 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.697+0000 7f6f567b7640 1 Processor -- start 2024-05-09T06:27:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.697+0000 7f6f567b7640 1 -- start start 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 0x7f6f501f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 0x7f6f501f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f500b7d00 con 0x7f6f500b9700 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6f500cfe70 con 0x7f6f500b4a10 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f567b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6f500d2ee0 con 0x7f6f500b9ae0 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f4f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f4f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44330/0 (socket says 172.21.15.1:44330) 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/1893839212 learned_addr learned my addr 172.21.15.1:0/1893839212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.698+0000 7f6f54d54640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 0x7f6f501f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 0x7f6f501f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 msgr2=0x7f6f501f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 0x7f6f501f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 msgr2=0x7f6f501f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 0x7f6f501f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 -- 172.21.15.1:0/1893839212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f500ce0b0 con 0x7f6f500b9700 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f54d54640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 0x7f6f501f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 0x7f6f501f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4f7fe640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f4402a300 tx=0x7f6f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44037040 con 0x7f6f500b9700 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f44042070 con 0x7f6f500b9700 2024-05-09T06:27:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f500c8ae0 con 0x7f6f500b9700 2024-05-09T06:27:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44030bb0 con 0x7f6f500b9700 2024-05-09T06:27:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.699+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f500c8d80 con 0x7f6f500b9700 2024-05-09T06:27:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.700+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f500752b0 con 0x7f6f500b9700 2024-05-09T06:27:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.701+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f44047460 con 0x7f6f500b9700 2024-05-09T06:27:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.701+0000 7f6f4cff9640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 0x7f6f3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.701+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6f440abdc0 con 0x7f6f500b9700 2024-05-09T06:27:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.701+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 0x7f6f3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.702+0000 7f6f4ffff640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 0x7f6f3c06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f500ca4c0 tx=0x7f6f40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.706+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f44045360 con 0x7f6f500b9700 2024-05-09T06:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.892+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f500b78e0 con 0x7f6f500b9700 2024-05-09T06:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.892+0000 7f6f4cff9640 1 -- 172.21.15.1:0/1893839212 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f6f4407a4c0 con 0x7f6f500b9700 2024-05-09T06:27:33.874 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 msgr2=0x7f6f3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 0x7f6f3c06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f500ca4c0 tx=0x7f6f40008040 comp rx=0 tx=0).stop 2024-05-09T06:27:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 msgr2=0x7f6f500cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f4402a300 tx=0x7f6f440046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 shutdown_connections 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6f3c06be30 0x7f6f3c06e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f500b9ae0 0x7f6f501f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f500b9700 0x7f6f500cd8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 --2- 172.21.15.1:0/1893839212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500b4a10 0x7f6f501f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 >> 172.21.15.1:0/1893839212 conn(0x7f6f500d3ea0 msgr2=0x7f6f5006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 shutdown_connections 2024-05-09T06:27:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:33.894+0000 7f6f567b7640 1 -- 172.21.15.1:0/1893839212 wait complete. 2024-05-09T06:27:33.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:27:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.013+0000 7fc870384640 1 Processor -- start 2024-05-09T06:27:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.013+0000 7fc870384640 1 -- start start 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc870384640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 0x7fc86815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc870384640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8680d45f0 0x7fc8680b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc870384640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc868161fe0 con 0x7fc8680d1c70 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc870384640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc86815f580 con 0x7fc8680d45f0 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc870384640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc8681622c0 con 0x7fc86815e190 2024-05-09T06:27:33.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8680d45f0 0x7fc8680b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8680d45f0 0x7fc8680b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44334/0 (socket says 172.21.15.1:44334) 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 -- 172.21.15.1:0/3353065900 learned_addr learned my addr 172.21.15.1:0/3353065900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e120640 1 --2- 172.21.15.1:0/3353065900 >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 0x7fc86815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 -- 172.21.15.1:0/3353065900 >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 msgr2=0x7fc86815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 --2- 172.21.15.1:0/3353065900 >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 0x7fc86815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 -- 172.21.15.1:0/3353065900 >> v1:172.21.15.6:6789/0 conn(0x7fc8680d1c70 legacy=0x7fc8680d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e921640 1 -- 172.21.15.1:0/3353065900 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8681625a0 con 0x7fc8680d45f0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.014+0000 7fc86e120640 1 --2- 172.21.15.1:0/3353065900 >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 0x7fc86815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.015+0000 7fc86e921640 1 --2- 172.21.15.1:0/3353065900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc8680b3de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8680d1bf0 tx=0x7fc86002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8315ff43a5fda604 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.015+0000 7fc86d11e640 1 -- 172.21.15.1:0/3353065900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860037070 con 0x7fc8680d45f0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.015+0000 7fc86d11e640 1 -- 172.21.15.1:0/3353065900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc860032440 con 0x7fc8680d45f0 2024-05-09T06:27:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.015+0000 7fc86d11e640 1 -- 172.21.15.1:0/3353065900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860032810 con 0x7fc8680d45f0 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 -- 172.21.15.1:0/3353065900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 msgr2=0x7fc8680b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 --2- 172.21.15.1:0/3353065900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc8680b3de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8680d1bf0 tx=0x7fc86002a230 comp rx=0 tx=0).stop 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 -- 172.21.15.1:0/3353065900 shutdown_connections 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 --2- 172.21.15.1:0/3353065900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc8680b3de0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 --2- 172.21.15.1:0/3353065900 >> v2:172.21.15.6:3300/0 conn(0x7fc86815e190 0x7fc86815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 -- 172.21.15.1:0/3353065900 >> 172.21.15.1:0/3353065900 conn(0x7fc868073340 msgr2=0x7fc86809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 -- 172.21.15.1:0/3353065900 shutdown_connections 2024-05-09T06:27:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.016+0000 7fc870384640 1 -- 172.21.15.1:0/3353065900 wait complete. 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.017+0000 7fc870384640 1 Processor -- start 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.017+0000 7fc870384640 1 -- start start 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8680d1c70 0x7fc86814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc86815e190 0x7fc8680b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680b56d0 con 0x7fc8680d45f0 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc868161c60 con 0x7fc8680d1c70 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc870384640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc8680b5850 con 0x7fc86815e190 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44348/0 (socket says 172.21.15.1:44348) 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 -- 172.21.15.1:0/1381723629 learned_addr learned my addr 172.21.15.1:0/1381723629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86e120640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8680d1c70 0x7fc86814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86e921640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc86815e190 0x7fc8680b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 -- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc86815e190 msgr2=0x7fc8680b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc86815e190 0x7fc8680b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 -- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8680d1c70 msgr2=0x7fc86814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8680d1c70 0x7fc86814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.018+0000 7fc86d91f640 1 -- 172.21.15.1:0/1381723629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86814fa30 con 0x7fc8680d45f0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc86d91f640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c430 tx=0x7fc85800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858011790 con 0x7fc8680d45f0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc858017070 con 0x7fc8680d45f0 2024-05-09T06:27:34.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8680b6cc0 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858011b70 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.019+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8680c8100 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.020+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8680752b0 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.020+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc858011d10 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.021+0000 7fc85effd640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 0x7fc84c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.021+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc85808b070 con 0x7fc8680d45f0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.021+0000 7fc86e120640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 0x7fc84c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.021+0000 7fc86e120640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 0x7fc84c06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc864008ee0 tx=0x7fc864008430 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.025+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc86814fa30 con 0x7fc8680d45f0 2024-05-09T06:27:34.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.213+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc86808e1c0 con 0x7fc8680d45f0 2024-05-09T06:27:34.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.213+0000 7fc85effd640 1 -- 172.21.15.1:0/1381723629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fc858059650 con 0x7fc8680d45f0 2024-05-09T06:27:34.195 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:34.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 msgr2=0x7fc84c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 0x7fc84c06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc864008ee0 tx=0x7fc864008430 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 msgr2=0x7fc868160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c430 tx=0x7fc85800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 shutdown_connections 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc84c06be30 0x7fc84c06e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc86815e190 0x7fc8680b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8680d45f0 0x7fc868160b90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 --2- 172.21.15.1:0/1381723629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8680d1c70 0x7fc86814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 >> 172.21.15.1:0/1381723629 conn(0x7fc868073340 msgr2=0x7fc8680d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 shutdown_connections 2024-05-09T06:27:34.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.215+0000 7fc870384640 1 -- 172.21.15.1:0/1381723629 wait complete. 2024-05-09T06:27:34.208 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:27:34.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.332+0000 7f2a3fea5640 1 Processor -- start 2024-05-09T06:27:34.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.333+0000 7f2a3fea5640 1 -- start start 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.333+0000 7f2a3fea5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a3815a4d0 0x7f2a380cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.333+0000 7f2a3fea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a38070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3fea5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a380cf1b0 con 0x7f2a38071440 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3fea5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3815a8b0 con 0x7f2a3815a4d0 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3fea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3806dc10 con 0x7f2a380cdb90 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3e442640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2a38071440 0x7f2a3815dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50806/0 (socket says 172.21.15.1:50806) 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3e442640 1 -- 172.21.15.1:0/4139369771 learned_addr learned my addr 172.21.15.1:0/4139369771 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3d440640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a38070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/4139369771 >> v2:172.21.15.6:3300/0 conn(0x7f2a3815a4d0 0x7f2a380cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004278080 0 0) 0x7f2a380cf1b0 con 0x7f2a38071440 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a24003520 con 0x7f2a38071440 2024-05-09T06:27:34.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 269172887 0 0) 0x7f2a24003520 con 0x7f2a38071440 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 msgr2=0x7f2a38070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a38070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 >> v2:172.21.15.6:3300/0 conn(0x7f2a3815a4d0 msgr2=0x7f2a380cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 --2- 172.21.15.1:0/4139369771 >> v2:172.21.15.6:3300/0 conn(0x7f2a3815a4d0 0x7f2a380cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a380cf490 con 0x7f2a38071440 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f2a300019f0 con 0x7f2a38071440 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 >> v1:172.21.15.1:6789/0 conn(0x7f2a38071440 legacy=0x7f2a3815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a240039b0 0x7f2a24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a24004d40 0x7f2a240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3815a8b0 con 0x7f2a240039b0 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3806dc10 con 0x7f2a24004d40 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2400dbd0 con 0x7f2a380cdb90 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a240039b0 0x7f2a24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.334+0000 7f2a3d440640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3e442640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a24004d40 0x7f2a240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 -- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 msgr2=0x7f2a24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a380cdb90 0x7f2a24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 -- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a24004d40 msgr2=0x7f2a240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a24004d40 0x7f2a240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 -- 172.21.15.1:0/4139369771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a24003520 con 0x7f2a240039b0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3e442640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a24004d40 0x7f2a240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.335+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a240039b0 0x7f2a24003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400aa40 tx=0x7f2a24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2416b42a8cc27c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34038070 con 0x7f2a240039b0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3402f070 con 0x7f2a240039b0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3cc3f640 1 -- 172.21.15.1:0/4139369771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400b040 con 0x7f2a240039b0 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 -- 172.21.15.1:0/4139369771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a240039b0 msgr2=0x7f2a24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/4139369771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a240039b0 0x7f2a24003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400aa40 tx=0x7f2a24005830 comp rx=0 tx=0).stop 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3e442640 1 -- 172.21.15.1:0/4139369771 reap_dead start 2024-05-09T06:27:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 -- 172.21.15.1:0/4139369771 shutdown_connections 2024-05-09T06:27:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 -- 172.21.15.1:0/4139369771 >> 172.21.15.1:0/4139369771 conn(0x7f2a380cc4d0 msgr2=0x7f2a380cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 -- 172.21.15.1:0/4139369771 shutdown_connections 2024-05-09T06:27:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.336+0000 7f2a3fea5640 1 -- 172.21.15.1:0/4139369771 wait complete. 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.337+0000 7f2a3fea5640 1 Processor -- start 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.337+0000 7f2a3fea5640 1 -- start start 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a240039b0 0x7f2a381f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a3815b440 0x7f2a381ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a380b2be0 con 0x7f2a381efef0 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2a380b2a60 con 0x7f2a3815b440 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3fea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2a38070f00 con 0x7f2a240039b0 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3e442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3e442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44378/0 (socket says 172.21.15.1:44378) 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3e442640 1 -- 172.21.15.1:0/2601888301 learned_addr learned my addr 172.21.15.1:0/2601888301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a240039b0 0x7f2a381f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.338+0000 7f2a3d440640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a3815b440 0x7f2a381ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 -- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a240039b0 msgr2=0x7f2a381f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a240039b0 0x7f2a381f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 -- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a3815b440 msgr2=0x7f2a381ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a3815b440 0x7f2a381ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 -- 172.21.15.1:0/2601888301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3815c0f0 con 0x7f2a381efef0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3e442640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a3804fb80 tx=0x7f2a3000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ea70 con 0x7f2a381efef0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30018070 con 0x7f2a381efef0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3815c3d0 con 0x7f2a381efef0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ee50 con 0x7f2a381efef0 2024-05-09T06:27:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.339+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3815c990 con 0x7f2a381efef0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.340+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a380752b0 con 0x7f2a381efef0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.341+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a3001d460 con 0x7f2a381efef0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.341+0000 7f2a2e7fc640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 0x7f2a1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.341+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2a3008b3a0 con 0x7f2a381efef0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.341+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 0x7f2a1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.342+0000 7f2a3dc41640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 0x7f2a1c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a34035000 tx=0x7f2a34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.346+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a30059a10 con 0x7f2a381efef0 2024-05-09T06:27:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.530+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a380cd590 con 0x7f2a381efef0 2024-05-09T06:27:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.530+0000 7f2a2e7fc640 1 -- 172.21.15.1:0/2601888301 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2a30059310 con 0x7f2a381efef0 2024-05-09T06:27:34.512 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 msgr2=0x7f2a1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 0x7f2a1c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a34035000 tx=0x7f2a34008030 comp rx=0 tx=0).stop 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 msgr2=0x7f2a380b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a3804fb80 tx=0x7f2a3000e530 comp rx=0 tx=0).stop 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 shutdown_connections 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.532+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a1c06be60 0x7f2a1c06e320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a381efef0 0x7f2a380b3dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a3815b440 0x7f2a381ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 --2- 172.21.15.1:0/2601888301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a240039b0 0x7f2a381f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 >> 172.21.15.1:0/2601888301 conn(0x7f2a380cc4d0 msgr2=0x7f2a38153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 shutdown_connections 2024-05-09T06:27:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.533+0000 7f2a3fea5640 1 -- 172.21.15.1:0/2601888301 wait complete. 2024-05-09T06:27:34.525 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:27:34.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.650+0000 7f4836930640 1 Processor -- start 2024-05-09T06:27:34.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.650+0000 7f4836930640 1 -- start start 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4836930640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f483015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4836930640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4830072030 con 0x7f48300d1640 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4836930640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4830072310 con 0x7f4830159b90 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4836930640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4830072cb0 con 0x7f483006b8f0 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4834ecd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f48300d1640 0x7f483006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50820/0 (socket says 172.21.15.1:50820) 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f4834ecd640 1 -- 172.21.15.1:0/534163765 learned_addr learned my addr 172.21.15.1:0/534163765 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f482f7fe640 1 --2- 172.21.15.1:0/534163765 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f483015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490317935 0 0) 0x7f4830072030 con 0x7f48300d1640 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.651+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4818003520 con 0x7f48300d1640 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947775195 0 0) 0x7f4830072310 con 0x7f4830159b90 2024-05-09T06:27:34.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4830072030 con 0x7f4830159b90 2024-05-09T06:27:34.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3781590083 0 0) 0x7f4818003520 con 0x7f48300d1640 2024-05-09T06:27:34.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482f7fe640 1 -- 172.21.15.1:0/534163765 >> v1:172.21.15.6:6789/0 conn(0x7f4830159b90 legacy=0x7f483006b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482f7fe640 1 -- 172.21.15.1:0/534163765 >> v1:172.21.15.1:6789/0 conn(0x7f48300d1640 legacy=0x7f483006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482f7fe640 1 -- 172.21.15.1:0/534163765 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300725f0 con 0x7f483006b8f0 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.652+0000 7f482f7fe640 1 --2- 172.21.15.1:0/534163765 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f483015c3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48240097d0 tx=0x7f482402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634c44dfbb4bf534 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f483006b8f0 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482402ab90 con 0x7f483006b8f0 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f482effd640 1 -- 172.21.15.1:0/534163765 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280019f0 con 0x7f483006b8f0 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 -- 172.21.15.1:0/534163765 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 msgr2=0x7f483015c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 --2- 172.21.15.1:0/534163765 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f483015c3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48240097d0 tx=0x7f482402a100 comp rx=0 tx=0).stop 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 -- 172.21.15.1:0/534163765 shutdown_connections 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 --2- 172.21.15.1:0/534163765 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f483015c3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 -- 172.21.15.1:0/534163765 >> 172.21.15.1:0/534163765 conn(0x7f48300d0b50 msgr2=0x7f48300d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 -- 172.21.15.1:0/534163765 shutdown_connections 2024-05-09T06:27:34.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.653+0000 7f4836930640 1 -- 172.21.15.1:0/534163765 wait complete. 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.654+0000 7f4836930640 1 Processor -- start 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 -- start start 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f48300c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48300d1640 0x7f48300cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f483015d580 con 0x7f4830159b90 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f483006c150 con 0x7f48300d1640 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.655+0000 7f4836930640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4830071ce0 con 0x7f483006b8f0 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44394/0 (socket says 172.21.15.1:44394) 2024-05-09T06:27:34.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 -- 172.21.15.1:0/4577777 learned_addr learned my addr 172.21.15.1:0/4577777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f482ffff640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f48300c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f482f7fe640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48300d1640 0x7f48300cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 -- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 msgr2=0x7f48300c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f48300c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 -- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48300d1640 msgr2=0x7f48300cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48300d1640 0x7f48300cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 -- 172.21.15.1:0/4577777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300c7310 con 0x7f4830159b90 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4834ecd640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4828001350 tx=0x7f482800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828017070 con 0x7f4830159b90 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482800ce20 con 0x7f4830159b90 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828012640 con 0x7f4830159b90 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48300c9350 con 0x7f4830159b90 2024-05-09T06:27:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.656+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48300c9910 con 0x7f4830159b90 2024-05-09T06:27:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.657+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48300752b0 con 0x7f4830159b90 2024-05-09T06:27:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.658+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f482801a070 con 0x7f4830159b90 2024-05-09T06:27:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.658+0000 7f482cff9640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.658+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f482808b050 con 0x7f4830159b90 2024-05-09T06:27:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.658+0000 7f482ffff640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.659+0000 7f482ffff640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 0x7f481c06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48200011e0 tx=0x7f4820009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.663+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4828059060 con 0x7f4830159b90 2024-05-09T06:27:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.845+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f48300c9ae0 con 0x7f4830159b90 2024-05-09T06:27:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.846+0000 7f482cff9640 1 -- 172.21.15.1:0/4577777 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f482805d360 con 0x7f4830159b90 2024-05-09T06:27:34.828 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 msgr2=0x7f481c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 0x7f481c06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48200011e0 tx=0x7f4820009040 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 msgr2=0x7f48300c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4828001350 tx=0x7f482800a430 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 shutdown_connections 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4830159b90 0x7f48300c6bb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48300d1640 0x7f48300cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 --2- 172.21.15.1:0/4577777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f483006b8f0 0x7f48300c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 >> 172.21.15.1:0/4577777 conn(0x7f48300d0b50 msgr2=0x7f4830160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 shutdown_connections 2024-05-09T06:27:34.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.848+0000 7f4836930640 1 -- 172.21.15.1:0/4577777 wait complete. 2024-05-09T06:27:34.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:27:34.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.963+0000 7f5e4e7b7640 1 Processor -- start 2024-05-09T06:27:34.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.963+0000 7f5e4e7b7640 1 -- start start 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.964+0000 7f5e4e7b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 0x7f5e480b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.964+0000 7f5e4e7b7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e480b33c0 con 0x7f5e480d2b10 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.964+0000 7f5e4e7b7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e480b3660 con 0x7f5e48161950 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.964+0000 7f5e4e7b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5e480d4dd0 con 0x7f5e480d4940 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e4cd54640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5e480d2b10 0x7f5e480b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:50836/0 (socket says 172.21.15.1:50836) 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e4cd54640 1 -- 172.21.15.1:0/1960839024 learned_addr learned my addr 172.21.15.1:0/1960839024 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:34.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e477fe640 1 --2- 172.21.15.1:0/1960839024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 0x7f5e480b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008123141 0 0) 0x7f5e480b33c0 con 0x7f5e480d2b10 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e30003520 con 0x7f5e480d2b10 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487082914 0 0) 0x7f5e480b3660 con 0x7f5e48161950 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e477fe640 1 -- 172.21.15.1:0/1960839024 >> v1:172.21.15.6:6789/0 conn(0x7f5e48161950 legacy=0x7f5e480d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e477fe640 1 -- 172.21.15.1:0/1960839024 >> v1:172.21.15.1:6789/0 conn(0x7f5e480d2b10 legacy=0x7f5e480b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.965+0000 7f5e477fe640 1 -- 172.21.15.1:0/1960839024 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e480b3940 con 0x7f5e480d4940 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e477fe640 1 --2- 172.21.15.1:0/1960839024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 0x7f5e480b4140 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0097d0 tx=0x7f5e3c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff68f37288f9174 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e480d4940 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c032440 con 0x7f5e480d4940 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e46ffd640 1 -- 172.21.15.1:0/1960839024 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0327f0 con 0x7f5e480d4940 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/1960839024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 msgr2=0x7f5e480b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/1960839024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 0x7f5e480b4140 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0097d0 tx=0x7f5e3c02a210 comp rx=0 tx=0).stop 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/1960839024 shutdown_connections 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/1960839024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e480d4940 0x7f5e480b4140 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.966+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/1960839024 >> 172.21.15.1:0/1960839024 conn(0x7f5e480d3e20 msgr2=0x7f5e480d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:34.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.967+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/1960839024 shutdown_connections 2024-05-09T06:27:34.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.967+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/1960839024 wait complete. 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.968+0000 7f5e4e7b7640 1 Processor -- start 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.968+0000 7f5e4e7b7640 1 -- start start 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e48161950 0x7f5e4814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e480a17c0 0x7f5e48160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480d1100 con 0x7f5e480d2b10 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480d0f80 con 0x7f5e480a17c0 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4e7b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5e480d1c00 con 0x7f5e48161950 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44400/0 (socket says 172.21.15.1:44400) 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 -- 172.21.15.1:0/2556484681 learned_addr learned my addr 172.21.15.1:0/2556484681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e477fe640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e48161950 0x7f5e4814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e4cd54640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e480a17c0 0x7f5e48160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 -- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e48161950 msgr2=0x7f5e4814f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e48161950 0x7f5e4814f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 -- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e480a17c0 msgr2=0x7f5e48160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e480a17c0 0x7f5e48160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 -- 172.21.15.1:0/2556484681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4814fdc0 con 0x7f5e480d2b10 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e47fff640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800c410 tx=0x7f5e3800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38017070 con 0x7f5e480d2b10 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.969+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e380035f0 con 0x7f5e480d2b10 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.970+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e480cd060 con 0x7f5e480d2b10 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.970+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e380039e0 con 0x7f5e480d2b10 2024-05-09T06:27:34.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.970+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e480cd620 con 0x7f5e480d2b10 2024-05-09T06:27:34.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.971+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e480752b0 con 0x7f5e480d2b10 2024-05-09T06:27:34.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.971+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e3801a070 con 0x7f5e480d2b10 2024-05-09T06:27:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.971+0000 7f5e44ff9640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 0x7f5e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.971+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5e3808a4d0 con 0x7f5e480d2b10 2024-05-09T06:27:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.971+0000 7f5e477fe640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 0x7f5e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.972+0000 7f5e477fe640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 0x7f5e3406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:34.976+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e380584f0 con 0x7f5e480d2b10 2024-05-09T06:27:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.171+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e480d21e0 con 0x7f5e480d2b10 2024-05-09T06:27:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.171+0000 7f5e44ff9640 1 -- 172.21.15.1:0/2556484681 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f5e3805c7f0 con 0x7f5e480d2b10 2024-05-09T06:27:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:27:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 msgr2=0x7f5e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 0x7f5e3406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 msgr2=0x7f5e480cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800c410 tx=0x7f5e3800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 shutdown_connections 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e3406be30 0x7f5e3406e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e480a17c0 0x7f5e48160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e48161950 0x7f5e4814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 --2- 172.21.15.1:0/2556484681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e480d2b10 0x7f5e480cc980 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 >> 172.21.15.1:0/2556484681 conn(0x7f5e480d3e20 msgr2=0x7f5e480b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 shutdown_connections 2024-05-09T06:27:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:35.173+0000 7f5e4e7b7640 1 -- 172.21.15.1:0/2556484681 wait complete. 2024-05-09T06:27:35.167 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:27:35.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:35.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T06:27:35.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[42366]: fuse finished with error 0 and tester_r 0 2024-05-09T06:27:35.193 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:27:41.298 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:27:41.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:41.298 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T06:27:41.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:41.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 Processor -- start 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 -- start start 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40d4940 0x7f41c40b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c40b33c0 con 0x7f41c40d2b10 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c40b3660 con 0x7f41c4161950 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.501+0000 7f41ca89a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40d4dd0 con 0x7f41c40d4940 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c8e37640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f41c40d2b10 0x7f41c40b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38334/0 (socket says 172.21.15.1:38334) 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c8e37640 1 -- 172.21.15.1:0/453748698 learned_addr learned my addr 172.21.15.1:0/453748698 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c37fe640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40d4940 0x7f41c40b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697260827 0 0) 0x7f41c40b33c0 con 0x7f41c40d2b10 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41ac003520 con 0x7f41c40d2b10 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695368203 0 0) 0x7f41c40b3660 con 0x7f41c4161950 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c40b33c0 con 0x7f41c4161950 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2486143089 0 0) 0x7f41ac003520 con 0x7f41c40d2b10 2024-05-09T06:27:41.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40d4940 msgr2=0x7f41c40b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40d4940 0x7f41c40b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 >> v1:172.21.15.6:6789/0 conn(0x7f41c4161950 legacy=0x7f41c40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40b3940 con 0x7f41c40d2b10 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f41bc001a10 con 0x7f41c40d2b10 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 >> v1:172.21.15.1:6789/0 conn(0x7f41c40d2b10 legacy=0x7f41c40b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41ac003b70 0x7f41ac003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41ac004800 0x7f41ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41ac0051c0 0x7f41ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40d4dd0 con 0x7f41ac0051c0 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac00bf00 con 0x7f41ac004800 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f41ac00c080 con 0x7f41ac003b70 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c8e37640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41ac0051c0 0x7f41ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c3fff640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41ac003b70 0x7f41ac003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.502+0000 7f41c37fe640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41ac004800 0x7f41ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 -- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41ac003b70 msgr2=0x7f41ac003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41ac003b70 0x7f41ac003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 -- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41ac004800 msgr2=0x7f41ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41ac004800 0x7f41ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 -- 172.21.15.1:0/453748698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac003520 con 0x7f41ac0051c0 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.503+0000 7f41c8e37640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41ac0051c0 0x7f41ac00b9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41c408d0b0 tx=0x7f41ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825cd05ead76e814 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00a430 con 0x7f41ac0051c0 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc02f070 con 0x7f41ac0051c0 2024-05-09T06:27:41.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41c2ffd640 1 -- 172.21.15.1:0/453748698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc02bba0 con 0x7f41ac0051c0 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 -- 172.21.15.1:0/453748698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41ac0051c0 msgr2=0x7f41ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 --2- 172.21.15.1:0/453748698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41ac0051c0 0x7f41ac00b9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41c408d0b0 tx=0x7f41ac005830 comp rx=0 tx=0).stop 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41c8e37640 1 -- 172.21.15.1:0/453748698 reap_dead start 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 -- 172.21.15.1:0/453748698 shutdown_connections 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 -- 172.21.15.1:0/453748698 >> 172.21.15.1:0/453748698 conn(0x7f41c40d3e20 msgr2=0x7f41c40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 -- 172.21.15.1:0/453748698 shutdown_connections 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.504+0000 7f41ca89a640 1 -- 172.21.15.1:0/453748698 wait complete. 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.505+0000 7f41ca89a640 1 Processor -- start 2024-05-09T06:27:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 -- start start 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40cca10 0x7f41c40b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41c40b6260 0x7f41c40caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40d1100 con 0x7f41c415c080 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40d0f80 con 0x7f41c40b6260 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.506+0000 7f41ca89a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40d1c00 con 0x7f41c40cca10 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52972/0 (socket says 172.21.15.1:52972) 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 -- 172.21.15.1:0/794225835 learned_addr learned my addr 172.21.15.1:0/794225835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c37fe640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40cca10 0x7f41c40b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c8e37640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41c40b6260 0x7f41c40caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 -- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40cca10 msgr2=0x7f41c40b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40cca10 0x7f41c40b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 -- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41c40b6260 msgr2=0x7f41c40caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41c40b6260 0x7f41c40caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 -- 172.21.15.1:0/794225835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40cceb0 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c3fff640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41b400f770 tx=0x7f41b400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b40050d0 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b4018070 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c40cd190 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b400d730 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.507+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c40cd750 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.508+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c40752b0 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.509+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41b400d8d0 con 0x7f41c415c080 2024-05-09T06:27:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.509+0000 7f41c0ff9640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 0x7f41b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.509+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f41b408b040 con 0x7f41c415c080 2024-05-09T06:27:41.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.509+0000 7f41c37fe640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 0x7f41b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.510+0000 7f41c37fe640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 0x7f41b006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41c415ea90 tx=0x7f41b8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.514+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41b40596b0 con 0x7f41c415c080 2024-05-09T06:27:41.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.699+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41c415ef70 con 0x7f41c415c080 2024-05-09T06:27:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.701+0000 7f41c0ff9640 1 -- 172.21.15.1:0/794225835 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f41b4058fb0 con 0x7f41c415c080 2024-05-09T06:27:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:41.683 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 msgr2=0x7f41b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 0x7f41b006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41c415ea90 tx=0x7f41b8009190 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 msgr2=0x7f41c40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41b400f770 tx=0x7f41b400fb70 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 shutdown_connections 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f41b006be30 0x7f41b006e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41c40b6260 0x7f41c40caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f41c40cca10 0x7f41c40b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 --2- 172.21.15.1:0/794225835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c415c080 0x7f41c40cc4d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 >> 172.21.15.1:0/794225835 conn(0x7f41c40d3e20 msgr2=0x7f41c40b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 shutdown_connections 2024-05-09T06:27:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.703+0000 7f41ca89a640 1 -- 172.21.15.1:0/794225835 wait complete. 2024-05-09T06:27:41.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.818+0000 7fc241264640 1 Processor -- start 2024-05-09T06:27:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.818+0000 7fc241264640 1 -- start start 2024-05-09T06:27:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc23c0b0080 0x7fc23c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0cbb60 0x7fc23c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc23c15fd70 0x7fc23c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0cbfe0 con 0x7fc23c0b0080 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0d04a0 con 0x7fc23c15fd70 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc241264640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0722c0 con 0x7fc23c0cbb60 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc23c0b0080 0x7fc23c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc23c0b0080 0x7fc23c071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52986/0 (socket says 172.21.15.1:52986) 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 -- 172.21.15.1:0/1675608862 learned_addr learned my addr 172.21.15.1:0/1675608862 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 -- 172.21.15.1:0/1675608862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0cbb60 msgr2=0x7fc23c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc233fff640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0cbb60 0x7fc23c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0cbb60 0x7fc23c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 -- 172.21.15.1:0/1675608862 >> v2:172.21.15.6:3300/0 conn(0x7fc23c15fd70 msgr2=0x7fc23c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23b7fe640 1 --2- 172.21.15.1:0/1675608862 >> v2:172.21.15.6:3300/0 conn(0x7fc23c15fd70 0x7fc23c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 --2- 172.21.15.1:0/1675608862 >> v2:172.21.15.6:3300/0 conn(0x7fc23c15fd70 0x7fc23c15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.819+0000 7fc23affd640 1 -- 172.21.15.1:0/1675608862 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0725a0 con 0x7fc23c0b0080 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.820+0000 7fc23affd640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c0b0080 0x7fc23c071ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0b2690 tx=0x7fc22c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a42c955248ca38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.820+0000 7fc23a7fc640 1 -- 172.21.15.1:0/1675608862 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c037070 con 0x7fc23c0b0080 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.820+0000 7fc23a7fc640 1 -- 172.21.15.1:0/1675608862 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c032440 con 0x7fc23c0b0080 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.820+0000 7fc23a7fc640 1 -- 172.21.15.1:0/1675608862 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c032810 con 0x7fc23c0b0080 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 -- 172.21.15.1:0/1675608862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c0b0080 msgr2=0x7fc23c071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c0b0080 0x7fc23c071ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0b2690 tx=0x7fc22c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 -- 172.21.15.1:0/1675608862 shutdown_connections 2024-05-09T06:27:41.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 --2- 172.21.15.1:0/1675608862 >> v2:172.21.15.6:3300/0 conn(0x7fc23c15fd70 0x7fc23c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0cbb60 0x7fc23c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 --2- 172.21.15.1:0/1675608862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c0b0080 0x7fc23c071ce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 -- 172.21.15.1:0/1675608862 >> 172.21.15.1:0/1675608862 conn(0x7fc23c0d0b50 msgr2=0x7fc23c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 -- 172.21.15.1:0/1675608862 shutdown_connections 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.821+0000 7fc241264640 1 -- 172.21.15.1:0/1675608862 wait complete. 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.822+0000 7fc241264640 1 Processor -- start 2024-05-09T06:27:41.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.822+0000 7fc241264640 1 -- start start 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0b0080 0x7fc23c1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 0x7fc23c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c15e7c0 con 0x7fc23c15fd70 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c072b60 con 0x7fc23c0cbb60 2024-05-09T06:27:41.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc241264640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c15f220 con 0x7fc23c0b0080 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53000/0 (socket says 172.21.15.1:53000) 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 -- 172.21.15.1:0/3817253129 learned_addr learned my addr 172.21.15.1:0/3817253129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23affd640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0b0080 0x7fc23c1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc233fff640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 0x7fc23c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 -- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0b0080 msgr2=0x7fc23c1f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0b0080 0x7fc23c1f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 -- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 msgr2=0x7fc23c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 0x7fc23c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc23b7fe640 1 -- 172.21.15.1:0/3817253129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0b46d0 con 0x7fc23c15fd70 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.823+0000 7fc233fff640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 0x7fc23c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc23b7fe640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc23400cc00 tx=0x7fc23400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400ce10 con 0x7fc23c15fd70 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc234003990 con 0x7fc23c15fd70 2024-05-09T06:27:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234003d60 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c0c5970 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.824+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c0c50d0 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.825+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0752b0 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.825+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc234003f00 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.826+0000 7fc2337fe640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 0x7fc21c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.826+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc23408b600 con 0x7fc23c15fd70 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.826+0000 7fc23affd640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 0x7fc21c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:41.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.826+0000 7fc23affd640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 0x7fc21c06e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc22c034000 tx=0x7fc22c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:41.830+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc23c0b46d0 con 0x7fc23c15fd70 2024-05-09T06:27:41.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.015+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc23c0c78e0 con 0x7fc23c15fd70 2024-05-09T06:27:41.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.016+0000 7fc2337fe640 1 -- 172.21.15.1:0/3817253129 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fc234014030 con 0x7fc23c15fd70 2024-05-09T06:27:41.999 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:41.999 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 msgr2=0x7fc21c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 0x7fc21c06e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc22c034000 tx=0x7fc22c02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 msgr2=0x7fc23c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc23400cc00 tx=0x7fc23400a5c0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 shutdown_connections 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc21c06be60 0x7fc21c06e320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc23c15fd70 0x7fc23c15d040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c0cbb60 0x7fc23c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.018+0000 7fc241264640 1 --2- 172.21.15.1:0/3817253129 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc23c0b0080 0x7fc23c1f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.019+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 >> 172.21.15.1:0/3817253129 conn(0x7fc23c0d0b50 msgr2=0x7fc23c0b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.019+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 shutdown_connections 2024-05-09T06:27:42.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.019+0000 7fc241264640 1 -- 172.21.15.1:0/3817253129 wait complete. 2024-05-09T06:27:42.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:27:42.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.136+0000 7fd360bd5640 1 Processor -- start 2024-05-09T06:27:42.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.136+0000 7fd360bd5640 1 -- start start 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd360bd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd35c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd360bd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd35c0d1c70 0x7fd35c0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd360bd5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c161fe0 con 0x7fd35c0b56d0 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd360bd5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd35c15f580 con 0x7fd35c0d1c70 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd360bd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c1622c0 con 0x7fd35c15e190 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.137+0000 7fd35ad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd35c0b56d0 0x7fd35c0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38350/0 (socket says 172.21.15.1:38350) 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35ad9d640 1 -- 172.21.15.1:0/1357027078 learned_addr learned my addr 172.21.15.1:0/1357027078 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:42.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd359d9b640 1 --2- 172.21.15.1:0/1357027078 >> v2:172.21.15.6:3300/0 conn(0x7fd35c0d1c70 0x7fd35c0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35a59c640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd35c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292220869 0 0) 0x7fd35c161fe0 con 0x7fd35c0b56d0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd348003520 con 0x7fd35c0b56d0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 827355921 0 0) 0x7fd348003520 con 0x7fd35c0b56d0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 msgr2=0x7fd35c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd35c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 >> v2:172.21.15.6:3300/0 conn(0x7fd35c0d1c70 msgr2=0x7fd35c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 --2- 172.21.15.1:0/1357027078 >> v2:172.21.15.6:3300/0 conn(0x7fd35c0d1c70 0x7fd35c0d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1625a0 con 0x7fd35c0b56d0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fd34c0019f0 con 0x7fd35c0b56d0 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35a59c640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd35c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 >> v1:172.21.15.1:6789/0 conn(0x7fd35c0b56d0 legacy=0x7fd35c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd3480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd348003ef0 0x7fd348004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348004d40 0x7fd3480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1622c0 con 0x7fd348004d40 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd34800b9c0 con 0x7fd348003ef0 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd34800bb40 con 0x7fd35c15e190 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348004d40 0x7fd3480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd35a59c640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd3480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.138+0000 7fd359d9b640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd348003ef0 0x7fd348004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 -- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 msgr2=0x7fd3480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd35c15e190 0x7fd3480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 -- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd348003ef0 msgr2=0x7fd348004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd348003ef0 0x7fd348004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 -- 172.21.15.1:0/1357027078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd348003520 con 0x7fd348004d40 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348004d40 0x7fd3480051c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00a8f0 tx=0x7fd348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd0133c4bfe3955 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00a4c0 con 0x7fd348004d40 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c02f070 con 0x7fd348004d40 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.139+0000 7fd35959a640 1 -- 172.21.15.1:0/1357027078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c02bd20 con 0x7fd348004d40 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 -- 172.21.15.1:0/1357027078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348004d40 msgr2=0x7fd3480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 --2- 172.21.15.1:0/1357027078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348004d40 0x7fd3480051c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00a8f0 tx=0x7fd348005830 comp rx=0 tx=0).stop 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd35ad9d640 1 -- 172.21.15.1:0/1357027078 reap_dead start 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 -- 172.21.15.1:0/1357027078 shutdown_connections 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 -- 172.21.15.1:0/1357027078 >> 172.21.15.1:0/1357027078 conn(0x7fd35c073340 msgr2=0x7fd35c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 -- 172.21.15.1:0/1357027078 shutdown_connections 2024-05-09T06:27:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.140+0000 7fd360bd5640 1 -- 172.21.15.1:0/1357027078 wait complete. 2024-05-09T06:27:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 Processor -- start 2024-05-09T06:27:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 -- start start 2024-05-09T06:27:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd348004d40 0x7fd35c1f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd35c1f4f60 0x7fd35c0d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c15f780 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c161c60 con 0x7fd35c1f4f60 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd360bd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c15f900 con 0x7fd348004d40 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd35ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd35ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53012/0 (socket says 172.21.15.1:53012) 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.142+0000 7fd35ad9d640 1 -- 172.21.15.1:0/3109684570 learned_addr learned my addr 172.21.15.1:0/3109684570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35a59c640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd348004d40 0x7fd35c1f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd359d9b640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd35c1f4f60 0x7fd35c0d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 -- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd348004d40 msgr2=0x7fd35c1f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd348004d40 0x7fd35c1f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 -- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd35c1f4f60 msgr2=0x7fd35c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd35c1f4f60 0x7fd35c0d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 -- 172.21.15.1:0/3109684570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1f80c0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd35ad9d640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00aa10 tx=0x7fd35c0d4dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c044070 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c034470 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1f83a0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00a4c0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.143+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1f8960 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.144+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c06c6d0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.145+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd34c047070 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.145+0000 7fd352ffd640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 0x7fd33c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.145+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd34c0ab1f0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.145+0000 7fd35a59c640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 0x7fd33c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.146+0000 7fd35a59c640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 0x7fd33c06e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd354009bb0 tx=0x7fd354009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.150+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd35c06c6d0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.370+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd35c0c77d0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.756+0000 7fd352ffd640 1 -- 172.21.15.1:0/3109684570 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fd35c0c77d0 con 0x7fd35c0c7bd0 2024-05-09T06:27:42.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 msgr2=0x7fd33c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 0x7fd33c06e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd354009bb0 tx=0x7fd354009190 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 msgr2=0x7fd35c0c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00aa10 tx=0x7fd35c0d4dc0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 shutdown_connections 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd33c06be30 0x7fd33c06e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35c0c7bd0 0x7fd35c0c8050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd35c1f4f60 0x7fd35c0d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 --2- 172.21.15.1:0/3109684570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd348004d40 0x7fd35c1f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 >> 172.21.15.1:0/3109684570 conn(0x7fd35c073340 msgr2=0x7fd35c0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 shutdown_connections 2024-05-09T06:27:42.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.758+0000 7fd360bd5640 1 -- 172.21.15.1:0/3109684570 wait complete. 2024-05-09T06:27:42.752 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-09T06:27:42.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:27:42.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.880+0000 7f8655c7d640 1 Processor -- start 2024-05-09T06:27:42.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.880+0000 7f8655c7d640 1 -- start start 2024-05-09T06:27:42.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f8655c7d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f865006b1e0 0x7f86500cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f8655c7d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8650072050 con 0x7f86500d06f0 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f8655c7d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8650072330 con 0x7f86500c41e0 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f8655c7d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8650072610 con 0x7f865006b1e0 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f865006b1e0 0x7f86500cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f86500d06f0 0x7f86500cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38364/0 (socket says 172.21.15.1:38364) 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864f7fe640 1 -- 172.21.15.1:0/3374327400 learned_addr learned my addr 172.21.15.1:0/3374327400 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864ffff640 1 -- 172.21.15.1:0/3374327400 >> v1:172.21.15.6:6789/0 conn(0x7f86500c41e0 legacy=0x7f86500ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864effd640 1 -- 172.21.15.1:0/3374327400 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810599533 0 0) 0x7f8650072050 con 0x7f86500d06f0 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864ffff640 1 -- 172.21.15.1:0/3374327400 >> v1:172.21.15.1:6789/0 conn(0x7f86500d06f0 legacy=0x7f86500cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.881+0000 7f864ffff640 1 -- 172.21.15.1:0/3374327400 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86500728f0 con 0x7f865006b1e0 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.882+0000 7f864ffff640 1 --2- 172.21.15.1:0/3374327400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f865006b1e0 0x7f86500cefe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8648009d90 tx=0x7f864802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f80a801912b0f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.882+0000 7f864effd640 1 -- 172.21.15.1:0/3374327400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648007740 con 0x7f865006b1e0 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.882+0000 7f864effd640 1 -- 172.21.15.1:0/3374327400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86480078e0 con 0x7f865006b1e0 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.882+0000 7f864effd640 1 -- 172.21.15.1:0/3374327400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648007cb0 con 0x7f865006b1e0 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 -- 172.21.15.1:0/3374327400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f865006b1e0 msgr2=0x7f86500cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 --2- 172.21.15.1:0/3374327400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f865006b1e0 0x7f86500cefe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8648009d90 tx=0x7f864802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 -- 172.21.15.1:0/3374327400 shutdown_connections 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 --2- 172.21.15.1:0/3374327400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f865006b1e0 0x7f86500cefe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 -- 172.21.15.1:0/3374327400 >> 172.21.15.1:0/3374327400 conn(0x7f86500d0b50 msgr2=0x7f86500d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:42.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 -- 172.21.15.1:0/3374327400 shutdown_connections 2024-05-09T06:27:42.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.883+0000 7f8655c7d640 1 -- 172.21.15.1:0/3374327400 wait complete. 2024-05-09T06:27:42.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.884+0000 7f8655c7d640 1 Processor -- start 2024-05-09T06:27:42.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.884+0000 7f8655c7d640 1 -- start start 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f865006b1e0 0x7f86501588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86500d06f0 0x7f8650159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86500ccf70 con 0x7f86500c41e0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86500ccdf0 con 0x7f86500d06f0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8655c7d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f86500cd0f0 con 0x7f865006b1e0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53042/0 (socket says 172.21.15.1:53042) 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 -- 172.21.15.1:0/4202346804 learned_addr learned my addr 172.21.15.1:0/4202346804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f864ffff640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86500d06f0 0x7f8650159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f864f7fe640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f865006b1e0 0x7f86501588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 -- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f865006b1e0 msgr2=0x7f86501588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f865006b1e0 0x7f86501588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 -- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86500d06f0 msgr2=0x7f8650159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86500d06f0 0x7f8650159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 -- 172.21.15.1:0/4202346804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8650159c60 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.885+0000 7f8647fff640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f863800c450 tx=0x7f863800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.886+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86380052d0 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.886+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8638005470 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.886+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501552e0 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.886+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638012870 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.886+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501f81b0 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.887+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86500752b0 con 0x7f86500c41e0 2024-05-09T06:27:42.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.887+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8638012a10 con 0x7f86500c41e0 2024-05-09T06:27:42.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.887+0000 7f864cff9640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 0x7f863006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:42.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.888+0000 7f864f7fe640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 0x7f863006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:42.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.888+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f863808adc0 con 0x7f86500c41e0 2024-05-09T06:27:42.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.888+0000 7f864f7fe640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 0x7f863006e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8640000f60 tx=0x7f8640009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:42.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:42.892+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8638059430 con 0x7f86500c41e0 2024-05-09T06:27:43.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.116+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f86501554b0 con 0x7f86500c41e0 2024-05-09T06:27:43.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.760+0000 7f864cff9640 1 -- 172.21.15.1:0/4202346804 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f8638058d30 con 0x7f86500c41e0 2024-05-09T06:27:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 msgr2=0x7f863006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 0x7f863006e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8640000f60 tx=0x7f8640009040 comp rx=0 tx=0).stop 2024-05-09T06:27:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 msgr2=0x7f86500c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f863800c450 tx=0x7f863800c910 comp rx=0 tx=0).stop 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 shutdown_connections 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f863006be30 0x7f863006e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86500d06f0 0x7f8650159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86500c41e0 0x7f86500c4fd0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f863800c450 tx=0x7f863800c910 comp rx=0 tx=0).stop 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 --2- 172.21.15.1:0/4202346804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f865006b1e0 0x7f86501588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 >> 172.21.15.1:0/4202346804 conn(0x7f86500d0b50 msgr2=0x7f86500ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 shutdown_connections 2024-05-09T06:27:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.763+0000 7f8655c7d640 1 -- 172.21.15.1:0/4202346804 wait complete. 2024-05-09T06:27:43.758 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:27:43.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:43.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.885+0000 7f9fd1d53640 1 Processor -- start 2024-05-09T06:27:43.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.886+0000 7f9fd1d53640 1 -- start start 2024-05-09T06:27:43.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.886+0000 7f9fd1d53640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9fcc0d4940 0x7f9fcc0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.886+0000 7f9fd1d53640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc0b33c0 con 0x7f9fcc161950 2024-05-09T06:27:43.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.886+0000 7f9fd1d53640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc0b3660 con 0x7f9fcc0d2b10 2024-05-09T06:27:43.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.886+0000 7f9fd1d53640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9fcc0d4dd0 con 0x7f9fcc0d4940 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcb7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9fcc161950 0x7f9fcc0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38372/0 (socket says 172.21.15.1:38372) 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/466231620 learned_addr learned my addr 172.21.15.1:0/466231620 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/466231620 >> v2:172.21.15.6:3300/0 conn(0x7f9fcc0d4940 0x7f9fcc0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3482733564 0 0) 0x7f9fcc0b33c0 con 0x7f9fcc161950 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb4003520 con 0x7f9fcc161950 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153089918 0 0) 0x7f9fcc0b3660 con 0x7f9fcc0d2b10 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fcc0b33c0 con 0x7f9fcc0d2b10 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2826771896 0 0) 0x7f9fb4003520 con 0x7f9fcc161950 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 >> v2:172.21.15.6:3300/0 conn(0x7f9fcc0d4940 msgr2=0x7f9fcc0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 --2- 172.21.15.1:0/466231620 >> v2:172.21.15.6:3300/0 conn(0x7f9fcc0d4940 0x7f9fcc0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 >> v1:172.21.15.6:6789/0 conn(0x7f9fcc0d2b10 legacy=0x7f9fcc0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc0b3940 con 0x7f9fcc161950 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f9fbc001670 con 0x7f9fcc161950 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/466231620 >> v2:172.21.15.6:3300/0 conn(0x7f9fcc0d4940 0x7f9fcc0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 >> v1:172.21.15.1:6789/0 conn(0x7f9fcc161950 legacy=0x7f9fcc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 0x7f9fb4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fb4004800 0x7f9fb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb40051c0 0x7f9fb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc0d4dd0 con 0x7f9fb40051c0 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400c040 con 0x7f9fb4003b40 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb400c240 con 0x7f9fb4004800 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 0x7f9fb4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcbfff640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb40051c0 0x7f9fb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.887+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fb4004800 0x7f9fb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fb4004800 msgr2=0x7f9fb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fb4004800 0x7f9fb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/466231620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb40051c0 msgr2=0x7f9fb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb40051c0 0x7f9fb400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/466231620 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb4003520 con 0x7f9fb4003b40 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcbfff640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb40051c0 0x7f9fb400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcbfff640 1 -- 172.21.15.1:0/466231620 reap_dead start 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.888+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 0x7f9fb4003f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c160 tx=0x7f9fb4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28c3812dd39e4cc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc007ca0 con 0x7f9fb4003b40 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc005ad0 con 0x7f9fb4003b40 2024-05-09T06:27:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fca7fc640 1 -- 172.21.15.1:0/466231620 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc007070 con 0x7f9fb4003b40 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 -- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 msgr2=0x7f9fb4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 0x7f9fb4003f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c160 tx=0x7f9fb4005830 comp rx=0 tx=0).stop 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 -- 172.21.15.1:0/466231620 shutdown_connections 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/466231620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4003b40 0x7f9fb4003f40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 -- 172.21.15.1:0/466231620 >> 172.21.15.1:0/466231620 conn(0x7f9fcc0d3e20 msgr2=0x7f9fcc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 -- 172.21.15.1:0/466231620 shutdown_connections 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.889+0000 7f9fd1d53640 1 -- 172.21.15.1:0/466231620 wait complete. 2024-05-09T06:27:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.891+0000 7f9fd1d53640 1 Processor -- start 2024-05-09T06:27:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.891+0000 7f9fd1d53640 1 -- start start 2024-05-09T06:27:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.891+0000 7f9fd1d53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.891+0000 7f9fd1d53640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 0x7f9fcc15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fcc15a930 0x7f9fcc06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc0d1100 con 0x7f9fb4003b40 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc0d0f80 con 0x7f9fcc159310 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc0d1c00 con 0x7f9fcc15a930 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53060/0 (socket says 172.21.15.1:53060) 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/1486073677 learned_addr learned my addr 172.21.15.1:0/1486073677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcbfff640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fcc15a930 0x7f9fcc06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 0x7f9fcc15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fcc15a930 msgr2=0x7f9fcc06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fcc15a930 0x7f9fcc06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 msgr2=0x7f9fcc15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 0x7f9fcc15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 -- 172.21.15.1:0/1486073677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc1f73e0 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 0x7f9fcc15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fcb7fe640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c280 tx=0x7f9fcc0b3b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc037040 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc041070 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc1f76c0 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0047e0 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.892+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1f7c80 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.894+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc0752b0 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.894+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fbc004980 con 0x7f9fb4003b40 2024-05-09T06:27:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.894+0000 7f9fb3fff640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 0x7f9fb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.894+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9fbc0aab20 con 0x7f9fb4003b40 2024-05-09T06:27:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.894+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 0x7f9fb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.895+0000 7f9fcaffd640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 0x7f9fb806e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ccfb0 tx=0x7f9fc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:43.899+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fbc030380 con 0x7f9fb4003b40 2024-05-09T06:27:44.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.118+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fcc0d21b0 con 0x7f9fb4003b40 2024-05-09T06:27:44.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.118+0000 7f9fb3fff640 1 -- 172.21.15.1:0/1486073677 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f9fbc079190 con 0x7f9fb4003b40 2024-05-09T06:27:44.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:44.101 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:44.101 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 msgr2=0x7f9fb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 0x7f9fb806e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ccfb0 tx=0x7f9fc0008040 comp rx=0 tx=0).stop 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 msgr2=0x7f9fcc0cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c280 tx=0x7f9fcc0b3b10 comp rx=0 tx=0).stop 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 shutdown_connections 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fb806be30 0x7f9fb806e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fcc15a930 0x7f9fcc06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fcc159310 0x7f9fcc15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 --2- 172.21.15.1:0/1486073677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fb4003b40 0x7f9fcc0cb900 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.120+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 >> 172.21.15.1:0/1486073677 conn(0x7f9fcc0d3e20 msgr2=0x7f9fcc06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.121+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 shutdown_connections 2024-05-09T06:27:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.121+0000 7f9fd1d53640 1 -- 172.21.15.1:0/1486073677 wait complete. 2024-05-09T06:27:44.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:44.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.236+0000 7fd52177e640 1 Processor -- start 2024-05-09T06:27:44.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.236+0000 7fd52177e640 1 -- start start 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd52177e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd51c15c320 0x7fd51c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd52177e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c0b0d00 0x7fd51c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd52177e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c0d2c80 con 0x7fd51c0d05e0 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd52177e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c0d29a0 con 0x7fd51c15c320 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd52177e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c0cd430 con 0x7fd51c0b0d00 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd51c15c320 0x7fd51c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd51c15c320 0x7fd51c0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53076/0 (socket says 172.21.15.1:53076) 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 -- 172.21.15.1:0/2970214458 learned_addr learned my addr 172.21.15.1:0/2970214458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51a7fc640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c0b0d00 0x7fd51c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 -- 172.21.15.1:0/2970214458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c0b0d00 msgr2=0x7fd51c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c0b0d00 0x7fd51c0d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 -- 172.21.15.1:0/2970214458 >> v1:172.21.15.6:6789/0 conn(0x7fd51c0d05e0 legacy=0x7fd51c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.237+0000 7fd51affd640 1 -- 172.21.15.1:0/2970214458 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0cd710 con 0x7fd51c15c320 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd51affd640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c15c320 0x7fd51c0b1250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009870 tx=0x7fd50c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03fb7f72d91e816 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd519ffb640 1 -- 172.21.15.1:0/2970214458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c037070 con 0x7fd51c15c320 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd519ffb640 1 -- 172.21.15.1:0/2970214458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50c02ae50 con 0x7fd51c15c320 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd519ffb640 1 -- 172.21.15.1:0/2970214458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c032860 con 0x7fd51c15c320 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd52177e640 1 -- 172.21.15.1:0/2970214458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c15c320 msgr2=0x7fd51c0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.238+0000 7fd52177e640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c15c320 0x7fd51c0b1250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009870 tx=0x7fd50c02a300 comp rx=0 tx=0).stop 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 -- 172.21.15.1:0/2970214458 shutdown_connections 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c0b0d00 0x7fd51c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 --2- 172.21.15.1:0/2970214458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c15c320 0x7fd51c0b1250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 -- 172.21.15.1:0/2970214458 >> 172.21.15.1:0/2970214458 conn(0x7fd51c0d1d70 msgr2=0x7fd51c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 -- 172.21.15.1:0/2970214458 shutdown_connections 2024-05-09T06:27:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.239+0000 7fd52177e640 1 -- 172.21.15.1:0/2970214458 wait complete. 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.240+0000 7fd52177e640 1 Processor -- start 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.240+0000 7fd52177e640 1 -- start start 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd51c0d05e0 0x7fd51c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c15c320 0x7fd51c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c070020 con 0x7fd51c0b0d00 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0cfde0 con 0x7fd51c0d05e0 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd52177e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c0701a0 con 0x7fd51c15c320 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53084/0 (socket says 172.21.15.1:53084) 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 -- 172.21.15.1:0/3113644957 learned_addr learned my addr 172.21.15.1:0/3113644957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51a7fc640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd51c0d05e0 0x7fd51c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51b7fe640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c15c320 0x7fd51c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 -- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c15c320 msgr2=0x7fd51c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c15c320 0x7fd51c0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 -- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd51c0d05e0 msgr2=0x7fd51c1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd51c0d05e0 0x7fd51c1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 -- 172.21.15.1:0/3113644957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c1f49f0 con 0x7fd51c0b0d00 2024-05-09T06:27:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd51affd640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000c00 tx=0x7fd50c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c041070 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.241+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50c03c660 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.242+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c1f4cd0 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.242+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c037040 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.242+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c0b4b80 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.243+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c0752b0 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.243+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd50c044070 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.244+0000 7fd5037fe640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.244+0000 7fd51a7fc640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.244+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd50c0aba00 con 0x7fd51c0b0d00 2024-05-09T06:27:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.244+0000 7fd51a7fc640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 0x7fd50806e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd51c06daf0 tx=0x7fd510009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.248+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd50c079b40 con 0x7fd51c0b0d00 2024-05-09T06:27:44.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.430+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd51c0b6bd0 con 0x7fd51c0b0d00 2024-05-09T06:27:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.431+0000 7fd5037fe640 1 -- 172.21.15.1:0/3113644957 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fd50c07de40 con 0x7fd51c0b0d00 2024-05-09T06:27:44.413 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:44.414 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 msgr2=0x7fd50806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 0x7fd50806e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd51c06daf0 tx=0x7fd510009190 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 msgr2=0x7fd51c0b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000c00 tx=0x7fd50c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 shutdown_connections 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd51c15c320 0x7fd51c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd51c0d05e0 0x7fd51c1f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 --2- 172.21.15.1:0/3113644957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd51c0b0d00 0x7fd51c0b6610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 >> 172.21.15.1:0/3113644957 conn(0x7fd51c0d1d70 msgr2=0x7fd51c0b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 shutdown_connections 2024-05-09T06:27:44.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.433+0000 7fd52177e640 1 -- 172.21.15.1:0/3113644957 wait complete. 2024-05-09T06:27:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:27:44.427 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:27:44.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.548+0000 7f822794c640 1 Processor -- start 2024-05-09T06:27:44.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.548+0000 7f822794c640 1 -- start start 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f822794c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8220099b30 0x7f82200b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f822794c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8220099930 con 0x7f82200c6860 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f822794c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8220162930 con 0x7f822015e4b0 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f822794c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f82200b1e20 con 0x7f8220099b30 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f8224ee7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8220099b30 0x7f82200b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f82256e8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f82200c6860 0x7f82200c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38382/0 (socket says 172.21.15.1:38382) 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f82256e8640 1 -- 172.21.15.1:0/3651223530 learned_addr learned my addr 172.21.15.1:0/3651223530 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:44.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f8224ee7640 1 -- 172.21.15.1:0/3651223530 >> v1:172.21.15.6:6789/0 conn(0x7f822015e4b0 legacy=0x7f822015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f8224ee7640 1 -- 172.21.15.1:0/3651223530 >> v1:172.21.15.1:6789/0 conn(0x7f82200c6860 legacy=0x7f82200c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.549+0000 7f821ffff640 1 -- 172.21.15.1:0/3651223530 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984467989 0 0) 0x7f8218001110 con 0x7f82200c6860 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.550+0000 7f8224ee7640 1 -- 172.21.15.1:0/3651223530 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220162c10 con 0x7f8220099b30 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.550+0000 7f8224ee7640 1 --2- 172.21.15.1:0/3651223530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8220099b30 0x7f82200b2aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82140097a0 tx=0x7f821402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a6a5b90130128 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.550+0000 7f821ffff640 1 -- 172.21.15.1:0/3651223530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214037070 con 0x7f8220099b30 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.550+0000 7f821ffff640 1 -- 172.21.15.1:0/3651223530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8214032440 con 0x7f8220099b30 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.550+0000 7f821ffff640 1 -- 172.21.15.1:0/3651223530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214032810 con 0x7f8220099b30 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 -- 172.21.15.1:0/3651223530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8220099b30 msgr2=0x7f82200b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 --2- 172.21.15.1:0/3651223530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8220099b30 0x7f82200b2aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82140097a0 tx=0x7f821402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 -- 172.21.15.1:0/3651223530 shutdown_connections 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 --2- 172.21.15.1:0/3651223530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8220099b30 0x7f82200b2aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 -- 172.21.15.1:0/3651223530 >> 172.21.15.1:0/3651223530 conn(0x7f82200b1fb0 msgr2=0x7f82200b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 -- 172.21.15.1:0/3651223530 shutdown_connections 2024-05-09T06:27:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.551+0000 7f822794c640 1 -- 172.21.15.1:0/3651223530 wait complete. 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.552+0000 7f822794c640 1 Processor -- start 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.552+0000 7f822794c640 1 -- start start 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8220099b30 0x7f82201f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f822015e4b0 0x7f82201521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f822015fa30 con 0x7f82200c6860 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f822015ee90 con 0x7f8220099b30 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f822794c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82200cdf50 con 0x7f822015e4b0 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53104/0 (socket says 172.21.15.1:53104) 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 -- 172.21.15.1:0/3655127579 learned_addr learned my addr 172.21.15.1:0/3655127579 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8225ee9640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f822015e4b0 0x7f82201521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f82256e8640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8220099b30 0x7f82201f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 -- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f822015e4b0 msgr2=0x7f82201521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f822015e4b0 0x7f82201521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 -- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8220099b30 msgr2=0x7f82201f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8220099b30 0x7f82201f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.553+0000 7f8224ee7640 1 -- 172.21.15.1:0/3655127579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82201558e0 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f8224ee7640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82140097a0 tx=0x7f82140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214037040 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8214004250 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8220155bc0 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214032e20 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.554+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8220156180 con 0x7f82200c6860 2024-05-09T06:27:44.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.555+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82200c60c0 con 0x7f82200c6860 2024-05-09T06:27:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.556+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8214030770 con 0x7f82200c6860 2024-05-09T06:27:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.556+0000 7f821dffb640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 0x7f820c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.556+0000 7f82256e8640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 0x7f820c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.556+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f82140ac5c0 con 0x7f82200c6860 2024-05-09T06:27:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.557+0000 7f82256e8640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 0x7f820c06e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8218000f60 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.561+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82200c60c0 con 0x7f82200c6860 2024-05-09T06:27:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.743+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82200cd850 con 0x7f820c06be60 2024-05-09T06:27:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.751+0000 7f821dffb640 1 -- 172.21.15.1:0/3655127579 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f82200cd850 con 0x7f820c06be60 2024-05-09T06:27:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:44.739 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 msgr2=0x7f820c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 0x7f820c06e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8218000f60 tx=0x7f8218009040 comp rx=0 tx=0).stop 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 msgr2=0x7f8220155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82140097a0 tx=0x7f82140046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 shutdown_connections 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f820c06be60 0x7f820c06e320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f822015e4b0 0x7f82201521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82200c6860 0x7f8220155090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 --2- 172.21.15.1:0/3655127579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8220099b30 0x7f82201f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 >> 172.21.15.1:0/3655127579 conn(0x7f82200b1fb0 msgr2=0x7f82200d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 shutdown_connections 2024-05-09T06:27:44.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.755+0000 7f822794c640 1 -- 172.21.15.1:0/3655127579 wait complete. 2024-05-09T06:27:44.748 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-05-09T06:27:43.073164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002695"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795027+0000","last_change":"2024-05-09T06:27:20.644248+0000","last_active":"2024-05-09T06:27:28.795027+0000","last_peered":"2024-05-09T06:27:28.795027+0000","last_clean":"2024-05-09T06:27:28.795027+0000","last_became_active":"2024-05-09T06:27:20.643791+0000","last_became_peered":"2024-05-09T06:27:20.643791+0000","last_unstale":"2024-05-09T06:27:28.795027+0000","last_undegraded":"2024-05-09T06:27:28.795027+0000","last_fullsized":"2024-05-09T06:27:28.795027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:48.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795308+0000","last_change":"2024-05-09T06:27:21.649501+0000","last_active":"2024-05-09T06:27:28.795308+0000","last_peered":"2024-05-09T06:27:28.795308+0000","last_clean":"2024-05-09T06:27:28.795308+0000","last_became_active":"2024-05-09T06:27:21.649193+0000","last_became_peered":"2024-05-09T06:27:21.649193+0000","last_unstale":"2024-05-09T06:27:28.795308+0000","last_undegraded":"2024-05-09T06:27:28.795308+0000","last_fullsized":"2024-05-09T06:27:28.795308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794757+0000","last_change":"2024-05-09T06:27:25.871323+0000","last_active":"2024-05-09T06:27:28.794757+0000","last_peered":"2024-05-09T06:27:28.794757+0000","last_clean":"2024-05-09T06:27:28.794757+0000","last_became_active":"2024-05-09T06:27:25.870947+0000","last_became_peered":"2024-05-09T06:27:25.870947+0000","last_unstale":"2024-05-09T06:27:28.794757+0000","last_undegraded":"2024-05-09T06:27:28.794757+0000","last_fullsized":"2024-05-09T06:27:28.794757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795538+0000","last_change":"2024-05-09T06:27:20.634799+0000","last_active":"2024-05-09T06:27:28.795538+0000","last_peered":"2024-05-09T06:27:28.795538+0000","last_clean":"2024-05-09T06:27:28.795538+0000","last_became_active":"2024-05-09T06:27:20.634300+0000","last_became_peered":"2024-05-09T06:27:20.634300+0000","last_unstale":"2024-05-09T06:27:28.795538+0000","last_undegraded":"2024-05-09T06:27:28.795538+0000","last_fullsized":"2024-05-09T06:27:28.795538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796489+0000","last_change":"2024-05-09T06:27:21.638390+0000","last_active":"2024-05-09T06:27:28.796489+0000","last_peered":"2024-05-09T06:27:28.796489+0000","last_clean":"2024-05-09T06:27:28.796489+0000","last_became_active":"2024-05-09T06:27:21.637720+0000","last_became_peered":"2024-05-09T06:27:21.637720+0000","last_unstale":"2024-05-09T06:27:28.796489+0000","last_undegraded":"2024-05-09T06:27:28.796489+0000","last_fullsized":"2024-05-09T06:27:28.796489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666410+0000","last_change":"2024-05-09T06:27:25.668634+0000","last_active":"2024-05-09T06:27:26.666410+0000","last_peered":"2024-05-09T06:27:26.666410+0000","last_clean":"2024-05-09T06:27:26.666410+0000","last_became_active":"2024-05-09T06:27:25.668153+0000","last_became_peered":"2024-05-09T06:27:25.668153+0000","last_unstale":"2024-05-09T06:27:26.666410+0000","last_undegraded":"2024-05-09T06:27:26.666410+0000","last_fullsized":"2024-05-09T06:27:26.666410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:46.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310311+0000","last_change":"2024-05-09T06:27:20.646888+0000","last_active":"2024-05-09T06:27:28.310311+0000","last_peered":"2024-05-09T06:27:28.310311+0000","last_clean":"2024-05-09T06:27:28.310311+0000","last_became_active":"2024-05-09T06:27:20.646603+0000","last_became_peered":"2024-05-09T06:27:20.646603+0000","last_unstale":"2024-05-09T06:27:28.310311+0000","last_undegraded":"2024-05-09T06:27:28.310311+0000","last_fullsized":"2024-05-09T06:27:28.310311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:06.656512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796678+0000","last_change":"2024-05-09T06:27:21.659688+0000","last_active":"2024-05-09T06:27:28.796678+0000","last_peered":"2024-05-09T06:27:28.796678+0000","last_clean":"2024-05-09T06:27:28.796678+0000","last_became_active":"2024-05-09T06:27:21.659407+0000","last_became_peered":"2024-05-09T06:27:21.659407+0000","last_unstale":"2024-05-09T06:27:28.796678+0000","last_undegraded":"2024-05-09T06:27:28.796678+0000","last_fullsized":"2024-05-09T06:27:28.796678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667001+0000","last_change":"2024-05-09T06:27:25.674815+0000","last_active":"2024-05-09T06:27:26.667001+0000","last_peered":"2024-05-09T06:27:26.667001+0000","last_clean":"2024-05-09T06:27:26.667001+0000","last_became_active":"2024-05-09T06:27:25.673855+0000","last_became_peered":"2024-05-09T06:27:25.673855+0000","last_unstale":"2024-05-09T06:27:26.667001+0000","last_undegraded":"2024-05-09T06:27:26.667001+0000","last_fullsized":"2024-05-09T06:27:26.667001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:16.160631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309929+0000","last_change":"2024-05-09T06:27:20.644933+0000","last_active":"2024-05-09T06:27:28.309929+0000","last_peered":"2024-05-09T06:27:28.309929+0000","last_clean":"2024-05-09T06:27:28.309929+0000","last_became_active":"2024-05-09T06:27:20.644019+0000","last_became_peered":"2024-05-09T06:27:20.644019+0000","last_unstale":"2024-05-09T06:27:28.309929+0000","last_undegraded":"2024-05-09T06:27:28.309929+0000","last_fullsized":"2024-05-09T06:27:28.309929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796776+0000","last_change":"2024-05-09T06:27:21.660829+0000","last_active":"2024-05-09T06:27:28.796776+0000","last_peered":"2024-05-09T06:27:28.796776+0000","last_clean":"2024-05-09T06:27:28.796776+0000","last_became_active":"2024-05-09T06:27:21.658349+0000","last_became_peered":"2024-05-09T06:27:21.658349+0000","last_unstale":"2024-05-09T06:27:28.796776+0000","last_undegraded":"2024-05-09T06:27:28.796776+0000","last_fullsized":"2024-05-09T06:27:28.796776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666466+0000","last_change":"2024-05-09T06:27:25.664257+0000","last_active":"2024-05-09T06:27:26.666466+0000","last_peered":"2024-05-09T06:27:26.666466+0000","last_clean":"2024-05-09T06:27:26.666466+0000","last_became_active":"2024-05-09T06:27:25.663787+0000","last_became_peered":"2024-05-09T06:27:25.663787+0000","last_unstale":"2024-05-09T06:27:26.666466+0000","last_undegraded":"2024-05-09T06:27:26.666466+0000","last_fullsized":"2024-05-09T06:27:26.666466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:57.007200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795391+0000","last_change":"2024-05-09T06:27:21.652107+0000","last_active":"2024-05-09T06:27:28.795391+0000","last_peered":"2024-05-09T06:27:28.795391+0000","last_clean":"2024-05-09T06:27:28.795391+0000","last_became_active":"2024-05-09T06:27:21.651764+0000","last_became_peered":"2024-05-09T06:27:21.651764+0000","last_unstale":"2024-05-09T06:27:28.795391+0000","last_undegraded":"2024-05-09T06:27:28.795391+0000","last_fullsized":"2024-05-09T06:27:28.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310074+0000","last_change":"2024-05-09T06:27:20.644636+0000","last_active":"2024-05-09T06:27:28.310074+0000","last_peered":"2024-05-09T06:27:28.310074+0000","last_clean":"2024-05-09T06:27:28.310074+0000","last_became_active":"2024-05-09T06:27:20.644155+0000","last_became_peered":"2024-05-09T06:27:20.644155+0000","last_unstale":"2024-05-09T06:27:28.310074+0000","last_undegraded":"2024-05-09T06:27:28.310074+0000","last_fullsized":"2024-05-09T06:27:28.310074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667637+0000","last_change":"2024-05-09T06:27:25.674517+0000","last_active":"2024-05-09T06:27:26.667637+0000","last_peered":"2024-05-09T06:27:26.667637+0000","last_clean":"2024-05-09T06:27:26.667637+0000","last_became_active":"2024-05-09T06:27:25.673256+0000","last_became_peered":"2024-05-09T06:27:25.673256+0000","last_unstale":"2024-05-09T06:27:26.667637+0000","last_undegraded":"2024-05-09T06:27:26.667637+0000","last_fullsized":"2024-05-09T06:27:26.667637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795461+0000","last_change":"2024-05-09T06:27:21.650596+0000","last_active":"2024-05-09T06:27:28.795461+0000","last_peered":"2024-05-09T06:27:28.795461+0000","last_clean":"2024-05-09T06:27:28.795461+0000","last_became_active":"2024-05-09T06:27:21.649943+0000","last_became_peered":"2024-05-09T06:27:21.649943+0000","last_unstale":"2024-05-09T06:27:28.795461+0000","last_undegraded":"2024-05-09T06:27:28.795461+0000","last_fullsized":"2024-05-09T06:27:28.795461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897815+0000","last_change":"2024-05-09T06:27:20.634659+0000","last_active":"2024-05-09T06:27:26.897815+0000","last_peered":"2024-05-09T06:27:26.897815+0000","last_clean":"2024-05-09T06:27:26.897815+0000","last_became_active":"2024-05-09T06:27:20.634344+0000","last_became_peered":"2024-05-09T06:27:20.634344+0000","last_unstale":"2024-05-09T06:27:26.897815+0000","last_undegraded":"2024-05-09T06:27:26.897815+0000","last_fullsized":"2024-05-09T06:27:26.897815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667419+0000","last_change":"2024-05-09T06:27:25.674355+0000","last_active":"2024-05-09T06:27:26.667419+0000","last_peered":"2024-05-09T06:27:26.667419+0000","last_clean":"2024-05-09T06:27:26.667419+0000","last_became_active":"2024-05-09T06:27:25.673950+0000","last_became_peered":"2024-05-09T06:27:25.673950+0000","last_unstale":"2024-05-09T06:27:26.667419+0000","last_undegraded":"2024-05-09T06:27:26.667419+0000","last_fullsized":"2024-05-09T06:27:26.667419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.231985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796292+0000","last_change":"2024-05-09T06:27:21.641405+0000","last_active":"2024-05-09T06:27:28.796292+0000","last_peered":"2024-05-09T06:27:28.796292+0000","last_clean":"2024-05-09T06:27:28.796292+0000","last_became_active":"2024-05-09T06:27:21.641019+0000","last_became_peered":"2024-05-09T06:27:21.641019+0000","last_unstale":"2024-05-09T06:27:28.796292+0000","last_undegraded":"2024-05-09T06:27:28.796292+0000","last_fullsized":"2024-05-09T06:27:28.796292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897255+0000","last_change":"2024-05-09T06:27:20.644807+0000","last_active":"2024-05-09T06:27:26.897255+0000","last_peered":"2024-05-09T06:27:26.897255+0000","last_clean":"2024-05-09T06:27:26.897255+0000","last_became_active":"2024-05-09T06:27:20.644510+0000","last_became_peered":"2024-05-09T06:27:20.644510+0000","last_unstale":"2024-05-09T06:27:26.897255+0000","last_undegraded":"2024-05-09T06:27:26.897255+0000","last_fullsized":"2024-05-09T06:27:26.897255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897895+0000","last_change":"2024-05-09T06:27:25.867962+0000","last_active":"2024-05-09T06:27:26.897895+0000","last_peered":"2024-05-09T06:27:26.897895+0000","last_clean":"2024-05-09T06:27:26.897895+0000","last_became_active":"2024-05-09T06:27:25.867593+0000","last_became_peered":"2024-05-09T06:27:25.867593+0000","last_unstale":"2024-05-09T06:27:26.897895+0000","last_undegraded":"2024-05-09T06:27:26.897895+0000","last_fullsized":"2024-05-09T06:27:26.897895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666600+0000","last_change":"2024-05-09T06:27:21.651627+0000","last_active":"2024-05-09T06:27:26.666600+0000","last_peered":"2024-05-09T06:27:26.666600+0000","last_clean":"2024-05-09T06:27:26.666600+0000","last_became_active":"2024-05-09T06:27:21.651021+0000","last_became_peered":"2024-05-09T06:27:21.651021+0000","last_unstale":"2024-05-09T06:27:26.666600+0000","last_undegraded":"2024-05-09T06:27:26.666600+0000","last_fullsized":"2024-05-09T06:27:26.666600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:04.963108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309528+0000","last_change":"2024-05-09T06:27:20.644691+0000","last_active":"2024-05-09T06:27:28.309528+0000","last_peered":"2024-05-09T06:27:28.309528+0000","last_clean":"2024-05-09T06:27:28.309528+0000","last_became_active":"2024-05-09T06:27:20.644291+0000","last_became_peered":"2024-05-09T06:27:20.644291+0000","last_unstale":"2024-05-09T06:27:28.309528+0000","last_undegraded":"2024-05-09T06:27:28.309528+0000","last_fullsized":"2024-05-09T06:27:28.309528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667629+0000","last_change":"2024-05-09T06:27:25.670582+0000","last_active":"2024-05-09T06:27:26.667629+0000","last_peered":"2024-05-09T06:27:26.667629+0000","last_clean":"2024-05-09T06:27:26.667629+0000","last_became_active":"2024-05-09T06:27:25.670180+0000","last_became_peered":"2024-05-09T06:27:25.670180+0000","last_unstale":"2024-05-09T06:27:26.667629+0000","last_undegraded":"2024-05-09T06:27:26.667629+0000","last_fullsized":"2024-05-09T06:27:26.667629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796762+0000","last_change":"2024-05-09T06:27:21.644419+0000","last_active":"2024-05-09T06:27:28.796762+0000","last_peered":"2024-05-09T06:27:28.796762+0000","last_clean":"2024-05-09T06:27:28.796762+0000","last_became_active":"2024-05-09T06:27:21.644054+0000","last_became_peered":"2024-05-09T06:27:21.644054+0000","last_unstale":"2024-05-09T06:27:28.796762+0000","last_undegraded":"2024-05-09T06:27:28.796762+0000","last_fullsized":"2024-05-09T06:27:28.796762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795166+0000","last_change":"2024-05-09T06:27:20.644413+0000","last_active":"2024-05-09T06:27:28.795166+0000","last_peered":"2024-05-09T06:27:28.795166+0000","last_clean":"2024-05-09T06:27:28.795166+0000","last_became_active":"2024-05-09T06:27:20.644063+0000","last_became_peered":"2024-05-09T06:27:20.644063+0000","last_unstale":"2024-05-09T06:27:28.795166+0000","last_undegraded":"2024-05-09T06:27:28.795166+0000","last_fullsized":"2024-05-09T06:27:28.795166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666413+0000","last_change":"2024-05-09T06:27:25.868219+0000","last_active":"2024-05-09T06:27:26.666413+0000","last_peered":"2024-05-09T06:27:26.666413+0000","last_clean":"2024-05-09T06:27:26.666413+0000","last_became_active":"2024-05-09T06:27:25.867618+0000","last_became_peered":"2024-05-09T06:27:25.867618+0000","last_unstale":"2024-05-09T06:27:26.666413+0000","last_undegraded":"2024-05-09T06:27:26.666413+0000","last_fullsized":"2024-05-09T06:27:26.666413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:59.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795981+0000","last_change":"2024-05-09T06:27:21.659207+0000","last_active":"2024-05-09T06:27:28.795981+0000","last_peered":"2024-05-09T06:27:28.795981+0000","last_clean":"2024-05-09T06:27:28.795981+0000","last_became_active":"2024-05-09T06:27:21.651367+0000","last_became_peered":"2024-05-09T06:27:21.651367+0000","last_unstale":"2024-05-09T06:27:28.795981+0000","last_undegraded":"2024-05-09T06:27:28.795981+0000","last_fullsized":"2024-05-09T06:27:28.795981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897683+0000","last_change":"2024-05-09T06:27:20.637053+0000","last_active":"2024-05-09T06:27:26.897683+0000","last_peered":"2024-05-09T06:27:26.897683+0000","last_clean":"2024-05-09T06:27:26.897683+0000","last_became_active":"2024-05-09T06:27:20.636755+0000","last_became_peered":"2024-05-09T06:27:20.636755+0000","last_unstale":"2024-05-09T06:27:26.897683+0000","last_undegraded":"2024-05-09T06:27:26.897683+0000","last_fullsized":"2024-05-09T06:27:26.897683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795768+0000","last_change":"2024-05-09T06:27:25.871255+0000","last_active":"2024-05-09T06:27:28.795768+0000","last_peered":"2024-05-09T06:27:28.795768+0000","last_clean":"2024-05-09T06:27:28.795768+0000","last_became_active":"2024-05-09T06:27:25.870862+0000","last_became_peered":"2024-05-09T06:27:25.870862+0000","last_unstale":"2024-05-09T06:27:28.795768+0000","last_undegraded":"2024-05-09T06:27:28.795768+0000","last_fullsized":"2024-05-09T06:27:28.795768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:53.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897799+0000","last_change":"2024-05-09T06:27:21.649868+0000","last_active":"2024-05-09T06:27:26.897799+0000","last_peered":"2024-05-09T06:27:26.897799+0000","last_clean":"2024-05-09T06:27:26.897799+0000","last_became_active":"2024-05-09T06:27:21.649515+0000","last_became_peered":"2024-05-09T06:27:21.649515+0000","last_unstale":"2024-05-09T06:27:26.897799+0000","last_undegraded":"2024-05-09T06:27:26.897799+0000","last_fullsized":"2024-05-09T06:27:26.897799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310415+0000","last_change":"2024-05-09T06:27:20.644414+0000","last_active":"2024-05-09T06:27:28.310415+0000","last_peered":"2024-05-09T06:27:28.310415+0000","last_clean":"2024-05-09T06:27:28.310415+0000","last_became_active":"2024-05-09T06:27:20.644077+0000","last_became_peered":"2024-05-09T06:27:20.644077+0000","last_unstale":"2024-05-09T06:27:28.310415+0000","last_undegraded":"2024-05-09T06:27:28.310415+0000","last_fullsized":"2024-05-09T06:27:28.310415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:34.718775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795535+0000","last_change":"2024-05-09T06:27:25.867794+0000","last_active":"2024-05-09T06:27:28.795535+0000","last_peered":"2024-05-09T06:27:28.795535+0000","last_clean":"2024-05-09T06:27:28.795535+0000","last_became_active":"2024-05-09T06:27:25.867351+0000","last_became_peered":"2024-05-09T06:27:25.867351+0000","last_unstale":"2024-05-09T06:27:28.795535+0000","last_undegraded":"2024-05-09T06:27:28.795535+0000","last_fullsized":"2024-05-09T06:27:28.795535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796829+0000","last_change":"2024-05-09T06:27:21.662543+0000","last_active":"2024-05-09T06:27:28.796829+0000","last_peered":"2024-05-09T06:27:28.796829+0000","last_clean":"2024-05-09T06:27:28.796829+0000","last_became_active":"2024-05-09T06:27:21.662266+0000","last_became_peered":"2024-05-09T06:27:21.662266+0000","last_unstale":"2024-05-09T06:27:28.796829+0000","last_undegraded":"2024-05-09T06:27:28.796829+0000","last_fullsized":"2024-05-09T06:27:28.796829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897904+0000","last_change":"2024-05-09T06:27:20.634145+0000","last_active":"2024-05-09T06:27:26.897904+0000","last_peered":"2024-05-09T06:27:26.897904+0000","last_clean":"2024-05-09T06:27:26.897904+0000","last_became_active":"2024-05-09T06:27:20.633864+0000","last_became_peered":"2024-05-09T06:27:20.633864+0000","last_unstale":"2024-05-09T06:27:26.897904+0000","last_undegraded":"2024-05-09T06:27:26.897904+0000","last_fullsized":"2024-05-09T06:27:26.897904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897405+0000","last_change":"2024-05-09T06:27:25.672524+0000","last_active":"2024-05-09T06:27:26.897405+0000","last_peered":"2024-05-09T06:27:26.897405+0000","last_clean":"2024-05-09T06:27:26.897405+0000","last_became_active":"2024-05-09T06:27:25.672004+0000","last_became_peered":"2024-05-09T06:27:25.672004+0000","last_unstale":"2024-05-09T06:27:26.897405+0000","last_undegraded":"2024-05-09T06:27:26.897405+0000","last_fullsized":"2024-05-09T06:27:26.897405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667430+0000","last_change":"2024-05-09T06:27:21.634887+0000","last_active":"2024-05-09T06:27:26.667430+0000","last_peered":"2024-05-09T06:27:26.667430+0000","last_clean":"2024-05-09T06:27:26.667430+0000","last_became_active":"2024-05-09T06:27:21.634609+0000","last_became_peered":"2024-05-09T06:27:21.634609+0000","last_unstale":"2024-05-09T06:27:26.667430+0000","last_undegraded":"2024-05-09T06:27:26.667430+0000","last_fullsized":"2024-05-09T06:27:26.667430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:59.874418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795267+0000","last_change":"2024-05-09T06:27:20.642066+0000","last_active":"2024-05-09T06:27:28.795267+0000","last_peered":"2024-05-09T06:27:28.795267+0000","last_clean":"2024-05-09T06:27:28.795267+0000","last_became_active":"2024-05-09T06:27:20.641753+0000","last_became_peered":"2024-05-09T06:27:20.641753+0000","last_unstale":"2024-05-09T06:27:28.795267+0000","last_undegraded":"2024-05-09T06:27:28.795267+0000","last_fullsized":"2024-05-09T06:27:28.795267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897884+0000","last_change":"2024-05-09T06:27:25.671230+0000","last_active":"2024-05-09T06:27:26.897884+0000","last_peered":"2024-05-09T06:27:26.897884+0000","last_clean":"2024-05-09T06:27:26.897884+0000","last_became_active":"2024-05-09T06:27:25.670778+0000","last_became_peered":"2024-05-09T06:27:25.670778+0000","last_unstale":"2024-05-09T06:27:26.897884+0000","last_undegraded":"2024-05-09T06:27:26.897884+0000","last_fullsized":"2024-05-09T06:27:26.897884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:20:39.476936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897252+0000","last_change":"2024-05-09T06:27:21.649626+0000","last_active":"2024-05-09T06:27:26.897252+0000","last_peered":"2024-05-09T06:27:26.897252+0000","last_clean":"2024-05-09T06:27:26.897252+0000","last_became_active":"2024-05-09T06:27:21.649332+0000","last_became_peered":"2024-05-09T06:27:21.649332+0000","last_unstale":"2024-05-09T06:27:26.897252+0000","last_undegraded":"2024-05-09T06:27:26.897252+0000","last_fullsized":"2024-05-09T06:27:26.897252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665959+0000","last_change":"2024-05-09T06:27:20.632572+0000","last_active":"2024-05-09T06:27:26.665959+0000","last_peered":"2024-05-09T06:27:26.665959+0000","last_clean":"2024-05-09T06:27:26.665959+0000","last_became_active":"2024-05-09T06:27:20.630286+0000","last_became_peered":"2024-05-09T06:27:20.630286+0000","last_unstale":"2024-05-09T06:27:26.665959+0000","last_undegraded":"2024-05-09T06:27:26.665959+0000","last_fullsized":"2024-05-09T06:27:26.665959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897808+0000","last_change":"2024-05-09T06:27:25.678228+0000","last_active":"2024-05-09T06:27:26.897808+0000","last_peered":"2024-05-09T06:27:26.897808+0000","last_clean":"2024-05-09T06:27:26.897808+0000","last_became_active":"2024-05-09T06:27:25.677638+0000","last_became_peered":"2024-05-09T06:27:25.677638+0000","last_unstale":"2024-05-09T06:27:26.897808+0000","last_undegraded":"2024-05-09T06:27:26.897808+0000","last_fullsized":"2024-05-09T06:27:26.897808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794917+0000","last_change":"2024-05-09T06:27:21.634999+0000","last_active":"2024-05-09T06:27:28.794917+0000","last_peered":"2024-05-09T06:27:28.794917+0000","last_clean":"2024-05-09T06:27:28.794917+0000","last_became_active":"2024-05-09T06:27:21.634688+0000","last_became_peered":"2024-05-09T06:27:21.634688+0000","last_unstale":"2024-05-09T06:27:28.794917+0000","last_undegraded":"2024-05-09T06:27:28.794917+0000","last_fullsized":"2024-05-09T06:27:28.794917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309529+0000","last_change":"2024-05-09T06:27:20.645301+0000","last_active":"2024-05-09T06:27:28.309529+0000","last_peered":"2024-05-09T06:27:28.309529+0000","last_clean":"2024-05-09T06:27:28.309529+0000","last_became_active":"2024-05-09T06:27:20.644309+0000","last_became_peered":"2024-05-09T06:27:20.644309+0000","last_unstale":"2024-05-09T06:27:28.309529+0000","last_undegraded":"2024-05-09T06:27:28.309529+0000","last_fullsized":"2024-05-09T06:27:28.309529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796649+0000","last_change":"2024-05-09T06:27:25.870805+0000","last_active":"2024-05-09T06:27:28.796649+0000","last_peered":"2024-05-09T06:27:28.796649+0000","last_clean":"2024-05-09T06:27:28.796649+0000","last_became_active":"2024-05-09T06:27:25.870405+0000","last_became_peered":"2024-05-09T06:27:25.870405+0000","last_unstale":"2024-05-09T06:27:28.796649+0000","last_undegraded":"2024-05-09T06:27:28.796649+0000","last_fullsized":"2024-05-09T06:27:28.796649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795597+0000","last_change":"2024-05-09T06:27:21.660077+0000","last_active":"2024-05-09T06:27:28.795597+0000","last_peered":"2024-05-09T06:27:28.795597+0000","last_clean":"2024-05-09T06:27:28.795597+0000","last_became_active":"2024-05-09T06:27:21.659816+0000","last_became_peered":"2024-05-09T06:27:21.659816+0000","last_unstale":"2024-05-09T06:27:28.795597+0000","last_undegraded":"2024-05-09T06:27:28.795597+0000","last_fullsized":"2024-05-09T06:27:28.795597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794955+0000","last_change":"2024-05-09T06:27:20.638709+0000","last_active":"2024-05-09T06:27:28.794955+0000","last_peered":"2024-05-09T06:27:28.794955+0000","last_clean":"2024-05-09T06:27:28.794955+0000","last_became_active":"2024-05-09T06:27:20.638404+0000","last_became_peered":"2024-05-09T06:27:20.638404+0000","last_unstale":"2024-05-09T06:27:28.794955+0000","last_undegraded":"2024-05-09T06:27:28.794955+0000","last_fullsized":"2024-05-09T06:27:28.794955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309694+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:27:28.309694+0000","last_peered":"2024-05-09T06:27:28.309694+0000","last_clean":"2024-05-09T06:27:28.309694+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:27:28.309694+0000","last_undegraded":"2024-05-09T06:27:28.309694+0000","last_fullsized":"2024-05-09T06:27:28.309694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795720+0000","last_change":"2024-05-09T06:27:25.871420+0000","last_active":"2024-05-09T06:27:28.795720+0000","last_peered":"2024-05-09T06:27:28.795720+0000","last_clean":"2024-05-09T06:27:28.795720+0000","last_became_active":"2024-05-09T06:27:25.870765+0000","last_became_peered":"2024-05-09T06:27:25.870765+0000","last_unstale":"2024-05-09T06:27:28.795720+0000","last_undegraded":"2024-05-09T06:27:28.795720+0000","last_fullsized":"2024-05-09T06:27:28.795720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310010+0000","last_change":"2024-05-09T06:27:21.665731+0000","last_active":"2024-05-09T06:27:28.310010+0000","last_peered":"2024-05-09T06:27:28.310010+0000","last_clean":"2024-05-09T06:27:28.310010+0000","last_became_active":"2024-05-09T06:27:21.665457+0000","last_became_peered":"2024-05-09T06:27:21.665457+0000","last_unstale":"2024-05-09T06:27:28.310010+0000","last_undegraded":"2024-05-09T06:27:28.310010+0000","last_fullsized":"2024-05-09T06:27:28.310010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666314+0000","last_change":"2024-05-09T06:27:20.639195+0000","last_active":"2024-05-09T06:27:26.666314+0000","last_peered":"2024-05-09T06:27:26.666314+0000","last_clean":"2024-05-09T06:27:26.666314+0000","last_became_active":"2024-05-09T06:27:20.638833+0000","last_became_peered":"2024-05-09T06:27:20.638833+0000","last_unstale":"2024-05-09T06:27:26.666314+0000","last_undegraded":"2024-05-09T06:27:26.666314+0000","last_fullsized":"2024-05-09T06:27:26.666314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795939+0000","last_change":"2024-05-09T06:27:25.868481+0000","last_active":"2024-05-09T06:27:28.795939+0000","last_peered":"2024-05-09T06:27:28.795939+0000","last_clean":"2024-05-09T06:27:28.795939+0000","last_became_active":"2024-05-09T06:27:25.867521+0000","last_became_peered":"2024-05-09T06:27:25.867521+0000","last_unstale":"2024-05-09T06:27:28.795939+0000","last_undegraded":"2024-05-09T06:27:28.795939+0000","last_fullsized":"2024-05-09T06:27:28.795939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796554+0000","last_change":"2024-05-09T06:27:21.663211+0000","last_active":"2024-05-09T06:27:28.796554+0000","last_peered":"2024-05-09T06:27:28.796554+0000","last_clean":"2024-05-09T06:27:28.796554+0000","last_became_active":"2024-05-09T06:27:21.662892+0000","last_became_peered":"2024-05-09T06:27:21.662892+0000","last_unstale":"2024-05-09T06:27:28.796554+0000","last_undegraded":"2024-05-09T06:27:28.796554+0000","last_fullsized":"2024-05-09T06:27:28.796554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897750+0000","last_change":"2024-05-09T06:27:20.638780+0000","last_active":"2024-05-09T06:27:26.897750+0000","last_peered":"2024-05-09T06:27:26.897750+0000","last_clean":"2024-05-09T06:27:26.897750+0000","last_became_active":"2024-05-09T06:27:20.638469+0000","last_became_peered":"2024-05-09T06:27:20.638469+0000","last_unstale":"2024-05-09T06:27:26.897750+0000","last_undegraded":"2024-05-09T06:27:26.897750+0000","last_fullsized":"2024-05-09T06:27:26.897750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795796+0000","last_change":"2024-05-09T06:27:25.673712+0000","last_active":"2024-05-09T06:27:28.795796+0000","last_peered":"2024-05-09T06:27:28.795796+0000","last_clean":"2024-05-09T06:27:28.795796+0000","last_became_active":"2024-05-09T06:27:25.673062+0000","last_became_peered":"2024-05-09T06:27:25.673062+0000","last_unstale":"2024-05-09T06:27:28.795796+0000","last_undegraded":"2024-05-09T06:27:28.795796+0000","last_fullsized":"2024-05-09T06:27:28.795796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310517+0000","last_change":"2024-05-09T06:27:21.652689+0000","last_active":"2024-05-09T06:27:28.310517+0000","last_peered":"2024-05-09T06:27:28.310517+0000","last_clean":"2024-05-09T06:27:28.310517+0000","last_became_active":"2024-05-09T06:27:21.652309+0000","last_became_peered":"2024-05-09T06:27:21.652309+0000","last_unstale":"2024-05-09T06:27:28.310517+0000","last_undegraded":"2024-05-09T06:27:28.310517+0000","last_fullsized":"2024-05-09T06:27:28.310517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:30.619441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666511+0000","last_change":"2024-05-09T06:27:20.631329+0000","last_active":"2024-05-09T06:27:26.666511+0000","last_peered":"2024-05-09T06:27:26.666511+0000","last_clean":"2024-05-09T06:27:26.666511+0000","last_became_active":"2024-05-09T06:27:20.631027+0000","last_became_peered":"2024-05-09T06:27:20.631027+0000","last_unstale":"2024-05-09T06:27:26.666511+0000","last_undegraded":"2024-05-09T06:27:26.666511+0000","last_fullsized":"2024-05-09T06:27:26.666511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795581+0000","last_change":"2024-05-09T06:27:25.868448+0000","last_active":"2024-05-09T06:27:28.795581+0000","last_peered":"2024-05-09T06:27:28.795581+0000","last_clean":"2024-05-09T06:27:28.795581+0000","last_became_active":"2024-05-09T06:27:25.867998+0000","last_became_peered":"2024-05-09T06:27:25.867998+0000","last_unstale":"2024-05-09T06:27:28.795581+0000","last_undegraded":"2024-05-09T06:27:28.795581+0000","last_fullsized":"2024-05-09T06:27:28.795581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310235+0000","last_change":"2024-05-09T06:27:21.654321+0000","last_active":"2024-05-09T06:27:28.310235+0000","last_peered":"2024-05-09T06:27:28.310235+0000","last_clean":"2024-05-09T06:27:28.310235+0000","last_became_active":"2024-05-09T06:27:21.654037+0000","last_became_peered":"2024-05-09T06:27:21.654037+0000","last_unstale":"2024-05-09T06:27:28.310235+0000","last_undegraded":"2024-05-09T06:27:28.310235+0000","last_fullsized":"2024-05-09T06:27:28.310235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667634+0000","last_change":"2024-05-09T06:27:20.634285+0000","last_active":"2024-05-09T06:27:26.667634+0000","last_peered":"2024-05-09T06:27:26.667634+0000","last_clean":"2024-05-09T06:27:26.667634+0000","last_became_active":"2024-05-09T06:27:20.633961+0000","last_became_peered":"2024-05-09T06:27:20.633961+0000","last_unstale":"2024-05-09T06:27:26.667634+0000","last_undegraded":"2024-05-09T06:27:26.667634+0000","last_fullsized":"2024-05-09T06:27:26.667634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:22.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666362+0000","last_change":"2024-05-09T06:27:25.666556+0000","last_active":"2024-05-09T06:27:26.666362+0000","last_peered":"2024-05-09T06:27:26.666362+0000","last_clean":"2024-05-09T06:27:26.666362+0000","last_became_active":"2024-05-09T06:27:25.666086+0000","last_became_peered":"2024-05-09T06:27:25.666086+0000","last_unstale":"2024-05-09T06:27:26.666362+0000","last_undegraded":"2024-05-09T06:27:26.666362+0000","last_fullsized":"2024-05-09T06:27:26.666362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:16.228053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309897+0000","last_change":"2024-05-09T06:27:21.653860+0000","last_active":"2024-05-09T06:27:28.309897+0000","last_peered":"2024-05-09T06:27:28.309897+0000","last_clean":"2024-05-09T06:27:28.309897+0000","last_became_active":"2024-05-09T06:27:21.653222+0000","last_became_peered":"2024-05-09T06:27:21.653222+0000","last_unstale":"2024-05-09T06:27:28.309897+0000","last_undegraded":"2024-05-09T06:27:28.309897+0000","last_fullsized":"2024-05-09T06:27:28.309897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665751+0000","last_change":"2024-05-09T06:27:20.632271+0000","last_active":"2024-05-09T06:27:26.665751+0000","last_peered":"2024-05-09T06:27:26.665751+0000","last_clean":"2024-05-09T06:27:26.665751+0000","last_became_active":"2024-05-09T06:27:20.631965+0000","last_became_peered":"2024-05-09T06:27:20.631965+0000","last_unstale":"2024-05-09T06:27:26.665751+0000","last_undegraded":"2024-05-09T06:27:26.665751+0000","last_fullsized":"2024-05-09T06:27:26.665751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796544+0000","last_change":"2024-05-09T06:27:25.673394+0000","last_active":"2024-05-09T06:27:28.796544+0000","last_peered":"2024-05-09T06:27:28.796544+0000","last_clean":"2024-05-09T06:27:28.796544+0000","last_became_active":"2024-05-09T06:27:25.672824+0000","last_became_peered":"2024-05-09T06:27:25.672824+0000","last_unstale":"2024-05-09T06:27:28.796544+0000","last_undegraded":"2024-05-09T06:27:28.796544+0000","last_fullsized":"2024-05-09T06:27:28.796544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795660+0000","last_change":"2024-05-09T06:27:21.658568+0000","last_active":"2024-05-09T06:27:28.795660+0000","last_peered":"2024-05-09T06:27:28.795660+0000","last_clean":"2024-05-09T06:27:28.795660+0000","last_became_active":"2024-05-09T06:27:21.658289+0000","last_became_peered":"2024-05-09T06:27:21.658289+0000","last_unstale":"2024-05-09T06:27:28.795660+0000","last_undegraded":"2024-05-09T06:27:28.795660+0000","last_fullsized":"2024-05-09T06:27:28.795660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.637847+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.637553+0000","last_became_peered":"2024-05-09T06:27:20.637553+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795481+0000","last_change":"2024-05-09T06:27:25.872027+0000","last_active":"2024-05-09T06:27:28.795481+0000","last_peered":"2024-05-09T06:27:28.795481+0000","last_clean":"2024-05-09T06:27:28.795481+0000","last_became_active":"2024-05-09T06:27:25.871616+0000","last_became_peered":"2024-05-09T06:27:25.871616+0000","last_unstale":"2024-05-09T06:27:28.795481+0000","last_undegraded":"2024-05-09T06:27:28.795481+0000","last_fullsized":"2024-05-09T06:27:28.795481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:58:18.735834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309942+0000","last_change":"2024-05-09T06:27:21.655596+0000","last_active":"2024-05-09T06:27:28.309942+0000","last_peered":"2024-05-09T06:27:28.309942+0000","last_clean":"2024-05-09T06:27:28.309942+0000","last_became_active":"2024-05-09T06:27:21.655327+0000","last_became_peered":"2024-05-09T06:27:21.655327+0000","last_unstale":"2024-05-09T06:27:28.309942+0000","last_undegraded":"2024-05-09T06:27:28.309942+0000","last_fullsized":"2024-05-09T06:27:28.309942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796614+0000","last_change":"2024-05-09T06:27:20.640461+0000","last_active":"2024-05-09T06:27:28.796614+0000","last_peered":"2024-05-09T06:27:28.796614+0000","last_clean":"2024-05-09T06:27:28.796614+0000","last_became_active":"2024-05-09T06:27:20.640164+0000","last_became_peered":"2024-05-09T06:27:20.640164+0000","last_unstale":"2024-05-09T06:27:28.796614+0000","last_undegraded":"2024-05-09T06:27:28.796614+0000","last_fullsized":"2024-05-09T06:27:28.796614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795963+0000","last_change":"2024-05-09T06:27:25.868054+0000","last_active":"2024-05-09T06:27:28.795963+0000","last_peered":"2024-05-09T06:27:28.795963+0000","last_clean":"2024-05-09T06:27:28.795963+0000","last_became_active":"2024-05-09T06:27:25.867638+0000","last_became_peered":"2024-05-09T06:27:25.867638+0000","last_unstale":"2024-05-09T06:27:28.795963+0000","last_undegraded":"2024-05-09T06:27:28.795963+0000","last_fullsized":"2024-05-09T06:27:28.795963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:05.780930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795875+0000","last_change":"2024-05-09T06:27:21.640283+0000","last_active":"2024-05-09T06:27:28.795875+0000","last_peered":"2024-05-09T06:27:28.795875+0000","last_clean":"2024-05-09T06:27:28.795875+0000","last_became_active":"2024-05-09T06:27:21.639981+0000","last_became_peered":"2024-05-09T06:27:21.639981+0000","last_unstale":"2024-05-09T06:27:28.795875+0000","last_undegraded":"2024-05-09T06:27:28.795875+0000","last_fullsized":"2024-05-09T06:27:28.795875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:57.154069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794456+0000","last_change":"2024-05-09T06:27:20.636469+0000","last_active":"2024-05-09T06:27:28.794456+0000","last_peered":"2024-05-09T06:27:28.794456+0000","last_clean":"2024-05-09T06:27:28.794456+0000","last_became_active":"2024-05-09T06:27:20.636200+0000","last_became_peered":"2024-05-09T06:27:20.636200+0000","last_unstale":"2024-05-09T06:27:28.794456+0000","last_undegraded":"2024-05-09T06:27:28.794456+0000","last_fullsized":"2024-05-09T06:27:28.794456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:02.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796704+0000","last_change":"2024-05-09T06:27:25.868001+0000","last_active":"2024-05-09T06:27:28.796704+0000","last_peered":"2024-05-09T06:27:28.796704+0000","last_clean":"2024-05-09T06:27:28.796704+0000","last_became_active":"2024-05-09T06:27:25.867561+0000","last_became_peered":"2024-05-09T06:27:25.867561+0000","last_unstale":"2024-05-09T06:27:28.796704+0000","last_undegraded":"2024-05-09T06:27:28.796704+0000","last_fullsized":"2024-05-09T06:27:28.796704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310020+0000","last_change":"2024-05-09T06:27:21.647526+0000","last_active":"2024-05-09T06:27:28.310020+0000","last_peered":"2024-05-09T06:27:28.310020+0000","last_clean":"2024-05-09T06:27:28.310020+0000","last_became_active":"2024-05-09T06:27:21.646721+0000","last_became_peered":"2024-05-09T06:27:21.646721+0000","last_unstale":"2024-05-09T06:27:28.310020+0000","last_undegraded":"2024-05-09T06:27:28.310020+0000","last_fullsized":"2024-05-09T06:27:28.310020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:31.226892+0000","last_change":"2024-05-09T06:27:20.634111+0000","last_active":"2024-05-09T06:27:31.226892+0000","last_peered":"2024-05-09T06:27:31.226892+0000","last_clean":"2024-05-09T06:27:31.226892+0000","last_became_active":"2024-05-09T06:27:20.633772+0000","last_became_peered":"2024-05-09T06:27:20.633772+0000","last_unstale":"2024-05-09T06:27:31.226892+0000","last_undegraded":"2024-05-09T06:27:31.226892+0000","last_fullsized":"2024-05-09T06:27:31.226892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897702+0000","last_change":"2024-05-09T06:27:25.676189+0000","last_active":"2024-05-09T06:27:26.897702+0000","last_peered":"2024-05-09T06:27:26.897702+0000","last_clean":"2024-05-09T06:27:26.897702+0000","last_became_active":"2024-05-09T06:27:25.672390+0000","last_became_peered":"2024-05-09T06:27:25.672390+0000","last_unstale":"2024-05-09T06:27:26.897702+0000","last_undegraded":"2024-05-09T06:27:26.897702+0000","last_fullsized":"2024-05-09T06:27:26.897702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667630+0000","last_change":"2024-05-09T06:27:21.637363+0000","last_active":"2024-05-09T06:27:26.667630+0000","last_peered":"2024-05-09T06:27:26.667630+0000","last_clean":"2024-05-09T06:27:26.667630+0000","last_became_active":"2024-05-09T06:27:21.637056+0000","last_became_peered":"2024-05-09T06:27:21.637056+0000","last_unstale":"2024-05-09T06:27:26.667630+0000","last_undegraded":"2024-05-09T06:27:26.667630+0000","last_fullsized":"2024-05-09T06:27:26.667630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897618+0000","last_change":"2024-05-09T06:27:20.641960+0000","last_active":"2024-05-09T06:27:26.897618+0000","last_peered":"2024-05-09T06:27:26.897618+0000","last_clean":"2024-05-09T06:27:26.897618+0000","last_became_active":"2024-05-09T06:27:20.641670+0000","last_became_peered":"2024-05-09T06:27:20.641670+0000","last_unstale":"2024-05-09T06:27:26.897618+0000","last_undegraded":"2024-05-09T06:27:26.897618+0000","last_fullsized":"2024-05-09T06:27:26.897618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667165+0000","last_change":"2024-05-09T06:27:25.675301+0000","last_active":"2024-05-09T06:27:26.667165+0000","last_peered":"2024-05-09T06:27:26.667165+0000","last_clean":"2024-05-09T06:27:26.667165+0000","last_became_active":"2024-05-09T06:27:25.674818+0000","last_became_peered":"2024-05-09T06:27:25.674818+0000","last_unstale":"2024-05-09T06:27:26.667165+0000","last_undegraded":"2024-05-09T06:27:26.667165+0000","last_fullsized":"2024-05-09T06:27:26.667165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:06.045330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796636+0000","last_change":"2024-05-09T06:27:21.638651+0000","last_active":"2024-05-09T06:27:28.796636+0000","last_peered":"2024-05-09T06:27:28.796636+0000","last_clean":"2024-05-09T06:27:28.796636+0000","last_became_active":"2024-05-09T06:27:21.638111+0000","last_became_peered":"2024-05-09T06:27:21.638111+0000","last_unstale":"2024-05-09T06:27:28.796636+0000","last_undegraded":"2024-05-09T06:27:28.796636+0000","last_fullsized":"2024-05-09T06:27:28.796636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796815+0000","last_change":"2024-05-09T06:27:20.639625+0000","last_active":"2024-05-09T06:27:28.796815+0000","last_peered":"2024-05-09T06:27:28.796815+0000","last_clean":"2024-05-09T06:27:28.796815+0000","last_became_active":"2024-05-09T06:27:20.639287+0000","last_became_peered":"2024-05-09T06:27:20.639287+0000","last_unstale":"2024-05-09T06:27:28.796815+0000","last_undegraded":"2024-05-09T06:27:28.796815+0000","last_fullsized":"2024-05-09T06:27:28.796815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796227+0000","last_change":"2024-05-09T06:27:25.669479+0000","last_active":"2024-05-09T06:27:28.796227+0000","last_peered":"2024-05-09T06:27:28.796227+0000","last_clean":"2024-05-09T06:27:28.796227+0000","last_became_active":"2024-05-09T06:27:25.668634+0000","last_became_peered":"2024-05-09T06:27:25.668634+0000","last_unstale":"2024-05-09T06:27:28.796227+0000","last_undegraded":"2024-05-09T06:27:28.796227+0000","last_fullsized":"2024-05-09T06:27:28.796227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:33.708720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796768+0000","last_change":"2024-05-09T06:27:21.657985+0000","last_active":"2024-05-09T06:27:28.796768+0000","last_peered":"2024-05-09T06:27:28.796768+0000","last_clean":"2024-05-09T06:27:28.796768+0000","last_became_active":"2024-05-09T06:27:21.653693+0000","last_became_peered":"2024-05-09T06:27:21.653693+0000","last_unstale":"2024-05-09T06:27:28.796768+0000","last_undegraded":"2024-05-09T06:27:28.796768+0000","last_fullsized":"2024-05-09T06:27:28.796768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796692+0000","last_change":"2024-05-09T06:27:20.639702+0000","last_active":"2024-05-09T06:27:28.796692+0000","last_peered":"2024-05-09T06:27:28.796692+0000","last_clean":"2024-05-09T06:27:28.796692+0000","last_became_active":"2024-05-09T06:27:20.639361+0000","last_became_peered":"2024-05-09T06:27:20.639361+0000","last_unstale":"2024-05-09T06:27:28.796692+0000","last_undegraded":"2024-05-09T06:27:28.796692+0000","last_fullsized":"2024-05-09T06:27:28.796692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:25.867964+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:25.867579+0000","last_became_peered":"2024-05-09T06:27:25.867579+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796481+0000","last_change":"2024-05-09T06:27:25.871150+0000","last_active":"2024-05-09T06:27:28.796481+0000","last_peered":"2024-05-09T06:27:28.796481+0000","last_clean":"2024-05-09T06:27:28.796481+0000","last_became_active":"2024-05-09T06:27:25.870707+0000","last_became_peered":"2024-05-09T06:27:25.870707+0000","last_unstale":"2024-05-09T06:27:28.796481+0000","last_undegraded":"2024-05-09T06:27:28.796481+0000","last_fullsized":"2024-05-09T06:27:28.796481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897140+0000","last_change":"2024-05-09T06:27:21.648383+0000","last_active":"2024-05-09T06:27:26.897140+0000","last_peered":"2024-05-09T06:27:26.897140+0000","last_clean":"2024-05-09T06:27:26.897140+0000","last_became_active":"2024-05-09T06:27:21.646149+0000","last_became_peered":"2024-05-09T06:27:21.646149+0000","last_unstale":"2024-05-09T06:27:26.897140+0000","last_undegraded":"2024-05-09T06:27:26.897140+0000","last_fullsized":"2024-05-09T06:27:26.897140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:20.639240+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:20.638885+0000","last_became_peered":"2024-05-09T06:27:20.638885+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796512+0000","last_change":"2024-05-09T06:27:25.673610+0000","last_active":"2024-05-09T06:27:28.796512+0000","last_peered":"2024-05-09T06:27:28.796512+0000","last_clean":"2024-05-09T06:27:28.796512+0000","last_became_active":"2024-05-09T06:27:25.673160+0000","last_became_peered":"2024-05-09T06:27:25.673160+0000","last_unstale":"2024-05-09T06:27:28.796512+0000","last_undegraded":"2024-05-09T06:27:28.796512+0000","last_fullsized":"2024-05-09T06:27:28.796512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796199+0000","last_change":"2024-05-09T06:27:21.645047+0000","last_active":"2024-05-09T06:27:28.796199+0000","last_peered":"2024-05-09T06:27:28.796199+0000","last_clean":"2024-05-09T06:27:28.796199+0000","last_became_active":"2024-05-09T06:27:21.644743+0000","last_became_peered":"2024-05-09T06:27:21.644743+0000","last_unstale":"2024-05-09T06:27:28.796199+0000","last_undegraded":"2024-05-09T06:27:28.796199+0000","last_fullsized":"2024-05-09T06:27:28.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795221+0000","last_change":"2024-05-09T06:27:20.635683+0000","last_active":"2024-05-09T06:27:28.795221+0000","last_peered":"2024-05-09T06:27:28.795221+0000","last_clean":"2024-05-09T06:27:28.795221+0000","last_became_active":"2024-05-09T06:27:20.635386+0000","last_became_peered":"2024-05-09T06:27:20.635386+0000","last_unstale":"2024-05-09T06:27:28.795221+0000","last_undegraded":"2024-05-09T06:27:28.795221+0000","last_fullsized":"2024-05-09T06:27:28.795221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.642531+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.642176+0000","last_became_peered":"2024-05-09T06:27:20.642176+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795979+0000","last_change":"2024-05-09T06:27:21.634797+0000","last_active":"2024-05-09T06:27:28.795979+0000","last_peered":"2024-05-09T06:27:28.795979+0000","last_clean":"2024-05-09T06:27:28.795979+0000","last_became_active":"2024-05-09T06:27:21.634458+0000","last_became_peered":"2024-05-09T06:27:21.634458+0000","last_unstale":"2024-05-09T06:27:28.795979+0000","last_undegraded":"2024-05-09T06:27:28.795979+0000","last_fullsized":"2024-05-09T06:27:28.795979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666255+0000","last_change":"2024-05-09T06:27:25.678471+0000","last_active":"2024-05-09T06:27:26.666255+0000","last_peered":"2024-05-09T06:27:26.666255+0000","last_clean":"2024-05-09T06:27:26.666255+0000","last_became_active":"2024-05-09T06:27:25.677981+0000","last_became_peered":"2024-05-09T06:27:25.677981+0000","last_unstale":"2024-05-09T06:27:26.666255+0000","last_undegraded":"2024-05-09T06:27:26.666255+0000","last_fullsized":"2024-05-09T06:27:26.666255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:20.609371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795054+0000","last_change":"2024-05-09T06:27:20.644121+0000","last_active":"2024-05-09T06:27:28.795054+0000","last_peered":"2024-05-09T06:27:28.795054+0000","last_clean":"2024-05-09T06:27:28.795054+0000","last_became_active":"2024-05-09T06:27:20.643795+0000","last_became_peered":"2024-05-09T06:27:20.643795+0000","last_unstale":"2024-05-09T06:27:28.795054+0000","last_undegraded":"2024-05-09T06:27:28.795054+0000","last_fullsized":"2024-05-09T06:27:28.795054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666578+0000","last_change":"2024-05-09T06:27:21.652395+0000","last_active":"2024-05-09T06:27:26.666578+0000","last_peered":"2024-05-09T06:27:26.666578+0000","last_clean":"2024-05-09T06:27:26.666578+0000","last_became_active":"2024-05-09T06:27:21.652019+0000","last_became_peered":"2024-05-09T06:27:21.652019+0000","last_unstale":"2024-05-09T06:27:26.666578+0000","last_undegraded":"2024-05-09T06:27:26.666578+0000","last_fullsized":"2024-05-09T06:27:26.666578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:37.219520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795669+0000","last_change":"2024-05-09T06:27:25.675291+0000","last_active":"2024-05-09T06:27:28.795669+0000","last_peered":"2024-05-09T06:27:28.795669+0000","last_clean":"2024-05-09T06:27:28.795669+0000","last_became_active":"2024-05-09T06:27:25.674422+0000","last_became_peered":"2024-05-09T06:27:25.674422+0000","last_unstale":"2024-05-09T06:27:28.795669+0000","last_undegraded":"2024-05-09T06:27:28.795669+0000","last_fullsized":"2024-05-09T06:27:28.795669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:18.601362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795268+0000","last_change":"2024-05-09T06:27:21.659311+0000","last_active":"2024-05-09T06:27:28.795268+0000","last_peered":"2024-05-09T06:27:28.795268+0000","last_clean":"2024-05-09T06:27:28.795268+0000","last_became_active":"2024-05-09T06:27:21.659039+0000","last_became_peered":"2024-05-09T06:27:21.659039+0000","last_unstale":"2024-05-09T06:27:28.795268+0000","last_undegraded":"2024-05-09T06:27:28.795268+0000","last_fullsized":"2024-05-09T06:27:28.795268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665756+0000","last_change":"2024-05-09T06:27:21.656834+0000","last_active":"2024-05-09T06:27:26.665756+0000","last_peered":"2024-05-09T06:27:26.665756+0000","last_clean":"2024-05-09T06:27:26.665756+0000","last_became_active":"2024-05-09T06:27:21.655466+0000","last_became_peered":"2024-05-09T06:27:21.655466+0000","last_unstale":"2024-05-09T06:27:26.665756+0000","last_undegraded":"2024-05-09T06:27:26.665756+0000","last_fullsized":"2024-05-09T06:27:26.665756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796581+0000","last_change":"2024-05-09T06:27:21.665177+0000","last_active":"2024-05-09T06:27:28.796581+0000","last_peered":"2024-05-09T06:27:28.796581+0000","last_clean":"2024-05-09T06:27:28.796581+0000","last_became_active":"2024-05-09T06:27:21.663959+0000","last_became_peered":"2024-05-09T06:27:21.663959+0000","last_unstale":"2024-05-09T06:27:28.796581+0000","last_undegraded":"2024-05-09T06:27:28.796581+0000","last_fullsized":"2024-05-09T06:27:28.796581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795891+0000","last_change":"2024-05-09T06:27:21.642136+0000","last_active":"2024-05-09T06:27:28.795891+0000","last_peered":"2024-05-09T06:27:28.795891+0000","last_clean":"2024-05-09T06:27:28.795891+0000","last_became_active":"2024-05-09T06:27:21.641855+0000","last_became_peered":"2024-05-09T06:27:21.641855+0000","last_unstale":"2024-05-09T06:27:28.795891+0000","last_undegraded":"2024-05-09T06:27:28.795891+0000","last_fullsized":"2024-05-09T06:27:28.795891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796263+0000","last_change":"2024-05-09T06:27:21.660067+0000","last_active":"2024-05-09T06:27:28.796263+0000","last_peered":"2024-05-09T06:27:28.796263+0000","last_clean":"2024-05-09T06:27:28.796263+0000","last_became_active":"2024-05-09T06:27:21.651525+0000","last_became_peered":"2024-05-09T06:27:21.651525+0000","last_unstale":"2024-05-09T06:27:28.796263+0000","last_undegraded":"2024-05-09T06:27:28.796263+0000","last_fullsized":"2024-05-09T06:27:28.796263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897431+0000","last_change":"2024-05-09T06:27:21.650694+0000","last_active":"2024-05-09T06:27:26.897431+0000","last_peered":"2024-05-09T06:27:26.897431+0000","last_clean":"2024-05-09T06:27:26.897431+0000","last_became_active":"2024-05-09T06:27:21.650404+0000","last_became_peered":"2024-05-09T06:27:21.650404+0000","last_unstale":"2024-05-09T06:27:26.897431+0000","last_undegraded":"2024-05-09T06:27:26.897431+0000","last_fullsized":"2024-05-09T06:27:26.897431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796327+0000","last_change":"2024-05-09T06:27:21.648723+0000","last_active":"2024-05-09T06:27:28.796327+0000","last_peered":"2024-05-09T06:27:28.796327+0000","last_clean":"2024-05-09T06:27:28.796327+0000","last_became_active":"2024-05-09T06:27:21.648434+0000","last_became_peered":"2024-05-09T06:27:21.648434+0000","last_unstale":"2024-05-09T06:27:28.796327+0000","last_undegraded":"2024-05-09T06:27:28.796327+0000","last_fullsized":"2024-05-09T06:27:28.796327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795585+0000","last_change":"2024-05-09T06:27:21.659502+0000","last_active":"2024-05-09T06:27:28.795585+0000","last_peered":"2024-05-09T06:27:28.795585+0000","last_clean":"2024-05-09T06:27:28.795585+0000","last_became_active":"2024-05-09T06:27:21.651268+0000","last_became_peered":"2024-05-09T06:27:21.651268+0000","last_unstale":"2024-05-09T06:27:28.795585+0000","last_undegraded":"2024-05-09T06:27:28.795585+0000","last_fullsized":"2024-05-09T06:27:28.795585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795603+0000","last_change":"2024-05-09T06:27:21.664539+0000","last_active":"2024-05-09T06:27:28.795603+0000","last_peered":"2024-05-09T06:27:28.795603+0000","last_clean":"2024-05-09T06:27:28.795603+0000","last_became_active":"2024-05-09T06:27:21.664267+0000","last_became_peered":"2024-05-09T06:27:21.664267+0000","last_unstale":"2024-05-09T06:27:28.795603+0000","last_undegraded":"2024-05-09T06:27:28.795603+0000","last_fullsized":"2024-05-09T06:27:28.795603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795097+0000","last_change":"2024-05-09T06:27:21.658842+0000","last_active":"2024-05-09T06:27:28.795097+0000","last_peered":"2024-05-09T06:27:28.795097+0000","last_clean":"2024-05-09T06:27:28.795097+0000","last_became_active":"2024-05-09T06:27:21.658522+0000","last_became_peered":"2024-05-09T06:27:21.658522+0000","last_unstale":"2024-05-09T06:27:28.795097+0000","last_undegraded":"2024-05-09T06:27:28.795097+0000","last_fullsized":"2024-05-09T06:27:28.795097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796440+0000","last_change":"2024-05-09T06:27:21.657936+0000","last_active":"2024-05-09T06:27:28.796440+0000","last_peered":"2024-05-09T06:27:28.796440+0000","last_clean":"2024-05-09T06:27:28.796440+0000","last_became_active":"2024-05-09T06:27:21.654771+0000","last_became_peered":"2024-05-09T06:27:21.654771+0000","last_unstale":"2024-05-09T06:27:28.796440+0000","last_undegraded":"2024-05-09T06:27:28.796440+0000","last_fullsized":"2024-05-09T06:27:28.796440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795635+0000","last_change":"2024-05-09T06:27:21.645805+0000","last_active":"2024-05-09T06:27:28.795635+0000","last_peered":"2024-05-09T06:27:28.795635+0000","last_clean":"2024-05-09T06:27:28.795635+0000","last_became_active":"2024-05-09T06:27:21.643759+0000","last_became_peered":"2024-05-09T06:27:21.643759+0000","last_unstale":"2024-05-09T06:27:28.795635+0000","last_undegraded":"2024-05-09T06:27:28.795635+0000","last_fullsized":"2024-05-09T06:27:28.795635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:12.851183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795676+0000","last_change":"2024-05-09T06:27:21.654407+0000","last_active":"2024-05-09T06:27:28.795676+0000","last_peered":"2024-05-09T06:27:28.795676+0000","last_clean":"2024-05-09T06:27:28.795676+0000","last_became_active":"2024-05-09T06:27:21.654126+0000","last_became_peered":"2024-05-09T06:27:21.654126+0000","last_unstale":"2024-05-09T06:27:28.795676+0000","last_undegraded":"2024-05-09T06:27:28.795676+0000","last_fullsized":"2024-05-09T06:27:28.795676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310344+0000","last_change":"2024-05-09T06:27:21.651773+0000","last_active":"2024-05-09T06:27:28.310344+0000","last_peered":"2024-05-09T06:27:28.310344+0000","last_clean":"2024-05-09T06:27:28.310344+0000","last_became_active":"2024-05-09T06:27:21.650069+0000","last_became_peered":"2024-05-09T06:27:21.650069+0000","last_unstale":"2024-05-09T06:27:28.310344+0000","last_undegraded":"2024-05-09T06:27:28.310344+0000","last_fullsized":"2024-05-09T06:27:28.310344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666446+0000","last_change":"2024-05-09T06:27:21.657123+0000","last_active":"2024-05-09T06:27:26.666446+0000","last_peered":"2024-05-09T06:27:26.666446+0000","last_clean":"2024-05-09T06:27:26.666446+0000","last_became_active":"2024-05-09T06:27:21.656787+0000","last_became_peered":"2024-05-09T06:27:21.656787+0000","last_unstale":"2024-05-09T06:27:26.666446+0000","last_undegraded":"2024-05-09T06:27:26.666446+0000","last_fullsized":"2024-05-09T06:27:26.666446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.797072+0000","last_change":"2024-05-09T06:27:21.660613+0000","last_active":"2024-05-09T06:27:28.797072+0000","last_peered":"2024-05-09T06:27:28.797072+0000","last_clean":"2024-05-09T06:27:28.797072+0000","last_became_active":"2024-05-09T06:27:21.654942+0000","last_became_peered":"2024-05-09T06:27:21.654942+0000","last_unstale":"2024-05-09T06:27:28.797072+0000","last_undegraded":"2024-05-09T06:27:28.797072+0000","last_fullsized":"2024-05-09T06:27:28.797072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795476+0000","last_change":"2024-05-09T06:27:21.658261+0000","last_active":"2024-05-09T06:27:28.795476+0000","last_peered":"2024-05-09T06:27:28.795476+0000","last_clean":"2024-05-09T06:27:28.795476+0000","last_became_active":"2024-05-09T06:27:21.658004+0000","last_became_peered":"2024-05-09T06:27:21.658004+0000","last_unstale":"2024-05-09T06:27:28.795476+0000","last_undegraded":"2024-05-09T06:27:28.795476+0000","last_fullsized":"2024-05-09T06:27:28.795476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897230+0000","last_change":"2024-05-09T06:27:21.654019+0000","last_active":"2024-05-09T06:27:26.897230+0000","last_peered":"2024-05-09T06:27:26.897230+0000","last_clean":"2024-05-09T06:27:26.897230+0000","last_became_active":"2024-05-09T06:27:21.653711+0000","last_became_peered":"2024-05-09T06:27:21.653711+0000","last_unstale":"2024-05-09T06:27:26.897230+0000","last_undegraded":"2024-05-09T06:27:26.897230+0000","last_fullsized":"2024-05-09T06:27:26.897230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795778+0000","last_change":"2024-05-09T06:27:21.645328+0000","last_active":"2024-05-09T06:27:28.795778+0000","last_peered":"2024-05-09T06:27:28.795778+0000","last_clean":"2024-05-09T06:27:28.795778+0000","last_became_active":"2024-05-09T06:27:21.644287+0000","last_became_peered":"2024-05-09T06:27:21.644287+0000","last_unstale":"2024-05-09T06:27:28.795778+0000","last_undegraded":"2024-05-09T06:27:28.795778+0000","last_fullsized":"2024-05-09T06:27:28.795778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:16.580568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795360+0000","last_change":"2024-05-09T06:27:21.643190+0000","last_active":"2024-05-09T06:27:28.795360+0000","last_peered":"2024-05-09T06:27:28.795360+0000","last_clean":"2024-05-09T06:27:28.795360+0000","last_became_active":"2024-05-09T06:27:21.642700+0000","last_became_peered":"2024-05-09T06:27:21.642700+0000","last_unstale":"2024-05-09T06:27:28.795360+0000","last_undegraded":"2024-05-09T06:27:28.795360+0000","last_fullsized":"2024-05-09T06:27:28.795360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667413+0000","last_change":"2024-05-09T06:27:21.652745+0000","last_active":"2024-05-09T06:27:26.667413+0000","last_peered":"2024-05-09T06:27:26.667413+0000","last_clean":"2024-05-09T06:27:26.667413+0000","last_became_active":"2024-05-09T06:27:21.651219+0000","last_became_peered":"2024-05-09T06:27:21.651219+0000","last_unstale":"2024-05-09T06:27:26.667413+0000","last_undegraded":"2024-05-09T06:27:26.667413+0000","last_fullsized":"2024-05-09T06:27:26.667413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795888+0000","last_change":"2024-05-09T06:27:21.663110+0000","last_active":"2024-05-09T06:27:28.795888+0000","last_peered":"2024-05-09T06:27:28.795888+0000","last_clean":"2024-05-09T06:27:28.795888+0000","last_became_active":"2024-05-09T06:27:21.662731+0000","last_became_peered":"2024-05-09T06:27:21.662731+0000","last_unstale":"2024-05-09T06:27:28.795888+0000","last_undegraded":"2024-05-09T06:27:28.795888+0000","last_fullsized":"2024-05-09T06:27:28.795888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796301+0000","last_change":"2024-05-09T06:27:21.659071+0000","last_active":"2024-05-09T06:27:28.796301+0000","last_peered":"2024-05-09T06:27:28.796301+0000","last_clean":"2024-05-09T06:27:28.796301+0000","last_became_active":"2024-05-09T06:27:21.658774+0000","last_became_peered":"2024-05-09T06:27:21.658774+0000","last_unstale":"2024-05-09T06:27:28.796301+0000","last_undegraded":"2024-05-09T06:27:28.796301+0000","last_fullsized":"2024-05-09T06:27:28.796301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310458+0000","last_change":"2024-05-09T06:27:21.665023+0000","last_active":"2024-05-09T06:27:28.310458+0000","last_peered":"2024-05-09T06:27:28.310458+0000","last_clean":"2024-05-09T06:27:28.310458+0000","last_became_active":"2024-05-09T06:27:21.664464+0000","last_became_peered":"2024-05-09T06:27:21.664464+0000","last_unstale":"2024-05-09T06:27:28.310458+0000","last_undegraded":"2024-05-09T06:27:28.310458+0000","last_fullsized":"2024-05-09T06:27:28.310458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795913+0000","last_change":"2024-05-09T06:27:21.661852+0000","last_active":"2024-05-09T06:27:28.795913+0000","last_peered":"2024-05-09T06:27:28.795913+0000","last_clean":"2024-05-09T06:27:28.795913+0000","last_became_active":"2024-05-09T06:27:21.660313+0000","last_became_peered":"2024-05-09T06:27:21.660313+0000","last_unstale":"2024-05-09T06:27:28.795913+0000","last_undegraded":"2024-05-09T06:27:28.795913+0000","last_fullsized":"2024-05-09T06:27:28.795913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309769+0000","last_change":"2024-05-09T06:27:21.654130+0000","last_active":"2024-05-09T06:27:28.309769+0000","last_peered":"2024-05-09T06:27:28.309769+0000","last_clean":"2024-05-09T06:27:28.309769+0000","last_became_active":"2024-05-09T06:27:21.653593+0000","last_became_peered":"2024-05-09T06:27:21.653593+0000","last_unstale":"2024-05-09T06:27:28.309769+0000","last_undegraded":"2024-05-09T06:27:28.309769+0000","last_fullsized":"2024-05-09T06:27:28.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796010+0000","last_change":"2024-05-09T06:27:21.654775+0000","last_active":"2024-05-09T06:27:28.796010+0000","last_peered":"2024-05-09T06:27:28.796010+0000","last_clean":"2024-05-09T06:27:28.796010+0000","last_became_active":"2024-05-09T06:27:21.654269+0000","last_became_peered":"2024-05-09T06:27:21.654269+0000","last_unstale":"2024-05-09T06:27:28.796010+0000","last_undegraded":"2024-05-09T06:27:28.796010+0000","last_fullsized":"2024-05-09T06:27:28.796010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309790+0000","last_change":"2024-05-09T06:27:21.649923+0000","last_active":"2024-05-09T06:27:28.309790+0000","last_peered":"2024-05-09T06:27:28.309790+0000","last_clean":"2024-05-09T06:27:28.309790+0000","last_became_active":"2024-05-09T06:27:21.648693+0000","last_became_peered":"2024-05-09T06:27:21.648693+0000","last_unstale":"2024-05-09T06:27:28.309790+0000","last_undegraded":"2024-05-09T06:27:28.309790+0000","last_fullsized":"2024-05-09T06:27:28.309790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796403+0000","last_change":"2024-05-09T06:27:21.664868+0000","last_active":"2024-05-09T06:27:28.796403+0000","last_peered":"2024-05-09T06:27:28.796403+0000","last_clean":"2024-05-09T06:27:28.796403+0000","last_became_active":"2024-05-09T06:27:21.663870+0000","last_became_peered":"2024-05-09T06:27:21.663870+0000","last_unstale":"2024-05-09T06:27:28.796403+0000","last_undegraded":"2024-05-09T06:27:28.796403+0000","last_fullsized":"2024-05-09T06:27:28.796403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796427+0000","last_change":"2024-05-09T06:27:21.651232+0000","last_active":"2024-05-09T06:27:28.796427+0000","last_peered":"2024-05-09T06:27:28.796427+0000","last_clean":"2024-05-09T06:27:28.796427+0000","last_became_active":"2024-05-09T06:27:21.650940+0000","last_became_peered":"2024-05-09T06:27:21.650940+0000","last_unstale":"2024-05-09T06:27:28.796427+0000","last_undegraded":"2024-05-09T06:27:28.796427+0000","last_fullsized":"2024-05-09T06:27:28.796427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897594+0000","last_change":"2024-05-09T06:27:21.648169+0000","last_active":"2024-05-09T06:27:26.897594+0000","last_peered":"2024-05-09T06:27:26.897594+0000","last_clean":"2024-05-09T06:27:26.897594+0000","last_became_active":"2024-05-09T06:27:21.647846+0000","last_became_peered":"2024-05-09T06:27:21.647846+0000","last_unstale":"2024-05-09T06:27:26.897594+0000","last_undegraded":"2024-05-09T06:27:26.897594+0000","last_fullsized":"2024-05-09T06:27:26.897594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310386+0000","last_change":"2024-05-09T06:27:21.649391+0000","last_active":"2024-05-09T06:27:28.310386+0000","last_peered":"2024-05-09T06:27:28.310386+0000","last_clean":"2024-05-09T06:27:28.310386+0000","last_became_active":"2024-05-09T06:27:21.648828+0000","last_became_peered":"2024-05-09T06:27:21.648828+0000","last_unstale":"2024-05-09T06:27:28.310386+0000","last_undegraded":"2024-05-09T06:27:28.310386+0000","last_fullsized":"2024-05-09T06:27:28.310386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:27:44.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:27:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.910+0000 7f09d4d16640 1 Processor -- start 2024-05-09T06:27:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.910+0000 7f09d4d16640 1 -- start start 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.911+0000 7f09d4d16640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f09d0161950 0x7f09d00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.911+0000 7f09d4d16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.911+0000 7f09d4d16640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d00b33c0 con 0x7f09d00d0f80 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.911+0000 7f09d4d16640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f09d00d4dd0 con 0x7f09d0161950 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.911+0000 7f09d4d16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09d00b3660 con 0x7f09d00d4940 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:32940/0 (socket says 172.21.15.1:32940) 2024-05-09T06:27:44.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 -- 172.21.15.1:0/4020650441 learned_addr learned my addr 172.21.15.1:0/4020650441 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:44.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09ce59c640 1 --2- 172.21.15.1:0/4020650441 >> v2:172.21.15.6:3300/0 conn(0x7f09d0161950 0x7f09d00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 -- 172.21.15.1:0/4020650441 >> v2:172.21.15.6:3300/0 conn(0x7f09d0161950 msgr2=0x7f09d00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:44.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 --2- 172.21.15.1:0/4020650441 >> v2:172.21.15.6:3300/0 conn(0x7f09d0161950 0x7f09d00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 -- 172.21.15.1:0/4020650441 >> v1:172.21.15.6:6789/0 conn(0x7f09d00d0f80 legacy=0x7f09d00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.912+0000 7f09cdd9b640 1 -- 172.21.15.1:0/4020650441 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d00b3940 con 0x7f09d00d4940 2024-05-09T06:27:44.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09cdd9b640 1 --2- 172.21.15.1:0/4020650441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f09d00b3590 tx=0x7f09b802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2648f77fe7beb592 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09cd59a640 1 -- 172.21.15.1:0/4020650441 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8037070 con 0x7f09d00d4940 2024-05-09T06:27:44.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09cd59a640 1 -- 172.21.15.1:0/4020650441 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b8032440 con 0x7f09d00d4940 2024-05-09T06:27:44.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09cd59a640 1 -- 172.21.15.1:0/4020650441 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8032810 con 0x7f09d00d4940 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09d4d16640 1 -- 172.21.15.1:0/4020650441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 msgr2=0x7f09d00d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.913+0000 7f09d4d16640 1 --2- 172.21.15.1:0/4020650441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f09d00b3590 tx=0x7f09b802a250 comp rx=0 tx=0).stop 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 -- 172.21.15.1:0/4020650441 shutdown_connections 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 --2- 172.21.15.1:0/4020650441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d00d4940 0x7f09d00d2940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 --2- 172.21.15.1:0/4020650441 >> v2:172.21.15.6:3300/0 conn(0x7f09d0161950 0x7f09d00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 -- 172.21.15.1:0/4020650441 >> 172.21.15.1:0/4020650441 conn(0x7f09d00d3e20 msgr2=0x7f09d00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 -- 172.21.15.1:0/4020650441 shutdown_connections 2024-05-09T06:27:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.914+0000 7f09d4d16640 1 -- 172.21.15.1:0/4020650441 wait complete. 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.915+0000 7f09d4d16640 1 Processor -- start 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.915+0000 7f09d4d16640 1 -- start start 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09d00d0f80 0x7f09d01e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 0x7f09d00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09d00b52d0 con 0x7f09d00cce90 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f09d00b5150 con 0x7f09d00d0f80 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09d4d16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09d00b5450 con 0x7f09d0161950 2024-05-09T06:27:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53120/0 (socket says 172.21.15.1:53120) 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 -- 172.21.15.1:0/691960498 learned_addr learned my addr 172.21.15.1:0/691960498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ce59c640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09d00d0f80 0x7f09d01e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09cdd9b640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 0x7f09d00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 -- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 msgr2=0x7f09d00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 0x7f09d00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 -- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09d00d0f80 msgr2=0x7f09d01e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09d00d0f80 0x7f09d01e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 -- 172.21.15.1:0/691960498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d00cd560 con 0x7f09d00cce90 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09cdd9b640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 0x7f09d00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09ced9d640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c3a0 tx=0x7f09c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.916+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0017070 con 0x7f09d00cce90 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.917+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c00052d0 con 0x7f09d00cce90 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.917+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d00cd840 con 0x7f09d00cce90 2024-05-09T06:27:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.917+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003830 con 0x7f09d00cce90 2024-05-09T06:27:44.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.917+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d014bc40 con 0x7f09d00cce90 2024-05-09T06:27:44.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.918+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d00752b0 con 0x7f09d00cce90 2024-05-09T06:27:44.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.918+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09c001a070 con 0x7f09d00cce90 2024-05-09T06:27:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.918+0000 7f09c6ffd640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 0x7f09b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.919+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f09c008a480 con 0x7f09d00cce90 2024-05-09T06:27:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.919+0000 7f09ce59c640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 0x7f09b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.919+0000 7f09ce59c640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 0x7f09b006e320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09c80073f0 tx=0x7f09c800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:44.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:44.923+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09c0058c40 con 0x7f09d00cce90 2024-05-09T06:27:45.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.102+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09d014b610 con 0x7f09b006be60 2024-05-09T06:27:45.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.110+0000 7f09c6ffd640 1 -- 172.21.15.1:0/691960498 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f09d014b610 con 0x7f09b006be60 2024-05-09T06:27:45.096 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:45.097 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 msgr2=0x7f09b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 0x7f09b006e320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09c80073f0 tx=0x7f09c800b040 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 msgr2=0x7f09d014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c3a0 tx=0x7f09c000c860 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 shutdown_connections 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006be60 0x7f09b006e320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09d00cce90 0x7f09d014d8e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09d0161950 0x7f09d00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 --2- 172.21.15.1:0/691960498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09d00d0f80 0x7f09d01e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.113+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 >> 172.21.15.1:0/691960498 conn(0x7f09d00d3e20 msgr2=0x7f09d00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.114+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 shutdown_connections 2024-05-09T06:27:45.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.114+0000 7f09d4d16640 1 -- 172.21.15.1:0/691960498 wait complete. 2024-05-09T06:27:45.107 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-09T06:27:45.073414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002624"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795027+0000","last_change":"2024-05-09T06:27:20.644248+0000","last_active":"2024-05-09T06:27:28.795027+0000","last_peered":"2024-05-09T06:27:28.795027+0000","last_clean":"2024-05-09T06:27:28.795027+0000","last_became_active":"2024-05-09T06:27:20.643791+0000","last_became_peered":"2024-05-09T06:27:20.643791+0000","last_unstale":"2024-05-09T06:27:28.795027+0000","last_undegraded":"2024-05-09T06:27:28.795027+0000","last_fullsized":"2024-05-09T06:27:28.795027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:48.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795308+0000","last_change":"2024-05-09T06:27:21.649501+0000","last_active":"2024-05-09T06:27:28.795308+0000","last_peered":"2024-05-09T06:27:28.795308+0000","last_clean":"2024-05-09T06:27:28.795308+0000","last_became_active":"2024-05-09T06:27:21.649193+0000","last_became_peered":"2024-05-09T06:27:21.649193+0000","last_unstale":"2024-05-09T06:27:28.795308+0000","last_undegraded":"2024-05-09T06:27:28.795308+0000","last_fullsized":"2024-05-09T06:27:28.795308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794757+0000","last_change":"2024-05-09T06:27:25.871323+0000","last_active":"2024-05-09T06:27:28.794757+0000","last_peered":"2024-05-09T06:27:28.794757+0000","last_clean":"2024-05-09T06:27:28.794757+0000","last_became_active":"2024-05-09T06:27:25.870947+0000","last_became_peered":"2024-05-09T06:27:25.870947+0000","last_unstale":"2024-05-09T06:27:28.794757+0000","last_undegraded":"2024-05-09T06:27:28.794757+0000","last_fullsized":"2024-05-09T06:27:28.794757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795538+0000","last_change":"2024-05-09T06:27:20.634799+0000","last_active":"2024-05-09T06:27:28.795538+0000","last_peered":"2024-05-09T06:27:28.795538+0000","last_clean":"2024-05-09T06:27:28.795538+0000","last_became_active":"2024-05-09T06:27:20.634300+0000","last_became_peered":"2024-05-09T06:27:20.634300+0000","last_unstale":"2024-05-09T06:27:28.795538+0000","last_undegraded":"2024-05-09T06:27:28.795538+0000","last_fullsized":"2024-05-09T06:27:28.795538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796489+0000","last_change":"2024-05-09T06:27:21.638390+0000","last_active":"2024-05-09T06:27:28.796489+0000","last_peered":"2024-05-09T06:27:28.796489+0000","last_clean":"2024-05-09T06:27:28.796489+0000","last_became_active":"2024-05-09T06:27:21.637720+0000","last_became_peered":"2024-05-09T06:27:21.637720+0000","last_unstale":"2024-05-09T06:27:28.796489+0000","last_undegraded":"2024-05-09T06:27:28.796489+0000","last_fullsized":"2024-05-09T06:27:28.796489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666410+0000","last_change":"2024-05-09T06:27:25.668634+0000","last_active":"2024-05-09T06:27:26.666410+0000","last_peered":"2024-05-09T06:27:26.666410+0000","last_clean":"2024-05-09T06:27:26.666410+0000","last_became_active":"2024-05-09T06:27:25.668153+0000","last_became_peered":"2024-05-09T06:27:25.668153+0000","last_unstale":"2024-05-09T06:27:26.666410+0000","last_undegraded":"2024-05-09T06:27:26.666410+0000","last_fullsized":"2024-05-09T06:27:26.666410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:46.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310311+0000","last_change":"2024-05-09T06:27:20.646888+0000","last_active":"2024-05-09T06:27:28.310311+0000","last_peered":"2024-05-09T06:27:28.310311+0000","last_clean":"2024-05-09T06:27:28.310311+0000","last_became_active":"2024-05-09T06:27:20.646603+0000","last_became_peered":"2024-05-09T06:27:20.646603+0000","last_unstale":"2024-05-09T06:27:28.310311+0000","last_undegraded":"2024-05-09T06:27:28.310311+0000","last_fullsized":"2024-05-09T06:27:28.310311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:06.656512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796678+0000","last_change":"2024-05-09T06:27:21.659688+0000","last_active":"2024-05-09T06:27:28.796678+0000","last_peered":"2024-05-09T06:27:28.796678+0000","last_clean":"2024-05-09T06:27:28.796678+0000","last_became_active":"2024-05-09T06:27:21.659407+0000","last_became_peered":"2024-05-09T06:27:21.659407+0000","last_unstale":"2024-05-09T06:27:28.796678+0000","last_undegraded":"2024-05-09T06:27:28.796678+0000","last_fullsized":"2024-05-09T06:27:28.796678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667001+0000","last_change":"2024-05-09T06:27:25.674815+0000","last_active":"2024-05-09T06:27:26.667001+0000","last_peered":"2024-05-09T06:27:26.667001+0000","last_clean":"2024-05-09T06:27:26.667001+0000","last_became_active":"2024-05-09T06:27:25.673855+0000","last_became_peered":"2024-05-09T06:27:25.673855+0000","last_unstale":"2024-05-09T06:27:26.667001+0000","last_undegraded":"2024-05-09T06:27:26.667001+0000","last_fullsized":"2024-05-09T06:27:26.667001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:16.160631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309929+0000","last_change":"2024-05-09T06:27:20.644933+0000","last_active":"2024-05-09T06:27:28.309929+0000","last_peered":"2024-05-09T06:27:28.309929+0000","last_clean":"2024-05-09T06:27:28.309929+0000","last_became_active":"2024-05-09T06:27:20.644019+0000","last_became_peered":"2024-05-09T06:27:20.644019+0000","last_unstale":"2024-05-09T06:27:28.309929+0000","last_undegraded":"2024-05-09T06:27:28.309929+0000","last_fullsized":"2024-05-09T06:27:28.309929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796776+0000","last_change":"2024-05-09T06:27:21.660829+0000","last_active":"2024-05-09T06:27:28.796776+0000","last_peered":"2024-05-09T06:27:28.796776+0000","last_clean":"2024-05-09T06:27:28.796776+0000","last_became_active":"2024-05-09T06:27:21.658349+0000","last_became_peered":"2024-05-09T06:27:21.658349+0000","last_unstale":"2024-05-09T06:27:28.796776+0000","last_undegraded":"2024-05-09T06:27:28.796776+0000","last_fullsized":"2024-05-09T06:27:28.796776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666466+0000","last_change":"2024-05-09T06:27:25.664257+0000","last_active":"2024-05-09T06:27:26.666466+0000","last_peered":"2024-05-09T06:27:26.666466+0000","last_clean":"2024-05-09T06:27:26.666466+0000","last_became_active":"2024-05-09T06:27:25.663787+0000","last_became_peered":"2024-05-09T06:27:25.663787+0000","last_unstale":"2024-05-09T06:27:26.666466+0000","last_undegraded":"2024-05-09T06:27:26.666466+0000","last_fullsized":"2024-05-09T06:27:26.666466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:57.007200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795391+0000","last_change":"2024-05-09T06:27:21.652107+0000","last_active":"2024-05-09T06:27:28.795391+0000","last_peered":"2024-05-09T06:27:28.795391+0000","last_clean":"2024-05-09T06:27:28.795391+0000","last_became_active":"2024-05-09T06:27:21.651764+0000","last_became_peered":"2024-05-09T06:27:21.651764+0000","last_unstale":"2024-05-09T06:27:28.795391+0000","last_undegraded":"2024-05-09T06:27:28.795391+0000","last_fullsized":"2024-05-09T06:27:28.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310074+0000","last_change":"2024-05-09T06:27:20.644636+0000","last_active":"2024-05-09T06:27:28.310074+0000","last_peered":"2024-05-09T06:27:28.310074+0000","last_clean":"2024-05-09T06:27:28.310074+0000","last_became_active":"2024-05-09T06:27:20.644155+0000","last_became_peered":"2024-05-09T06:27:20.644155+0000","last_unstale":"2024-05-09T06:27:28.310074+0000","last_undegraded":"2024-05-09T06:27:28.310074+0000","last_fullsized":"2024-05-09T06:27:28.310074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667637+0000","last_change":"2024-05-09T06:27:25.674517+0000","last_active":"2024-05-09T06:27:26.667637+0000","last_peered":"2024-05-09T06:27:26.667637+0000","last_clean":"2024-05-09T06:27:26.667637+0000","last_became_active":"2024-05-09T06:27:25.673256+0000","last_became_peered":"2024-05-09T06:27:25.673256+0000","last_unstale":"2024-05-09T06:27:26.667637+0000","last_undegraded":"2024-05-09T06:27:26.667637+0000","last_fullsized":"2024-05-09T06:27:26.667637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795461+0000","last_change":"2024-05-09T06:27:21.650596+0000","last_active":"2024-05-09T06:27:28.795461+0000","last_peered":"2024-05-09T06:27:28.795461+0000","last_clean":"2024-05-09T06:27:28.795461+0000","last_became_active":"2024-05-09T06:27:21.649943+0000","last_became_peered":"2024-05-09T06:27:21.649943+0000","last_unstale":"2024-05-09T06:27:28.795461+0000","last_undegraded":"2024-05-09T06:27:28.795461+0000","last_fullsized":"2024-05-09T06:27:28.795461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897815+0000","last_change":"2024-05-09T06:27:20.634659+0000","last_active":"2024-05-09T06:27:26.897815+0000","last_peered":"2024-05-09T06:27:26.897815+0000","last_clean":"2024-05-09T06:27:26.897815+0000","last_became_active":"2024-05-09T06:27:20.634344+0000","last_became_peered":"2024-05-09T06:27:20.634344+0000","last_unstale":"2024-05-09T06:27:26.897815+0000","last_undegraded":"2024-05-09T06:27:26.897815+0000","last_fullsized":"2024-05-09T06:27:26.897815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667419+0000","last_change":"2024-05-09T06:27:25.674355+0000","last_active":"2024-05-09T06:27:26.667419+0000","last_peered":"2024-05-09T06:27:26.667419+0000","last_clean":"2024-05-09T06:27:26.667419+0000","last_became_active":"2024-05-09T06:27:25.673950+0000","last_became_peered":"2024-05-09T06:27:25.673950+0000","last_unstale":"2024-05-09T06:27:26.667419+0000","last_undegraded":"2024-05-09T06:27:26.667419+0000","last_fullsized":"2024-05-09T06:27:26.667419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.231985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796292+0000","last_change":"2024-05-09T06:27:21.641405+0000","last_active":"2024-05-09T06:27:28.796292+0000","last_peered":"2024-05-09T06:27:28.796292+0000","last_clean":"2024-05-09T06:27:28.796292+0000","last_became_active":"2024-05-09T06:27:21.641019+0000","last_became_peered":"2024-05-09T06:27:21.641019+0000","last_unstale":"2024-05-09T06:27:28.796292+0000","last_undegraded":"2024-05-09T06:27:28.796292+0000","last_fullsized":"2024-05-09T06:27:28.796292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897255+0000","last_change":"2024-05-09T06:27:20.644807+0000","last_active":"2024-05-09T06:27:26.897255+0000","last_peered":"2024-05-09T06:27:26.897255+0000","last_clean":"2024-05-09T06:27:26.897255+0000","last_became_active":"2024-05-09T06:27:20.644510+0000","last_became_peered":"2024-05-09T06:27:20.644510+0000","last_unstale":"2024-05-09T06:27:26.897255+0000","last_undegraded":"2024-05-09T06:27:26.897255+0000","last_fullsized":"2024-05-09T06:27:26.897255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897895+0000","last_change":"2024-05-09T06:27:25.867962+0000","last_active":"2024-05-09T06:27:26.897895+0000","last_peered":"2024-05-09T06:27:26.897895+0000","last_clean":"2024-05-09T06:27:26.897895+0000","last_became_active":"2024-05-09T06:27:25.867593+0000","last_became_peered":"2024-05-09T06:27:25.867593+0000","last_unstale":"2024-05-09T06:27:26.897895+0000","last_undegraded":"2024-05-09T06:27:26.897895+0000","last_fullsized":"2024-05-09T06:27:26.897895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666600+0000","last_change":"2024-05-09T06:27:21.651627+0000","last_active":"2024-05-09T06:27:26.666600+0000","last_peered":"2024-05-09T06:27:26.666600+0000","last_clean":"2024-05-09T06:27:26.666600+0000","last_became_active":"2024-05-09T06:27:21.651021+0000","last_became_peered":"2024-05-09T06:27:21.651021+0000","last_unstale":"2024-05-09T06:27:26.666600+0000","last_undegraded":"2024-05-09T06:27:26.666600+0000","last_fullsized":"2024-05-09T06:27:26.666600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:04.963108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309528+0000","last_change":"2024-05-09T06:27:20.644691+0000","last_active":"2024-05-09T06:27:28.309528+0000","last_peered":"2024-05-09T06:27:28.309528+0000","last_clean":"2024-05-09T06:27:28.309528+0000","last_became_active":"2024-05-09T06:27:20.644291+0000","last_became_peered":"2024-05-09T06:27:20.644291+0000","last_unstale":"2024-05-09T06:27:28.309528+0000","last_undegraded":"2024-05-09T06:27:28.309528+0000","last_fullsized":"2024-05-09T06:27:28.309528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667629+0000","last_change":"2024-05-09T06:27:25.670582+0000","last_active":"2024-05-09T06:27:26.667629+0000","last_peered":"2024-05-09T06:27:26.667629+0000","last_clean":"2024-05-09T06:27:26.667629+0000","last_became_active":"2024-05-09T06:27:25.670180+0000","last_became_peered":"2024-05-09T06:27:25.670180+0000","last_unstale":"2024-05-09T06:27:26.667629+0000","last_undegraded":"2024-05-09T06:27:26.667629+0000","last_fullsized":"2024-05-09T06:27:26.667629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796762+0000","last_change":"2024-05-09T06:27:21.644419+0000","last_active":"2024-05-09T06:27:28.796762+0000","last_peered":"2024-05-09T06:27:28.796762+0000","last_clean":"2024-05-09T06:27:28.796762+0000","last_became_active":"2024-05-09T06:27:21.644054+0000","last_became_peered":"2024-05-09T06:27:21.644054+0000","last_unstale":"2024-05-09T06:27:28.796762+0000","last_undegraded":"2024-05-09T06:27:28.796762+0000","last_fullsized":"2024-05-09T06:27:28.796762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795166+0000","last_change":"2024-05-09T06:27:20.644413+0000","last_active":"2024-05-09T06:27:28.795166+0000","last_peered":"2024-05-09T06:27:28.795166+0000","last_clean":"2024-05-09T06:27:28.795166+0000","last_became_active":"2024-05-09T06:27:20.644063+0000","last_became_peered":"2024-05-09T06:27:20.644063+0000","last_unstale":"2024-05-09T06:27:28.795166+0000","last_undegraded":"2024-05-09T06:27:28.795166+0000","last_fullsized":"2024-05-09T06:27:28.795166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666413+0000","last_change":"2024-05-09T06:27:25.868219+0000","last_active":"2024-05-09T06:27:26.666413+0000","last_peered":"2024-05-09T06:27:26.666413+0000","last_clean":"2024-05-09T06:27:26.666413+0000","last_became_active":"2024-05-09T06:27:25.867618+0000","last_became_peered":"2024-05-09T06:27:25.867618+0000","last_unstale":"2024-05-09T06:27:26.666413+0000","last_undegraded":"2024-05-09T06:27:26.666413+0000","last_fullsized":"2024-05-09T06:27:26.666413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:59.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795981+0000","last_change":"2024-05-09T06:27:21.659207+0000","last_active":"2024-05-09T06:27:28.795981+0000","last_peered":"2024-05-09T06:27:28.795981+0000","last_clean":"2024-05-09T06:27:28.795981+0000","last_became_active":"2024-05-09T06:27:21.651367+0000","last_became_peered":"2024-05-09T06:27:21.651367+0000","last_unstale":"2024-05-09T06:27:28.795981+0000","last_undegraded":"2024-05-09T06:27:28.795981+0000","last_fullsized":"2024-05-09T06:27:28.795981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897683+0000","last_change":"2024-05-09T06:27:20.637053+0000","last_active":"2024-05-09T06:27:26.897683+0000","last_peered":"2024-05-09T06:27:26.897683+0000","last_clean":"2024-05-09T06:27:26.897683+0000","last_became_active":"2024-05-09T06:27:20.636755+0000","last_became_peered":"2024-05-09T06:27:20.636755+0000","last_unstale":"2024-05-09T06:27:26.897683+0000","last_undegraded":"2024-05-09T06:27:26.897683+0000","last_fullsized":"2024-05-09T06:27:26.897683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795768+0000","last_change":"2024-05-09T06:27:25.871255+0000","last_active":"2024-05-09T06:27:28.795768+0000","last_peered":"2024-05-09T06:27:28.795768+0000","last_clean":"2024-05-09T06:27:28.795768+0000","last_became_active":"2024-05-09T06:27:25.870862+0000","last_became_peered":"2024-05-09T06:27:25.870862+0000","last_unstale":"2024-05-09T06:27:28.795768+0000","last_undegraded":"2024-05-09T06:27:28.795768+0000","last_fullsized":"2024-05-09T06:27:28.795768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:53.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897799+0000","last_change":"2024-05-09T06:27:21.649868+0000","last_active":"2024-05-09T06:27:26.897799+0000","last_peered":"2024-05-09T06:27:26.897799+0000","last_clean":"2024-05-09T06:27:26.897799+0000","last_became_active":"2024-05-09T06:27:21.649515+0000","last_became_peered":"2024-05-09T06:27:21.649515+0000","last_unstale":"2024-05-09T06:27:26.897799+0000","last_undegraded":"2024-05-09T06:27:26.897799+0000","last_fullsized":"2024-05-09T06:27:26.897799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310415+0000","last_change":"2024-05-09T06:27:20.644414+0000","last_active":"2024-05-09T06:27:28.310415+0000","last_peered":"2024-05-09T06:27:28.310415+0000","last_clean":"2024-05-09T06:27:28.310415+0000","last_became_active":"2024-05-09T06:27:20.644077+0000","last_became_peered":"2024-05-09T06:27:20.644077+0000","last_unstale":"2024-05-09T06:27:28.310415+0000","last_undegraded":"2024-05-09T06:27:28.310415+0000","last_fullsized":"2024-05-09T06:27:28.310415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:34.718775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795535+0000","last_change":"2024-05-09T06:27:25.867794+0000","last_active":"2024-05-09T06:27:28.795535+0000","last_peered":"2024-05-09T06:27:28.795535+0000","last_clean":"2024-05-09T06:27:28.795535+0000","last_became_active":"2024-05-09T06:27:25.867351+0000","last_became_peered":"2024-05-09T06:27:25.867351+0000","last_unstale":"2024-05-09T06:27:28.795535+0000","last_undegraded":"2024-05-09T06:27:28.795535+0000","last_fullsized":"2024-05-09T06:27:28.795535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796829+0000","last_change":"2024-05-09T06:27:21.662543+0000","last_active":"2024-05-09T06:27:28.796829+0000","last_peered":"2024-05-09T06:27:28.796829+0000","last_clean":"2024-05-09T06:27:28.796829+0000","last_became_active":"2024-05-09T06:27:21.662266+0000","last_became_peered":"2024-05-09T06:27:21.662266+0000","last_unstale":"2024-05-09T06:27:28.796829+0000","last_undegraded":"2024-05-09T06:27:28.796829+0000","last_fullsized":"2024-05-09T06:27:28.796829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897904+0000","last_change":"2024-05-09T06:27:20.634145+0000","last_active":"2024-05-09T06:27:26.897904+0000","last_peered":"2024-05-09T06:27:26.897904+0000","last_clean":"2024-05-09T06:27:26.897904+0000","last_became_active":"2024-05-09T06:27:20.633864+0000","last_became_peered":"2024-05-09T06:27:20.633864+0000","last_unstale":"2024-05-09T06:27:26.897904+0000","last_undegraded":"2024-05-09T06:27:26.897904+0000","last_fullsized":"2024-05-09T06:27:26.897904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897405+0000","last_change":"2024-05-09T06:27:25.672524+0000","last_active":"2024-05-09T06:27:26.897405+0000","last_peered":"2024-05-09T06:27:26.897405+0000","last_clean":"2024-05-09T06:27:26.897405+0000","last_became_active":"2024-05-09T06:27:25.672004+0000","last_became_peered":"2024-05-09T06:27:25.672004+0000","last_unstale":"2024-05-09T06:27:26.897405+0000","last_undegraded":"2024-05-09T06:27:26.897405+0000","last_fullsized":"2024-05-09T06:27:26.897405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667430+0000","last_change":"2024-05-09T06:27:21.634887+0000","last_active":"2024-05-09T06:27:26.667430+0000","last_peered":"2024-05-09T06:27:26.667430+0000","last_clean":"2024-05-09T06:27:26.667430+0000","last_became_active":"2024-05-09T06:27:21.634609+0000","last_became_peered":"2024-05-09T06:27:21.634609+0000","last_unstale":"2024-05-09T06:27:26.667430+0000","last_undegraded":"2024-05-09T06:27:26.667430+0000","last_fullsized":"2024-05-09T06:27:26.667430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:59.874418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795267+0000","last_change":"2024-05-09T06:27:20.642066+0000","last_active":"2024-05-09T06:27:28.795267+0000","last_peered":"2024-05-09T06:27:28.795267+0000","last_clean":"2024-05-09T06:27:28.795267+0000","last_became_active":"2024-05-09T06:27:20.641753+0000","last_became_peered":"2024-05-09T06:27:20.641753+0000","last_unstale":"2024-05-09T06:27:28.795267+0000","last_undegraded":"2024-05-09T06:27:28.795267+0000","last_fullsized":"2024-05-09T06:27:28.795267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897884+0000","last_change":"2024-05-09T06:27:25.671230+0000","last_active":"2024-05-09T06:27:26.897884+0000","last_peered":"2024-05-09T06:27:26.897884+0000","last_clean":"2024-05-09T06:27:26.897884+0000","last_became_active":"2024-05-09T06:27:25.670778+0000","last_became_peered":"2024-05-09T06:27:25.670778+0000","last_unstale":"2024-05-09T06:27:26.897884+0000","last_undegraded":"2024-05-09T06:27:26.897884+0000","last_fullsized":"2024-05-09T06:27:26.897884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:20:39.476936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897252+0000","last_change":"2024-05-09T06:27:21.649626+0000","last_active":"2024-05-09T06:27:26.897252+0000","last_peered":"2024-05-09T06:27:26.897252+0000","last_clean":"2024-05-09T06:27:26.897252+0000","last_became_active":"2024-05-09T06:27:21.649332+0000","last_became_peered":"2024-05-09T06:27:21.649332+0000","last_unstale":"2024-05-09T06:27:26.897252+0000","last_undegraded":"2024-05-09T06:27:26.897252+0000","last_fullsized":"2024-05-09T06:27:26.897252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665959+0000","last_change":"2024-05-09T06:27:20.632572+0000","last_active":"2024-05-09T06:27:26.665959+0000","last_peered":"2024-05-09T06:27:26.665959+0000","last_clean":"2024-05-09T06:27:26.665959+0000","last_became_active":"2024-05-09T06:27:20.630286+0000","last_became_peered":"2024-05-09T06:27:20.630286+0000","last_unstale":"2024-05-09T06:27:26.665959+0000","last_undegraded":"2024-05-09T06:27:26.665959+0000","last_fullsized":"2024-05-09T06:27:26.665959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897808+0000","last_change":"2024-05-09T06:27:25.678228+0000","last_active":"2024-05-09T06:27:26.897808+0000","last_peered":"2024-05-09T06:27:26.897808+0000","last_clean":"2024-05-09T06:27:26.897808+0000","last_became_active":"2024-05-09T06:27:25.677638+0000","last_became_peered":"2024-05-09T06:27:25.677638+0000","last_unstale":"2024-05-09T06:27:26.897808+0000","last_undegraded":"2024-05-09T06:27:26.897808+0000","last_fullsized":"2024-05-09T06:27:26.897808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794917+0000","last_change":"2024-05-09T06:27:21.634999+0000","last_active":"2024-05-09T06:27:28.794917+0000","last_peered":"2024-05-09T06:27:28.794917+0000","last_clean":"2024-05-09T06:27:28.794917+0000","last_became_active":"2024-05-09T06:27:21.634688+0000","last_became_peered":"2024-05-09T06:27:21.634688+0000","last_unstale":"2024-05-09T06:27:28.794917+0000","last_undegraded":"2024-05-09T06:27:28.794917+0000","last_fullsized":"2024-05-09T06:27:28.794917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309529+0000","last_change":"2024-05-09T06:27:20.645301+0000","last_active":"2024-05-09T06:27:28.309529+0000","last_peered":"2024-05-09T06:27:28.309529+0000","last_clean":"2024-05-09T06:27:28.309529+0000","last_became_active":"2024-05-09T06:27:20.644309+0000","last_became_peered":"2024-05-09T06:27:20.644309+0000","last_unstale":"2024-05-09T06:27:28.309529+0000","last_undegraded":"2024-05-09T06:27:28.309529+0000","last_fullsized":"2024-05-09T06:27:28.309529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796649+0000","last_change":"2024-05-09T06:27:25.870805+0000","last_active":"2024-05-09T06:27:28.796649+0000","last_peered":"2024-05-09T06:27:28.796649+0000","last_clean":"2024-05-09T06:27:28.796649+0000","last_became_active":"2024-05-09T06:27:25.870405+0000","last_became_peered":"2024-05-09T06:27:25.870405+0000","last_unstale":"2024-05-09T06:27:28.796649+0000","last_undegraded":"2024-05-09T06:27:28.796649+0000","last_fullsized":"2024-05-09T06:27:28.796649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795597+0000","last_change":"2024-05-09T06:27:21.660077+0000","last_active":"2024-05-09T06:27:28.795597+0000","last_peered":"2024-05-09T06:27:28.795597+0000","last_clean":"2024-05-09T06:27:28.795597+0000","last_became_active":"2024-05-09T06:27:21.659816+0000","last_became_peered":"2024-05-09T06:27:21.659816+0000","last_unstale":"2024-05-09T06:27:28.795597+0000","last_undegraded":"2024-05-09T06:27:28.795597+0000","last_fullsized":"2024-05-09T06:27:28.795597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794955+0000","last_change":"2024-05-09T06:27:20.638709+0000","last_active":"2024-05-09T06:27:28.794955+0000","last_peered":"2024-05-09T06:27:28.794955+0000","last_clean":"2024-05-09T06:27:28.794955+0000","last_became_active":"2024-05-09T06:27:20.638404+0000","last_became_peered":"2024-05-09T06:27:20.638404+0000","last_unstale":"2024-05-09T06:27:28.794955+0000","last_undegraded":"2024-05-09T06:27:28.794955+0000","last_fullsized":"2024-05-09T06:27:28.794955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309694+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:27:28.309694+0000","last_peered":"2024-05-09T06:27:28.309694+0000","last_clean":"2024-05-09T06:27:28.309694+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:27:28.309694+0000","last_undegraded":"2024-05-09T06:27:28.309694+0000","last_fullsized":"2024-05-09T06:27:28.309694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795720+0000","last_change":"2024-05-09T06:27:25.871420+0000","last_active":"2024-05-09T06:27:28.795720+0000","last_peered":"2024-05-09T06:27:28.795720+0000","last_clean":"2024-05-09T06:27:28.795720+0000","last_became_active":"2024-05-09T06:27:25.870765+0000","last_became_peered":"2024-05-09T06:27:25.870765+0000","last_unstale":"2024-05-09T06:27:28.795720+0000","last_undegraded":"2024-05-09T06:27:28.795720+0000","last_fullsized":"2024-05-09T06:27:28.795720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310010+0000","last_change":"2024-05-09T06:27:21.665731+0000","last_active":"2024-05-09T06:27:28.310010+0000","last_peered":"2024-05-09T06:27:28.310010+0000","last_clean":"2024-05-09T06:27:28.310010+0000","last_became_active":"2024-05-09T06:27:21.665457+0000","last_became_peered":"2024-05-09T06:27:21.665457+0000","last_unstale":"2024-05-09T06:27:28.310010+0000","last_undegraded":"2024-05-09T06:27:28.310010+0000","last_fullsized":"2024-05-09T06:27:28.310010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666314+0000","last_change":"2024-05-09T06:27:20.639195+0000","last_active":"2024-05-09T06:27:26.666314+0000","last_peered":"2024-05-09T06:27:26.666314+0000","last_clean":"2024-05-09T06:27:26.666314+0000","last_became_active":"2024-05-09T06:27:20.638833+0000","last_became_peered":"2024-05-09T06:27:20.638833+0000","last_unstale":"2024-05-09T06:27:26.666314+0000","last_undegraded":"2024-05-09T06:27:26.666314+0000","last_fullsized":"2024-05-09T06:27:26.666314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795939+0000","last_change":"2024-05-09T06:27:25.868481+0000","last_active":"2024-05-09T06:27:28.795939+0000","last_peered":"2024-05-09T06:27:28.795939+0000","last_clean":"2024-05-09T06:27:28.795939+0000","last_became_active":"2024-05-09T06:27:25.867521+0000","last_became_peered":"2024-05-09T06:27:25.867521+0000","last_unstale":"2024-05-09T06:27:28.795939+0000","last_undegraded":"2024-05-09T06:27:28.795939+0000","last_fullsized":"2024-05-09T06:27:28.795939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796554+0000","last_change":"2024-05-09T06:27:21.663211+0000","last_active":"2024-05-09T06:27:28.796554+0000","last_peered":"2024-05-09T06:27:28.796554+0000","last_clean":"2024-05-09T06:27:28.796554+0000","last_became_active":"2024-05-09T06:27:21.662892+0000","last_became_peered":"2024-05-09T06:27:21.662892+0000","last_unstale":"2024-05-09T06:27:28.796554+0000","last_undegraded":"2024-05-09T06:27:28.796554+0000","last_fullsized":"2024-05-09T06:27:28.796554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897750+0000","last_change":"2024-05-09T06:27:20.638780+0000","last_active":"2024-05-09T06:27:26.897750+0000","last_peered":"2024-05-09T06:27:26.897750+0000","last_clean":"2024-05-09T06:27:26.897750+0000","last_became_active":"2024-05-09T06:27:20.638469+0000","last_became_peered":"2024-05-09T06:27:20.638469+0000","last_unstale":"2024-05-09T06:27:26.897750+0000","last_undegraded":"2024-05-09T06:27:26.897750+0000","last_fullsized":"2024-05-09T06:27:26.897750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795796+0000","last_change":"2024-05-09T06:27:25.673712+0000","last_active":"2024-05-09T06:27:28.795796+0000","last_peered":"2024-05-09T06:27:28.795796+0000","last_clean":"2024-05-09T06:27:28.795796+0000","last_became_active":"2024-05-09T06:27:25.673062+0000","last_became_peered":"2024-05-09T06:27:25.673062+0000","last_unstale":"2024-05-09T06:27:28.795796+0000","last_undegraded":"2024-05-09T06:27:28.795796+0000","last_fullsized":"2024-05-09T06:27:28.795796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310517+0000","last_change":"2024-05-09T06:27:21.652689+0000","last_active":"2024-05-09T06:27:28.310517+0000","last_peered":"2024-05-09T06:27:28.310517+0000","last_clean":"2024-05-09T06:27:28.310517+0000","last_became_active":"2024-05-09T06:27:21.652309+0000","last_became_peered":"2024-05-09T06:27:21.652309+0000","last_unstale":"2024-05-09T06:27:28.310517+0000","last_undegraded":"2024-05-09T06:27:28.310517+0000","last_fullsized":"2024-05-09T06:27:28.310517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:30.619441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666511+0000","last_change":"2024-05-09T06:27:20.631329+0000","last_active":"2024-05-09T06:27:26.666511+0000","last_peered":"2024-05-09T06:27:26.666511+0000","last_clean":"2024-05-09T06:27:26.666511+0000","last_became_active":"2024-05-09T06:27:20.631027+0000","last_became_peered":"2024-05-09T06:27:20.631027+0000","last_unstale":"2024-05-09T06:27:26.666511+0000","last_undegraded":"2024-05-09T06:27:26.666511+0000","last_fullsized":"2024-05-09T06:27:26.666511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795581+0000","last_change":"2024-05-09T06:27:25.868448+0000","last_active":"2024-05-09T06:27:28.795581+0000","last_peered":"2024-05-09T06:27:28.795581+0000","last_clean":"2024-05-09T06:27:28.795581+0000","last_became_active":"2024-05-09T06:27:25.867998+0000","last_became_peered":"2024-05-09T06:27:25.867998+0000","last_unstale":"2024-05-09T06:27:28.795581+0000","last_undegraded":"2024-05-09T06:27:28.795581+0000","last_fullsized":"2024-05-09T06:27:28.795581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310235+0000","last_change":"2024-05-09T06:27:21.654321+0000","last_active":"2024-05-09T06:27:28.310235+0000","last_peered":"2024-05-09T06:27:28.310235+0000","last_clean":"2024-05-09T06:27:28.310235+0000","last_became_active":"2024-05-09T06:27:21.654037+0000","last_became_peered":"2024-05-09T06:27:21.654037+0000","last_unstale":"2024-05-09T06:27:28.310235+0000","last_undegraded":"2024-05-09T06:27:28.310235+0000","last_fullsized":"2024-05-09T06:27:28.310235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667634+0000","last_change":"2024-05-09T06:27:20.634285+0000","last_active":"2024-05-09T06:27:26.667634+0000","last_peered":"2024-05-09T06:27:26.667634+0000","last_clean":"2024-05-09T06:27:26.667634+0000","last_became_active":"2024-05-09T06:27:20.633961+0000","last_became_peered":"2024-05-09T06:27:20.633961+0000","last_unstale":"2024-05-09T06:27:26.667634+0000","last_undegraded":"2024-05-09T06:27:26.667634+0000","last_fullsized":"2024-05-09T06:27:26.667634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:22.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666362+0000","last_change":"2024-05-09T06:27:25.666556+0000","last_active":"2024-05-09T06:27:26.666362+0000","last_peered":"2024-05-09T06:27:26.666362+0000","last_clean":"2024-05-09T06:27:26.666362+0000","last_became_active":"2024-05-09T06:27:25.666086+0000","last_became_peered":"2024-05-09T06:27:25.666086+0000","last_unstale":"2024-05-09T06:27:26.666362+0000","last_undegraded":"2024-05-09T06:27:26.666362+0000","last_fullsized":"2024-05-09T06:27:26.666362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:16.228053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309897+0000","last_change":"2024-05-09T06:27:21.653860+0000","last_active":"2024-05-09T06:27:28.309897+0000","last_peered":"2024-05-09T06:27:28.309897+0000","last_clean":"2024-05-09T06:27:28.309897+0000","last_became_active":"2024-05-09T06:27:21.653222+0000","last_became_peered":"2024-05-09T06:27:21.653222+0000","last_unstale":"2024-05-09T06:27:28.309897+0000","last_undegraded":"2024-05-09T06:27:28.309897+0000","last_fullsized":"2024-05-09T06:27:28.309897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665751+0000","last_change":"2024-05-09T06:27:20.632271+0000","last_active":"2024-05-09T06:27:26.665751+0000","last_peered":"2024-05-09T06:27:26.665751+0000","last_clean":"2024-05-09T06:27:26.665751+0000","last_became_active":"2024-05-09T06:27:20.631965+0000","last_became_peered":"2024-05-09T06:27:20.631965+0000","last_unstale":"2024-05-09T06:27:26.665751+0000","last_undegraded":"2024-05-09T06:27:26.665751+0000","last_fullsized":"2024-05-09T06:27:26.665751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796544+0000","last_change":"2024-05-09T06:27:25.673394+0000","last_active":"2024-05-09T06:27:28.796544+0000","last_peered":"2024-05-09T06:27:28.796544+0000","last_clean":"2024-05-09T06:27:28.796544+0000","last_became_active":"2024-05-09T06:27:25.672824+0000","last_became_peered":"2024-05-09T06:27:25.672824+0000","last_unstale":"2024-05-09T06:27:28.796544+0000","last_undegraded":"2024-05-09T06:27:28.796544+0000","last_fullsized":"2024-05-09T06:27:28.796544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795660+0000","last_change":"2024-05-09T06:27:21.658568+0000","last_active":"2024-05-09T06:27:28.795660+0000","last_peered":"2024-05-09T06:27:28.795660+0000","last_clean":"2024-05-09T06:27:28.795660+0000","last_became_active":"2024-05-09T06:27:21.658289+0000","last_became_peered":"2024-05-09T06:27:21.658289+0000","last_unstale":"2024-05-09T06:27:28.795660+0000","last_undegraded":"2024-05-09T06:27:28.795660+0000","last_fullsized":"2024-05-09T06:27:28.795660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.637847+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.637553+0000","last_became_peered":"2024-05-09T06:27:20.637553+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795481+0000","last_change":"2024-05-09T06:27:25.872027+0000","last_active":"2024-05-09T06:27:28.795481+0000","last_peered":"2024-05-09T06:27:28.795481+0000","last_clean":"2024-05-09T06:27:28.795481+0000","last_became_active":"2024-05-09T06:27:25.871616+0000","last_became_peered":"2024-05-09T06:27:25.871616+0000","last_unstale":"2024-05-09T06:27:28.795481+0000","last_undegraded":"2024-05-09T06:27:28.795481+0000","last_fullsized":"2024-05-09T06:27:28.795481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:58:18.735834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309942+0000","last_change":"2024-05-09T06:27:21.655596+0000","last_active":"2024-05-09T06:27:28.309942+0000","last_peered":"2024-05-09T06:27:28.309942+0000","last_clean":"2024-05-09T06:27:28.309942+0000","last_became_active":"2024-05-09T06:27:21.655327+0000","last_became_peered":"2024-05-09T06:27:21.655327+0000","last_unstale":"2024-05-09T06:27:28.309942+0000","last_undegraded":"2024-05-09T06:27:28.309942+0000","last_fullsized":"2024-05-09T06:27:28.309942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796614+0000","last_change":"2024-05-09T06:27:20.640461+0000","last_active":"2024-05-09T06:27:28.796614+0000","last_peered":"2024-05-09T06:27:28.796614+0000","last_clean":"2024-05-09T06:27:28.796614+0000","last_became_active":"2024-05-09T06:27:20.640164+0000","last_became_peered":"2024-05-09T06:27:20.640164+0000","last_unstale":"2024-05-09T06:27:28.796614+0000","last_undegraded":"2024-05-09T06:27:28.796614+0000","last_fullsized":"2024-05-09T06:27:28.796614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795963+0000","last_change":"2024-05-09T06:27:25.868054+0000","last_active":"2024-05-09T06:27:28.795963+0000","last_peered":"2024-05-09T06:27:28.795963+0000","last_clean":"2024-05-09T06:27:28.795963+0000","last_became_active":"2024-05-09T06:27:25.867638+0000","last_became_peered":"2024-05-09T06:27:25.867638+0000","last_unstale":"2024-05-09T06:27:28.795963+0000","last_undegraded":"2024-05-09T06:27:28.795963+0000","last_fullsized":"2024-05-09T06:27:28.795963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:05.780930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795875+0000","last_change":"2024-05-09T06:27:21.640283+0000","last_active":"2024-05-09T06:27:28.795875+0000","last_peered":"2024-05-09T06:27:28.795875+0000","last_clean":"2024-05-09T06:27:28.795875+0000","last_became_active":"2024-05-09T06:27:21.639981+0000","last_became_peered":"2024-05-09T06:27:21.639981+0000","last_unstale":"2024-05-09T06:27:28.795875+0000","last_undegraded":"2024-05-09T06:27:28.795875+0000","last_fullsized":"2024-05-09T06:27:28.795875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:57.154069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794456+0000","last_change":"2024-05-09T06:27:20.636469+0000","last_active":"2024-05-09T06:27:28.794456+0000","last_peered":"2024-05-09T06:27:28.794456+0000","last_clean":"2024-05-09T06:27:28.794456+0000","last_became_active":"2024-05-09T06:27:20.636200+0000","last_became_peered":"2024-05-09T06:27:20.636200+0000","last_unstale":"2024-05-09T06:27:28.794456+0000","last_undegraded":"2024-05-09T06:27:28.794456+0000","last_fullsized":"2024-05-09T06:27:28.794456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:02.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796704+0000","last_change":"2024-05-09T06:27:25.868001+0000","last_active":"2024-05-09T06:27:28.796704+0000","last_peered":"2024-05-09T06:27:28.796704+0000","last_clean":"2024-05-09T06:27:28.796704+0000","last_became_active":"2024-05-09T06:27:25.867561+0000","last_became_peered":"2024-05-09T06:27:25.867561+0000","last_unstale":"2024-05-09T06:27:28.796704+0000","last_undegraded":"2024-05-09T06:27:28.796704+0000","last_fullsized":"2024-05-09T06:27:28.796704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310020+0000","last_change":"2024-05-09T06:27:21.647526+0000","last_active":"2024-05-09T06:27:28.310020+0000","last_peered":"2024-05-09T06:27:28.310020+0000","last_clean":"2024-05-09T06:27:28.310020+0000","last_became_active":"2024-05-09T06:27:21.646721+0000","last_became_peered":"2024-05-09T06:27:21.646721+0000","last_unstale":"2024-05-09T06:27:28.310020+0000","last_undegraded":"2024-05-09T06:27:28.310020+0000","last_fullsized":"2024-05-09T06:27:28.310020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:31.226892+0000","last_change":"2024-05-09T06:27:20.634111+0000","last_active":"2024-05-09T06:27:31.226892+0000","last_peered":"2024-05-09T06:27:31.226892+0000","last_clean":"2024-05-09T06:27:31.226892+0000","last_became_active":"2024-05-09T06:27:20.633772+0000","last_became_peered":"2024-05-09T06:27:20.633772+0000","last_unstale":"2024-05-09T06:27:31.226892+0000","last_undegraded":"2024-05-09T06:27:31.226892+0000","last_fullsized":"2024-05-09T06:27:31.226892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897702+0000","last_change":"2024-05-09T06:27:25.676189+0000","last_active":"2024-05-09T06:27:26.897702+0000","last_peered":"2024-05-09T06:27:26.897702+0000","last_clean":"2024-05-09T06:27:26.897702+0000","last_became_active":"2024-05-09T06:27:25.672390+0000","last_became_peered":"2024-05-09T06:27:25.672390+0000","last_unstale":"2024-05-09T06:27:26.897702+0000","last_undegraded":"2024-05-09T06:27:26.897702+0000","last_fullsized":"2024-05-09T06:27:26.897702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667630+0000","last_change":"2024-05-09T06:27:21.637363+0000","last_active":"2024-05-09T06:27:26.667630+0000","last_peered":"2024-05-09T06:27:26.667630+0000","last_clean":"2024-05-09T06:27:26.667630+0000","last_became_active":"2024-05-09T06:27:21.637056+0000","last_became_peered":"2024-05-09T06:27:21.637056+0000","last_unstale":"2024-05-09T06:27:26.667630+0000","last_undegraded":"2024-05-09T06:27:26.667630+0000","last_fullsized":"2024-05-09T06:27:26.667630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897618+0000","last_change":"2024-05-09T06:27:20.641960+0000","last_active":"2024-05-09T06:27:26.897618+0000","last_peered":"2024-05-09T06:27:26.897618+0000","last_clean":"2024-05-09T06:27:26.897618+0000","last_became_active":"2024-05-09T06:27:20.641670+0000","last_became_peered":"2024-05-09T06:27:20.641670+0000","last_unstale":"2024-05-09T06:27:26.897618+0000","last_undegraded":"2024-05-09T06:27:26.897618+0000","last_fullsized":"2024-05-09T06:27:26.897618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667165+0000","last_change":"2024-05-09T06:27:25.675301+0000","last_active":"2024-05-09T06:27:26.667165+0000","last_peered":"2024-05-09T06:27:26.667165+0000","last_clean":"2024-05-09T06:27:26.667165+0000","last_became_active":"2024-05-09T06:27:25.674818+0000","last_became_peered":"2024-05-09T06:27:25.674818+0000","last_unstale":"2024-05-09T06:27:26.667165+0000","last_undegraded":"2024-05-09T06:27:26.667165+0000","last_fullsized":"2024-05-09T06:27:26.667165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:06.045330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796636+0000","last_change":"2024-05-09T06:27:21.638651+0000","last_active":"2024-05-09T06:27:28.796636+0000","last_peered":"2024-05-09T06:27:28.796636+0000","last_clean":"2024-05-09T06:27:28.796636+0000","last_became_active":"2024-05-09T06:27:21.638111+0000","last_became_peered":"2024-05-09T06:27:21.638111+0000","last_unstale":"2024-05-09T06:27:28.796636+0000","last_undegraded":"2024-05-09T06:27:28.796636+0000","last_fullsized":"2024-05-09T06:27:28.796636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796815+0000","last_change":"2024-05-09T06:27:20.639625+0000","last_active":"2024-05-09T06:27:28.796815+0000","last_peered":"2024-05-09T06:27:28.796815+0000","last_clean":"2024-05-09T06:27:28.796815+0000","last_became_active":"2024-05-09T06:27:20.639287+0000","last_became_peered":"2024-05-09T06:27:20.639287+0000","last_unstale":"2024-05-09T06:27:28.796815+0000","last_undegraded":"2024-05-09T06:27:28.796815+0000","last_fullsized":"2024-05-09T06:27:28.796815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796227+0000","last_change":"2024-05-09T06:27:25.669479+0000","last_active":"2024-05-09T06:27:28.796227+0000","last_peered":"2024-05-09T06:27:28.796227+0000","last_clean":"2024-05-09T06:27:28.796227+0000","last_became_active":"2024-05-09T06:27:25.668634+0000","last_became_peered":"2024-05-09T06:27:25.668634+0000","last_unstale":"2024-05-09T06:27:28.796227+0000","last_undegraded":"2024-05-09T06:27:28.796227+0000","last_fullsized":"2024-05-09T06:27:28.796227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:33.708720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796768+0000","last_change":"2024-05-09T06:27:21.657985+0000","last_active":"2024-05-09T06:27:28.796768+0000","last_peered":"2024-05-09T06:27:28.796768+0000","last_clean":"2024-05-09T06:27:28.796768+0000","last_became_active":"2024-05-09T06:27:21.653693+0000","last_became_peered":"2024-05-09T06:27:21.653693+0000","last_unstale":"2024-05-09T06:27:28.796768+0000","last_undegraded":"2024-05-09T06:27:28.796768+0000","last_fullsized":"2024-05-09T06:27:28.796768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796692+0000","last_change":"2024-05-09T06:27:20.639702+0000","last_active":"2024-05-09T06:27:28.796692+0000","last_peered":"2024-05-09T06:27:28.796692+0000","last_clean":"2024-05-09T06:27:28.796692+0000","last_became_active":"2024-05-09T06:27:20.639361+0000","last_became_peered":"2024-05-09T06:27:20.639361+0000","last_unstale":"2024-05-09T06:27:28.796692+0000","last_undegraded":"2024-05-09T06:27:28.796692+0000","last_fullsized":"2024-05-09T06:27:28.796692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:25.867964+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:25.867579+0000","last_became_peered":"2024-05-09T06:27:25.867579+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796481+0000","last_change":"2024-05-09T06:27:25.871150+0000","last_active":"2024-05-09T06:27:28.796481+0000","last_peered":"2024-05-09T06:27:28.796481+0000","last_clean":"2024-05-09T06:27:28.796481+0000","last_became_active":"2024-05-09T06:27:25.870707+0000","last_became_peered":"2024-05-09T06:27:25.870707+0000","last_unstale":"2024-05-09T06:27:28.796481+0000","last_undegraded":"2024-05-09T06:27:28.796481+0000","last_fullsized":"2024-05-09T06:27:28.796481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897140+0000","last_change":"2024-05-09T06:27:21.648383+0000","last_active":"2024-05-09T06:27:26.897140+0000","last_peered":"2024-05-09T06:27:26.897140+0000","last_clean":"2024-05-09T06:27:26.897140+0000","last_became_active":"2024-05-09T06:27:21.646149+0000","last_became_peered":"2024-05-09T06:27:21.646149+0000","last_unstale":"2024-05-09T06:27:26.897140+0000","last_undegraded":"2024-05-09T06:27:26.897140+0000","last_fullsized":"2024-05-09T06:27:26.897140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:20.639240+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:20.638885+0000","last_became_peered":"2024-05-09T06:27:20.638885+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796512+0000","last_change":"2024-05-09T06:27:25.673610+0000","last_active":"2024-05-09T06:27:28.796512+0000","last_peered":"2024-05-09T06:27:28.796512+0000","last_clean":"2024-05-09T06:27:28.796512+0000","last_became_active":"2024-05-09T06:27:25.673160+0000","last_became_peered":"2024-05-09T06:27:25.673160+0000","last_unstale":"2024-05-09T06:27:28.796512+0000","last_undegraded":"2024-05-09T06:27:28.796512+0000","last_fullsized":"2024-05-09T06:27:28.796512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796199+0000","last_change":"2024-05-09T06:27:21.645047+0000","last_active":"2024-05-09T06:27:28.796199+0000","last_peered":"2024-05-09T06:27:28.796199+0000","last_clean":"2024-05-09T06:27:28.796199+0000","last_became_active":"2024-05-09T06:27:21.644743+0000","last_became_peered":"2024-05-09T06:27:21.644743+0000","last_unstale":"2024-05-09T06:27:28.796199+0000","last_undegraded":"2024-05-09T06:27:28.796199+0000","last_fullsized":"2024-05-09T06:27:28.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795221+0000","last_change":"2024-05-09T06:27:20.635683+0000","last_active":"2024-05-09T06:27:28.795221+0000","last_peered":"2024-05-09T06:27:28.795221+0000","last_clean":"2024-05-09T06:27:28.795221+0000","last_became_active":"2024-05-09T06:27:20.635386+0000","last_became_peered":"2024-05-09T06:27:20.635386+0000","last_unstale":"2024-05-09T06:27:28.795221+0000","last_undegraded":"2024-05-09T06:27:28.795221+0000","last_fullsized":"2024-05-09T06:27:28.795221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.642531+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.642176+0000","last_became_peered":"2024-05-09T06:27:20.642176+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795979+0000","last_change":"2024-05-09T06:27:21.634797+0000","last_active":"2024-05-09T06:27:28.795979+0000","last_peered":"2024-05-09T06:27:28.795979+0000","last_clean":"2024-05-09T06:27:28.795979+0000","last_became_active":"2024-05-09T06:27:21.634458+0000","last_became_peered":"2024-05-09T06:27:21.634458+0000","last_unstale":"2024-05-09T06:27:28.795979+0000","last_undegraded":"2024-05-09T06:27:28.795979+0000","last_fullsized":"2024-05-09T06:27:28.795979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666255+0000","last_change":"2024-05-09T06:27:25.678471+0000","last_active":"2024-05-09T06:27:26.666255+0000","last_peered":"2024-05-09T06:27:26.666255+0000","last_clean":"2024-05-09T06:27:26.666255+0000","last_became_active":"2024-05-09T06:27:25.677981+0000","last_became_peered":"2024-05-09T06:27:25.677981+0000","last_unstale":"2024-05-09T06:27:26.666255+0000","last_undegraded":"2024-05-09T06:27:26.666255+0000","last_fullsized":"2024-05-09T06:27:26.666255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:20.609371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795054+0000","last_change":"2024-05-09T06:27:20.644121+0000","last_active":"2024-05-09T06:27:28.795054+0000","last_peered":"2024-05-09T06:27:28.795054+0000","last_clean":"2024-05-09T06:27:28.795054+0000","last_became_active":"2024-05-09T06:27:20.643795+0000","last_became_peered":"2024-05-09T06:27:20.643795+0000","last_unstale":"2024-05-09T06:27:28.795054+0000","last_undegraded":"2024-05-09T06:27:28.795054+0000","last_fullsized":"2024-05-09T06:27:28.795054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666578+0000","last_change":"2024-05-09T06:27:21.652395+0000","last_active":"2024-05-09T06:27:26.666578+0000","last_peered":"2024-05-09T06:27:26.666578+0000","last_clean":"2024-05-09T06:27:26.666578+0000","last_became_active":"2024-05-09T06:27:21.652019+0000","last_became_peered":"2024-05-09T06:27:21.652019+0000","last_unstale":"2024-05-09T06:27:26.666578+0000","last_undegraded":"2024-05-09T06:27:26.666578+0000","last_fullsized":"2024-05-09T06:27:26.666578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:37.219520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795669+0000","last_change":"2024-05-09T06:27:25.675291+0000","last_active":"2024-05-09T06:27:28.795669+0000","last_peered":"2024-05-09T06:27:28.795669+0000","last_clean":"2024-05-09T06:27:28.795669+0000","last_became_active":"2024-05-09T06:27:25.674422+0000","last_became_peered":"2024-05-09T06:27:25.674422+0000","last_unstale":"2024-05-09T06:27:28.795669+0000","last_undegraded":"2024-05-09T06:27:28.795669+0000","last_fullsized":"2024-05-09T06:27:28.795669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:18.601362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795268+0000","last_change":"2024-05-09T06:27:21.659311+0000","last_active":"2024-05-09T06:27:28.795268+0000","last_peered":"2024-05-09T06:27:28.795268+0000","last_clean":"2024-05-09T06:27:28.795268+0000","last_became_active":"2024-05-09T06:27:21.659039+0000","last_became_peered":"2024-05-09T06:27:21.659039+0000","last_unstale":"2024-05-09T06:27:28.795268+0000","last_undegraded":"2024-05-09T06:27:28.795268+0000","last_fullsized":"2024-05-09T06:27:28.795268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665756+0000","last_change":"2024-05-09T06:27:21.656834+0000","last_active":"2024-05-09T06:27:26.665756+0000","last_peered":"2024-05-09T06:27:26.665756+0000","last_clean":"2024-05-09T06:27:26.665756+0000","last_became_active":"2024-05-09T06:27:21.655466+0000","last_became_peered":"2024-05-09T06:27:21.655466+0000","last_unstale":"2024-05-09T06:27:26.665756+0000","last_undegraded":"2024-05-09T06:27:26.665756+0000","last_fullsized":"2024-05-09T06:27:26.665756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796581+0000","last_change":"2024-05-09T06:27:21.665177+0000","last_active":"2024-05-09T06:27:28.796581+0000","last_peered":"2024-05-09T06:27:28.796581+0000","last_clean":"2024-05-09T06:27:28.796581+0000","last_became_active":"2024-05-09T06:27:21.663959+0000","last_became_peered":"2024-05-09T06:27:21.663959+0000","last_unstale":"2024-05-09T06:27:28.796581+0000","last_undegraded":"2024-05-09T06:27:28.796581+0000","last_fullsized":"2024-05-09T06:27:28.796581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795891+0000","last_change":"2024-05-09T06:27:21.642136+0000","last_active":"2024-05-09T06:27:28.795891+0000","last_peered":"2024-05-09T06:27:28.795891+0000","last_clean":"2024-05-09T06:27:28.795891+0000","last_became_active":"2024-05-09T06:27:21.641855+0000","last_became_peered":"2024-05-09T06:27:21.641855+0000","last_unstale":"2024-05-09T06:27:28.795891+0000","last_undegraded":"2024-05-09T06:27:28.795891+0000","last_fullsized":"2024-05-09T06:27:28.795891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796263+0000","last_change":"2024-05-09T06:27:21.660067+0000","last_active":"2024-05-09T06:27:28.796263+0000","last_peered":"2024-05-09T06:27:28.796263+0000","last_clean":"2024-05-09T06:27:28.796263+0000","last_became_active":"2024-05-09T06:27:21.651525+0000","last_became_peered":"2024-05-09T06:27:21.651525+0000","last_unstale":"2024-05-09T06:27:28.796263+0000","last_undegraded":"2024-05-09T06:27:28.796263+0000","last_fullsized":"2024-05-09T06:27:28.796263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897431+0000","last_change":"2024-05-09T06:27:21.650694+0000","last_active":"2024-05-09T06:27:26.897431+0000","last_peered":"2024-05-09T06:27:26.897431+0000","last_clean":"2024-05-09T06:27:26.897431+0000","last_became_active":"2024-05-09T06:27:21.650404+0000","last_became_peered":"2024-05-09T06:27:21.650404+0000","last_unstale":"2024-05-09T06:27:26.897431+0000","last_undegraded":"2024-05-09T06:27:26.897431+0000","last_fullsized":"2024-05-09T06:27:26.897431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796327+0000","last_change":"2024-05-09T06:27:21.648723+0000","last_active":"2024-05-09T06:27:28.796327+0000","last_peered":"2024-05-09T06:27:28.796327+0000","last_clean":"2024-05-09T06:27:28.796327+0000","last_became_active":"2024-05-09T06:27:21.648434+0000","last_became_peered":"2024-05-09T06:27:21.648434+0000","last_unstale":"2024-05-09T06:27:28.796327+0000","last_undegraded":"2024-05-09T06:27:28.796327+0000","last_fullsized":"2024-05-09T06:27:28.796327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795585+0000","last_change":"2024-05-09T06:27:21.659502+0000","last_active":"2024-05-09T06:27:28.795585+0000","last_peered":"2024-05-09T06:27:28.795585+0000","last_clean":"2024-05-09T06:27:28.795585+0000","last_became_active":"2024-05-09T06:27:21.651268+0000","last_became_peered":"2024-05-09T06:27:21.651268+0000","last_unstale":"2024-05-09T06:27:28.795585+0000","last_undegraded":"2024-05-09T06:27:28.795585+0000","last_fullsized":"2024-05-09T06:27:28.795585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795603+0000","last_change":"2024-05-09T06:27:21.664539+0000","last_active":"2024-05-09T06:27:28.795603+0000","last_peered":"2024-05-09T06:27:28.795603+0000","last_clean":"2024-05-09T06:27:28.795603+0000","last_became_active":"2024-05-09T06:27:21.664267+0000","last_became_peered":"2024-05-09T06:27:21.664267+0000","last_unstale":"2024-05-09T06:27:28.795603+0000","last_undegraded":"2024-05-09T06:27:28.795603+0000","last_fullsized":"2024-05-09T06:27:28.795603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795097+0000","last_change":"2024-05-09T06:27:21.658842+0000","last_active":"2024-05-09T06:27:28.795097+0000","last_peered":"2024-05-09T06:27:28.795097+0000","last_clean":"2024-05-09T06:27:28.795097+0000","last_became_active":"2024-05-09T06:27:21.658522+0000","last_became_peered":"2024-05-09T06:27:21.658522+0000","last_unstale":"2024-05-09T06:27:28.795097+0000","last_undegraded":"2024-05-09T06:27:28.795097+0000","last_fullsized":"2024-05-09T06:27:28.795097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796440+0000","last_change":"2024-05-09T06:27:21.657936+0000","last_active":"2024-05-09T06:27:28.796440+0000","last_peered":"2024-05-09T06:27:28.796440+0000","last_clean":"2024-05-09T06:27:28.796440+0000","last_became_active":"2024-05-09T06:27:21.654771+0000","last_became_peered":"2024-05-09T06:27:21.654771+0000","last_unstale":"2024-05-09T06:27:28.796440+0000","last_undegraded":"2024-05-09T06:27:28.796440+0000","last_fullsized":"2024-05-09T06:27:28.796440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795635+0000","last_change":"2024-05-09T06:27:21.645805+0000","last_active":"2024-05-09T06:27:28.795635+0000","last_peered":"2024-05-09T06:27:28.795635+0000","last_clean":"2024-05-09T06:27:28.795635+0000","last_became_active":"2024-05-09T06:27:21.643759+0000","last_became_peered":"2024-05-09T06:27:21.643759+0000","last_unstale":"2024-05-09T06:27:28.795635+0000","last_undegraded":"2024-05-09T06:27:28.795635+0000","last_fullsized":"2024-05-09T06:27:28.795635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:12.851183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795676+0000","last_change":"2024-05-09T06:27:21.654407+0000","last_active":"2024-05-09T06:27:28.795676+0000","last_peered":"2024-05-09T06:27:28.795676+0000","last_clean":"2024-05-09T06:27:28.795676+0000","last_became_active":"2024-05-09T06:27:21.654126+0000","last_became_peered":"2024-05-09T06:27:21.654126+0000","last_unstale":"2024-05-09T06:27:28.795676+0000","last_undegraded":"2024-05-09T06:27:28.795676+0000","last_fullsized":"2024-05-09T06:27:28.795676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310344+0000","last_change":"2024-05-09T06:27:21.651773+0000","last_active":"2024-05-09T06:27:28.310344+0000","last_peered":"2024-05-09T06:27:28.310344+0000","last_clean":"2024-05-09T06:27:28.310344+0000","last_became_active":"2024-05-09T06:27:21.650069+0000","last_became_peered":"2024-05-09T06:27:21.650069+0000","last_unstale":"2024-05-09T06:27:28.310344+0000","last_undegraded":"2024-05-09T06:27:28.310344+0000","last_fullsized":"2024-05-09T06:27:28.310344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666446+0000","last_change":"2024-05-09T06:27:21.657123+0000","last_active":"2024-05-09T06:27:26.666446+0000","last_peered":"2024-05-09T06:27:26.666446+0000","last_clean":"2024-05-09T06:27:26.666446+0000","last_became_active":"2024-05-09T06:27:21.656787+0000","last_became_peered":"2024-05-09T06:27:21.656787+0000","last_unstale":"2024-05-09T06:27:26.666446+0000","last_undegraded":"2024-05-09T06:27:26.666446+0000","last_fullsized":"2024-05-09T06:27:26.666446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.797072+0000","last_change":"2024-05-09T06:27:21.660613+0000","last_active":"2024-05-09T06:27:28.797072+0000","last_peered":"2024-05-09T06:27:28.797072+0000","last_clean":"2024-05-09T06:27:28.797072+0000","last_became_active":"2024-05-09T06:27:21.654942+0000","last_became_peered":"2024-05-09T06:27:21.654942+0000","last_unstale":"2024-05-09T06:27:28.797072+0000","last_undegraded":"2024-05-09T06:27:28.797072+0000","last_fullsized":"2024-05-09T06:27:28.797072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795476+0000","last_change":"2024-05-09T06:27:21.658261+0000","last_active":"2024-05-09T06:27:28.795476+0000","last_peered":"2024-05-09T06:27:28.795476+0000","last_clean":"2024-05-09T06:27:28.795476+0000","last_became_active":"2024-05-09T06:27:21.658004+0000","last_became_peered":"2024-05-09T06:27:21.658004+0000","last_unstale":"2024-05-09T06:27:28.795476+0000","last_undegraded":"2024-05-09T06:27:28.795476+0000","last_fullsized":"2024-05-09T06:27:28.795476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897230+0000","last_change":"2024-05-09T06:27:21.654019+0000","last_active":"2024-05-09T06:27:26.897230+0000","last_peered":"2024-05-09T06:27:26.897230+0000","last_clean":"2024-05-09T06:27:26.897230+0000","last_became_active":"2024-05-09T06:27:21.653711+0000","last_became_peered":"2024-05-09T06:27:21.653711+0000","last_unstale":"2024-05-09T06:27:26.897230+0000","last_undegraded":"2024-05-09T06:27:26.897230+0000","last_fullsized":"2024-05-09T06:27:26.897230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795778+0000","last_change":"2024-05-09T06:27:21.645328+0000","last_active":"2024-05-09T06:27:28.795778+0000","last_peered":"2024-05-09T06:27:28.795778+0000","last_clean":"2024-05-09T06:27:28.795778+0000","last_became_active":"2024-05-09T06:27:21.644287+0000","last_became_peered":"2024-05-09T06:27:21.644287+0000","last_unstale":"2024-05-09T06:27:28.795778+0000","last_undegraded":"2024-05-09T06:27:28.795778+0000","last_fullsized":"2024-05-09T06:27:28.795778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:16.580568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795360+0000","last_change":"2024-05-09T06:27:21.643190+0000","last_active":"2024-05-09T06:27:28.795360+0000","last_peered":"2024-05-09T06:27:28.795360+0000","last_clean":"2024-05-09T06:27:28.795360+0000","last_became_active":"2024-05-09T06:27:21.642700+0000","last_became_peered":"2024-05-09T06:27:21.642700+0000","last_unstale":"2024-05-09T06:27:28.795360+0000","last_undegraded":"2024-05-09T06:27:28.795360+0000","last_fullsized":"2024-05-09T06:27:28.795360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667413+0000","last_change":"2024-05-09T06:27:21.652745+0000","last_active":"2024-05-09T06:27:26.667413+0000","last_peered":"2024-05-09T06:27:26.667413+0000","last_clean":"2024-05-09T06:27:26.667413+0000","last_became_active":"2024-05-09T06:27:21.651219+0000","last_became_peered":"2024-05-09T06:27:21.651219+0000","last_unstale":"2024-05-09T06:27:26.667413+0000","last_undegraded":"2024-05-09T06:27:26.667413+0000","last_fullsized":"2024-05-09T06:27:26.667413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795888+0000","last_change":"2024-05-09T06:27:21.663110+0000","last_active":"2024-05-09T06:27:28.795888+0000","last_peered":"2024-05-09T06:27:28.795888+0000","last_clean":"2024-05-09T06:27:28.795888+0000","last_became_active":"2024-05-09T06:27:21.662731+0000","last_became_peered":"2024-05-09T06:27:21.662731+0000","last_unstale":"2024-05-09T06:27:28.795888+0000","last_undegraded":"2024-05-09T06:27:28.795888+0000","last_fullsized":"2024-05-09T06:27:28.795888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796301+0000","last_change":"2024-05-09T06:27:21.659071+0000","last_active":"2024-05-09T06:27:28.796301+0000","last_peered":"2024-05-09T06:27:28.796301+0000","last_clean":"2024-05-09T06:27:28.796301+0000","last_became_active":"2024-05-09T06:27:21.658774+0000","last_became_peered":"2024-05-09T06:27:21.658774+0000","last_unstale":"2024-05-09T06:27:28.796301+0000","last_undegraded":"2024-05-09T06:27:28.796301+0000","last_fullsized":"2024-05-09T06:27:28.796301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310458+0000","last_change":"2024-05-09T06:27:21.665023+0000","last_active":"2024-05-09T06:27:28.310458+0000","last_peered":"2024-05-09T06:27:28.310458+0000","last_clean":"2024-05-09T06:27:28.310458+0000","last_became_active":"2024-05-09T06:27:21.664464+0000","last_became_peered":"2024-05-09T06:27:21.664464+0000","last_unstale":"2024-05-09T06:27:28.310458+0000","last_undegraded":"2024-05-09T06:27:28.310458+0000","last_fullsized":"2024-05-09T06:27:28.310458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795913+0000","last_change":"2024-05-09T06:27:21.661852+0000","last_active":"2024-05-09T06:27:28.795913+0000","last_peered":"2024-05-09T06:27:28.795913+0000","last_clean":"2024-05-09T06:27:28.795913+0000","last_became_active":"2024-05-09T06:27:21.660313+0000","last_became_peered":"2024-05-09T06:27:21.660313+0000","last_unstale":"2024-05-09T06:27:28.795913+0000","last_undegraded":"2024-05-09T06:27:28.795913+0000","last_fullsized":"2024-05-09T06:27:28.795913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309769+0000","last_change":"2024-05-09T06:27:21.654130+0000","last_active":"2024-05-09T06:27:28.309769+0000","last_peered":"2024-05-09T06:27:28.309769+0000","last_clean":"2024-05-09T06:27:28.309769+0000","last_became_active":"2024-05-09T06:27:21.653593+0000","last_became_peered":"2024-05-09T06:27:21.653593+0000","last_unstale":"2024-05-09T06:27:28.309769+0000","last_undegraded":"2024-05-09T06:27:28.309769+0000","last_fullsized":"2024-05-09T06:27:28.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796010+0000","last_change":"2024-05-09T06:27:21.654775+0000","last_active":"2024-05-09T06:27:28.796010+0000","last_peered":"2024-05-09T06:27:28.796010+0000","last_clean":"2024-05-09T06:27:28.796010+0000","last_became_active":"2024-05-09T06:27:21.654269+0000","last_became_peered":"2024-05-09T06:27:21.654269+0000","last_unstale":"2024-05-09T06:27:28.796010+0000","last_undegraded":"2024-05-09T06:27:28.796010+0000","last_fullsized":"2024-05-09T06:27:28.796010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309790+0000","last_change":"2024-05-09T06:27:21.649923+0000","last_active":"2024-05-09T06:27:28.309790+0000","last_peered":"2024-05-09T06:27:28.309790+0000","last_clean":"2024-05-09T06:27:28.309790+0000","last_became_active":"2024-05-09T06:27:21.648693+0000","last_became_peered":"2024-05-09T06:27:21.648693+0000","last_unstale":"2024-05-09T06:27:28.309790+0000","last_undegraded":"2024-05-09T06:27:28.309790+0000","last_fullsized":"2024-05-09T06:27:28.309790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796403+0000","last_change":"2024-05-09T06:27:21.664868+0000","last_active":"2024-05-09T06:27:28.796403+0000","last_peered":"2024-05-09T06:27:28.796403+0000","last_clean":"2024-05-09T06:27:28.796403+0000","last_became_active":"2024-05-09T06:27:21.663870+0000","last_became_peered":"2024-05-09T06:27:21.663870+0000","last_unstale":"2024-05-09T06:27:28.796403+0000","last_undegraded":"2024-05-09T06:27:28.796403+0000","last_fullsized":"2024-05-09T06:27:28.796403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796427+0000","last_change":"2024-05-09T06:27:21.651232+0000","last_active":"2024-05-09T06:27:28.796427+0000","last_peered":"2024-05-09T06:27:28.796427+0000","last_clean":"2024-05-09T06:27:28.796427+0000","last_became_active":"2024-05-09T06:27:21.650940+0000","last_became_peered":"2024-05-09T06:27:21.650940+0000","last_unstale":"2024-05-09T06:27:28.796427+0000","last_undegraded":"2024-05-09T06:27:28.796427+0000","last_fullsized":"2024-05-09T06:27:28.796427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897594+0000","last_change":"2024-05-09T06:27:21.648169+0000","last_active":"2024-05-09T06:27:26.897594+0000","last_peered":"2024-05-09T06:27:26.897594+0000","last_clean":"2024-05-09T06:27:26.897594+0000","last_became_active":"2024-05-09T06:27:21.647846+0000","last_became_peered":"2024-05-09T06:27:21.647846+0000","last_unstale":"2024-05-09T06:27:26.897594+0000","last_undegraded":"2024-05-09T06:27:26.897594+0000","last_fullsized":"2024-05-09T06:27:26.897594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310386+0000","last_change":"2024-05-09T06:27:21.649391+0000","last_active":"2024-05-09T06:27:28.310386+0000","last_peered":"2024-05-09T06:27:28.310386+0000","last_clean":"2024-05-09T06:27:28.310386+0000","last_became_active":"2024-05-09T06:27:21.648828+0000","last_became_peered":"2024-05-09T06:27:21.648828+0000","last_unstale":"2024-05-09T06:27:28.310386+0000","last_undegraded":"2024-05-09T06:27:28.310386+0000","last_fullsized":"2024-05-09T06:27:28.310386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:27:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:27:45.112 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T06:27:45.112 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T06:27:45.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T06:27:45.112 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T06:27:45.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.268+0000 7ffad04a5640 1 Processor -- start 2024-05-09T06:27:45.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.268+0000 7ffad04a5640 1 -- start start 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffad04a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 0x7ffac815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffad04a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac80aef90 con 0x7ffac80ae9b0 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffad04a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac80b3530 con 0x7ffac806e4f0 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffad04a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffac80b3b80 con 0x7ffac815e050 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffacea42640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ffac80ae9b0 0x7ffac80b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38398/0 (socket says 172.21.15.1:38398) 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.269+0000 7ffacea42640 1 -- 172.21.15.1:0/4011551793 learned_addr learned my addr 172.21.15.1:0/4011551793 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7fface241640 1 --2- 172.21.15.1:0/4011551793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 0x7ffac815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595484199 0 0) 0x7ffac80aef90 con 0x7ffac80ae9b0 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffab0003520 con 0x7ffac80ae9b0 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 442469269 0 0) 0x7ffac80b3530 con 0x7ffac806e4f0 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffac80aef90 con 0x7ffac806e4f0 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7fface241640 1 -- 172.21.15.1:0/4011551793 >> v1:172.21.15.6:6789/0 conn(0x7ffac806e4f0 legacy=0x7ffac80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7fface241640 1 -- 172.21.15.1:0/4011551793 >> v1:172.21.15.1:6789/0 conn(0x7ffac80ae9b0 legacy=0x7ffac80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.270+0000 7fface241640 1 -- 172.21.15.1:0/4011551793 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac80b3810 con 0x7ffac815e050 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7fface241640 1 --2- 172.21.15.1:0/4011551793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 0x7ffac815e430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097d0 tx=0x7ffac002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82f66707f21f823 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0037070 con 0x7ffac815e050 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac0032440 con 0x7ffac815e050 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffacd23f640 1 -- 172.21.15.1:0/4011551793 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00327f0 con 0x7ffac815e050 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffad04a5640 1 -- 172.21.15.1:0/4011551793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 msgr2=0x7ffac815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffad04a5640 1 --2- 172.21.15.1:0/4011551793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 0x7ffac815e430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097d0 tx=0x7ffac002a210 comp rx=0 tx=0).stop 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffad04a5640 1 -- 172.21.15.1:0/4011551793 shutdown_connections 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffad04a5640 1 --2- 172.21.15.1:0/4011551793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac815e050 0x7ffac815e430 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.271+0000 7ffad04a5640 1 -- 172.21.15.1:0/4011551793 >> 172.21.15.1:0/4011551793 conn(0x7ffac806d270 msgr2=0x7ffac806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.272+0000 7ffad04a5640 1 -- 172.21.15.1:0/4011551793 shutdown_connections 2024-05-09T06:27:45.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.272+0000 7ffad04a5640 1 -- 172.21.15.1:0/4011551793 wait complete. 2024-05-09T06:27:45.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 Processor -- start 2024-05-09T06:27:45.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 -- start start 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac806e4f0 0x7ffac813cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac80ae9b0 0x7ffac8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffac806ead0 con 0x7ffac815e050 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffac806e950 con 0x7ffac806e4f0 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.273+0000 7ffad04a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffac806ec50 con 0x7ffac80ae9b0 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53130/0 (socket says 172.21.15.1:53130) 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 -- 172.21.15.1:0/1920156195 learned_addr learned my addr 172.21.15.1:0/1920156195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7fface241640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac806e4f0 0x7ffac813cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacda40640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac80ae9b0 0x7ffac8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 -- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac80ae9b0 msgr2=0x7ffac8158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac80ae9b0 0x7ffac8158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 -- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac806e4f0 msgr2=0x7ffac813cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac806e4f0 0x7ffac813cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 -- 172.21.15.1:0/1920156195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac81506b0 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffacea42640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffab800cb40 tx=0x7ffab800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8017070 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab800aca0 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac8150990 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8012600 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.274+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac8157b70 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.275+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac8150b60 con 0x7ffac815e050 2024-05-09T06:27:45.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.276+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffab801a070 con 0x7ffac815e050 2024-05-09T06:27:45.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.276+0000 7ffac6ffd640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 0x7ffab406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.276+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffab808af40 con 0x7ffac815e050 2024-05-09T06:27:45.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.276+0000 7fface241640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 0x7ffab406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.277+0000 7fface241640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 0x7ffab406e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffac0034000 tx=0x7ffac002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.281+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffab8058ee0 con 0x7ffac815e050 2024-05-09T06:27:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.512+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ffac80752b0 con 0x7ffac815e050 2024-05-09T06:27:45.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.515+0000 7ffac6ffd640 1 -- 172.21.15.1:0/1920156195 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172932 (secure 0 0 0) 0x7ffab805d1e0 con 0x7ffac815e050 2024-05-09T06:27:45.499 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:45.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 msgr2=0x7ffab406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 0x7ffab406e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffac0034000 tx=0x7ffac002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 msgr2=0x7ffac813f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffab800cb40 tx=0x7ffab800a060 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 shutdown_connections 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffab406be30 0x7ffab406e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac815e050 0x7ffac813f910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffac80ae9b0 0x7ffac8158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 --2- 172.21.15.1:0/1920156195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac806e4f0 0x7ffac813cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 >> 172.21.15.1:0/1920156195 conn(0x7ffac806d270 msgr2=0x7ffac815bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 shutdown_connections 2024-05-09T06:27:45.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.518+0000 7ffad04a5640 1 -- 172.21.15.1:0/1920156195 wait complete. 2024-05-09T06:27:45.511 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2620481298},{"type":"v1","addr":"172.21.15.1:6833","nonce":2620481298}]},"active_addr":"172.21.15.1:6833/2620481298","active_change":"2024-05-09T06:27:01.060238+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2542828478}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3187516281}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2155164325}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1891905510}]}]} 2024-05-09T06:27:45.514 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T06:27:45.514 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T06:27:45.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.635+0000 7f8dd14f5640 1 Processor -- start 2024-05-09T06:27:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.635+0000 7f8dd14f5640 1 -- start start 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dcc161950 0x7f8dcc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d4940 0x7f8dcc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8dcc0d0f80 0x7f8dcc0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8dcc0d4dd0 con 0x7f8dcc161950 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8dcc0b33c0 con 0x7f8dcc0d0f80 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dd14f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b3660 con 0x7f8dcc0d4940 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dcaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dcc161950 0x7f8dcc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dcaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dcc161950 0x7f8dcc0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53146/0 (socket says 172.21.15.1:53146) 2024-05-09T06:27:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.636+0000 7f8dcaffd640 1 -- 172.21.15.1:0/810182648 learned_addr learned my addr 172.21.15.1:0/810182648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcb7fe640 1 --2- 172.21.15.1:0/810182648 >> v2:172.21.15.6:3300/0 conn(0x7f8dcc0d0f80 0x7f8dcc0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dca7fc640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d4940 0x7f8dcc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcaffd640 1 -- 172.21.15.1:0/810182648 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d4940 msgr2=0x7f8dcc0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d4940 0x7f8dcc0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcaffd640 1 -- 172.21.15.1:0/810182648 >> v2:172.21.15.6:3300/0 conn(0x7f8dcc0d0f80 msgr2=0x7f8dcc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/810182648 >> v2:172.21.15.6:3300/0 conn(0x7f8dcc0d0f80 0x7f8dcc0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.637+0000 7f8dcaffd640 1 -- 172.21.15.1:0/810182648 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc0b3940 con 0x7f8dcc161950 2024-05-09T06:27:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc0d2400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc08c710 tx=0x7f8dbc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5567777ba20e1a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dc9ffb640 1 -- 172.21.15.1:0/810182648 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc036070 con 0x7f8dcc161950 2024-05-09T06:27:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dc9ffb640 1 -- 172.21.15.1:0/810182648 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dbc031600 con 0x7f8dcc161950 2024-05-09T06:27:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dc9ffb640 1 -- 172.21.15.1:0/810182648 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc0319d0 con 0x7f8dcc161950 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 -- 172.21.15.1:0/810182648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 msgr2=0x7f8dcc0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc0d2400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc08c710 tx=0x7f8dbc02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 -- 172.21.15.1:0/810182648 shutdown_connections 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/810182648 >> v2:172.21.15.6:3300/0 conn(0x7f8dcc0d0f80 0x7f8dcc0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d4940 0x7f8dcc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/810182648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc0d2400 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 -- 172.21.15.1:0/810182648 >> 172.21.15.1:0/810182648 conn(0x7f8dcc0d3e20 msgr2=0x7f8dcc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 -- 172.21.15.1:0/810182648 shutdown_connections 2024-05-09T06:27:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.638+0000 7f8dd14f5640 1 -- 172.21.15.1:0/810182648 wait complete. 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.640+0000 7f8dd14f5640 1 Processor -- start 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.640+0000 7f8dd14f5640 1 -- start start 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d0f80 0x7f8dcc09f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dcc0b7680 0x7f8dcc0b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0d1d80 con 0x7f8dcc161950 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0d1c00 con 0x7f8dcc0b7680 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc0d1f00 con 0x7f8dcc0d0f80 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53160/0 (socket says 172.21.15.1:53160) 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 -- 172.21.15.1:0/2622733229 learned_addr learned my addr 172.21.15.1:0/2622733229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d0f80 0x7f8dcc09f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dcb7fe640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dcc0b7680 0x7f8dcc0b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 -- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d0f80 msgr2=0x7f8dcc09f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d0f80 0x7f8dcc09f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 -- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dcc0b7680 msgr2=0x7f8dcc0b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dcc0b7680 0x7f8dcc0b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 -- 172.21.15.1:0/2622733229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc0b8420 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dca7fc640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e9e0 tx=0x7f8db400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4017070 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db400c970 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc0b8140 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.641+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db400cd60 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.642+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc0cc2c0 con 0x7f8dcc161950 2024-05-09T06:27:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.642+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc0752b0 con 0x7f8dcc161950 2024-05-09T06:27:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.643+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8db401a070 con 0x7f8dcc161950 2024-05-09T06:27:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.644+0000 7f8dc37fe640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 0x7f8dac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.644+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8db408afd0 con 0x7f8dcc161950 2024-05-09T06:27:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.644+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 0x7f8dac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.644+0000 7f8dcaffd640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 0x7f8dac06e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc033000 tx=0x7f8dbc009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.648+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8db4058ff0 con 0x7f8dcc161950 2024-05-09T06:27:45.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.834+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dcc0d21e0 con 0x7f8dcc161950 2024-05-09T06:27:45.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.836+0000 7f8dc37fe640 1 -- 172.21.15.1:0/2622733229 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f8db405d2f0 con 0x7f8dcc161950 2024-05-09T06:27:45.818 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:45.818 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 msgr2=0x7f8dac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 0x7f8dac06e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc033000 tx=0x7f8dbc009000 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 msgr2=0x7f8dcc15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e9e0 tx=0x7f8db400eea0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 shutdown_connections 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8dac06be60 0x7f8dac06e320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dcc0b7680 0x7f8dcc0b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dcc161950 0x7f8dcc15df10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 --2- 172.21.15.1:0/2622733229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dcc0d0f80 0x7f8dcc09f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.837+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 >> 172.21.15.1:0/2622733229 conn(0x7f8dcc0d3e20 msgr2=0x7f8dcc0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.838+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 shutdown_connections 2024-05-09T06:27:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.838+0000 7f8dd14f5640 1 -- 172.21.15.1:0/2622733229 wait complete. 2024-05-09T06:27:45.831 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T06:27:45.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:45.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.953+0000 7f0c8211b640 1 Processor -- start 2024-05-09T06:27:45.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.953+0000 7f0c8211b640 1 -- start start 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c8211b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c7c0d2be0 0x7f0c7c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c8211b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c8211b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c7c0b3ae0 con 0x7f0c7c15e170 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c8211b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0c7c0b0c50 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c8211b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7c0cf7b0 con 0x7f0c7c0cefa0 2024-05-09T06:27:45.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c7c0d2be0 0x7f0c7c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0c7c15e170 0x7f0c7c0d24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38412/0 (socket says 172.21.15.1:38412) 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7b7fe640 1 -- 172.21.15.1:0/4182282554 learned_addr learned my addr 172.21.15.1:0/4182282554 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7bfff640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7affd640 1 -- 172.21.15.1:0/4182282554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 msgr2=0x7f0c7c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7affd640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7affd640 1 -- 172.21.15.1:0/4182282554 >> v1:172.21.15.1:6789/0 conn(0x7f0c7c15e170 legacy=0x7f0c7c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7a7fc640 1 -- 172.21.15.1:0/4182282554 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557422837 0 0) 0x7f0c7c0b3ae0 con 0x7f0c7c15e170 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.954+0000 7f0c7affd640 1 -- 172.21.15.1:0/4182282554 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c0cfa90 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.955+0000 7f0c7affd640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0cea60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009790 tx=0x7f0c7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc667cb4e77379c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.955+0000 7f0c7a7fc640 1 -- 172.21.15.1:0/4182282554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70037070 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.955+0000 7f0c7a7fc640 1 -- 172.21.15.1:0/4182282554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70032440 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.955+0000 7f0c7a7fc640 1 -- 172.21.15.1:0/4182282554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70032810 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 -- 172.21.15.1:0/4182282554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c0d2be0 msgr2=0x7f0c7c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0cea60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009790 tx=0x7f0c7002a180 comp rx=0 tx=0).stop 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 -- 172.21.15.1:0/4182282554 shutdown_connections 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 --2- 172.21.15.1:0/4182282554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0cea60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 -- 172.21.15.1:0/4182282554 >> 172.21.15.1:0/4182282554 conn(0x7f0c7c073340 msgr2=0x7f0c7c09d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 -- 172.21.15.1:0/4182282554 shutdown_connections 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.956+0000 7f0c8211b640 1 -- 172.21.15.1:0/4182282554 wait complete. 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.957+0000 7f0c8211b640 1 Processor -- start 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.957+0000 7f0c8211b640 1 -- start start 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c0d0330 con 0x7f0c7c15e170 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c0d0050 con 0x7f0c7c0d2be0 2024-05-09T06:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c8211b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7c15e550 con 0x7f0c7c0cefa0 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53172/0 (socket says 172.21.15.1:53172) 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c7bfff640 1 -- 172.21.15.1:0/1366124521 learned_addr learned my addr 172.21.15.1:0/1366124521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.958+0000 7f0c7b7fe640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7affd640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 -- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 msgr2=0x7f0c7c152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 -- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c7c0d2be0 msgr2=0x7f0c7c0b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0b92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 -- 172.21.15.1:0/1366124521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c0b9f60 con 0x7f0c7c15e170 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7b7fe640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c7bfff640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007e60 tx=0x7f0c7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74014420 con 0x7f0c7c15e170 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c74019070 con 0x7f0c7c15e170 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c0b9c80 con 0x7f0c7c15e170 2024-05-09T06:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74014820 con 0x7f0c7c15e170 2024-05-09T06:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.959+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c15eb50 con 0x7f0c7c15e170 2024-05-09T06:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.960+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7c0752b0 con 0x7f0c7c15e170 2024-05-09T06:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.961+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c740149c0 con 0x7f0c7c15e170 2024-05-09T06:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.961+0000 7f0c63fff640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.961+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0c7408b930 con 0x7f0c7c15e170 2024-05-09T06:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.961+0000 7f0c7b7fe640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.962+0000 7f0c7b7fe640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 0x7f0c6806e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f60 tx=0x7f0c6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:45.965+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c7405a030 con 0x7f0c7c15e170 2024-05-09T06:27:46.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.149+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c7c0d2390 con 0x7f0c7c15e170 2024-05-09T06:27:46.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.150+0000 7f0c63fff640 1 -- 172.21.15.1:0/1366124521 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f0c74059930 con 0x7f0c7c15e170 2024-05-09T06:27:46.132 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:46.133 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 msgr2=0x7f0c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 0x7f0c6806e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f60 tx=0x7f0c6c009040 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 msgr2=0x7f0c7c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c74007e60 tx=0x7f0c7400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 shutdown_connections 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6806be30 0x7f0c6806e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c15e170 0x7f0c7c06cb40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c7c0d2be0 0x7f0c7c0b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 --2- 172.21.15.1:0/1366124521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c7c0cefa0 0x7f0c7c152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 >> 172.21.15.1:0/1366124521 conn(0x7f0c7c073340 msgr2=0x7f0c7c0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 shutdown_connections 2024-05-09T06:27:46.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.152+0000 7f0c8211b640 1 -- 172.21.15.1:0/1366124521 wait complete. 2024-05-09T06:27:46.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T06:27:46.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T06:27:46.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T06:27:46.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T06:27:46.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T06:27:46.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T06:27:46.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T06:27:46.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T06:27:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.343+0000 7fc4f9f7d640 1 Processor -- start 2024-05-09T06:27:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.344+0000 7fc4f9f7d640 1 -- start start 2024-05-09T06:27:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.344+0000 7fc4f9f7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 0x7fc4f40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f9f7d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f40cfaa0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f9f7d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f40cfd80 con 0x7fc4f40cec50 2024-05-09T06:27:46.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f9f7d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f40d0060 con 0x7fc4f415be60 2024-05-09T06:27:46.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f2ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc4f40cd8d0 0x7fc4f40ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38428/0 (socket says 172.21.15.1:38428) 2024-05-09T06:27:46.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f2ffd640 1 -- 172.21.15.1:0/3465019327 learned_addr learned my addr 172.21.15.1:0/3465019327 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.345+0000 7fc4f37fe640 1 --2- 172.21.15.1:0/3465019327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 0x7fc4f40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090770591 0 0) 0x7fc4f40cfaa0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4dc003520 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45353801 0 0) 0x7fc4f40cfd80 con 0x7fc4f40cec50 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4f40cfaa0 con 0x7fc4f40cec50 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f37fe640 1 -- 172.21.15.1:0/3465019327 >> v1:172.21.15.6:6789/0 conn(0x7fc4f40cec50 legacy=0x7fc4f40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f37fe640 1 -- 172.21.15.1:0/3465019327 >> v1:172.21.15.1:6789/0 conn(0x7fc4f40cd8d0 legacy=0x7fc4f40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.346+0000 7fc4f37fe640 1 -- 172.21.15.1:0/3465019327 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f40d0340 con 0x7fc4f415be60 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7ff4e605d640 1 Processor -- start 2024-05-09T06:27:46.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7ff4e605d640 1 -- start start 2024-05-09T06:27:46.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fc4f37fe640 1 --2- 172.21.15.1:0/3465019327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 0x7fc4f40cd390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40097d0 tx=0x7fc4e402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aee170f5ea15f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4037070 con 0x7fc4f415be60 2024-05-09T06:27:46.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e4032440 con 0x7fc4f415be60 2024-05-09T06:27:46.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fc4f27fc640 1 -- 172.21.15.1:0/3465019327 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4032850 con 0x7fc4f415be60 2024-05-09T06:27:46.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fded5fdc640 1 Processor -- start 2024-05-09T06:27:46.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7fded5fdc640 1 -- start start 2024-05-09T06:27:46.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7f1c3359e640 1 Processor -- start 2024-05-09T06:27:46.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7f1c3359e640 1 -- start start 2024-05-09T06:27:46.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/3465019327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 msgr2=0x7fc4f40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4e605d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 0x7ff4e00d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4e605d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e015e9d0 con 0x7ff4e00b2f30 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4e605d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e00d14f0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4e605d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e00b3480 con 0x7ff4e00b53a0 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fc4f9f7d640 1 --2- 172.21.15.1:0/3465019327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 0x7fc4f40cd390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40097d0 tx=0x7fc4e402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/3465019327 shutdown_connections 2024-05-09T06:27:46.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4df7fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff4e015e5f0 0x7ff4e00d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35058/0 (socket says 172.21.15.1:35058) 2024-05-09T06:27:46.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4df7fe640 1 -- 172.21.15.1:0/959894937 learned_addr learned my addr 172.21.15.1:0/959894937 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7ff4deffd640 1 --2- 172.21.15.1:0/959894937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 0x7ff4e00d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fded5fdc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 0x7fded015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fded5fdc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 0x7fded006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fc4f9f7d640 1 --2- 172.21.15.1:0/3465019327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f415be60 0x7fc4f40cd390 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fdecf7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 0x7fded015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7f39d62bb640 1 Processor -- start 2024-05-09T06:27:46.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.347+0000 7f39d62bb640 1 -- start start 2024-05-09T06:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.348+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/3465019327 >> 172.21.15.1:0/3465019327 conn(0x7fc4f40d1bc0 msgr2=0x7fc4f40d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d62bb640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f39d0072ca0 0x7f39d00ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d62bb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f39d00d04a0 0x7f39d00aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d62bb640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d00cdad0 con 0x7f39d0162430 2024-05-09T06:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d62bb640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f39d00cd880 con 0x7f39d0072ca0 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d62bb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f39d00ce6e0 con 0x7f39d00d04a0 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d52b9640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f39d0162430 0x7f39d00ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35066/0 (socket says 172.21.15.1:35066) 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d52b9640 1 -- 172.21.15.1:0/397134826 learned_addr learned my addr 172.21.15.1:0/397134826 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fdecf7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 0x7fded015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53180/0 (socket says 172.21.15.1:53180) 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d5aba640 1 --2- 172.21.15.1:0/397134826 >> v2:172.21.15.6:3300/0 conn(0x7f39d00d04a0 0x7f39d00aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fded5fdc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded015d2c0 con 0x7fded015f8f0 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fded5fdc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fded006e540 con 0x7fded0145590 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fded5fdc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fded006f620 con 0x7fded006ec80 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f1c3359e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1c3415a6e0 0x7f1c340b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f1c3359e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c3415ea40 0x7f1c34074450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fdeceffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 0x7fded006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fdecffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fded015f8f0 0x7fded015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38450/0 (socket says 172.21.15.1:38450) 2024-05-09T06:27:46.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7fdecffff640 1 -- 172.21.15.1:0/2020386297 learned_addr learned my addr 172.21.15.1:0/2020386297 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f1c3359e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c340ce0e0 con 0x7f1c340cc9f0 2024-05-09T06:27:46.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f1c3359e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1c340ce3e0 con 0x7f1c3415a6e0 2024-05-09T06:27:46.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f1c3359e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3415eec0 con 0x7f1c3415ea40 2024-05-09T06:27:46.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f39d4ab8640 1 --2- 172.21.15.1:0/397134826 >> v2:172.21.15.1:3300/0 conn(0x7f39d0072ca0 0x7f39d00ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39c7fff640 1 -- 172.21.15.1:0/397134826 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1678169678 0 0) 0x7f39d00cdad0 con 0x7f39d0162430 2024-05-09T06:27:46.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c3415ea40 0x7f1c34074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39c7fff640 1 -- 172.21.15.1:0/397134826 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39bc003520 con 0x7f39d0162430 2024-05-09T06:27:46.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39d5aba640 1 -- 172.21.15.1:0/397134826 >> v2:172.21.15.1:3300/0 conn(0x7f39d0072ca0 msgr2=0x7f39d00ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39d5aba640 1 --2- 172.21.15.1:0/397134826 >> v2:172.21.15.1:3300/0 conn(0x7f39d0072ca0 0x7f39d00ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f9780f5f640 1 Processor -- start 2024-05-09T06:27:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7f9780f5f640 1 -- start start 2024-05-09T06:27:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39d5aba640 1 -- 172.21.15.1:0/397134826 >> v1:172.21.15.6:6789/0 conn(0x7f39d0162430 legacy=0x7f39d00ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f39d5aba640 1 -- 172.21.15.1:0/397134826 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00cddb0 con 0x7f39d00d04a0 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c3415ea40 0x7f1c34074450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33446/0 (socket says 172.21.15.1:33446) 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 -- 172.21.15.1:0/3790081602 learned_addr learned my addr 172.21.15.1:0/3790081602 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7ff4deffd640 1 -- 172.21.15.1:0/959894937 >> v1:172.21.15.6:6789/0 conn(0x7ff4e015e5f0 legacy=0x7ff4e00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7ff4deffd640 1 -- 172.21.15.1:0/959894937 >> v1:172.21.15.1:6789/0 conn(0x7ff4e00b2f30 legacy=0x7ff4e00b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.349+0000 7ff4deffd640 1 -- 172.21.15.1:0/959894937 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e00cf820 con 0x7ff4e00b53a0 2024-05-09T06:27:46.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c3259c640 1 --2- 172.21.15.1:0/3790081602 >> v2:172.21.15.1:3300/0 conn(0x7f1c3415a6e0 0x7f1c340b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7ff2660c7640 1 Processor -- start 2024-05-09T06:27:46.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 -- 172.21.15.1:0/3790081602 >> v2:172.21.15.1:3300/0 conn(0x7f1c3415a6e0 msgr2=0x7f1c340b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 --2- 172.21.15.1:0/3790081602 >> v2:172.21.15.1:3300/0 conn(0x7f1c3415a6e0 0x7f1c340b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 -- 172.21.15.1:0/3790081602 >> v1:172.21.15.1:6789/0 conn(0x7f1c340cc9f0 legacy=0x7f1c340727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7fdeceffd640 1 -- 172.21.15.1:0/2020386297 >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 msgr2=0x7fded015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7fdeceffd640 1 --2- 172.21.15.1:0/2020386297 >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 0x7fded015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7fdeceffd640 1 -- 172.21.15.1:0/2020386297 >> v1:172.21.15.1:6789/0 conn(0x7fded015f8f0 legacy=0x7fded015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7fdeceffd640 1 -- 172.21.15.1:0/2020386297 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded006d200 con 0x7fded006ec80 2024-05-09T06:27:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7ff2660c7640 1 -- start start 2024-05-09T06:27:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.350+0000 7f1c31d9b640 1 -- 172.21.15.1:0/3790081602 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3406d590 con 0x7f1c3415ea40 2024-05-09T06:27:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f9780f5f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f977c1446b0 0x7f977c15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f9780f5f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f977c0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f9780f5f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977c144a90 con 0x7f977c15ed80 2024-05-09T06:27:46.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f9780f5f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f977c06dd00 con 0x7f977c1446b0 2024-05-09T06:27:46.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f9780f5f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f977c15f2c0 con 0x7f977c0cdc00 2024-05-09T06:27:46.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/3465019327 shutdown_connections 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977affd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f977c15ed80 0x7f977c0cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35078/0 (socket says 172.21.15.1:35078) 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977affd640 1 -- 172.21.15.1:0/2304666312 learned_addr learned my addr 172.21.15.1:0/2304666312 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977bfff640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f977c0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2304666312 >> v2:172.21.15.1:3300/0 conn(0x7f977c1446b0 0x7f977c15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325665816 0 0) 0x7f977c144a90 con 0x7f977c15ed80 2024-05-09T06:27:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/3465019327 wait complete. 2024-05-09T06:27:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9764003520 con 0x7f977c15ed80 2024-05-09T06:27:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f39d5aba640 1 --2- 172.21.15.1:0/397134826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d00d04a0 0x7f39d00aecb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39d00d0950 tx=0x7f39c002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6283dbabf3c57b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.351+0000 7f1c31d9b640 1 --2- 172.21.15.1:0/3790081602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c3415ea40 0x7f1c34074450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c3406cbb0 tx=0x7f1c2802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f2547e374a89e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7fdeceffd640 1 --2- 172.21.15.1:0/2020386297 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 0x7fded006f0e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdec40097a0 tx=0x7fdec402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df4e5df5a6e2e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383717579 0 0) 0x7f9764003520 con 0x7f977c15ed80 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 msgr2=0x7f977c0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f977c0cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 >> v2:172.21.15.1:3300/0 conn(0x7f977c1446b0 msgr2=0x7f977c15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 --2- 172.21.15.1:0/2304666312 >> v2:172.21.15.1:3300/0 conn(0x7f977c1446b0 0x7f977c15e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f39c7fff640 1 -- 172.21.15.1:0/397134826 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0037070 con 0x7f39d00d04a0 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f39c7fff640 1 -- 172.21.15.1:0/397134826 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c0032440 con 0x7f39d00d04a0 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c15f5a0 con 0x7f977c15ed80 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f9774001670 con 0x7f977c15ed80 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 >> v1:172.21.15.6:6789/0 conn(0x7f977c15ed80 legacy=0x7f977c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f97640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9764003f30 0x7f9764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9764004d40 0x7f97640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f977c06dd00 con 0x7f9764003f30 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f977c15f2c0 con 0x7f9764004d40 2024-05-09T06:27:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f976400ba10 con 0x7f977c0cdc00 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2660c7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff26015ae30 0x7ff2600ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2660c7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2600b3c50 0x7ff2600b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2660c7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2600af210 con 0x7ff2600aecb0 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2660c7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600b2f20 con 0x7ff26015ae30 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2660c7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600b3170 con 0x7ff2600b3c50 2024-05-09T06:27:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f97640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2658c6640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2600b3c50 0x7ff2600b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2648c4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff2600aecb0 0x7ff26006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38464/0 (socket says 172.21.15.1:38464) 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2648c4640 1 -- 172.21.15.1:0/637016259 learned_addr learned my addr 172.21.15.1:0/637016259 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2650c5640 1 --2- 172.21.15.1:0/637016259 >> v2:172.21.15.1:3300/0 conn(0x7ff26015ae30 0x7ff2600ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f977bfff640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9764004d40 0x7f97640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f1c3159a640 1 -- 172.21.15.1:0/3790081602 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28037070 con 0x7f1c3415ea40 2024-05-09T06:27:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f1c3159a640 1 -- 172.21.15.1:0/3790081602 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c28032440 con 0x7f1c3415ea40 2024-05-09T06:27:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7f1c3159a640 1 -- 172.21.15.1:0/3790081602 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28032810 con 0x7f1c3415ea40 2024-05-09T06:27:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7fdece7fc640 1 -- 172.21.15.1:0/2020386297 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4037070 con 0x7fded006ec80 2024-05-09T06:27:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7fdece7fc640 1 -- 172.21.15.1:0/2020386297 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec402ae50 con 0x7fded006ec80 2024-05-09T06:27:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff4deffd640 1 --2- 172.21.15.1:0/959894937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 0x7ff4e00d0d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009750 tx=0x7ff4c802a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4084845a760998ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f39d62bb640 1 -- 172.21.15.1:0/397134826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d00d04a0 msgr2=0x7f39d00aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f39d62bb640 1 --2- 172.21.15.1:0/397134826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d00d04a0 0x7f39d00aecb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39d00d0950 tx=0x7f39c002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7ff4de7fc640 1 -- 172.21.15.1:0/959894937 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8037070 con 0x7ff4e00b53a0 2024-05-09T06:27:46.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7ff4de7fc640 1 -- 172.21.15.1:0/959894937 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c802adb0 con 0x7ff4e00b53a0 2024-05-09T06:27:46.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7fc4f9f7d640 1 Processor -- start 2024-05-09T06:27:46.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7fc4f9f7d640 1 -- start start 2024-05-09T06:27:46.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f1c3359e640 1 -- 172.21.15.1:0/3790081602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c3415ea40 msgr2=0x7f1c34074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2650c5640 1 -- 172.21.15.1:0/637016259 >> v2:172.21.15.6:3300/0 conn(0x7ff2600b3c50 msgr2=0x7ff2600b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7ff4de7fc640 1 -- 172.21.15.1:0/959894937 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8032660 con 0x7ff4e00b53a0 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2650c5640 1 --2- 172.21.15.1:0/637016259 >> v2:172.21.15.6:3300/0 conn(0x7ff2600b3c50 0x7ff2600b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2650c5640 1 -- 172.21.15.1:0/637016259 >> v1:172.21.15.1:6789/0 conn(0x7ff2600aecb0 legacy=0x7ff26006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.352+0000 7ff2650c5640 1 -- 172.21.15.1:0/637016259 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600b3450 con 0x7ff26015ae30 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f977b7fe640 1 -- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9764004d40 msgr2=0x7f97640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9764004d40 0x7f97640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f977b7fe640 1 -- 172.21.15.1:0/2304666312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9764003f30 msgr2=0x7f9764004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9764003f30 0x7f9764004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f977b7fe640 1 -- 172.21.15.1:0/2304666312 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c15f5a0 con 0x7f977c0cdc00 2024-05-09T06:27:46.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 -- 172.21.15.1:0/397134826 shutdown_connections 2024-05-09T06:27:46.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 --2- 172.21.15.1:0/397134826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d00d04a0 0x7f39d00aecb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 --2- 172.21.15.1:0/397134826 >> v2:172.21.15.1:3300/0 conn(0x7f39d0072ca0 0x7f39d00ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 -- 172.21.15.1:0/397134826 >> 172.21.15.1:0/397134826 conn(0x7f39d00cf980 msgr2=0x7f39d00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7f1c3359e640 1 --2- 172.21.15.1:0/3790081602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c3415ea40 0x7f1c34074450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c3406cbb0 tx=0x7f1c2802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.353+0000 7fdece7fc640 1 -- 172.21.15.1:0/2020386297 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4037070 con 0x7fded006ec80 2024-05-09T06:27:46.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 -- 172.21.15.1:0/2020386297 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 msgr2=0x7fded006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 --2- 172.21.15.1:0/2020386297 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 0x7fded006f0e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdec40097a0 tx=0x7fdec402a100 comp rx=0 tx=0).stop 2024-05-09T06:27:46.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 -- 172.21.15.1:0/3790081602 shutdown_connections 2024-05-09T06:27:46.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 -- 172.21.15.1:0/397134826 shutdown_connections 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff4e605d640 1 -- 172.21.15.1:0/959894937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 msgr2=0x7ff4e00d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff4e605d640 1 --2- 172.21.15.1:0/959894937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 0x7ff4e00d0d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009750 tx=0x7ff4c802a0a0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff4e605d640 1 -- 172.21.15.1:0/959894937 shutdown_connections 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff4e605d640 1 --2- 172.21.15.1:0/959894937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b53a0 0x7ff4e00d0d60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff4e605d640 1 -- 172.21.15.1:0/959894937 >> 172.21.15.1:0/959894937 conn(0x7ff4e00d3e90 msgr2=0x7ff4e00d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 -- 172.21.15.1:0/2020386297 shutdown_connections 2024-05-09T06:27:46.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 --2- 172.21.15.1:0/2020386297 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded006ec80 0x7fded006f0e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 --2- 172.21.15.1:0/2020386297 >> v2:172.21.15.1:3300/0 conn(0x7fded0145590 0x7fded015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f39d62bb640 1 -- 172.21.15.1:0/397134826 wait complete. 2024-05-09T06:27:46.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff2650c5640 1 --2- 172.21.15.1:0/637016259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff26015ae30 0x7ff2600ae770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b29c0 tx=0x7ff25c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db84bf1e63c8d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 -- 172.21.15.1:0/2020386297 >> 172.21.15.1:0/2020386297 conn(0x7fded00cc4b0 msgr2=0x7fded00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 -- 172.21.15.1:0/2020386297 shutdown_connections 2024-05-09T06:27:46.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7fded5fdc640 1 -- 172.21.15.1:0/2020386297 wait complete. 2024-05-09T06:27:46.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff257fff640 1 -- 172.21.15.1:0/637016259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c037070 con 0x7ff26015ae30 2024-05-09T06:27:46.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff257fff640 1 -- 172.21.15.1:0/637016259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c02ae00 con 0x7ff26015ae30 2024-05-09T06:27:46.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f9f7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 0x7fc4f40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7ff257fff640 1 -- 172.21.15.1:0/637016259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c032660 con 0x7ff26015ae30 2024-05-09T06:27:46.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f9f7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 0x7fc4f40ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f9f7d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f415be60 0x7fc4f41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f97640039b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9770009af0 tx=0x7f9764005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4245c5645c2e857b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7ff4e605d640 1 -- 172.21.15.1:0/959894937 shutdown_connections 2024-05-09T06:27:46.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 0x7fc4f40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 --2- 172.21.15.1:0/3790081602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c3415ea40 0x7f1c34074450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 --2- 172.21.15.1:0/3790081602 >> v2:172.21.15.1:3300/0 conn(0x7f1c3415a6e0 0x7f1c340b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 -- 172.21.15.1:0/3790081602 >> 172.21.15.1:0/3790081602 conn(0x7f1c340b1d30 msgr2=0x7f1c340b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 -- 172.21.15.1:0/3790081602 shutdown_connections 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.354+0000 7f1c3359e640 1 -- 172.21.15.1:0/3790081602 wait complete. 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7ff2660c7640 1 -- 172.21.15.1:0/637016259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff26015ae30 msgr2=0x7ff2600ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7ff2660c7640 1 --2- 172.21.15.1:0/637016259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff26015ae30 0x7ff2600ae770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b29c0 tx=0x7ff25c02a0b0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770009690 con 0x7f977c0cdc00 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9770032410 con 0x7f977c0cdc00 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7f977a7fc640 1 -- 172.21.15.1:0/2304666312 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770009690 con 0x7f977c0cdc00 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 0x7fc4f40cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53222/0 (socket says 172.21.15.1:53222) 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f2ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 0x7fc4f40ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7fc4f2ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 0x7fc4f40ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33070/0 (socket says 172.21.15.1:33070) 2024-05-09T06:27:46.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7f013c064640 1 Processor -- start 2024-05-09T06:27:46.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 -- 172.21.15.1:0/1456409787 learned_addr learned my addr 172.21.15.1:0/1456409787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f415c240 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40cf790 con 0x7fc4f415be60 2024-05-09T06:27:46.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f415c3c0 con 0x7fc4f40cec50 2024-05-09T06:27:46.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f3fff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f415be60 0x7fc4f41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7f013c064640 1 -- start start 2024-05-09T06:27:46.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7f39d62bb640 1 Processor -- start 2024-05-09T06:27:46.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 -- 172.21.15.1:0/637016259 shutdown_connections 2024-05-09T06:27:46.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 --2- 172.21.15.1:0/637016259 >> v2:172.21.15.6:3300/0 conn(0x7ff2600b3c50 0x7ff2600b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 --2- 172.21.15.1:0/637016259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff26015ae30 0x7ff2600ae770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 -- 172.21.15.1:0/637016259 >> 172.21.15.1:0/637016259 conn(0x7ff2600b1610 msgr2=0x7ff2600b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 -- 172.21.15.1:0/637016259 shutdown_connections 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.355+0000 7ff4e605d640 1 -- 172.21.15.1:0/959894937 wait complete. 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 -- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 msgr2=0x7fc4f40ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 0x7fc4f40ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7fded5fdc640 1 Processor -- start 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 -- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f415be60 msgr2=0x7fc4f41549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f415be60 0x7fc4f41549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7fc4f37fe640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4154ef0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7ff4e605d640 1 Processor -- start 2024-05-09T06:27:46.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f9780f5f640 1 -- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 msgr2=0x7f97640039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f9780f5f640 1 --2- 172.21.15.1:0/2304666312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c0cdc00 0x7f97640039b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9770009af0 tx=0x7f9764005830 comp rx=0 tx=0).stop 2024-05-09T06:27:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f977bfff640 1 -- 172.21.15.1:0/2304666312 reap_dead start 2024-05-09T06:27:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f1c3359e640 1 Processor -- start 2024-05-09T06:27:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f013c064640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f013415a800 0x7f013415e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7fded5fdc640 1 -- start start 2024-05-09T06:27:46.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f1c3359e640 1 -- start start 2024-05-09T06:27:46.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f9780f5f640 1 -- 172.21.15.1:0/2304666312 shutdown_connections 2024-05-09T06:27:46.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f9780f5f640 1 -- 172.21.15.1:0/2304666312 >> 172.21.15.1:0/2304666312 conn(0x7f977c0d0cc0 msgr2=0x7f977c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f013c064640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f01340ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7f39d62bb640 1 -- start start 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 0x7f39d00af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d00d04a0 0x7f39d00ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d0162430 0x7f39d00adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39d006f510 con 0x7f39d00d04a0 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00ce370 con 0x7f39d0162430 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d62bb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f39d0162810 con 0x7f39d0072ca0 2024-05-09T06:27:46.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f9780f5f640 1 -- 172.21.15.1:0/2304666312 shutdown_connections 2024-05-09T06:27:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f013c064640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340ce2e0 con 0x7f01340cfe00 2024-05-09T06:27:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f013c064640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f013406c110 con 0x7f013415a800 2024-05-09T06:27:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7f39d52b9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 0x7f39d00af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 0x7f39d00af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33072/0 (socket says 172.21.15.1:33072) 2024-05-09T06:27:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 -- 172.21.15.1:0/1285366115 learned_addr learned my addr 172.21.15.1:0/1285366115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f9780f5f640 1 -- 172.21.15.1:0/2304666312 wait complete. 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f0139e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f013415a800 0x7f013415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f0139e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f013415a800 0x7f013415e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53236/0 (socket says 172.21.15.1:53236) 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f0139e00640 1 -- 172.21.15.1:0/3534466757 learned_addr learned my addr 172.21.15.1:0/3534466757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f01340b1950 con 0x7f01340ad2b0 2024-05-09T06:27:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d5aba640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d0162430 0x7f39d00adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7fc4f2ffd640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4f40cec50 0x7fc4f40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7fc4f3fff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4f415be60 0x7fc4f41549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 -- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d0162430 msgr2=0x7f39d00adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f0139e00640 1 -- 172.21.15.1:0/3534466757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 msgr2=0x7f01340ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7fc4f37fe640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 0x7fc4f40cb930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40097d0 tx=0x7fc4e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4042020 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e403d730 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4037040 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.357+0000 7ff4e605d640 1 -- start start 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7ff4e605d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b2f30 0x7ff4e00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4e605d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4e00b53a0 0x7ff4e01f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4e605d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 0x7ff4e014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4e605d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e00cfde0 con 0x7ff4e00b53a0 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4e605d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e00d12a0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4e605d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e00b4020 con 0x7ff4e00b2f30 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39d0162430 0x7f39d00adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 -- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d00d04a0 msgr2=0x7f39d00ad760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:27:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d00d04a0 0x7f39d00ad760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f39d52b9640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d014a6e0 con 0x7f39d0072ca0 2024-05-09T06:27:46.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f013a601640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f01340ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c340cc9f0 0x7f1c341fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f40b60d0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fded5fdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 0x7fded00ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c3415a6e0 0x7f1c34159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 0x7f1c340c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7f0139e00640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f01340ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fded5fdc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fded0145590 0x7fded01430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fdecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 0x7fded00ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3406dcd0 con 0x7f1c3415ea40 2024-05-09T06:27:46.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fdecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 0x7fded00ac060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53250/0 (socket says 172.21.15.1:53250) 2024-05-09T06:27:46.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3406db50 con 0x7f1c340cc9f0 2024-05-09T06:27:46.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c3359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c34074990 con 0x7f1c3415a6e0 2024-05-09T06:27:46.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f0139e00640 1 -- 172.21.15.1:0/3534466757 >> v1:172.21.15.6:6789/0 conn(0x7f01340cfe00 legacy=0x7f013415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c32d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 0x7f1c340c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fdecf7fe640 1 -- 172.21.15.1:0/3999720789 learned_addr learned my addr 172.21.15.1:0/3999720789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c32d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 0x7f1c340c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53254/0 (socket says 172.21.15.1:53254) 2024-05-09T06:27:46.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c32d9d640 1 -- 172.21.15.1:0/1579346318 learned_addr learned my addr 172.21.15.1:0/1579346318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f1c31d9b640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c3415a6e0 0x7f1c34159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f1c3259c640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c340cc9f0 0x7f1c341fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f40b6690 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.356+0000 7ff2660c7640 1 -- 172.21.15.1:0/637016259 wait complete. 2024-05-09T06:27:46.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7ff2660c7640 1 Processor -- start 2024-05-09T06:27:46.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.358+0000 7ff2660c7640 1 -- start start 2024-05-09T06:27:46.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 0x7ff2600a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f9780f5f640 1 Processor -- start 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f9780f5f640 1 -- start start 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2600b3c50 0x7ff26013f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff26015ae30 0x7ff26013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff26006d410 con 0x7ff2600aecb0 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff26006d290 con 0x7ff26015ae30 2024-05-09T06:27:46.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2660c7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff2600cd880 con 0x7ff2600b3c50 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7f0139e00640 1 -- 172.21.15.1:0/3534466757 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340ce5a0 con 0x7f013415a800 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f0139e00640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 0x7f013415e490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f013000bb10 tx=0x7f013002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5534627b6f8b0832 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f39d52b9640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 0x7f39d00af720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c440 tx=0x7f39cc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f0138dfe640 1 -- 172.21.15.1:0/3534466757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130036020 con 0x7f013415a800 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f0138dfe640 1 -- 172.21.15.1:0/3534466757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130032820 con 0x7f013415a800 2024-05-09T06:27:46.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f0138dfe640 1 -- 172.21.15.1:0/3534466757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130032bf0 con 0x7f013415a800 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2650c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 0x7ff2600a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2650c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 0x7ff2600a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53262/0 (socket says 172.21.15.1:53262) 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff2650c5640 1 -- 172.21.15.1:0/1819226019 learned_addr learned my addr 172.21.15.1:0/1819226019 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc017070 con 0x7f39d0072ca0 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc005ce0 con 0x7f39d0072ca0 2024-05-09T06:27:46.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0072b0 con 0x7f39d0072ca0 2024-05-09T06:27:46.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7fdeceffd640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fded0145590 0x7fded01430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f1c32d9d640 1 -- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c3415a6e0 msgr2=0x7f1c34159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f1c32d9d640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c3415a6e0 0x7f1c34159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f1c32d9d640 1 -- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c340cc9f0 msgr2=0x7f1c341fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f1c32d9d640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c340cc9f0 0x7f1c341fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0cdc00 0x7f977c137020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c13f9d0 0x7f977c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15d4f0 0x7f977c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0b3c40 con 0x7f977c0cdc00 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0cf1d0 con 0x7f977c13f9d0 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f9780f5f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f977c0b3dc0 con 0x7f977c15d4f0 2024-05-09T06:27:46.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 msgr2=0x7f013415e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 0x7f013415e490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f013000bb10 tx=0x7f013002c920 comp rx=0 tx=0).stop 2024-05-09T06:27:46.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f977bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15d4f0 0x7f977c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f39d62bb640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d014a9c0 con 0x7f39d0072ca0 2024-05-09T06:27:46.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f1c32d9d640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c340b5e90 con 0x7f1c3415ea40 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f977bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15d4f0 0x7f977c15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33104/0 (socket says 172.21.15.1:33104) 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f977bfff640 1 -- 172.21.15.1:0/2597647555 learned_addr learned my addr 172.21.15.1:0/2597647555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 shutdown_connections 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f01340ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 --2- 172.21.15.1:0/3534466757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 0x7f013415e490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 >> 172.21.15.1:0/3534466757 conn(0x7f01340d0260 msgr2=0x7f01340d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.359+0000 7ff4deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4e00b53a0 0x7ff4e01f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 0x7ff4e014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 0x7ff4e014d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33484/0 (socket says 172.21.15.1:33484) 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 -- 172.21.15.1:0/692342152 learned_addr learned my addr 172.21.15.1:0/692342152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4df7fe640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b2f30 0x7ff4e00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 -- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b2f30 msgr2=0x7ff4e00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4e00b2f30 0x7ff4e00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.360+0000 7ff4dffff640 1 -- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4e00b53a0 msgr2=0x7ff4e01f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7ff4dffff640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4e00b53a0 0x7ff4e01f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7ff4dffff640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01f8070 con 0x7ff4e015e5f0 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7ff4deffd640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4e00b53a0 0x7ff4e01f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7ff4dffff640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 0x7ff4e014d250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800c430 tx=0x7ff4d800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f39d62bb640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d014af80 con 0x7f39d0072ca0 2024-05-09T06:27:46.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7f977affd640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c13f9d0 0x7f977c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fded5fdc640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded015f8f0 0x7fded00ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fded5fdc640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fded006d780 con 0x7fded006ec80 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fded5fdc640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fded006e0b0 con 0x7fded0145590 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fded5fdc640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fded006da60 con 0x7fded015f8f0 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2650c5640 1 -- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2600b3c50 msgr2=0x7ff26013f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2650c5640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2600b3c50 0x7ff26013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2658c6640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff26015ae30 0x7ff26013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2650c5640 1 -- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff26015ae30 msgr2=0x7ff26013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2650c5640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff26015ae30 0x7ff26013f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff2650c5640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600a9ab0 con 0x7ff2600aecb0 2024-05-09T06:27:46.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8015e30 con 0x7ff4e015e5f0 2024-05-09T06:27:46.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.361+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4e4045020 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d80050d0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4f40cbfa0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0cdc00 0x7f977c137020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fc4dbfff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4e006be30 0x7fc4e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 shutdown_connections 2024-05-09T06:27:46.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f39d62bb640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39d00752b0 con 0x7f39d0072ca0 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977affd640 1 -- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15d4f0 msgr2=0x7f977c15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977affd640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15d4f0 0x7f977c15d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977affd640 1 -- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0cdc00 msgr2=0x7f977c137020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977affd640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0cdc00 0x7f977c137020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977affd640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c137560 con 0x7f977c13f9d0 2024-05-09T06:27:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fdecffff640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded015f8f0 0x7fded00ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0cdc00 0x7f977c137020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:46.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f013c064640 1 -- 172.21.15.1:0/3534466757 wait complete. 2024-05-09T06:27:46.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4e40abbd0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fc4f2ffd640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4e006be30 0x7fc4e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7fc4dbfff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7fc4e0075530 0x7fc4e0077970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4e40093e0 con 0x7fc4e0075530 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4e4075ef0 con 0x7fc4f40cd8d0 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f977affd640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c13f9d0 0x7f977c13fe50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f977400c6d0 tx=0x7f977400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7ff2650c5640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 0x7ff2600a9480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009890 tx=0x7ff25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fdecf7fe640 1 -- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded015f8f0 msgr2=0x7fded00ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fdecf7fe640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fded015f8f0 0x7fded00ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fdecf7fe640 1 -- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fded0145590 msgr2=0x7fded01430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fdecf7fe640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fded0145590 0x7fded01430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fdecf7fe640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded00c4b50 con 0x7fded006ec80 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774018070 con 0x7f977c13f9d0 2024-05-09T06:27:46.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7fc4f2ffd640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4e006be30 0x7fc4e006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415f080 tx=0x7fc4e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.362+0000 7f1c32d9d640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 0x7f1c340c4d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00c780 tx=0x7f1c2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c004b60 con 0x7f1c3415ea40 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c341fc6c0 con 0x7f1c3415ea40 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c34154c80 con 0x7f1c3415ea40 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c004d00 con 0x7f1c3415ea40 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f9780f5f640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1581a0 con 0x7f977c13f9d0 2024-05-09T06:27:46.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00aa90 con 0x7f1c3415ea40 2024-05-09T06:27:46.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7f9780f5f640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c158760 con 0x7f977c13f9d0 2024-05-09T06:27:46.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7fdecf7fe640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 0x7fded00ac060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdec000d800 tx=0x7fdec000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7fc4f3fff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7fc4e0075530 0x7fc4e0077970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97740072d0 con 0x7f977c13f9d0 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01f8350 con 0x7ff4e015e5f0 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.363+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01f45c0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8007020 con 0x7ff4e015e5f0 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c18001030 con 0x7f1c3415ea40 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c041070 con 0x7ff2600aecb0 2024-05-09T06:27:46.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2600a9d90 con 0x7ff2600aecb0 2024-05-09T06:27:46.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7f013c064640 1 Processor -- start 2024-05-09T06:27:46.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7f013c064640 1 -- start start 2024-05-09T06:27:46.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.364+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2600aa350 con 0x7ff2600aecb0 2024-05-09T06:27:46.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39cc01a070 con 0x7f39d0072ca0 2024-05-09T06:27:46.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec000b6c0 con 0x7fded006ec80 2024-05-09T06:27:46.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c03c460 con 0x7ff2600aecb0 2024-05-09T06:27:46.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c037040 con 0x7ff2600aecb0 2024-05-09T06:27:46.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7fded5fdc640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded00ac7d0 con 0x7fded006ec80 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7fded5fdc640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded00acd90 con 0x7fded006ec80 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7f013c064640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f0134156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.365+0000 7f013c064640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 0x7f0134157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f013c064640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 0x7f01341582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f013c064640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f013415abe0 con 0x7f013415a800 2024-05-09T06:27:46.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f013c064640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f01340b15e0 con 0x7f01340cfe00 2024-05-09T06:27:46.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f013c064640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f013415ad60 con 0x7f01340ad2b0 2024-05-09T06:27:46.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c2c00ac30 con 0x7f1c3415ea40 2024-05-09T06:27:46.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 0x7f0134157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 0x7f0134157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33518/0 (socket says 172.21.15.1:33518) 2024-05-09T06:27:46.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f39c5ffb640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f39b406bf00 0x7f39b406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 -- 172.21.15.1:0/1333445735 learned_addr learned my addr 172.21.15.1:0/1333445735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f0139e00640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f0134156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0019070 con 0x7fded006ec80 2024-05-09T06:27:46.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 -- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 msgr2=0x7f0134156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97740047e0 con 0x7f977c13f9d0 2024-05-09T06:27:46.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff240001030 con 0x7ff2600aecb0 2024-05-09T06:27:46.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f9780f5f640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f975c001030 con 0x7f977c13f9d0 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01340ad2b0 0x7f0134156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 -- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 msgr2=0x7f01341582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f013415a800 0x7f01341582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f01395ff640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340b4160 con 0x7f01340cfe00 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fded00acfd0 con 0x7fded006ec80 2024-05-09T06:27:46.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f1c22ffd640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c1406be30 0x7f1c1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.366+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f977401b070 con 0x7f977c13f9d0 2024-05-09T06:27:46.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7fc4f3fff640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7fc4e0075530 0x7fc4e0077970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1c2c08bee0 con 0x7f1c3415ea40 2024-05-09T06:27:46.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c22ffd640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f1c14075530 0x7f1c14077970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c2c025e50 con 0x7f1c14075530 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c08c2d0 con 0x7f1c3415ea40 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c3259c640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c1406be30 0x7f1c1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec000ba60 con 0x7fded006ec80 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f1c31d9b640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f1c14075530 0x7f1c14077970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff25c03a070 con 0x7ff2600aecb0 2024-05-09T06:27:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e40093e0 con 0x7fc4e0075530 2024-05-09T06:27:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f39d4ab8640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f39b406bf00 0x7f39b406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.367+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f39cc08a8c0 con 0x7f39d0072ca0 2024-05-09T06:27:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f39c5ffb640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f39b4075670 0x7f39b4077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39cc036ec0 con 0x7f39b4075670 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f39cc08acb0 con 0x7f39d0072ca0 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f39d5aba640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f39b4075670 0x7f39b4077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f39d4ab8640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f39b406bf00 0x7f39b406e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39d006ebe0 tx=0x7f39c80097b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4e00752b0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7ff255ffb640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff24406be30 0x7ff24406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f1c3259c640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c1406be30 0x7f1c1406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009b40 tx=0x7f1c24009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff25c0abb50 con 0x7ff2600aecb0 2024-05-09T06:27:46.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7ff2648c4640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff24406be30 0x7ff24406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7f9763fff640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976806bf30 0x7f976806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.368+0000 7ff255ffb640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7ff244075520 0x7ff244077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff25c006810 con 0x7ff244075520 2024-05-09T06:27:46.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f977408a7a0 con 0x7f977c13f9d0 2024-05-09T06:27:46.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976806bf30 0x7f976806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f01395ff640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 0x7f0134157db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f012400c400 tx=0x7f012400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7ff2658c6640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7ff244075520 0x7ff244077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f9763fff640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f97680756a0 0x7f9768077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff25c075cc0 con 0x7ff2600aecb0 2024-05-09T06:27:46.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97740072d0 con 0x7f97680756a0 2024-05-09T06:27:46.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7ff2648c4640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff24406be30 0x7ff24406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff26014cc40 tx=0x7ff250007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f1c31d9b640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f1c14075530 0x7f1c14077970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f977408f1e0 con 0x7f977c13f9d0 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240052d0 con 0x7f01340cfe00 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0124005490 con 0x7f01340cfe00 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124007260 con 0x7f01340cfe00 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01340b4440 con 0x7f01340cfe00 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01340b4a00 con 0x7f01340cfe00 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f39d5aba640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f39b4075670 0x7f39b4077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c2c025e50 con 0x7f1c14075530 2024-05-09T06:27:46.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.369+0000 7f977bfff640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f97680756a0 0x7f9768077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976806bf30 0x7f976806e3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9770039000 tx=0x7f9770008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7f977bfff640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f97680756a0 0x7f9768077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4d80071c0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7ff2658c6640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7ff244075520 0x7ff244077960 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.370+0000 7ff4d7fff640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff4c006bf00 0x7ff4c006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0134158830 con 0x7f01340cfe00 2024-05-09T06:27:46.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdec000bc00 con 0x7fded006ec80 2024-05-09T06:27:46.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97740072d0 con 0x7f97680756a0 2024-05-09T06:27:46.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff25c006810 con 0x7ff244075520 2024-05-09T06:27:46.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39cc036ec0 con 0x7f39b4075670 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeb7fff640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdebc06be30 0x7fdebc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdec008bfe0 con 0x7fded006ec80 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeb7fff640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7fdebc0755b0 0x7fdebc0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdec0001d30 con 0x7fdebc0755b0 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdec008c3d0 con 0x7fded006ec80 2024-05-09T06:27:46.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7fdeceffd640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdebc06be30 0x7fdebc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff4d8092b20 con 0x7ff4e015e5f0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7ff4df7fe640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff4c006bf00 0x7ff4c006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7ff4df7fe640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff4c006bf00 0x7ff4c006e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00012e0 tx=0x7ff4d0005bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.371+0000 7ff4d7fff640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7ff4c0075670 0x7ff4c0077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4d80050d0 con 0x7ff4c0075670 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff4d80972a0 con 0x7ff4e015e5f0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.372+0000 7ff4deffd640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7ff4c0075670 0x7ff4c0077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.373+0000 7ff4deffd640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7ff4c0075670 0x7ff4c0077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.373+0000 7fdeceffd640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdebc06be30 0x7fdebc06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdec4034000 tx=0x7fdec402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.373+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0124007400 con 0x7f01340cfe00 2024-05-09T06:27:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.373+0000 7fdecffff640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7fdebc0755b0 0x7fdebc0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.374+0000 7fdecffff640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7fdebc0755b0 0x7fdebc0779f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.374+0000 7f012a7fc640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f011806be30 0x7f011806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.374+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4d80050d0 con 0x7ff4c0075670 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.374+0000 7f0139e00640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f011806be30 0x7f011806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.375+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f012408a780 con 0x7f01340cfe00 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.375+0000 7f012a7fc640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f01180754b0 0x7f01180778f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.375+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0124009e50 con 0x7f01180754b0 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.376+0000 7f013a601640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f01180754b0 0x7f01180778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.376+0000 7f0139e00640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f011806be30 0x7f011806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0130037000 tx=0x7f0130008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.376+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0124091050 con 0x7f01340cfe00 2024-05-09T06:27:46.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.375+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdec0001d30 con 0x7fdebc0755b0 2024-05-09T06:27:46.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.376+0000 7f013a601640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f01180754b0 0x7f01180778f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.377+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0124009e50 con 0x7f01180754b0 2024-05-09T06:27:46.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.397+0000 7fc4f9f7d640 1 -- 172.21.15.1:0/1456409787 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4f40752b0 con 0x7fc4e0075530 2024-05-09T06:27:46.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.398+0000 7fc4dbfff640 1 -- 172.21.15.1:0/1456409787 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc4f40752b0 con 0x7fc4e0075530 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.399+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7fc4e0075530 msgr2=0x7fc4e0077970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.399+0000 7fc4d9ffb640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7fc4e0075530 0x7fc4e0077970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.399+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4e006be30 msgr2=0x7fc4e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.399+0000 7fc4d9ffb640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4e006be30 0x7fc4e006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415f080 tx=0x7fc4e8009040 comp rx=0 tx=0).stop 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.400+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 msgr2=0x7fc4f40cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.400+0000 7fc4d9ffb640 1 --2- 172.21.15.1:0/1456409787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4f40cd8d0 0x7fc4f40cb930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40097d0 tx=0x7fc4e40046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.400+0000 7fc4f3fff640 1 -- 172.21.15.1:0/1456409787 reap_dead start 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.400+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 shutdown_connections 2024-05-09T06:27:46.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.400+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 >> 172.21.15.1:0/1456409787 conn(0x7fc4f40d1bc0 msgr2=0x7fc4f40cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.401+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 shutdown_connections 2024-05-09T06:27:46.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.401+0000 7fc4d9ffb640 1 -- 172.21.15.1:0/1456409787 wait complete. 2024-05-09T06:27:46.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.404+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff240002d50 con 0x7ff244075520 2024-05-09T06:27:46.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.404+0000 7f9780f5f640 1 -- 172.21.15.1:0/2597647555 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f975c002d50 con 0x7f97680756a0 2024-05-09T06:27:46.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.404+0000 7f39d62bb640 1 -- 172.21.15.1:0/1285366115 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39d00752b0 con 0x7f39b4075670 2024-05-09T06:27:46.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7f9763fff640 1 -- 172.21.15.1:0/2597647555 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f975c002d50 con 0x7f97680756a0 2024-05-09T06:27:46.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7ff255ffb640 1 -- 172.21.15.1:0/1819226019 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff240002d50 con 0x7ff244075520 2024-05-09T06:27:46.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7f39c5ffb640 1 -- 172.21.15.1:0/1285366115 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f39d00752b0 con 0x7f39b4075670 2024-05-09T06:27:46.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fded00752b0 con 0x7fdebc0755b0 2024-05-09T06:27:46.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f97680756a0 msgr2=0x7f9768077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7f9761ffb640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f97680756a0 0x7f9768077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f39b4075670 msgr2=0x7f39b4077ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f39b37fe640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f39b4075670 0x7f39b4077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c18002dd0 con 0x7f1c14075530 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976806bf30 msgr2=0x7f976806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976806bf30 0x7f976806e3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9770039000 tx=0x7f9770008030 comp rx=0 tx=0).stop 2024-05-09T06:27:46.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c13f9d0 msgr2=0x7f977c13fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 --2- 172.21.15.1:0/2597647555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c13f9d0 0x7f977c13fe50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f977400c6d0 tx=0x7f977400cb90 comp rx=0 tx=0).stop 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f977bfff640 1 -- 172.21.15.1:0/2597647555 reap_dead start 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7ff244075520 msgr2=0x7ff244077960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.405+0000 7ff2660c7640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7ff244075520 0x7ff244077960 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff24406be30 msgr2=0x7ff24406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff24406be30 0x7ff24406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff26014cc40 tx=0x7ff250007a40 comp rx=0 tx=0).stop 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 msgr2=0x7ff2600a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 --2- 172.21.15.1:0/1819226019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2600aecb0 0x7ff2600a9480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009890 tx=0x7ff25c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2658c6640 1 -- 172.21.15.1:0/1819226019 reap_dead start 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 shutdown_connections 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 >> 172.21.15.1:0/2597647555 conn(0x7f977c0d0cc0 msgr2=0x7f977c0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 shutdown_connections 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 >> 172.21.15.1:0/1819226019 conn(0x7ff2600b1610 msgr2=0x7ff2600af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f39b406bf00 msgr2=0x7f39b406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f39b37fe640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f39b406bf00 0x7f39b406e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39d006ebe0 tx=0x7f39c80097b0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 shutdown_connections 2024-05-09T06:27:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 msgr2=0x7f39d00af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39b37fe640 1 --2- 172.21.15.1:0/1285366115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39d0072ca0 0x7f39d00af720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c440 tx=0x7f39cc00c900 comp rx=0 tx=0).stop 2024-05-09T06:27:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7ff2660c7640 1 -- 172.21.15.1:0/1819226019 wait complete. 2024-05-09T06:27:46.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb7fff640 1 -- 172.21.15.1:0/3999720789 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fded00752b0 con 0x7fdebc0755b0 2024-05-09T06:27:46.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39d5aba640 1 -- 172.21.15.1:0/1285366115 reap_dead start 2024-05-09T06:27:46.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 shutdown_connections 2024-05-09T06:27:46.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 >> 172.21.15.1:0/1285366115 conn(0x7f39d00cf980 msgr2=0x7f39d00b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f1c22ffd640 1 -- 172.21.15.1:0/1579346318 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1c18002dd0 con 0x7f1c14075530 2024-05-09T06:27:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.406+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 shutdown_connections 2024-05-09T06:27:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 shutdown_connections 2024-05-09T06:27:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7fdebc0755b0 msgr2=0x7fdebc0779f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7fdebc0755b0 0x7fdebc0779f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f9761ffb640 1 -- 172.21.15.1:0/2597647555 wait complete. 2024-05-09T06:27:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f39b37fe640 1 -- 172.21.15.1:0/1285366115 wait complete. 2024-05-09T06:27:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f1c14075530 msgr2=0x7f1c14077970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7f1c3359e640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f1c14075530 0x7f1c14077970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdebc06be30 msgr2=0x7fdebc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdebc06be30 0x7fdebc06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdec4034000 tx=0x7fdec402e000 comp rx=0 tx=0).stop 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 msgr2=0x7fded00ac060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.407+0000 7fdeb5ffb640 1 --2- 172.21.15.1:0/3999720789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fded006ec80 0x7fded00ac060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdec000d800 tx=0x7fdec000dcc0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7fdecffff640 1 -- 172.21.15.1:0/3999720789 reap_dead start 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 shutdown_connections 2024-05-09T06:27:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 >> 172.21.15.1:0/3999720789 conn(0x7fded00cc4b0 msgr2=0x7fded0069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c1406be30 msgr2=0x7f1c1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c1406be30 0x7f1c1406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009b40 tx=0x7f1c24009190 comp rx=0 tx=0).stop 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 msgr2=0x7f1c340c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 --2- 172.21.15.1:0/1579346318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3415ea40 0x7f1c340c4d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00c780 tx=0x7f1c2c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 shutdown_connections 2024-05-09T06:27:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff4e00d16f0 con 0x7ff4c0075670 2024-05-09T06:27:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7fdeb5ffb640 1 -- 172.21.15.1:0/3999720789 wait complete. 2024-05-09T06:27:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c32d9d640 1 -- 172.21.15.1:0/1579346318 reap_dead start 2024-05-09T06:27:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 shutdown_connections 2024-05-09T06:27:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.408+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 >> 172.21.15.1:0/1579346318 conn(0x7f1c340b1d30 msgr2=0x7f1c340b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.409+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 shutdown_connections 2024-05-09T06:27:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.409+0000 7f1c3359e640 1 -- 172.21.15.1:0/1579346318 wait complete. 2024-05-09T06:27:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.409+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0134068300 con 0x7f01180754b0 2024-05-09T06:27:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.409+0000 7ff4d7fff640 1 -- 172.21.15.1:0/692342152 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff4e00d16f0 con 0x7ff4c0075670 2024-05-09T06:27:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7ff4c0075670 msgr2=0x7ff4c0077ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7f012a7fc640 1 -- 172.21.15.1:0/1333445735 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0134068300 con 0x7f01180754b0 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7ff4c0075670 0x7ff4c0077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff4c006bf00 msgr2=0x7ff4c006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff4c006bf00 0x7ff4c006e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00012e0 tx=0x7ff4d0005bf0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 msgr2=0x7ff4e014d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4e605d640 1 --2- 172.21.15.1:0/692342152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4e015e5f0 0x7ff4e014d250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800c430 tx=0x7ff4d800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.410+0000 7ff4dffff640 1 -- 172.21.15.1:0/692342152 reap_dead start 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 shutdown_connections 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 >> 172.21.15.1:0/692342152 conn(0x7ff4e00d3e90 msgr2=0x7ff4e00d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f01180754b0 msgr2=0x7f01180778f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f01180754b0 0x7f01180778f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 shutdown_connections 2024-05-09T06:27:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7ff4e605d640 1 -- 172.21.15.1:0/692342152 wait complete. 2024-05-09T06:27:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f011806be30 msgr2=0x7f011806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f011806be30 0x7f011806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0130037000 tx=0x7f0130008040 comp rx=0 tx=0).stop 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 msgr2=0x7f0134157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 --2- 172.21.15.1:0/1333445735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01340cfe00 0x7f0134157db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f012400c400 tx=0x7f012400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013a601640 1 -- 172.21.15.1:0/1333445735 reap_dead start 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 shutdown_connections 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 >> 172.21.15.1:0/1333445735 conn(0x7f01340d0260 msgr2=0x7f01340ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.411+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 shutdown_connections 2024-05-09T06:27:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.412+0000 7f013c064640 1 -- 172.21.15.1:0/1333445735 wait complete. 2024-05-09T06:27:46.447 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.447 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T06:27:46.448 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T06:27:46.449 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:46.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T06:27:46.450 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T06:27:46.451 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T06:27:46.452 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.452 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T06:27:46.453 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:46.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T06:27:46.454 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:46.454 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T06:27:46.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.683+0000 7f1b0fa2c640 1 Processor -- start 2024-05-09T06:27:46.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.683+0000 7f1b0fa2c640 1 -- start start 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.684+0000 7f1b0fa2c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b080d4940 0x7f1b080b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.684+0000 7f1b0fa2c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 0x7f1b080b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.684+0000 7f1b0fa2c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b080b33c0 con 0x7f1b08161950 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.684+0000 7f1b0fa2c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080d4dd0 con 0x7f1b080d4940 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.684+0000 7f1b0fa2c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080b3660 con 0x7f1b080d2b10 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f1b0d7c8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1b08161950 0x7f1b080d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38478/0 (socket says 172.21.15.1:38478) 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f1b0d7c8640 1 -- 172.21.15.1:0/3527154126 learned_addr learned my addr 172.21.15.1:0/3527154126 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/3527154126 >> v2:172.21.15.1:3300/0 conn(0x7f1b080d4940 0x7f1b080b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f1b0dfc9640 1 --2- 172.21.15.1:0/3527154126 >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 0x7f1b080b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f75cd790640 1 Processor -- start 2024-05-09T06:27:46.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.685+0000 7f75cd790640 1 -- start start 2024-05-09T06:27:46.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f1b0cfc7640 1 -- 172.21.15.1:0/3527154126 >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 msgr2=0x7f1b080b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/3527154126 >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 0x7f1b080b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f1b0cfc7640 1 -- 172.21.15.1:0/3527154126 >> v1:172.21.15.1:6789/0 conn(0x7f1b08161950 legacy=0x7f1b080d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f1b0cfc7640 1 -- 172.21.15.1:0/3527154126 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080b3940 con 0x7f1b080d4940 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f1b0dfc9640 1 --2- 172.21.15.1:0/3527154126 >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 0x7f1b080b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f3eda8bb640 1 Processor -- start 2024-05-09T06:27:46.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7fd20cce0640 1 Processor -- start 2024-05-09T06:27:46.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7fd20cce0640 1 -- start start 2024-05-09T06:27:46.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7fdcff729640 1 Processor -- start 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75cd790640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 0x7f75c81633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75cd790640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c8163900 con 0x7f75c8071f50 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75cd790640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c8070c10 con 0x7f75c806d440 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75cd790640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75c8071ac0 con 0x7f75c815ff70 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75c77fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f75c806d440 0x7f75c80cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35096/0 (socket says 172.21.15.1:35096) 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75c77fe640 1 -- 172.21.15.1:0/1537951697 learned_addr learned my addr 172.21.15.1:0/1537951697 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f75c6ffd640 1 --2- 172.21.15.1:0/1537951697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 0x7f75c81633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f75c5ffb640 1 -- 172.21.15.1:0/1537951697 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308514443 0 0) 0x7f75c8070c10 con 0x7f75c806d440 2024-05-09T06:27:46.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f073e17e640 1 Processor -- start 2024-05-09T06:27:46.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f75c5ffb640 1 -- 172.21.15.1:0/1537951697 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75b0003520 con 0x7f75c806d440 2024-05-09T06:27:46.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f75c6ffd640 1 -- 172.21.15.1:0/1537951697 >> v1:172.21.15.6:6789/0 conn(0x7f75c806d440 legacy=0x7f75c80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f75c6ffd640 1 -- 172.21.15.1:0/1537951697 >> v1:172.21.15.1:6789/0 conn(0x7f75c8071f50 legacy=0x7f75c806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f75c6ffd640 1 -- 172.21.15.1:0/1537951697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c8160350 con 0x7f75c815ff70 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f073e17e640 1 -- start start 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7fdcff729640 1 -- start start 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.686+0000 7f3eda8bb640 1 -- start start 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3eda8bb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3eda8bb640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3ecc038e60 0x7f3ecc03a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3eda8bb640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ecc051b20 con 0x7f3ecc0516c0 2024-05-09T06:27:46.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3eda8bb640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3ecc056a30 con 0x7f3ecc038e60 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3eda8bb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc051e60 con 0x7f3ecc0e1210 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd20cce0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd20815ac80 0x7fd2080aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd20cce0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd20806ca60 0x7fd2080ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd20cce0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20806cfd0 con 0x7fd2080cacd0 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd20cce0640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd2080711c0 con 0x7fd20806ca60 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd20cce0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2080714a0 con 0x7fd20815ac80 2024-05-09T06:27:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3ed98b9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3ed98b9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33138/0 (socket says 172.21.15.1:33138) 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.687+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/3527154126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d4940 0x7f1b080b4140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b0808e540 tx=0x7f1b0002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5a0124fed9275 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fdcff729640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdcf81629e0 0x7fdcf8162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fdcff729640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf8071c70 con 0x7fdcf814be30 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fdcff729640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf8071f50 con 0x7fdcf8163360 2024-05-09T06:27:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fdcff729640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf806f0c0 con 0x7fdcf81629e0 2024-05-09T06:27:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7f3ed98b9640 1 -- 172.21.15.1:0/4099937990 learned_addr learned my addr 172.21.15.1:0/4099937990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fdcfd4c5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdcf814be30 0x7fdcf806cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38502/0 (socket says 172.21.15.1:38502) 2024-05-09T06:27:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f3eda0ba640 1 --2- 172.21.15.1:0/4099937990 >> v2:172.21.15.1:3300/0 conn(0x7f3ecc038e60 0x7f3ecc03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.688+0000 7fd206ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd20806ca60 0x7fd2080ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd20806ca60 0x7fd2080ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53272/0 (socket says 172.21.15.1:53272) 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 -- 172.21.15.1:0/2926502605 learned_addr learned my addr 172.21.15.1:0/2926502605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fdcfd4c5640 1 -- 172.21.15.1:0/2764170216 learned_addr learned my addr 172.21.15.1:0/2764170216 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f073e17e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0730009dd0 0x7f07300e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f073e17e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f07300516c0 0x7f073000a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f1afffff640 1 -- 172.21.15.1:0/3527154126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007740 con 0x7f1b080d4940 2024-05-09T06:27:46.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f1afffff640 1 -- 172.21.15.1:0/3527154126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b000078e0 con 0x7f1b080d4940 2024-05-09T06:27:46.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f75c6ffd640 1 --2- 172.21.15.1:0/1537951697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 0x7f75c81633c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097d0 tx=0x7f75bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550ab2b78e8ced5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fdcfccc4640 1 --2- 172.21.15.1:0/2764170216 >> v2:172.21.15.6:3300/0 conn(0x7fdcf81629e0 0x7fdcf8162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 -- 172.21.15.1:0/2926502605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd20815ac80 msgr2=0x7fd2080aec30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 --2- 172.21.15.1:0/2926502605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd20815ac80 0x7fd2080aec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 -- 172.21.15.1:0/2926502605 >> v1:172.21.15.6:6789/0 conn(0x7fd2080cacd0 legacy=0x7fd208071ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7fd206ffd640 1 -- 172.21.15.1:0/2926502605 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd208071780 con 0x7fd20806ca60 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f073e17e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073000a6f0 con 0x7f07300e4d20 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f07300516c0 0x7f073000a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f07300516c0 0x7f073000a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33548/0 (socket says 172.21.15.1:33548) 2024-05-09T06:27:46.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 -- 172.21.15.1:0/3677398445 learned_addr learned my addr 172.21.15.1:0/3677398445 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7fdcfccc4640 1 -- 172.21.15.1:0/2764170216 >> v1:172.21.15.6:6789/0 conn(0x7fdcf8163360 legacy=0x7fdcf80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7fdcfccc4640 1 -- 172.21.15.1:0/2764170216 >> v1:172.21.15.1:6789/0 conn(0x7fdcf814be30 legacy=0x7fdcf806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7fdcfccc4640 1 -- 172.21.15.1:0/2764170216 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf8072230 con 0x7fdcf81629e0 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0730051eb0 con 0x7f0730009dd0 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0730052190 con 0x7f07300516c0 2024-05-09T06:27:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1afffff640 1 -- 172.21.15.1:0/3527154126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007cb0 con 0x7f1b080d4940 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f07367fc640 1 -- 172.21.15.1:0/3677398445 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358239669 0 0) 0x7f073000a6f0 con 0x7f07300e4d20 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f07367fc640 1 -- 172.21.15.1:0/3677398445 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0728003520 con 0x7f07300e4d20 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f75c5ffb640 1 -- 172.21.15.1:0/1537951697 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc037070 con 0x7f75c815ff70 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 -- 172.21.15.1:0/3677398445 >> v2:172.21.15.1:3300/0 conn(0x7f0730009dd0 msgr2=0x7f07300e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 --2- 172.21.15.1:0/3677398445 >> v2:172.21.15.1:3300/0 conn(0x7f0730009dd0 0x7f07300e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f0736ffd640 1 -- 172.21.15.1:0/3677398445 >> v1:172.21.15.6:6789/0 conn(0x7f07300e4d20 legacy=0x7f07300e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/3527154126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d4940 msgr2=0x7f1b080b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 --2- 172.21.15.1:0/3527154126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d4940 0x7f1b080b4140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b0808e540 tx=0x7f1b0002a570 comp rx=0 tx=0).stop 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/3527154126 shutdown_connections 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 --2- 172.21.15.1:0/3527154126 >> v2:172.21.15.6:3300/0 conn(0x7f1b080d2b10 0x7f1b080b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 --2- 172.21.15.1:0/3527154126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d4940 0x7f1b080b4140 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/3527154126 >> 172.21.15.1:0/3527154126 conn(0x7f1b080d3e20 msgr2=0x7f1b080d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.690+0000 7f75c5ffb640 1 -- 172.21.15.1:0/1537951697 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc032440 con 0x7f75c815ff70 2024-05-09T06:27:46.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f43f70ca640 1 Processor -- start 2024-05-09T06:27:46.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f0736ffd640 1 -- 172.21.15.1:0/3677398445 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07300e2cc0 con 0x7f07300516c0 2024-05-09T06:27:46.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75c5ffb640 1 -- 172.21.15.1:0/1537951697 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc032810 con 0x7f75c815ff70 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f3ed98b9640 1 -- 172.21.15.1:0/4099937990 >> v2:172.21.15.1:3300/0 conn(0x7f3ecc038e60 msgr2=0x7f3ecc03a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f3ed98b9640 1 --2- 172.21.15.1:0/4099937990 >> v2:172.21.15.1:3300/0 conn(0x7f3ecc038e60 0x7f3ecc03a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f3ed98b9640 1 -- 172.21.15.1:0/4099937990 >> v1:172.21.15.1:6789/0 conn(0x7f3ecc0516c0 legacy=0x7f3ecc038750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.689+0000 7f3ed98b9640 1 -- 172.21.15.1:0/4099937990 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc052140 con 0x7f3ecc0e1210 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f3ed98b9640 1 --2- 172.21.15.1:0/4099937990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009870 tx=0x7f3ec402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae7369817c9429d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7fd206ffd640 1 --2- 172.21.15.1:0/2926502605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20806ca60 0x7fd2080ca790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bb10 tx=0x7fd1fc02eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70007d1aa02a744a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f6c54729640 1 Processor -- start 2024-05-09T06:27:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7fdcfccc4640 1 --2- 172.21.15.1:0/2764170216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf81629e0 0x7fdcf8162e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdce80097a0 tx=0x7fdce802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b3ae24380f6927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f43f70ca640 1 -- start start 2024-05-09T06:27:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f3ed88b7640 1 -- 172.21.15.1:0/4099937990 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4034750 con 0x7f3ecc0e1210 2024-05-09T06:27:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f3ed88b7640 1 -- 172.21.15.1:0/4099937990 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec4039070 con 0x7f3ecc0e1210 2024-05-09T06:27:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7fdceffff640 1 -- 172.21.15.1:0/2764170216 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8037070 con 0x7fdcf81629e0 2024-05-09T06:27:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7fdceffff640 1 -- 172.21.15.1:0/2764170216 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce802acd0 con 0x7fdcf81629e0 2024-05-09T06:27:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7fdceffff640 1 -- 172.21.15.1:0/2764170216 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8032660 con 0x7fdcf81629e0 2024-05-09T06:27:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f6c54729640 1 -- start start 2024-05-09T06:27:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f3ed88b7640 1 -- 172.21.15.1:0/4099937990 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4034af0 con 0x7f3ecc0e1210 2024-05-09T06:27:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75cd790640 1 -- 172.21.15.1:0/1537951697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 msgr2=0x7f75c81633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75cd790640 1 --2- 172.21.15.1:0/1537951697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 0x7f75c81633c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097d0 tx=0x7f75bc02a210 comp rx=0 tx=0).stop 2024-05-09T06:27:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/3527154126 shutdown_connections 2024-05-09T06:27:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/3527154126 wait complete. 2024-05-09T06:27:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75cd790640 1 -- 172.21.15.1:0/1537951697 shutdown_connections 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75cd790640 1 --2- 172.21.15.1:0/1537951697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c815ff70 0x7f75c81633c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.691+0000 7f75cd790640 1 -- 172.21.15.1:0/1537951697 >> 172.21.15.1:0/1537951697 conn(0x7f75c80cc4b0 msgr2=0x7f75c80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f75cd790640 1 -- 172.21.15.1:0/1537951697 shutdown_connections 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f0736ffd640 1 --2- 172.21.15.1:0/3677398445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07300516c0 0x7f073000a1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0730004370 tx=0x7f072402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7600ad3a75bf0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f75cd790640 1 -- 172.21.15.1:0/1537951697 wait complete. 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 -- 172.21.15.1:0/4099937990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 msgr2=0x7f3ecc0563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/4099937990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009870 tx=0x7f3ec402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7fd2067fc640 1 -- 172.21.15.1:0/2926502605 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc032860 con 0x7fd20806ca60 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 -- 172.21.15.1:0/4099937990 shutdown_connections 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/4099937990 >> v2:172.21.15.1:3300/0 conn(0x7f3ecc038e60 0x7f3ecc03a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/4099937990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0e1210 0x7f3ecc0563a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 -- 172.21.15.1:0/4099937990 >> 172.21.15.1:0/4099937990 conn(0x7f3ecc053750 msgr2=0x7f3ecc053b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f07367fc640 1 -- 172.21.15.1:0/3677398445 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724037070 con 0x7f07300516c0 2024-05-09T06:27:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f07367fc640 1 -- 172.21.15.1:0/3677398445 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0724032440 con 0x7f07300516c0 2024-05-09T06:27:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f07367fc640 1 -- 172.21.15.1:0/3677398445 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240327f0 con 0x7f07300516c0 2024-05-09T06:27:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7fd2067fc640 1 -- 172.21.15.1:0/2926502605 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc032a00 con 0x7fd20806ca60 2024-05-09T06:27:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 -- 172.21.15.1:0/4099937990 shutdown_connections 2024-05-09T06:27:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7f3eda8bb640 1 -- 172.21.15.1:0/4099937990 wait complete. 2024-05-09T06:27:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.692+0000 7fd2067fc640 1 -- 172.21.15.1:0/2926502605 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc032dd0 con 0x7fd20806ca60 2024-05-09T06:27:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c54729640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c4c15c8e0 0x7f6c4c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fd20cce0640 1 -- 172.21.15.1:0/2926502605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20806ca60 msgr2=0x7fd2080ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fd20cce0640 1 --2- 172.21.15.1:0/2926502605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20806ca60 0x7fd2080ca790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bb10 tx=0x7fd1fc02eae0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f1b0fa2c640 1 Processor -- start 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f1b0fa2c640 1 -- start start 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c54729640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c4c0b2f80 0x7f6c4c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c54729640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c4c15ccc0 con 0x7f6c4c06e9f0 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c54729640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4c1632f0 con 0x7f6c4c0b2f80 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c54729640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4c1635d0 con 0x7f6c4c15c8e0 2024-05-09T06:27:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07300516c0 msgr2=0x7f073000a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 --2- 172.21.15.1:0/3677398445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07300516c0 0x7f073000a1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0730004370 tx=0x7f072402a290 comp rx=0 tx=0).stop 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 shutdown_connections 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 --2- 172.21.15.1:0/3677398445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07300516c0 0x7f073000a1b0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0730004370 tx=0x7f072402a290 comp rx=0 tx=0).stop 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 --2- 172.21.15.1:0/3677398445 >> v2:172.21.15.1:3300/0 conn(0x7f0730009dd0 0x7f07300e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 >> 172.21.15.1:0/3677398445 conn(0x7f07300537e0 msgr2=0x7f0730053bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 shutdown_connections 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f073e17e640 1 -- 172.21.15.1:0/3677398445 wait complete. 2024-05-09T06:27:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fd20cce0640 1 -- 172.21.15.1:0/2926502605 shutdown_connections 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fd20cce0640 1 --2- 172.21.15.1:0/2926502605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20806ca60 0x7fd2080ca790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fdcff729640 1 -- 172.21.15.1:0/2764170216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf81629e0 msgr2=0x7fdcf8162e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fdcff729640 1 --2- 172.21.15.1:0/2764170216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf81629e0 0x7fdcf8162e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdce80097a0 tx=0x7fdce802a040 comp rx=0 tx=0).stop 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fdcff729640 1 -- 172.21.15.1:0/2764170216 shutdown_connections 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fdcff729640 1 --2- 172.21.15.1:0/2764170216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf81629e0 0x7fdcf8162e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7fdcff729640 1 -- 172.21.15.1:0/2764170216 >> 172.21.15.1:0/2764170216 conn(0x7fdcf80cc4b0 msgr2=0x7fdcf80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c524c5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c4c15c8e0 0x7f6c4c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c51cc4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6c4c06e9f0 0x7f6c4c1628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38508/0 (socket says 172.21.15.1:38508) 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7fd20cce0640 1 --2- 172.21.15.1:0/2926502605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd20815ac80 0x7fd2080aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7fd20cce0640 1 -- 172.21.15.1:0/2926502605 >> 172.21.15.1:0/2926502605 conn(0x7fd2080b1530 msgr2=0x7fd2080b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f43f70ca640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f43f0162670 0x7f43f00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f43f70ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43f00c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f43f70ca640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f0162a50 con 0x7f43f00c51c0 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f43f70ca640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f43f00b4740 con 0x7f43f0162670 2024-05-09T06:27:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f43f70ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f43f00c5630 con 0x7f43f00b42e0 2024-05-09T06:27:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43f5667640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f43f00c51c0 0x7f43f0166130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35142/0 (socket says 172.21.15.1:35142) 2024-05-09T06:27:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43f5667640 1 -- 172.21.15.1:0/2600388947 learned_addr learned my addr 172.21.15.1:0/2600388947 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f75cd790640 1 Processor -- start 2024-05-09T06:27:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7fd20cce0640 1 -- 172.21.15.1:0/2926502605 shutdown_connections 2024-05-09T06:27:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7fd20cce0640 1 -- 172.21.15.1:0/2926502605 wait complete. 2024-05-09T06:27:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f75cd790640 1 -- start start 2024-05-09T06:27:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.693+0000 7f6c51cc4640 1 -- 172.21.15.1:0/1490478701 learned_addr learned my addr 172.21.15.1:0/1490478701 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f3eda8bb640 1 Processor -- start 2024-05-09T06:27:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949058572 0 0) 0x7f43e8001490 con 0x7f43f00c51c0 2024-05-09T06:27:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d8003520 con 0x7f43f00c51c0 2024-05-09T06:27:46.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43effff640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43f00c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43f4e66640 1 --2- 172.21.15.1:0/2600388947 >> v2:172.21.15.1:3300/0 conn(0x7f43f0162670 0x7f43f00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f1b0fa2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f3eda8bb640 1 -- start start 2024-05-09T06:27:46.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f6c52cc6640 1 --2- 172.21.15.1:0/1490478701 >> v2:172.21.15.1:3300/0 conn(0x7f6c4c0b2f80 0x7f6c4c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550989690 0 0) 0x7f43d8003520 con 0x7f43f00c51c0 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 msgr2=0x7f43f00c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43f00c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 >> v2:172.21.15.1:3300/0 conn(0x7f43f0162670 msgr2=0x7f43f00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 --2- 172.21.15.1:0/2600388947 >> v2:172.21.15.1:3300/0 conn(0x7f43f0162670 0x7f43f00cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.694+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f00b1440 con 0x7f43f00c51c0 2024-05-09T06:27:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0d7c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f43e8001bc0 con 0x7f43f00c51c0 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 >> v1:172.21.15.6:6789/0 conn(0x7f43f00c51c0 legacy=0x7f43f0166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0d7c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53326/0 (socket says 172.21.15.1:53326) 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0d7c8640 1 -- 172.21.15.1:0/1084232891 learned_addr learned my addr 172.21.15.1:0/1084232891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0fa2c640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b08161950 0x7f1b080b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0fa2c640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b080b8fa0 0x7f1b080b9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d80039b0 0x7f43d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43d8004d40 0x7f43d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080d1100 con 0x7f1b080d2b10 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080d0f80 con 0x7f1b080b8fa0 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1b080d1c00 con 0x7f1b08161950 2024-05-09T06:27:46.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c514c3640 1 -- 172.21.15.1:0/1490478701 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285621806 0 0) 0x7f6c4c15ccc0 con 0x7f6c4c06e9f0 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c514c3640 1 -- 172.21.15.1:0/1490478701 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c38003520 con 0x7f6c4c06e9f0 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f75cd790640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43f00c5630 con 0x7f43d80039b0 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800ba70 con 0x7f43d8004d40 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f43d800bc20 con 0x7f43f00b42e0 2024-05-09T06:27:46.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43f5667640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43d8004d40 0x7f43d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43effff640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f75cd790640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 0x7f75c814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f3eda8bb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 0x7f3ecc0e1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f75c6ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f75c6ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33190/0 (socket says 172.21.15.1:33190) 2024-05-09T06:27:46.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f3eda8bb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 0x7f3ecc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f073e17e640 1 Processor -- start 2024-05-09T06:27:46.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f3eda8bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 0x7f3ecc0dc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda8bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc0e15f0 con 0x7f3ecc0e1210 2024-05-09T06:27:46.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda8bb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc03a150 con 0x7f3ecc038e60 2024-05-09T06:27:46.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0dfc9640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b080b8fa0 0x7f1b080b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b08161950 0x7f1b080b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda8bb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc0e1770 con 0x7f3ecc0516c0 2024-05-09T06:27:46.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3ed98b9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 0x7f3ecc0e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f43f4e66640 1 --2- 172.21.15.1:0/2600388947 >> v2:172.21.15.1:3300/0 conn(0x7f43f0162670 0x7f43f00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:46.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3ed98b9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 0x7f3ecc0e1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33586/0 (socket says 172.21.15.1:33586) 2024-05-09T06:27:46.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3ed98b9640 1 -- 172.21.15.1:0/3392008599 learned_addr learned my addr 172.21.15.1:0/3392008599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 0x7f3ecc0dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f073e17e640 1 -- start start 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 -- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b08161950 msgr2=0x7f1b080b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b08161950 0x7f1b080b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 -- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b080b8fa0 msgr2=0x7f1b080b9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b080b8fa0 0x7f1b080b9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080c91a0 con 0x7f1b080d2b10 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c524c5640 1 -- 172.21.15.1:0/1490478701 >> v2:172.21.15.1:3300/0 conn(0x7f6c4c0b2f80 msgr2=0x7f6c4c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c524c5640 1 --2- 172.21.15.1:0/1490478701 >> v2:172.21.15.1:3300/0 conn(0x7f6c4c0b2f80 0x7f6c4c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c524c5640 1 -- 172.21.15.1:0/1490478701 >> v1:172.21.15.1:6789/0 conn(0x7f6c4c06e9f0 legacy=0x7f6c4c1628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.695+0000 7f6c524c5640 1 -- 172.21.15.1:0/1490478701 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c0713f0 con 0x7f6c4c15c8e0 2024-05-09T06:27:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7fd20cce0640 1 Processor -- start 2024-05-09T06:27:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3ed90b8640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 0x7f3ecc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 -- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 msgr2=0x7f3ecc03a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7fdcff729640 1 -- 172.21.15.1:0/2764170216 shutdown_connections 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7fd20cce0640 1 -- start start 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f1b0d7c8640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c5a0 tx=0x7f1af800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75cd790640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75c815ff70 0x7f75c8160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75cd790640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8071470 con 0x7f75c8071f50 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75cd790640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8071190 con 0x7f75c815ff70 2024-05-09T06:27:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75cd790640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75c8071750 con 0x7f75c806d440 2024-05-09T06:27:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75c67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 0x7f75c814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75c67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 0x7f75c814a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53332/0 (socket says 172.21.15.1:53332) 2024-05-09T06:27:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f75c67fc640 1 -- 172.21.15.1:0/3766807674 learned_addr learned my addr 172.21.15.1:0/3766807674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f6c524c5640 1 --2- 172.21.15.1:0/1490478701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15c8e0 0x7f6c4c06e4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c06c5f0 tx=0x7f6c4802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2aafa9303a85b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 0x7f3ecc03a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 -- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 msgr2=0x7f3ecc0e1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 0x7f3ecc0e1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f3eda0ba640 1 -- 172.21.15.1:0/3392008599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc17d510 con 0x7f3ecc0e1210 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eda0ba640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 0x7f3ecc0dc360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406ab70 tx=0x7f3ed4068d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3ed90b8640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 0x7f3ecc03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f75c6ffd640 1 -- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75c815ff70 msgr2=0x7f75c8160e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f75c6ffd640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75c815ff70 0x7f75c8160e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f75c6ffd640 1 -- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 msgr2=0x7f75c814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f75c6ffd640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 0x7f75c814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f75c6ffd640 1 -- 172.21.15.1:0/3766807674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c80a9600 con 0x7f75c806d440 2024-05-09T06:27:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8018070 con 0x7f1b080d2b10 2024-05-09T06:27:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4069530 con 0x7f3ecc0e1210 2024-05-09T06:27:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4069710 con 0x7f3ecc0e1210 2024-05-09T06:27:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4073300 con 0x7f3ecc0e1210 2024-05-09T06:27:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f43f5667640 1 -- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 msgr2=0x7f43d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f43f5667640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f00b42e0 0x7f43d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f43f5667640 1 -- 172.21.15.1:0/2600388947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d80039b0 msgr2=0x7f43d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc17d7b0 con 0x7f3ecc0e1210 2024-05-09T06:27:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f073e17e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc17dd70 con 0x7f3ecc0e1210 2024-05-09T06:27:46.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f43f5667640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d80039b0 0x7f43d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.696+0000 7f43f5667640 1 -- 172.21.15.1:0/2600388947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8003520 con 0x7f43d8004d40 2024-05-09T06:27:46.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f43f5667640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43d8004d40 0x7f43d80051c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43e800a9b0 tx=0x7f43d8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a1755397b8fbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f75c6ffd640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75bc031000 tx=0x7f75bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f073e17e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07300e4d20 0x7f07300502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f6c514c3640 1 -- 172.21.15.1:0/1490478701 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48007740 con 0x7f6c4c15c8e0 2024-05-09T06:27:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f6c514c3640 1 -- 172.21.15.1:0/1490478701 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c480078e0 con 0x7f6c4c15c8e0 2024-05-09T06:27:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e800a5d0 con 0x7f43d8004d40 2024-05-09T06:27:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e802f070 con 0x7f43d8004d40 2024-05-09T06:27:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7fd20cce0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 0x7fd208151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f07377fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7fd20cce0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f07377fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33602/0 (socket says 172.21.15.1:33602) 2024-05-09T06:27:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f07377fe640 1 -- 172.21.15.1:0/591391160 learned_addr learned my addr 172.21.15.1:0/591391160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7fd20cce0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20815ac80 0x7fd2080b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f073e17e640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07300c8140 0x7f07300d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7fdcff729640 1 -- 172.21.15.1:0/2764170216 wait complete. 2024-05-09T06:27:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7fdcff729640 1 Processor -- start 2024-05-09T06:27:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7f0736ffd640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07300e4d20 0x7f07300502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.698+0000 7fd206ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f07300e44d0 con 0x7f07300c8140 2024-05-09T06:27:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7fdcff729640 1 -- start start 2024-05-09T06:27:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f07300e1550 con 0x7f0730009dd0 2024-05-09T06:27:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f07300e4650 con 0x7f07300e4d20 2024-05-09T06:27:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7fd206ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33214/0 (socket says 172.21.15.1:33214) 2024-05-09T06:27:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 -- 172.21.15.1:0/1490478701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15c8e0 msgr2=0x7f6c4c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 --2- 172.21.15.1:0/1490478701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15c8e0 0x7f6c4c06e4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c06c5f0 tx=0x7f6c4802a680 comp rx=0 tx=0).stop 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7fd206ffd640 1 -- 172.21.15.1:0/1430103642 learned_addr learned my addr 172.21.15.1:0/1430103642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43ef7fe640 1 -- 172.21.15.1:0/2600388947 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80070f0 con 0x7f43d8004d40 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af80074e0 con 0x7f1b080d2b10 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8007690 con 0x7f1b080d2b10 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b080cb020 con 0x7f1b080d2b10 2024-05-09T06:27:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.697+0000 7f1b0fa2c640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b080cb5e0 con 0x7f1b080d2b10 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 -- 172.21.15.1:0/1490478701 shutdown_connections 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 --2- 172.21.15.1:0/1490478701 >> v2:172.21.15.1:3300/0 conn(0x7f6c4c0b2f80 0x7f6c4c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 --2- 172.21.15.1:0/1490478701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15c8e0 0x7f6c4c06e4b0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c06c5f0 tx=0x7f6c4802a680 comp rx=0 tx=0).stop 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f6c54729640 1 -- 172.21.15.1:0/1490478701 >> 172.21.15.1:0/1490478701 conn(0x7f6c4c0cfed0 msgr2=0x7f6c4c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43f70ca640 1 -- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43d8004d40 msgr2=0x7f43d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43f70ca640 1 --2- 172.21.15.1:0/2600388947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43d8004d40 0x7f43d80051c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43e800a9b0 tx=0x7f43d8005830 comp rx=0 tx=0).stop 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43f5667640 1 -- 172.21.15.1:0/2600388947 reap_dead start 2024-05-09T06:27:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd208070fa0 con 0x7fd20815ac80 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc042020 con 0x7f75c806d440 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 -- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07300e4d20 msgr2=0x7f07300502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07300e4d20 0x7f07300502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 -- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07300c8140 msgr2=0x7f07300d8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07300c8140 0x7f07300d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07300d9330 con 0x7f0730009dd0 2024-05-09T06:27:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43f70ca640 1 -- 172.21.15.1:0/2600388947 shutdown_connections 2024-05-09T06:27:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f43f70ca640 1 -- 172.21.15.1:0/2600388947 >> 172.21.15.1:0/2600388947 conn(0x7f43f00b37c0 msgr2=0x7f43f00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd208070e20 con 0x7fd20806ca60 2024-05-09T06:27:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c80a98c0 con 0x7f75c806d440 2024-05-09T06:27:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.699+0000 7f07377fe640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b4d0 tx=0x7f072c00b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2080723b0 con 0x7fd2080cacd0 2024-05-09T06:27:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c80a9ba0 con 0x7f75c806d440 2024-05-09T06:27:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc005710 con 0x7f3ecc0e1210 2024-05-09T06:27:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc03d750 con 0x7f75c806d440 2024-05-09T06:27:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc037040 con 0x7f75c806d440 2024-05-09T06:27:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c014070 con 0x7f0730009dd0 2024-05-09T06:27:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c010870 con 0x7f0730009dd0 2024-05-09T06:27:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 0x7fdcf82016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07300dbb70 con 0x7f0730009dd0 2024-05-09T06:27:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd2077fe640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 0x7fd208151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 0x7fdcf81f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd206ffd640 1 -- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 msgr2=0x7fd208151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae8005290 con 0x7f1b080d2b10 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf806e7e0 con 0x7fdcf81629e0 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf814c210 con 0x7fdcf8163360 2024-05-09T06:27:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fdcff729640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf806e960 con 0x7fdcf814be30 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073004f800 con 0x7f0730009dd0 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c010c60 con 0x7f0730009dd0 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ed40734a0 con 0x7f3ecc0e1210 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33628/0 (socket says 172.21.15.1:33628) 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 -- 172.21.15.1:0/2949262425 learned_addr learned my addr 172.21.15.1:0/2949262425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfccc4640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 0x7fdcf81f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd206ffd640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 0x7fd208151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd206ffd640 1 -- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20815ac80 msgr2=0x7fd2080b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd206ffd640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20815ac80 0x7fd2080b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7fd206ffd640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080b2630 con 0x7fd2080cacd0 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fd2077fe640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 0x7fd208151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.700+0000 7f6c54729640 1 -- 172.21.15.1:0/1490478701 shutdown_connections 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7f3eca7fc640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 0x7f3eb806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3ed41065e0 con 0x7f3ecc0e1210 2024-05-09T06:27:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 -- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 msgr2=0x7fdcf82016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fd206ffd640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b2c0 tx=0x7fd1fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfd4c5640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 0x7fdcf82016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 0x7fdcf82016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 -- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 msgr2=0x7fdcf81f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 0x7fdcf81f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7fdcfdcc6640 1 -- 172.21.15.1:0/2949262425 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf8201bf0 con 0x7fdcf8163360 2024-05-09T06:27:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7f3ed98b9640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 0x7f3eb806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.701+0000 7f6c54729640 1 -- 172.21.15.1:0/1490478701 wait complete. 2024-05-09T06:27:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcfccc4640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 0x7fdcf81f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7f43f70ca640 1 -- 172.21.15.1:0/2600388947 shutdown_connections 2024-05-09T06:27:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcfd4c5640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 0x7fdcf82016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcfdcc6640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c580 tx=0x7fdcf400ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc049070 con 0x7fd2080cacd0 2024-05-09T06:27:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7f43f70ca640 1 -- 172.21.15.1:0/2600388947 wait complete. 2024-05-09T06:27:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc030800 con 0x7fd2080cacd0 2024-05-09T06:27:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc030bf0 con 0x7fd2080cacd0 2024-05-09T06:27:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7f3ed98b9640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 0x7f3eb806e3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009870 tx=0x7f3ec402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2080b28d0 con 0x7fd2080cacd0 2024-05-09T06:27:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f072c017070 con 0x7f0730009dd0 2024-05-09T06:27:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4017070 con 0x7fdcf8163360 2024-05-09T06:27:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf40035f0 con 0x7fdcf8163360 2024-05-09T06:27:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.702+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40039e0 con 0x7fdcf8163360 2024-05-09T06:27:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20815b060 con 0x7fd2080cacd0 2024-05-09T06:27:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0718005290 con 0x7f0730009dd0 2024-05-09T06:27:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8005290 con 0x7f75c806d440 2024-05-09T06:27:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7f0723fff640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 0x7f071c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7f6c54729640 1 Processor -- start 2024-05-09T06:27:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.704+0000 7f6c54729640 1 -- start start 2024-05-09T06:27:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.704+0000 7f0736ffd640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 0x7f071c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf80b3c00 con 0x7fdcf8163360 2024-05-09T06:27:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.703+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf80b41c0 con 0x7fdcf8163360 2024-05-09T06:27:46.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.704+0000 7f0736ffd640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 0x7f071c06e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f072402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f072c08ada0 con 0x7f0730009dd0 2024-05-09T06:27:46.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75bc045020 con 0x7f75c806d440 2024-05-09T06:27:46.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf80685e0 con 0x7fdcf8163360 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c4c06e9f0 0x7f6c4c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c4c0b2f80 0x7f6c4c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c0b5250 con 0x7f6c4c0b2f80 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c162fb0 con 0x7f6c4c15a220 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c54729640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4c0b53d0 con 0x7f6c4c06e9f0 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.704+0000 7f43f70ca640 1 Processor -- start 2024-05-09T06:27:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.704+0000 7f43f70ca640 1 -- start start 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1fc04c070 con 0x7fd2080cacd0 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 0x7f43f01fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c52cc6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c52cc6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33640/0 (socket says 172.21.15.1:33640) 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c51cc4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c4c0b2f80 0x7f6c4c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c52cc6640 1 -- 172.21.15.1:0/3779270071 learned_addr learned my addr 172.21.15.1:0/3779270071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 0x7f43f01f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f6c524c5640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c4c06e9f0 0x7f6c4c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f4e66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 0x7f43f01fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f4e66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 0x7f43f01fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53382/0 (socket says 172.21.15.1:53382) 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f75af7fe640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 0x7f75b406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7fd1effff640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 0x7fd1f406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43f015db50 0x7f43f00d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43f006f580 con 0x7f43d8004d40 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43f006f2a0 con 0x7f43f015db50 2024-05-09T06:27:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7f43f70ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f43f00aeaa0 con 0x7f43f01606e0 2024-05-09T06:27:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43f4e66640 1 -- 172.21.15.1:0/1709951786 learned_addr learned my addr 172.21.15.1:0/1709951786 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 -- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c4c06e9f0 msgr2=0x7f6c4c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43f5667640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43f015db50 0x7f43f00d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 0x7f43f01f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c4c06e9f0 0x7f6c4c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 -- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c4c0b2f80 msgr2=0x7f6c4c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd1fc0b0a10 con 0x7fd2080cacd0 2024-05-09T06:27:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7fd2077fe640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 0x7fd1f406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.705+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20804f660 con 0x7fd2080cacd0 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c4c0b2f80 0x7f6c4c15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 -- 172.21.15.1:0/3779270071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c166ce0 con 0x7f6c4c15a220 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 -- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43f015db50 msgr2=0x7f43f00d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43f015db50 0x7f43f00d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 -- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 msgr2=0x7f43f01fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 0x7f43f01fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 -- 172.21.15.1:0/1709951786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f0160b60 con 0x7f43f01606e0 2024-05-09T06:27:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75bc0abaf0 con 0x7f75c806d440 2024-05-09T06:27:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f6c52cc6640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d600 tx=0x7f6c4400dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f43effff640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 0x7f43f01f9790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43e4011960 tx=0x7f43e4011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.706+0000 7f75c67fc640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 0x7f75b406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.707+0000 7f75c67fc640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 0x7f75b406e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75c81fb5b0 tx=0x7f75b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.707+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcf401a070 con 0x7fdcf8163360 2024-05-09T06:27:46.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.707+0000 7fd2077fe640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 0x7fd1f406e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800a810 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.707+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ed40d4c30 con 0x7f3ecc0e1210 2024-05-09T06:27:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.708+0000 7fdcedffb640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 0x7fdcdc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.708+0000 7fdcfd4c5640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 0x7fdcdc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.708+0000 7fdcfd4c5640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 0x7fdcdc06e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0001e30 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1af801b070 con 0x7f1b080d2b10 2024-05-09T06:27:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdcf40566e0 con 0x7fdcf8163360 2024-05-09T06:27:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44018020 con 0x7f6c4c15a220 2024-05-09T06:27:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c15e5b0 con 0x7f6c4c15a220 2024-05-09T06:27:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f1afdffb640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 0x7f1aec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f072c058ea0 con 0x7f0730009dd0 2024-05-09T06:27:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.709+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c15eb70 con 0x7f6c4c15a220 2024-05-09T06:27:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.710+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 0x7f1aec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.710+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1af808a8c0 con 0x7f1b080d2b10 2024-05-09T06:27:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.710+0000 7f1b0cfc7640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 0x7f1aec06e370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b00034000 tx=0x7f1b0002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.710+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4c0752b0 con 0x7f6c4c15a220 2024-05-09T06:27:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.711+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75bc04c030 con 0x7f75c806d440 2024-05-09T06:27:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.714+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c44014620 con 0x7f6c4c15a220 2024-05-09T06:27:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.714+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c440149d0 con 0x7f6c4c15a220 2024-05-09T06:27:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcf80685e0 con 0x7fdcf8163360 2024-05-09T06:27:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1fc036030 con 0x7fd2080cacd0 2024-05-09T06:27:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.715+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e400fa30 con 0x7f43f01606e0 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01fbf70 con 0x7f43f01606e0 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01fc530 con 0x7f43f01606e0 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c4401b020 con 0x7f6c4c15a220 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f6c42ffd640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 0x7f6c3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e400fbd0 con 0x7f43f01606e0 2024-05-09T06:27:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.716+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4014440 con 0x7f43f01606e0 2024-05-09T06:27:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.717+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c44057770 con 0x7f6c4c15a220 2024-05-09T06:27:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.718+0000 7f6c524c5640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 0x7f6c3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.718+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f00752b0 con 0x7f43f01606e0 2024-05-09T06:27:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.719+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1af8022020 con 0x7f1b080d2b10 2024-05-09T06:27:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.718+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43e4012040 con 0x7f43f01606e0 2024-05-09T06:27:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.719+0000 7f6c524c5640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 0x7f6c3006e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c48034000 tx=0x7f6c4802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.719+0000 7f43ed7fa640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 0x7f43dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.719+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c44022030 con 0x7f6c4c15a220 2024-05-09T06:27:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.719+0000 7f43f4e66640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 0x7f43dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.720+0000 7f43f4e66640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 0x7f43dc06e340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006d40 tx=0x7f43e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.720+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f43e408b570 con 0x7f43f01606e0 2024-05-09T06:27:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:46.724+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f43e4019030 con 0x7f43f01606e0 2024-05-09T06:27:47.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.136+0000 7f073e17e640 1 -- 172.21.15.1:0/591391160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0718005850 con 0x7f0730009dd0 2024-05-09T06:27:47.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.137+0000 7f0723fff640 1 -- 172.21.15.1:0/591391160 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f072c05d1a0 con 0x7f0730009dd0 2024-05-09T06:27:47.120 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.138+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3ecc0df3d0 con 0x7f3ecc0e1210 2024-05-09T06:27:47.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.139+0000 7f3eca7fc640 1 -- 172.21.15.1:0/3392008599 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ed40d4530 con 0x7f3ecc0e1210 2024-05-09T06:27:47.121 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.140+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f75a8005580 con 0x7f75c806d440 2024-05-09T06:27:47.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 msgr2=0x7f071c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 0x7f071c06e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f072402e000 comp rx=0 tx=0).stop 2024-05-09T06:27:47.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f75af7fe640 1 -- 172.21.15.1:0/3766807674 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75bc07a210 con 0x7f75c806d440 2024-05-09T06:27:47.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 msgr2=0x7f0730176dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b4d0 tx=0x7f072c00b990 comp rx=0 tx=0).stop 2024-05-09T06:27:47.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.141+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 shutdown_connections 2024-05-09T06:27:47.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f071c06be80 0x7f071c06e340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.124 INFO:teuthology.orchestra.run.smithi001.stdout:55834574856 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07300c8140 0x7f07300d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07300e4d20 0x7f07300502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 --2- 172.21.15.1:0/591391160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0730009dd0 0x7f0730176dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 >> 172.21.15.1:0/591391160 conn(0x7f07300537e0 msgr2=0x7f07300548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 shutdown_connections 2024-05-09T06:27:47.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f0721ffb640 1 -- 172.21.15.1:0/591391160 wait complete. 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 msgr2=0x7f3eb806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.142+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 0x7f3eb806e3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009870 tx=0x7f3ec402e000 comp rx=0 tx=0).stop 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 msgr2=0x7f3ecc0dc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 0x7f3ecc0dc360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406ab70 tx=0x7f3ed4068d30 comp rx=0 tx=0).stop 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 shutdown_connections 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3eb806bf00 0x7f3eb806e3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0e1210 0x7f3ecc0dc360 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ecc0516c0 0x7f3ecc03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 --2- 172.21.15.1:0/3392008599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ecc038e60 0x7f3ecc0e1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.143+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 >> 172.21.15.1:0/3392008599 conn(0x7f3ecc053750 msgr2=0x7f3ecc009140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.144+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 shutdown_connections 2024-05-09T06:27:47.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.144+0000 7f3eda8bb640 1 -- 172.21.15.1:0/3392008599 wait complete. 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 msgr2=0x7f75b406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 0x7f75b406e3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75c81fb5b0 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 msgr2=0x7f75c81fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75bc031000 tx=0x7f75bc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 shutdown_connections 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75b406bf00 0x7f75b406e3c0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75c81fb5b0 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75c815ff70 0x7f75c8160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c8071f50 0x7f75c814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 --2- 172.21.15.1:0/3766807674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75c806d440 0x7f75c81fbfd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.145+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 >> 172.21.15.1:0/3766807674 conn(0x7f75c80cc4b0 msgr2=0x7f75c80cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.146+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 shutdown_connections 2024-05-09T06:27:47.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.146+0000 7f75cd790640 1 -- 172.21.15.1:0/3766807674 wait complete. 2024-05-09T06:27:47.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.147+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd2080af180 con 0x7fd2080cacd0 2024-05-09T06:27:47.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.151+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdcf80752b0 con 0x7fdcf8163360 2024-05-09T06:27:47.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.151+0000 7fd1effff640 1 -- 172.21.15.1:0/1430103642 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1fc07efd0 con 0x7fd2080cacd0 2024-05-09T06:27:47.133 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.152+0000 7fdcedffb640 1 -- 172.21.15.1:0/2949262425 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdcf8201bf0 con 0x7fdcf8163360 2024-05-09T06:27:47.134 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.152+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1ae8005850 con 0x7f1b080d2b10 2024-05-09T06:27:47.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.152+0000 7f1afdffb640 1 -- 172.21.15.1:0/1084232891 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1af80590a0 con 0x7f1b080d2b10 2024-05-09T06:27:47.135 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 msgr2=0x7fd1f406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 0x7fd1f406e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800a810 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 msgr2=0x7fd208151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b2c0 tx=0x7fd1fc004730 comp rx=0 tx=0).stop 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f43f00b7c30 con 0x7f43f01606e0 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 msgr2=0x7fdcdc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 0x7fdcdc06e340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0001e30 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 msgr2=0x7fdcf8164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c580 tx=0x7fdcf400ca40 comp rx=0 tx=0).stop 2024-05-09T06:27:47.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 shutdown_connections 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd1f406be80 0x7fd1f406e340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd20815ac80 0x7fd2080b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2080cacd0 0x7fd208151a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 --2- 172.21.15.1:0/1430103642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd20806ca60 0x7fd208151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 >> 172.21.15.1:0/1430103642 conn(0x7fd2080b1530 msgr2=0x7fd2081580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.155+0000 7f43ed7fa640 1 -- 172.21.15.1:0/1709951786 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43e4059680 con 0x7f43f01606e0 2024-05-09T06:27:47.139 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:47.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 shutdown_connections 2024-05-09T06:27:47.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7fd20cce0640 1 -- 172.21.15.1:0/1430103642 wait complete. 2024-05-09T06:27:47.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6c4c157a80 con 0x7f6c4c15a220 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 msgr2=0x7f1aec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 0x7f1aec06e370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b00034000 tx=0x7f1b0002f000 comp rx=0 tx=0).stop 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 msgr2=0x7f1b080c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.156+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c5a0 tx=0x7f1af800c9a0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 shutdown_connections 2024-05-09T06:27:47.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1aec06beb0 0x7f1aec06e370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b080b8fa0 0x7f1b080b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b08161950 0x7f1b080b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 --2- 172.21.15.1:0/1084232891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b080d2b10 0x7f1b080c9850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 >> 172.21.15.1:0/1084232891 conn(0x7f1b080d3e20 msgr2=0x7f1b080ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 shutdown_connections 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f1af37fe640 1 -- 172.21.15.1:0/1084232891 wait complete. 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7f6c42ffd640 1 -- 172.21.15.1:0/3779270071 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c44022030 con 0x7f6c4c15a220 2024-05-09T06:27:47.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 shutdown_connections 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdcdc06be80 0x7fdcdc06e340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcf8163360 0x7fdcf8164f60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcf81629e0 0x7fdcf81f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 --2- 172.21.15.1:0/2949262425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdcf814be30 0x7fdcf82016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.157+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 >> 172.21.15.1:0/2949262425 conn(0x7fdcf80cc4b0 msgr2=0x7fdcf80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.158+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 shutdown_connections 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.158+0000 7fdcff729640 1 -- 172.21.15.1:0/2949262425 wait complete. 2024-05-09T06:27:47.143 INFO:teuthology.orchestra.run.smithi001.stdout:55834574856 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 msgr2=0x7f43dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 0x7f43dc06e340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006d40 tx=0x7f43e0008040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 msgr2=0x7f43f01f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 0x7f43f01f9790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43e4011960 tx=0x7f43e4011e20 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 shutdown_connections 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43dc06be80 0x7f43dc06e340 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006d40 tx=0x7f43e0008040 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43f015db50 0x7f43f00d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43f01606e0 0x7f43f01f9790 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 --2- 172.21.15.1:0/1709951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43d8004d40 0x7f43f01fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.159+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 >> 172.21.15.1:0/1709951786 conn(0x7f43f00b37c0 msgr2=0x7f43f00cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.160+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 shutdown_connections 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.160+0000 7f43f70ca640 1 -- 172.21.15.1:0/1709951786 wait complete. 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.160+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 msgr2=0x7f6c3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 0x7f6c3006e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c48034000 tx=0x7f6c4802f000 comp rx=0 tx=0).stop 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 msgr2=0x7f6c4c15a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d600 tx=0x7f6c4400dac0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 shutdown_connections 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c3006be30 0x7f6c3006e2f0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c48034000 tx=0x7f6c4802f000 comp rx=0 tx=0).stop 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c4c15a220 0x7f6c4c15a6d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c4c0b2f80 0x7f6c4c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 --2- 172.21.15.1:0/3779270071 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c4c06e9f0 0x7f6c4c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:47.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.161+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 >> 172.21.15.1:0/3779270071 conn(0x7f6c4c0cfed0 msgr2=0x7f6c4c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:47.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-05-09T06:27:47.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.164+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 shutdown_connections 2024-05-09T06:27:47.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:47.164+0000 7f6c54729640 1 -- 172.21.15.1:0/3779270071 wait complete. 2024-05-09T06:27:47.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-05-09T06:27:47.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-05-09T06:27:47.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-05-09T06:27:47.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-05-09T06:27:47.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-05-09T06:27:47.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-05-09T06:27:47.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-05-09T06:27:47.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:27:47.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:27:48.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T06:27:48.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T06:27:48.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T06:27:48.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T06:27:48.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T06:27:48.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T06:27:48.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T06:27:48.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T06:27:48.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.338+0000 7fb3d1858640 1 Processor -- start 2024-05-09T06:27:48.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.338+0000 7fb3d1858640 1 -- start start 2024-05-09T06:27:48.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.339+0000 7f036d55a640 1 Processor -- start 2024-05-09T06:27:48.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.339+0000 7f036d55a640 1 -- start start 2024-05-09T06:27:48.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7f036d55a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0360048b00 0x7f03600526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d1858640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3c40e75d0 0x7fb3c4050430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d1858640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c40070c0 0x7fb3c4053b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d1858640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c4050d10 con 0x7fb3c4049220 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d1858640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c4051890 con 0x7fb3c40e75d0 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d1858640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c4050ff0 con 0x7fb3c40070c0 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d0856640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3c40e75d0 0x7fb3c4050430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d0856640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3c40e75d0 0x7fb3c4050430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56040/0 (socket says 172.21.15.1:56040) 2024-05-09T06:27:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7fb3d0856640 1 -- 172.21.15.1:0/2700488607 learned_addr learned my addr 172.21.15.1:0/2700488607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7f0366ffd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f03600eaca0 0x7f03600483f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:43038/0 (socket says 172.21.15.1:43038) 2024-05-09T06:27:48.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7f03667fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0360048b00 0x7f03600526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.341+0000 7f03667fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0360048b00 0x7f03600526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56050/0 (socket says 172.21.15.1:56050) 2024-05-09T06:27:48.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7f0366ffd640 1 -- 172.21.15.1:0/3265101973 learned_addr learned my addr 172.21.15.1:0/3265101973 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:48.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036003a320 con 0x7f036003b0e0 2024-05-09T06:27:48.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036003a600 con 0x7f03600eaca0 2024-05-09T06:27:48.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0360052bf0 con 0x7f0360048b00 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.343+0000 7f03667fc640 1 -- 172.21.15.1:0/3265101973 >> v1:172.21.15.6:6789/0 conn(0x7f03600eaca0 legacy=0x7f03600483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.343+0000 7f03667fc640 1 -- 172.21.15.1:0/3265101973 >> v1:172.21.15.1:6789/0 conn(0x7f036003b0e0 legacy=0x7f03600537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.343+0000 7f03667fc640 1 -- 172.21.15.1:0/3265101973 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036003a8e0 con 0x7f0360048b00 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.344+0000 7f03667fc640 1 --2- 172.21.15.1:0/3265101973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0360048b00 0x7f03600526b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f035c0097a0 tx=0x7f035c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b7c47ff0d1612 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7fb3d0856640 1 -- 172.21.15.1:0/2700488607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c40070c0 msgr2=0x7fb3c4053b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7fb3d0856640 1 --2- 172.21.15.1:0/2700488607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c40070c0 0x7fb3c4053b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7fb3d0856640 1 -- 172.21.15.1:0/2700488607 >> v1:172.21.15.6:6789/0 conn(0x7fb3c4049220 legacy=0x7fb3c40069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.342+0000 7fb3d0856640 1 -- 172.21.15.1:0/2700488607 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c40512d0 con 0x7fb3c40e75d0 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.344+0000 7fb3d0856640 1 --2- 172.21.15.1:0/2700488607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4050430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb20daa56765b812 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.344+0000 7fb3cb7fe640 1 -- 172.21.15.1:0/2700488607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0037070 con 0x7fb3c40e75d0 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.344+0000 7fb3cb7fe640 1 -- 172.21.15.1:0/2700488607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0032440 con 0x7fb3c40e75d0 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.345+0000 7fb3cb7fe640 1 -- 172.21.15.1:0/2700488607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0032810 con 0x7fb3c40e75d0 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.345+0000 7fb3d1858640 1 -- 172.21.15.1:0/2700488607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40e75d0 msgr2=0x7fb3c4050430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.345+0000 7fb3d1858640 1 --2- 172.21.15.1:0/2700488607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4050430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.345+0000 7f0365ffb640 1 -- 172.21.15.1:0/3265101973 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c037070 con 0x7f0360048b00 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7f0365ffb640 1 -- 172.21.15.1:0/3265101973 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c032440 con 0x7f0360048b00 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7f0365ffb640 1 -- 172.21.15.1:0/3265101973 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c032850 con 0x7f0360048b00 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 -- 172.21.15.1:0/2700488607 shutdown_connections 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 --2- 172.21.15.1:0/2700488607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c40070c0 0x7fb3c4053b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 --2- 172.21.15.1:0/2700488607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4050430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 -- 172.21.15.1:0/2700488607 >> 172.21.15.1:0/2700488607 conn(0x7fb3c4055bd0 msgr2=0x7fb3c4055fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0360048b00 msgr2=0x7f03600526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 --2- 172.21.15.1:0/3265101973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0360048b00 0x7f03600526b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f035c0097a0 tx=0x7f035c02a140 comp rx=0 tx=0).stop 2024-05-09T06:27:48.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 -- 172.21.15.1:0/2700488607 shutdown_connections 2024-05-09T06:27:48.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.346+0000 7fb3d1858640 1 -- 172.21.15.1:0/2700488607 wait complete. 2024-05-09T06:27:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 shutdown_connections 2024-05-09T06:27:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 --2- 172.21.15.1:0/3265101973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0360048b00 0x7f03600526b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 >> 172.21.15.1:0/3265101973 conn(0x7f036001e6f0 msgr2=0x7f0360055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 shutdown_connections 2024-05-09T06:27:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.347+0000 7f036d55a640 1 -- 172.21.15.1:0/3265101973 wait complete. 2024-05-09T06:27:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7f036d55a640 1 Processor -- start 2024-05-09T06:27:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7f036d55a640 1 -- start start 2024-05-09T06:27:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7f036d55a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 0x7f03600d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.348+0000 7fb3d1858640 1 Processor -- start 2024-05-09T06:27:48.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.348+0000 7fb3d1858640 1 -- start start 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 0x7fb3c40e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4186d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4050970 con 0x7fb3c40070c0 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c40e79b0 con 0x7fb3c40e75d0 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1858640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c4050af0 con 0x7fb3c4170f90 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.350+0000 7fb3d1057640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7f036d55a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0360048b00 0x7f03601854b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43324/0 (socket says 172.21.15.1:43324) 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 -- 172.21.15.1:0/1427202437 learned_addr learned my addr 172.21.15.1:0/1427202437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d0856640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 0x7fb3c40e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 -- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c40e75d0 msgr2=0x7fb3c4186d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4186d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 -- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 msgr2=0x7fb3c40e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 0x7fb3c40e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d1057640 1 -- 172.21.15.1:0/1427202437 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c4171470 con 0x7fb3c4170f90 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.351+0000 7fb3d0856640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 0x7fb3c40e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7fb3d1057640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc06ab70 tx=0x7fb3cc074100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7f0366ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 0x7f03600d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7f0366ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 0x7f03600d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43334/0 (socket says 172.21.15.1:43334) 2024-05-09T06:27:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.352+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc069550 con 0x7fb3c4170f90 2024-05-09T06:27:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.353+0000 7fb3d1858640 1 -- 172.21.15.1:0/1427202437 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c4171750 con 0x7fb3c4170f90 2024-05-09T06:27:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.354+0000 7f036d55a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 0x7f036000b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.354+0000 7f036d55a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f036003a0d0 con 0x7f03600eaca0 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.354+0000 7f036d55a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03600eb310 con 0x7f0360048b00 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.354+0000 7f036d55a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f036003ac50 con 0x7f036003b0e0 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.353+0000 7fb3d1858640 1 -- 172.21.15.1:0/1427202437 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c4171d10 con 0x7fb3c4170f90 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.353+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3cc08f070 con 0x7fb3c4170f90 2024-05-09T06:27:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.353+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc073300 con 0x7fb3c4170f90 2024-05-09T06:27:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.354+0000 7fb3d1858640 1 -- 172.21.15.1:0/1427202437 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac005290 con 0x7fb3c4170f90 2024-05-09T06:27:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.355+0000 7f0366ffd640 1 -- 172.21.15.1:0/2236577842 learned_addr learned my addr 172.21.15.1:0/2236577842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.355+0000 7f03677fe640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 0x7f036000b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 -- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 msgr2=0x7f03600d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 0x7f03600d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 -- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0360048b00 msgr2=0x7f03601854b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0360048b00 0x7f03601854b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 -- 172.21.15.1:0/2236577842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03600d6ff0 con 0x7f03600eaca0 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7f03677fe640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 0x7f036000b390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0368069d50 tx=0x7f0368074100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3cc0734a0 con 0x7fb3c4170f90 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7fb3c97fa640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 0x7fb3b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.356+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb3cc1053e0 con 0x7fb3c4170f90 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7fb3d0856640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 0x7fb3b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368069550 con 0x7f03600eaca0 2024-05-09T06:27:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036808f070 con 0x7f03600eaca0 2024-05-09T06:27:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368073300 con 0x7f03600eaca0 2024-05-09T06:27:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601897e0 con 0x7f03600eaca0 2024-05-09T06:27:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.357+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0360189da0 con 0x7f03600eaca0 2024-05-09T06:27:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.358+0000 7fb3d0856640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 0x7fb3b006e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0034000 tx=0x7fb3c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.360+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03680734a0 con 0x7f03600eaca0 2024-05-09T06:27:48.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.360+0000 7f034f7fe640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 0x7f035406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.360+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0368105cd0 con 0x7f03600eaca0 2024-05-09T06:27:48.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.361+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0348005290 con 0x7f03600eaca0 2024-05-09T06:27:48.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.361+0000 7f0366ffd640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 0x7f035406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.362+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3cc0d33f0 con 0x7fb3c4170f90 2024-05-09T06:27:48.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.370+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03600d6ff0 con 0x7f03600eaca0 2024-05-09T06:27:48.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.371+0000 7f0366ffd640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 0x7f035406e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0358005880 tx=0x7f0358009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.391+0000 7f75eb59e640 1 Processor -- start 2024-05-09T06:27:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.391+0000 7f75eb59e640 1 -- start start 2024-05-09T06:27:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.394+0000 7f7abbbab640 1 Processor -- start 2024-05-09T06:27:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75eb59e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f75e40507a0 0x7f75e4006fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75eb59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f75e40e3130 0x7f75e40074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75eb59e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40dfa40 con 0x7f75e40df660 2024-05-09T06:27:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75eb59e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f75e4006b20 con 0x7f75e40e3130 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75eb59e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f75e40e3630 con 0x7f75e40507a0 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75ead9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f75e40e3130 0x7f75e40074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75ead9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f75e40e3130 0x7f75e40074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56070/0 (socket says 172.21.15.1:56070) 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.393+0000 7f75ead9d640 1 -- 172.21.15.1:0/2963348766 learned_addr learned my addr 172.21.15.1:0/2963348766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.394+0000 7f75ead9d640 1 -- 172.21.15.1:0/2963348766 >> v2:172.21.15.6:3300/0 conn(0x7f75e40507a0 msgr2=0x7f75e4006fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.394+0000 7f75ead9d640 1 --2- 172.21.15.1:0/2963348766 >> v2:172.21.15.6:3300/0 conn(0x7f75e40507a0 0x7f75e4006fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.394+0000 7f75ead9d640 1 -- 172.21.15.1:0/2963348766 >> v1:172.21.15.1:6789/0 conn(0x7f75e40df660 legacy=0x7f75e4050090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:27:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.394+0000 7f75ead9d640 1 -- 172.21.15.1:0/2963348766 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40e3910 con 0x7f75e40e3130 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.395+0000 7f7abbbab640 1 -- start start 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 0x7f7ab406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 0x7f7ab40ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7ab40ceb70 0x7f7ab40cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab406ffe0 con 0x7f7ab40ceb70 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab4144c40 con 0x7f7ab4144860 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7abbbab640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab406f160 con 0x7f7ab406ed00 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7aba148640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7ab40ceb70 0x7f7ab40cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7aba148640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7ab40ceb70 0x7f7ab40cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56078/0 (socket says 172.21.15.1:56078) 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f7aba148640 1 -- 172.21.15.1:0/2574474692 learned_addr learned my addr 172.21.15.1:0/2574474692 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7ab9146640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 0x7f7ab40ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7ab9947640 1 --2- 172.21.15.1:0/2574474692 >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 0x7f7ab406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7aba148640 1 -- 172.21.15.1:0/2574474692 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 msgr2=0x7f7ab40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7aba148640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 0x7f7ab40ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7aba148640 1 -- 172.21.15.1:0/2574474692 >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 msgr2=0x7f7ab406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7aba148640 1 --2- 172.21.15.1:0/2574474692 >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 0x7f7ab406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7aba148640 1 -- 172.21.15.1:0/2574474692 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40ce150 con 0x7f7ab40ceb70 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7ab9947640 1 --2- 172.21.15.1:0/2574474692 >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 0x7f7ab406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f7ab9146640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 0x7f7ab40ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.395+0000 7f75ead9d640 1 --2- 172.21.15.1:0/2963348766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40e3130 0x7f75e40074f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75ec02d750 tx=0x7f75ec06af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7fc4729cb0093 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f75e959a640 1 -- 172.21.15.1:0/2963348766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b3070 con 0x7f75e40e3130 2024-05-09T06:27:48.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7f75e959a640 1 -- 172.21.15.1:0/2963348766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0ae680 con 0x7f75e40e3130 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 -- 172.21.15.1:0/2963348766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40e3130 msgr2=0x7f75e40074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2963348766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40e3130 0x7f75e40074f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75ec02d750 tx=0x7f75ec06af80 comp rx=0 tx=0).stop 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 -- 172.21.15.1:0/2963348766 shutdown_connections 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2963348766 >> v2:172.21.15.6:3300/0 conn(0x7f75e40507a0 0x7f75e4006fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2963348766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40e3130 0x7f75e40074f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f75eb59e640 1 -- 172.21.15.1:0/2963348766 >> 172.21.15.1:0/2963348766 conn(0x7f75e4055350 msgr2=0x7f75e4055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7fcdcb59e640 1 Processor -- start 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.396+0000 7fcdcb59e640 1 -- start start 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7fcdcb59e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcdcc159d20 0x7fcdcc0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7fcdcb59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcdcc0cb1e0 0x7fcdcc070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7fcdcb59e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdcc0cb6f0 con 0x7fcdcc071150 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7fcdcb59e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcdcc0715a0 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7fcdcb59e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcdcc0cb9d0 con 0x7fcdcc159d20 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7f75eb59e640 1 -- 172.21.15.1:0/2963348766 shutdown_connections 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.397+0000 7f712f2a5640 1 Processor -- start 2024-05-09T06:27:48.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7f712f2a5640 1 -- start start 2024-05-09T06:27:48.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcdcc159d20 0x7fcdcc0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcdcc159d20 0x7fcdcc0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:46606/0 (socket says 172.21.15.1:46606) 2024-05-09T06:27:48.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 -- 172.21.15.1:0/169553456 learned_addr learned my addr 172.21.15.1:0/169553456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f7aba148640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab40ceb70 0x7f7ab40cefd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ab404f830 tx=0x7f7ab002e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfa9dec5f43744e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7f75eb59e640 1 -- 172.21.15.1:0/2963348766 wait complete. 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712f2a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f712806ca70 0x7f712806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712f2a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71280c9e20 con 0x7f71280cbc20 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712f2a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71280ca0c0 con 0x7f7128159d00 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712f2a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f712806d410 con 0x7f712806ca70 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712d842640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f71280cbc20 0x7f71280cc070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58722/0 (socket says 172.21.15.1:58722) 2024-05-09T06:27:48.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.399+0000 7f712d842640 1 -- 172.21.15.1:0/2721629119 learned_addr learned my addr 172.21.15.1:0/2721629119 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:48.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 -- 172.21.15.1:0/169553456 >> v2:172.21.15.1:3300/0 conn(0x7fcdcc0cb1e0 msgr2=0x7fcdcc070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:48.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 --2- 172.21.15.1:0/169553456 >> v2:172.21.15.1:3300/0 conn(0x7fcdcc0cb1e0 0x7fcdcc070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 -- 172.21.15.1:0/169553456 >> v1:172.21.15.6:6789/0 conn(0x7fcdcc071150 legacy=0x7fcdcc15d900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:48.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.398+0000 7fcdca59c640 1 -- 172.21.15.1:0/169553456 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc0cd560 con 0x7fcdcc159d20 2024-05-09T06:27:48.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f712c840640 1 --2- 172.21.15.1:0/2721629119 >> v2:172.21.15.1:3300/0 conn(0x7f712806ca70 0x7f712806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f7ab8945640 1 -- 172.21.15.1:0/2574474692 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab003a070 con 0x7f7ab40ceb70 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f7ab8945640 1 -- 172.21.15.1:0/2574474692 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab0033710 con 0x7f7ab40ceb70 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f7ab8945640 1 -- 172.21.15.1:0/2574474692 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0033ae0 con 0x7f7ab40ceb70 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f75eb59e640 1 Processor -- start 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f75eb59e640 1 -- start start 2024-05-09T06:27:48.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f711ffff640 1 -- 172.21.15.1:0/2721629119 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301026681 0 0) 0x7f71280ca0c0 con 0x7f7128159d00 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f712c840640 1 -- 172.21.15.1:0/2721629119 >> v1:172.21.15.6:6789/0 conn(0x7f7128159d00 legacy=0x7f71280cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f712c840640 1 -- 172.21.15.1:0/2721629119 >> v1:172.21.15.1:6789/0 conn(0x7f71280cbc20 legacy=0x7f71280cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7f712c840640 1 -- 172.21.15.1:0/2721629119 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71280ca3a0 con 0x7f712806ca70 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7fcdca59c640 1 --2- 172.21.15.1:0/169553456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0caca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0741e0 tx=0x7fcdc402ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de1ae70df820908 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7fcdc959a640 1 -- 172.21.15.1:0/169553456 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4035070 con 0x7fcdcc159d20 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7fcdc959a640 1 -- 172.21.15.1:0/169553456 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc402f950 con 0x7fcdcc159d20 2024-05-09T06:27:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.400+0000 7fcdc959a640 1 -- 172.21.15.1:0/169553456 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400be60 con 0x7fcdcc159d20 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f272bfff640 1 Processor -- start 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 -- 172.21.15.1:0/2574474692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab40ceb70 msgr2=0x7f7ab40cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab40ceb70 0x7f7ab40cefd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ab404f830 tx=0x7f7ab002e4e0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 -- 172.21.15.1:0/2574474692 shutdown_connections 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab40ceb70 0x7f7ab40cefd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 --2- 172.21.15.1:0/2574474692 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab406ed00 0x7f7ab40ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f75eb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40507a0 0x7f75e40d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f75eb59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 --2- 172.21.15.1:0/2574474692 >> v2:172.21.15.6:3300/0 conn(0x7f7ab4144860 0x7f7ab406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f7abbbab640 1 -- 172.21.15.1:0/2574474692 >> 172.21.15.1:0/2574474692 conn(0x7f7ab40d0c00 msgr2=0x7f7ab40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f712c840640 1 --2- 172.21.15.1:0/2721629119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712806ca70 0x7f712806ced0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f712400a940 tx=0x7f712402b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c223cb93ef88aac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f272bfff640 1 -- start start 2024-05-09T06:27:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f7abbbab640 1 -- 172.21.15.1:0/2574474692 shutdown_connections 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.401+0000 7f75eb59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75e40d5d00 0x7f75e40dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75e9d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75e9d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43338/0 (socket says 172.21.15.1:43338) 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75e9d9b640 1 -- 172.21.15.1:0/2238580474 learned_addr learned my addr 172.21.15.1:0/2238580474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75e4054fb0 con 0x7f75e40507a0 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75ead9d640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75e40d5d00 0x7f75e40dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40e3ed0 con 0x7f75e40d5d00 2024-05-09T06:27:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f711ffff640 1 -- 172.21.15.1:0/2721629119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400b040 con 0x7f712806ca70 2024-05-09T06:27:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75e4055130 con 0x7f75e40df660 2024-05-09T06:27:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f711ffff640 1 -- 172.21.15.1:0/2721629119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124030510 con 0x7f712806ca70 2024-05-09T06:27:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75ea59c640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40507a0 0x7f75e40d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c0aea90 0x7f272c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272bfff640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272c0ceb00 con 0x7f272c0ce720 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272bfff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272c0cef70 con 0x7f272c0cd430 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272bfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f272c0cf250 con 0x7f272c0aea90 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7fcdcb59e640 1 -- 172.21.15.1:0/169553456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc159d20 msgr2=0x7fcdcc0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/169553456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0caca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0741e0 tx=0x7fcdc402ec40 comp rx=0 tx=0).stop 2024-05-09T06:27:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 -- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75e40d5d00 msgr2=0x7f75e40dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272b7fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f272c0cd430 0x7f272c06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:43058/0 (socket says 172.21.15.1:43058) 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272b7fe640 1 -- 172.21.15.1:0/2755622697 learned_addr learned my addr 172.21.15.1:0/2755622697 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75e40d5d00 0x7f75e40dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 -- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40507a0 msgr2=0x7f75e40d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40507a0 0x7f75e40d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e404e960 con 0x7f75e40df660 2024-05-09T06:27:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.402+0000 7f7abbbab640 1 -- 172.21.15.1:0/2574474692 wait complete. 2024-05-09T06:27:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f272a7fc640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c0aea90 0x7f272c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f75e9d9b640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b450 tx=0x7f75dc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 -- 172.21.15.1:0/2721629119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712806ca70 msgr2=0x7f712806ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 --2- 172.21.15.1:0/2721629119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712806ca70 0x7f712806ced0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f712400a940 tx=0x7f712402b5e0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 -- 172.21.15.1:0/2721629119 shutdown_connections 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 --2- 172.21.15.1:0/2721629119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712806ca70 0x7f712806ced0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 -- 172.21.15.1:0/2721629119 >> 172.21.15.1:0/2721629119 conn(0x7f71280b14c0 msgr2=0x7f71280b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f712f2a5640 1 -- 172.21.15.1:0/2721629119 shutdown_connections 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816137639 0 0) 0x7f272c0cef70 con 0x7f272c0cd430 2024-05-09T06:27:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.403+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2714003520 con 0x7f272c0cd430 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc015020 con 0x7f75e40df660 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e404ec40 con 0x7f75e40df660 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc011850 con 0x7f75e40df660 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2957951587 0 0) 0x7f2714003520 con 0x7f272c0cd430 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c0aea90 msgr2=0x7f272c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:48.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c0aea90 0x7f272c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 >> v1:172.21.15.1:6789/0 conn(0x7f272c0ce720 legacy=0x7f272c0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0c9f50 con 0x7f272c0cd430 2024-05-09T06:27:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f271c001650 con 0x7f272c0cd430 2024-05-09T06:27:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 >> v1:172.21.15.6:6789/0 conn(0x7f272c0cd430 legacy=0x7f272c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e40d52a0 con 0x7f75e40df660 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2714003a10 0x7f2714003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2714004800 0x7f2714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27140052e0 0x7f271400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0cf250 con 0x7f27140052e0 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f271400c230 con 0x7f2714004800 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f271400c430 con 0x7f2714003a10 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f7abbbab640 1 Processor -- start 2024-05-09T06:27:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc011c40 con 0x7f75e40df660 2024-05-09T06:27:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f272b7fe640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27140052e0 0x7f271400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7fcdcb59e640 1 -- 172.21.15.1:0/169553456 shutdown_connections 2024-05-09T06:27:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/169553456 >> v2:172.21.15.1:3300/0 conn(0x7fcdcc0cb1e0 0x7fcdcc070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/169553456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0caca0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7fcdcb59e640 1 -- 172.21.15.1:0/169553456 >> 172.21.15.1:0/169553456 conn(0x7fcdcc0cf860 msgr2=0x7fcdcc0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f7abbbab640 1 -- start start 2024-05-09T06:27:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7fcdcb59e640 1 -- 172.21.15.1:0/169553456 shutdown_connections 2024-05-09T06:27:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7fcdcb59e640 1 -- 172.21.15.1:0/169553456 wait complete. 2024-05-09T06:27:48.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f272b7fe640 1 -- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2714003a10 msgr2=0x7f2714003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:27:48.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f272b7fe640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2714003a10 0x7f2714003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f272b7fe640 1 -- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2714004800 msgr2=0x7f2714004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:27:48.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f272b7fe640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2714004800 0x7f2714004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f7abbbab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f272b7fe640 1 -- 172.21.15.1:0/2755622697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0c9f50 con 0x7f27140052e0 2024-05-09T06:27:48.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f272b7fe640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27140052e0 0x7f271400b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c260 tx=0x7f2714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e068c0c878f1ca22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f7ab9947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f7ab9947640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56118/0 (socket says 172.21.15.1:56118) 2024-05-09T06:27:48.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75dc018020 con 0x7f75e40df660 2024-05-09T06:27:48.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.404+0000 7f712f2a5640 1 -- 172.21.15.1:0/2721629119 wait complete. 2024-05-09T06:27:48.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f7ab9947640 1 -- 172.21.15.1:0/1760851538 learned_addr learned my addr 172.21.15.1:0/1760851538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.405+0000 7f712f2a5640 1 Processor -- start 2024-05-09T06:27:48.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ab40ceb70 0x7f7ab41e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.406+0000 7f712f2a5640 1 -- start start 2024-05-09T06:27:48.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab4144860 0x7f7ab41e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406f990 con 0x7f7ab406ed00 2024-05-09T06:27:48.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7fcdcb59e640 1 Processor -- start 2024-05-09T06:27:48.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406f6b0 con 0x7f7ab40ceb70 2024-05-09T06:27:48.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab406fc70 con 0x7f7ab4144860 2024-05-09T06:27:48.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7fcdcb59e640 1 -- start start 2024-05-09T06:27:48.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7aba148640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab4144860 0x7f7ab41e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f7ab9146640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ab40ceb70 0x7f7ab41e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712f2a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 0x7f71280b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d0005290 con 0x7f75e40df660 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f75daffd640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 0x7f75cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f75ea59c640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 0x7f75cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c036070 con 0x7f27140052e0 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c005ad0 con 0x7f27140052e0 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f2729ffb640 1 -- 172.21.15.1:0/2755622697 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c036070 con 0x7f27140052e0 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75dc08b100 con 0x7f75e40df660 2024-05-09T06:27:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc071150 0x7fcdcc06f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc15a100 con 0x7fcdcc159d20 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc0cdbb0 con 0x7fcdcc071150 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdcb59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcdcc0af110 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f7ab9947640 1 -- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab4144860 msgr2=0x7f7ab41e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f7ab9947640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab4144860 0x7f7ab41e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f7ab9947640 1 -- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ab40ceb70 msgr2=0x7f7ab41e1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdc9d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7fcdca59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc071150 0x7fcdcc06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43402/0 (socket says 172.21.15.1:43402) 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 -- 172.21.15.1:0/4138828500 learned_addr learned my addr 172.21.15.1:0/4138828500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdcad9d640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f7ab9947640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ab40ceb70 0x7f7ab41e1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f7ab9947640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab4142160 con 0x7f7ab406ed00 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 -- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc071150 msgr2=0x7fcdcc06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc071150 0x7fcdcc06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 -- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 msgr2=0x7fcdcc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 -- 172.21.15.1:0/4138828500 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc14fa30 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f7ab9947640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f272bfff640 1 -- 172.21.15.1:0/2755622697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27140052e0 msgr2=0x7f271400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f272bfff640 1 --2- 172.21.15.1:0/2755622697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27140052e0 0x7f271400b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c260 tx=0x7f2714005830 comp rx=0 tx=0).stop 2024-05-09T06:27:48.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdcad9d640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:48.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f272b7fe640 1 -- 172.21.15.1:0/2755622697 reap_dead start 2024-05-09T06:27:48.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f75ea59c640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 0x7f75cc06e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006fd0 tx=0x7f75e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712f2a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712f2a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7128159d00 0x7f71281561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712d041640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 0x7f71280b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712d041640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 0x7f71280b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:46630/0 (socket says 172.21.15.1:46630) 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.407+0000 7f712c840640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56126/0 (socket says 172.21.15.1:56126) 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712d041640 1 -- 172.21.15.1:0/1720834926 learned_addr learned my addr 172.21.15.1:0/1720834926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712f2a5640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71280cf360 con 0x7f71280cbc20 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712f2a5640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f71280cee30 con 0x7f712806ca70 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712f2a5640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f712815a0e0 con 0x7f7128159d00 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712d842640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7128159d00 0x7f71281561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004030 con 0x7f7ab406ed00 2024-05-09T06:27:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 -- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7128159d00 msgr2=0x7f71281561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7128159d00 0x7f71281561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 -- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 msgr2=0x7f71280b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 0x7f71280b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.408+0000 7f712c840640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7128156a10 con 0x7f71280cbc20 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f712c840640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7124032000 tx=0x7f7124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7fcdc9d9b640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000b4a0 tx=0x7fcdc000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f272bfff640 1 -- 172.21.15.1:0/2755622697 shutdown_connections 2024-05-09T06:27:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.409+0000 7f272bfff640 1 -- 172.21.15.1:0/2755622697 >> 172.21.15.1:0/2755622697 conn(0x7f272c0b1530 msgr2=0x7f272c0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa40041d0 con 0x7f7ab406ed00 2024-05-09T06:27:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab41423d0 con 0x7f7ab406ed00 2024-05-09T06:27:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f272bfff640 1 -- 172.21.15.1:0/2755622697 shutdown_connections 2024-05-09T06:27:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f272bfff640 1 -- 172.21.15.1:0/2755622697 wait complete. 2024-05-09T06:27:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40096a0 con 0x7f7ab406ed00 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712403b460 con 0x7f71280cbc20 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124040070 con 0x7f71280cbc20 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712403b800 con 0x7f71280cbc20 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f712f2a5640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712814aa00 con 0x7f71280cbc20 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f712f2a5640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712814afc0 con 0x7f71280cbc20 2024-05-09T06:27:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.410+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41f0980 con 0x7f7ab406ed00 2024-05-09T06:27:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.412+0000 7f272bfff640 1 Processor -- start 2024-05-09T06:27:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.412+0000 7f272bfff640 1 -- start start 2024-05-09T06:27:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab40752b0 con 0x7f7ab406ed00 2024-05-09T06:27:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c1f81b0 0x7f272c1644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f272c1f8590 0x7f272c159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0cae60 con 0x7f272c1f8590 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f272c06cc80 con 0x7f272c0b2620 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272bfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f272c0cf530 con 0x7f272c1f81b0 2024-05-09T06:27:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0014070 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc0010610 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0010a00 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75dc01f020 con 0x7f75e40df660 2024-05-09T06:27:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f272b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:46644/0 (socket says 172.21.15.1:46644) 2024-05-09T06:27:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 -- 172.21.15.1:0/3323908393 learned_addr learned my addr 172.21.15.1:0/3323908393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc14fd10 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1457b0 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272affd640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c1f81b0 0x7f272c1644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 -- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c1f81b0 msgr2=0x7f272c1644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c1f81b0 0x7f272c1644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 -- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f272c1f8590 msgr2=0x7f272c159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f272c1f8590 0x7f272c159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f272b7fe640 1 -- 172.21.15.1:0/3323908393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0b2b50 con 0x7f272c0b2620 2024-05-09T06:27:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7aa4009840 con 0x7f7ab406ed00 2024-05-09T06:27:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f7aaa7fc640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 0x7f7a9806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f272b7fe640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c260 tx=0x7f271c001a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7aa408b3e0 con 0x7f7ab406ed00 2024-05-09T06:27:48.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c036070 con 0x7f272c0b2620 2024-05-09T06:27:48.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc04f660 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c0b2e30 con 0x7f272c0b2620 2024-05-09T06:27:48.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.412+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712804f660 con 0x7f71280cbc20 2024-05-09T06:27:48.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.413+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7124030510 con 0x7f71280cbc20 2024-05-09T06:27:48.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c0b33f0 con 0x7f272c0b2620 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f711dffb640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 0x7f710c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f71240aba10 con 0x7f71280cbc20 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.414+0000 7f712d041640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 0x7f710c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.415+0000 7f712d041640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 0x7f710c06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71280b0970 tx=0x7f7118009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.416+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c042070 con 0x7f272c0b2620 2024-05-09T06:27:48.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.416+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c036070 con 0x7f272c0b2620 2024-05-09T06:27:48.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.416+0000 7f7bd8d22640 1 Processor -- start 2024-05-09T06:27:48.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.416+0000 7f7bd8d22640 1 -- start start 2024-05-09T06:27:48.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.416+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdc0017070 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.417+0000 7fcdbaffd640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 0x7fcdac06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.417+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcdc008ab20 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7ab9146640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 0x7f7a9806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd8d22640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7bd40cf180 0x7f7bd40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd8d22640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd40ce0e0 con 0x7f7bd40cdab0 2024-05-09T06:27:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd8d22640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd40ce3c0 con 0x7f7bd406e6d0 2024-05-09T06:27:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd8d22640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd40cdec0 con 0x7f7bd40cf180 2024-05-09T06:27:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f271c031440 con 0x7f272c0b2620 2024-05-09T06:27:48.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd2d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7bd40cdab0 0x7f7bd40b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58742/0 (socket says 172.21.15.1:58742) 2024-05-09T06:27:48.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd2d9d640 1 -- 172.21.15.1:0/1745658412 learned_addr learned my addr 172.21.15.1:0/1745658412 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:48.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7fcdca59c640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 0x7fcdac06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7fcdca59c640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 0x7fcdac06e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdc403a000 tx=0x7fcdc400d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7bd1d9b640 1 --2- 172.21.15.1:0/1745658412 >> v2:172.21.15.1:3300/0 conn(0x7f7bd40cf180 0x7f7bd40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.418+0000 7f7ab9146640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 0x7f7a9806e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00bbf0 tx=0x7f7aac00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c005290 con 0x7f272c0b2620 2024-05-09T06:27:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f7bd159a640 1 -- 172.21.15.1:0/1745658412 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150493992 0 0) 0x7f7bd40ce3c0 con 0x7f7bd406e6d0 2024-05-09T06:27:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f7bd159a640 1 -- 172.21.15.1:0/1745658412 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bbc003520 con 0x7f7bd406e6d0 2024-05-09T06:27:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f7bd1d9b640 1 -- 172.21.15.1:0/1745658412 >> v1:172.21.15.6:6789/0 conn(0x7f7bd406e6d0 legacy=0x7f7bd406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f7bd1d9b640 1 -- 172.21.15.1:0/1745658412 >> v1:172.21.15.1:6789/0 conn(0x7f7bd40cdab0 legacy=0x7f7bd40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f2713fff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 0x7f271806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f271c0058d0 con 0x7f272c0b2620 2024-05-09T06:27:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f272affd640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 0x7f271806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.419+0000 7f7bd1d9b640 1 -- 172.21.15.1:0/1745658412 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd40ce6a0 con 0x7f7bd40cf180 2024-05-09T06:27:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.420+0000 7f272affd640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 0x7f271806e6f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2724007fd0 tx=0x7f2724009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.420+0000 7f7bd1d9b640 1 --2- 172.21.15.1:0/1745658412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cf180 0x7f7bd40cd570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80097a0 tx=0x7f7bc802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a91213a31f549c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.421+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcdc0058bb0 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.424+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f271c0790f0 con 0x7f272c0b2620 2024-05-09T06:27:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.425+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7aa4059a30 con 0x7f7ab406ed00 2024-05-09T06:27:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.433+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f712407a0f0 con 0x7f71280cbc20 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.433+0000 7f7bd159a640 1 -- 172.21.15.1:0/1745658412 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8037070 con 0x7f7bd40cf180 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.433+0000 7f7bd159a640 1 -- 172.21.15.1:0/1745658412 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8032440 con 0x7f7bd40cf180 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.434+0000 7f7bd8d22640 1 -- 172.21.15.1:0/1745658412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cf180 msgr2=0x7f7bd40cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.434+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/1745658412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cf180 0x7f7bd40cd570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80097a0 tx=0x7f7bc802a230 comp rx=0 tx=0).stop 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.434+0000 7f7bd8d22640 1 -- 172.21.15.1:0/1745658412 shutdown_connections 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.434+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/1745658412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cf180 0x7f7bd40cd570 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80097a0 tx=0x7f7bc802a230 comp rx=0 tx=0).stop 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.434+0000 7f7bd8d22640 1 -- 172.21.15.1:0/1745658412 >> 172.21.15.1:0/1745658412 conn(0x7f7bd40cc4b0 msgr2=0x7f7bd40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.435+0000 7f7bd8d22640 1 -- 172.21.15.1:0/1745658412 shutdown_connections 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.435+0000 7f7bd8d22640 1 -- 172.21.15.1:0/1745658412 wait complete. 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.437+0000 7f7bd8d22640 1 Processor -- start 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.437+0000 7f7bd8d22640 1 -- start start 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd406e6d0 0x7f7bd415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cdab0 0x7f7bd41e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4070ed0 con 0x7f7bd40cdab0 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd40cec60 con 0x7f7bd406e6d0 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.438+0000 7f7bd8d22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd41447c0 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43414/0 (socket says 172.21.15.1:43414) 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 -- 172.21.15.1:0/2857268697 learned_addr learned my addr 172.21.15.1:0/2857268697 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 -- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd406e6d0 msgr2=0x7f7bd415a700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:27:48.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd406e6d0 0x7f7bd415a700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 -- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cdab0 msgr2=0x7f7bd41e5320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cdab0 0x7f7bd41e5320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 -- 172.21.15.1:0/2857268697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd415c0c0 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.439+0000 7f7bd2d9d640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0021f0 tx=0x7f7bcc00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.440+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00ad40 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.440+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd415c3a0 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.440+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd415cc40 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.443+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc019070 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.443+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc015b60 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.444+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bcc007070 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.444+0000 7f7bc2ffd640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 0x7f7bb406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.445+0000 7f7bd259c640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 0x7f7bb406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.446+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7bcc093db0 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.446+0000 7f7bd259c640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 0x7f7bb406e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4001560 tx=0x7f7bc4007840 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:48.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.447+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8005290 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.452+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bcc062400 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.776+0000 7fb3d1858640 1 -- 172.21.15.1:0/1427202437 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb3ac005580 con 0x7fb3c4170f90 2024-05-09T06:27:48.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.777+0000 7fb3c97fa640 1 -- 172.21.15.1:0/1427202437 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3cc0d76f0 con 0x7fb3c4170f90 2024-05-09T06:27:48.759 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.780+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 msgr2=0x7fb3b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.780+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 0x7fb3b006e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0034000 tx=0x7fb3c002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:48.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.780+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 msgr2=0x7fb3c4172f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.780+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc06ab70 tx=0x7fb3cc074100 comp rx=0 tx=0).stop 2024-05-09T06:27:48.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.780+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 shutdown_connections 2024-05-09T06:27:48.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3b006be30 0x7fb3b006e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3c4170f90 0x7fb3c4172f10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c40e75d0 0x7fb3c4186d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 --2- 172.21.15.1:0/1427202437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c40070c0 0x7fb3c40e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 >> 172.21.15.1:0/1427202437 conn(0x7fb3c4055bd0 msgr2=0x7fb3c40069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 shutdown_connections 2024-05-09T06:27:48.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.781+0000 7fb3b6ffd640 1 -- 172.21.15.1:0/1427202437 wait complete. 2024-05-09T06:27:48.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-05-09T06:27:48.780 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.801+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0348005580 con 0x7f03600eaca0 2024-05-09T06:27:48.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.802+0000 7f034f7fe640 1 -- 172.21.15.1:0/2236577842 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03680d4360 con 0x7f03600eaca0 2024-05-09T06:27:48.785 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.805+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 msgr2=0x7f035406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.805+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 0x7f035406e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0358005880 tx=0x7f0358009040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 msgr2=0x7f036000b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 0x7f036000b390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0368069d50 tx=0x7f0368074100 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 shutdown_connections 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f035406bf00 0x7f035406e3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03600eaca0 0x7f036000b390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0360048b00 0x7f03601854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 --2- 172.21.15.1:0/2236577842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f036003b0e0 0x7f03600d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 >> 172.21.15.1:0/2236577842 conn(0x7f036001e6f0 msgr2=0x7f0360047f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 shutdown_connections 2024-05-09T06:27:48.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.806+0000 7f036d55a640 1 -- 172.21.15.1:0/2236577842 wait complete. 2024-05-09T06:27:48.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-05-09T06:27:48.804 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.825+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f75d0005580 con 0x7f75e40df660 2024-05-09T06:27:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.826+0000 7f75daffd640 1 -- 172.21.15.1:0/2238580474 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75dc0596d0 con 0x7f75e40df660 2024-05-09T06:27:48.807 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 msgr2=0x7f75cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 0x7f75cc06e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006fd0 tx=0x7f75e0008040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 msgr2=0x7f75e40d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b450 tx=0x7f75dc00b910 comp rx=0 tx=0).stop 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 shutdown_connections 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75cc06be60 0x7f75cc06e320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75e40d5d00 0x7f75e40dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75e40df660 0x7f75e40d57c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 --2- 172.21.15.1:0/2238580474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75e40507a0 0x7f75e40d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 >> 172.21.15.1:0/2238580474 conn(0x7f75e4055350 msgr2=0x7f75e4168ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 shutdown_connections 2024-05-09T06:27:48.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.828+0000 7f75eb59e640 1 -- 172.21.15.1:0/2238580474 wait complete. 2024-05-09T06:27:48.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.835+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7bb8005580 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.835+0000 7f7bc2ffd640 1 -- 172.21.15.1:0/2857268697 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7bcc061d00 con 0x7f7bd41f3bd0 2024-05-09T06:27:48.818 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.837+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 msgr2=0x7f7bb406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.837+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 0x7f7bb406e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4001560 tx=0x7f7bc4007840 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.837+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 msgr2=0x7f7bd40b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.837+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0021f0 tx=0x7f7bcc00a430 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 shutdown_connections 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb406bf30 0x7f7bb406e3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd41f3bd0 0x7f7bd40b4310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd40cdab0 0x7f7bd41e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 --2- 172.21.15.1:0/2857268697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd406e6d0 0x7f7bd415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 >> 172.21.15.1:0/2857268697 conn(0x7f7bd40cc4b0 msgr2=0x7f7bd40b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.838+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 shutdown_connections 2024-05-09T06:27:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.839+0000 7f7bd8d22640 1 -- 172.21.15.1:0/2857268697 wait complete. 2024-05-09T06:27:48.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-05-09T06:27:48.825 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-05-09T06:27:48.832 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.851+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcdcc0b02b0 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.851+0000 7fcdbaffd640 1 -- 172.21.15.1:0/4138828500 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcdc005ceb0 con 0x7fcdcc0cb1e0 2024-05-09T06:27:48.833 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 msgr2=0x7fcdac06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 0x7fcdac06e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdc403a000 tx=0x7fcdc400d040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 msgr2=0x7fcdcc145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000b4a0 tx=0x7fcdc000b960 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 shutdown_connections 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdac06bf00 0x7fcdac06e3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc159d20 0x7fcdcc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdcc0cb1e0 0x7fcdcc145270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 --2- 172.21.15.1:0/4138828500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdcc071150 0x7fcdcc06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 >> 172.21.15.1:0/4138828500 conn(0x7fcdcc0cf860 msgr2=0x7fcdcc0af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 shutdown_connections 2024-05-09T06:27:48.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.855+0000 7fcdcb59e640 1 -- 172.21.15.1:0/4138828500 wait complete. 2024-05-09T06:27:48.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.856+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f270c005580 con 0x7f272c0b2620 2024-05-09T06:27:48.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.856+0000 7f2713fff640 1 -- 172.21.15.1:0/3323908393 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f271c0790f0 con 0x7f272c0b2620 2024-05-09T06:27:48.839 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 msgr2=0x7f271806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 0x7f271806e6f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2724007fd0 tx=0x7f2724009040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 msgr2=0x7f272c1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c260 tx=0x7f271c001a20 comp rx=0 tx=0).stop 2024-05-09T06:27:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 shutdown_connections 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f271806c230 0x7f271806e6f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f272c0b2620 0x7f272c1598a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f272c1f8590 0x7f272c159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 --2- 172.21.15.1:0/3323908393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f272c1f81b0 0x7f272c1644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.858+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 >> 172.21.15.1:0/3323908393 conn(0x7f272c0b1530 msgr2=0x7f272c0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.859+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 shutdown_connections 2024-05-09T06:27:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.859+0000 7f272bfff640 1 -- 172.21.15.1:0/3323908393 wait complete. 2024-05-09T06:27:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.860+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7ab40cf5b0 con 0x7f7ab406ed00 2024-05-09T06:27:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.860+0000 7f7aaa7fc640 1 -- 172.21.15.1:0/1760851538 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7aa4059330 con 0x7f7ab406ed00 2024-05-09T06:27:48.844 INFO:teuthology.orchestra.run.smithi001.stdout:55834574857 2024-05-09T06:27:48.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.862+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 msgr2=0x7f7a9806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.862+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 0x7f7a9806e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00bbf0 tx=0x7f7aac00b040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.862+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 msgr2=0x7f7ab413bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.862+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 shutdown_connections 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9806bf00 0x7f7a9806e3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ab4144860 0x7f7ab41e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ab40ceb70 0x7f7ab41e1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 --2- 172.21.15.1:0/1760851538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab406ed00 0x7f7ab413bb90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 >> 172.21.15.1:0/1760851538 conn(0x7f7ab40d0c00 msgr2=0x7f7ab40cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 shutdown_connections 2024-05-09T06:27:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.863+0000 7f7abbbab640 1 -- 172.21.15.1:0/1760851538 wait complete. 2024-05-09T06:27:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.864+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f712814b190 con 0x7f71280cbc20 2024-05-09T06:27:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.865+0000 7f711dffb640 1 -- 172.21.15.1:0/1720834926 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71240799f0 con 0x7f71280cbc20 2024-05-09T06:27:48.848 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T06:27:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 msgr2=0x7f710c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 0x7f710c06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71280b0970 tx=0x7f7118009040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 msgr2=0x7f7128155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7124032000 tx=0x7f7124004730 comp rx=0 tx=0).stop 2024-05-09T06:27:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 shutdown_connections 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f710c06bf00 0x7f710c06e3c0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71280b0970 tx=0x7f7118009040 comp rx=0 tx=0).stop 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7128159d00 0x7f71281561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71280cbc20 0x7f7128155c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 --2- 172.21.15.1:0/1720834926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f712806ca70 0x7f71280b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 >> 172.21.15.1:0/1720834926 conn(0x7f71280b14c0 msgr2=0x7f7128158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 shutdown_connections 2024-05-09T06:27:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:48.867+0000 7f71137fe640 1 -- 172.21.15.1:0/1720834926 wait complete. 2024-05-09T06:27:48.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-05-09T06:27:48.853 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-05-09T06:27:48.855 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-05-09T06:27:48.860 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-05-09T06:27:48.862 DEBUG:teuthology.parallel:result is None 2024-05-09T06:27:48.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T06:27:48.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:27:49.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.024+0000 7fd2ade9a640 1 Processor -- start 2024-05-09T06:27:49.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2ade9a640 1 -- start start 2024-05-09T06:27:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2ade9a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 0x7fd2a80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2ade9a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80b33c0 con 0x7fd2a80d4940 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2ade9a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80b3660 con 0x7fd2a8161950 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2ade9a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a80d4dd0 con 0x7fd2a80d2b10 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.025+0000 7fd2a6ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd2a80d4940 0x7fd2a80b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58758/0 (socket says 172.21.15.1:58758) 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a6ffd640 1 -- 172.21.15.1:0/2095609605 learned_addr learned my addr 172.21.15.1:0/2095609605 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a7fff640 1 --2- 172.21.15.1:0/2095609605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 0x7fd2a80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504594040 0 0) 0x7fd2a80b33c0 con 0x7fd2a80d4940 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd290003520 con 0x7fd2a80d4940 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3624400880 0 0) 0x7fd2a80b3660 con 0x7fd2a8161950 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a80b33c0 con 0x7fd2a8161950 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a7fff640 1 -- 172.21.15.1:0/2095609605 >> v1:172.21.15.6:6789/0 conn(0x7fd2a8161950 legacy=0x7fd2a80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a7fff640 1 -- 172.21.15.1:0/2095609605 >> v1:172.21.15.1:6789/0 conn(0x7fd2a80d4940 legacy=0x7fd2a80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.026+0000 7fd2a7fff640 1 -- 172.21.15.1:0/2095609605 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a80b3940 con 0x7fd2a80d2b10 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2a7fff640 1 --2- 172.21.15.1:0/2095609605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 0x7fd2a80b4850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009dc0 tx=0x7fd2a002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626a2f50c8e98bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0007740 con 0x7fd2a80d2b10 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a00078e0 con 0x7fd2a80d2b10 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2a67fc640 1 -- 172.21.15.1:0/2095609605 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0007c90 con 0x7fd2a80d2b10 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2095609605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 msgr2=0x7fd2a80b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2095609605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 0x7fd2a80b4850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009dc0 tx=0x7fd2a002a6d0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2095609605 shutdown_connections 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2095609605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a80d2b10 0x7fd2a80b4850 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.027+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2095609605 >> 172.21.15.1:0/2095609605 conn(0x7fd2a80d3e20 msgr2=0x7fd2a80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.028+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2095609605 shutdown_connections 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.028+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2095609605 wait complete. 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.029+0000 7fd2ade9a640 1 Processor -- start 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.029+0000 7fd2ade9a640 1 -- start start 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a8161950 0x7fd2a81f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 0x7fd2a806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a80d1100 con 0x7fd2a80d2b10 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a80d0f80 con 0x7fd2a81f7460 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a80d1c00 con 0x7fd2a8161950 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56174/0 (socket says 172.21.15.1:56174) 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 -- 172.21.15.1:0/2582471080 learned_addr learned my addr 172.21.15.1:0/2582471080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a6ffd640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a8161950 0x7fd2a81f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a7fff640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 0x7fd2a806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 -- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a8161950 msgr2=0x7fd2a81f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a8161950 0x7fd2a81f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 -- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 msgr2=0x7fd2a806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 0x7fd2a806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 -- 172.21.15.1:0/2582471080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81f7970 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a7fff640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 0x7fd2a806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2a77fe640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c430 tx=0x7fd29800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298017070 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2980052f0 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81f7c50 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298003830 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.030+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a80b6e90 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.031+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a80752b0 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.032+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd29801a070 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.032+0000 7fd28ffff640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.032+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd29808a640 con 0x7fd2a80d2b10 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.032+0000 7fd2a6ffd640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.033+0000 7fd2a6ffd640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 0x7fd29406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2a806db20 tx=0x7fd29c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.037+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd298058cc0 con 0x7fd2a80d2b10 2024-05-09T06:27:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.215+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2a80cdd00 con 0x7fd29406be30 2024-05-09T06:27:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.223+0000 7fd28ffff640 1 -- 172.21.15.1:0/2582471080 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7fd2a80cdd00 con 0x7fd29406be30 2024-05-09T06:27:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 msgr2=0x7fd29406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 0x7fd29406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2a806db20 tx=0x7fd29c009040 comp rx=0 tx=0).stop 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 msgr2=0x7fd2a80a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c430 tx=0x7fd29800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 shutdown_connections 2024-05-09T06:27:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2a81f7460 0x7fd2a806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2a8161950 0x7fd2a81f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 --2- 172.21.15.1:0/2582471080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80d2b10 0x7fd2a80a1500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 >> 172.21.15.1:0/2582471080 conn(0x7fd2a80d3e20 msgr2=0x7fd2a80b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.226+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 shutdown_connections 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.227+0000 7fd2ade9a640 1 -- 172.21.15.1:0/2582471080 wait complete. 2024-05-09T06:27:49.239 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-05-09T06:27:49.074539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002924"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795027+0000","last_change":"2024-05-09T06:27:20.644248+0000","last_active":"2024-05-09T06:27:28.795027+0000","last_peered":"2024-05-09T06:27:28.795027+0000","last_clean":"2024-05-09T06:27:28.795027+0000","last_became_active":"2024-05-09T06:27:20.643791+0000","last_became_peered":"2024-05-09T06:27:20.643791+0000","last_unstale":"2024-05-09T06:27:28.795027+0000","last_undegraded":"2024-05-09T06:27:28.795027+0000","last_fullsized":"2024-05-09T06:27:28.795027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:48.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795308+0000","last_change":"2024-05-09T06:27:21.649501+0000","last_active":"2024-05-09T06:27:28.795308+0000","last_peered":"2024-05-09T06:27:28.795308+0000","last_clean":"2024-05-09T06:27:28.795308+0000","last_became_active":"2024-05-09T06:27:21.649193+0000","last_became_peered":"2024-05-09T06:27:21.649193+0000","last_unstale":"2024-05-09T06:27:28.795308+0000","last_undegraded":"2024-05-09T06:27:28.795308+0000","last_fullsized":"2024-05-09T06:27:28.795308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794757+0000","last_change":"2024-05-09T06:27:25.871323+0000","last_active":"2024-05-09T06:27:28.794757+0000","last_peered":"2024-05-09T06:27:28.794757+0000","last_clean":"2024-05-09T06:27:28.794757+0000","last_became_active":"2024-05-09T06:27:25.870947+0000","last_became_peered":"2024-05-09T06:27:25.870947+0000","last_unstale":"2024-05-09T06:27:28.794757+0000","last_undegraded":"2024-05-09T06:27:28.794757+0000","last_fullsized":"2024-05-09T06:27:28.794757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795538+0000","last_change":"2024-05-09T06:27:20.634799+0000","last_active":"2024-05-09T06:27:28.795538+0000","last_peered":"2024-05-09T06:27:28.795538+0000","last_clean":"2024-05-09T06:27:28.795538+0000","last_became_active":"2024-05-09T06:27:20.634300+0000","last_became_peered":"2024-05-09T06:27:20.634300+0000","last_unstale":"2024-05-09T06:27:28.795538+0000","last_undegraded":"2024-05-09T06:27:28.795538+0000","last_fullsized":"2024-05-09T06:27:28.795538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796489+0000","last_change":"2024-05-09T06:27:21.638390+0000","last_active":"2024-05-09T06:27:28.796489+0000","last_peered":"2024-05-09T06:27:28.796489+0000","last_clean":"2024-05-09T06:27:28.796489+0000","last_became_active":"2024-05-09T06:27:21.637720+0000","last_became_peered":"2024-05-09T06:27:21.637720+0000","last_unstale":"2024-05-09T06:27:28.796489+0000","last_undegraded":"2024-05-09T06:27:28.796489+0000","last_fullsized":"2024-05-09T06:27:28.796489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666410+0000","last_change":"2024-05-09T06:27:25.668634+0000","last_active":"2024-05-09T06:27:26.666410+0000","last_peered":"2024-05-09T06:27:26.666410+0000","last_clean":"2024-05-09T06:27:26.666410+0000","last_became_active":"2024-05-09T06:27:25.668153+0000","last_became_peered":"2024-05-09T06:27:25.668153+0000","last_unstale":"2024-05-09T06:27:26.666410+0000","last_undegraded":"2024-05-09T06:27:26.666410+0000","last_fullsized":"2024-05-09T06:27:26.666410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:46.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310311+0000","last_change":"2024-05-09T06:27:20.646888+0000","last_active":"2024-05-09T06:27:28.310311+0000","last_peered":"2024-05-09T06:27:28.310311+0000","last_clean":"2024-05-09T06:27:28.310311+0000","last_became_active":"2024-05-09T06:27:20.646603+0000","last_became_peered":"2024-05-09T06:27:20.646603+0000","last_unstale":"2024-05-09T06:27:28.310311+0000","last_undegraded":"2024-05-09T06:27:28.310311+0000","last_fullsized":"2024-05-09T06:27:28.310311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:06.656512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796678+0000","last_change":"2024-05-09T06:27:21.659688+0000","last_active":"2024-05-09T06:27:28.796678+0000","last_peered":"2024-05-09T06:27:28.796678+0000","last_clean":"2024-05-09T06:27:28.796678+0000","last_became_active":"2024-05-09T06:27:21.659407+0000","last_became_peered":"2024-05-09T06:27:21.659407+0000","last_unstale":"2024-05-09T06:27:28.796678+0000","last_undegraded":"2024-05-09T06:27:28.796678+0000","last_fullsized":"2024-05-09T06:27:28.796678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667001+0000","last_change":"2024-05-09T06:27:25.674815+0000","last_active":"2024-05-09T06:27:26.667001+0000","last_peered":"2024-05-09T06:27:26.667001+0000","last_clean":"2024-05-09T06:27:26.667001+0000","last_became_active":"2024-05-09T06:27:25.673855+0000","last_became_peered":"2024-05-09T06:27:25.673855+0000","last_unstale":"2024-05-09T06:27:26.667001+0000","last_undegraded":"2024-05-09T06:27:26.667001+0000","last_fullsized":"2024-05-09T06:27:26.667001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:16.160631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309929+0000","last_change":"2024-05-09T06:27:20.644933+0000","last_active":"2024-05-09T06:27:28.309929+0000","last_peered":"2024-05-09T06:27:28.309929+0000","last_clean":"2024-05-09T06:27:28.309929+0000","last_became_active":"2024-05-09T06:27:20.644019+0000","last_became_peered":"2024-05-09T06:27:20.644019+0000","last_unstale":"2024-05-09T06:27:28.309929+0000","last_undegraded":"2024-05-09T06:27:28.309929+0000","last_fullsized":"2024-05-09T06:27:28.309929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796776+0000","last_change":"2024-05-09T06:27:21.660829+0000","last_active":"2024-05-09T06:27:28.796776+0000","last_peered":"2024-05-09T06:27:28.796776+0000","last_clean":"2024-05-09T06:27:28.796776+0000","last_became_active":"2024-05-09T06:27:21.658349+0000","last_became_peered":"2024-05-09T06:27:21.658349+0000","last_unstale":"2024-05-09T06:27:28.796776+0000","last_undegraded":"2024-05-09T06:27:28.796776+0000","last_fullsized":"2024-05-09T06:27:28.796776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666466+0000","last_change":"2024-05-09T06:27:25.664257+0000","last_active":"2024-05-09T06:27:26.666466+0000","last_peered":"2024-05-09T06:27:26.666466+0000","last_clean":"2024-05-09T06:27:26.666466+0000","last_became_active":"2024-05-09T06:27:25.663787+0000","last_became_peered":"2024-05-09T06:27:25.663787+0000","last_unstale":"2024-05-09T06:27:26.666466+0000","last_undegraded":"2024-05-09T06:27:26.666466+0000","last_fullsized":"2024-05-09T06:27:26.666466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:57.007200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795391+0000","last_change":"2024-05-09T06:27:21.652107+0000","last_active":"2024-05-09T06:27:28.795391+0000","last_peered":"2024-05-09T06:27:28.795391+0000","last_clean":"2024-05-09T06:27:28.795391+0000","last_became_active":"2024-05-09T06:27:21.651764+0000","last_became_peered":"2024-05-09T06:27:21.651764+0000","last_unstale":"2024-05-09T06:27:28.795391+0000","last_undegraded":"2024-05-09T06:27:28.795391+0000","last_fullsized":"2024-05-09T06:27:28.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310074+0000","last_change":"2024-05-09T06:27:20.644636+0000","last_active":"2024-05-09T06:27:28.310074+0000","last_peered":"2024-05-09T06:27:28.310074+0000","last_clean":"2024-05-09T06:27:28.310074+0000","last_became_active":"2024-05-09T06:27:20.644155+0000","last_became_peered":"2024-05-09T06:27:20.644155+0000","last_unstale":"2024-05-09T06:27:28.310074+0000","last_undegraded":"2024-05-09T06:27:28.310074+0000","last_fullsized":"2024-05-09T06:27:28.310074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667637+0000","last_change":"2024-05-09T06:27:25.674517+0000","last_active":"2024-05-09T06:27:26.667637+0000","last_peered":"2024-05-09T06:27:26.667637+0000","last_clean":"2024-05-09T06:27:26.667637+0000","last_became_active":"2024-05-09T06:27:25.673256+0000","last_became_peered":"2024-05-09T06:27:25.673256+0000","last_unstale":"2024-05-09T06:27:26.667637+0000","last_undegraded":"2024-05-09T06:27:26.667637+0000","last_fullsized":"2024-05-09T06:27:26.667637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795461+0000","last_change":"2024-05-09T06:27:21.650596+0000","last_active":"2024-05-09T06:27:28.795461+0000","last_peered":"2024-05-09T06:27:28.795461+0000","last_clean":"2024-05-09T06:27:28.795461+0000","last_became_active":"2024-05-09T06:27:21.649943+0000","last_became_peered":"2024-05-09T06:27:21.649943+0000","last_unstale":"2024-05-09T06:27:28.795461+0000","last_undegraded":"2024-05-09T06:27:28.795461+0000","last_fullsized":"2024-05-09T06:27:28.795461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897815+0000","last_change":"2024-05-09T06:27:20.634659+0000","last_active":"2024-05-09T06:27:26.897815+0000","last_peered":"2024-05-09T06:27:26.897815+0000","last_clean":"2024-05-09T06:27:26.897815+0000","last_became_active":"2024-05-09T06:27:20.634344+0000","last_became_peered":"2024-05-09T06:27:20.634344+0000","last_unstale":"2024-05-09T06:27:26.897815+0000","last_undegraded":"2024-05-09T06:27:26.897815+0000","last_fullsized":"2024-05-09T06:27:26.897815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667419+0000","last_change":"2024-05-09T06:27:25.674355+0000","last_active":"2024-05-09T06:27:26.667419+0000","last_peered":"2024-05-09T06:27:26.667419+0000","last_clean":"2024-05-09T06:27:26.667419+0000","last_became_active":"2024-05-09T06:27:25.673950+0000","last_became_peered":"2024-05-09T06:27:25.673950+0000","last_unstale":"2024-05-09T06:27:26.667419+0000","last_undegraded":"2024-05-09T06:27:26.667419+0000","last_fullsized":"2024-05-09T06:27:26.667419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.231985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796292+0000","last_change":"2024-05-09T06:27:21.641405+0000","last_active":"2024-05-09T06:27:28.796292+0000","last_peered":"2024-05-09T06:27:28.796292+0000","last_clean":"2024-05-09T06:27:28.796292+0000","last_became_active":"2024-05-09T06:27:21.641019+0000","last_became_peered":"2024-05-09T06:27:21.641019+0000","last_unstale":"2024-05-09T06:27:28.796292+0000","last_undegraded":"2024-05-09T06:27:28.796292+0000","last_fullsized":"2024-05-09T06:27:28.796292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897255+0000","last_change":"2024-05-09T06:27:20.644807+0000","last_active":"2024-05-09T06:27:26.897255+0000","last_peered":"2024-05-09T06:27:26.897255+0000","last_clean":"2024-05-09T06:27:26.897255+0000","last_became_active":"2024-05-09T06:27:20.644510+0000","last_became_peered":"2024-05-09T06:27:20.644510+0000","last_unstale":"2024-05-09T06:27:26.897255+0000","last_undegraded":"2024-05-09T06:27:26.897255+0000","last_fullsized":"2024-05-09T06:27:26.897255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897895+0000","last_change":"2024-05-09T06:27:25.867962+0000","last_active":"2024-05-09T06:27:26.897895+0000","last_peered":"2024-05-09T06:27:26.897895+0000","last_clean":"2024-05-09T06:27:26.897895+0000","last_became_active":"2024-05-09T06:27:25.867593+0000","last_became_peered":"2024-05-09T06:27:25.867593+0000","last_unstale":"2024-05-09T06:27:26.897895+0000","last_undegraded":"2024-05-09T06:27:26.897895+0000","last_fullsized":"2024-05-09T06:27:26.897895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666600+0000","last_change":"2024-05-09T06:27:21.651627+0000","last_active":"2024-05-09T06:27:26.666600+0000","last_peered":"2024-05-09T06:27:26.666600+0000","last_clean":"2024-05-09T06:27:26.666600+0000","last_became_active":"2024-05-09T06:27:21.651021+0000","last_became_peered":"2024-05-09T06:27:21.651021+0000","last_unstale":"2024-05-09T06:27:26.666600+0000","last_undegraded":"2024-05-09T06:27:26.666600+0000","last_fullsized":"2024-05-09T06:27:26.666600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:04.963108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309528+0000","last_change":"2024-05-09T06:27:20.644691+0000","last_active":"2024-05-09T06:27:28.309528+0000","last_peered":"2024-05-09T06:27:28.309528+0000","last_clean":"2024-05-09T06:27:28.309528+0000","last_became_active":"2024-05-09T06:27:20.644291+0000","last_became_peered":"2024-05-09T06:27:20.644291+0000","last_unstale":"2024-05-09T06:27:28.309528+0000","last_undegraded":"2024-05-09T06:27:28.309528+0000","last_fullsized":"2024-05-09T06:27:28.309528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667629+0000","last_change":"2024-05-09T06:27:25.670582+0000","last_active":"2024-05-09T06:27:26.667629+0000","last_peered":"2024-05-09T06:27:26.667629+0000","last_clean":"2024-05-09T06:27:26.667629+0000","last_became_active":"2024-05-09T06:27:25.670180+0000","last_became_peered":"2024-05-09T06:27:25.670180+0000","last_unstale":"2024-05-09T06:27:26.667629+0000","last_undegraded":"2024-05-09T06:27:26.667629+0000","last_fullsized":"2024-05-09T06:27:26.667629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796762+0000","last_change":"2024-05-09T06:27:21.644419+0000","last_active":"2024-05-09T06:27:28.796762+0000","last_peered":"2024-05-09T06:27:28.796762+0000","last_clean":"2024-05-09T06:27:28.796762+0000","last_became_active":"2024-05-09T06:27:21.644054+0000","last_became_peered":"2024-05-09T06:27:21.644054+0000","last_unstale":"2024-05-09T06:27:28.796762+0000","last_undegraded":"2024-05-09T06:27:28.796762+0000","last_fullsized":"2024-05-09T06:27:28.796762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795166+0000","last_change":"2024-05-09T06:27:20.644413+0000","last_active":"2024-05-09T06:27:28.795166+0000","last_peered":"2024-05-09T06:27:28.795166+0000","last_clean":"2024-05-09T06:27:28.795166+0000","last_became_active":"2024-05-09T06:27:20.644063+0000","last_became_peered":"2024-05-09T06:27:20.644063+0000","last_unstale":"2024-05-09T06:27:28.795166+0000","last_undegraded":"2024-05-09T06:27:28.795166+0000","last_fullsized":"2024-05-09T06:27:28.795166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666413+0000","last_change":"2024-05-09T06:27:25.868219+0000","last_active":"2024-05-09T06:27:26.666413+0000","last_peered":"2024-05-09T06:27:26.666413+0000","last_clean":"2024-05-09T06:27:26.666413+0000","last_became_active":"2024-05-09T06:27:25.867618+0000","last_became_peered":"2024-05-09T06:27:25.867618+0000","last_unstale":"2024-05-09T06:27:26.666413+0000","last_undegraded":"2024-05-09T06:27:26.666413+0000","last_fullsized":"2024-05-09T06:27:26.666413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:59.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795981+0000","last_change":"2024-05-09T06:27:21.659207+0000","last_active":"2024-05-09T06:27:28.795981+0000","last_peered":"2024-05-09T06:27:28.795981+0000","last_clean":"2024-05-09T06:27:28.795981+0000","last_became_active":"2024-05-09T06:27:21.651367+0000","last_became_peered":"2024-05-09T06:27:21.651367+0000","last_unstale":"2024-05-09T06:27:28.795981+0000","last_undegraded":"2024-05-09T06:27:28.795981+0000","last_fullsized":"2024-05-09T06:27:28.795981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897683+0000","last_change":"2024-05-09T06:27:20.637053+0000","last_active":"2024-05-09T06:27:26.897683+0000","last_peered":"2024-05-09T06:27:26.897683+0000","last_clean":"2024-05-09T06:27:26.897683+0000","last_became_active":"2024-05-09T06:27:20.636755+0000","last_became_peered":"2024-05-09T06:27:20.636755+0000","last_unstale":"2024-05-09T06:27:26.897683+0000","last_undegraded":"2024-05-09T06:27:26.897683+0000","last_fullsized":"2024-05-09T06:27:26.897683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795768+0000","last_change":"2024-05-09T06:27:25.871255+0000","last_active":"2024-05-09T06:27:28.795768+0000","last_peered":"2024-05-09T06:27:28.795768+0000","last_clean":"2024-05-09T06:27:28.795768+0000","last_became_active":"2024-05-09T06:27:25.870862+0000","last_became_peered":"2024-05-09T06:27:25.870862+0000","last_unstale":"2024-05-09T06:27:28.795768+0000","last_undegraded":"2024-05-09T06:27:28.795768+0000","last_fullsized":"2024-05-09T06:27:28.795768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:53.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897799+0000","last_change":"2024-05-09T06:27:21.649868+0000","last_active":"2024-05-09T06:27:26.897799+0000","last_peered":"2024-05-09T06:27:26.897799+0000","last_clean":"2024-05-09T06:27:26.897799+0000","last_became_active":"2024-05-09T06:27:21.649515+0000","last_became_peered":"2024-05-09T06:27:21.649515+0000","last_unstale":"2024-05-09T06:27:26.897799+0000","last_undegraded":"2024-05-09T06:27:26.897799+0000","last_fullsized":"2024-05-09T06:27:26.897799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310415+0000","last_change":"2024-05-09T06:27:20.644414+0000","last_active":"2024-05-09T06:27:28.310415+0000","last_peered":"2024-05-09T06:27:28.310415+0000","last_clean":"2024-05-09T06:27:28.310415+0000","last_became_active":"2024-05-09T06:27:20.644077+0000","last_became_peered":"2024-05-09T06:27:20.644077+0000","last_unstale":"2024-05-09T06:27:28.310415+0000","last_undegraded":"2024-05-09T06:27:28.310415+0000","last_fullsized":"2024-05-09T06:27:28.310415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:34.718775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795535+0000","last_change":"2024-05-09T06:27:25.867794+0000","last_active":"2024-05-09T06:27:28.795535+0000","last_peered":"2024-05-09T06:27:28.795535+0000","last_clean":"2024-05-09T06:27:28.795535+0000","last_became_active":"2024-05-09T06:27:25.867351+0000","last_became_peered":"2024-05-09T06:27:25.867351+0000","last_unstale":"2024-05-09T06:27:28.795535+0000","last_undegraded":"2024-05-09T06:27:28.795535+0000","last_fullsized":"2024-05-09T06:27:28.795535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796829+0000","last_change":"2024-05-09T06:27:21.662543+0000","last_active":"2024-05-09T06:27:28.796829+0000","last_peered":"2024-05-09T06:27:28.796829+0000","last_clean":"2024-05-09T06:27:28.796829+0000","last_became_active":"2024-05-09T06:27:21.662266+0000","last_became_peered":"2024-05-09T06:27:21.662266+0000","last_unstale":"2024-05-09T06:27:28.796829+0000","last_undegraded":"2024-05-09T06:27:28.796829+0000","last_fullsized":"2024-05-09T06:27:28.796829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897904+0000","last_change":"2024-05-09T06:27:20.634145+0000","last_active":"2024-05-09T06:27:26.897904+0000","last_peered":"2024-05-09T06:27:26.897904+0000","last_clean":"2024-05-09T06:27:26.897904+0000","last_became_active":"2024-05-09T06:27:20.633864+0000","last_became_peered":"2024-05-09T06:27:20.633864+0000","last_unstale":"2024-05-09T06:27:26.897904+0000","last_undegraded":"2024-05-09T06:27:26.897904+0000","last_fullsized":"2024-05-09T06:27:26.897904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897405+0000","last_change":"2024-05-09T06:27:25.672524+0000","last_active":"2024-05-09T06:27:26.897405+0000","last_peered":"2024-05-09T06:27:26.897405+0000","last_clean":"2024-05-09T06:27:26.897405+0000","last_became_active":"2024-05-09T06:27:25.672004+0000","last_became_peered":"2024-05-09T06:27:25.672004+0000","last_unstale":"2024-05-09T06:27:26.897405+0000","last_undegraded":"2024-05-09T06:27:26.897405+0000","last_fullsized":"2024-05-09T06:27:26.897405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667430+0000","last_change":"2024-05-09T06:27:21.634887+0000","last_active":"2024-05-09T06:27:26.667430+0000","last_peered":"2024-05-09T06:27:26.667430+0000","last_clean":"2024-05-09T06:27:26.667430+0000","last_became_active":"2024-05-09T06:27:21.634609+0000","last_became_peered":"2024-05-09T06:27:21.634609+0000","last_unstale":"2024-05-09T06:27:26.667430+0000","last_undegraded":"2024-05-09T06:27:26.667430+0000","last_fullsized":"2024-05-09T06:27:26.667430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:59.874418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795267+0000","last_change":"2024-05-09T06:27:20.642066+0000","last_active":"2024-05-09T06:27:28.795267+0000","last_peered":"2024-05-09T06:27:28.795267+0000","last_clean":"2024-05-09T06:27:28.795267+0000","last_became_active":"2024-05-09T06:27:20.641753+0000","last_became_peered":"2024-05-09T06:27:20.641753+0000","last_unstale":"2024-05-09T06:27:28.795267+0000","last_undegraded":"2024-05-09T06:27:28.795267+0000","last_fullsized":"2024-05-09T06:27:28.795267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897884+0000","last_change":"2024-05-09T06:27:25.671230+0000","last_active":"2024-05-09T06:27:26.897884+0000","last_peered":"2024-05-09T06:27:26.897884+0000","last_clean":"2024-05-09T06:27:26.897884+0000","last_became_active":"2024-05-09T06:27:25.670778+0000","last_became_peered":"2024-05-09T06:27:25.670778+0000","last_unstale":"2024-05-09T06:27:26.897884+0000","last_undegraded":"2024-05-09T06:27:26.897884+0000","last_fullsized":"2024-05-09T06:27:26.897884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:20:39.476936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897252+0000","last_change":"2024-05-09T06:27:21.649626+0000","last_active":"2024-05-09T06:27:26.897252+0000","last_peered":"2024-05-09T06:27:26.897252+0000","last_clean":"2024-05-09T06:27:26.897252+0000","last_became_active":"2024-05-09T06:27:21.649332+0000","last_became_peered":"2024-05-09T06:27:21.649332+0000","last_unstale":"2024-05-09T06:27:26.897252+0000","last_undegraded":"2024-05-09T06:27:26.897252+0000","last_fullsized":"2024-05-09T06:27:26.897252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665959+0000","last_change":"2024-05-09T06:27:20.632572+0000","last_active":"2024-05-09T06:27:26.665959+0000","last_peered":"2024-05-09T06:27:26.665959+0000","last_clean":"2024-05-09T06:27:26.665959+0000","last_became_active":"2024-05-09T06:27:20.630286+0000","last_became_peered":"2024-05-09T06:27:20.630286+0000","last_unstale":"2024-05-09T06:27:26.665959+0000","last_undegraded":"2024-05-09T06:27:26.665959+0000","last_fullsized":"2024-05-09T06:27:26.665959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897808+0000","last_change":"2024-05-09T06:27:25.678228+0000","last_active":"2024-05-09T06:27:26.897808+0000","last_peered":"2024-05-09T06:27:26.897808+0000","last_clean":"2024-05-09T06:27:26.897808+0000","last_became_active":"2024-05-09T06:27:25.677638+0000","last_became_peered":"2024-05-09T06:27:25.677638+0000","last_unstale":"2024-05-09T06:27:26.897808+0000","last_undegraded":"2024-05-09T06:27:26.897808+0000","last_fullsized":"2024-05-09T06:27:26.897808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794917+0000","last_change":"2024-05-09T06:27:21.634999+0000","last_active":"2024-05-09T06:27:28.794917+0000","last_peered":"2024-05-09T06:27:28.794917+0000","last_clean":"2024-05-09T06:27:28.794917+0000","last_became_active":"2024-05-09T06:27:21.634688+0000","last_became_peered":"2024-05-09T06:27:21.634688+0000","last_unstale":"2024-05-09T06:27:28.794917+0000","last_undegraded":"2024-05-09T06:27:28.794917+0000","last_fullsized":"2024-05-09T06:27:28.794917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309529+0000","last_change":"2024-05-09T06:27:20.645301+0000","last_active":"2024-05-09T06:27:28.309529+0000","last_peered":"2024-05-09T06:27:28.309529+0000","last_clean":"2024-05-09T06:27:28.309529+0000","last_became_active":"2024-05-09T06:27:20.644309+0000","last_became_peered":"2024-05-09T06:27:20.644309+0000","last_unstale":"2024-05-09T06:27:28.309529+0000","last_undegraded":"2024-05-09T06:27:28.309529+0000","last_fullsized":"2024-05-09T06:27:28.309529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796649+0000","last_change":"2024-05-09T06:27:25.870805+0000","last_active":"2024-05-09T06:27:28.796649+0000","last_peered":"2024-05-09T06:27:28.796649+0000","last_clean":"2024-05-09T06:27:28.796649+0000","last_became_active":"2024-05-09T06:27:25.870405+0000","last_became_peered":"2024-05-09T06:27:25.870405+0000","last_unstale":"2024-05-09T06:27:28.796649+0000","last_undegraded":"2024-05-09T06:27:28.796649+0000","last_fullsized":"2024-05-09T06:27:28.796649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795597+0000","last_change":"2024-05-09T06:27:21.660077+0000","last_active":"2024-05-09T06:27:28.795597+0000","last_peered":"2024-05-09T06:27:28.795597+0000","last_clean":"2024-05-09T06:27:28.795597+0000","last_became_active":"2024-05-09T06:27:21.659816+0000","last_became_peered":"2024-05-09T06:27:21.659816+0000","last_unstale":"2024-05-09T06:27:28.795597+0000","last_undegraded":"2024-05-09T06:27:28.795597+0000","last_fullsized":"2024-05-09T06:27:28.795597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794955+0000","last_change":"2024-05-09T06:27:20.638709+0000","last_active":"2024-05-09T06:27:28.794955+0000","last_peered":"2024-05-09T06:27:28.794955+0000","last_clean":"2024-05-09T06:27:28.794955+0000","last_became_active":"2024-05-09T06:27:20.638404+0000","last_became_peered":"2024-05-09T06:27:20.638404+0000","last_unstale":"2024-05-09T06:27:28.794955+0000","last_undegraded":"2024-05-09T06:27:28.794955+0000","last_fullsized":"2024-05-09T06:27:28.794955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309694+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:27:28.309694+0000","last_peered":"2024-05-09T06:27:28.309694+0000","last_clean":"2024-05-09T06:27:28.309694+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:27:28.309694+0000","last_undegraded":"2024-05-09T06:27:28.309694+0000","last_fullsized":"2024-05-09T06:27:28.309694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795720+0000","last_change":"2024-05-09T06:27:25.871420+0000","last_active":"2024-05-09T06:27:28.795720+0000","last_peered":"2024-05-09T06:27:28.795720+0000","last_clean":"2024-05-09T06:27:28.795720+0000","last_became_active":"2024-05-09T06:27:25.870765+0000","last_became_peered":"2024-05-09T06:27:25.870765+0000","last_unstale":"2024-05-09T06:27:28.795720+0000","last_undegraded":"2024-05-09T06:27:28.795720+0000","last_fullsized":"2024-05-09T06:27:28.795720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310010+0000","last_change":"2024-05-09T06:27:21.665731+0000","last_active":"2024-05-09T06:27:28.310010+0000","last_peered":"2024-05-09T06:27:28.310010+0000","last_clean":"2024-05-09T06:27:28.310010+0000","last_became_active":"2024-05-09T06:27:21.665457+0000","last_became_peered":"2024-05-09T06:27:21.665457+0000","last_unstale":"2024-05-09T06:27:28.310010+0000","last_undegraded":"2024-05-09T06:27:28.310010+0000","last_fullsized":"2024-05-09T06:27:28.310010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666314+0000","last_change":"2024-05-09T06:27:20.639195+0000","last_active":"2024-05-09T06:27:26.666314+0000","last_peered":"2024-05-09T06:27:26.666314+0000","last_clean":"2024-05-09T06:27:26.666314+0000","last_became_active":"2024-05-09T06:27:20.638833+0000","last_became_peered":"2024-05-09T06:27:20.638833+0000","last_unstale":"2024-05-09T06:27:26.666314+0000","last_undegraded":"2024-05-09T06:27:26.666314+0000","last_fullsized":"2024-05-09T06:27:26.666314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795939+0000","last_change":"2024-05-09T06:27:25.868481+0000","last_active":"2024-05-09T06:27:28.795939+0000","last_peered":"2024-05-09T06:27:28.795939+0000","last_clean":"2024-05-09T06:27:28.795939+0000","last_became_active":"2024-05-09T06:27:25.867521+0000","last_became_peered":"2024-05-09T06:27:25.867521+0000","last_unstale":"2024-05-09T06:27:28.795939+0000","last_undegraded":"2024-05-09T06:27:28.795939+0000","last_fullsized":"2024-05-09T06:27:28.795939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796554+0000","last_change":"2024-05-09T06:27:21.663211+0000","last_active":"2024-05-09T06:27:28.796554+0000","last_peered":"2024-05-09T06:27:28.796554+0000","last_clean":"2024-05-09T06:27:28.796554+0000","last_became_active":"2024-05-09T06:27:21.662892+0000","last_became_peered":"2024-05-09T06:27:21.662892+0000","last_unstale":"2024-05-09T06:27:28.796554+0000","last_undegraded":"2024-05-09T06:27:28.796554+0000","last_fullsized":"2024-05-09T06:27:28.796554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897750+0000","last_change":"2024-05-09T06:27:20.638780+0000","last_active":"2024-05-09T06:27:26.897750+0000","last_peered":"2024-05-09T06:27:26.897750+0000","last_clean":"2024-05-09T06:27:26.897750+0000","last_became_active":"2024-05-09T06:27:20.638469+0000","last_became_peered":"2024-05-09T06:27:20.638469+0000","last_unstale":"2024-05-09T06:27:26.897750+0000","last_undegraded":"2024-05-09T06:27:26.897750+0000","last_fullsized":"2024-05-09T06:27:26.897750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795796+0000","last_change":"2024-05-09T06:27:25.673712+0000","last_active":"2024-05-09T06:27:28.795796+0000","last_peered":"2024-05-09T06:27:28.795796+0000","last_clean":"2024-05-09T06:27:28.795796+0000","last_became_active":"2024-05-09T06:27:25.673062+0000","last_became_peered":"2024-05-09T06:27:25.673062+0000","last_unstale":"2024-05-09T06:27:28.795796+0000","last_undegraded":"2024-05-09T06:27:28.795796+0000","last_fullsized":"2024-05-09T06:27:28.795796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310517+0000","last_change":"2024-05-09T06:27:21.652689+0000","last_active":"2024-05-09T06:27:28.310517+0000","last_peered":"2024-05-09T06:27:28.310517+0000","last_clean":"2024-05-09T06:27:28.310517+0000","last_became_active":"2024-05-09T06:27:21.652309+0000","last_became_peered":"2024-05-09T06:27:21.652309+0000","last_unstale":"2024-05-09T06:27:28.310517+0000","last_undegraded":"2024-05-09T06:27:28.310517+0000","last_fullsized":"2024-05-09T06:27:28.310517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:30.619441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666511+0000","last_change":"2024-05-09T06:27:20.631329+0000","last_active":"2024-05-09T06:27:26.666511+0000","last_peered":"2024-05-09T06:27:26.666511+0000","last_clean":"2024-05-09T06:27:26.666511+0000","last_became_active":"2024-05-09T06:27:20.631027+0000","last_became_peered":"2024-05-09T06:27:20.631027+0000","last_unstale":"2024-05-09T06:27:26.666511+0000","last_undegraded":"2024-05-09T06:27:26.666511+0000","last_fullsized":"2024-05-09T06:27:26.666511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795581+0000","last_change":"2024-05-09T06:27:25.868448+0000","last_active":"2024-05-09T06:27:28.795581+0000","last_peered":"2024-05-09T06:27:28.795581+0000","last_clean":"2024-05-09T06:27:28.795581+0000","last_became_active":"2024-05-09T06:27:25.867998+0000","last_became_peered":"2024-05-09T06:27:25.867998+0000","last_unstale":"2024-05-09T06:27:28.795581+0000","last_undegraded":"2024-05-09T06:27:28.795581+0000","last_fullsized":"2024-05-09T06:27:28.795581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310235+0000","last_change":"2024-05-09T06:27:21.654321+0000","last_active":"2024-05-09T06:27:28.310235+0000","last_peered":"2024-05-09T06:27:28.310235+0000","last_clean":"2024-05-09T06:27:28.310235+0000","last_became_active":"2024-05-09T06:27:21.654037+0000","last_became_peered":"2024-05-09T06:27:21.654037+0000","last_unstale":"2024-05-09T06:27:28.310235+0000","last_undegraded":"2024-05-09T06:27:28.310235+0000","last_fullsized":"2024-05-09T06:27:28.310235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667634+0000","last_change":"2024-05-09T06:27:20.634285+0000","last_active":"2024-05-09T06:27:26.667634+0000","last_peered":"2024-05-09T06:27:26.667634+0000","last_clean":"2024-05-09T06:27:26.667634+0000","last_became_active":"2024-05-09T06:27:20.633961+0000","last_became_peered":"2024-05-09T06:27:20.633961+0000","last_unstale":"2024-05-09T06:27:26.667634+0000","last_undegraded":"2024-05-09T06:27:26.667634+0000","last_fullsized":"2024-05-09T06:27:26.667634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:22.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666362+0000","last_change":"2024-05-09T06:27:25.666556+0000","last_active":"2024-05-09T06:27:26.666362+0000","last_peered":"2024-05-09T06:27:26.666362+0000","last_clean":"2024-05-09T06:27:26.666362+0000","last_became_active":"2024-05-09T06:27:25.666086+0000","last_became_peered":"2024-05-09T06:27:25.666086+0000","last_unstale":"2024-05-09T06:27:26.666362+0000","last_undegraded":"2024-05-09T06:27:26.666362+0000","last_fullsized":"2024-05-09T06:27:26.666362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:16.228053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309897+0000","last_change":"2024-05-09T06:27:21.653860+0000","last_active":"2024-05-09T06:27:28.309897+0000","last_peered":"2024-05-09T06:27:28.309897+0000","last_clean":"2024-05-09T06:27:28.309897+0000","last_became_active":"2024-05-09T06:27:21.653222+0000","last_became_peered":"2024-05-09T06:27:21.653222+0000","last_unstale":"2024-05-09T06:27:28.309897+0000","last_undegraded":"2024-05-09T06:27:28.309897+0000","last_fullsized":"2024-05-09T06:27:28.309897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665751+0000","last_change":"2024-05-09T06:27:20.632271+0000","last_active":"2024-05-09T06:27:26.665751+0000","last_peered":"2024-05-09T06:27:26.665751+0000","last_clean":"2024-05-09T06:27:26.665751+0000","last_became_active":"2024-05-09T06:27:20.631965+0000","last_became_peered":"2024-05-09T06:27:20.631965+0000","last_unstale":"2024-05-09T06:27:26.665751+0000","last_undegraded":"2024-05-09T06:27:26.665751+0000","last_fullsized":"2024-05-09T06:27:26.665751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796544+0000","last_change":"2024-05-09T06:27:25.673394+0000","last_active":"2024-05-09T06:27:28.796544+0000","last_peered":"2024-05-09T06:27:28.796544+0000","last_clean":"2024-05-09T06:27:28.796544+0000","last_became_active":"2024-05-09T06:27:25.672824+0000","last_became_peered":"2024-05-09T06:27:25.672824+0000","last_unstale":"2024-05-09T06:27:28.796544+0000","last_undegraded":"2024-05-09T06:27:28.796544+0000","last_fullsized":"2024-05-09T06:27:28.796544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795660+0000","last_change":"2024-05-09T06:27:21.658568+0000","last_active":"2024-05-09T06:27:28.795660+0000","last_peered":"2024-05-09T06:27:28.795660+0000","last_clean":"2024-05-09T06:27:28.795660+0000","last_became_active":"2024-05-09T06:27:21.658289+0000","last_became_peered":"2024-05-09T06:27:21.658289+0000","last_unstale":"2024-05-09T06:27:28.795660+0000","last_undegraded":"2024-05-09T06:27:28.795660+0000","last_fullsized":"2024-05-09T06:27:28.795660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.637847+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.637553+0000","last_became_peered":"2024-05-09T06:27:20.637553+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795481+0000","last_change":"2024-05-09T06:27:25.872027+0000","last_active":"2024-05-09T06:27:28.795481+0000","last_peered":"2024-05-09T06:27:28.795481+0000","last_clean":"2024-05-09T06:27:28.795481+0000","last_became_active":"2024-05-09T06:27:25.871616+0000","last_became_peered":"2024-05-09T06:27:25.871616+0000","last_unstale":"2024-05-09T06:27:28.795481+0000","last_undegraded":"2024-05-09T06:27:28.795481+0000","last_fullsized":"2024-05-09T06:27:28.795481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:58:18.735834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309942+0000","last_change":"2024-05-09T06:27:21.655596+0000","last_active":"2024-05-09T06:27:28.309942+0000","last_peered":"2024-05-09T06:27:28.309942+0000","last_clean":"2024-05-09T06:27:28.309942+0000","last_became_active":"2024-05-09T06:27:21.655327+0000","last_became_peered":"2024-05-09T06:27:21.655327+0000","last_unstale":"2024-05-09T06:27:28.309942+0000","last_undegraded":"2024-05-09T06:27:28.309942+0000","last_fullsized":"2024-05-09T06:27:28.309942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796614+0000","last_change":"2024-05-09T06:27:20.640461+0000","last_active":"2024-05-09T06:27:28.796614+0000","last_peered":"2024-05-09T06:27:28.796614+0000","last_clean":"2024-05-09T06:27:28.796614+0000","last_became_active":"2024-05-09T06:27:20.640164+0000","last_became_peered":"2024-05-09T06:27:20.640164+0000","last_unstale":"2024-05-09T06:27:28.796614+0000","last_undegraded":"2024-05-09T06:27:28.796614+0000","last_fullsized":"2024-05-09T06:27:28.796614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795963+0000","last_change":"2024-05-09T06:27:25.868054+0000","last_active":"2024-05-09T06:27:28.795963+0000","last_peered":"2024-05-09T06:27:28.795963+0000","last_clean":"2024-05-09T06:27:28.795963+0000","last_became_active":"2024-05-09T06:27:25.867638+0000","last_became_peered":"2024-05-09T06:27:25.867638+0000","last_unstale":"2024-05-09T06:27:28.795963+0000","last_undegraded":"2024-05-09T06:27:28.795963+0000","last_fullsized":"2024-05-09T06:27:28.795963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:05.780930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795875+0000","last_change":"2024-05-09T06:27:21.640283+0000","last_active":"2024-05-09T06:27:28.795875+0000","last_peered":"2024-05-09T06:27:28.795875+0000","last_clean":"2024-05-09T06:27:28.795875+0000","last_became_active":"2024-05-09T06:27:21.639981+0000","last_became_peered":"2024-05-09T06:27:21.639981+0000","last_unstale":"2024-05-09T06:27:28.795875+0000","last_undegraded":"2024-05-09T06:27:28.795875+0000","last_fullsized":"2024-05-09T06:27:28.795875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:57.154069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794456+0000","last_change":"2024-05-09T06:27:20.636469+0000","last_active":"2024-05-09T06:27:28.794456+0000","last_peered":"2024-05-09T06:27:28.794456+0000","last_clean":"2024-05-09T06:27:28.794456+0000","last_became_active":"2024-05-09T06:27:20.636200+0000","last_became_peered":"2024-05-09T06:27:20.636200+0000","last_unstale":"2024-05-09T06:27:28.794456+0000","last_undegraded":"2024-05-09T06:27:28.794456+0000","last_fullsized":"2024-05-09T06:27:28.794456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:02.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796704+0000","last_change":"2024-05-09T06:27:25.868001+0000","last_active":"2024-05-09T06:27:28.796704+0000","last_peered":"2024-05-09T06:27:28.796704+0000","last_clean":"2024-05-09T06:27:28.796704+0000","last_became_active":"2024-05-09T06:27:25.867561+0000","last_became_peered":"2024-05-09T06:27:25.867561+0000","last_unstale":"2024-05-09T06:27:28.796704+0000","last_undegraded":"2024-05-09T06:27:28.796704+0000","last_fullsized":"2024-05-09T06:27:28.796704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310020+0000","last_change":"2024-05-09T06:27:21.647526+0000","last_active":"2024-05-09T06:27:28.310020+0000","last_peered":"2024-05-09T06:27:28.310020+0000","last_clean":"2024-05-09T06:27:28.310020+0000","last_became_active":"2024-05-09T06:27:21.646721+0000","last_became_peered":"2024-05-09T06:27:21.646721+0000","last_unstale":"2024-05-09T06:27:28.310020+0000","last_undegraded":"2024-05-09T06:27:28.310020+0000","last_fullsized":"2024-05-09T06:27:28.310020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:31.226892+0000","last_change":"2024-05-09T06:27:20.634111+0000","last_active":"2024-05-09T06:27:31.226892+0000","last_peered":"2024-05-09T06:27:31.226892+0000","last_clean":"2024-05-09T06:27:31.226892+0000","last_became_active":"2024-05-09T06:27:20.633772+0000","last_became_peered":"2024-05-09T06:27:20.633772+0000","last_unstale":"2024-05-09T06:27:31.226892+0000","last_undegraded":"2024-05-09T06:27:31.226892+0000","last_fullsized":"2024-05-09T06:27:31.226892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897702+0000","last_change":"2024-05-09T06:27:25.676189+0000","last_active":"2024-05-09T06:27:26.897702+0000","last_peered":"2024-05-09T06:27:26.897702+0000","last_clean":"2024-05-09T06:27:26.897702+0000","last_became_active":"2024-05-09T06:27:25.672390+0000","last_became_peered":"2024-05-09T06:27:25.672390+0000","last_unstale":"2024-05-09T06:27:26.897702+0000","last_undegraded":"2024-05-09T06:27:26.897702+0000","last_fullsized":"2024-05-09T06:27:26.897702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667630+0000","last_change":"2024-05-09T06:27:21.637363+0000","last_active":"2024-05-09T06:27:26.667630+0000","last_peered":"2024-05-09T06:27:26.667630+0000","last_clean":"2024-05-09T06:27:26.667630+0000","last_became_active":"2024-05-09T06:27:21.637056+0000","last_became_peered":"2024-05-09T06:27:21.637056+0000","last_unstale":"2024-05-09T06:27:26.667630+0000","last_undegraded":"2024-05-09T06:27:26.667630+0000","last_fullsized":"2024-05-09T06:27:26.667630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897618+0000","last_change":"2024-05-09T06:27:20.641960+0000","last_active":"2024-05-09T06:27:26.897618+0000","last_peered":"2024-05-09T06:27:26.897618+0000","last_clean":"2024-05-09T06:27:26.897618+0000","last_became_active":"2024-05-09T06:27:20.641670+0000","last_became_peered":"2024-05-09T06:27:20.641670+0000","last_unstale":"2024-05-09T06:27:26.897618+0000","last_undegraded":"2024-05-09T06:27:26.897618+0000","last_fullsized":"2024-05-09T06:27:26.897618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667165+0000","last_change":"2024-05-09T06:27:25.675301+0000","last_active":"2024-05-09T06:27:26.667165+0000","last_peered":"2024-05-09T06:27:26.667165+0000","last_clean":"2024-05-09T06:27:26.667165+0000","last_became_active":"2024-05-09T06:27:25.674818+0000","last_became_peered":"2024-05-09T06:27:25.674818+0000","last_unstale":"2024-05-09T06:27:26.667165+0000","last_undegraded":"2024-05-09T06:27:26.667165+0000","last_fullsized":"2024-05-09T06:27:26.667165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:06.045330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796636+0000","last_change":"2024-05-09T06:27:21.638651+0000","last_active":"2024-05-09T06:27:28.796636+0000","last_peered":"2024-05-09T06:27:28.796636+0000","last_clean":"2024-05-09T06:27:28.796636+0000","last_became_active":"2024-05-09T06:27:21.638111+0000","last_became_peered":"2024-05-09T06:27:21.638111+0000","last_unstale":"2024-05-09T06:27:28.796636+0000","last_undegraded":"2024-05-09T06:27:28.796636+0000","last_fullsized":"2024-05-09T06:27:28.796636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796815+0000","last_change":"2024-05-09T06:27:20.639625+0000","last_active":"2024-05-09T06:27:28.796815+0000","last_peered":"2024-05-09T06:27:28.796815+0000","last_clean":"2024-05-09T06:27:28.796815+0000","last_became_active":"2024-05-09T06:27:20.639287+0000","last_became_peered":"2024-05-09T06:27:20.639287+0000","last_unstale":"2024-05-09T06:27:28.796815+0000","last_undegraded":"2024-05-09T06:27:28.796815+0000","last_fullsized":"2024-05-09T06:27:28.796815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796227+0000","last_change":"2024-05-09T06:27:25.669479+0000","last_active":"2024-05-09T06:27:28.796227+0000","last_peered":"2024-05-09T06:27:28.796227+0000","last_clean":"2024-05-09T06:27:28.796227+0000","last_became_active":"2024-05-09T06:27:25.668634+0000","last_became_peered":"2024-05-09T06:27:25.668634+0000","last_unstale":"2024-05-09T06:27:28.796227+0000","last_undegraded":"2024-05-09T06:27:28.796227+0000","last_fullsized":"2024-05-09T06:27:28.796227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:33.708720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796768+0000","last_change":"2024-05-09T06:27:21.657985+0000","last_active":"2024-05-09T06:27:28.796768+0000","last_peered":"2024-05-09T06:27:28.796768+0000","last_clean":"2024-05-09T06:27:28.796768+0000","last_became_active":"2024-05-09T06:27:21.653693+0000","last_became_peered":"2024-05-09T06:27:21.653693+0000","last_unstale":"2024-05-09T06:27:28.796768+0000","last_undegraded":"2024-05-09T06:27:28.796768+0000","last_fullsized":"2024-05-09T06:27:28.796768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796692+0000","last_change":"2024-05-09T06:27:20.639702+0000","last_active":"2024-05-09T06:27:28.796692+0000","last_peered":"2024-05-09T06:27:28.796692+0000","last_clean":"2024-05-09T06:27:28.796692+0000","last_became_active":"2024-05-09T06:27:20.639361+0000","last_became_peered":"2024-05-09T06:27:20.639361+0000","last_unstale":"2024-05-09T06:27:28.796692+0000","last_undegraded":"2024-05-09T06:27:28.796692+0000","last_fullsized":"2024-05-09T06:27:28.796692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:25.867964+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:25.867579+0000","last_became_peered":"2024-05-09T06:27:25.867579+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796481+0000","last_change":"2024-05-09T06:27:25.871150+0000","last_active":"2024-05-09T06:27:28.796481+0000","last_peered":"2024-05-09T06:27:28.796481+0000","last_clean":"2024-05-09T06:27:28.796481+0000","last_became_active":"2024-05-09T06:27:25.870707+0000","last_became_peered":"2024-05-09T06:27:25.870707+0000","last_unstale":"2024-05-09T06:27:28.796481+0000","last_undegraded":"2024-05-09T06:27:28.796481+0000","last_fullsized":"2024-05-09T06:27:28.796481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897140+0000","last_change":"2024-05-09T06:27:21.648383+0000","last_active":"2024-05-09T06:27:26.897140+0000","last_peered":"2024-05-09T06:27:26.897140+0000","last_clean":"2024-05-09T06:27:26.897140+0000","last_became_active":"2024-05-09T06:27:21.646149+0000","last_became_peered":"2024-05-09T06:27:21.646149+0000","last_unstale":"2024-05-09T06:27:26.897140+0000","last_undegraded":"2024-05-09T06:27:26.897140+0000","last_fullsized":"2024-05-09T06:27:26.897140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:20.639240+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:20.638885+0000","last_became_peered":"2024-05-09T06:27:20.638885+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796512+0000","last_change":"2024-05-09T06:27:25.673610+0000","last_active":"2024-05-09T06:27:28.796512+0000","last_peered":"2024-05-09T06:27:28.796512+0000","last_clean":"2024-05-09T06:27:28.796512+0000","last_became_active":"2024-05-09T06:27:25.673160+0000","last_became_peered":"2024-05-09T06:27:25.673160+0000","last_unstale":"2024-05-09T06:27:28.796512+0000","last_undegraded":"2024-05-09T06:27:28.796512+0000","last_fullsized":"2024-05-09T06:27:28.796512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796199+0000","last_change":"2024-05-09T06:27:21.645047+0000","last_active":"2024-05-09T06:27:28.796199+0000","last_peered":"2024-05-09T06:27:28.796199+0000","last_clean":"2024-05-09T06:27:28.796199+0000","last_became_active":"2024-05-09T06:27:21.644743+0000","last_became_peered":"2024-05-09T06:27:21.644743+0000","last_unstale":"2024-05-09T06:27:28.796199+0000","last_undegraded":"2024-05-09T06:27:28.796199+0000","last_fullsized":"2024-05-09T06:27:28.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795221+0000","last_change":"2024-05-09T06:27:20.635683+0000","last_active":"2024-05-09T06:27:28.795221+0000","last_peered":"2024-05-09T06:27:28.795221+0000","last_clean":"2024-05-09T06:27:28.795221+0000","last_became_active":"2024-05-09T06:27:20.635386+0000","last_became_peered":"2024-05-09T06:27:20.635386+0000","last_unstale":"2024-05-09T06:27:28.795221+0000","last_undegraded":"2024-05-09T06:27:28.795221+0000","last_fullsized":"2024-05-09T06:27:28.795221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.642531+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.642176+0000","last_became_peered":"2024-05-09T06:27:20.642176+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795979+0000","last_change":"2024-05-09T06:27:21.634797+0000","last_active":"2024-05-09T06:27:28.795979+0000","last_peered":"2024-05-09T06:27:28.795979+0000","last_clean":"2024-05-09T06:27:28.795979+0000","last_became_active":"2024-05-09T06:27:21.634458+0000","last_became_peered":"2024-05-09T06:27:21.634458+0000","last_unstale":"2024-05-09T06:27:28.795979+0000","last_undegraded":"2024-05-09T06:27:28.795979+0000","last_fullsized":"2024-05-09T06:27:28.795979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666255+0000","last_change":"2024-05-09T06:27:25.678471+0000","last_active":"2024-05-09T06:27:26.666255+0000","last_peered":"2024-05-09T06:27:26.666255+0000","last_clean":"2024-05-09T06:27:26.666255+0000","last_became_active":"2024-05-09T06:27:25.677981+0000","last_became_peered":"2024-05-09T06:27:25.677981+0000","last_unstale":"2024-05-09T06:27:26.666255+0000","last_undegraded":"2024-05-09T06:27:26.666255+0000","last_fullsized":"2024-05-09T06:27:26.666255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:20.609371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795054+0000","last_change":"2024-05-09T06:27:20.644121+0000","last_active":"2024-05-09T06:27:28.795054+0000","last_peered":"2024-05-09T06:27:28.795054+0000","last_clean":"2024-05-09T06:27:28.795054+0000","last_became_active":"2024-05-09T06:27:20.643795+0000","last_became_peered":"2024-05-09T06:27:20.643795+0000","last_unstale":"2024-05-09T06:27:28.795054+0000","last_undegraded":"2024-05-09T06:27:28.795054+0000","last_fullsized":"2024-05-09T06:27:28.795054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666578+0000","last_change":"2024-05-09T06:27:21.652395+0000","last_active":"2024-05-09T06:27:26.666578+0000","last_peered":"2024-05-09T06:27:26.666578+0000","last_clean":"2024-05-09T06:27:26.666578+0000","last_became_active":"2024-05-09T06:27:21.652019+0000","last_became_peered":"2024-05-09T06:27:21.652019+0000","last_unstale":"2024-05-09T06:27:26.666578+0000","last_undegraded":"2024-05-09T06:27:26.666578+0000","last_fullsized":"2024-05-09T06:27:26.666578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:37.219520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795669+0000","last_change":"2024-05-09T06:27:25.675291+0000","last_active":"2024-05-09T06:27:28.795669+0000","last_peered":"2024-05-09T06:27:28.795669+0000","last_clean":"2024-05-09T06:27:28.795669+0000","last_became_active":"2024-05-09T06:27:25.674422+0000","last_became_peered":"2024-05-09T06:27:25.674422+0000","last_unstale":"2024-05-09T06:27:28.795669+0000","last_undegraded":"2024-05-09T06:27:28.795669+0000","last_fullsized":"2024-05-09T06:27:28.795669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:18.601362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795268+0000","last_change":"2024-05-09T06:27:21.659311+0000","last_active":"2024-05-09T06:27:28.795268+0000","last_peered":"2024-05-09T06:27:28.795268+0000","last_clean":"2024-05-09T06:27:28.795268+0000","last_became_active":"2024-05-09T06:27:21.659039+0000","last_became_peered":"2024-05-09T06:27:21.659039+0000","last_unstale":"2024-05-09T06:27:28.795268+0000","last_undegraded":"2024-05-09T06:27:28.795268+0000","last_fullsized":"2024-05-09T06:27:28.795268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665756+0000","last_change":"2024-05-09T06:27:21.656834+0000","last_active":"2024-05-09T06:27:26.665756+0000","last_peered":"2024-05-09T06:27:26.665756+0000","last_clean":"2024-05-09T06:27:26.665756+0000","last_became_active":"2024-05-09T06:27:21.655466+0000","last_became_peered":"2024-05-09T06:27:21.655466+0000","last_unstale":"2024-05-09T06:27:26.665756+0000","last_undegraded":"2024-05-09T06:27:26.665756+0000","last_fullsized":"2024-05-09T06:27:26.665756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796581+0000","last_change":"2024-05-09T06:27:21.665177+0000","last_active":"2024-05-09T06:27:28.796581+0000","last_peered":"2024-05-09T06:27:28.796581+0000","last_clean":"2024-05-09T06:27:28.796581+0000","last_became_active":"2024-05-09T06:27:21.663959+0000","last_became_peered":"2024-05-09T06:27:21.663959+0000","last_unstale":"2024-05-09T06:27:28.796581+0000","last_undegraded":"2024-05-09T06:27:28.796581+0000","last_fullsized":"2024-05-09T06:27:28.796581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795891+0000","last_change":"2024-05-09T06:27:21.642136+0000","last_active":"2024-05-09T06:27:28.795891+0000","last_peered":"2024-05-09T06:27:28.795891+0000","last_clean":"2024-05-09T06:27:28.795891+0000","last_became_active":"2024-05-09T06:27:21.641855+0000","last_became_peered":"2024-05-09T06:27:21.641855+0000","last_unstale":"2024-05-09T06:27:28.795891+0000","last_undegraded":"2024-05-09T06:27:28.795891+0000","last_fullsized":"2024-05-09T06:27:28.795891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796263+0000","last_change":"2024-05-09T06:27:21.660067+0000","last_active":"2024-05-09T06:27:28.796263+0000","last_peered":"2024-05-09T06:27:28.796263+0000","last_clean":"2024-05-09T06:27:28.796263+0000","last_became_active":"2024-05-09T06:27:21.651525+0000","last_became_peered":"2024-05-09T06:27:21.651525+0000","last_unstale":"2024-05-09T06:27:28.796263+0000","last_undegraded":"2024-05-09T06:27:28.796263+0000","last_fullsized":"2024-05-09T06:27:28.796263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897431+0000","last_change":"2024-05-09T06:27:21.650694+0000","last_active":"2024-05-09T06:27:26.897431+0000","last_peered":"2024-05-09T06:27:26.897431+0000","last_clean":"2024-05-09T06:27:26.897431+0000","last_became_active":"2024-05-09T06:27:21.650404+0000","last_became_peered":"2024-05-09T06:27:21.650404+0000","last_unstale":"2024-05-09T06:27:26.897431+0000","last_undegraded":"2024-05-09T06:27:26.897431+0000","last_fullsized":"2024-05-09T06:27:26.897431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796327+0000","last_change":"2024-05-09T06:27:21.648723+0000","last_active":"2024-05-09T06:27:28.796327+0000","last_peered":"2024-05-09T06:27:28.796327+0000","last_clean":"2024-05-09T06:27:28.796327+0000","last_became_active":"2024-05-09T06:27:21.648434+0000","last_became_peered":"2024-05-09T06:27:21.648434+0000","last_unstale":"2024-05-09T06:27:28.796327+0000","last_undegraded":"2024-05-09T06:27:28.796327+0000","last_fullsized":"2024-05-09T06:27:28.796327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795585+0000","last_change":"2024-05-09T06:27:21.659502+0000","last_active":"2024-05-09T06:27:28.795585+0000","last_peered":"2024-05-09T06:27:28.795585+0000","last_clean":"2024-05-09T06:27:28.795585+0000","last_became_active":"2024-05-09T06:27:21.651268+0000","last_became_peered":"2024-05-09T06:27:21.651268+0000","last_unstale":"2024-05-09T06:27:28.795585+0000","last_undegraded":"2024-05-09T06:27:28.795585+0000","last_fullsized":"2024-05-09T06:27:28.795585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795603+0000","last_change":"2024-05-09T06:27:21.664539+0000","last_active":"2024-05-09T06:27:28.795603+0000","last_peered":"2024-05-09T06:27:28.795603+0000","last_clean":"2024-05-09T06:27:28.795603+0000","last_became_active":"2024-05-09T06:27:21.664267+0000","last_became_peered":"2024-05-09T06:27:21.664267+0000","last_unstale":"2024-05-09T06:27:28.795603+0000","last_undegraded":"2024-05-09T06:27:28.795603+0000","last_fullsized":"2024-05-09T06:27:28.795603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795097+0000","last_change":"2024-05-09T06:27:21.658842+0000","last_active":"2024-05-09T06:27:28.795097+0000","last_peered":"2024-05-09T06:27:28.795097+0000","last_clean":"2024-05-09T06:27:28.795097+0000","last_became_active":"2024-05-09T06:27:21.658522+0000","last_became_peered":"2024-05-09T06:27:21.658522+0000","last_unstale":"2024-05-09T06:27:28.795097+0000","last_undegraded":"2024-05-09T06:27:28.795097+0000","last_fullsized":"2024-05-09T06:27:28.795097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796440+0000","last_change":"2024-05-09T06:27:21.657936+0000","last_active":"2024-05-09T06:27:28.796440+0000","last_peered":"2024-05-09T06:27:28.796440+0000","last_clean":"2024-05-09T06:27:28.796440+0000","last_became_active":"2024-05-09T06:27:21.654771+0000","last_became_peered":"2024-05-09T06:27:21.654771+0000","last_unstale":"2024-05-09T06:27:28.796440+0000","last_undegraded":"2024-05-09T06:27:28.796440+0000","last_fullsized":"2024-05-09T06:27:28.796440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795635+0000","last_change":"2024-05-09T06:27:21.645805+0000","last_active":"2024-05-09T06:27:28.795635+0000","last_peered":"2024-05-09T06:27:28.795635+0000","last_clean":"2024-05-09T06:27:28.795635+0000","last_became_active":"2024-05-09T06:27:21.643759+0000","last_became_peered":"2024-05-09T06:27:21.643759+0000","last_unstale":"2024-05-09T06:27:28.795635+0000","last_undegraded":"2024-05-09T06:27:28.795635+0000","last_fullsized":"2024-05-09T06:27:28.795635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:12.851183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795676+0000","last_change":"2024-05-09T06:27:21.654407+0000","last_active":"2024-05-09T06:27:28.795676+0000","last_peered":"2024-05-09T06:27:28.795676+0000","last_clean":"2024-05-09T06:27:28.795676+0000","last_became_active":"2024-05-09T06:27:21.654126+0000","last_became_peered":"2024-05-09T06:27:21.654126+0000","last_unstale":"2024-05-09T06:27:28.795676+0000","last_undegraded":"2024-05-09T06:27:28.795676+0000","last_fullsized":"2024-05-09T06:27:28.795676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310344+0000","last_change":"2024-05-09T06:27:21.651773+0000","last_active":"2024-05-09T06:27:28.310344+0000","last_peered":"2024-05-09T06:27:28.310344+0000","last_clean":"2024-05-09T06:27:28.310344+0000","last_became_active":"2024-05-09T06:27:21.650069+0000","last_became_peered":"2024-05-09T06:27:21.650069+0000","last_unstale":"2024-05-09T06:27:28.310344+0000","last_undegraded":"2024-05-09T06:27:28.310344+0000","last_fullsized":"2024-05-09T06:27:28.310344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666446+0000","last_change":"2024-05-09T06:27:21.657123+0000","last_active":"2024-05-09T06:27:26.666446+0000","last_peered":"2024-05-09T06:27:26.666446+0000","last_clean":"2024-05-09T06:27:26.666446+0000","last_became_active":"2024-05-09T06:27:21.656787+0000","last_became_peered":"2024-05-09T06:27:21.656787+0000","last_unstale":"2024-05-09T06:27:26.666446+0000","last_undegraded":"2024-05-09T06:27:26.666446+0000","last_fullsized":"2024-05-09T06:27:26.666446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.797072+0000","last_change":"2024-05-09T06:27:21.660613+0000","last_active":"2024-05-09T06:27:28.797072+0000","last_peered":"2024-05-09T06:27:28.797072+0000","last_clean":"2024-05-09T06:27:28.797072+0000","last_became_active":"2024-05-09T06:27:21.654942+0000","last_became_peered":"2024-05-09T06:27:21.654942+0000","last_unstale":"2024-05-09T06:27:28.797072+0000","last_undegraded":"2024-05-09T06:27:28.797072+0000","last_fullsized":"2024-05-09T06:27:28.797072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795476+0000","last_change":"2024-05-09T06:27:21.658261+0000","last_active":"2024-05-09T06:27:28.795476+0000","last_peered":"2024-05-09T06:27:28.795476+0000","last_clean":"2024-05-09T06:27:28.795476+0000","last_became_active":"2024-05-09T06:27:21.658004+0000","last_became_peered":"2024-05-09T06:27:21.658004+0000","last_unstale":"2024-05-09T06:27:28.795476+0000","last_undegraded":"2024-05-09T06:27:28.795476+0000","last_fullsized":"2024-05-09T06:27:28.795476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897230+0000","last_change":"2024-05-09T06:27:21.654019+0000","last_active":"2024-05-09T06:27:26.897230+0000","last_peered":"2024-05-09T06:27:26.897230+0000","last_clean":"2024-05-09T06:27:26.897230+0000","last_became_active":"2024-05-09T06:27:21.653711+0000","last_became_peered":"2024-05-09T06:27:21.653711+0000","last_unstale":"2024-05-09T06:27:26.897230+0000","last_undegraded":"2024-05-09T06:27:26.897230+0000","last_fullsized":"2024-05-09T06:27:26.897230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795778+0000","last_change":"2024-05-09T06:27:21.645328+0000","last_active":"2024-05-09T06:27:28.795778+0000","last_peered":"2024-05-09T06:27:28.795778+0000","last_clean":"2024-05-09T06:27:28.795778+0000","last_became_active":"2024-05-09T06:27:21.644287+0000","last_became_peered":"2024-05-09T06:27:21.644287+0000","last_unstale":"2024-05-09T06:27:28.795778+0000","last_undegraded":"2024-05-09T06:27:28.795778+0000","last_fullsized":"2024-05-09T06:27:28.795778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:16.580568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795360+0000","last_change":"2024-05-09T06:27:21.643190+0000","last_active":"2024-05-09T06:27:28.795360+0000","last_peered":"2024-05-09T06:27:28.795360+0000","last_clean":"2024-05-09T06:27:28.795360+0000","last_became_active":"2024-05-09T06:27:21.642700+0000","last_became_peered":"2024-05-09T06:27:21.642700+0000","last_unstale":"2024-05-09T06:27:28.795360+0000","last_undegraded":"2024-05-09T06:27:28.795360+0000","last_fullsized":"2024-05-09T06:27:28.795360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667413+0000","last_change":"2024-05-09T06:27:21.652745+0000","last_active":"2024-05-09T06:27:26.667413+0000","last_peered":"2024-05-09T06:27:26.667413+0000","last_clean":"2024-05-09T06:27:26.667413+0000","last_became_active":"2024-05-09T06:27:21.651219+0000","last_became_peered":"2024-05-09T06:27:21.651219+0000","last_unstale":"2024-05-09T06:27:26.667413+0000","last_undegraded":"2024-05-09T06:27:26.667413+0000","last_fullsized":"2024-05-09T06:27:26.667413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795888+0000","last_change":"2024-05-09T06:27:21.663110+0000","last_active":"2024-05-09T06:27:28.795888+0000","last_peered":"2024-05-09T06:27:28.795888+0000","last_clean":"2024-05-09T06:27:28.795888+0000","last_became_active":"2024-05-09T06:27:21.662731+0000","last_became_peered":"2024-05-09T06:27:21.662731+0000","last_unstale":"2024-05-09T06:27:28.795888+0000","last_undegraded":"2024-05-09T06:27:28.795888+0000","last_fullsized":"2024-05-09T06:27:28.795888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796301+0000","last_change":"2024-05-09T06:27:21.659071+0000","last_active":"2024-05-09T06:27:28.796301+0000","last_peered":"2024-05-09T06:27:28.796301+0000","last_clean":"2024-05-09T06:27:28.796301+0000","last_became_active":"2024-05-09T06:27:21.658774+0000","last_became_peered":"2024-05-09T06:27:21.658774+0000","last_unstale":"2024-05-09T06:27:28.796301+0000","last_undegraded":"2024-05-09T06:27:28.796301+0000","last_fullsized":"2024-05-09T06:27:28.796301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310458+0000","last_change":"2024-05-09T06:27:21.665023+0000","last_active":"2024-05-09T06:27:28.310458+0000","last_peered":"2024-05-09T06:27:28.310458+0000","last_clean":"2024-05-09T06:27:28.310458+0000","last_became_active":"2024-05-09T06:27:21.664464+0000","last_became_peered":"2024-05-09T06:27:21.664464+0000","last_unstale":"2024-05-09T06:27:28.310458+0000","last_undegraded":"2024-05-09T06:27:28.310458+0000","last_fullsized":"2024-05-09T06:27:28.310458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795913+0000","last_change":"2024-05-09T06:27:21.661852+0000","last_active":"2024-05-09T06:27:28.795913+0000","last_peered":"2024-05-09T06:27:28.795913+0000","last_clean":"2024-05-09T06:27:28.795913+0000","last_became_active":"2024-05-09T06:27:21.660313+0000","last_became_peered":"2024-05-09T06:27:21.660313+0000","last_unstale":"2024-05-09T06:27:28.795913+0000","last_undegraded":"2024-05-09T06:27:28.795913+0000","last_fullsized":"2024-05-09T06:27:28.795913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309769+0000","last_change":"2024-05-09T06:27:21.654130+0000","last_active":"2024-05-09T06:27:28.309769+0000","last_peered":"2024-05-09T06:27:28.309769+0000","last_clean":"2024-05-09T06:27:28.309769+0000","last_became_active":"2024-05-09T06:27:21.653593+0000","last_became_peered":"2024-05-09T06:27:21.653593+0000","last_unstale":"2024-05-09T06:27:28.309769+0000","last_undegraded":"2024-05-09T06:27:28.309769+0000","last_fullsized":"2024-05-09T06:27:28.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796010+0000","last_change":"2024-05-09T06:27:21.654775+0000","last_active":"2024-05-09T06:27:28.796010+0000","last_peered":"2024-05-09T06:27:28.796010+0000","last_clean":"2024-05-09T06:27:28.796010+0000","last_became_active":"2024-05-09T06:27:21.654269+0000","last_became_peered":"2024-05-09T06:27:21.654269+0000","last_unstale":"2024-05-09T06:27:28.796010+0000","last_undegraded":"2024-05-09T06:27:28.796010+0000","last_fullsized":"2024-05-09T06:27:28.796010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309790+0000","last_change":"2024-05-09T06:27:21.649923+0000","last_active":"2024-05-09T06:27:28.309790+0000","last_peered":"2024-05-09T06:27:28.309790+0000","last_clean":"2024-05-09T06:27:28.309790+0000","last_became_active":"2024-05-09T06:27:21.648693+0000","last_became_peered":"2024-05-09T06:27:21.648693+0000","last_unstale":"2024-05-09T06:27:28.309790+0000","last_undegraded":"2024-05-09T06:27:28.309790+0000","last_fullsized":"2024-05-09T06:27:28.309790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796403+0000","last_change":"2024-05-09T06:27:21.664868+0000","last_active":"2024-05-09T06:27:28.796403+0000","last_peered":"2024-05-09T06:27:28.796403+0000","last_clean":"2024-05-09T06:27:28.796403+0000","last_became_active":"2024-05-09T06:27:21.663870+0000","last_became_peered":"2024-05-09T06:27:21.663870+0000","last_unstale":"2024-05-09T06:27:28.796403+0000","last_undegraded":"2024-05-09T06:27:28.796403+0000","last_fullsized":"2024-05-09T06:27:28.796403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796427+0000","last_change":"2024-05-09T06:27:21.651232+0000","last_active":"2024-05-09T06:27:28.796427+0000","last_peered":"2024-05-09T06:27:28.796427+0000","last_clean":"2024-05-09T06:27:28.796427+0000","last_became_active":"2024-05-09T06:27:21.650940+0000","last_became_peered":"2024-05-09T06:27:21.650940+0000","last_unstale":"2024-05-09T06:27:28.796427+0000","last_undegraded":"2024-05-09T06:27:28.796427+0000","last_fullsized":"2024-05-09T06:27:28.796427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897594+0000","last_change":"2024-05-09T06:27:21.648169+0000","last_active":"2024-05-09T06:27:26.897594+0000","last_peered":"2024-05-09T06:27:26.897594+0000","last_clean":"2024-05-09T06:27:26.897594+0000","last_became_active":"2024-05-09T06:27:21.647846+0000","last_became_peered":"2024-05-09T06:27:21.647846+0000","last_unstale":"2024-05-09T06:27:26.897594+0000","last_undegraded":"2024-05-09T06:27:26.897594+0000","last_fullsized":"2024-05-09T06:27:26.897594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310386+0000","last_change":"2024-05-09T06:27:21.649391+0000","last_active":"2024-05-09T06:27:28.310386+0000","last_peered":"2024-05-09T06:27:28.310386+0000","last_clean":"2024-05-09T06:27:28.310386+0000","last_became_active":"2024-05-09T06:27:21.648828+0000","last_became_peered":"2024-05-09T06:27:21.648828+0000","last_unstale":"2024-05-09T06:27:28.310386+0000","last_undegraded":"2024-05-09T06:27:28.310386+0000","last_fullsized":"2024-05-09T06:27:28.310386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:27:49.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:27:49.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.366+0000 7f658fde9640 1 Processor -- start 2024-05-09T06:27:49.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.366+0000 7f658fde9640 1 -- start start 2024-05-09T06:27:49.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658fde9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65881454b0 0x7f65880b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658fde9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65880b2a60 0x7f65880b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658fde9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65880b5230 con 0x7f65880b4da0 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658fde9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6588071c20 con 0x7f65880b2a60 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658fde9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f658806eb40 con 0x7f65881454b0 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658d384640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65880b2a60 0x7f65880b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658e386640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f65880b4da0 0x7f658815f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58774/0 (socket says 172.21.15.1:58774) 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658e386640 1 -- 172.21.15.1:0/2920283539 learned_addr learned my addr 172.21.15.1:0/2920283539 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.367+0000 7f658db85640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65881454b0 0x7f65880b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658d384640 1 -- 172.21.15.1:0/2920283539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65881454b0 msgr2=0x7f65880b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658d384640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65881454b0 0x7f65880b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658d384640 1 -- 172.21.15.1:0/2920283539 >> v1:172.21.15.1:6789/0 conn(0x7f65880b4da0 legacy=0x7f658815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658d384640 1 -- 172.21.15.1:0/2920283539 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658806ee20 con 0x7f65880b2a60 2024-05-09T06:27:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658cb83640 1 -- 172.21.15.1:0/2920283539 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320704670 0 0) 0x7f65880b5230 con 0x7f65880b4da0 2024-05-09T06:27:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658d384640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f65880b4860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097a0 tx=0x7f657c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e22fed49e871653 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658cb83640 1 -- 172.21.15.1:0/2920283539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c037070 con 0x7f65880b2a60 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658cb83640 1 -- 172.21.15.1:0/2920283539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c032440 con 0x7f65880b2a60 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.368+0000 7f658cb83640 1 -- 172.21.15.1:0/2920283539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c032810 con 0x7f65880b2a60 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 -- 172.21.15.1:0/2920283539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 msgr2=0x7f65880b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f65880b4860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097a0 tx=0x7f657c02a180 comp rx=0 tx=0).stop 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 -- 172.21.15.1:0/2920283539 shutdown_connections 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f65880b4860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 --2- 172.21.15.1:0/2920283539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65881454b0 0x7f65880b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 -- 172.21.15.1:0/2920283539 >> 172.21.15.1:0/2920283539 conn(0x7f65880cc4b0 msgr2=0x7f65880cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 -- 172.21.15.1:0/2920283539 shutdown_connections 2024-05-09T06:27:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.369+0000 7f658fde9640 1 -- 172.21.15.1:0/2920283539 wait complete. 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.370+0000 7f658fde9640 1 Processor -- start 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.370+0000 7f658fde9640 1 -- start start 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65880b4da0 0x7f65881e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65881454b0 0x7f65881e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6588145890 con 0x7f65880b2a60 2024-05-09T06:27:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f658806dbc0 con 0x7f65881454b0 2024-05-09T06:27:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658fde9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f658806e0e0 con 0x7f65880b4da0 2024-05-09T06:27:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658db85640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658db85640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56198/0 (socket says 172.21.15.1:56198) 2024-05-09T06:27:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658db85640 1 -- 172.21.15.1:0/686514060 learned_addr learned my addr 172.21.15.1:0/686514060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658e386640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65881454b0 0x7f65881e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658d384640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65880b4da0 0x7f65881e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658db85640 1 -- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65880b4da0 msgr2=0x7f65881e7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.371+0000 7f658db85640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65880b4da0 0x7f65881e7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658db85640 1 -- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65881454b0 msgr2=0x7f65881e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658db85640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65881454b0 0x7f65881e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658db85640 1 -- 172.21.15.1:0/686514060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65881e1ce0 con 0x7f65880b2a60 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658db85640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f658000d830 tx=0x7f658000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658000b6c0 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580019070 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65881e1fc0 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658000ba80 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.372+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658815bcc0 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.373+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65880752b0 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.373+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f658000bc20 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.374+0000 7f65867fc640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 0x7f657406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.374+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f658008b5e0 con 0x7f65880b2a60 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.374+0000 7f658d384640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 0x7f657406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.374+0000 7f658d384640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 0x7f657406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f657c034000 tx=0x7f657c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.378+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f658001c360 con 0x7f65880b2a60 2024-05-09T06:27:49.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.560+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65881e2190 con 0x7f657406be30 2024-05-09T06:27:49.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.568+0000 7f65867fc640 1 -- 172.21.15.1:0/686514060 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f65881e2190 con 0x7f657406be30 2024-05-09T06:27:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:49.555 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 msgr2=0x7f657406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 0x7f657406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f657c034000 tx=0x7f657c02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 msgr2=0x7f658815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f658000d830 tx=0x7f658000dcf0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 shutdown_connections 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f657406be30 0x7f657406e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65881454b0 0x7f65881e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65880b4da0 0x7f65881e7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 --2- 172.21.15.1:0/686514060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65880b2a60 0x7f658815b780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 >> 172.21.15.1:0/686514060 conn(0x7f65880cc4b0 msgr2=0x7f65880b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 shutdown_connections 2024-05-09T06:27:49.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.572+0000 7f658fde9640 1 -- 172.21.15.1:0/686514060 wait complete. 2024-05-09T06:27:49.565 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-05-09T06:27:49.074539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002924"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795027+0000","last_change":"2024-05-09T06:27:20.644248+0000","last_active":"2024-05-09T06:27:28.795027+0000","last_peered":"2024-05-09T06:27:28.795027+0000","last_clean":"2024-05-09T06:27:28.795027+0000","last_became_active":"2024-05-09T06:27:20.643791+0000","last_became_peered":"2024-05-09T06:27:20.643791+0000","last_unstale":"2024-05-09T06:27:28.795027+0000","last_undegraded":"2024-05-09T06:27:28.795027+0000","last_fullsized":"2024-05-09T06:27:28.795027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:48.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795308+0000","last_change":"2024-05-09T06:27:21.649501+0000","last_active":"2024-05-09T06:27:28.795308+0000","last_peered":"2024-05-09T06:27:28.795308+0000","last_clean":"2024-05-09T06:27:28.795308+0000","last_became_active":"2024-05-09T06:27:21.649193+0000","last_became_peered":"2024-05-09T06:27:21.649193+0000","last_unstale":"2024-05-09T06:27:28.795308+0000","last_undegraded":"2024-05-09T06:27:28.795308+0000","last_fullsized":"2024-05-09T06:27:28.795308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794757+0000","last_change":"2024-05-09T06:27:25.871323+0000","last_active":"2024-05-09T06:27:28.794757+0000","last_peered":"2024-05-09T06:27:28.794757+0000","last_clean":"2024-05-09T06:27:28.794757+0000","last_became_active":"2024-05-09T06:27:25.870947+0000","last_became_peered":"2024-05-09T06:27:25.870947+0000","last_unstale":"2024-05-09T06:27:28.794757+0000","last_undegraded":"2024-05-09T06:27:28.794757+0000","last_fullsized":"2024-05-09T06:27:28.794757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795538+0000","last_change":"2024-05-09T06:27:20.634799+0000","last_active":"2024-05-09T06:27:28.795538+0000","last_peered":"2024-05-09T06:27:28.795538+0000","last_clean":"2024-05-09T06:27:28.795538+0000","last_became_active":"2024-05-09T06:27:20.634300+0000","last_became_peered":"2024-05-09T06:27:20.634300+0000","last_unstale":"2024-05-09T06:27:28.795538+0000","last_undegraded":"2024-05-09T06:27:28.795538+0000","last_fullsized":"2024-05-09T06:27:28.795538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796489+0000","last_change":"2024-05-09T06:27:21.638390+0000","last_active":"2024-05-09T06:27:28.796489+0000","last_peered":"2024-05-09T06:27:28.796489+0000","last_clean":"2024-05-09T06:27:28.796489+0000","last_became_active":"2024-05-09T06:27:21.637720+0000","last_became_peered":"2024-05-09T06:27:21.637720+0000","last_unstale":"2024-05-09T06:27:28.796489+0000","last_undegraded":"2024-05-09T06:27:28.796489+0000","last_fullsized":"2024-05-09T06:27:28.796489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666410+0000","last_change":"2024-05-09T06:27:25.668634+0000","last_active":"2024-05-09T06:27:26.666410+0000","last_peered":"2024-05-09T06:27:26.666410+0000","last_clean":"2024-05-09T06:27:26.666410+0000","last_became_active":"2024-05-09T06:27:25.668153+0000","last_became_peered":"2024-05-09T06:27:25.668153+0000","last_unstale":"2024-05-09T06:27:26.666410+0000","last_undegraded":"2024-05-09T06:27:26.666410+0000","last_fullsized":"2024-05-09T06:27:26.666410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:46.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310311+0000","last_change":"2024-05-09T06:27:20.646888+0000","last_active":"2024-05-09T06:27:28.310311+0000","last_peered":"2024-05-09T06:27:28.310311+0000","last_clean":"2024-05-09T06:27:28.310311+0000","last_became_active":"2024-05-09T06:27:20.646603+0000","last_became_peered":"2024-05-09T06:27:20.646603+0000","last_unstale":"2024-05-09T06:27:28.310311+0000","last_undegraded":"2024-05-09T06:27:28.310311+0000","last_fullsized":"2024-05-09T06:27:28.310311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:06.656512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796678+0000","last_change":"2024-05-09T06:27:21.659688+0000","last_active":"2024-05-09T06:27:28.796678+0000","last_peered":"2024-05-09T06:27:28.796678+0000","last_clean":"2024-05-09T06:27:28.796678+0000","last_became_active":"2024-05-09T06:27:21.659407+0000","last_became_peered":"2024-05-09T06:27:21.659407+0000","last_unstale":"2024-05-09T06:27:28.796678+0000","last_undegraded":"2024-05-09T06:27:28.796678+0000","last_fullsized":"2024-05-09T06:27:28.796678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667001+0000","last_change":"2024-05-09T06:27:25.674815+0000","last_active":"2024-05-09T06:27:26.667001+0000","last_peered":"2024-05-09T06:27:26.667001+0000","last_clean":"2024-05-09T06:27:26.667001+0000","last_became_active":"2024-05-09T06:27:25.673855+0000","last_became_peered":"2024-05-09T06:27:25.673855+0000","last_unstale":"2024-05-09T06:27:26.667001+0000","last_undegraded":"2024-05-09T06:27:26.667001+0000","last_fullsized":"2024-05-09T06:27:26.667001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:16.160631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309929+0000","last_change":"2024-05-09T06:27:20.644933+0000","last_active":"2024-05-09T06:27:28.309929+0000","last_peered":"2024-05-09T06:27:28.309929+0000","last_clean":"2024-05-09T06:27:28.309929+0000","last_became_active":"2024-05-09T06:27:20.644019+0000","last_became_peered":"2024-05-09T06:27:20.644019+0000","last_unstale":"2024-05-09T06:27:28.309929+0000","last_undegraded":"2024-05-09T06:27:28.309929+0000","last_fullsized":"2024-05-09T06:27:28.309929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796776+0000","last_change":"2024-05-09T06:27:21.660829+0000","last_active":"2024-05-09T06:27:28.796776+0000","last_peered":"2024-05-09T06:27:28.796776+0000","last_clean":"2024-05-09T06:27:28.796776+0000","last_became_active":"2024-05-09T06:27:21.658349+0000","last_became_peered":"2024-05-09T06:27:21.658349+0000","last_unstale":"2024-05-09T06:27:28.796776+0000","last_undegraded":"2024-05-09T06:27:28.796776+0000","last_fullsized":"2024-05-09T06:27:28.796776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666466+0000","last_change":"2024-05-09T06:27:25.664257+0000","last_active":"2024-05-09T06:27:26.666466+0000","last_peered":"2024-05-09T06:27:26.666466+0000","last_clean":"2024-05-09T06:27:26.666466+0000","last_became_active":"2024-05-09T06:27:25.663787+0000","last_became_peered":"2024-05-09T06:27:25.663787+0000","last_unstale":"2024-05-09T06:27:26.666466+0000","last_undegraded":"2024-05-09T06:27:26.666466+0000","last_fullsized":"2024-05-09T06:27:26.666466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:57.007200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795391+0000","last_change":"2024-05-09T06:27:21.652107+0000","last_active":"2024-05-09T06:27:28.795391+0000","last_peered":"2024-05-09T06:27:28.795391+0000","last_clean":"2024-05-09T06:27:28.795391+0000","last_became_active":"2024-05-09T06:27:21.651764+0000","last_became_peered":"2024-05-09T06:27:21.651764+0000","last_unstale":"2024-05-09T06:27:28.795391+0000","last_undegraded":"2024-05-09T06:27:28.795391+0000","last_fullsized":"2024-05-09T06:27:28.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310074+0000","last_change":"2024-05-09T06:27:20.644636+0000","last_active":"2024-05-09T06:27:28.310074+0000","last_peered":"2024-05-09T06:27:28.310074+0000","last_clean":"2024-05-09T06:27:28.310074+0000","last_became_active":"2024-05-09T06:27:20.644155+0000","last_became_peered":"2024-05-09T06:27:20.644155+0000","last_unstale":"2024-05-09T06:27:28.310074+0000","last_undegraded":"2024-05-09T06:27:28.310074+0000","last_fullsized":"2024-05-09T06:27:28.310074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667637+0000","last_change":"2024-05-09T06:27:25.674517+0000","last_active":"2024-05-09T06:27:26.667637+0000","last_peered":"2024-05-09T06:27:26.667637+0000","last_clean":"2024-05-09T06:27:26.667637+0000","last_became_active":"2024-05-09T06:27:25.673256+0000","last_became_peered":"2024-05-09T06:27:25.673256+0000","last_unstale":"2024-05-09T06:27:26.667637+0000","last_undegraded":"2024-05-09T06:27:26.667637+0000","last_fullsized":"2024-05-09T06:27:26.667637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795461+0000","last_change":"2024-05-09T06:27:21.650596+0000","last_active":"2024-05-09T06:27:28.795461+0000","last_peered":"2024-05-09T06:27:28.795461+0000","last_clean":"2024-05-09T06:27:28.795461+0000","last_became_active":"2024-05-09T06:27:21.649943+0000","last_became_peered":"2024-05-09T06:27:21.649943+0000","last_unstale":"2024-05-09T06:27:28.795461+0000","last_undegraded":"2024-05-09T06:27:28.795461+0000","last_fullsized":"2024-05-09T06:27:28.795461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897815+0000","last_change":"2024-05-09T06:27:20.634659+0000","last_active":"2024-05-09T06:27:26.897815+0000","last_peered":"2024-05-09T06:27:26.897815+0000","last_clean":"2024-05-09T06:27:26.897815+0000","last_became_active":"2024-05-09T06:27:20.634344+0000","last_became_peered":"2024-05-09T06:27:20.634344+0000","last_unstale":"2024-05-09T06:27:26.897815+0000","last_undegraded":"2024-05-09T06:27:26.897815+0000","last_fullsized":"2024-05-09T06:27:26.897815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667419+0000","last_change":"2024-05-09T06:27:25.674355+0000","last_active":"2024-05-09T06:27:26.667419+0000","last_peered":"2024-05-09T06:27:26.667419+0000","last_clean":"2024-05-09T06:27:26.667419+0000","last_became_active":"2024-05-09T06:27:25.673950+0000","last_became_peered":"2024-05-09T06:27:25.673950+0000","last_unstale":"2024-05-09T06:27:26.667419+0000","last_undegraded":"2024-05-09T06:27:26.667419+0000","last_fullsized":"2024-05-09T06:27:26.667419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.231985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796292+0000","last_change":"2024-05-09T06:27:21.641405+0000","last_active":"2024-05-09T06:27:28.796292+0000","last_peered":"2024-05-09T06:27:28.796292+0000","last_clean":"2024-05-09T06:27:28.796292+0000","last_became_active":"2024-05-09T06:27:21.641019+0000","last_became_peered":"2024-05-09T06:27:21.641019+0000","last_unstale":"2024-05-09T06:27:28.796292+0000","last_undegraded":"2024-05-09T06:27:28.796292+0000","last_fullsized":"2024-05-09T06:27:28.796292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897255+0000","last_change":"2024-05-09T06:27:20.644807+0000","last_active":"2024-05-09T06:27:26.897255+0000","last_peered":"2024-05-09T06:27:26.897255+0000","last_clean":"2024-05-09T06:27:26.897255+0000","last_became_active":"2024-05-09T06:27:20.644510+0000","last_became_peered":"2024-05-09T06:27:20.644510+0000","last_unstale":"2024-05-09T06:27:26.897255+0000","last_undegraded":"2024-05-09T06:27:26.897255+0000","last_fullsized":"2024-05-09T06:27:26.897255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897895+0000","last_change":"2024-05-09T06:27:25.867962+0000","last_active":"2024-05-09T06:27:26.897895+0000","last_peered":"2024-05-09T06:27:26.897895+0000","last_clean":"2024-05-09T06:27:26.897895+0000","last_became_active":"2024-05-09T06:27:25.867593+0000","last_became_peered":"2024-05-09T06:27:25.867593+0000","last_unstale":"2024-05-09T06:27:26.897895+0000","last_undegraded":"2024-05-09T06:27:26.897895+0000","last_fullsized":"2024-05-09T06:27:26.897895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666600+0000","last_change":"2024-05-09T06:27:21.651627+0000","last_active":"2024-05-09T06:27:26.666600+0000","last_peered":"2024-05-09T06:27:26.666600+0000","last_clean":"2024-05-09T06:27:26.666600+0000","last_became_active":"2024-05-09T06:27:21.651021+0000","last_became_peered":"2024-05-09T06:27:21.651021+0000","last_unstale":"2024-05-09T06:27:26.666600+0000","last_undegraded":"2024-05-09T06:27:26.666600+0000","last_fullsized":"2024-05-09T06:27:26.666600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:04.963108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309528+0000","last_change":"2024-05-09T06:27:20.644691+0000","last_active":"2024-05-09T06:27:28.309528+0000","last_peered":"2024-05-09T06:27:28.309528+0000","last_clean":"2024-05-09T06:27:28.309528+0000","last_became_active":"2024-05-09T06:27:20.644291+0000","last_became_peered":"2024-05-09T06:27:20.644291+0000","last_unstale":"2024-05-09T06:27:28.309528+0000","last_undegraded":"2024-05-09T06:27:28.309528+0000","last_fullsized":"2024-05-09T06:27:28.309528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667629+0000","last_change":"2024-05-09T06:27:25.670582+0000","last_active":"2024-05-09T06:27:26.667629+0000","last_peered":"2024-05-09T06:27:26.667629+0000","last_clean":"2024-05-09T06:27:26.667629+0000","last_became_active":"2024-05-09T06:27:25.670180+0000","last_became_peered":"2024-05-09T06:27:25.670180+0000","last_unstale":"2024-05-09T06:27:26.667629+0000","last_undegraded":"2024-05-09T06:27:26.667629+0000","last_fullsized":"2024-05-09T06:27:26.667629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:21.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796762+0000","last_change":"2024-05-09T06:27:21.644419+0000","last_active":"2024-05-09T06:27:28.796762+0000","last_peered":"2024-05-09T06:27:28.796762+0000","last_clean":"2024-05-09T06:27:28.796762+0000","last_became_active":"2024-05-09T06:27:21.644054+0000","last_became_peered":"2024-05-09T06:27:21.644054+0000","last_unstale":"2024-05-09T06:27:28.796762+0000","last_undegraded":"2024-05-09T06:27:28.796762+0000","last_fullsized":"2024-05-09T06:27:28.796762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795166+0000","last_change":"2024-05-09T06:27:20.644413+0000","last_active":"2024-05-09T06:27:28.795166+0000","last_peered":"2024-05-09T06:27:28.795166+0000","last_clean":"2024-05-09T06:27:28.795166+0000","last_became_active":"2024-05-09T06:27:20.644063+0000","last_became_peered":"2024-05-09T06:27:20.644063+0000","last_unstale":"2024-05-09T06:27:28.795166+0000","last_undegraded":"2024-05-09T06:27:28.795166+0000","last_fullsized":"2024-05-09T06:27:28.795166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666413+0000","last_change":"2024-05-09T06:27:25.868219+0000","last_active":"2024-05-09T06:27:26.666413+0000","last_peered":"2024-05-09T06:27:26.666413+0000","last_clean":"2024-05-09T06:27:26.666413+0000","last_became_active":"2024-05-09T06:27:25.867618+0000","last_became_peered":"2024-05-09T06:27:25.867618+0000","last_unstale":"2024-05-09T06:27:26.666413+0000","last_undegraded":"2024-05-09T06:27:26.666413+0000","last_fullsized":"2024-05-09T06:27:26.666413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:59.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795981+0000","last_change":"2024-05-09T06:27:21.659207+0000","last_active":"2024-05-09T06:27:28.795981+0000","last_peered":"2024-05-09T06:27:28.795981+0000","last_clean":"2024-05-09T06:27:28.795981+0000","last_became_active":"2024-05-09T06:27:21.651367+0000","last_became_peered":"2024-05-09T06:27:21.651367+0000","last_unstale":"2024-05-09T06:27:28.795981+0000","last_undegraded":"2024-05-09T06:27:28.795981+0000","last_fullsized":"2024-05-09T06:27:28.795981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897683+0000","last_change":"2024-05-09T06:27:20.637053+0000","last_active":"2024-05-09T06:27:26.897683+0000","last_peered":"2024-05-09T06:27:26.897683+0000","last_clean":"2024-05-09T06:27:26.897683+0000","last_became_active":"2024-05-09T06:27:20.636755+0000","last_became_peered":"2024-05-09T06:27:20.636755+0000","last_unstale":"2024-05-09T06:27:26.897683+0000","last_undegraded":"2024-05-09T06:27:26.897683+0000","last_fullsized":"2024-05-09T06:27:26.897683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795768+0000","last_change":"2024-05-09T06:27:25.871255+0000","last_active":"2024-05-09T06:27:28.795768+0000","last_peered":"2024-05-09T06:27:28.795768+0000","last_clean":"2024-05-09T06:27:28.795768+0000","last_became_active":"2024-05-09T06:27:25.870862+0000","last_became_peered":"2024-05-09T06:27:25.870862+0000","last_unstale":"2024-05-09T06:27:28.795768+0000","last_undegraded":"2024-05-09T06:27:28.795768+0000","last_fullsized":"2024-05-09T06:27:28.795768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:01:53.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897799+0000","last_change":"2024-05-09T06:27:21.649868+0000","last_active":"2024-05-09T06:27:26.897799+0000","last_peered":"2024-05-09T06:27:26.897799+0000","last_clean":"2024-05-09T06:27:26.897799+0000","last_became_active":"2024-05-09T06:27:21.649515+0000","last_became_peered":"2024-05-09T06:27:21.649515+0000","last_unstale":"2024-05-09T06:27:26.897799+0000","last_undegraded":"2024-05-09T06:27:26.897799+0000","last_fullsized":"2024-05-09T06:27:26.897799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310415+0000","last_change":"2024-05-09T06:27:20.644414+0000","last_active":"2024-05-09T06:27:28.310415+0000","last_peered":"2024-05-09T06:27:28.310415+0000","last_clean":"2024-05-09T06:27:28.310415+0000","last_became_active":"2024-05-09T06:27:20.644077+0000","last_became_peered":"2024-05-09T06:27:20.644077+0000","last_unstale":"2024-05-09T06:27:28.310415+0000","last_undegraded":"2024-05-09T06:27:28.310415+0000","last_fullsized":"2024-05-09T06:27:28.310415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:34.718775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795535+0000","last_change":"2024-05-09T06:27:25.867794+0000","last_active":"2024-05-09T06:27:28.795535+0000","last_peered":"2024-05-09T06:27:28.795535+0000","last_clean":"2024-05-09T06:27:28.795535+0000","last_became_active":"2024-05-09T06:27:25.867351+0000","last_became_peered":"2024-05-09T06:27:25.867351+0000","last_unstale":"2024-05-09T06:27:28.795535+0000","last_undegraded":"2024-05-09T06:27:28.795535+0000","last_fullsized":"2024-05-09T06:27:28.795535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796829+0000","last_change":"2024-05-09T06:27:21.662543+0000","last_active":"2024-05-09T06:27:28.796829+0000","last_peered":"2024-05-09T06:27:28.796829+0000","last_clean":"2024-05-09T06:27:28.796829+0000","last_became_active":"2024-05-09T06:27:21.662266+0000","last_became_peered":"2024-05-09T06:27:21.662266+0000","last_unstale":"2024-05-09T06:27:28.796829+0000","last_undegraded":"2024-05-09T06:27:28.796829+0000","last_fullsized":"2024-05-09T06:27:28.796829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897904+0000","last_change":"2024-05-09T06:27:20.634145+0000","last_active":"2024-05-09T06:27:26.897904+0000","last_peered":"2024-05-09T06:27:26.897904+0000","last_clean":"2024-05-09T06:27:26.897904+0000","last_became_active":"2024-05-09T06:27:20.633864+0000","last_became_peered":"2024-05-09T06:27:20.633864+0000","last_unstale":"2024-05-09T06:27:26.897904+0000","last_undegraded":"2024-05-09T06:27:26.897904+0000","last_fullsized":"2024-05-09T06:27:26.897904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897405+0000","last_change":"2024-05-09T06:27:25.672524+0000","last_active":"2024-05-09T06:27:26.897405+0000","last_peered":"2024-05-09T06:27:26.897405+0000","last_clean":"2024-05-09T06:27:26.897405+0000","last_became_active":"2024-05-09T06:27:25.672004+0000","last_became_peered":"2024-05-09T06:27:25.672004+0000","last_unstale":"2024-05-09T06:27:26.897405+0000","last_undegraded":"2024-05-09T06:27:26.897405+0000","last_fullsized":"2024-05-09T06:27:26.897405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667430+0000","last_change":"2024-05-09T06:27:21.634887+0000","last_active":"2024-05-09T06:27:26.667430+0000","last_peered":"2024-05-09T06:27:26.667430+0000","last_clean":"2024-05-09T06:27:26.667430+0000","last_became_active":"2024-05-09T06:27:21.634609+0000","last_became_peered":"2024-05-09T06:27:21.634609+0000","last_unstale":"2024-05-09T06:27:26.667430+0000","last_undegraded":"2024-05-09T06:27:26.667430+0000","last_fullsized":"2024-05-09T06:27:26.667430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:59.874418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795267+0000","last_change":"2024-05-09T06:27:20.642066+0000","last_active":"2024-05-09T06:27:28.795267+0000","last_peered":"2024-05-09T06:27:28.795267+0000","last_clean":"2024-05-09T06:27:28.795267+0000","last_became_active":"2024-05-09T06:27:20.641753+0000","last_became_peered":"2024-05-09T06:27:20.641753+0000","last_unstale":"2024-05-09T06:27:28.795267+0000","last_undegraded":"2024-05-09T06:27:28.795267+0000","last_fullsized":"2024-05-09T06:27:28.795267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897884+0000","last_change":"2024-05-09T06:27:25.671230+0000","last_active":"2024-05-09T06:27:26.897884+0000","last_peered":"2024-05-09T06:27:26.897884+0000","last_clean":"2024-05-09T06:27:26.897884+0000","last_became_active":"2024-05-09T06:27:25.670778+0000","last_became_peered":"2024-05-09T06:27:25.670778+0000","last_unstale":"2024-05-09T06:27:26.897884+0000","last_undegraded":"2024-05-09T06:27:26.897884+0000","last_fullsized":"2024-05-09T06:27:26.897884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:20:39.476936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897252+0000","last_change":"2024-05-09T06:27:21.649626+0000","last_active":"2024-05-09T06:27:26.897252+0000","last_peered":"2024-05-09T06:27:26.897252+0000","last_clean":"2024-05-09T06:27:26.897252+0000","last_became_active":"2024-05-09T06:27:21.649332+0000","last_became_peered":"2024-05-09T06:27:21.649332+0000","last_unstale":"2024-05-09T06:27:26.897252+0000","last_undegraded":"2024-05-09T06:27:26.897252+0000","last_fullsized":"2024-05-09T06:27:26.897252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665959+0000","last_change":"2024-05-09T06:27:20.632572+0000","last_active":"2024-05-09T06:27:26.665959+0000","last_peered":"2024-05-09T06:27:26.665959+0000","last_clean":"2024-05-09T06:27:26.665959+0000","last_became_active":"2024-05-09T06:27:20.630286+0000","last_became_peered":"2024-05-09T06:27:20.630286+0000","last_unstale":"2024-05-09T06:27:26.665959+0000","last_undegraded":"2024-05-09T06:27:26.665959+0000","last_fullsized":"2024-05-09T06:27:26.665959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897808+0000","last_change":"2024-05-09T06:27:25.678228+0000","last_active":"2024-05-09T06:27:26.897808+0000","last_peered":"2024-05-09T06:27:26.897808+0000","last_clean":"2024-05-09T06:27:26.897808+0000","last_became_active":"2024-05-09T06:27:25.677638+0000","last_became_peered":"2024-05-09T06:27:25.677638+0000","last_unstale":"2024-05-09T06:27:26.897808+0000","last_undegraded":"2024-05-09T06:27:26.897808+0000","last_fullsized":"2024-05-09T06:27:26.897808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794917+0000","last_change":"2024-05-09T06:27:21.634999+0000","last_active":"2024-05-09T06:27:28.794917+0000","last_peered":"2024-05-09T06:27:28.794917+0000","last_clean":"2024-05-09T06:27:28.794917+0000","last_became_active":"2024-05-09T06:27:21.634688+0000","last_became_peered":"2024-05-09T06:27:21.634688+0000","last_unstale":"2024-05-09T06:27:28.794917+0000","last_undegraded":"2024-05-09T06:27:28.794917+0000","last_fullsized":"2024-05-09T06:27:28.794917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309529+0000","last_change":"2024-05-09T06:27:20.645301+0000","last_active":"2024-05-09T06:27:28.309529+0000","last_peered":"2024-05-09T06:27:28.309529+0000","last_clean":"2024-05-09T06:27:28.309529+0000","last_became_active":"2024-05-09T06:27:20.644309+0000","last_became_peered":"2024-05-09T06:27:20.644309+0000","last_unstale":"2024-05-09T06:27:28.309529+0000","last_undegraded":"2024-05-09T06:27:28.309529+0000","last_fullsized":"2024-05-09T06:27:28.309529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796649+0000","last_change":"2024-05-09T06:27:25.870805+0000","last_active":"2024-05-09T06:27:28.796649+0000","last_peered":"2024-05-09T06:27:28.796649+0000","last_clean":"2024-05-09T06:27:28.796649+0000","last_became_active":"2024-05-09T06:27:25.870405+0000","last_became_peered":"2024-05-09T06:27:25.870405+0000","last_unstale":"2024-05-09T06:27:28.796649+0000","last_undegraded":"2024-05-09T06:27:28.796649+0000","last_fullsized":"2024-05-09T06:27:28.796649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:22.208044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795597+0000","last_change":"2024-05-09T06:27:21.660077+0000","last_active":"2024-05-09T06:27:28.795597+0000","last_peered":"2024-05-09T06:27:28.795597+0000","last_clean":"2024-05-09T06:27:28.795597+0000","last_became_active":"2024-05-09T06:27:21.659816+0000","last_became_peered":"2024-05-09T06:27:21.659816+0000","last_unstale":"2024-05-09T06:27:28.795597+0000","last_undegraded":"2024-05-09T06:27:28.795597+0000","last_fullsized":"2024-05-09T06:27:28.795597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794955+0000","last_change":"2024-05-09T06:27:20.638709+0000","last_active":"2024-05-09T06:27:28.794955+0000","last_peered":"2024-05-09T06:27:28.794955+0000","last_clean":"2024-05-09T06:27:28.794955+0000","last_became_active":"2024-05-09T06:27:20.638404+0000","last_became_peered":"2024-05-09T06:27:20.638404+0000","last_unstale":"2024-05-09T06:27:28.794955+0000","last_undegraded":"2024-05-09T06:27:28.794955+0000","last_fullsized":"2024-05-09T06:27:28.794955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309694+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:27:28.309694+0000","last_peered":"2024-05-09T06:27:28.309694+0000","last_clean":"2024-05-09T06:27:28.309694+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:27:28.309694+0000","last_undegraded":"2024-05-09T06:27:28.309694+0000","last_fullsized":"2024-05-09T06:27:28.309694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795720+0000","last_change":"2024-05-09T06:27:25.871420+0000","last_active":"2024-05-09T06:27:28.795720+0000","last_peered":"2024-05-09T06:27:28.795720+0000","last_clean":"2024-05-09T06:27:28.795720+0000","last_became_active":"2024-05-09T06:27:25.870765+0000","last_became_peered":"2024-05-09T06:27:25.870765+0000","last_unstale":"2024-05-09T06:27:28.795720+0000","last_undegraded":"2024-05-09T06:27:28.795720+0000","last_fullsized":"2024-05-09T06:27:28.795720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:11.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310010+0000","last_change":"2024-05-09T06:27:21.665731+0000","last_active":"2024-05-09T06:27:28.310010+0000","last_peered":"2024-05-09T06:27:28.310010+0000","last_clean":"2024-05-09T06:27:28.310010+0000","last_became_active":"2024-05-09T06:27:21.665457+0000","last_became_peered":"2024-05-09T06:27:21.665457+0000","last_unstale":"2024-05-09T06:27:28.310010+0000","last_undegraded":"2024-05-09T06:27:28.310010+0000","last_fullsized":"2024-05-09T06:27:28.310010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666314+0000","last_change":"2024-05-09T06:27:20.639195+0000","last_active":"2024-05-09T06:27:26.666314+0000","last_peered":"2024-05-09T06:27:26.666314+0000","last_clean":"2024-05-09T06:27:26.666314+0000","last_became_active":"2024-05-09T06:27:20.638833+0000","last_became_peered":"2024-05-09T06:27:20.638833+0000","last_unstale":"2024-05-09T06:27:26.666314+0000","last_undegraded":"2024-05-09T06:27:26.666314+0000","last_fullsized":"2024-05-09T06:27:26.666314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795939+0000","last_change":"2024-05-09T06:27:25.868481+0000","last_active":"2024-05-09T06:27:28.795939+0000","last_peered":"2024-05-09T06:27:28.795939+0000","last_clean":"2024-05-09T06:27:28.795939+0000","last_became_active":"2024-05-09T06:27:25.867521+0000","last_became_peered":"2024-05-09T06:27:25.867521+0000","last_unstale":"2024-05-09T06:27:28.795939+0000","last_undegraded":"2024-05-09T06:27:28.795939+0000","last_fullsized":"2024-05-09T06:27:28.795939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:08.393833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796554+0000","last_change":"2024-05-09T06:27:21.663211+0000","last_active":"2024-05-09T06:27:28.796554+0000","last_peered":"2024-05-09T06:27:28.796554+0000","last_clean":"2024-05-09T06:27:28.796554+0000","last_became_active":"2024-05-09T06:27:21.662892+0000","last_became_peered":"2024-05-09T06:27:21.662892+0000","last_unstale":"2024-05-09T06:27:28.796554+0000","last_undegraded":"2024-05-09T06:27:28.796554+0000","last_fullsized":"2024-05-09T06:27:28.796554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897750+0000","last_change":"2024-05-09T06:27:20.638780+0000","last_active":"2024-05-09T06:27:26.897750+0000","last_peered":"2024-05-09T06:27:26.897750+0000","last_clean":"2024-05-09T06:27:26.897750+0000","last_became_active":"2024-05-09T06:27:20.638469+0000","last_became_peered":"2024-05-09T06:27:20.638469+0000","last_unstale":"2024-05-09T06:27:26.897750+0000","last_undegraded":"2024-05-09T06:27:26.897750+0000","last_fullsized":"2024-05-09T06:27:26.897750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:07.022180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795796+0000","last_change":"2024-05-09T06:27:25.673712+0000","last_active":"2024-05-09T06:27:28.795796+0000","last_peered":"2024-05-09T06:27:28.795796+0000","last_clean":"2024-05-09T06:27:28.795796+0000","last_became_active":"2024-05-09T06:27:25.673062+0000","last_became_peered":"2024-05-09T06:27:25.673062+0000","last_unstale":"2024-05-09T06:27:28.795796+0000","last_undegraded":"2024-05-09T06:27:28.795796+0000","last_fullsized":"2024-05-09T06:27:28.795796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310517+0000","last_change":"2024-05-09T06:27:21.652689+0000","last_active":"2024-05-09T06:27:28.310517+0000","last_peered":"2024-05-09T06:27:28.310517+0000","last_clean":"2024-05-09T06:27:28.310517+0000","last_became_active":"2024-05-09T06:27:21.652309+0000","last_became_peered":"2024-05-09T06:27:21.652309+0000","last_unstale":"2024-05-09T06:27:28.310517+0000","last_undegraded":"2024-05-09T06:27:28.310517+0000","last_fullsized":"2024-05-09T06:27:28.310517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:30.619441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666511+0000","last_change":"2024-05-09T06:27:20.631329+0000","last_active":"2024-05-09T06:27:26.666511+0000","last_peered":"2024-05-09T06:27:26.666511+0000","last_clean":"2024-05-09T06:27:26.666511+0000","last_became_active":"2024-05-09T06:27:20.631027+0000","last_became_peered":"2024-05-09T06:27:20.631027+0000","last_unstale":"2024-05-09T06:27:26.666511+0000","last_undegraded":"2024-05-09T06:27:26.666511+0000","last_fullsized":"2024-05-09T06:27:26.666511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795581+0000","last_change":"2024-05-09T06:27:25.868448+0000","last_active":"2024-05-09T06:27:28.795581+0000","last_peered":"2024-05-09T06:27:28.795581+0000","last_clean":"2024-05-09T06:27:28.795581+0000","last_became_active":"2024-05-09T06:27:25.867998+0000","last_became_peered":"2024-05-09T06:27:25.867998+0000","last_unstale":"2024-05-09T06:27:28.795581+0000","last_undegraded":"2024-05-09T06:27:28.795581+0000","last_fullsized":"2024-05-09T06:27:28.795581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:01.997341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310235+0000","last_change":"2024-05-09T06:27:21.654321+0000","last_active":"2024-05-09T06:27:28.310235+0000","last_peered":"2024-05-09T06:27:28.310235+0000","last_clean":"2024-05-09T06:27:28.310235+0000","last_became_active":"2024-05-09T06:27:21.654037+0000","last_became_peered":"2024-05-09T06:27:21.654037+0000","last_unstale":"2024-05-09T06:27:28.310235+0000","last_undegraded":"2024-05-09T06:27:28.310235+0000","last_fullsized":"2024-05-09T06:27:28.310235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667634+0000","last_change":"2024-05-09T06:27:20.634285+0000","last_active":"2024-05-09T06:27:26.667634+0000","last_peered":"2024-05-09T06:27:26.667634+0000","last_clean":"2024-05-09T06:27:26.667634+0000","last_became_active":"2024-05-09T06:27:20.633961+0000","last_became_peered":"2024-05-09T06:27:20.633961+0000","last_unstale":"2024-05-09T06:27:26.667634+0000","last_undegraded":"2024-05-09T06:27:26.667634+0000","last_fullsized":"2024-05-09T06:27:26.667634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:22.773284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666362+0000","last_change":"2024-05-09T06:27:25.666556+0000","last_active":"2024-05-09T06:27:26.666362+0000","last_peered":"2024-05-09T06:27:26.666362+0000","last_clean":"2024-05-09T06:27:26.666362+0000","last_became_active":"2024-05-09T06:27:25.666086+0000","last_became_peered":"2024-05-09T06:27:25.666086+0000","last_unstale":"2024-05-09T06:27:26.666362+0000","last_undegraded":"2024-05-09T06:27:26.666362+0000","last_fullsized":"2024-05-09T06:27:26.666362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:16.228053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309897+0000","last_change":"2024-05-09T06:27:21.653860+0000","last_active":"2024-05-09T06:27:28.309897+0000","last_peered":"2024-05-09T06:27:28.309897+0000","last_clean":"2024-05-09T06:27:28.309897+0000","last_became_active":"2024-05-09T06:27:21.653222+0000","last_became_peered":"2024-05-09T06:27:21.653222+0000","last_unstale":"2024-05-09T06:27:28.309897+0000","last_undegraded":"2024-05-09T06:27:28.309897+0000","last_fullsized":"2024-05-09T06:27:28.309897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665751+0000","last_change":"2024-05-09T06:27:20.632271+0000","last_active":"2024-05-09T06:27:26.665751+0000","last_peered":"2024-05-09T06:27:26.665751+0000","last_clean":"2024-05-09T06:27:26.665751+0000","last_became_active":"2024-05-09T06:27:20.631965+0000","last_became_peered":"2024-05-09T06:27:20.631965+0000","last_unstale":"2024-05-09T06:27:26.665751+0000","last_undegraded":"2024-05-09T06:27:26.665751+0000","last_fullsized":"2024-05-09T06:27:26.665751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796544+0000","last_change":"2024-05-09T06:27:25.673394+0000","last_active":"2024-05-09T06:27:28.796544+0000","last_peered":"2024-05-09T06:27:28.796544+0000","last_clean":"2024-05-09T06:27:28.796544+0000","last_became_active":"2024-05-09T06:27:25.672824+0000","last_became_peered":"2024-05-09T06:27:25.672824+0000","last_unstale":"2024-05-09T06:27:28.796544+0000","last_undegraded":"2024-05-09T06:27:28.796544+0000","last_fullsized":"2024-05-09T06:27:28.796544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795660+0000","last_change":"2024-05-09T06:27:21.658568+0000","last_active":"2024-05-09T06:27:28.795660+0000","last_peered":"2024-05-09T06:27:28.795660+0000","last_clean":"2024-05-09T06:27:28.795660+0000","last_became_active":"2024-05-09T06:27:21.658289+0000","last_became_peered":"2024-05-09T06:27:21.658289+0000","last_unstale":"2024-05-09T06:27:28.795660+0000","last_undegraded":"2024-05-09T06:27:28.795660+0000","last_fullsized":"2024-05-09T06:27:28.795660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.637847+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.637553+0000","last_became_peered":"2024-05-09T06:27:20.637553+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:57:51.515755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795481+0000","last_change":"2024-05-09T06:27:25.872027+0000","last_active":"2024-05-09T06:27:28.795481+0000","last_peered":"2024-05-09T06:27:28.795481+0000","last_clean":"2024-05-09T06:27:28.795481+0000","last_became_active":"2024-05-09T06:27:25.871616+0000","last_became_peered":"2024-05-09T06:27:25.871616+0000","last_unstale":"2024-05-09T06:27:28.795481+0000","last_undegraded":"2024-05-09T06:27:28.795481+0000","last_fullsized":"2024-05-09T06:27:28.795481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:58:18.735834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309942+0000","last_change":"2024-05-09T06:27:21.655596+0000","last_active":"2024-05-09T06:27:28.309942+0000","last_peered":"2024-05-09T06:27:28.309942+0000","last_clean":"2024-05-09T06:27:28.309942+0000","last_became_active":"2024-05-09T06:27:21.655327+0000","last_became_peered":"2024-05-09T06:27:21.655327+0000","last_unstale":"2024-05-09T06:27:28.309942+0000","last_undegraded":"2024-05-09T06:27:28.309942+0000","last_fullsized":"2024-05-09T06:27:28.309942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:07.475580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796614+0000","last_change":"2024-05-09T06:27:20.640461+0000","last_active":"2024-05-09T06:27:28.796614+0000","last_peered":"2024-05-09T06:27:28.796614+0000","last_clean":"2024-05-09T06:27:28.796614+0000","last_became_active":"2024-05-09T06:27:20.640164+0000","last_became_peered":"2024-05-09T06:27:20.640164+0000","last_unstale":"2024-05-09T06:27:28.796614+0000","last_undegraded":"2024-05-09T06:27:28.796614+0000","last_fullsized":"2024-05-09T06:27:28.796614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795963+0000","last_change":"2024-05-09T06:27:25.868054+0000","last_active":"2024-05-09T06:27:28.795963+0000","last_peered":"2024-05-09T06:27:28.795963+0000","last_clean":"2024-05-09T06:27:28.795963+0000","last_became_active":"2024-05-09T06:27:25.867638+0000","last_became_peered":"2024-05-09T06:27:25.867638+0000","last_unstale":"2024-05-09T06:27:28.795963+0000","last_undegraded":"2024-05-09T06:27:28.795963+0000","last_fullsized":"2024-05-09T06:27:28.795963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:05.780930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795875+0000","last_change":"2024-05-09T06:27:21.640283+0000","last_active":"2024-05-09T06:27:28.795875+0000","last_peered":"2024-05-09T06:27:28.795875+0000","last_clean":"2024-05-09T06:27:28.795875+0000","last_became_active":"2024-05-09T06:27:21.639981+0000","last_became_peered":"2024-05-09T06:27:21.639981+0000","last_unstale":"2024-05-09T06:27:28.795875+0000","last_undegraded":"2024-05-09T06:27:28.795875+0000","last_fullsized":"2024-05-09T06:27:28.795875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:57.154069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794456+0000","last_change":"2024-05-09T06:27:20.636469+0000","last_active":"2024-05-09T06:27:28.794456+0000","last_peered":"2024-05-09T06:27:28.794456+0000","last_clean":"2024-05-09T06:27:28.794456+0000","last_became_active":"2024-05-09T06:27:20.636200+0000","last_became_peered":"2024-05-09T06:27:20.636200+0000","last_unstale":"2024-05-09T06:27:28.794456+0000","last_undegraded":"2024-05-09T06:27:28.794456+0000","last_fullsized":"2024-05-09T06:27:28.794456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:02.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796704+0000","last_change":"2024-05-09T06:27:25.868001+0000","last_active":"2024-05-09T06:27:28.796704+0000","last_peered":"2024-05-09T06:27:28.796704+0000","last_clean":"2024-05-09T06:27:28.796704+0000","last_became_active":"2024-05-09T06:27:25.867561+0000","last_became_peered":"2024-05-09T06:27:25.867561+0000","last_unstale":"2024-05-09T06:27:28.796704+0000","last_undegraded":"2024-05-09T06:27:28.796704+0000","last_fullsized":"2024-05-09T06:27:28.796704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310020+0000","last_change":"2024-05-09T06:27:21.647526+0000","last_active":"2024-05-09T06:27:28.310020+0000","last_peered":"2024-05-09T06:27:28.310020+0000","last_clean":"2024-05-09T06:27:28.310020+0000","last_became_active":"2024-05-09T06:27:21.646721+0000","last_became_peered":"2024-05-09T06:27:21.646721+0000","last_unstale":"2024-05-09T06:27:28.310020+0000","last_undegraded":"2024-05-09T06:27:28.310020+0000","last_fullsized":"2024-05-09T06:27:28.310020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:31.226892+0000","last_change":"2024-05-09T06:27:20.634111+0000","last_active":"2024-05-09T06:27:31.226892+0000","last_peered":"2024-05-09T06:27:31.226892+0000","last_clean":"2024-05-09T06:27:31.226892+0000","last_became_active":"2024-05-09T06:27:20.633772+0000","last_became_peered":"2024-05-09T06:27:20.633772+0000","last_unstale":"2024-05-09T06:27:31.226892+0000","last_undegraded":"2024-05-09T06:27:31.226892+0000","last_fullsized":"2024-05-09T06:27:31.226892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:04:58.870334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897702+0000","last_change":"2024-05-09T06:27:25.676189+0000","last_active":"2024-05-09T06:27:26.897702+0000","last_peered":"2024-05-09T06:27:26.897702+0000","last_clean":"2024-05-09T06:27:26.897702+0000","last_became_active":"2024-05-09T06:27:25.672390+0000","last_became_peered":"2024-05-09T06:27:25.672390+0000","last_unstale":"2024-05-09T06:27:26.897702+0000","last_undegraded":"2024-05-09T06:27:26.897702+0000","last_fullsized":"2024-05-09T06:27:26.897702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:34.648244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667630+0000","last_change":"2024-05-09T06:27:21.637363+0000","last_active":"2024-05-09T06:27:26.667630+0000","last_peered":"2024-05-09T06:27:26.667630+0000","last_clean":"2024-05-09T06:27:26.667630+0000","last_became_active":"2024-05-09T06:27:21.637056+0000","last_became_peered":"2024-05-09T06:27:21.637056+0000","last_unstale":"2024-05-09T06:27:26.667630+0000","last_undegraded":"2024-05-09T06:27:26.667630+0000","last_fullsized":"2024-05-09T06:27:26.667630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897618+0000","last_change":"2024-05-09T06:27:20.641960+0000","last_active":"2024-05-09T06:27:26.897618+0000","last_peered":"2024-05-09T06:27:26.897618+0000","last_clean":"2024-05-09T06:27:26.897618+0000","last_became_active":"2024-05-09T06:27:20.641670+0000","last_became_peered":"2024-05-09T06:27:20.641670+0000","last_unstale":"2024-05-09T06:27:26.897618+0000","last_undegraded":"2024-05-09T06:27:26.897618+0000","last_fullsized":"2024-05-09T06:27:26.897618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:36.215437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667165+0000","last_change":"2024-05-09T06:27:25.675301+0000","last_active":"2024-05-09T06:27:26.667165+0000","last_peered":"2024-05-09T06:27:26.667165+0000","last_clean":"2024-05-09T06:27:26.667165+0000","last_became_active":"2024-05-09T06:27:25.674818+0000","last_became_peered":"2024-05-09T06:27:25.674818+0000","last_unstale":"2024-05-09T06:27:26.667165+0000","last_undegraded":"2024-05-09T06:27:26.667165+0000","last_fullsized":"2024-05-09T06:27:26.667165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:06.045330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796636+0000","last_change":"2024-05-09T06:27:21.638651+0000","last_active":"2024-05-09T06:27:28.796636+0000","last_peered":"2024-05-09T06:27:28.796636+0000","last_clean":"2024-05-09T06:27:28.796636+0000","last_became_active":"2024-05-09T06:27:21.638111+0000","last_became_peered":"2024-05-09T06:27:21.638111+0000","last_unstale":"2024-05-09T06:27:28.796636+0000","last_undegraded":"2024-05-09T06:27:28.796636+0000","last_fullsized":"2024-05-09T06:27:28.796636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796815+0000","last_change":"2024-05-09T06:27:20.639625+0000","last_active":"2024-05-09T06:27:28.796815+0000","last_peered":"2024-05-09T06:27:28.796815+0000","last_clean":"2024-05-09T06:27:28.796815+0000","last_became_active":"2024-05-09T06:27:20.639287+0000","last_became_peered":"2024-05-09T06:27:20.639287+0000","last_unstale":"2024-05-09T06:27:28.796815+0000","last_undegraded":"2024-05-09T06:27:28.796815+0000","last_fullsized":"2024-05-09T06:27:28.796815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796227+0000","last_change":"2024-05-09T06:27:25.669479+0000","last_active":"2024-05-09T06:27:28.796227+0000","last_peered":"2024-05-09T06:27:28.796227+0000","last_clean":"2024-05-09T06:27:28.796227+0000","last_became_active":"2024-05-09T06:27:25.668634+0000","last_became_peered":"2024-05-09T06:27:25.668634+0000","last_unstale":"2024-05-09T06:27:28.796227+0000","last_undegraded":"2024-05-09T06:27:28.796227+0000","last_fullsized":"2024-05-09T06:27:28.796227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:33.708720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796768+0000","last_change":"2024-05-09T06:27:21.657985+0000","last_active":"2024-05-09T06:27:28.796768+0000","last_peered":"2024-05-09T06:27:28.796768+0000","last_clean":"2024-05-09T06:27:28.796768+0000","last_became_active":"2024-05-09T06:27:21.653693+0000","last_became_peered":"2024-05-09T06:27:21.653693+0000","last_unstale":"2024-05-09T06:27:28.796768+0000","last_undegraded":"2024-05-09T06:27:28.796768+0000","last_fullsized":"2024-05-09T06:27:28.796768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796692+0000","last_change":"2024-05-09T06:27:20.639702+0000","last_active":"2024-05-09T06:27:28.796692+0000","last_peered":"2024-05-09T06:27:28.796692+0000","last_clean":"2024-05-09T06:27:28.796692+0000","last_became_active":"2024-05-09T06:27:20.639361+0000","last_became_peered":"2024-05-09T06:27:20.639361+0000","last_unstale":"2024-05-09T06:27:28.796692+0000","last_undegraded":"2024-05-09T06:27:28.796692+0000","last_fullsized":"2024-05-09T06:27:28.796692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:25.867964+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:25.867579+0000","last_became_peered":"2024-05-09T06:27:25.867579+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:06.555652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796481+0000","last_change":"2024-05-09T06:27:25.871150+0000","last_active":"2024-05-09T06:27:28.796481+0000","last_peered":"2024-05-09T06:27:28.796481+0000","last_clean":"2024-05-09T06:27:28.796481+0000","last_became_active":"2024-05-09T06:27:25.870707+0000","last_became_peered":"2024-05-09T06:27:25.870707+0000","last_unstale":"2024-05-09T06:27:28.796481+0000","last_undegraded":"2024-05-09T06:27:28.796481+0000","last_fullsized":"2024-05-09T06:27:28.796481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:54.627889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897140+0000","last_change":"2024-05-09T06:27:21.648383+0000","last_active":"2024-05-09T06:27:26.897140+0000","last_peered":"2024-05-09T06:27:26.897140+0000","last_clean":"2024-05-09T06:27:26.897140+0000","last_became_active":"2024-05-09T06:27:21.646149+0000","last_became_peered":"2024-05-09T06:27:21.646149+0000","last_unstale":"2024-05-09T06:27:26.897140+0000","last_undegraded":"2024-05-09T06:27:26.897140+0000","last_fullsized":"2024-05-09T06:27:26.897140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794935+0000","last_change":"2024-05-09T06:27:20.639240+0000","last_active":"2024-05-09T06:27:28.794935+0000","last_peered":"2024-05-09T06:27:28.794935+0000","last_clean":"2024-05-09T06:27:28.794935+0000","last_became_active":"2024-05-09T06:27:20.638885+0000","last_became_peered":"2024-05-09T06:27:20.638885+0000","last_unstale":"2024-05-09T06:27:28.794935+0000","last_undegraded":"2024-05-09T06:27:28.794935+0000","last_fullsized":"2024-05-09T06:27:28.794935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796512+0000","last_change":"2024-05-09T06:27:25.673610+0000","last_active":"2024-05-09T06:27:28.796512+0000","last_peered":"2024-05-09T06:27:28.796512+0000","last_clean":"2024-05-09T06:27:28.796512+0000","last_became_active":"2024-05-09T06:27:25.673160+0000","last_became_peered":"2024-05-09T06:27:25.673160+0000","last_unstale":"2024-05-09T06:27:28.796512+0000","last_undegraded":"2024-05-09T06:27:28.796512+0000","last_fullsized":"2024-05-09T06:27:28.796512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:50.303726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796199+0000","last_change":"2024-05-09T06:27:21.645047+0000","last_active":"2024-05-09T06:27:28.796199+0000","last_peered":"2024-05-09T06:27:28.796199+0000","last_clean":"2024-05-09T06:27:28.796199+0000","last_became_active":"2024-05-09T06:27:21.644743+0000","last_became_peered":"2024-05-09T06:27:21.644743+0000","last_unstale":"2024-05-09T06:27:28.796199+0000","last_undegraded":"2024-05-09T06:27:28.796199+0000","last_fullsized":"2024-05-09T06:27:28.796199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795221+0000","last_change":"2024-05-09T06:27:20.635683+0000","last_active":"2024-05-09T06:27:28.795221+0000","last_peered":"2024-05-09T06:27:28.795221+0000","last_clean":"2024-05-09T06:27:28.795221+0000","last_became_active":"2024-05-09T06:27:20.635386+0000","last_became_peered":"2024-05-09T06:27:20.635386+0000","last_unstale":"2024-05-09T06:27:28.795221+0000","last_undegraded":"2024-05-09T06:27:28.795221+0000","last_fullsized":"2024-05-09T06:27:28.795221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:46.960514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.794910+0000","last_change":"2024-05-09T06:27:20.642531+0000","last_active":"2024-05-09T06:27:28.794910+0000","last_peered":"2024-05-09T06:27:28.794910+0000","last_clean":"2024-05-09T06:27:28.794910+0000","last_became_active":"2024-05-09T06:27:20.642176+0000","last_became_peered":"2024-05-09T06:27:20.642176+0000","last_unstale":"2024-05-09T06:27:28.794910+0000","last_undegraded":"2024-05-09T06:27:28.794910+0000","last_fullsized":"2024-05-09T06:27:28.794910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:03.959024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795979+0000","last_change":"2024-05-09T06:27:21.634797+0000","last_active":"2024-05-09T06:27:28.795979+0000","last_peered":"2024-05-09T06:27:28.795979+0000","last_clean":"2024-05-09T06:27:28.795979+0000","last_became_active":"2024-05-09T06:27:21.634458+0000","last_became_peered":"2024-05-09T06:27:21.634458+0000","last_unstale":"2024-05-09T06:27:28.795979+0000","last_undegraded":"2024-05-09T06:27:28.795979+0000","last_fullsized":"2024-05-09T06:27:28.795979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666255+0000","last_change":"2024-05-09T06:27:25.678471+0000","last_active":"2024-05-09T06:27:26.666255+0000","last_peered":"2024-05-09T06:27:26.666255+0000","last_clean":"2024-05-09T06:27:26.666255+0000","last_became_active":"2024-05-09T06:27:25.677981+0000","last_became_peered":"2024-05-09T06:27:25.677981+0000","last_unstale":"2024-05-09T06:27:26.666255+0000","last_undegraded":"2024-05-09T06:27:26.666255+0000","last_fullsized":"2024-05-09T06:27:26.666255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:20.609371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795054+0000","last_change":"2024-05-09T06:27:20.644121+0000","last_active":"2024-05-09T06:27:28.795054+0000","last_peered":"2024-05-09T06:27:28.795054+0000","last_clean":"2024-05-09T06:27:28.795054+0000","last_became_active":"2024-05-09T06:27:20.643795+0000","last_became_peered":"2024-05-09T06:27:20.643795+0000","last_unstale":"2024-05-09T06:27:28.795054+0000","last_undegraded":"2024-05-09T06:27:28.795054+0000","last_fullsized":"2024-05-09T06:27:28.795054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:19.611416+0000","last_clean_scrub_stamp":"2024-05-09T06:27:19.611416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:04.460444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666578+0000","last_change":"2024-05-09T06:27:21.652395+0000","last_active":"2024-05-09T06:27:26.666578+0000","last_peered":"2024-05-09T06:27:26.666578+0000","last_clean":"2024-05-09T06:27:26.666578+0000","last_became_active":"2024-05-09T06:27:21.652019+0000","last_became_peered":"2024-05-09T06:27:21.652019+0000","last_unstale":"2024-05-09T06:27:26.666578+0000","last_undegraded":"2024-05-09T06:27:26.666578+0000","last_fullsized":"2024-05-09T06:27:26.666578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:37.219520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795669+0000","last_change":"2024-05-09T06:27:25.675291+0000","last_active":"2024-05-09T06:27:28.795669+0000","last_peered":"2024-05-09T06:27:28.795669+0000","last_clean":"2024-05-09T06:27:28.795669+0000","last_became_active":"2024-05-09T06:27:25.674422+0000","last_became_peered":"2024-05-09T06:27:25.674422+0000","last_unstale":"2024-05-09T06:27:28.795669+0000","last_undegraded":"2024-05-09T06:27:28.795669+0000","last_fullsized":"2024-05-09T06:27:28.795669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:24.644302+0000","last_clean_scrub_stamp":"2024-05-09T06:27:24.644302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:18.601362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795268+0000","last_change":"2024-05-09T06:27:21.659311+0000","last_active":"2024-05-09T06:27:28.795268+0000","last_peered":"2024-05-09T06:27:28.795268+0000","last_clean":"2024-05-09T06:27:28.795268+0000","last_became_active":"2024-05-09T06:27:21.659039+0000","last_became_peered":"2024-05-09T06:27:21.659039+0000","last_unstale":"2024-05-09T06:27:28.795268+0000","last_undegraded":"2024-05-09T06:27:28.795268+0000","last_fullsized":"2024-05-09T06:27:28.795268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.665756+0000","last_change":"2024-05-09T06:27:21.656834+0000","last_active":"2024-05-09T06:27:26.665756+0000","last_peered":"2024-05-09T06:27:26.665756+0000","last_clean":"2024-05-09T06:27:26.665756+0000","last_became_active":"2024-05-09T06:27:21.655466+0000","last_became_peered":"2024-05-09T06:27:21.655466+0000","last_unstale":"2024-05-09T06:27:26.665756+0000","last_undegraded":"2024-05-09T06:27:26.665756+0000","last_fullsized":"2024-05-09T06:27:26.665756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796581+0000","last_change":"2024-05-09T06:27:21.665177+0000","last_active":"2024-05-09T06:27:28.796581+0000","last_peered":"2024-05-09T06:27:28.796581+0000","last_clean":"2024-05-09T06:27:28.796581+0000","last_became_active":"2024-05-09T06:27:21.663959+0000","last_became_peered":"2024-05-09T06:27:21.663959+0000","last_unstale":"2024-05-09T06:27:28.796581+0000","last_undegraded":"2024-05-09T06:27:28.796581+0000","last_fullsized":"2024-05-09T06:27:28.796581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:55.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795891+0000","last_change":"2024-05-09T06:27:21.642136+0000","last_active":"2024-05-09T06:27:28.795891+0000","last_peered":"2024-05-09T06:27:28.795891+0000","last_clean":"2024-05-09T06:27:28.795891+0000","last_became_active":"2024-05-09T06:27:21.641855+0000","last_became_peered":"2024-05-09T06:27:21.641855+0000","last_unstale":"2024-05-09T06:27:28.795891+0000","last_undegraded":"2024-05-09T06:27:28.795891+0000","last_fullsized":"2024-05-09T06:27:28.795891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:59.087180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796263+0000","last_change":"2024-05-09T06:27:21.660067+0000","last_active":"2024-05-09T06:27:28.796263+0000","last_peered":"2024-05-09T06:27:28.796263+0000","last_clean":"2024-05-09T06:27:28.796263+0000","last_became_active":"2024-05-09T06:27:21.651525+0000","last_became_peered":"2024-05-09T06:27:21.651525+0000","last_unstale":"2024-05-09T06:27:28.796263+0000","last_undegraded":"2024-05-09T06:27:28.796263+0000","last_fullsized":"2024-05-09T06:27:28.796263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:02.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897431+0000","last_change":"2024-05-09T06:27:21.650694+0000","last_active":"2024-05-09T06:27:26.897431+0000","last_peered":"2024-05-09T06:27:26.897431+0000","last_clean":"2024-05-09T06:27:26.897431+0000","last_became_active":"2024-05-09T06:27:21.650404+0000","last_became_peered":"2024-05-09T06:27:21.650404+0000","last_unstale":"2024-05-09T06:27:26.897431+0000","last_undegraded":"2024-05-09T06:27:26.897431+0000","last_fullsized":"2024-05-09T06:27:26.897431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796327+0000","last_change":"2024-05-09T06:27:21.648723+0000","last_active":"2024-05-09T06:27:28.796327+0000","last_peered":"2024-05-09T06:27:28.796327+0000","last_clean":"2024-05-09T06:27:28.796327+0000","last_became_active":"2024-05-09T06:27:21.648434+0000","last_became_peered":"2024-05-09T06:27:21.648434+0000","last_unstale":"2024-05-09T06:27:28.796327+0000","last_undegraded":"2024-05-09T06:27:28.796327+0000","last_fullsized":"2024-05-09T06:27:28.796327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795585+0000","last_change":"2024-05-09T06:27:21.659502+0000","last_active":"2024-05-09T06:27:28.795585+0000","last_peered":"2024-05-09T06:27:28.795585+0000","last_clean":"2024-05-09T06:27:28.795585+0000","last_became_active":"2024-05-09T06:27:21.651268+0000","last_became_peered":"2024-05-09T06:27:21.651268+0000","last_unstale":"2024-05-09T06:27:28.795585+0000","last_undegraded":"2024-05-09T06:27:28.795585+0000","last_fullsized":"2024-05-09T06:27:28.795585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795603+0000","last_change":"2024-05-09T06:27:21.664539+0000","last_active":"2024-05-09T06:27:28.795603+0000","last_peered":"2024-05-09T06:27:28.795603+0000","last_clean":"2024-05-09T06:27:28.795603+0000","last_became_active":"2024-05-09T06:27:21.664267+0000","last_became_peered":"2024-05-09T06:27:21.664267+0000","last_unstale":"2024-05-09T06:27:28.795603+0000","last_undegraded":"2024-05-09T06:27:28.795603+0000","last_fullsized":"2024-05-09T06:27:28.795603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795097+0000","last_change":"2024-05-09T06:27:21.658842+0000","last_active":"2024-05-09T06:27:28.795097+0000","last_peered":"2024-05-09T06:27:28.795097+0000","last_clean":"2024-05-09T06:27:28.795097+0000","last_became_active":"2024-05-09T06:27:21.658522+0000","last_became_peered":"2024-05-09T06:27:21.658522+0000","last_unstale":"2024-05-09T06:27:28.795097+0000","last_undegraded":"2024-05-09T06:27:28.795097+0000","last_fullsized":"2024-05-09T06:27:28.795097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796440+0000","last_change":"2024-05-09T06:27:21.657936+0000","last_active":"2024-05-09T06:27:28.796440+0000","last_peered":"2024-05-09T06:27:28.796440+0000","last_clean":"2024-05-09T06:27:28.796440+0000","last_became_active":"2024-05-09T06:27:21.654771+0000","last_became_peered":"2024-05-09T06:27:21.654771+0000","last_unstale":"2024-05-09T06:27:28.796440+0000","last_undegraded":"2024-05-09T06:27:28.796440+0000","last_fullsized":"2024-05-09T06:27:28.796440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:12.199250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795635+0000","last_change":"2024-05-09T06:27:21.645805+0000","last_active":"2024-05-09T06:27:28.795635+0000","last_peered":"2024-05-09T06:27:28.795635+0000","last_clean":"2024-05-09T06:27:28.795635+0000","last_became_active":"2024-05-09T06:27:21.643759+0000","last_became_peered":"2024-05-09T06:27:21.643759+0000","last_unstale":"2024-05-09T06:27:28.795635+0000","last_undegraded":"2024-05-09T06:27:28.795635+0000","last_fullsized":"2024-05-09T06:27:28.795635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:12.851183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795676+0000","last_change":"2024-05-09T06:27:21.654407+0000","last_active":"2024-05-09T06:27:28.795676+0000","last_peered":"2024-05-09T06:27:28.795676+0000","last_clean":"2024-05-09T06:27:28.795676+0000","last_became_active":"2024-05-09T06:27:21.654126+0000","last_became_peered":"2024-05-09T06:27:21.654126+0000","last_unstale":"2024-05-09T06:27:28.795676+0000","last_undegraded":"2024-05-09T06:27:28.795676+0000","last_fullsized":"2024-05-09T06:27:28.795676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:35.546658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310344+0000","last_change":"2024-05-09T06:27:21.651773+0000","last_active":"2024-05-09T06:27:28.310344+0000","last_peered":"2024-05-09T06:27:28.310344+0000","last_clean":"2024-05-09T06:27:28.310344+0000","last_became_active":"2024-05-09T06:27:21.650069+0000","last_became_peered":"2024-05-09T06:27:21.650069+0000","last_unstale":"2024-05-09T06:27:28.310344+0000","last_undegraded":"2024-05-09T06:27:28.310344+0000","last_fullsized":"2024-05-09T06:27:28.310344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:04.365030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.666446+0000","last_change":"2024-05-09T06:27:21.657123+0000","last_active":"2024-05-09T06:27:26.666446+0000","last_peered":"2024-05-09T06:27:26.666446+0000","last_clean":"2024-05-09T06:27:26.666446+0000","last_became_active":"2024-05-09T06:27:21.656787+0000","last_became_peered":"2024-05-09T06:27:21.656787+0000","last_unstale":"2024-05-09T06:27:26.666446+0000","last_undegraded":"2024-05-09T06:27:26.666446+0000","last_fullsized":"2024-05-09T06:27:26.666446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:07.660596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.797072+0000","last_change":"2024-05-09T06:27:21.660613+0000","last_active":"2024-05-09T06:27:28.797072+0000","last_peered":"2024-05-09T06:27:28.797072+0000","last_clean":"2024-05-09T06:27:28.797072+0000","last_became_active":"2024-05-09T06:27:21.654942+0000","last_became_peered":"2024-05-09T06:27:21.654942+0000","last_unstale":"2024-05-09T06:27:28.797072+0000","last_undegraded":"2024-05-09T06:27:28.797072+0000","last_fullsized":"2024-05-09T06:27:28.797072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795476+0000","last_change":"2024-05-09T06:27:21.658261+0000","last_active":"2024-05-09T06:27:28.795476+0000","last_peered":"2024-05-09T06:27:28.795476+0000","last_clean":"2024-05-09T06:27:28.795476+0000","last_became_active":"2024-05-09T06:27:21.658004+0000","last_became_peered":"2024-05-09T06:27:21.658004+0000","last_unstale":"2024-05-09T06:27:28.795476+0000","last_undegraded":"2024-05-09T06:27:28.795476+0000","last_fullsized":"2024-05-09T06:27:28.795476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:01.203183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897230+0000","last_change":"2024-05-09T06:27:21.654019+0000","last_active":"2024-05-09T06:27:26.897230+0000","last_peered":"2024-05-09T06:27:26.897230+0000","last_clean":"2024-05-09T06:27:26.897230+0000","last_became_active":"2024-05-09T06:27:21.653711+0000","last_became_peered":"2024-05-09T06:27:21.653711+0000","last_unstale":"2024-05-09T06:27:26.897230+0000","last_undegraded":"2024-05-09T06:27:26.897230+0000","last_fullsized":"2024-05-09T06:27:26.897230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795778+0000","last_change":"2024-05-09T06:27:21.645328+0000","last_active":"2024-05-09T06:27:28.795778+0000","last_peered":"2024-05-09T06:27:28.795778+0000","last_clean":"2024-05-09T06:27:28.795778+0000","last_became_active":"2024-05-09T06:27:21.644287+0000","last_became_peered":"2024-05-09T06:27:21.644287+0000","last_unstale":"2024-05-09T06:27:28.795778+0000","last_undegraded":"2024-05-09T06:27:28.795778+0000","last_fullsized":"2024-05-09T06:27:28.795778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:16.580568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795360+0000","last_change":"2024-05-09T06:27:21.643190+0000","last_active":"2024-05-09T06:27:28.795360+0000","last_peered":"2024-05-09T06:27:28.795360+0000","last_clean":"2024-05-09T06:27:28.795360+0000","last_became_active":"2024-05-09T06:27:21.642700+0000","last_became_peered":"2024-05-09T06:27:21.642700+0000","last_unstale":"2024-05-09T06:27:28.795360+0000","last_undegraded":"2024-05-09T06:27:28.795360+0000","last_fullsized":"2024-05-09T06:27:28.795360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.667413+0000","last_change":"2024-05-09T06:27:21.652745+0000","last_active":"2024-05-09T06:27:26.667413+0000","last_peered":"2024-05-09T06:27:26.667413+0000","last_clean":"2024-05-09T06:27:26.667413+0000","last_became_active":"2024-05-09T06:27:21.651219+0000","last_became_peered":"2024-05-09T06:27:21.651219+0000","last_unstale":"2024-05-09T06:27:26.667413+0000","last_undegraded":"2024-05-09T06:27:26.667413+0000","last_fullsized":"2024-05-09T06:27:26.667413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:08.026264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795888+0000","last_change":"2024-05-09T06:27:21.663110+0000","last_active":"2024-05-09T06:27:28.795888+0000","last_peered":"2024-05-09T06:27:28.795888+0000","last_clean":"2024-05-09T06:27:28.795888+0000","last_became_active":"2024-05-09T06:27:21.662731+0000","last_became_peered":"2024-05-09T06:27:21.662731+0000","last_unstale":"2024-05-09T06:27:28.795888+0000","last_undegraded":"2024-05-09T06:27:28.795888+0000","last_fullsized":"2024-05-09T06:27:28.795888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796301+0000","last_change":"2024-05-09T06:27:21.659071+0000","last_active":"2024-05-09T06:27:28.796301+0000","last_peered":"2024-05-09T06:27:28.796301+0000","last_clean":"2024-05-09T06:27:28.796301+0000","last_became_active":"2024-05-09T06:27:21.658774+0000","last_became_peered":"2024-05-09T06:27:21.658774+0000","last_unstale":"2024-05-09T06:27:28.796301+0000","last_undegraded":"2024-05-09T06:27:28.796301+0000","last_fullsized":"2024-05-09T06:27:28.796301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310458+0000","last_change":"2024-05-09T06:27:21.665023+0000","last_active":"2024-05-09T06:27:28.310458+0000","last_peered":"2024-05-09T06:27:28.310458+0000","last_clean":"2024-05-09T06:27:28.310458+0000","last_became_active":"2024-05-09T06:27:21.664464+0000","last_became_peered":"2024-05-09T06:27:21.664464+0000","last_unstale":"2024-05-09T06:27:28.310458+0000","last_undegraded":"2024-05-09T06:27:28.310458+0000","last_fullsized":"2024-05-09T06:27:28.310458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:50.599086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.795913+0000","last_change":"2024-05-09T06:27:21.661852+0000","last_active":"2024-05-09T06:27:28.795913+0000","last_peered":"2024-05-09T06:27:28.795913+0000","last_clean":"2024-05-09T06:27:28.795913+0000","last_became_active":"2024-05-09T06:27:21.660313+0000","last_became_peered":"2024-05-09T06:27:21.660313+0000","last_unstale":"2024-05-09T06:27:28.795913+0000","last_undegraded":"2024-05-09T06:27:28.795913+0000","last_fullsized":"2024-05-09T06:27:28.795913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:43.940944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309769+0000","last_change":"2024-05-09T06:27:21.654130+0000","last_active":"2024-05-09T06:27:28.309769+0000","last_peered":"2024-05-09T06:27:28.309769+0000","last_clean":"2024-05-09T06:27:28.309769+0000","last_became_active":"2024-05-09T06:27:21.653593+0000","last_became_peered":"2024-05-09T06:27:21.653593+0000","last_unstale":"2024-05-09T06:27:28.309769+0000","last_undegraded":"2024-05-09T06:27:28.309769+0000","last_fullsized":"2024-05-09T06:27:28.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:52.978398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796010+0000","last_change":"2024-05-09T06:27:21.654775+0000","last_active":"2024-05-09T06:27:28.796010+0000","last_peered":"2024-05-09T06:27:28.796010+0000","last_clean":"2024-05-09T06:27:28.796010+0000","last_became_active":"2024-05-09T06:27:21.654269+0000","last_became_peered":"2024-05-09T06:27:21.654269+0000","last_unstale":"2024-05-09T06:27:28.796010+0000","last_undegraded":"2024-05-09T06:27:28.796010+0000","last_fullsized":"2024-05-09T06:27:28.796010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:12.131828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.309790+0000","last_change":"2024-05-09T06:27:21.649923+0000","last_active":"2024-05-09T06:27:28.309790+0000","last_peered":"2024-05-09T06:27:28.309790+0000","last_clean":"2024-05-09T06:27:28.309790+0000","last_became_active":"2024-05-09T06:27:21.648693+0000","last_became_peered":"2024-05-09T06:27:21.648693+0000","last_unstale":"2024-05-09T06:27:28.309790+0000","last_undegraded":"2024-05-09T06:27:28.309790+0000","last_fullsized":"2024-05-09T06:27:28.309790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:17.261877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796403+0000","last_change":"2024-05-09T06:27:21.664868+0000","last_active":"2024-05-09T06:27:28.796403+0000","last_peered":"2024-05-09T06:27:28.796403+0000","last_clean":"2024-05-09T06:27:28.796403+0000","last_became_active":"2024-05-09T06:27:21.663870+0000","last_became_peered":"2024-05-09T06:27:21.663870+0000","last_unstale":"2024-05-09T06:27:28.796403+0000","last_undegraded":"2024-05-09T06:27:28.796403+0000","last_fullsized":"2024-05-09T06:27:28.796403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:55:42.162883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.796427+0000","last_change":"2024-05-09T06:27:21.651232+0000","last_active":"2024-05-09T06:27:28.796427+0000","last_peered":"2024-05-09T06:27:28.796427+0000","last_clean":"2024-05-09T06:27:28.796427+0000","last_became_active":"2024-05-09T06:27:21.650940+0000","last_became_peered":"2024-05-09T06:27:21.650940+0000","last_unstale":"2024-05-09T06:27:28.796427+0000","last_undegraded":"2024-05-09T06:27:28.796427+0000","last_fullsized":"2024-05-09T06:27:28.796427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:22.812128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:26.897594+0000","last_change":"2024-05-09T06:27:21.648169+0000","last_active":"2024-05-09T06:27:26.897594+0000","last_peered":"2024-05-09T06:27:26.897594+0000","last_clean":"2024-05-09T06:27:26.897594+0000","last_became_active":"2024-05-09T06:27:21.647846+0000","last_became_peered":"2024-05-09T06:27:21.647846+0000","last_unstale":"2024-05-09T06:27:26.897594+0000","last_undegraded":"2024-05-09T06:27:26.897594+0000","last_fullsized":"2024-05-09T06:27:26.897594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:57.850003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-09T06:27:28.310386+0000","last_change":"2024-05-09T06:27:21.649391+0000","last_active":"2024-05-09T06:27:28.310386+0000","last_peered":"2024-05-09T06:27:28.310386+0000","last_clean":"2024-05-09T06:27:28.310386+0000","last_became_active":"2024-05-09T06:27:21.648828+0000","last_became_peered":"2024-05-09T06:27:21.648828+0000","last_unstale":"2024-05-09T06:27:28.310386+0000","last_undegraded":"2024-05-09T06:27:28.310386+0000","last_fullsized":"2024-05-09T06:27:28.310386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:20.615500+0000","last_clean_scrub_stamp":"2024-05-09T06:27:20.615500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:46.274923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:27:49.570 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T06:27:49.570 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T06:27:49.570 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T06:27:49.570 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:27:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.728+0000 7f814abf9640 1 Processor -- start 2024-05-09T06:27:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f814abf9640 1 -- start start 2024-05-09T06:27:49.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f814abf9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f81440d4940 0x7f81440b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f814abf9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440b33c0 con 0x7f8144161950 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f814abf9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440b3660 con 0x7f81440d2b10 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f814abf9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f81440d4dd0 con 0x7f81440d4940 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f8148995640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8144161950 0x7f81440d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58778/0 (socket says 172.21.15.1:58778) 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.729+0000 7f8148995640 1 -- 172.21.15.1:0/932544561 learned_addr learned my addr 172.21.15.1:0/932544561 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f8143fff640 1 --2- 172.21.15.1:0/932544561 >> v2:172.21.15.6:3300/0 conn(0x7f81440d4940 0x7f81440b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557749516 0 0) 0x7f81440b33c0 con 0x7f8144161950 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f812c003520 con 0x7f8144161950 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 281606375 0 0) 0x7f812c003520 con 0x7f8144161950 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 >> v2:172.21.15.6:3300/0 conn(0x7f81440d4940 msgr2=0x7f81440b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 --2- 172.21.15.1:0/932544561 >> v2:172.21.15.6:3300/0 conn(0x7f81440d4940 0x7f81440b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 >> v1:172.21.15.6:6789/0 conn(0x7f81440d2b10 legacy=0x7f81440b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440b3940 con 0x7f8144161950 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f813c0019f0 con 0x7f8144161950 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f8143fff640 1 --2- 172.21.15.1:0/932544561 >> v2:172.21.15.6:3300/0 conn(0x7f81440d4940 0x7f81440b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 >> v1:172.21.15.1:6789/0 conn(0x7f8144161950 legacy=0x7f81440d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81440d4dd0 con 0x7f812c0051c0 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f812c00c060 con 0x7f812c0039b0 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f812c00c260 con 0x7f812c004800 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f8149196640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f8148995640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.730+0000 7f8143fff640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 -- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c004800 msgr2=0x7f812c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c004800 0x7f812c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 -- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f812c0039b0 msgr2=0x7f812c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f812c0039b0 0x7f812c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 -- 172.21.15.1:0/932544561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c003520 con 0x7f812c0051c0 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 -- 172.21.15.1:0/932544561 reap_dead start 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f8149196640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 0x7f812c00b9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8134001720 tx=0x7f812c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdfcbcd73fe7384 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134001370 con 0x7f812c0051c0 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81340052d0 con 0x7f812c0051c0 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.731+0000 7f81437fe640 1 -- 172.21.15.1:0/932544561 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813402abc0 con 0x7f812c0051c0 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 -- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 msgr2=0x7f812c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 0x7f812c00b9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8134001720 tx=0x7f812c005830 comp rx=0 tx=0).stop 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 -- 172.21.15.1:0/932544561 shutdown_connections 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 --2- 172.21.15.1:0/932544561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f812c0051c0 0x7f812c00b9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 -- 172.21.15.1:0/932544561 >> 172.21.15.1:0/932544561 conn(0x7f81440d3e20 msgr2=0x7f81440d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 -- 172.21.15.1:0/932544561 shutdown_connections 2024-05-09T06:27:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.732+0000 7f814abf9640 1 -- 172.21.15.1:0/932544561 wait complete. 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.733+0000 7f814abf9640 1 Processor -- start 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.733+0000 7f814abf9640 1 -- start start 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c0051c0 0x7f81440ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81441535e0 0x7f8144160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81440d1100 con 0x7f8144153180 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f81440d0f80 con 0x7f81441535e0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f814abf9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f81440d1c00 con 0x7f812c0051c0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f8143fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f8143fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56208/0 (socket says 172.21.15.1:56208) 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f8143fff640 1 -- 172.21.15.1:0/711054822 learned_addr learned my addr 172.21.15.1:0/711054822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f8148995640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c0051c0 0x7f81440ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.734+0000 7f8149196640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81441535e0 0x7f8144160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 -- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c0051c0 msgr2=0x7f81440ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c0051c0 0x7f81440ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 -- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81441535e0 msgr2=0x7f8144160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81441535e0 0x7f8144160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 -- 172.21.15.1:0/711054822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440cb2d0 con 0x7f8144153180 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f8143fff640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8138011ac0 tx=0x7f8138011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800fc10 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813800fdb0 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81440cb5b0 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138004260 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.735+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81440cbb70 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.736+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81440752b0 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.736+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8138012040 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.737+0000 7f81417fa640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.737+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f813808b700 con 0x7f8144153180 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.737+0000 7f8148995640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.737+0000 7f8148995640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 0x7f813006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f813c004c20 tx=0x7f813c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.741+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8138019030 con 0x7f8144153180 2024-05-09T06:27:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.963+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8144068460 con 0x7f8144153180 2024-05-09T06:27:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.964+0000 7f81417fa640 1 -- 172.21.15.1:0/711054822 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8138059d70 con 0x7f8144153180 2024-05-09T06:27:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:49.946 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:27:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 msgr2=0x7f813006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 0x7f813006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f813c004c20 tx=0x7f813c006040 comp rx=0 tx=0).stop 2024-05-09T06:27:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 msgr2=0x7f81441604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8138011ac0 tx=0x7f8138011f80 comp rx=0 tx=0).stop 2024-05-09T06:27:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 shutdown_connections 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f813006be30 0x7f813006e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81441535e0 0x7f8144160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8144153180 0x7f81441604f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 --2- 172.21.15.1:0/711054822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f812c0051c0 0x7f81440ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 >> 172.21.15.1:0/711054822 conn(0x7f81440d3e20 msgr2=0x7f81440b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 shutdown_connections 2024-05-09T06:27:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:49.966+0000 7f814abf9640 1 -- 172.21.15.1:0/711054822 wait complete. 2024-05-09T06:27:49.959 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T06:27:49.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:50.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.081+0000 7fa7c99ce640 1 Processor -- start 2024-05-09T06:27:50.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.081+0000 7fa7c99ce640 1 -- start start 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c99ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c99ce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa7c415e5b0 0x7fa7c40d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c99ce640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c406c2f0 con 0x7fa7c415e1d0 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c99ce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c40cc3c0 con 0x7fa7c415e5b0 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c99ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c40ccc60 con 0x7fa7c406ffa0 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c37fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa7c415e5b0 0x7fa7c40d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c37fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa7c415e5b0 0x7fa7c40d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:46724/0 (socket says 172.21.15.1:46724) 2024-05-09T06:27:50.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:43468/0 (socket says 172.21.15.1:43468) 2024-05-09T06:27:50.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c37fe640 1 -- 172.21.15.1:0/501625075 learned_addr learned my addr 172.21.15.1:0/501625075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:50.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.082+0000 7fa7c2ffd640 1 --1- 172.21.15.1:0/501625075 >> v1:172.21.15.6:6789/0 conn(0x7fa7c415e1d0 0x7fa7c406f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:43094/0 (socket says 172.21.15.1:43094) 2024-05-09T06:27:50.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7c37fe640 1 -- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 msgr2=0x7fa7c40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:50.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7c37fe640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7c37fe640 1 -- 172.21.15.1:0/501625075 >> v1:172.21.15.6:6789/0 conn(0x7fa7c415e1d0 legacy=0x7fa7c406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7c37fe640 1 -- 172.21.15.1:0/501625075 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c406f500 con 0x7fa7c415e5b0 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7bbfff640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.083+0000 7fa7c37fe640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c40d04f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1d87fa31209995 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c27fc640 1 -- 172.21.15.1:0/501625075 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc036070 con 0x7fa7c415e5b0 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c27fc640 1 -- 172.21.15.1:0/501625075 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc031600 con 0x7fa7c415e5b0 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c27fc640 1 -- 172.21.15.1:0/501625075 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0319d0 con 0x7fa7c415e5b0 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 -- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 msgr2=0x7fa7c40d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c40d04f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc02a680 comp rx=0 tx=0).stop 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 -- 172.21.15.1:0/501625075 shutdown_connections 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c40d04f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/501625075 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 -- 172.21.15.1:0/501625075 >> 172.21.15.1:0/501625075 conn(0x7fa7c40d1d70 msgr2=0x7fa7c40d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 -- 172.21.15.1:0/501625075 shutdown_connections 2024-05-09T06:27:50.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.084+0000 7fa7c99ce640 1 -- 172.21.15.1:0/501625075 wait complete. 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.085+0000 7fa7c99ce640 1 Processor -- start 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.085+0000 7fa7c99ce640 1 -- start start 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c406f1b0 con 0x7fa7c415e1d0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40cc8f0 con 0x7fa7c415e5b0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c99ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c40ce100 con 0x7fa7c406ffa0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7bbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56214/0 (socket says 172.21.15.1:56214) 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7bbfff640 1 -- 172.21.15.1:0/489237082 learned_addr learned my addr 172.21.15.1:0/489237082 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c2ffd640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.086+0000 7fa7c37fe640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 -- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 msgr2=0x7fa7c40c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 -- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 msgr2=0x7fa7c415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c415ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 -- 172.21.15.1:0/489237082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c40c8200 con 0x7fa7c415e1d0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c37fe640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7bbfff640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ebe0 tx=0x7fa7ac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac017070 con 0x7fa7c415e1d0 2024-05-09T06:27:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ac00cc30 con 0x7fa7c415e1d0 2024-05-09T06:27:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac013470 con 0x7fa7c415e1d0 2024-05-09T06:27:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c40c6f20 con 0x7fa7c415e1d0 2024-05-09T06:27:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.087+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c40c74e0 con 0x7fa7c415e1d0 2024-05-09T06:27:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.088+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c40752b0 con 0x7fa7c415e1d0 2024-05-09T06:27:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.089+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7ac01a070 con 0x7fa7c415e1d0 2024-05-09T06:27:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.089+0000 7fa7c0ff9640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 0x7fa7a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.089+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa7ac08b030 con 0x7fa7c415e1d0 2024-05-09T06:27:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.089+0000 7fa7c2ffd640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 0x7fa7a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.090+0000 7fa7c2ffd640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 0x7fa7a406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4007fd0 tx=0x7fa7b4007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.093+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7ac059050 con 0x7fa7c415e1d0 2024-05-09T06:27:50.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.279+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c40c78e0 con 0x7fa7c415e1d0 2024-05-09T06:27:50.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.280+0000 7fa7c0ff9640 1 -- 172.21.15.1:0/489237082 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fa7ac05d350 con 0x7fa7c415e1d0 2024-05-09T06:27:50.263 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:50.263 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:50.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 msgr2=0x7fa7a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 0x7fa7a406e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4007fd0 tx=0x7fa7b4007e80 comp rx=0 tx=0).stop 2024-05-09T06:27:50.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 msgr2=0x7fa7c41f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00ebe0 tx=0x7fa7ac00c520 comp rx=0 tx=0).stop 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 shutdown_connections 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa7a406be60 0x7fa7a406e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7c415e5b0 0x7fa7c415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c415e1d0 0x7fa7c41f7db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 --2- 172.21.15.1:0/489237082 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa7c406ffa0 0x7fa7c40c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 >> 172.21.15.1:0/489237082 conn(0x7fa7c40d1d70 msgr2=0x7fa7c406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 shutdown_connections 2024-05-09T06:27:50.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.282+0000 7fa7c99ce640 1 -- 172.21.15.1:0/489237082 wait complete. 2024-05-09T06:27:50.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:27:50.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.401+0000 7f3e95163640 1 Processor -- start 2024-05-09T06:27:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.401+0000 7f3e95163640 1 -- start start 2024-05-09T06:27:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e95163640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e9015e190 0x7f3e9015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e95163640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e90161fe0 con 0x7f3e900d45f0 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e95163640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e901622c0 con 0x7f3e900d1c70 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e95163640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3e9015f580 con 0x7f3e9015e190 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e8ed9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e9015e190 0x7f3e9015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e8f59e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3e900d45f0 0x7f3e900b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58794/0 (socket says 172.21.15.1:58794) 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.402+0000 7f3e8f59e640 1 -- 172.21.15.1:0/3249068132 learned_addr learned my addr 172.21.15.1:0/3249068132 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/3249068132 >> v1:172.21.15.6:6789/0 conn(0x7f3e900d1c70 legacy=0x7f3e900d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/3249068132 >> v1:172.21.15.1:6789/0 conn(0x7f3e900d45f0 legacy=0x7f3e900b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/3249068132 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e901625a0 con 0x7f3e9015e190 2024-05-09T06:27:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8dd9b640 1 -- 172.21.15.1:0/3249068132 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713552300 0 0) 0x7f3e90161fe0 con 0x7f3e900d45f0 2024-05-09T06:27:50.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8ed9d640 1 --2- 172.21.15.1:0/3249068132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e9015e190 0x7f3e9015f780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787d34bd662ffb1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.403+0000 7f3e8dd9b640 1 -- 172.21.15.1:0/3249068132 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88037070 con 0x7f3e9015e190 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e8dd9b640 1 -- 172.21.15.1:0/3249068132 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e88032440 con 0x7f3e9015e190 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e8dd9b640 1 -- 172.21.15.1:0/3249068132 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88032810 con 0x7f3e9015e190 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 -- 172.21.15.1:0/3249068132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e9015e190 msgr2=0x7f3e9015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 --2- 172.21.15.1:0/3249068132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e9015e190 0x7f3e9015f780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802a230 comp rx=0 tx=0).stop 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 -- 172.21.15.1:0/3249068132 shutdown_connections 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 --2- 172.21.15.1:0/3249068132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e9015e190 0x7f3e9015f780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 -- 172.21.15.1:0/3249068132 >> 172.21.15.1:0/3249068132 conn(0x7f3e90073340 msgr2=0x7f3e9009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 -- 172.21.15.1:0/3249068132 shutdown_connections 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.404+0000 7f3e95163640 1 -- 172.21.15.1:0/3249068132 wait complete. 2024-05-09T06:27:50.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 Processor -- start 2024-05-09T06:27:50.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 -- start start 2024-05-09T06:27:50.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e900d45f0 0x7f3e900b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e9015e190 0x7f3e900b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e900b56d0 con 0x7f3e900d1c70 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90161c60 con 0x7f3e900d45f0 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.406+0000 7f3e95163640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3e900b5850 con 0x7f3e9015e190 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56226/0 (socket says 172.21.15.1:56226) 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/2962228035 learned_addr learned my addr 172.21.15.1:0/2962228035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8e59c640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e900d45f0 0x7f3e900b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8f59e640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e9015e190 0x7f3e900b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e9015e190 msgr2=0x7f3e900b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e9015e190 0x7f3e900b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e900d45f0 msgr2=0x7f3e900b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e900d45f0 0x7f3e900b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 -- 172.21.15.1:0/2962228035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e900cd080 con 0x7f3e900d1c70 2024-05-09T06:27:50.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e8ed9d640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000c00 tx=0x7f3e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88041070 con 0x7f3e900d1c70 2024-05-09T06:27:50.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8803c930 con 0x7f3e900d1c70 2024-05-09T06:27:50.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e900cd360 con 0x7f3e900d1c70 2024-05-09T06:27:50.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88037040 con 0x7f3e900d1c70 2024-05-09T06:27:50.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.407+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e900cd920 con 0x7f3e900d1c70 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.408+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e900752b0 con 0x7f3e900d1c70 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.409+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e88044070 con 0x7f3e900d1c70 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.409+0000 7f3e877fe640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.409+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3e880abce0 con 0x7f3e900d1c70 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.410+0000 7f3e8e59c640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.410+0000 7f3e8e59c640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 0x7f3e7006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e901613e0 tx=0x7f3e78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.414+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e88079eb0 con 0x7f3e900d1c70 2024-05-09T06:27:50.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.598+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e9008e1c0 con 0x7f3e900d1c70 2024-05-09T06:27:50.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.598+0000 7f3e877fe640 1 -- 172.21.15.1:0/2962228035 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f3e8807e1b0 con 0x7f3e900d1c70 2024-05-09T06:27:50.580 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 msgr2=0x7f3e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 0x7f3e7006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e901613e0 tx=0x7f3e78009040 comp rx=0 tx=0).stop 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 msgr2=0x7f3e900b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000c00 tx=0x7f3e880046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 shutdown_connections 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e9015e190 0x7f3e900b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e900d45f0 0x7f3e900b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.600+0000 7f3e95163640 1 --2- 172.21.15.1:0/2962228035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e900d1c70 0x7f3e900b7e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.601+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 >> 172.21.15.1:0/2962228035 conn(0x7f3e90073340 msgr2=0x7f3e900b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.601+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 shutdown_connections 2024-05-09T06:27:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.601+0000 7f3e95163640 1 -- 172.21.15.1:0/2962228035 wait complete. 2024-05-09T06:27:50.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:27:50.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.718+0000 7fe90b258640 1 Processor -- start 2024-05-09T06:27:50.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.718+0000 7fe90b258640 1 -- start start 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe90b258640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 0x7fe90415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe90b258640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe904161fe0 con 0x7fe9040d1c70 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe90b258640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9041622c0 con 0x7fe9040d45f0 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe90b258640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe90415f580 con 0x7fe90415e190 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe903fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe9040d1c70 0x7fe9040d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58800/0 (socket says 172.21.15.1:58800) 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe903fff640 1 -- 172.21.15.1:0/3058606854 learned_addr learned my addr 172.21.15.1:0/3058606854 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe908ff4640 1 --2- 172.21.15.1:0/3058606854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 0x7fe90415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621515935 0 0) 0x7fe904161fe0 con 0x7fe9040d1c70 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.719+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8f0003520 con 0x7fe9040d1c70 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.720+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139865684 0 0) 0x7fe9041622c0 con 0x7fe9040d45f0 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.720+0000 7fe908ff4640 1 -- 172.21.15.1:0/3058606854 >> v1:172.21.15.6:6789/0 conn(0x7fe9040d45f0 legacy=0x7fe9040b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.720+0000 7fe908ff4640 1 -- 172.21.15.1:0/3058606854 >> v1:172.21.15.1:6789/0 conn(0x7fe9040d1c70 legacy=0x7fe9040d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.720+0000 7fe908ff4640 1 -- 172.21.15.1:0/3058606854 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9041625a0 con 0x7fe90415e190 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.720+0000 7fe908ff4640 1 --2- 172.21.15.1:0/3058606854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 0x7fe90415f780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0097d0 tx=0x7fe8ec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ca646795e553b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec037070 con 0x7fe90415e190 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec032440 con 0x7fe90415e190 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe9037fe640 1 -- 172.21.15.1:0/3058606854 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec032810 con 0x7fe90415e190 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 -- 172.21.15.1:0/3058606854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 msgr2=0x7fe90415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 --2- 172.21.15.1:0/3058606854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 0x7fe90415f780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0097d0 tx=0x7fe8ec02a210 comp rx=0 tx=0).stop 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 -- 172.21.15.1:0/3058606854 shutdown_connections 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 --2- 172.21.15.1:0/3058606854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe90415e190 0x7fe90415f780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 -- 172.21.15.1:0/3058606854 >> 172.21.15.1:0/3058606854 conn(0x7fe904073340 msgr2=0x7fe90409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 -- 172.21.15.1:0/3058606854 shutdown_connections 2024-05-09T06:27:50.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.721+0000 7fe90b258640 1 -- 172.21.15.1:0/3058606854 wait complete. 2024-05-09T06:27:50.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.722+0000 7fe90b258640 1 Processor -- start 2024-05-09T06:27:50.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.722+0000 7fe90b258640 1 -- start start 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9040d1c70 0x7fe904159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9040d45f0 0x7fe9040c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040b56d0 con 0x7fe90415e190 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe904161c60 con 0x7fe9040d45f0 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe90b258640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe9040b5850 con 0x7fe9040d1c70 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.723+0000 7fe9097f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56242/0 (socket says 172.21.15.1:56242) 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 -- 172.21.15.1:0/3974938603 learned_addr learned my addr 172.21.15.1:0/3974938603 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe903fff640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9040d45f0 0x7fe9040c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe908ff4640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9040d1c70 0x7fe904159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 -- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9040d1c70 msgr2=0x7fe904159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9040d1c70 0x7fe904159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 -- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9040d45f0 msgr2=0x7fe9040c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9040d45f0 0x7fe9040c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 -- 172.21.15.1:0/3974938603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9040ccb20 con 0x7fe90415e190 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9097f5640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c730 tx=0x7fe8fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc003da0 con 0x7fe90415e190 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00ce40 con 0x7fe90415e190 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0052b0 con 0x7fe90415e190 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9040cce00 con 0x7fe90415e190 2024-05-09T06:27:50.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.724+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9040cd3c0 con 0x7fe90415e190 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.725+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9040752b0 con 0x7fe90415e190 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.726+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8fc005450 con 0x7fe90415e190 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.726+0000 7fe9017fa640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 0x7fe8e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.726+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe8fc08afd0 con 0x7fe90415e190 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.726+0000 7fe908ff4640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 0x7fe8e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:50.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.727+0000 7fe908ff4640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 0x7fe8e406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec034000 tx=0x7fe8ec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.731+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8fc058ff0 con 0x7fe90415e190 2024-05-09T06:27:50.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.912+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe90408e1c0 con 0x7fe90415e190 2024-05-09T06:27:50.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.913+0000 7fe9017fa640 1 -- 172.21.15.1:0/3974938603 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fe8fc05d2f0 con 0x7fe90415e190 2024-05-09T06:27:50.895 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 msgr2=0x7fe8e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 0x7fe8e406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec034000 tx=0x7fe8ec02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 msgr2=0x7fe9040c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c730 tx=0x7fe8fc00cbf0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 shutdown_connections 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe8e406be30 0x7fe8e406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90415e190 0x7fe9040c8b20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9040d45f0 0x7fe9040c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 --2- 172.21.15.1:0/3974938603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9040d1c70 0x7fe904159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:50.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 >> 172.21.15.1:0/3974938603 conn(0x7fe904073340 msgr2=0x7fe904160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:50.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 shutdown_connections 2024-05-09T06:27:50.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:50.915+0000 7fe90b258640 1 -- 172.21.15.1:0/3974938603 wait complete. 2024-05-09T06:27:50.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:27:51.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.032+0000 7f843229e640 1 Processor -- start 2024-05-09T06:27:51.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.032+0000 7f843229e640 1 -- start start 2024-05-09T06:27:51.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c158a50 0x7f842c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f842c15c4a0 0x7f842c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f842c06e6d0 0x7f842c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f842c158e30 con 0x7f842c06e6d0 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f842c072480 con 0x7f842c15c4a0 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843229e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f842c15b420 con 0x7f842c158a50 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f842c06e6d0 0x7f842c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f842c06e6d0 0x7f842c06eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56256/0 (socket says 172.21.15.1:56256) 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 -- 172.21.15.1:0/122991049 learned_addr learned my addr 172.21.15.1:0/122991049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f842bfff640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c158a50 0x7f842c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f8423fff640 1 --2- 172.21.15.1:0/122991049 >> v2:172.21.15.6:3300/0 conn(0x7f842c15c4a0 0x7f842c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 -- 172.21.15.1:0/122991049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c158a50 msgr2=0x7f842c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c158a50 0x7f842c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 -- 172.21.15.1:0/122991049 >> v2:172.21.15.6:3300/0 conn(0x7f842c15c4a0 msgr2=0x7f842c071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 --2- 172.21.15.1:0/122991049 >> v2:172.21.15.6:3300/0 conn(0x7f842c15c4a0 0x7f842c071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.033+0000 7f843083b640 1 -- 172.21.15.1:0/122991049 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c15b6e0 con 0x7f842c06e6d0 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.034+0000 7f843083b640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c06eb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b2770 tx=0x7f842402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cd3e3f9e2ed5e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.034+0000 7f842b7fe640 1 -- 172.21.15.1:0/122991049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424036070 con 0x7f842c06e6d0 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.034+0000 7f842b7fe640 1 -- 172.21.15.1:0/122991049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8424031600 con 0x7f842c06e6d0 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.034+0000 7f842b7fe640 1 -- 172.21.15.1:0/122991049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84240319d0 con 0x7f842c06e6d0 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 -- 172.21.15.1:0/122991049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 msgr2=0x7f842c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c06eb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b2770 tx=0x7f842402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 -- 172.21.15.1:0/122991049 shutdown_connections 2024-05-09T06:27:51.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c06eb80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 --2- 172.21.15.1:0/122991049 >> v2:172.21.15.6:3300/0 conn(0x7f842c15c4a0 0x7f842c071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 --2- 172.21.15.1:0/122991049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c158a50 0x7f842c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 -- 172.21.15.1:0/122991049 >> 172.21.15.1:0/122991049 conn(0x7f842c0cc4d0 msgr2=0x7f842c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 -- 172.21.15.1:0/122991049 shutdown_connections 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.035+0000 7f843229e640 1 -- 172.21.15.1:0/122991049 wait complete. 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.036+0000 7f843229e640 1 Processor -- start 2024-05-09T06:27:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.036+0000 7f843229e640 1 -- start start 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158a50 0x7f842c145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c15c4a0 0x7f842c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f842c15bf80 con 0x7f842c06e6d0 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f842c15bca0 con 0x7f842c158a50 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843229e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f842c070ea0 con 0x7f842c15c4a0 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56266/0 (socket says 172.21.15.1:56266) 2024-05-09T06:27:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 -- 172.21.15.1:0/2025964894 learned_addr learned my addr 172.21.15.1:0/2025964894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f843083b640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c15c4a0 0x7f842c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f8423fff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158a50 0x7f842c145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 -- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c15c4a0 msgr2=0x7f842c06d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c15c4a0 0x7f842c06d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 -- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158a50 msgr2=0x7f842c145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158a50 0x7f842c145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.037+0000 7f842bfff640 1 -- 172.21.15.1:0/2025964894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c14d4b0 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f842bfff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e890 tx=0x7f841c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c017070 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c00c940 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c14d790 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00cd10 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.038+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c14fe60 con 0x7f842c06e6d0 2024-05-09T06:27:51.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.039+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c0752b0 con 0x7f842c06e6d0 2024-05-09T06:27:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.039+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f841c01a070 con 0x7f842c06e6d0 2024-05-09T06:27:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.040+0000 7f84297fa640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 0x7f840c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.040+0000 7f8423fff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 0x7f840c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.040+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f841c08b3f0 con 0x7f842c06e6d0 2024-05-09T06:27:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.040+0000 7f8423fff640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 0x7f840c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b52d0 tx=0x7f8414009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.044+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f841c059410 con 0x7f842c06e6d0 2024-05-09T06:27:51.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.229+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f842c15a1d0 con 0x7f842c06e6d0 2024-05-09T06:27:51.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.229+0000 7f84297fa640 1 -- 172.21.15.1:0/2025964894 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f841c05d710 con 0x7f842c06e6d0 2024-05-09T06:27:51.212 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:27:51.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 msgr2=0x7f840c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 0x7f840c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b52d0 tx=0x7f8414009190 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 msgr2=0x7f842c14c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e890 tx=0x7f841c00ed50 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 shutdown_connections 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f840c06be30 0x7f840c06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842c15c4a0 0x7f842c06d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158a50 0x7f842c145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 --2- 172.21.15.1:0/2025964894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f842c06e6d0 0x7f842c14c700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 >> 172.21.15.1:0/2025964894 conn(0x7f842c0cc4d0 msgr2=0x7f842c15a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 shutdown_connections 2024-05-09T06:27:51.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.231+0000 7f843229e640 1 -- 172.21.15.1:0/2025964894 wait complete. 2024-05-09T06:27:51.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:27:51.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.349+0000 7fd581d22640 1 Processor -- start 2024-05-09T06:27:51.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.349+0000 7fd581d22640 1 -- start start 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd581d22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd581d22640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd57c0d0a10 0x7fd57c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd581d22640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c0d1350 con 0x7fd57c0ceaa0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd581d22640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0b6e50 con 0x7fd57c0d0a10 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd581d22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c0d2760 con 0x7fd57c15e0f0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd57bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd57c0ceaa0 0x7fd57c0ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58812/0 (socket says 172.21.15.1:58812) 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd57bfff640 1 -- 172.21.15.1:0/3729045636 learned_addr learned my addr 172.21.15.1:0/3729045636 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd57affd640 1 --2- 172.21.15.1:0/3729045636 >> v2:172.21.15.6:3300/0 conn(0x7fd57c0d0a10 0x7fd57c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.350+0000 7fd57b7fe640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57a7fc640 1 -- 172.21.15.1:0/3729045636 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1999831962 0 0) 0x7fd57c0d1350 con 0x7fd57c0ceaa0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57a7fc640 1 -- 172.21.15.1:0/3729045636 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd564003520 con 0x7fd57c0ceaa0 2024-05-09T06:27:51.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57affd640 1 -- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 msgr2=0x7fd57c0d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57affd640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57affd640 1 -- 172.21.15.1:0/3729045636 >> v1:172.21.15.1:6789/0 conn(0x7fd57c0ceaa0 legacy=0x7fd57c0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57affd640 1 -- 172.21.15.1:0/3729045636 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0d2a40 con 0x7fd57c0d0a10 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.351+0000 7fd57affd640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0d0e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097a0 tx=0x7fd57002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797e9b8f68352ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd57a7fc640 1 -- 172.21.15.1:0/3729045636 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570037070 con 0x7fd57c0d0a10 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd57a7fc640 1 -- 172.21.15.1:0/3729045636 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570032440 con 0x7fd57c0d0a10 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd57a7fc640 1 -- 172.21.15.1:0/3729045636 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700327f0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 -- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0d0a10 msgr2=0x7fd57c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0d0e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5700097a0 tx=0x7fd57002a180 comp rx=0 tx=0).stop 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 -- 172.21.15.1:0/3729045636 shutdown_connections 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0d0e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 --2- 172.21.15.1:0/3729045636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 -- 172.21.15.1:0/3729045636 >> 172.21.15.1:0/3729045636 conn(0x7fd57c09d3b0 msgr2=0x7fd57c0d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 -- 172.21.15.1:0/3729045636 shutdown_connections 2024-05-09T06:27:51.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.352+0000 7fd581d22640 1 -- 172.21.15.1:0/3729045636 wait complete. 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.353+0000 7fd581d22640 1 Processor -- start 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 -- start start 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0ceaa0 0x7fd57c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c15e4d0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0b70a0 con 0x7fd57c0ceaa0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd581d22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c15e650 con 0x7fd57c15e0f0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.354+0000 7fd57affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56280/0 (socket says 172.21.15.1:56280) 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 -- 172.21.15.1:0/386713366 learned_addr learned my addr 172.21.15.1:0/386713366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57b7fe640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0ceaa0 0x7fd57c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 -- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 msgr2=0x7fd57c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 -- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0ceaa0 msgr2=0x7fd57c146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0ceaa0 0x7fd57c146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 -- 172.21.15.1:0/386713366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0ca6d0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57bfff640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd57affd640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd57002a1c0 tx=0x7fd5700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570037040 con 0x7fd57c0d0a10 2024-05-09T06:27:51.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570042070 con 0x7fd57c0d0a10 2024-05-09T06:27:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c0ca9b0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003b930 con 0x7fd57c0d0a10 2024-05-09T06:27:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.355+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c146f50 con 0x7fd57c0d0a10 2024-05-09T06:27:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.356+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c0752b0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.357+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd57003bad0 con 0x7fd57c0d0a10 2024-05-09T06:27:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.357+0000 7fd563fff640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 0x7fd56806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.357+0000 7fd57b7fe640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 0x7fd56806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.357+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd5700ab880 con 0x7fd57c0d0a10 2024-05-09T06:27:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.358+0000 7fd57b7fe640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 0x7fd56806e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009aa0 tx=0x7fd56c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.362+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd570079a50 con 0x7fd57c0d0a10 2024-05-09T06:27:51.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.542+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd57c069120 con 0x7fd57c0d0a10 2024-05-09T06:27:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.543+0000 7fd563fff640 1 -- 172.21.15.1:0/386713366 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fd57007dd50 con 0x7fd57c0d0a10 2024-05-09T06:27:51.525 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 msgr2=0x7fd56806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 0x7fd56806e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009aa0 tx=0x7fd56c009190 comp rx=0 tx=0).stop 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 msgr2=0x7fd57c0c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd57002a1c0 tx=0x7fd5700046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 shutdown_connections 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56806be30 0x7fd56806e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd57c15e0f0 0x7fd57c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd57c0d0a10 0x7fd57c0c7b10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 --2- 172.21.15.1:0/386713366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd57c0ceaa0 0x7fd57c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 >> 172.21.15.1:0/386713366 conn(0x7fd57c09d3b0 msgr2=0x7fd57c0b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 shutdown_connections 2024-05-09T06:27:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.545+0000 7fd581d22640 1 -- 172.21.15.1:0/386713366 wait complete. 2024-05-09T06:27:51.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:51.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.660+0000 7f5fbd8d2640 1 Processor -- start 2024-05-09T06:27:51.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.660+0000 7f5fbd8d2640 1 -- start start 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5fb80c6860 0x7f5fb8099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80b2aa0 0x7f5fb815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fb80c6c40 0x7f5fb815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb80b1e20 con 0x7f5fb80c6c40 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb815ee90 con 0x7f5fb80c6860 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fbd8d2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb8099930 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fb80c6c40 0x7f5fb815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fb80c6c40 0x7f5fb815f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56290/0 (socket says 172.21.15.1:56290) 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 -- 172.21.15.1:0/3861994555 learned_addr learned my addr 172.21.15.1:0/3861994555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:51.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb67fc640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80b2aa0 0x7f5fb815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb6ffd640 1 --2- 172.21.15.1:0/3861994555 >> v2:172.21.15.6:3300/0 conn(0x7f5fb80c6860 0x7f5fb8099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 -- 172.21.15.1:0/3861994555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80b2aa0 msgr2=0x7f5fb815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80b2aa0 0x7f5fb815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 -- 172.21.15.1:0/3861994555 >> v2:172.21.15.6:3300/0 conn(0x7f5fb80c6860 msgr2=0x7f5fb8099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 --2- 172.21.15.1:0/3861994555 >> v2:172.21.15.6:3300/0 conn(0x7f5fb80c6860 0x7f5fb8099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.661+0000 7f5fb77fe640 1 -- 172.21.15.1:0/3861994555 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8162930 con 0x7f5fb80c6c40 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.662+0000 7f5fb77fe640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80c6c40 0x7f5fb815f320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80722c0 tx=0x7f5fb002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cee2f7f9f3390d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.662+0000 7f5fb5ffb640 1 -- 172.21.15.1:0/3861994555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0036070 con 0x7f5fb80c6c40 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.662+0000 7f5fb5ffb640 1 -- 172.21.15.1:0/3861994555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb0031600 con 0x7f5fb80c6c40 2024-05-09T06:27:51.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.662+0000 7f5fb5ffb640 1 -- 172.21.15.1:0/3861994555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00319d0 con 0x7f5fb80c6c40 2024-05-09T06:27:51.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3861994555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80c6c40 msgr2=0x7f5fb815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80c6c40 0x7f5fb815f320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80722c0 tx=0x7f5fb002a610 comp rx=0 tx=0).stop 2024-05-09T06:27:51.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3861994555 shutdown_connections 2024-05-09T06:27:51.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80c6c40 0x7f5fb815f320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3861994555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80b2aa0 0x7f5fb815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3861994555 >> v2:172.21.15.6:3300/0 conn(0x7f5fb80c6860 0x7f5fb8099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3861994555 >> 172.21.15.1:0/3861994555 conn(0x7f5fb80b1fb0 msgr2=0x7f5fb80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3861994555 shutdown_connections 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.663+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3861994555 wait complete. 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.664+0000 7f5fbd8d2640 1 Processor -- start 2024-05-09T06:27:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.664+0000 7f5fbd8d2640 1 -- start start 2024-05-09T06:27:51.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 0x7f5fb80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80c6c40 0x7f5fb81f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80c70d0 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb815eb20 con 0x7f5fb80c6860 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fbd8d2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb815f860 con 0x7f5fb80c6c40 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56296/0 (socket says 172.21.15.1:56296) 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 -- 172.21.15.1:0/3945187039 learned_addr learned my addr 172.21.15.1:0/3945187039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb77fe640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80c6c40 0x7f5fb81f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb67fc640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 0x7f5fb80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 -- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80c6c40 msgr2=0x7f5fb81f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80c6c40 0x7f5fb81f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 -- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 msgr2=0x7f5fb80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 0x7f5fb80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 -- 172.21.15.1:0/3945187039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb80c74f0 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb67fc640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 0x7f5fb80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.665+0000 7f5fb6ffd640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e8a0 tx=0x7f5fa800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.666+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8017070 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.666+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa800c940 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.666+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb80c77d0 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.666+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800cd30 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.666+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb80c7d90 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.667+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb80cd330 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.667+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fa801a070 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.668+0000 7f5f9ffff640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 0x7f5fa406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.668+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5fa808b300 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.668+0000 7f5fb67fc640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 0x7f5fa406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.668+0000 7f5fb67fc640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 0x7f5fa406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80d0280 tx=0x7f5fac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.672+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5fb80cd330 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.893+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fb80752b0 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.894+0000 7f5f9ffff640 1 -- 172.21.15.1:0/3945187039 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f5fa8059320 con 0x7f5fb80b2aa0 2024-05-09T06:27:51.876 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:51.876 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:51.877 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:27:51.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 msgr2=0x7f5fa406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 0x7f5fa406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80d0280 tx=0x7f5fac008040 comp rx=0 tx=0).stop 2024-05-09T06:27:51.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 msgr2=0x7f5fb80c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e8a0 tx=0x7f5fa800ed60 comp rx=0 tx=0).stop 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 shutdown_connections 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5fa406be60 0x7f5fa406e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5fb80c6c40 0x7f5fb81f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fb80c6860 0x7f5fb80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 --2- 172.21.15.1:0/3945187039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fb80b2aa0 0x7f5fb80c9ed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 >> 172.21.15.1:0/3945187039 conn(0x7f5fb80b1fb0 msgr2=0x7f5fb80d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 shutdown_connections 2024-05-09T06:27:51.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:51.896+0000 7f5fbd8d2640 1 -- 172.21.15.1:0/3945187039 wait complete. 2024-05-09T06:27:51.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:51.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.010+0000 7fd4c103f640 1 Processor -- start 2024-05-09T06:27:51.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.010+0000 7fd4c103f640 1 -- start start 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4c103f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4c103f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4bc0d2710 0x7fd4bc0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4c103f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4bc0d2b10 con 0x7fd4bc1617a0 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4c103f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd4bc0d1480 con 0x7fd4bc0b2dd0 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4c103f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd4bc0b5720 con 0x7fd4bc0d2710 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4ba59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4bad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd4bc1617a0 0x7fd4bc0b5010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58818/0 (socket says 172.21.15.1:58818) 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4ba59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56312/0 (socket says 172.21.15.1:56312) 2024-05-09T06:27:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4bad9d640 1 -- 172.21.15.1:0/1643015039 learned_addr learned my addr 172.21.15.1:0/1643015039 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.011+0000 7fd4bb59e640 1 --2- 172.21.15.1:0/1643015039 >> v2:172.21.15.6:3300/0 conn(0x7fd4bc0d2710 0x7fd4bc0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4ba59c640 1 -- 172.21.15.1:0/1643015039 >> v2:172.21.15.6:3300/0 conn(0x7fd4bc0d2710 msgr2=0x7fd4bc0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4ba59c640 1 --2- 172.21.15.1:0/1643015039 >> v2:172.21.15.6:3300/0 conn(0x7fd4bc0d2710 0x7fd4bc0ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4b9d9b640 1 -- 172.21.15.1:0/1643015039 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747004152 0 0) 0x7fd4bc0d2b10 con 0x7fd4bc1617a0 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4ba59c640 1 -- 172.21.15.1:0/1643015039 >> v1:172.21.15.1:6789/0 conn(0x7fd4bc1617a0 legacy=0x7fd4bc0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4ba59c640 1 -- 172.21.15.1:0/1643015039 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc0d0870 con 0x7fd4bc0b2dd0 2024-05-09T06:27:51.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.012+0000 7fd4ba59c640 1 --2- 172.21.15.1:0/1643015039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc08e0b0 tx=0x7fd4a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edd481b67deb4d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:51.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4b9d9b640 1 -- 172.21.15.1:0/1643015039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4037070 con 0x7fd4bc0b2dd0 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4b9d9b640 1 -- 172.21.15.1:0/1643015039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4032440 con 0x7fd4bc0b2dd0 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4b9d9b640 1 -- 172.21.15.1:0/1643015039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032810 con 0x7fd4bc0b2dd0 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 -- 172.21.15.1:0/1643015039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc0b2dd0 msgr2=0x7fd4bc0d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1643015039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc08e0b0 tx=0x7fd4a402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 -- 172.21.15.1:0/1643015039 shutdown_connections 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1643015039 >> v2:172.21.15.6:3300/0 conn(0x7fd4bc0d2710 0x7fd4bc0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1643015039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0d21d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 -- 172.21.15.1:0/1643015039 >> 172.21.15.1:0/1643015039 conn(0x7fd4bc09d180 msgr2=0x7fd4bc0d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 -- 172.21.15.1:0/1643015039 shutdown_connections 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.013+0000 7fd4c103f640 1 -- 172.21.15.1:0/1643015039 wait complete. 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 Processor -- start 2024-05-09T06:27:51.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 -- start start 2024-05-09T06:27:51.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4bc0d2710 0x7fd4bc1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:51.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc0d1110 con 0x7fd4bc1617a0 2024-05-09T06:27:51.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc0d0e30 con 0x7fd4bc0d2710 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.015+0000 7fd4c103f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd4bc161b80 con 0x7fd4bc0b2dd0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56322/0 (socket says 172.21.15.1:56322) 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 -- 172.21.15.1:0/1247259046 learned_addr learned my addr 172.21.15.1:0/1247259046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4ba59c640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4bc0d2710 0x7fd4bc1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bad9d640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 -- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4bc0b2dd0 msgr2=0x7fd4bc0a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 -- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4bc0d2710 msgr2=0x7fd4bc1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4bc0d2710 0x7fd4bc1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 -- 172.21.15.1:0/1247259046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc15dc00 con 0x7fd4bc1617a0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4bb59e640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400e930 tx=0x7fd4b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4017070 con 0x7fd4bc1617a0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b400c980 con 0x7fd4bc1617a0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc15dee0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400cd70 con 0x7fd4bc1617a0 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.016+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc15e4a0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.017+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc0740a0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.018+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4b401a070 con 0x7fd4bc1617a0 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.018+0000 7fd4b37fe640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 0x7fd49c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.018+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd4b408b580 con 0x7fd4bc1617a0 2024-05-09T06:27:52.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.018+0000 7fd4bad9d640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 0x7fd49c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.019+0000 7fd4bad9d640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 0x7fd49c06e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0012e0 tx=0x7fd4ac001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.023+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4bc0740a0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.209+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4bc0a1cf0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.210+0000 7fd4b37fe640 1 -- 172.21.15.1:0/1247259046 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7fd4bc0a1cf0 con 0x7fd4bc1617a0 2024-05-09T06:27:52.193 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:52.193 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 msgr2=0x7fd49c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 0x7fd49c06e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0012e0 tx=0x7fd4ac001630 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 msgr2=0x7fd4bc1f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400e930 tx=0x7fd4b400edf0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 shutdown_connections 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd49c06be60 0x7fd49c06e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4bc1617a0 0x7fd4bc1f7f90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4bc0d2710 0x7fd4bc1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 --2- 172.21.15.1:0/1247259046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4bc0b2dd0 0x7fd4bc0a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 >> 172.21.15.1:0/1247259046 conn(0x7fd4bc09d180 msgr2=0x7fd4bc0d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 shutdown_connections 2024-05-09T06:27:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.212+0000 7fd4c103f640 1 -- 172.21.15.1:0/1247259046 wait complete. 2024-05-09T06:27:52.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:27:52.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:27:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.330+0000 7fe6c105d640 1 Processor -- start 2024-05-09T06:27:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.330+0000 7fe6c105d640 1 -- start start 2024-05-09T06:27:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6c105d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6bc0d45f0 0x7fe6bc15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6c105d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6bc161fe0 con 0x7fe6bc0d1c70 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6c105d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6bc1622c0 con 0x7fe6bc15e190 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6c105d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe6bc15f580 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6bb59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6bc0d45f0 0x7fe6bc15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6ba59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe6bc0d1c70 0x7fe6bc0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58822/0 (socket says 172.21.15.1:58822) 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.331+0000 7fe6ba59c640 1 -- 172.21.15.1:0/3506059161 learned_addr learned my addr 172.21.15.1:0/3506059161 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.332+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3506059161 >> v1:172.21.15.6:6789/0 conn(0x7fe6bc15e190 legacy=0x7fe6bc0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.332+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3506059161 >> v1:172.21.15.1:6789/0 conn(0x7fe6bc0d1c70 legacy=0x7fe6bc0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.332+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3506059161 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc1625a0 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.332+0000 7fe6b9d9b640 1 -- 172.21.15.1:0/3506059161 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091217072 0 0) 0x7fe6bc161fe0 con 0x7fe6bc0d1c70 2024-05-09T06:27:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.332+0000 7fe6bb59e640 1 --2- 172.21.15.1:0/3506059161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc0d45f0 0x7fe6bc15f780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940f9f90cb2db655 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6b9d9b640 1 -- 172.21.15.1:0/3506059161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac037070 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6b9d9b640 1 -- 172.21.15.1:0/3506059161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac032440 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6b9d9b640 1 -- 172.21.15.1:0/3506059161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac032810 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 -- 172.21.15.1:0/3506059161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc0d45f0 msgr2=0x7fe6bc15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3506059161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc0d45f0 0x7fe6bc15f780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac02a230 comp rx=0 tx=0).stop 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 -- 172.21.15.1:0/3506059161 shutdown_connections 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3506059161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc0d45f0 0x7fe6bc15f780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 -- 172.21.15.1:0/3506059161 >> 172.21.15.1:0/3506059161 conn(0x7fe6bc073340 msgr2=0x7fe6bc09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 -- 172.21.15.1:0/3506059161 shutdown_connections 2024-05-09T06:27:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.333+0000 7fe6c105d640 1 -- 172.21.15.1:0/3506059161 wait complete. 2024-05-09T06:27:52.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.335+0000 7fe6c105d640 1 Processor -- start 2024-05-09T06:27:52.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.335+0000 7fe6c105d640 1 -- start start 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.335+0000 7fe6c105d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 0x7fe6bc15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.335+0000 7fe6c105d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 0x7fe6bc0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.335+0000 7fe6c105d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6c105d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc0b56d0 con 0x7fe6bc15e190 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6c105d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc161c60 con 0x7fe6bc0d1c70 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6c105d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc0b5850 con 0x7fe6bc0d45f0 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56326/0 (socket says 172.21.15.1:56326) 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6ba59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 0x7fe6bc0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3683872715 learned_addr learned my addr 172.21.15.1:0/3683872715 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bad9d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 0x7fe6bc15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 msgr2=0x7fe6bc0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 0x7fe6bc0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 msgr2=0x7fe6bc15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 0x7fe6bc15cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 -- 172.21.15.1:0/3683872715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc0c7f00 con 0x7fe6bc15e190 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6ba59c640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 0x7fe6bc0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bad9d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 0x7fe6bc15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6bb59e640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac037040 con 0x7fe6bc15e190 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac042070 con 0x7fe6bc15e190 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc0c81e0 con 0x7fe6bc15e190 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac030b30 con 0x7fe6bc15e190 2024-05-09T06:27:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.336+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc0c87a0 con 0x7fe6bc15e190 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.338+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6bc0752b0 con 0x7fe6bc15e190 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.338+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6ac030cd0 con 0x7fe6bc15e190 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.338+0000 7fe6b37fe640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 0x7fe69c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.339+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe6ac0abc30 con 0x7fe6bc15e190 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.339+0000 7fe6bad9d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 0x7fe69c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.339+0000 7fe6bad9d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 0x7fe69c06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40012e0 tx=0x7fe6b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.343+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6ac079c50 con 0x7fe6bc15e190 2024-05-09T06:27:52.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.526+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6bc08e1c0 con 0x7fe6bc15e190 2024-05-09T06:27:52.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.526+0000 7fe6b37fe640 1 -- 172.21.15.1:0/3683872715 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fe6ac07df50 con 0x7fe6bc15e190 2024-05-09T06:27:52.508 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.528+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 msgr2=0x7fe69c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.528+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 0x7fe69c06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40012e0 tx=0x7fe6b4009040 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.528+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 msgr2=0x7fe6bc0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.528+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac0046c0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 shutdown_connections 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe69c06be30 0x7fe69c06e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6bc15e190 0x7fe6bc0cdb40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6bc0d45f0 0x7fe6bc0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 --2- 172.21.15.1:0/3683872715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6bc0d1c70 0x7fe6bc15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 >> 172.21.15.1:0/3683872715 conn(0x7fe6bc073340 msgr2=0x7fe6bc15c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 shutdown_connections 2024-05-09T06:27:52.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.529+0000 7fe6c105d640 1 -- 172.21.15.1:0/3683872715 wait complete. 2024-05-09T06:27:52.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:27:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.647+0000 7fc209de7640 1 Processor -- start 2024-05-09T06:27:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.647+0000 7fc209de7640 1 -- start start 2024-05-09T06:27:52.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc209de7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc2040c41e0 0x7fc2040ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc209de7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc20406b1e0 0x7fc2040cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc209de7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc204072050 con 0x7fc2040d06f0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc209de7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc204072330 con 0x7fc2040c41e0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc209de7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc204072610 con 0x7fc20406b1e0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc202ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc2040c41e0 0x7fc2040ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc2037fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc2040d06f0 0x7fc2040cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58832/0 (socket says 172.21.15.1:58832) 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc2037fe640 1 -- 172.21.15.1:0/1541549286 learned_addr learned my addr 172.21.15.1:0/1541549286 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc203fff640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc20406b1e0 0x7fc2040cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc202ffd640 1 -- 172.21.15.1:0/1541549286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc20406b1e0 msgr2=0x7fc2040cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc202ffd640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc20406b1e0 0x7fc2040cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc202ffd640 1 -- 172.21.15.1:0/1541549286 >> v1:172.21.15.1:6789/0 conn(0x7fc2040d06f0 legacy=0x7fc2040cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc2027fc640 1 -- 172.21.15.1:0/1541549286 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740493430 0 0) 0x7fc204072050 con 0x7fc2040d06f0 2024-05-09T06:27:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.648+0000 7fc202ffd640 1 -- 172.21.15.1:0/1541549286 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040728f0 con 0x7fc2040c41e0 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.649+0000 7fc202ffd640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2040c41e0 0x7fc2040ce8d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80097a0 tx=0x7fc1f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3282b4e3fc59ad04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.649+0000 7fc2027fc640 1 -- 172.21.15.1:0/1541549286 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8037070 con 0x7fc2040c41e0 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.649+0000 7fc2027fc640 1 -- 172.21.15.1:0/1541549286 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f8032440 con 0x7fc2040c41e0 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.649+0000 7fc2027fc640 1 -- 172.21.15.1:0/1541549286 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8032810 con 0x7fc2040c41e0 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 -- 172.21.15.1:0/1541549286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2040c41e0 msgr2=0x7fc2040ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2040c41e0 0x7fc2040ce8d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80097a0 tx=0x7fc1f802a180 comp rx=0 tx=0).stop 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 -- 172.21.15.1:0/1541549286 shutdown_connections 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc20406b1e0 0x7fc2040cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 --2- 172.21.15.1:0/1541549286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2040c41e0 0x7fc2040ce8d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 -- 172.21.15.1:0/1541549286 >> 172.21.15.1:0/1541549286 conn(0x7fc2040d0b50 msgr2=0x7fc2040d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 -- 172.21.15.1:0/1541549286 shutdown_connections 2024-05-09T06:27:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.650+0000 7fc209de7640 1 -- 172.21.15.1:0/1541549286 wait complete. 2024-05-09T06:27:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.651+0000 7fc209de7640 1 Processor -- start 2024-05-09T06:27:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.651+0000 7fc209de7640 1 -- start start 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 0x7fc2040a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 0x7fc2041e3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040ccf70 con 0x7fc20406b1e0 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040ccdf0 con 0x7fc2040d06f0 2024-05-09T06:27:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc209de7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc2040cd0f0 con 0x7fc2040c41e0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56344/0 (socket says 172.21.15.1:56344) 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 -- 172.21.15.1:0/2468590039 learned_addr learned my addr 172.21.15.1:0/2468590039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc202ffd640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 0x7fc2040a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc203fff640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 0x7fc2041e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 -- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 msgr2=0x7fc2040a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 0x7fc2040a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 -- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 msgr2=0x7fc2041e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 0x7fc2041e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc2037fe640 1 -- 172.21.15.1:0/2468590039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040a0dd0 con 0x7fc20406b1e0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc203fff640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 0x7fc2041e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.652+0000 7fc202ffd640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 0x7fc2040a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc2037fe640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c460 tx=0x7fc1f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4018070 con 0x7fc20406b1e0 2024-05-09T06:27:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f4007500 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2040a1070 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004a20 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.653+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2040a1630 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.654+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2040752b0 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.654+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1f401b070 con 0x7fc20406b1e0 2024-05-09T06:27:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.655+0000 7fc1ebfff640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 0x7fc1f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.655+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc1f408a3c0 con 0x7fc20406b1e0 2024-05-09T06:27:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.655+0000 7fc202ffd640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 0x7fc1f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.655+0000 7fc202ffd640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 0x7fc1f006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8034000 tx=0x7fc1f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.659+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1f4058ac0 con 0x7fc20406b1e0 2024-05-09T06:27:52.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.842+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc204072f30 con 0x7fc20406b1e0 2024-05-09T06:27:52.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.842+0000 7fc1ebfff640 1 -- 172.21.15.1:0/2468590039 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fc1f40583c0 con 0x7fc20406b1e0 2024-05-09T06:27:52.824 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 msgr2=0x7fc1f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 0x7fc1f006e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8034000 tx=0x7fc1f802e000 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 msgr2=0x7fc2041f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c460 tx=0x7fc1f400c920 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 shutdown_connections 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc1f006be30 0x7fc1f006e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2040d06f0 0x7fc2041e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2040c41e0 0x7fc2040a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 --2- 172.21.15.1:0/2468590039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc20406b1e0 0x7fc2041f8000 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.844+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 >> 172.21.15.1:0/2468590039 conn(0x7fc2040d0b50 msgr2=0x7fc20406ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.845+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 shutdown_connections 2024-05-09T06:27:52.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.845+0000 7fc209de7640 1 -- 172.21.15.1:0/2468590039 wait complete. 2024-05-09T06:27:52.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:27:52.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.962+0000 7f9cce134640 1 Processor -- start 2024-05-09T06:27:52.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.962+0000 7f9cce134640 1 -- start start 2024-05-09T06:27:52.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cce134640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9cc815e390 0x7f9cc80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cce134640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9cc80b1900 0x7f9cc80cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cce134640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc815e900 con 0x7f9cc80cf990 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cce134640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc815ebe0 con 0x7f9cc815e390 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cce134640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc815eec0 con 0x7f9cc80b1900 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9cc815e390 0x7f9cc80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc6ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9cc80cf990 0x7f9cc80b11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58840/0 (socket says 172.21.15.1:58840) 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc6ffd640 1 -- 172.21.15.1:0/1934655981 learned_addr learned my addr 172.21.15.1:0/1934655981 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc7fff640 1 --2- 172.21.15.1:0/1934655981 >> v2:172.21.15.6:3300/0 conn(0x7f9cc80b1900 0x7f9cc80cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc77fe640 1 -- 172.21.15.1:0/1934655981 >> v2:172.21.15.6:3300/0 conn(0x7f9cc80b1900 msgr2=0x7f9cc80cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc77fe640 1 --2- 172.21.15.1:0/1934655981 >> v2:172.21.15.6:3300/0 conn(0x7f9cc80b1900 0x7f9cc80cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc77fe640 1 -- 172.21.15.1:0/1934655981 >> v1:172.21.15.1:6789/0 conn(0x7f9cc80cf990 legacy=0x7f9cc80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc77fe640 1 -- 172.21.15.1:0/1934655981 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc815f1a0 con 0x7f9cc815e390 2024-05-09T06:27:52.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.963+0000 7f9cc67fc640 1 -- 172.21.15.1:0/1934655981 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841699571 0 0) 0x7f9cc815e900 con 0x7f9cc80cf990 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.964+0000 7f9cc77fe640 1 --2- 172.21.15.1:0/1934655981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc815e390 0x7f9cc80cf450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80721e0 tx=0x7f9cb002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00b273ef85104eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.964+0000 7f9cc67fc640 1 -- 172.21.15.1:0/1934655981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037070 con 0x7f9cc815e390 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.964+0000 7f9cc67fc640 1 -- 172.21.15.1:0/1934655981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0032440 con 0x7f9cc815e390 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.964+0000 7f9cc67fc640 1 -- 172.21.15.1:0/1934655981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb002ae60 con 0x7f9cc815e390 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 -- 172.21.15.1:0/1934655981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc815e390 msgr2=0x7f9cc80cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 --2- 172.21.15.1:0/1934655981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc815e390 0x7f9cc80cf450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80721e0 tx=0x7f9cb002a110 comp rx=0 tx=0).stop 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 -- 172.21.15.1:0/1934655981 shutdown_connections 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 --2- 172.21.15.1:0/1934655981 >> v2:172.21.15.6:3300/0 conn(0x7f9cc80b1900 0x7f9cc80cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 --2- 172.21.15.1:0/1934655981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc815e390 0x7f9cc80cf450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 -- 172.21.15.1:0/1934655981 >> 172.21.15.1:0/1934655981 conn(0x7f9cc80b1da0 msgr2=0x7f9cc80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 -- 172.21.15.1:0/1934655981 shutdown_connections 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.965+0000 7f9cce134640 1 -- 172.21.15.1:0/1934655981 wait complete. 2024-05-09T06:27:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.966+0000 7f9cce134640 1 Processor -- start 2024-05-09T06:27:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.966+0000 7f9cce134640 1 -- start start 2024-05-09T06:27:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cc80b1900 0x7f9cc80b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cc815e390 0x7f9cc815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80cffb0 con 0x7f9cc80cf990 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8162ba0 con 0x7f9cc815e390 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cce134640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc80cce20 con 0x7f9cc80b1900 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56364/0 (socket says 172.21.15.1:56364) 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 -- 172.21.15.1:0/1273967005 learned_addr learned my addr 172.21.15.1:0/1273967005 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc7fff640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cc815e390 0x7f9cc815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc77fe640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cc80b1900 0x7f9cc80b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 -- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cc80b1900 msgr2=0x7f9cc80b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cc80b1900 0x7f9cc80b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 -- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cc815e390 msgr2=0x7f9cc815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cc815e390 0x7f9cc815cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.967+0000 7f9cc6ffd640 1 -- 172.21.15.1:0/1273967005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc80b2e50 con 0x7f9cc80cf990 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cc6ffd640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c4d0 tx=0x7f9cb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8018070 con 0x7f9cc80cf990 2024-05-09T06:27:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb8007500 con 0x7f9cc80cf990 2024-05-09T06:27:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc80adf90 con 0x7f9cc80cf990 2024-05-09T06:27:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004a20 con 0x7f9cc80cf990 2024-05-09T06:27:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.968+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc80ae530 con 0x7f9cc80cf990 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.969+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc80752b0 con 0x7f9cc80cf990 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.969+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cb8004bc0 con 0x7f9cc80cf990 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.970+0000 7f9cbffff640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 0x7f9ca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.970+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9cb808a3b0 con 0x7f9cc80cf990 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.970+0000 7f9cc77fe640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 0x7f9ca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.970+0000 7f9cc77fe640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 0x7f9ca806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0034000 tx=0x7f9cb002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:52.974+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9cb80583e0 con 0x7f9cc80cf990 2024-05-09T06:27:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.158+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cc81616a0 con 0x7f9cc80cf990 2024-05-09T06:27:53.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.158+0000 7f9cbffff640 1 -- 172.21.15.1:0/1273967005 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f9cb805c6e0 con 0x7f9cc80cf990 2024-05-09T06:27:53.140 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:27:53.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 msgr2=0x7f9ca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 0x7f9ca806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0034000 tx=0x7f9cb002e000 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 msgr2=0x7f9cc80ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c4d0 tx=0x7f9cb800c990 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 shutdown_connections 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ca806be30 0x7f9ca806e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cc815e390 0x7f9cc815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cc80cf990 0x7f9cc80ada50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 --2- 172.21.15.1:0/1273967005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cc80b1900 0x7f9cc80b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 >> 172.21.15.1:0/1273967005 conn(0x7f9cc80b1da0 msgr2=0x7f9cc8162320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 shutdown_connections 2024-05-09T06:27:53.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.160+0000 7f9cce134640 1 -- 172.21.15.1:0/1273967005 wait complete. 2024-05-09T06:27:53.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:27:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.278+0000 7fbf28f63640 1 Processor -- start 2024-05-09T06:27:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.278+0000 7fbf28f63640 1 -- start start 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf28f63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbf240d1c70 0x7fbf2415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf28f63640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbf240d45f0 0x7fbf240b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf28f63640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf24161fe0 con 0x7fbf2415e190 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf28f63640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2415f580 con 0x7fbf240d45f0 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf28f63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbf241622c0 con 0x7fbf240d1c70 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbf240d45f0 0x7fbf240b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbf240d45f0 0x7fbf240b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56368/0 (socket says 172.21.15.1:56368) 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 -- 172.21.15.1:0/2403450355 learned_addr learned my addr 172.21.15.1:0/2403450355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf19d9b640 1 --2- 172.21.15.1:0/2403450355 >> v2:172.21.15.6:3300/0 conn(0x7fbf240d1c70 0x7fbf2415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 -- 172.21.15.1:0/2403450355 >> v2:172.21.15.6:3300/0 conn(0x7fbf240d1c70 msgr2=0x7fbf2415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 --2- 172.21.15.1:0/2403450355 >> v2:172.21.15.6:3300/0 conn(0x7fbf240d1c70 0x7fbf2415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 -- 172.21.15.1:0/2403450355 >> v1:172.21.15.6:6789/0 conn(0x7fbf2415e190 legacy=0x7fbf240d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.279+0000 7fbf22d9d640 1 -- 172.21.15.1:0/2403450355 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241625a0 con 0x7fbf240d45f0 2024-05-09T06:27:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf22d9d640 1 --2- 172.21.15.1:0/2403450355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d45f0 0x7fbf240b3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf240b6c30 tx=0x7fbf1c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72a81aa72a727fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf21d9b640 1 -- 172.21.15.1:0/2403450355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c036070 con 0x7fbf240d45f0 2024-05-09T06:27:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf21d9b640 1 -- 172.21.15.1:0/2403450355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c031600 con 0x7fbf240d45f0 2024-05-09T06:27:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf21d9b640 1 -- 172.21.15.1:0/2403450355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0319d0 con 0x7fbf240d45f0 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 -- 172.21.15.1:0/2403450355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d45f0 msgr2=0x7fbf240b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 --2- 172.21.15.1:0/2403450355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d45f0 0x7fbf240b3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf240b6c30 tx=0x7fbf1c02a6f0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 -- 172.21.15.1:0/2403450355 shutdown_connections 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 --2- 172.21.15.1:0/2403450355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d45f0 0x7fbf240b3de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 --2- 172.21.15.1:0/2403450355 >> v2:172.21.15.6:3300/0 conn(0x7fbf240d1c70 0x7fbf2415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.280+0000 7fbf28f63640 1 -- 172.21.15.1:0/2403450355 >> 172.21.15.1:0/2403450355 conn(0x7fbf24073340 msgr2=0x7fbf2409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.281+0000 7fbf28f63640 1 -- 172.21.15.1:0/2403450355 shutdown_connections 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.281+0000 7fbf28f63640 1 -- 172.21.15.1:0/2403450355 wait complete. 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.282+0000 7fbf28f63640 1 Processor -- start 2024-05-09T06:27:53.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.282+0000 7fbf28f63640 1 -- start start 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf240d45f0 0x7fbf2414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf2415e190 0x7fbf240c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240b56d0 con 0x7fbf240d1c70 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24161c60 con 0x7fbf240d45f0 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf28f63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbf240b5850 con 0x7fbf2415e190 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56376/0 (socket says 172.21.15.1:56376) 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 -- 172.21.15.1:0/3297379873 learned_addr learned my addr 172.21.15.1:0/3297379873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf19d9b640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf240d45f0 0x7fbf2414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf22d9d640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf2415e190 0x7fbf240c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 -- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf2415e190 msgr2=0x7fbf240c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf2415e190 0x7fbf240c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 -- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf240d45f0 msgr2=0x7fbf2414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf240d45f0 0x7fbf2414b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 -- 172.21.15.1:0/3297379873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241f3b80 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.283+0000 7fbf2259c640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b3f0 tx=0x7fbf1400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.284+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14015020 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.284+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf14011830 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.284+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf241f3e40 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.284+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011c20 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.284+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241f4400 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.285+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf240752b0 con 0x7fbf240d1c70 2024-05-09T06:27:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.285+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf14018020 con 0x7fbf240d1c70 2024-05-09T06:27:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.286+0000 7fbf1b7fe640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.286+0000 7fbf19d9b640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.286+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf1408b050 con 0x7fbf240d1c70 2024-05-09T06:27:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.286+0000 7fbf19d9b640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 0x7fbf0406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf24161330 tx=0x7fbf0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.290+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf1401f020 con 0x7fbf240d1c70 2024-05-09T06:27:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.473+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbf2408e1c0 con 0x7fbf240d1c70 2024-05-09T06:27:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.473+0000 7fbf1b7fe640 1 -- 172.21.15.1:0/3297379873 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fbf140172c0 con 0x7fbf240d1c70 2024-05-09T06:27:53.455 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 msgr2=0x7fbf0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 0x7fbf0406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf24161330 tx=0x7fbf0c008040 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 msgr2=0x7fbf24148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b3f0 tx=0x7fbf1400b8b0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 shutdown_connections 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf2415e190 0x7fbf240c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf240d45f0 0x7fbf2414b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 --2- 172.21.15.1:0/3297379873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf240d1c70 0x7fbf24148610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 >> 172.21.15.1:0/3297379873 conn(0x7fbf24073340 msgr2=0x7fbf2406bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 shutdown_connections 2024-05-09T06:27:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.475+0000 7fbf28f63640 1 -- 172.21.15.1:0/3297379873 wait complete. 2024-05-09T06:27:53.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 Processor -- start 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 -- start start 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38e40b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e40b33c0 con 0x7f38e40d4940 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e40b3660 con 0x7f38e4161950 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.591+0000 7f38e95d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38e40d4dd0 con 0x7f38e40d2b10 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e27fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f38e40d4940 0x7f38e40b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58850/0 (socket says 172.21.15.1:58850) 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e27fc640 1 -- 172.21.15.1:0/1755714441 learned_addr learned my addr 172.21.15.1:0/1755714441 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:53.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e37fe640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38e40b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661863979 0 0) 0x7f38e40b33c0 con 0x7f38e40d4940 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38cc003520 con 0x7f38e40d4940 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 642423475 0 0) 0x7f38cc003520 con 0x7f38e40d4940 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 msgr2=0x7f38e40b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38e40b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 >> v1:172.21.15.6:6789/0 conn(0x7f38e4161950 legacy=0x7f38e40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e40b3940 con 0x7f38e40d4940 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f38d8001690 con 0x7f38e40d4940 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 >> v1:172.21.15.1:6789/0 conn(0x7f38e40d4940 legacy=0x7f38e40b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc0039e0 0x7f38cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38cc004800 0x7f38cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38e40d4dd0 con 0x7f38cc004800 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc00baa0 con 0x7f38cc0039e0 2024-05-09T06:27:53.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc00bca0 con 0x7f38e40d2b10 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e27fc640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38cc004800 0x7f38cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e37fe640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.592+0000 7f38e2ffd640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc0039e0 0x7f38cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 -- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 msgr2=0x7f38cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40d2b10 0x7f38cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 -- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc0039e0 msgr2=0x7f38cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc0039e0 0x7f38cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 -- 172.21.15.1:0/1755714441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc003520 con 0x7f38cc004800 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.593+0000 7f38e27fc640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38cc004800 0x7f38cc004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38e40b3590 tx=0x7f38cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e7bcd1f69db96d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8007d30 con 0x7f38cc004800 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d8005af0 con 0x7f38cc004800 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e1ffb640 1 -- 172.21.15.1:0/1755714441 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8007070 con 0x7f38cc004800 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 -- 172.21.15.1:0/1755714441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38cc004800 msgr2=0x7f38cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 --2- 172.21.15.1:0/1755714441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38cc004800 0x7f38cc004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38e40b3590 tx=0x7f38cc005830 comp rx=0 tx=0).stop 2024-05-09T06:27:53.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e37fe640 1 -- 172.21.15.1:0/1755714441 reap_dead start 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 -- 172.21.15.1:0/1755714441 shutdown_connections 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 -- 172.21.15.1:0/1755714441 >> 172.21.15.1:0/1755714441 conn(0x7f38e40d3e20 msgr2=0x7f38e40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 -- 172.21.15.1:0/1755714441 shutdown_connections 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.594+0000 7f38e95d5640 1 -- 172.21.15.1:0/1755714441 wait complete. 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.595+0000 7f38e95d5640 1 Processor -- start 2024-05-09T06:27:53.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 -- start start 2024-05-09T06:27:53.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc004800 0x7f38e41f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40cd910 0x7f38e41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38e40d1100 con 0x7f38e40b8d00 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38e40d0f80 con 0x7f38cc004800 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e95d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38e40d1c00 con 0x7f38e40cd910 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56396/0 (socket says 172.21.15.1:56396) 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.596+0000 7f38e37fe640 1 -- 172.21.15.1:0/254846908 learned_addr learned my addr 172.21.15.1:0/254846908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e27fc640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40cd910 0x7f38e41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e2ffd640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc004800 0x7f38e41f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 -- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40cd910 msgr2=0x7f38e41f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40cd910 0x7f38e41f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 -- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc004800 msgr2=0x7f38e41f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc004800 0x7f38e41f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 -- 172.21.15.1:0/254846908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e40b6ae0 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e37fe640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38dc011c80 tx=0x7f38dc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc003990 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc01c070 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e40cd3e0 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc003d50 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.597+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e41f45f0 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.598+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e40752b0 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.599+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38dc004030 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.599+0000 7f38cb7fe640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 0x7f38d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.599+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f38dc08bb90 con 0x7f38e40b8d00 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.599+0000 7f38e2ffd640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 0x7f38d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.600+0000 7f38e2ffd640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 0x7f38d006e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38e409e480 tx=0x7f38d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.604+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38dc010040 con 0x7f38e40b8d00 2024-05-09T06:27:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.831+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38e40d21e0 con 0x7f38e40b8d00 2024-05-09T06:27:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.831+0000 7f38cb7fe640 1 -- 172.21.15.1:0/254846908 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f38dc05a200 con 0x7f38e40b8d00 2024-05-09T06:27:53.814 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:53.814 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 msgr2=0x7f38d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 0x7f38d006e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38e409e480 tx=0x7f38d4009040 comp rx=0 tx=0).stop 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 msgr2=0x7f38e415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38dc011c80 tx=0x7f38dc00f580 comp rx=0 tx=0).stop 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 shutdown_connections 2024-05-09T06:27:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38d006be30 0x7f38d006e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38e40b8d00 0x7f38e415d6e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38e40cd910 0x7f38e41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 --2- 172.21.15.1:0/254846908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38cc004800 0x7f38e41f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 >> 172.21.15.1:0/254846908 conn(0x7f38e40d3e20 msgr2=0x7f38e40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 shutdown_connections 2024-05-09T06:27:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.834+0000 7f38e95d5640 1 -- 172.21.15.1:0/254846908 wait complete. 2024-05-09T06:27:53.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:27:53.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.949+0000 7f0244e38640 1 Processor -- start 2024-05-09T06:27:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.949+0000 7f0244e38640 1 -- start start 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f0244e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f02400cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f0244e38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02400d04a0 con 0x7f02400b0080 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f0244e38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02400722c0 con 0x7f0240071ce0 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f0244e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f02400725a0 con 0x7f024015f3f0 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f023e59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f02400b0080 0x7f024015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58854/0 (socket says 172.21.15.1:58854) 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.950+0000 7f023e59c640 1 -- 172.21.15.1:0/1451774719 learned_addr learned my addr 172.21.15.1:0/1451774719 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023ed9d640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f02400cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585209063 0 0) 0x7f02400d04a0 con 0x7f02400b0080 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f022c003520 con 0x7f02400b0080 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201148093 0 0) 0x7f02400722c0 con 0x7f0240071ce0 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02400d04a0 con 0x7f0240071ce0 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 201069816 0 0) 0x7f022c003520 con 0x7f02400b0080 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 msgr2=0x7f02400cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f02400cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 >> v1:172.21.15.6:6789/0 conn(0x7f0240071ce0 legacy=0x7f024015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0240072880 con 0x7f02400b0080 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f02300019f0 con 0x7f02400b0080 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 >> v1:172.21.15.1:6789/0 conn(0x7f02400b0080 legacy=0x7f024015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f022c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f022c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f022c004d40 0x7f022c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f02400725a0 con 0x7f022c004120 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f022c00bb00 con 0x7f022c004d40 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f022c00bd00 con 0x7f024015f3f0 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f022c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023e59c640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f022c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023ed9d640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f022c004d40 0x7f022c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 -- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 msgr2=0x7f022c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f024015f3f0 0x7f022c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 -- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f022c004d40 msgr2=0x7f022c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f022c004d40 0x7f022c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.951+0000 7f023dd9b640 1 -- 172.21.15.1:0/1451774719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c003520 con 0x7f022c004120 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.952+0000 7f023dd9b640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f022c004800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02280018d0 tx=0x7f022c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505fe6fbceed11e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.952+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280014f0 con 0x7f022c004120 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.952+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280052d0 con 0x7f022c004120 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.952+0000 7f023d59a640 1 -- 172.21.15.1:0/1451774719 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022802abe0 con 0x7f022c004120 2024-05-09T06:27:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 -- 172.21.15.1:0/1451774719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 msgr2=0x7f022c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 --2- 172.21.15.1:0/1451774719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f022c004800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02280018d0 tx=0x7f022c005830 comp rx=0 tx=0).stop 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f023ed9d640 1 -- 172.21.15.1:0/1451774719 reap_dead start 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 -- 172.21.15.1:0/1451774719 shutdown_connections 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 -- 172.21.15.1:0/1451774719 >> 172.21.15.1:0/1451774719 conn(0x7f02400d0b50 msgr2=0x7f02400d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 -- 172.21.15.1:0/1451774719 shutdown_connections 2024-05-09T06:27:53.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.953+0000 7f0244e38640 1 -- 172.21.15.1:0/1451774719 wait complete. 2024-05-09T06:27:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.954+0000 7f0244e38640 1 Processor -- start 2024-05-09T06:27:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.954+0000 7f0244e38640 1 -- start start 2024-05-09T06:27:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02400c53a0 0x7f024015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02400c5820 0x7f024015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f024015ff40 con 0x7f022c004120 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f024015e7c0 con 0x7f02400c5820 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f0244e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f02400cbb60 con 0x7f02400c53a0 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56416/0 (socket says 172.21.15.1:56416) 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 -- 172.21.15.1:0/435897830 learned_addr learned my addr 172.21.15.1:0/435897830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023dd9b640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02400c53a0 0x7f024015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023ed9d640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02400c5820 0x7f024015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 -- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02400c53a0 msgr2=0x7f024015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02400c53a0 0x7f024015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 -- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02400c5820 msgr2=0x7f024015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02400c5820 0x7f024015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.955+0000 7f023e59c640 1 -- 172.21.15.1:0/435897830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400c7910 con 0x7f022c004120 2024-05-09T06:27:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f023e59c640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0230007970 tx=0x7f023000a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230017020 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023000acb0 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023000ae60 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02400b52f0 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.956+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02400c7c20 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.957+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02400752b0 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.957+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f023001a020 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.958+0000 7f0236ffd640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 0x7f022006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.958+0000 7f023dd9b640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 0x7f022006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.958+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f023008bb60 con 0x7f022c004120 2024-05-09T06:27:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.958+0000 7f023dd9b640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 0x7f022006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0228032000 tx=0x7f0228009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:53.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:53.962+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0230021020 con 0x7f022c004120 2024-05-09T06:27:54.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.146+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02400d1280 con 0x7f022c004120 2024-05-09T06:27:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.147+0000 7f0236ffd640 1 -- 172.21.15.1:0/435897830 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18382 (secure 0 0 0) 0x7f02400c7910 con 0x7f022c004120 2024-05-09T06:27:54.130 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:54.130 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":24,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:27:26.035902+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 msgr2=0x7f022006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 0x7f022006e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0228032000 tx=0x7f0228009030 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 msgr2=0x7f02400c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0230007970 tx=0x7f023000a600 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 shutdown_connections 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f022006be30 0x7f022006e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02400c5820 0x7f024015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02400c53a0 0x7f024015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 --2- 172.21.15.1:0/435897830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f022c004120 0x7f02400c73d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.149+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 >> 172.21.15.1:0/435897830 conn(0x7f02400d0b50 msgr2=0x7f02400ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.150+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 shutdown_connections 2024-05-09T06:27:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.150+0000 7f0244e38640 1 -- 172.21.15.1:0/435897830 wait complete. 2024-05-09T06:27:54.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:27:54.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.265+0000 7f829fa76640 1 Processor -- start 2024-05-09T06:27:54.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.265+0000 7f829fa76640 1 -- start start 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829fa76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82980b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829fa76640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82980b4f70 0x7f8298070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829fa76640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829806eb40 con 0x7f82981454b0 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829fa76640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f829806ee20 con 0x7f82980b4f70 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829fa76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f829806d8e0 con 0x7f82980b2a60 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82980b4f70 0x7f8298070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82980b4f70 0x7f8298070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56432/0 (socket says 172.21.15.1:56432) 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 -- 172.21.15.1:0/1481039107 learned_addr learned my addr 172.21.15.1:0/1481039107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829d011640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82980b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 -- 172.21.15.1:0/1481039107 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 msgr2=0x7f82980b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:54.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82980b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 -- 172.21.15.1:0/1481039107 >> v1:172.21.15.6:6789/0 conn(0x7f82981454b0 legacy=0x7f82980b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.266+0000 7f829e013640 1 -- 172.21.15.1:0/1481039107 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829806dbc0 con 0x7f82980b4f70 2024-05-09T06:27:54.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.267+0000 7f829e013640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82980b4f70 0x7f8298070c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f829002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34faf0bd39bb48ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:54.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.267+0000 7f829c810640 1 -- 172.21.15.1:0/1481039107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290036070 con 0x7f82980b4f70 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.267+0000 7f829c810640 1 -- 172.21.15.1:0/1481039107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8290031600 con 0x7f82980b4f70 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.267+0000 7f829c810640 1 -- 172.21.15.1:0/1481039107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900319d0 con 0x7f82980b4f70 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 -- 172.21.15.1:0/1481039107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82980b4f70 msgr2=0x7f8298070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82980b4f70 0x7f8298070c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f829002a710 comp rx=0 tx=0).stop 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 -- 172.21.15.1:0/1481039107 shutdown_connections 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82980b4f70 0x7f8298070c10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 --2- 172.21.15.1:0/1481039107 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82980b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 -- 172.21.15.1:0/1481039107 >> 172.21.15.1:0/1481039107 conn(0x7f82980cc4b0 msgr2=0x7f82980cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 -- 172.21.15.1:0/1481039107 shutdown_connections 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.268+0000 7f829fa76640 1 -- 172.21.15.1:0/1481039107 wait complete. 2024-05-09T06:27:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.269+0000 7f829fa76640 1 Processor -- start 2024-05-09T06:27:54.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.269+0000 7f829fa76640 1 -- start start 2024-05-09T06:27:54.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82981f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82980b4f70 0x7f8298133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f829806e260 con 0x7f82981454b0 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f829806e0e0 con 0x7f82980b4f70 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829fa76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8298071150 con 0x7f82980b2a60 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56440/0 (socket says 172.21.15.1:56440) 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 -- 172.21.15.1:0/2827206781 learned_addr learned my addr 172.21.15.1:0/2827206781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829d812640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82981f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829d011640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82980b4f70 0x7f8298133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 -- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 msgr2=0x7f82981f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82981f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 -- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82980b4f70 msgr2=0x7f8298133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82980b4f70 0x7f8298133a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.270+0000 7f829e013640 1 -- 172.21.15.1:0/2827206781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8298133f40 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f829e013640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f8290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290041020 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8290007800 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8298134210 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290036040 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.271+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981347d0 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.272+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82980b3f00 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.272+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8290044020 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.273+0000 7f828e7fc640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 0x7f827c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.273+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f82900abd90 con 0x7f82981454b0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.273+0000 7f829d812640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 0x7f827c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:54.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.273+0000 7f829d812640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 0x7f827c06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82940016e0 tx=0x7f8294009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:54.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.277+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82980b3f00 con 0x7f82981454b0 2024-05-09T06:27:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.498+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82980752b0 con 0x7f82981454b0 2024-05-09T06:27:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.499+0000 7f828e7fc640 1 -- 172.21.15.1:0/2827206781 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f829004b030 con 0x7f82981454b0 2024-05-09T06:27:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:27:54.481 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:27:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 msgr2=0x7f827c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 0x7f827c06e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82940016e0 tx=0x7f8294009040 comp rx=0 tx=0).stop 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 msgr2=0x7f82980ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f8290004a40 comp rx=0 tx=0).stop 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 shutdown_connections 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f827c06be30 0x7f827c06e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82981454b0 0x7f82980ae770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82980b4f70 0x7f8298133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 --2- 172.21.15.1:0/2827206781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82980b2a60 0x7f82981f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 >> 172.21.15.1:0/2827206781 conn(0x7f82980cc4b0 msgr2=0x7f82980b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 shutdown_connections 2024-05-09T06:27:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:54.501+0000 7f829fa76640 1 -- 172.21.15.1:0/2827206781 wait complete. 2024-05-09T06:27:54.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:27:20.997818+0000', 'modified': '2024-05-09T06:27:43.117905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6834/278895687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6832', 'nonce': 278895687}, {'type': 'v1', 'addr': '172.21.15.6:6834', 'nonce': 278895687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-05-09T06:27:54.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:27:54.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T06:27:55.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.025+0000 7fb9c6910640 1 Processor -- start 2024-05-09T06:27:55.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.025+0000 7fb9c6910640 1 -- start start 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9c6910640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb9c0071ce0 0x7fb9c00cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9c6910640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c00d04a0 con 0x7fb9c00b0080 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9c6910640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c00722c0 con 0x7fb9c015ece0 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9c6910640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c00725a0 con 0x7fb9c0071ce0 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb9c00b0080 0x7fb9c015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58866/0 (socket says 172.21.15.1:58866) 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bf7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb9c0071ce0 0x7fb9c00cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bffff640 1 -- 172.21.15.1:0/1661222321 learned_addr learned my addr 172.21.15.1:0/1661222321 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/1661222321 >> v1:172.21.15.6:6789/0 conn(0x7fb9c015ece0 legacy=0x7fb9c015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/1661222321 >> v1:172.21.15.1:6789/0 conn(0x7fb9c00b0080 legacy=0x7fb9c015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/1661222321 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c0072880 con 0x7fb9c0071ce0 2024-05-09T06:27:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.026+0000 7fb9beffd640 1 -- 172.21.15.1:0/1661222321 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062485239 0 0) 0x7fb9c00d04a0 con 0x7fb9c00b0080 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.027+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/1661222321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c0071ce0 0x7fb9c00cbb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097a0 tx=0x7fb9b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a3265b8bccab53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.027+0000 7fb9beffd640 1 -- 172.21.15.1:0/1661222321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4037070 con 0x7fb9c0071ce0 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.027+0000 7fb9beffd640 1 -- 172.21.15.1:0/1661222321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4032440 con 0x7fb9c0071ce0 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.027+0000 7fb9beffd640 1 -- 172.21.15.1:0/1661222321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4032810 con 0x7fb9c0071ce0 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 -- 172.21.15.1:0/1661222321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c0071ce0 msgr2=0x7fb9c00cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 --2- 172.21.15.1:0/1661222321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c0071ce0 0x7fb9c00cbb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097a0 tx=0x7fb9b402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 -- 172.21.15.1:0/1661222321 shutdown_connections 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 --2- 172.21.15.1:0/1661222321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c0071ce0 0x7fb9c00cbb60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 -- 172.21.15.1:0/1661222321 >> 172.21.15.1:0/1661222321 conn(0x7fb9c00d0b50 msgr2=0x7fb9c00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 -- 172.21.15.1:0/1661222321 shutdown_connections 2024-05-09T06:27:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.028+0000 7fb9c6910640 1 -- 172.21.15.1:0/1661222321 wait complete. 2024-05-09T06:27:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.029+0000 7fb9c6910640 1 Processor -- start 2024-05-09T06:27:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.029+0000 7fb9c6910640 1 -- start start 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 0x7fb9c00c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 0x7fb9c015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c015ff40 con 0x7fb9c015ece0 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c015e7c0 con 0x7fb9c00b0080 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c6910640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c006b1e0 con 0x7fb9c0071ce0 2024-05-09T06:27:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56458/0 (socket says 172.21.15.1:56458) 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 -- 172.21.15.1:0/3833531846 learned_addr learned my addr 172.21.15.1:0/3833531846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 0x7fb9c015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 0x7fb9c00c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 -- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 msgr2=0x7fb9c00c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 0x7fb9c00c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 -- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 msgr2=0x7fb9c015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 0x7fb9c015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9c4ead640 1 -- 172.21.15.1:0/3833531846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c00c78d0 con 0x7fb9c015ece0 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 0x7fb9c00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.030+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 0x7fb9c015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b970 tx=0x7fb9b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9bcff9640 1 -- 172.21.15.1:0/3833531846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8019070 con 0x7fb9c015ece0 2024-05-09T06:27:55.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9bcff9640 1 -- 172.21.15.1:0/3833531846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b800cb40 con 0x7fb9c015ece0 2024-05-09T06:27:55.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9bcff9640 1 -- 172.21.15.1:0/3833531846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8014440 con 0x7fb9c015ece0 2024-05-09T06:27:55.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c015d2c0 con 0x7fb9c015ece0 2024-05-09T06:27:55.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.031+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c00c7310 con 0x7fb9c015ece0 2024-05-09T06:27:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.032+0000 7fb9bcff9640 1 -- 172.21.15.1:0/3833531846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9b801c070 con 0x7fb9c015ece0 2024-05-09T06:27:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.033+0000 7fb9bcff9640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 0x7fb9ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.033+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 0x7fb9ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.033+0000 7fb9bcff9640 1 -- 172.21.15.1:0/3833531846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb9b808ae60 con 0x7fb9c015ece0 2024-05-09T06:27:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.034+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 0x7fb9ac06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000f60 tx=0x7fb9b0007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.034+0000 7fb9c961d740 1 Processor -- start 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.034+0000 7fb9c961d740 1 -- start start 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b50a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 0x55e244b6e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e244a894d0 con 0x55e244b26350 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e244a313e0 con 0x55e244b50670 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e244a85490 con 0x55e244b6df10 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56466/0 (socket says 172.21.15.1:56466) 2024-05-09T06:27:55.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 -- 172.21.15.1:0/4271883708 learned_addr learned my addr 172.21.15.1:0/4271883708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 0x55e244b6e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 -- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 msgr2=0x55e244b6e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9bffff640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b50a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 0x55e244b6e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 -- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 msgr2=0x55e244b50a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b50a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 -- 172.21.15.1:0/4271883708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24479d2d0 con 0x55e244b26350 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 0x55e244b6e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9bffff640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.035+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8088f50 tx=0x7fb9b8059570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6b96508305fcd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9a5ffb640 1 -- 172.21.15.1:0/4271883708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8087070 con 0x55e244b26350 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9a5ffb640 1 -- 172.21.15.1:0/4271883708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b8061910 con 0x55e244b26350 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9a5ffb640 1 -- 172.21.15.1:0/4271883708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80553a0 con 0x55e244b26350 2024-05-09T06:27:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 -- 172.21.15.1:0/4271883708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 msgr2=0x55e244b6c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8088f50 tx=0x7fb9b8059570 comp rx=0 tx=0).stop 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 -- 172.21.15.1:0/4271883708 shutdown_connections 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b26350 0x55e244b6c5d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b6df10 0x55e244b6e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 --2- 172.21.15.1:0/4271883708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 -- 172.21.15.1:0/4271883708 >> 172.21.15.1:0/4271883708 conn(0x55e244b22c80 msgr2=0x55e244b6b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 -- 172.21.15.1:0/4271883708 shutdown_connections 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.036+0000 7fb9c961d740 1 -- 172.21.15.1:0/4271883708 wait complete. 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.037+0000 7fb9c961d740 1 -- start start 2024-05-09T06:27:55.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.037+0000 7fb9c961d740 1 Processor -- start 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b4d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b1f770 0x55e244b1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e244b270f0 con 0x55e244b6df10 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e244a448e0 con 0x55e244b50670 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e244b20bb0 con 0x55e244b1f770 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c961d740 10 client.0 _ll_register_callbacks cb 0x55e244a119c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:27:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56478/0 (socket says 172.21.15.1:56478) 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/3212183234 learned_addr learned my addr 172.21.15.1:0/3212183234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b1f770 0x55e244b1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b4d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b1f770 msgr2=0x55e244b1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b1f770 0x55e244b1fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 msgr2=0x55e244b4d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b4d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 -- 172.21.15.1:0/3212183234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2448676b0 con 0x55e244b6df10 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402a7d0 tx=0x7fb9b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0d73896806878a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb99effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:27:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.038+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4037040 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb9b4047de0 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b8061910 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb9b4032a50 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 10 client.4863 fetch_fsmap learned FSMap version 9 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55e244b64840 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b8061910 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fb9b404e050 con 0x55e244b6df10 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 10 client.4863 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 20 client.4863 populate_metadata read hostname 'smithi001' 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244b7bf00 0x55e244b7c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 4 client.4863 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 --> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e244a96c00 con 0x55e244b7bf00 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.039+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244b7bf00 0x55e244b7c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.040+0000 7fb9bffff640 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244b7bf00 0x55e244b7c2e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6043070223b792 server_cookie=e9b8cf81a19f01cf in_seq=0 out_seq=0 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.040+0000 7fb99effd640 10 client.4863 ms_handle_connect on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb99effd640 1 -- 172.21.15.1:0/3212183234 <== mds.0 v2:172.21.15.6:6832/278895687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e244a96c00 con 0x55e244b7bf00 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb99effd640 10 client.4863 handle_command_reply: tid=0 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 client.4863 shutdown 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 msgr2=0x55e244b53010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402a7d0 tx=0x7fb9b4004730 comp rx=0 tx=0).stop 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 shutdown_connections 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244b7bf00 0x55e244b7c2e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b1f770 0x55e244b1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b53010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb99effd640 0 client.4863 ms_handle_reset on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 --2- 172.21.15.1:0/3212183234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b50670 0x55e244b4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 >> 172.21.15.1:0/3212183234 conn(0x55e244b26350 msgr2=0x55e244b6ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 shutdown_connections 2024-05-09T06:27:55.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 1 -- 172.21.15.1:0/3212183234 wait complete. 2024-05-09T06:27:55.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.041+0000 7fb9c961d740 20 client.4863 trim_cache size 0 max 16384 2024-05-09T06:27:55.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 Processor -- start 2024-05-09T06:27:55.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 -- start start 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b1bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b6df10 0x55e244b26940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c0200aa0 con 0x55e244b1f770 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00c6d70 con 0x55e244b6df10 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c0200450 con 0x55e244b50670 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.060+0000 7fb9bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56480/0 (socket says 172.21.15.1:56480) 2024-05-09T06:27:55.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 -- 172.21.15.1:0/2500519803 learned_addr learned my addr 172.21.15.1:0/2500519803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 -- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 msgr2=0x55e244b1bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b1bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 -- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b6df10 msgr2=0x55e244b26940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b6df10 0x55e244b26940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 -- 172.21.15.1:0/2500519803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e244b53380 con 0x55e244b1f770 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9bffff640 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007350 tx=0x7fb9b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2886770c86825a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9a57fa640 1 -- 172.21.15.1:0/2500519803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0014a70 con 0x55e244b1f770 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9a57fa640 1 -- 172.21.15.1:0/2500519803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b0029070 con 0x55e244b1f770 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9a57fa640 1 -- 172.21.15.1:0/2500519803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0014e30 con 0x55e244b1f770 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9c961d740 1 -- 172.21.15.1:0/2500519803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 msgr2=0x55e244b27720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.061+0000 7fb9c961d740 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007350 tx=0x7fb9b0012000 comp rx=0 tx=0).stop 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 -- 172.21.15.1:0/2500519803 shutdown_connections 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b6df10 0x55e244b26940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b1bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 --2- 172.21.15.1:0/2500519803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b1f770 0x55e244b27720 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 -- 172.21.15.1:0/2500519803 >> 172.21.15.1:0/2500519803 conn(0x55e244b26350 msgr2=0x55e244b4ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 -- 172.21.15.1:0/2500519803 shutdown_connections 2024-05-09T06:27:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 -- 172.21.15.1:0/2500519803 wait complete. 2024-05-09T06:27:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 -- start start 2024-05-09T06:27:55.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.062+0000 7fb9c961d740 1 Processor -- start 2024-05-09T06:27:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 0x55e244b6ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e244b53550 con 0x55e244b6df10 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e244b64b20 con 0x55e244b1f770 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e244b62d30 con 0x55e244b50670 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c961d740 10 client.0 _ll_register_callbacks cb 0x7fb9b4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56490/0 (socket says 172.21.15.1:56490) 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 -- 172.21.15.1:0/1814243131 learned_addr learned my addr 172.21.15.1:0/1814243131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9bf7fe640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9bffff640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 0x55e244b6ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 -- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 msgr2=0x55e244b84f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b84f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 -- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 msgr2=0x55e244b6ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 0x55e244b6ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9c4ead640 1 -- 172.21.15.1:0/1814243131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e244b4ff40 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.063+0000 7fb9bffff640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 0x55e244b6ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c4ead640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9b805d2a0 tx=0x7fb9b8088e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bef205670c8b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8087070 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb9b8061680 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b0029070 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb9b807e050 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 10 client.4869 fetch_fsmap learned FSMap version 9 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55e244be5b60 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b0029070 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fb9b8081020 con 0x55e244b6df10 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 10 client.4869 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:27:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 20 client.4869 populate_metadata read hostname 'smithi001' 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244bce160 0x55e244b6a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 4 client.4869 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 --> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] -- command(tid 0: {"prefix": "status"}) -- 0x55e244a90f50 con 0x55e244bce160 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.064+0000 7fb9bffff640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244bce160 0x55e244b6a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb99effd640 10 client.4869 ms_handle_connect on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9bffff640 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244bce160 0x55e244b6a410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926c7506d33ecac3 server_cookie=f3319ae3ae9cfbda in_seq=0 out_seq=0 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb99effd640 1 -- 172.21.15.1:0/1814243131 <== mds.0 v2:172.21.15.6:6832/278895687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55e244a90f50 con 0x55e244bce160 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb99effd640 10 client.4869 handle_command_reply: tid=0 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 client.4869 shutdown 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 msgr2=0x55e244b4fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9b805d2a0 tx=0x7fb9b8088e80 comp rx=0 tx=0).stop 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 shutdown_connections 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:6832/278895687,v1:172.21.15.6:6834/278895687] conn(0x55e244bce160 0x55e244b6a410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e244b6df10 0x55e244b4fa00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e244b50670 0x55e244b84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb99effd640 0 client.4869 ms_handle_reset on v2:172.21.15.6:6832/278895687 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 --2- 172.21.15.1:0/1814243131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e244b1f770 0x55e244b6ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 >> 172.21.15.1:0/1814243131 conn(0x55e244b26350 msgr2=0x55e244b8f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 shutdown_connections 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 1 -- 172.21.15.1:0/1814243131 wait complete. 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.065+0000 7fb9c961d740 20 client.4869 trim_cache size 0 max 16384 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4290, 2024-05-09T06:27:55.053 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 33.429642530000002, 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 9, 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 24, 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 24, 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 40.45540123 2024-05-09T06:27:55.054 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 msgr2=0x7fb9ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 0x7fb9ac06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000f60 tx=0x7fb9b0007c10 comp rx=0 tx=0).stop 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 msgr2=0x7fb9c015dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b970 tx=0x7fb9b800be30 comp rx=0 tx=0).stop 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 shutdown_connections 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb9ac06be30 0x7fb9ac06e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9c015ece0 0x7fb9c015dc20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c00b0080 0x7fb9c015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 --2- 172.21.15.1:0/3833531846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9c0071ce0 0x7fb9c00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 >> 172.21.15.1:0/3833531846 conn(0x7fb9c00d0b50 msgr2=0x7fb9c00b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 shutdown_connections 2024-05-09T06:27:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.066+0000 7fb9c6910640 1 -- 172.21.15.1:0/3833531846 wait complete. 2024-05-09T06:27:55.059 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T06:27:55.068 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T06:27:55.068 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T06:27:55.068 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T06:27:55.068 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T06:27:55.088 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T06:27:55.088 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-09T06:27:55.089 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:55.089 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T06:27:55.089 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:27:55.089 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:27:55.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T06:27:55.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.090 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T06:27:55.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.110 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-05-09T06:27:55.182 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2024-05-09T06:27:55.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:27:55.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.184 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:27:55.200 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T06:27:55.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T06:27:55.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-09T06:27:55.375 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T06:27:55.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:27:55.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.499+0000 7f97e30b3640 1 Processor -- start 2024-05-09T06:27:55.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.499+0000 7f97e30b3640 1 -- start start 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e30b3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97dc06e6d0 0x7f97dc06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e30b3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 0x7f97dc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e30b3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc0ce0e0 con 0x7f97dc0cdab0 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e30b3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc0cdec0 con 0x7f97dc06e6d0 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e30b3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc0ce3c0 con 0x7f97dc0cf180 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e0e4f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97dc06e6d0 0x7f97dc06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e0e4f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97dc06e6d0 0x7f97dc06eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56506/0 (socket says 172.21.15.1:56506) 2024-05-09T06:27:55.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.500+0000 7f97e0e4f640 1 -- 172.21.15.1:0/996595698 learned_addr learned my addr 172.21.15.1:0/996595698 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97dbfff640 1 --2- 172.21.15.1:0/996595698 >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 0x7f97dc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97e0e4f640 1 -- 172.21.15.1:0/996595698 >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 msgr2=0x7f97dc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97e0e4f640 1 --2- 172.21.15.1:0/996595698 >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 0x7f97dc0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97e0e4f640 1 -- 172.21.15.1:0/996595698 >> v1:172.21.15.6:6789/0 conn(0x7f97dc0cdab0 legacy=0x7f97dc15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97e0e4f640 1 -- 172.21.15.1:0/996595698 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0ce6a0 con 0x7f97dc06e6d0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97dbfff640 1 --2- 172.21.15.1:0/996595698 >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 0x7f97dc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97e0e4f640 1 --2- 172.21.15.1:0/996595698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc06e6d0 0x7f97dc06eab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b27c0 tx=0x7f97cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2bf2f5ce09e2ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.501+0000 7f97db7fe640 1 -- 172.21.15.1:0/996595698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc037070 con 0x7f97dc06e6d0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97db7fe640 1 -- 172.21.15.1:0/996595698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc032440 con 0x7f97dc06e6d0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97db7fe640 1 -- 172.21.15.1:0/996595698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc032810 con 0x7f97dc06e6d0 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 -- 172.21.15.1:0/996595698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc06e6d0 msgr2=0x7f97dc06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 --2- 172.21.15.1:0/996595698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc06e6d0 0x7f97dc06eab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b27c0 tx=0x7f97cc02a230 comp rx=0 tx=0).stop 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 -- 172.21.15.1:0/996595698 shutdown_connections 2024-05-09T06:27:55.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 --2- 172.21.15.1:0/996595698 >> v2:172.21.15.6:3300/0 conn(0x7f97dc0cf180 0x7f97dc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 --2- 172.21.15.1:0/996595698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc06e6d0 0x7f97dc06eab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 -- 172.21.15.1:0/996595698 >> 172.21.15.1:0/996595698 conn(0x7f97dc0cc4b0 msgr2=0x7f97dc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 -- 172.21.15.1:0/996595698 shutdown_connections 2024-05-09T06:27:55.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.502+0000 7f97e30b3640 1 -- 172.21.15.1:0/996595698 wait complete. 2024-05-09T06:27:55.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.503+0000 7f97e30b3640 1 Processor -- start 2024-05-09T06:27:55.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.503+0000 7f97e30b3640 1 -- start start 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97dc06e6d0 0x7f97dc143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97dc0cf180 0x7f97dc12ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc06eff0 con 0x7f97dc0cdab0 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0cec60 con 0x7f97dc06e6d0 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97e30b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc070ed0 con 0x7f97dc0cf180 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56514/0 (socket says 172.21.15.1:56514) 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.504+0000 7f97dbfff640 1 -- 172.21.15.1:0/1519231772 learned_addr learned my addr 172.21.15.1:0/1519231772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97e0e4f640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97dc06e6d0 0x7f97dc143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 -- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97dc0cf180 msgr2=0x7f97dc12ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97dc0cf180 0x7f97dc12ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 -- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97dc06e6d0 msgr2=0x7f97dc143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97dc06e6d0 0x7f97dc143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 -- 172.21.15.1:0/1519231772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc130450 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97dbfff640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c820 tx=0x7f97d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0015070 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d0007a30 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc130730 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0007e00 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.505+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc130cf0 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.506+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc0752b0 con 0x7f97dc0cdab0 2024-05-09T06:27:55.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.506+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97d0018070 con 0x7f97dc0cdab0 2024-05-09T06:27:55.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.507+0000 7f97d97fa640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 0x7f97c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:55.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.507+0000 7f97e0e4f640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 0x7f97c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:55.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.507+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f97d008b110 con 0x7f97dc0cdab0 2024-05-09T06:27:55.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.508+0000 7f97e0e4f640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 0x7f97c806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97cc034000 tx=0x7f97cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:55.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.512+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97d0059780 con 0x7f97dc0cdab0 2024-05-09T06:27:55.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.734+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f97dc06cf50 con 0x7f97dc0cdab0 2024-05-09T06:27:55.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.734+0000 7f97d97fa640 1 -- 172.21.15.1:0/1519231772 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f97d0059080 con 0x7f97dc0cdab0 2024-05-09T06:27:55.717 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 msgr2=0x7f97c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 0x7f97c806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97cc034000 tx=0x7f97cc02e000 comp rx=0 tx=0).stop 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 msgr2=0x7f97dc168640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c820 tx=0x7f97d000cce0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 shutdown_connections 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f97c806be60 0x7f97c806e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97dc0cf180 0x7f97dc12ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97dc0cdab0 0x7f97dc168640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 --2- 172.21.15.1:0/1519231772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97dc06e6d0 0x7f97dc143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 >> 172.21.15.1:0/1519231772 conn(0x7f97dc0cc4b0 msgr2=0x7f97dc13a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 shutdown_connections 2024-05-09T06:27:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55.737+0000 7f97e30b3640 1 -- 172.21.15.1:0/1519231772 wait complete. 2024-05-09T06:27:55.730 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:27:55.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:27:55.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:27:55.730 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:27:55.730 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:55.731 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:27:55.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:27:55.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:27:55.731 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:27:55.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.731 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:27:55.747 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:55.747 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:27:55.748 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42667sec preferred_lft 42667sec 2024-05-09T06:27:55.749 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:27:55.749 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:55.749 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:55.749 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:55.749 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:27:55.749 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:27:55.749 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:55.749 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:55.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:27:55.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-05-09T06:27:55.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-05-09T06:27:55.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:27:55.750 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:55.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:55.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.891 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:27:55.915 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:27:55.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:55.917 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:27:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:27:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:27:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:27:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:27:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:27:55.974 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:55.974 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:55.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:27:55.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:27:55.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:27:55.975 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:56.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:56.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:56.104 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:27:56.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:56.120 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-09T06:27:56.176 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:56.176 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:56.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:56.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T06:27:56.176 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:56.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:56.297 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T06:27:56.297 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:56.297 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:56.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T06:27:56.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:27:56.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T06:27:56.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T06:27:56.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T06:27:56.298 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:56.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:56.415 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:56.415 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:56.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:27:56.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:27:56.415 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:56.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:56.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:56.494 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:27:56.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:56.495 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:56.511 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:27:56.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:56.511 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:56.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:56.633 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:56.691 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:56.692 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:56.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:56.693 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:56.747 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:27:56.747 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:27:56.789 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:56.815 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:56.872 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:56.872 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:56.872 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:56.873 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:56.874 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:56.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:56.875 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:56.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:27:56.892+0000 7f49b313a480 -1 init, newargv = 0x559c6f334540 newargc=15 2024-05-09T06:27:56.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[45835]: starting ceph client 2024-05-09T06:27:56.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[45835]: starting fuse 2024-05-09T06:27:56.892 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:27:56.892 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:27:56.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:27:56.892 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:27:56.893 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:27:56.894 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:27:57.014 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.45835.asok 2024-05-09T06:27:57.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:57.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45835.asok 2024-05-09T06:27:57.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.025 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45835.asok status 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "45835", 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:27:57.165 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4881, 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 4881 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3555917288 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:57.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3555917288 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.4881 192.168.0.1:0/3555917288", 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/3555917288", 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 9, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 24, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:27:57.167 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:27:57.173 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:27:57.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.295+0000 7f93699a2640 1 Processor -- start 2024-05-09T06:27:57.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.295+0000 7f93699a2640 1 -- start start 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93699a2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9364071ce0 0x7f93640cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93699a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f936415ece0 0x7f936415f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93699a2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93640d04a0 con 0x7f93640b0080 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93699a2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f936415f670 con 0x7f9364071ce0 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93699a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f93640722c0 con 0x7f936415ece0 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9364071ce0 0x7f93640cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9364071ce0 0x7f93640cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56524/0 (socket says 172.21.15.1:56524) 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 -- 172.21.15.1:0/321645846 learned_addr learned my addr 172.21.15.1:0/321645846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f93637fe640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f936415ece0 0x7f936415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 -- 172.21.15.1:0/321645846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f936415ece0 msgr2=0x7f936415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f936415ece0 0x7f936415f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 -- 172.21.15.1:0/321645846 >> v1:172.21.15.6:6789/0 conn(0x7f93640b0080 legacy=0x7f936415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.296+0000 7f935a7fc640 1 -- 172.21.15.1:0/321645846 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93640725a0 con 0x7f9364071ce0 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.297+0000 7f935a7fc640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9364071ce0 0x7f93640cbb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f934c009870 tx=0x7f934c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b537450dabf777 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.297+0000 7f93627fc640 1 -- 172.21.15.1:0/321645846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c037070 con 0x7f9364071ce0 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.297+0000 7f93627fc640 1 -- 172.21.15.1:0/321645846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c032440 con 0x7f9364071ce0 2024-05-09T06:27:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.297+0000 7f93627fc640 1 -- 172.21.15.1:0/321645846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c032810 con 0x7f9364071ce0 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 -- 172.21.15.1:0/321645846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9364071ce0 msgr2=0x7f93640cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9364071ce0 0x7f93640cbb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f934c009870 tx=0x7f934c02a270 comp rx=0 tx=0).stop 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 -- 172.21.15.1:0/321645846 shutdown_connections 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f936415ece0 0x7f936415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 --2- 172.21.15.1:0/321645846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9364071ce0 0x7f93640cbb60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 -- 172.21.15.1:0/321645846 >> 172.21.15.1:0/321645846 conn(0x7f93640d0b50 msgr2=0x7f93640d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 -- 172.21.15.1:0/321645846 shutdown_connections 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.298+0000 7f93699a2640 1 -- 172.21.15.1:0/321645846 wait complete. 2024-05-09T06:27:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.299+0000 7f93699a2640 1 Processor -- start 2024-05-09T06:27:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.299+0000 7f93699a2640 1 -- start start 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9364071ce0 0x7f936415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93640b0080 0x7f93640c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f936415e7c0 con 0x7f936415ece0 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9364072b60 con 0x7f9364071ce0 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93699a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f936415ff40 con 0x7f93640b0080 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56538/0 (socket says 172.21.15.1:56538) 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 -- 172.21.15.1:0/3410877046 learned_addr learned my addr 172.21.15.1:0/3410877046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f935a7fc640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93640b0080 0x7f93640c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f9362ffd640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9364071ce0 0x7f936415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 -- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93640b0080 msgr2=0x7f93640c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93640b0080 0x7f93640c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 -- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9364071ce0 msgr2=0x7f936415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9364071ce0 0x7f936415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 -- 172.21.15.1:0/3410877046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93640b4490 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.300+0000 7f93637fe640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f935c00efc0 tx=0x7f935c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.301+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c018020 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.301+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c014480 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.301+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93640b4770 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.301+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c014870 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.301+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93640b4d30 con 0x7f936415ece0 2024-05-09T06:27:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.302+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93640752b0 con 0x7f936415ece0 2024-05-09T06:27:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.302+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f935c005d10 con 0x7f936415ece0 2024-05-09T06:27:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.303+0000 7f9360ff9640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 0x7f934406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.303+0000 7f9362ffd640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 0x7f934406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.303+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f935c08be50 con 0x7f936415ece0 2024-05-09T06:27:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.303+0000 7f9362ffd640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 0x7f934406e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9354001640 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.307+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f935c022030 con 0x7f936415ece0 2024-05-09T06:27:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.529+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93640b67f0 con 0x7f936415ece0 2024-05-09T06:27:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.529+0000 7f9360ff9640 1 -- 172.21.15.1:0/3410877046 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f935c010040 con 0x7f936415ece0 2024-05-09T06:27:57.511 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 msgr2=0x7f934406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 0x7f934406e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9354001640 tx=0x7f9354009040 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 msgr2=0x7f93640b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f935c00efc0 tx=0x7f935c00c490 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 shutdown_connections 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f934406be60 0x7f934406e320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f936415ece0 0x7f93640b5530 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93640b0080 0x7f93640c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 --2- 172.21.15.1:0/3410877046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9364071ce0 0x7f936415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 >> 172.21.15.1:0/3410877046 conn(0x7f93640d0b50 msgr2=0x7f93640cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 shutdown_connections 2024-05-09T06:27:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57.531+0000 7f93699a2640 1 -- 172.21.15.1:0/3410877046 wait complete. 2024-05-09T06:27:57.524 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:27:57.524 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:27:57.524 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:27:57.524 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-09T06:27:57.524 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T06:27:57.525 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:27:57.525 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:27:57.525 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:27:57.525 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-09T06:27:57.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.525 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42665sec preferred_lft 42665sec 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:27:57.542 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link tentative 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link tentative 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:57.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.543 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:27:57.607 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:27:57.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.609 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:27:57.666 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:27:57.666 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:27:57.666 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:27:57.666 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:27:57.666 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:27:57.666 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:57.666 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:57.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:27:57.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:27:57.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:27:57.667 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:57.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:57.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.796 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:27:57.812 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:27:57.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.812 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-09T06:27:57.867 INFO:teuthology.orchestra.run.smithi001.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-09T06:27:57.867 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:57.868 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:57.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:27:57.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-09T06:27:57.868 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:57.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:57.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T06:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 6a:ca:ef:81:f8:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::68ca:efff:fe81:f84b/64 scope link tentative 2024-05-09T06:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:58.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T06:27:58.080 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 6a:ca:ef:81:f8:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::68ca:efff:fe81:f84b/64 scope link tentative 2024-05-09T06:27:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:58.082 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-09T06:27:58.083 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:58.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:58.238 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:58.239 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:27:58.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.1 up 2024-05-09T06:27:58.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T06:27:58.239 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:27:58.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:58.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:58.318 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-09T06:27:58.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:58.318 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-09T06:27:58.334 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-09T06:27:58.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:27:58.335 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-09T06:27:58.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:58.456 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:58.514 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:58.515 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:27:58.516 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:27:58.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:58.517 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:58.569 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:27:58.570 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-05-09T06:27:58.570 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-09T06:27:58.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:27:58.637 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:27:58.694 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:27:58.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:27:58.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:27:58.697 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:27:58.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:58.698 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:27:58.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi001.stderr:2024-05-09T06:27:58.715+0000 7f8d0f979480 -1 init, newargv = 0x55b7b4cec3b0 newargc=15 2024-05-09T06:27:58.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi001.stderr:ceph-fuse[46333]: starting ceph client 2024-05-09T06:27:58.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi001.stderr:ceph-fuse[46333]: starting fuse 2024-05-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi001.stdout:43 2024-05-09T06:27:58.721 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-09T06:27:58.721 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:27:58.722 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:27:58.723 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:27:58.723 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.2")) 2024-05-09T06:27:58.723 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:27:58.842 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.2.46333.asok 2024-05-09T06:27:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:58.852 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46333.asok 2024-05-09T06:27:58.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:58.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46333.asok status 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "2", 2024-05-09T06:27:58.991 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "46333", 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4896, 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 4896 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:58.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.2:0", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 460624060 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.2:0", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 460624060 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.4896 192.168.0.2:0/460624060", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.2:0/460624060", 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 9, 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 24, 2024-05-09T06:27:58.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:27:58.994 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:27:58.994 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:27:58.994 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:27:59.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:27:59.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.122+0000 7f1e7011d640 1 Processor -- start 2024-05-09T06:27:59.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.122+0000 7f1e7011d640 1 -- start start 2024-05-09T06:27:59.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e7011d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1e680c6860 0x7f1e68099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e7011d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1e6815e4b0 0x7f1e6815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e7011d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e68099930 con 0x7f1e680b2aa0 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e7011d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1e680b1e20 con 0x7f1e6815e4b0 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e7011d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6815ee90 con 0x7f1e680c6860 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6e6ba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1e6815e4b0 0x7f1e6815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6d6b8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1e680b2aa0 0x7f1e680c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45678/0 (socket says 172.21.15.1:45678) 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6d6b8640 1 -- 172.21.15.1:0/869315395 learned_addr learned my addr 172.21.15.1:0/869315395 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6deb9640 1 --2- 172.21.15.1:0/869315395 >> v2:172.21.15.6:3300/0 conn(0x7f1e680c6860 0x7f1e68099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6e6ba640 1 -- 172.21.15.1:0/869315395 >> v2:172.21.15.6:3300/0 conn(0x7f1e680c6860 msgr2=0x7f1e68099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6e6ba640 1 --2- 172.21.15.1:0/869315395 >> v2:172.21.15.6:3300/0 conn(0x7f1e680c6860 0x7f1e68099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6e6ba640 1 -- 172.21.15.1:0/869315395 >> v1:172.21.15.1:6789/0 conn(0x7f1e680b2aa0 legacy=0x7f1e680c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6e6ba640 1 -- 172.21.15.1:0/869315395 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e68162930 con 0x7f1e6815e4b0 2024-05-09T06:27:59.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.123+0000 7f1e6ceb7640 1 -- 172.21.15.1:0/869315395 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111737716 0 0) 0x7f1e64001110 con 0x7f1e680b2aa0 2024-05-09T06:27:59.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.124+0000 7f1e6e6ba640 1 --2- 172.21.15.1:0/869315395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e6815e4b0 0x7f1e6815f320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e680722c0 tx=0x7f1e6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a49d8980363e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.124+0000 7f1e6ceb7640 1 -- 172.21.15.1:0/869315395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60007740 con 0x7f1e6815e4b0 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.124+0000 7f1e6ceb7640 1 -- 172.21.15.1:0/869315395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600078e0 con 0x7f1e6815e4b0 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.124+0000 7f1e6ceb7640 1 -- 172.21.15.1:0/869315395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60007cb0 con 0x7f1e6815e4b0 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 -- 172.21.15.1:0/869315395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e6815e4b0 msgr2=0x7f1e6815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 --2- 172.21.15.1:0/869315395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e6815e4b0 0x7f1e6815f320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e680722c0 tx=0x7f1e6002a680 comp rx=0 tx=0).stop 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 -- 172.21.15.1:0/869315395 shutdown_connections 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 --2- 172.21.15.1:0/869315395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e6815e4b0 0x7f1e6815f320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 --2- 172.21.15.1:0/869315395 >> v2:172.21.15.6:3300/0 conn(0x7f1e680c6860 0x7f1e68099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 -- 172.21.15.1:0/869315395 >> 172.21.15.1:0/869315395 conn(0x7f1e680b1fb0 msgr2=0x7f1e680b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 -- 172.21.15.1:0/869315395 shutdown_connections 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.125+0000 7f1e7011d640 1 -- 172.21.15.1:0/869315395 wait complete. 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.126+0000 7f1e7011d640 1 Processor -- start 2024-05-09T06:27:59.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.126+0000 7f1e7011d640 1 -- start start 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e680b2aa0 0x7f1e681495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e6815e4b0 0x7f1e68161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6815f860 con 0x7f1e680c6860 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6815eb20 con 0x7f1e680b2aa0 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e7011d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6815f9e0 con 0x7f1e6815e4b0 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58904/0 (socket says 172.21.15.1:58904) 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 -- 172.21.15.1:0/3375896610 learned_addr learned my addr 172.21.15.1:0/3375896610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6deb9640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e680b2aa0 0x7f1e681495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6e6ba640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e6815e4b0 0x7f1e68161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 -- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e6815e4b0 msgr2=0x7f1e68161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e6815e4b0 0x7f1e68161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 -- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e680b2aa0 msgr2=0x7f1e681495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e680b2aa0 0x7f1e681495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 -- 172.21.15.1:0/3375896610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680cab50 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.127+0000 7f1e6d6b8640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c780 tx=0x7f1e6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.128+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64018070 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.128+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e640072d0 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.128+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e680cae30 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.128+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640047e0 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.128+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e680cb3f0 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.129+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e680752b0 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.129+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6401b070 con 0x7f1e680c6860 2024-05-09T06:27:59.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.130+0000 7f1e5a7fc640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:27:59.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.130+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1e6408a5b0 con 0x7f1e680c6860 2024-05-09T06:27:59.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.130+0000 7f1e6deb9640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:27:59.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.130+0000 7f1e6deb9640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009ad0 tx=0x7f1e5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:27:59.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.135+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e68099930 con 0x7f1e680c6860 2024-05-09T06:27:59.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.357+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e6806ce30 con 0x7f1e680c6860 2024-05-09T06:27:59.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.358+0000 7f1e5a7fc640 1 -- 172.21.15.1:0/3375896610 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1e64058b90 con 0x7f1e680c6860 2024-05-09T06:27:59.340 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T06:27:59.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 msgr2=0x7f1e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009ad0 tx=0x7f1e5c008040 comp rx=0 tx=0).stop 2024-05-09T06:27:59.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 msgr2=0x7f1e68161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:27:59.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c780 tx=0x7f1e6400cc40 comp rx=0 tx=0).stop 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 shutdown_connections 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e6815e4b0 0x7f1e68161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e680c6860 0x7f1e68161130 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 --2- 172.21.15.1:0/3375896610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e680b2aa0 0x7f1e681495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 >> 172.21.15.1:0/3375896610 conn(0x7f1e680b1fb0 msgr2=0x7f1e680c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 shutdown_connections 2024-05-09T06:27:59.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:27:59.360+0000 7f1e7011d640 1 -- 172.21.15.1:0/3375896610 wait complete. 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:27:59.353 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:27:59.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:59.354 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:59.374 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42675sec preferred_lft 42675sec 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:27:59.375 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:27:59.376 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-05-09T06:27:59.376 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:27:59.376 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:27:59.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:59.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:27:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:59.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:59.572 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:27:59.598 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-05-09T06:27:59.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:59.600 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-05-09T06:27:59.671 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-05-09T06:27:59.672 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:27:59.672 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:27:59.672 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:27:59.672 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:27:59.672 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:27:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:27:59.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:27:59.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:27:59.988 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T06:28:00.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:00.005 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-05-09T06:28:00.060 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:00.060 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:00.061 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:00.061 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T06:28:00.061 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:00.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:00.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:00.181 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T06:28:00.181 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:00.181 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:00.181 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T06:28:00.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:28:00.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T06:28:00.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T06:28:00.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T06:28:00.182 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:00.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:00.303 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:00.309 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:00.309 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:00.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-05-09T06:28:00.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:28:00.309 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:00.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:00.390 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-09T06:28:00.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:00.391 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:00.406 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:28:00.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:00.407 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:00.462 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T06:28:00.529 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:00.586 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T06:28:00.587 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T06:28:00.588 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T06:28:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T06:28:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T06:28:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:00.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:00.589 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T06:28:00.642 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:28:00.642 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-09T06:28:00.685 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T06:28:00.711 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:00.768 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T06:28:00.768 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T06:28:00.769 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T06:28:00.770 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:00.771 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:00.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:00.771 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T06:28:00.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:2024-05-09T06:28:00.801+0000 7f3fda64a480 -1 init, newargv = 0x55a0578f8120 newargc=15 2024-05-09T06:28:00.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[41812]: starting ceph client 2024-05-09T06:28:00.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[41812]: starting fuse 2024-05-09T06:28:00.799 INFO:teuthology.orchestra.run.smithi006.stdout:42 2024-05-09T06:28:00.800 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> import re 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> import os 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2024-05-09T06:28:00.800 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> return f 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:00.801 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.1")) 2024-05-09T06:28:00.802 DEBUG:teuthology.orchestra.run.smithi006:> ' 2024-05-09T06:28:00.921 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.1.41812.asok 2024-05-09T06:28:00.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:00.931 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41812.asok 2024-05-09T06:28:00.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:00.931 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41812.asok status 2024-05-09T06:28:01.070 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T06:28:01.070 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2024-05-09T06:28:01.070 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:28:01.070 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:28:01.070 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "1", 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "41812", 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4622, 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4622 2024-05-09T06:28:01.071 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3792361118 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3792361118 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4622 192.168.0.1:0/3792361118", 2024-05-09T06:28:01.072 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/3792361118", 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 9, 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 24, 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2024-05-09T06:28:01.073 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T06:28:01.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:28:01.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.216+0000 7fd2cac12640 1 Processor -- start 2024-05-09T06:28:01.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.216+0000 7fd2cac12640 1 -- start start 2024-05-09T06:28:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2cac12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c4099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2cac12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2c415e4b0 0x7fd2c415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2cac12640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c4099930 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2cac12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c40b1e20 con 0x7fd2c415e4b0 2024-05-09T06:28:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2cac12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c415ee90 con 0x7fd2c40c6860 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c91af640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2c415e4b0 0x7fd2c415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c91af640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2c415e4b0 0x7fd2c415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58912/0 (socket says 172.21.15.1:58912) 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c91af640 1 -- 172.21.15.1:0/3066713631 learned_addr learned my addr 172.21.15.1:0/3066713631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c89ae640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c4099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c91af640 1 -- 172.21.15.1:0/3066713631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 msgr2=0x7fd2c4099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.217+0000 7fd2c91af640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c4099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.218+0000 7fd2c91af640 1 -- 172.21.15.1:0/3066713631 >> v1:172.21.15.6:6789/0 conn(0x7fd2c40b2aa0 legacy=0x7fd2c40c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.218+0000 7fd2c91af640 1 -- 172.21.15.1:0/3066713631 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c4162930 con 0x7fd2c415e4b0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.218+0000 7fd2c91af640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c415f320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3721d195be071336 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.218+0000 7fd2c37fe640 1 -- 172.21.15.1:0/3066713631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc036070 con 0x7fd2c415e4b0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.218+0000 7fd2c37fe640 1 -- 172.21.15.1:0/3066713631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc031600 con 0x7fd2c415e4b0 2024-05-09T06:28:01.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2c37fe640 1 -- 172.21.15.1:0/3066713631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0319d0 con 0x7fd2c415e4b0 2024-05-09T06:28:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 -- 172.21.15.1:0/3066713631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 msgr2=0x7fd2c415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c415f320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02a710 comp rx=0 tx=0).stop 2024-05-09T06:28:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 -- 172.21.15.1:0/3066713631 shutdown_connections 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c415f320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 --2- 172.21.15.1:0/3066713631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c4099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 -- 172.21.15.1:0/3066713631 >> 172.21.15.1:0/3066713631 conn(0x7fd2c40b1fb0 msgr2=0x7fd2c40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 -- 172.21.15.1:0/3066713631 shutdown_connections 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.219+0000 7fd2cac12640 1 -- 172.21.15.1:0/3066713631 wait complete. 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.220+0000 7fd2cac12640 1 Processor -- start 2024-05-09T06:28:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.220+0000 7fd2cac12640 1 -- start start 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c41566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c415f860 con 0x7fd2c415e4b0 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c415eb20 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2cac12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c415f9e0 con 0x7fd2c40c6860 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2c91af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2c89ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2c89ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:38014/0 (socket says 172.21.15.1:38014) 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.221+0000 7fd2c3fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c41566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 -- 172.21.15.1:0/1498993639 learned_addr learned my addr 172.21.15.1:0/1498993639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c91af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58914/0 (socket says 172.21.15.1:58914) 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 -- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 msgr2=0x7fd2c41566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c41566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 -- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 msgr2=0x7fd2c41ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 -- 172.21.15.1:0/1498993639 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c40cbbf0 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c91af640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c89ae640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e9b0 tx=0x7fd2b400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b400c9a0 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400cd90 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c40cbed0 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.222+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41e7c00 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.223+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40752b0 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.224+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2b401a070 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.224+0000 7fd2c17fa640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 0x7fd2b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.224+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fd2b408b5c0 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.224+0000 7fd2c3fff640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 0x7fd2b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.225+0000 7fd2c3fff640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 0x7fd2b006e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40b16a0 tx=0x7fd2b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:01.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.229+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2b4059490 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.452+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd2c406ce30 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.452+0000 7fd2c17fa640 1 -- 172.21.15.1:0/1498993639 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd2b405d790 con 0x7fd2c40b2aa0 2024-05-09T06:28:01.434 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 msgr2=0x7fd2b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 0x7fd2b006e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40b16a0 tx=0x7fd2b8009040 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 msgr2=0x7fd2c40cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e9b0 tx=0x7fd2b400ee70 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 shutdown_connections 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2b006be60 0x7fd2b006e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c415e4b0 0x7fd2c41ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c40c6860 0x7fd2c41566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 --2- 172.21.15.1:0/1498993639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40b2aa0 0x7fd2c40cb3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 >> 172.21.15.1:0/1498993639 conn(0x7fd2c40b1fb0 msgr2=0x7fd2c415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 shutdown_connections 2024-05-09T06:28:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:01.455+0000 7fd2cac12640 1 -- 172.21.15.1:0/1498993639 wait complete. 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:28:01.448 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-09T06:28:01.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.448 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:01.466 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42673sec preferred_lft 42673sec 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 9e:10:bf:b0:ea:8f brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.467 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::20bc:99ff:feea:fda5/64 scope link 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run.smithi006.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 9e:10:bf:b0:ea:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::9c10:bfff:feb0:ea8f/64 scope link tentative 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.468 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:28:01.531 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-05-09T06:28:01.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.533 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-05-09T06:28:01.589 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-05-09T06:28:01.589 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:28:01.589 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:28:01.589 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:28:01.589 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:28:01.589 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:01.590 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:01.590 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:28:01.590 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:28:01.590 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:28:01.590 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:01.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:01.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:01.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.720 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T06:28:01.736 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T06:28:01.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.737 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-05-09T06:28:01.792 INFO:teuthology.orchestra.run.smithi006.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-09T06:28:01.793 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:01.793 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:01.793 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:28:01.793 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-09T06:28:01.793 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:01.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:01.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:01.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.912 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T06:28:01.937 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:30:35:c2:6d:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b830:35ff:fec2:6dda/64 scope link 2024-05-09T06:28:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:01.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:01.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:30:35:c2:6d:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:28:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b830:35ff:fec2:6dda/64 scope link 2024-05-09T06:28:02.006 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:02.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-09T06:28:02.007 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:02.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:02.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:02.168 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:02.168 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T06:28:02.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.1 up 2024-05-09T06:28:02.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T06:28:02.168 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T06:28:02.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:02.247 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-09T06:28:02.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:02.248 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:02.263 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-09T06:28:02.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:02.263 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:02.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T06:28:02.386 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:02.443 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T06:28:02.443 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T06:28:02.443 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T06:28:02.443 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:28:02.445 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:28:02.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:02.446 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T06:28:02.499 INFO:teuthology.orchestra.run.smithi006.stdout:42 2024-05-09T06:28:02.499 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-05-09T06:28:02.499 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-09T06:28:02.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T06:28:02.568 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:02.625 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T06:28:02.626 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T06:28:02.627 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:28:02.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:02.628 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T06:28:02.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:2024-05-09T06:28:02.646+0000 7f9f743b4480 -1 init, newargv = 0x5601bf954fe0 newargc=15 2024-05-09T06:28:02.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:ceph-fuse[42258]: starting ceph client 2024-05-09T06:28:02.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:ceph-fuse[42258]: starting fuse 2024-05-09T06:28:02.645 INFO:teuthology.orchestra.run.smithi006.stdout:42 2024-05-09T06:28:02.645 INFO:teuthology.orchestra.run.smithi006.stdout:43 2024-05-09T06:28:02.645 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> import re 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> import os 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2024-05-09T06:28:02.645 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:28:02.646 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> return f 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.3")) 2024-05-09T06:28:02.647 DEBUG:teuthology.orchestra.run.smithi006:> ' 2024-05-09T06:28:02.766 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.3.42258.asok 2024-05-09T06:28:02.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T06:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:02.777 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42258.asok 2024-05-09T06:28:02.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:02.777 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42258.asok status 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "3", 2024-05-09T06:28:02.915 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "42258", 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4747, 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4747 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:02.916 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1287284883 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1287284883 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4747 192.168.0.2:0/1287284883", 2024-05-09T06:28:02.917 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.2:0/1287284883", 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 9, 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 26, 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2024-05-09T06:28:02.918 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T06:28:02.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:02.923 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:28:02.923 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:02.944 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:28:02.944 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:02.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:02.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:03.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.016 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:28:03.016 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-09T06:28:03.076 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:28:03.076 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-09T06:28:03.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-09T06:28:03.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.146 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2024-05-09T06:28:03.146 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:03.163 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2024-05-09T06:28:03.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:03.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.163 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:03.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.231 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2024-05-09T06:28:03.231 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:03.288 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2024-05-09T06:28:03.288 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:03.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:03.288 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:03.354 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-09T06:28:03.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:03.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.805+0000 7ff563770640 1 Processor -- start 2024-05-09T06:28:03.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.805+0000 7ff563770640 1 -- start start 2024-05-09T06:28:03.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff563770640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff55c06ffa0 0x7ff55c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff563770640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff55c06c2f0 con 0x7ff55c15e1d0 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff563770640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff55c06f500 con 0x7ff55c15e5b0 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff563770640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff55c0cc3c0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff560d0b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff55c06ffa0 0x7ff55c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff56150c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff55c15e1d0 0x7ff55c06f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45692/0 (socket says 172.21.15.1:45692) 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.806+0000 7ff56150c640 1 -- 172.21.15.1:0/1642754505 learned_addr learned my addr 172.21.15.1:0/1642754505 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff560d0b640 1 -- 172.21.15.1:0/1642754505 >> v1:172.21.15.6:6789/0 conn(0x7ff55c15e5b0 legacy=0x7ff55c0d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff54ffff640 1 -- 172.21.15.1:0/1642754505 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713054853 0 0) 0x7ff558001110 con 0x7ff55c15e1d0 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff560d0b640 1 -- 172.21.15.1:0/1642754505 >> v1:172.21.15.1:6789/0 conn(0x7ff55c15e1d0 legacy=0x7ff55c06f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff560d0b640 1 -- 172.21.15.1:0/1642754505 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c0cc610 con 0x7ff55c06ffa0 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff560d0b640 1 --2- 172.21.15.1:0/1642754505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c0cffb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff5500097a0 tx=0x7ff55002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689335ea0f260727 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.807+0000 7ff54ffff640 1 -- 172.21.15.1:0/1642754505 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550037070 con 0x7ff55c06ffa0 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff54ffff640 1 -- 172.21.15.1:0/1642754505 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff550032440 con 0x7ff55c06ffa0 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff54ffff640 1 -- 172.21.15.1:0/1642754505 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550032810 con 0x7ff55c06ffa0 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 -- 172.21.15.1:0/1642754505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 msgr2=0x7ff55c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 --2- 172.21.15.1:0/1642754505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c0cffb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff5500097a0 tx=0x7ff55002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 -- 172.21.15.1:0/1642754505 shutdown_connections 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 --2- 172.21.15.1:0/1642754505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c0cffb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 -- 172.21.15.1:0/1642754505 >> 172.21.15.1:0/1642754505 conn(0x7ff55c0d1d70 msgr2=0x7ff55c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 -- 172.21.15.1:0/1642754505 shutdown_connections 2024-05-09T06:28:03.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.808+0000 7ff563770640 1 -- 172.21.15.1:0/1642754505 wait complete. 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.809+0000 7ff563770640 1 Processor -- start 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.809+0000 7ff563770640 1 -- start start 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 0x7ff55c0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 0x7ff55c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c06f1b0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c0ccc60 con 0x7ff55c15e1d0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff563770640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c0ce100 con 0x7ff55c15e5b0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff56150c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff56150c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58934/0 (socket says 172.21.15.1:58934) 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff56150c640 1 -- 172.21.15.1:0/1034687915 learned_addr learned my addr 172.21.15.1:0/1034687915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff560d0b640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 0x7ff55c0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.810+0000 7ff561d0d640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 0x7ff55c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 -- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 msgr2=0x7ff55c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 0x7ff55c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 -- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 msgr2=0x7ff55c0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 0x7ff55c0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 -- 172.21.15.1:0/1034687915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c161980 con 0x7ff55c06ffa0 2024-05-09T06:28:03.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff561d0d640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 0x7ff55c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff560d0b640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 0x7ff55c0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff56150c640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c780 tx=0x7ff55800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558018070 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5580072d0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c0c94d0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5580047e0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.811+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c0c9d70 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.812+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c0752b0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.813+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff55801b070 con 0x7ff55c06ffa0 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.813+0000 7ff54dffb640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 0x7ff54006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.813+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff55808a570 con 0x7ff55c06ffa0 2024-05-09T06:28:03.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.813+0000 7ff560d0b640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 0x7ff54006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.814+0000 7ff560d0b640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 0x7ff54006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff550034000 tx=0x7ff55002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:03.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:03.818+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff55c06c2f0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.003+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff55c0b0cc0 con 0x7ff55c06ffa0 2024-05-09T06:28:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.004+0000 7ff54dffb640 1 -- 172.21.15.1:0/1034687915 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7ff558058b20 con 0x7ff55c06ffa0 2024-05-09T06:28:03.986 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:03.986 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 msgr2=0x7ff54006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 0x7ff54006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff550034000 tx=0x7ff55002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 msgr2=0x7ff55c1f3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c780 tx=0x7ff55800cc40 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 shutdown_connections 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff54006be30 0x7ff54006e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff55c15e5b0 0x7ff55c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff55c15e1d0 0x7ff55c0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 --2- 172.21.15.1:0/1034687915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55c06ffa0 0x7ff55c1f3b20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 >> 172.21.15.1:0/1034687915 conn(0x7ff55c0d1d70 msgr2=0x7ff55c0aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 shutdown_connections 2024-05-09T06:28:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.006+0000 7ff563770640 1 -- 172.21.15.1:0/1034687915 wait complete. 2024-05-09T06:28:03.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:04.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:04.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.124+0000 7fb99f909640 1 Processor -- start 2024-05-09T06:28:04.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.124+0000 7fb99f909640 1 -- start start 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99f909640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9981585d0 0x7fb9980b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99f909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb998146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99f909640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb998071aa0 con 0x7fb9980715d0 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99f909640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb998071d80 con 0x7fb9981585d0 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99f909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb998072060 con 0x7fb998162260 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99cea4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb9980715d0 0x7fb998161b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45706/0 (socket says 172.21.15.1:45706) 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99cea4640 1 -- 172.21.15.1:0/696621989 learned_addr learned my addr 172.21.15.1:0/696621989 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99dea6640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb998146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.125+0000 7fb99d6a5640 1 --2- 172.21.15.1:0/696621989 >> v2:172.21.15.6:3300/0 conn(0x7fb9981585d0 0x7fb9980b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738631618 0 0) 0x7fb998071aa0 con 0x7fb9980715d0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb980003520 con 0x7fb9980715d0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3728334028 0 0) 0x7fb980003520 con 0x7fb9980715d0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 msgr2=0x7fb998146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb998146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 >> v2:172.21.15.6:3300/0 conn(0x7fb9981585d0 msgr2=0x7fb9980b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 --2- 172.21.15.1:0/696621989 >> v2:172.21.15.6:3300/0 conn(0x7fb9981585d0 0x7fb9980b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb998072340 con 0x7fb9980715d0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb98c001670 con 0x7fb9980715d0 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 >> v1:172.21.15.1:6789/0 conn(0x7fb9980715d0 legacy=0x7fb998161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9800039b0 0x7fb980003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb980004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb9800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb998071d80 con 0x7fb980004d40 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb998072060 con 0x7fb9800039b0 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb98000b9c0 con 0x7fb998162260 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb9800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99cea4640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb980004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99d6a5640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9800039b0 0x7fb980003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 -- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 msgr2=0x7fb980004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb998162260 0x7fb980004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 -- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9800039b0 msgr2=0x7fb980003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9800039b0 0x7fb980003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.126+0000 7fb99dea6640 1 -- 172.21.15.1:0/696621989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb980003520 con 0x7fb980004d40 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.127+0000 7fb99dea6640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb9800051c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e170 tx=0x7fb980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7e3bc25f15a3e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.127+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988034020 con 0x7fb980004d40 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.127+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9880305f0 con 0x7fb980004d40 2024-05-09T06:28:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.127+0000 7fb997fff640 1 -- 172.21.15.1:0/696621989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988009c10 con 0x7fb980004d40 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 -- 172.21.15.1:0/696621989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 msgr2=0x7fb9800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 --2- 172.21.15.1:0/696621989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb9800051c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e170 tx=0x7fb980005830 comp rx=0 tx=0).stop 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99dea6640 1 -- 172.21.15.1:0/696621989 reap_dead start 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 -- 172.21.15.1:0/696621989 shutdown_connections 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 -- 172.21.15.1:0/696621989 >> 172.21.15.1:0/696621989 conn(0x7fb9980b1ba0 msgr2=0x7fb9980b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 -- 172.21.15.1:0/696621989 shutdown_connections 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.128+0000 7fb99f909640 1 -- 172.21.15.1:0/696621989 wait complete. 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.129+0000 7fb99f909640 1 Processor -- start 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.129+0000 7fb99f909640 1 -- start start 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 0x7fb998155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 0x7fb99815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9980b4190 con 0x7fb980004d40 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981589b0 con 0x7fb9980cebd0 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99f909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb9980b4310 con 0x7fb9980cc960 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58962/0 (socket says 172.21.15.1:58962) 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 -- 172.21.15.1:0/3595159175 learned_addr learned my addr 172.21.15.1:0/3595159175 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99cea4640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 0x7fb998155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99dea6640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 0x7fb99815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 -- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 msgr2=0x7fb998155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 0x7fb998155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 -- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 msgr2=0x7fb99815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 0x7fb99815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99d6a5640 1 -- 172.21.15.1:0/3595159175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9980cf570 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99cea4640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 0x7fb998155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.130+0000 7fb99dea6640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 0x7fb99815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb99d6a5640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb990011a70 tx=0x7fb990011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99000fb50 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99000fcf0 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb998158dd0 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99000fea0 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.131+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb998159390 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.132+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb998074610 con 0x7fb980004d40 2024-05-09T06:28:04.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.132+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb990012040 con 0x7fb980004d40 2024-05-09T06:28:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.133+0000 7fb995ffb640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 0x7fb98406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.133+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb99008b9e0 con 0x7fb980004d40 2024-05-09T06:28:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.133+0000 7fb99cea4640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 0x7fb98406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.134+0000 7fb99cea4640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 0x7fb98406e320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb99814f080 tx=0x7fb98c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.137+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb990019030 con 0x7fb980004d40 2024-05-09T06:28:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.318+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb998158bc0 con 0x7fb980004d40 2024-05-09T06:28:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.319+0000 7fb995ffb640 1 -- 172.21.15.1:0/3595159175 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb990059940 con 0x7fb980004d40 2024-05-09T06:28:04.301 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 msgr2=0x7fb98406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 0x7fb98406e320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb99814f080 tx=0x7fb98c009040 comp rx=0 tx=0).stop 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 msgr2=0x7fb99814e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb990011a70 tx=0x7fb990011f30 comp rx=0 tx=0).stop 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 shutdown_connections 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98406be60 0x7fb98406e320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9980cebd0 0x7fb99815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9980cc960 0x7fb998155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 --2- 172.21.15.1:0/3595159175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb980004d40 0x7fb99814e7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 >> 172.21.15.1:0/3595159175 conn(0x7fb9980b1ba0 msgr2=0x7fb9980ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 shutdown_connections 2024-05-09T06:28:04.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.321+0000 7fb99f909640 1 -- 172.21.15.1:0/3595159175 wait complete. 2024-05-09T06:28:04.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:04.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.438+0000 7fbf69563640 1 Processor -- start 2024-05-09T06:28:04.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.438+0000 7fbf69563640 1 -- start start 2024-05-09T06:28:04.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf69563640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbf64158a50 0x7fbf640718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf69563640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf64071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf69563640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6415b420 con 0x7fbf6406e6d0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf69563640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbf64158e30 con 0x7fbf64158a50 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf69563640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbf64072480 con 0x7fbf6415c4a0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf637fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbf6406e6d0 0x7fbf640cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45720/0 (socket says 172.21.15.1:45720) 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf637fe640 1 -- 172.21.15.1:0/1113119308 learned_addr learned my addr 172.21.15.1:0/1113119308 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/1113119308 >> v2:172.21.15.6:3300/0 conn(0x7fbf64158a50 0x7fbf640718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078375487 0 0) 0x7fbf6415b420 con 0x7fbf6406e6d0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf627fc640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf64071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.439+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf50003520 con 0x7fbf6406e6d0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870560547 0 0) 0x7fbf50003520 con 0x7fbf6406e6d0 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 msgr2=0x7fbf64071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf64071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 >> v2:172.21.15.6:3300/0 conn(0x7fbf64158a50 msgr2=0x7fbf640718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 --2- 172.21.15.1:0/1113119308 >> v2:172.21.15.6:3300/0 conn(0x7fbf64158a50 0x7fbf640718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6415b6e0 con 0x7fbf6406e6d0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fbf5c001670 con 0x7fbf6406e6d0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 >> v1:172.21.15.1:6789/0 conn(0x7fbf6406e6d0 legacy=0x7fbf640cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf500039b0 0x7fbf50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf50004800 0x7fbf50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64158e30 con 0x7fbf500039b0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64072480 con 0x7fbf50004800 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5000dbd0 con 0x7fbf6415c4a0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf500039b0 0x7fbf50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf637fe640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf627fc640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf50004800 0x7fbf50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 -- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 msgr2=0x7fbf500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf6415c4a0 0x7fbf500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 -- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf50004800 msgr2=0x7fbf50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf50004800 0x7fbf50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.440+0000 7fbf62ffd640 1 -- 172.21.15.1:0/1113119308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf50003520 con 0x7fbf500039b0 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf500039b0 0x7fbf50003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fc0 tx=0x7fbf50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4ba44c0ef434d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54009c10 con 0x7fbf500039b0 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf54036070 con 0x7fbf500039b0 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf61ffb640 1 -- 172.21.15.1:0/1113119308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5402d070 con 0x7fbf500039b0 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf69563640 1 -- 172.21.15.1:0/1113119308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf500039b0 msgr2=0x7fbf50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf69563640 1 --2- 172.21.15.1:0/1113119308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf500039b0 0x7fbf50003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fc0 tx=0x7fbf50005830 comp rx=0 tx=0).stop 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.441+0000 7fbf637fe640 1 -- 172.21.15.1:0/1113119308 reap_dead start 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.442+0000 7fbf69563640 1 -- 172.21.15.1:0/1113119308 shutdown_connections 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.442+0000 7fbf69563640 1 -- 172.21.15.1:0/1113119308 >> 172.21.15.1:0/1113119308 conn(0x7fbf640cc4d0 msgr2=0x7fbf640cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.442+0000 7fbf69563640 1 -- 172.21.15.1:0/1113119308 shutdown_connections 2024-05-09T06:28:04.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.442+0000 7fbf69563640 1 -- 172.21.15.1:0/1113119308 wait complete. 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.443+0000 7fbf69563640 1 Processor -- start 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.443+0000 7fbf69563640 1 -- start start 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf500039b0 0x7fbf6406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf64149040 0x7fbf64148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415bf80 con 0x7fbf640b4970 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415bca0 con 0x7fbf500039b0 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf69563640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbf64070ea0 con 0x7fbf64149040 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58976/0 (socket says 172.21.15.1:58976) 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 -- 172.21.15.1:0/109938040 learned_addr learned my addr 172.21.15.1:0/109938040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf500039b0 0x7fbf6406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf627fc640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf64149040 0x7fbf64148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 -- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf64149040 msgr2=0x7fbf64148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf64149040 0x7fbf64148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 -- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf500039b0 msgr2=0x7fbf6406d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf500039b0 0x7fbf6406d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 -- 172.21.15.1:0/109938040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6406dba0 con 0x7fbf640b4970 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf637fe640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbf640739f0 tx=0x7fbf5c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00c5b0 con 0x7fbf640b4970 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.444+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c018070 con 0x7fbf640b4970 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.445+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00c990 con 0x7fbf640b4970 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.445+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6406de80 con 0x7fbf640b4970 2024-05-09T06:28:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.445+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641566e0 con 0x7fbf640b4970 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.446+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf640752b0 con 0x7fbf640b4970 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.446+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf5c00cb30 con 0x7fbf640b4970 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.447+0000 7fbf5b7fe640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.447+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.447+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbf5c08add0 con 0x7fbf640b4970 2024-05-09T06:28:04.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.447+0000 7fbf62ffd640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 0x7fbf4406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf54035000 tx=0x7fbf5402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.451+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf5c059370 con 0x7fbf640b4970 2024-05-09T06:28:04.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.632+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf640cf4b0 con 0x7fbf640b4970 2024-05-09T06:28:04.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.633+0000 7fbf5b7fe640 1 -- 172.21.15.1:0/109938040 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbf5c05f030 con 0x7fbf640b4970 2024-05-09T06:28:04.615 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 msgr2=0x7fbf4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 0x7fbf4406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf54035000 tx=0x7fbf5402f000 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 msgr2=0x7fbf640b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbf640739f0 tx=0x7fbf5c00eb90 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 shutdown_connections 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf640b4970 0x7fbf640b4df0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf64149040 0x7fbf64148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 --2- 172.21.15.1:0/109938040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf500039b0 0x7fbf6406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 >> 172.21.15.1:0/109938040 conn(0x7fbf640cc4d0 msgr2=0x7fbf64159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 shutdown_connections 2024-05-09T06:28:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.635+0000 7fbf69563640 1 -- 172.21.15.1:0/109938040 wait complete. 2024-05-09T06:28:04.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:04.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.759+0000 7f396df16640 1 Processor -- start 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.759+0000 7f396df16640 1 -- start start 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f39680d06f0 0x7f39680cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680c41e0 0x7f39680ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f396806b1e0 0x7f3968161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f39680cd160 con 0x7f39680d06f0 2024-05-09T06:28:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3968072050 con 0x7f396806b1e0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f396df16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3968072330 con 0x7f39680c41e0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f39677fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f39680d06f0 0x7f39680cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f39677fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f39680d06f0 0x7f39680cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58992/0 (socket says 172.21.15.1:58992) 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f39677fe640 1 -- 172.21.15.1:0/1188227822 learned_addr learned my addr 172.21.15.1:0/1188227822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f3967fff640 1 --2- 172.21.15.1:0/1188227822 >> v2:172.21.15.6:3300/0 conn(0x7f396806b1e0 0x7f3968161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.760+0000 7f3966ffd640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680c41e0 0x7f39680ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 -- 172.21.15.1:0/1188227822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680c41e0 msgr2=0x7f39680ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680c41e0 0x7f39680ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 -- 172.21.15.1:0/1188227822 >> v2:172.21.15.6:3300/0 conn(0x7f396806b1e0 msgr2=0x7f3968161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 --2- 172.21.15.1:0/1188227822 >> v2:172.21.15.6:3300/0 conn(0x7f396806b1e0 0x7f3968161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 -- 172.21.15.1:0/1188227822 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3968072610 con 0x7f39680d06f0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39677fe640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680d06f0 0x7f39680cc6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f39680b2690 tx=0x7f395802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6e4fefa8ce7176 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39667fc640 1 -- 172.21.15.1:0/1188227822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958036070 con 0x7f39680d06f0 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39667fc640 1 -- 172.21.15.1:0/1188227822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3958031600 con 0x7f39680d06f0 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.761+0000 7f39667fc640 1 -- 172.21.15.1:0/1188227822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580319d0 con 0x7f39680d06f0 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 -- 172.21.15.1:0/1188227822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680d06f0 msgr2=0x7f39680cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680d06f0 0x7f39680cc6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f39680b2690 tx=0x7f395802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 -- 172.21.15.1:0/1188227822 shutdown_connections 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 --2- 172.21.15.1:0/1188227822 >> v2:172.21.15.6:3300/0 conn(0x7f396806b1e0 0x7f3968161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680c41e0 0x7f39680ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 --2- 172.21.15.1:0/1188227822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680d06f0 0x7f39680cc6e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 -- 172.21.15.1:0/1188227822 >> 172.21.15.1:0/1188227822 conn(0x7f39680d0b50 msgr2=0x7f39680d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 -- 172.21.15.1:0/1188227822 shutdown_connections 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.762+0000 7f396df16640 1 -- 172.21.15.1:0/1188227822 wait complete. 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.763+0000 7f396df16640 1 Processor -- start 2024-05-09T06:28:04.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.763+0000 7f396df16640 1 -- start start 2024-05-09T06:28:04.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f396806b1e0 0x7f39680c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680d06f0 0x7f39680c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39681620c0 con 0x7f39680c41e0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3968071ce0 con 0x7f396806b1e0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f396df16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3968162240 con 0x7f39680d06f0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f3966ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f3966ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59008/0 (socket says 172.21.15.1:59008) 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f3966ffd640 1 -- 172.21.15.1:0/1513843005 learned_addr learned my addr 172.21.15.1:0/1513843005 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f39677fe640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f396806b1e0 0x7f39680c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.764+0000 7f3967fff640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680d06f0 0x7f39680c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 -- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680d06f0 msgr2=0x7f39680c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680d06f0 0x7f39680c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 -- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f396806b1e0 msgr2=0x7f39680c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f396806b1e0 0x7f39680c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 -- 172.21.15.1:0/1513843005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680c64e0 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f3966ffd640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d7d0 tx=0x7f395c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00b6c0 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c019070 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39680c96b0 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00bac0 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.765+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39680c6750 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.766+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39680752b0 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.766+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f395c00bc60 con 0x7f39680c41e0 2024-05-09T06:28:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.767+0000 7f394ffff640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 0x7f395406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.767+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f395c08b5b0 con 0x7f39680c41e0 2024-05-09T06:28:04.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.767+0000 7f39677fe640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 0x7f395406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.767+0000 7f39677fe640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 0x7f395406e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3958033000 tx=0x7f3958009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.771+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f39680c96b0 con 0x7f39680c41e0 2024-05-09T06:28:04.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.953+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3968072f30 con 0x7f39680c41e0 2024-05-09T06:28:04.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.953+0000 7f394ffff640 1 -- 172.21.15.1:0/1513843005 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f395c059bf0 con 0x7f39680c41e0 2024-05-09T06:28:04.936 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 msgr2=0x7f395406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 0x7f395406e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3958033000 tx=0x7f3958009000 comp rx=0 tx=0).stop 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 msgr2=0x7f39680c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d7d0 tx=0x7f395c00dc90 comp rx=0 tx=0).stop 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 shutdown_connections 2024-05-09T06:28:04.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f395406be30 0x7f395406e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f39680d06f0 0x7f39680c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39680c41e0 0x7f39680c9170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 --2- 172.21.15.1:0/1513843005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f396806b1e0 0x7f39680c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.955+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 >> 172.21.15.1:0/1513843005 conn(0x7f39680d0b50 msgr2=0x7f396806e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.956+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 shutdown_connections 2024-05-09T06:28:04.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:04.956+0000 7f396df16640 1 -- 172.21.15.1:0/1513843005 wait complete. 2024-05-09T06:28:04.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:05.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.076+0000 7fb81e714640 1 Processor -- start 2024-05-09T06:28:05.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.076+0000 7fb81e714640 1 -- start start 2024-05-09T06:28:05.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.077+0000 7fb81e714640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb8180d1c70 0x7fb81815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.077+0000 7fb81e714640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb81e714640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb818161fe0 con 0x7fb81815e190 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb81e714640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb81815f580 con 0x7fb8180d1c70 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb81e714640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb8181622c0 con 0x7fb8180d45f0 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb8180d1c70 0x7fb81815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb8180d1c70 0x7fb81815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59014/0 (socket says 172.21.15.1:59014) 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 -- 172.21.15.1:0/2403689607 learned_addr learned my addr 172.21.15.1:0/2403689607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 -- 172.21.15.1:0/2403689607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 msgr2=0x7fb8180b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 --2- 172.21.15.1:0/2403689607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 -- 172.21.15.1:0/2403689607 >> v1:172.21.15.6:6789/0 conn(0x7fb81815e190 legacy=0x7fb8180d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.078+0000 7fb8177fe640 1 -- 172.21.15.1:0/2403689607 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181625a0 con 0x7fb8180d1c70 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb8177fe640 1 --2- 172.21.15.1:0/2403689607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8180d1c70 0x7fb81815f780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009870 tx=0x7fb80c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cd48dc1ba1512f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb816ffd640 1 -- 172.21.15.1:0/2403689607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c037070 con 0x7fb8180d1c70 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb816ffd640 1 -- 172.21.15.1:0/2403689607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c032440 con 0x7fb8180d1c70 2024-05-09T06:28:05.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb816ffd640 1 -- 172.21.15.1:0/2403689607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c032810 con 0x7fb8180d1c70 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 -- 172.21.15.1:0/2403689607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8180d1c70 msgr2=0x7fb81815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 --2- 172.21.15.1:0/2403689607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8180d1c70 0x7fb81815f780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009870 tx=0x7fb80c02a210 comp rx=0 tx=0).stop 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 -- 172.21.15.1:0/2403689607 shutdown_connections 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 --2- 172.21.15.1:0/2403689607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 --2- 172.21.15.1:0/2403689607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8180d1c70 0x7fb81815f780 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 -- 172.21.15.1:0/2403689607 >> 172.21.15.1:0/2403689607 conn(0x7fb818073340 msgr2=0x7fb81809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.079+0000 7fb81e714640 1 -- 172.21.15.1:0/2403689607 shutdown_connections 2024-05-09T06:28:05.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.080+0000 7fb81e714640 1 -- 172.21.15.1:0/2403689607 wait complete. 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.081+0000 7fb81e714640 1 Processor -- start 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.081+0000 7fb81e714640 1 -- start start 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8180d1c70 0x7fb8180cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180b56d0 con 0x7fb81815e190 2024-05-09T06:28:05.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb818161c60 con 0x7fb8180d1c70 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180b5850 con 0x7fb8180d45f0 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59022/0 (socket says 172.21.15.1:59022) 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 -- 172.21.15.1:0/3575239312 learned_addr learned my addr 172.21.15.1:0/3575239312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb817fff640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8180d1c70 0x7fb8180cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb8177fe640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 -- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 msgr2=0x7fb8180cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 -- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8180d1c70 msgr2=0x7fb8180cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8180d1c70 0x7fb8180cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 -- 172.21.15.1:0/3575239312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180cb6b0 con 0x7fb81815e190 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81ccb1640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b9e0 tx=0x7fb81000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810019070 con 0x7fb81815e190 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81000cb40 con 0x7fb81815e190 2024-05-09T06:28:05.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810014440 con 0x7fb81815e190 2024-05-09T06:28:05.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.082+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8180cc1f0 con 0x7fb81815e190 2024-05-09T06:28:05.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.083+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8180cbc30 con 0x7fb81815e190 2024-05-09T06:28:05.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.084+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8180752b0 con 0x7fb81815e190 2024-05-09T06:28:05.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.084+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb81001c070 con 0x7fb81815e190 2024-05-09T06:28:05.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.085+0000 7fb814ff9640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.085+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb81008ae50 con 0x7fb81815e190 2024-05-09T06:28:05.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.085+0000 7fb817fff640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.085+0000 7fb817fff640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 0x7fb80406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb81815e720 tx=0x7fb808009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.089+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb810058e40 con 0x7fb81815e190 2024-05-09T06:28:05.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.271+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb81808e1c0 con 0x7fb81815e190 2024-05-09T06:28:05.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.272+0000 7fb814ff9640 1 -- 172.21.15.1:0/3575239312 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb81005d140 con 0x7fb81815e190 2024-05-09T06:28:05.254 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 msgr2=0x7fb80406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 0x7fb80406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb81815e720 tx=0x7fb808009060 comp rx=0 tx=0).stop 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 msgr2=0x7fb8180b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b9e0 tx=0x7fb81000bea0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 shutdown_connections 2024-05-09T06:28:05.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb80406be30 0x7fb80406e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb81815e190 0x7fb8180b2e40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180d45f0 0x7fb8180cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 --2- 172.21.15.1:0/3575239312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8180d1c70 0x7fb8180cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 >> 172.21.15.1:0/3575239312 conn(0x7fb818073340 msgr2=0x7fb81815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 shutdown_connections 2024-05-09T06:28:05.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.274+0000 7fb81e714640 1 -- 172.21.15.1:0/3575239312 wait complete. 2024-05-09T06:28:05.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.389+0000 7f083329a640 1 Processor -- start 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 -- start start 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 0x7f082c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f082c0d2b10 0x7f082c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c0b33c0 con 0x7f082c161950 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f082c0d4dd0 con 0x7f082c0d2b10 2024-05-09T06:28:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.390+0000 7f083329a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f082c0b3660 con 0x7f082c0d4940 2024-05-09T06:28:05.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f082c0d2b10 0x7f082c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0830835640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 0x7f082c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f082c0d2b10 0x7f082c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:38100/0 (socket says 172.21.15.1:38100) 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831036640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f082c161950 0x7f082c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:38366/0 (socket says 172.21.15.1:38366) 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 -- 172.21.15.1:0/1118284310 learned_addr learned my addr 172.21.15.1:0/1118284310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 -- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 msgr2=0x7f082c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 0x7f082c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 -- 172.21.15.1:0/1118284310 >> v1:172.21.15.6:6789/0 conn(0x7f082c161950 legacy=0x7f082c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0831837640 1 -- 172.21.15.1:0/1118284310 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c0b3940 con 0x7f082c0d2b10 2024-05-09T06:28:05.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.391+0000 7f0830835640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 0x7f082c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:05.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.392+0000 7f0831837640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b4680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f081c009e50 tx=0x7f081c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f35dfb1170be612 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.392+0000 7f082bfff640 1 -- 172.21.15.1:0/1118284310 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c036070 con 0x7f082c0d2b10 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.392+0000 7f082bfff640 1 -- 172.21.15.1:0/1118284310 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c031600 con 0x7f082c0d2b10 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.392+0000 7f082bfff640 1 -- 172.21.15.1:0/1118284310 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0319d0 con 0x7f082c0d2b10 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 -- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 msgr2=0x7f082c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b4680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f081c009e50 tx=0x7f081c02a710 comp rx=0 tx=0).stop 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 -- 172.21.15.1:0/1118284310 shutdown_connections 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b4680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 --2- 172.21.15.1:0/1118284310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c0d4940 0x7f082c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 -- 172.21.15.1:0/1118284310 >> 172.21.15.1:0/1118284310 conn(0x7f082c0d3e20 msgr2=0x7f082c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 -- 172.21.15.1:0/1118284310 shutdown_connections 2024-05-09T06:28:05.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.393+0000 7f083329a640 1 -- 172.21.15.1:0/1118284310 wait complete. 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.394+0000 7f083329a640 1 Processor -- start 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.394+0000 7f083329a640 1 -- start start 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c161950 0x7f082c151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f082c0d1100 con 0x7f082c0cbde0 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f082c0d0f80 con 0x7f082c0d2b10 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f083329a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f082c0d1c00 con 0x7f082c161950 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f0831837640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f0831837640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59026/0 (socket says 172.21.15.1:59026) 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f0831837640 1 -- 172.21.15.1:0/1685186054 learned_addr learned my addr 172.21.15.1:0/1685186054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f0831036640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.395+0000 7f0830835640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c161950 0x7f082c151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 -- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c161950 msgr2=0x7f082c151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c161950 0x7f082c151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 -- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 msgr2=0x7f082c0b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 -- 172.21.15.1:0/1685186054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c0cb020 con 0x7f082c0cbde0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0831837640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f081c02ad00 tx=0x7f081c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c036030 con 0x7f082c0cbde0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c040070 con 0x7f082c0cbde0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c009510 con 0x7f082c0cbde0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c0cb300 con 0x7f082c0cbde0 2024-05-09T06:28:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.396+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c0cb8c0 con 0x7f082c0cbde0 2024-05-09T06:28:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.397+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c0cbac0 con 0x7f082c0cbde0 2024-05-09T06:28:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.398+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f081c007c70 con 0x7f082c0cbde0 2024-05-09T06:28:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.398+0000 7f0829ffb640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.398+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f081c0ab850 con 0x7f082c0cbde0 2024-05-09T06:28:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.398+0000 7f0831036640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.399+0000 7f0831036640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 0x7f081806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0824000f60 tx=0x7f0824007e20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.403+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f082c0cbac0 con 0x7f082c0cbde0 2024-05-09T06:28:05.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.585+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f082c0752b0 con 0x7f082c0cbde0 2024-05-09T06:28:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.586+0000 7f0829ffb640 1 -- 172.21.15.1:0/1685186054 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f081c0797c0 con 0x7f082c0cbde0 2024-05-09T06:28:05.568 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:05.569 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 msgr2=0x7f081806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 0x7f081806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0824000f60 tx=0x7f0824007e20 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 msgr2=0x7f082c0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f081c02ad00 tx=0x7f081c004a40 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 shutdown_connections 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082c0cbde0 0x7f082c0cc270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f082c161950 0x7f082c151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 --2- 172.21.15.1:0/1685186054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f082c0d2b10 0x7f082c0b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 >> 172.21.15.1:0/1685186054 conn(0x7f082c0d3e20 msgr2=0x7f082c0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 shutdown_connections 2024-05-09T06:28:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.588+0000 7f083329a640 1 -- 172.21.15.1:0/1685186054 wait complete. 2024-05-09T06:28:05.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:05.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.709+0000 7f0b43a95640 1 Processor -- start 2024-05-09T06:28:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.709+0000 7f0b43a95640 1 -- start start 2024-05-09T06:28:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b43a95640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c15a4d0 0x7f0b3c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b43a95640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0b3c071610 0x7f0b3c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b43a95640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b3c0cf1b0 con 0x7f0b3c0cdb90 2024-05-09T06:28:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b43a95640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3c15a8b0 con 0x7f0b3c071610 2024-05-09T06:28:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b43a95640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3c06dc10 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b42032640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0b3c071610 0x7f0b3c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b41831640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c15a4d0 0x7f0b3c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b41030640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f0b3c0cdb90 0x7f0b3c070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:38376/0 (socket says 172.21.15.1:38376) 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b42032640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0b3c071610 0x7f0b3c06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:38124/0 (socket says 172.21.15.1:38124) 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.710+0000 7f0b41030640 1 -- 172.21.15.1:0/2983788134 learned_addr learned my addr 172.21.15.1:0/2983788134 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.711+0000 7f0b42032640 1 -- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c15a4d0 msgr2=0x7f0b3c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.711+0000 7f0b42032640 1 --2- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c15a4d0 0x7f0b3c0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.711+0000 7f0b42032640 1 -- 172.21.15.1:0/2983788134 >> v1:172.21.15.6:6789/0 conn(0x7f0b3c0cdb90 legacy=0x7f0b3c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.711+0000 7f0b42032640 1 -- 172.21.15.1:0/2983788134 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c0cf490 con 0x7f0b3c071610 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.711+0000 7f0b42032640 1 --2- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c071610 0x7f0b3c06e0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7bb85039148072a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b4082f640 1 -- 172.21.15.1:0/2983788134 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34007740 con 0x7f0b3c071610 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b4082f640 1 -- 172.21.15.1:0/2983788134 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b340078e0 con 0x7f0b3c071610 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b4082f640 1 -- 172.21.15.1:0/2983788134 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34007cb0 con 0x7f0b3c071610 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 -- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c071610 msgr2=0x7f0b3c06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c071610 0x7f0b3c06e0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3402a660 comp rx=0 tx=0).stop 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 -- 172.21.15.1:0/2983788134 shutdown_connections 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c071610 0x7f0b3c06e0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2983788134 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c15a4d0 0x7f0b3c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 -- 172.21.15.1:0/2983788134 >> 172.21.15.1:0/2983788134 conn(0x7f0b3c0cc4d0 msgr2=0x7f0b3c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 -- 172.21.15.1:0/2983788134 shutdown_connections 2024-05-09T06:28:05.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.712+0000 7f0b43a95640 1 -- 172.21.15.1:0/2983788134 wait complete. 2024-05-09T06:28:05.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.713+0000 7f0b43a95640 1 Processor -- start 2024-05-09T06:28:05.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 -- start start 2024-05-09T06:28:05.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c071610 0x7f0b3c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c0cdb90 0x7f0b3c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c0b2be0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c0b2a60 con 0x7f0b3c0cdb90 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b43a95640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3c06cd30 con 0x7f0b3c071610 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.714+0000 7f0b42032640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59034/0 (socket says 172.21.15.1:59034) 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 -- 172.21.15.1:0/2283955641 learned_addr learned my addr 172.21.15.1:0/2283955641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b41030640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c0cdb90 0x7f0b3c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b41831640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c071610 0x7f0b3c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 -- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c071610 msgr2=0x7f0b3c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c071610 0x7f0b3c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 -- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c0cdb90 msgr2=0x7f0b3c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c0cdb90 0x7f0b3c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 -- 172.21.15.1:0/2283955641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c0b40e0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b42032640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009eb0 tx=0x7f0b34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34040070 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c0b43c0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34031c60 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34009450 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.715+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c1431a0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.716+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3c142ce0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.717+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b34039040 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.717+0000 7f0b327fc640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 0x7f0b2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.718+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b340ac2d0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.718+0000 7f0b41831640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 0x7f0b2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.718+0000 7f0b41831640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 0x7f0b2006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009aa0 tx=0x7f0b38009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.722+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b3c142ce0 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.907+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b3c0cef00 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.908+0000 7f0b327fc640 1 -- 172.21.15.1:0/2283955641 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0b3c0cef00 con 0x7f0b3c15a4d0 2024-05-09T06:28:05.890 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 msgr2=0x7f0b2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 0x7f0b2006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009aa0 tx=0x7f0b38009190 comp rx=0 tx=0).stop 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 msgr2=0x7f0b3c146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009eb0 tx=0x7f0b34004a40 comp rx=0 tx=0).stop 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 shutdown_connections 2024-05-09T06:28:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b2006be60 0x7f0b2006e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c15a4d0 0x7f0b3c146260 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b3c0cdb90 0x7f0b3c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 --2- 172.21.15.1:0/2283955641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b3c071610 0x7f0b3c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 >> 172.21.15.1:0/2283955641 conn(0x7f0b3c0cc4d0 msgr2=0x7f0b3c14e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 shutdown_connections 2024-05-09T06:28:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:05.910+0000 7f0b43a95640 1 -- 172.21.15.1:0/2283955641 wait complete. 2024-05-09T06:28:05.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:06.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.028+0000 7f9d7cef5640 1 Processor -- start 2024-05-09T06:28:06.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.028+0000 7f9d7cef5640 1 -- start start 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7cef5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9d7806ede0 0x7f9d78165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7cef5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7815b420 con 0x7f9d78158a50 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7cef5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7815b6e0 con 0x7f9d78071fd0 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7cef5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9d78158e30 con 0x7f9d7806ede0 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9d78158a50 0x7f9d780718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45728/0 (socket says 172.21.15.1:45728) 2024-05-09T06:28:06.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7659c640 1 -- 172.21.15.1:0/2480757718 learned_addr learned my addr 172.21.15.1:0/2480757718 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d76d9d640 1 --2- 172.21.15.1:0/2480757718 >> v2:172.21.15.6:3300/0 conn(0x7f9d7806ede0 0x7f9d78165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060295727 0 0) 0x7f9d7815b420 con 0x7f9d78158a50 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d60003520 con 0x7f9d78158a50 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d76d9d640 1 -- 172.21.15.1:0/2480757718 >> v1:172.21.15.6:6789/0 conn(0x7f9d78071fd0 legacy=0x7f9d7806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d76d9d640 1 -- 172.21.15.1:0/2480757718 >> v1:172.21.15.1:6789/0 conn(0x7f9d78158a50 legacy=0x7f9d780718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675715821 0 0) 0x7f9d60003520 con 0x7f9d78158a50 2024-05-09T06:28:06.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.029+0000 7f9d76d9d640 1 -- 172.21.15.1:0/2480757718 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7815b9c0 con 0x7f9d7806ede0 2024-05-09T06:28:06.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.030+0000 7f9d76d9d640 1 --2- 172.21.15.1:0/2480757718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d7806ede0 0x7f9d78165b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009d90 tx=0x7f9d7002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f87fee114bf2ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.030+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68001690 con 0x7f9d7806ede0 2024-05-09T06:28:06.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.030+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d70007540 con 0x7f9d7806ede0 2024-05-09T06:28:06.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.030+0000 7f9d7559a640 1 -- 172.21.15.1:0/2480757718 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d700078f0 con 0x7f9d7806ede0 2024-05-09T06:28:06.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 -- 172.21.15.1:0/2480757718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d7806ede0 msgr2=0x7f9d78165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/2480757718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d7806ede0 0x7f9d78165b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009d90 tx=0x7f9d7002a680 comp rx=0 tx=0).stop 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 -- 172.21.15.1:0/2480757718 shutdown_connections 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/2480757718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d7806ede0 0x7f9d78165b70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 -- 172.21.15.1:0/2480757718 >> 172.21.15.1:0/2480757718 conn(0x7f9d780cc4d0 msgr2=0x7f9d780cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 -- 172.21.15.1:0/2480757718 shutdown_connections 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.031+0000 7f9d7cef5640 1 -- 172.21.15.1:0/2480757718 wait complete. 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.032+0000 7f9d7cef5640 1 Processor -- start 2024-05-09T06:28:06.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.032+0000 7f9d7cef5640 1 -- start start 2024-05-09T06:28:06.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d78071fd0 0x7f9d78159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d78158a50 0x7f9d7815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7815bf80 con 0x7f9d7806ede0 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78072480 con 0x7f9d78158a50 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7cef5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9d78070ea0 con 0x7f9d78071fd0 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59042/0 (socket says 172.21.15.1:59042) 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d7659c640 1 -- 172.21.15.1:0/3238336894 learned_addr learned my addr 172.21.15.1:0/3238336894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d76d9d640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d78158a50 0x7f9d7815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.033+0000 7f9d75d9b640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d78071fd0 0x7f9d78159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 -- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d78071fd0 msgr2=0x7f9d78159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d78071fd0 0x7f9d78159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 -- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d78158a50 msgr2=0x7f9d7815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d78158a50 0x7f9d7815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 -- 172.21.15.1:0/3238336894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d780af340 con 0x7f9d7806ede0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7659c640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d68001200 tx=0x7f9d6800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68018070 con 0x7f9d7806ede0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d68007500 con 0x7f9d7806ede0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004a20 con 0x7f9d7806ede0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d780a9b80 con 0x7f9d7806ede0 2024-05-09T06:28:06.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.034+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d780aa140 con 0x7f9d7806ede0 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.035+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d780752b0 con 0x7f9d7806ede0 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.036+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d6801b070 con 0x7f9d7806ede0 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.036+0000 7f9d66ffd640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.036+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9d6808a460 con 0x7f9d7806ede0 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.036+0000 7f9d75d9b640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.037+0000 7f9d75d9b640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 0x7f9d5806e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d780abfa0 tx=0x7f9d6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.041+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d68058450 con 0x7f9d7806ede0 2024-05-09T06:28:06.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.224+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d780aa520 con 0x7f9d7806ede0 2024-05-09T06:28:06.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.225+0000 7f9d66ffd640 1 -- 172.21.15.1:0/3238336894 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f9d6805c750 con 0x7f9d7806ede0 2024-05-09T06:28:06.207 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.226+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 msgr2=0x7f9d5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.226+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 0x7f9d5806e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d780abfa0 tx=0x7f9d6c009040 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.226+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 msgr2=0x7f9d780acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.226+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d68001200 tx=0x7f9d6800c9b0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 shutdown_connections 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d5806be30 0x7f9d5806e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d78158a50 0x7f9d7815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d78071fd0 0x7f9d78159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 --2- 172.21.15.1:0/3238336894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d7806ede0 0x7f9d780acbc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 >> 172.21.15.1:0/3238336894 conn(0x7f9d780cc4d0 msgr2=0x7f9d781592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 shutdown_connections 2024-05-09T06:28:06.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.227+0000 7f9d7cef5640 1 -- 172.21.15.1:0/3238336894 wait complete. 2024-05-09T06:28:06.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.344+0000 7fa1ffd5a640 1 Processor -- start 2024-05-09T06:28:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.345+0000 7fa1ffd5a640 1 -- start start 2024-05-09T06:28:06.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.345+0000 7fa1ffd5a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1f815e190 0x7fa1f815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.345+0000 7fa1ffd5a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f8161fe0 con 0x7fa1f80d45f0 2024-05-09T06:28:06.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.345+0000 7fa1ffd5a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f81622c0 con 0x7fa1f80d1c70 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.345+0000 7fa1ffd5a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f815f580 con 0x7fa1f815e190 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fe2f7640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa1f80d45f0 0x7fa1f80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45732/0 (socket says 172.21.15.1:45732) 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fe2f7640 1 -- 172.21.15.1:0/1158538586 learned_addr learned my addr 172.21.15.1:0/1158538586 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277187938 0 0) 0x7fa1f8161fe0 con 0x7fa1f80d45f0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/1158538586 >> v2:172.21.15.6:3300/0 conn(0x7fa1f815e190 0x7fa1f815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e0003520 con 0x7fa1f80d45f0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 556181139 0 0) 0x7fa1e0003520 con 0x7fa1f80d45f0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 >> v2:172.21.15.6:3300/0 conn(0x7fa1f815e190 msgr2=0x7fa1f815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 --2- 172.21.15.1:0/1158538586 >> v2:172.21.15.6:3300/0 conn(0x7fa1f815e190 0x7fa1f815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 >> v1:172.21.15.6:6789/0 conn(0x7fa1f80d1c70 legacy=0x7fa1f80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81625a0 con 0x7fa1f80d45f0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fa1e80019f0 con 0x7fa1f80d45f0 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 >> v1:172.21.15.1:6789/0 conn(0x7fa1f80d45f0 legacy=0x7fa1f80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00039b0 0x7fa1e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 0x7fa1e0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1e0005080 0x7fa1e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e000bfb0 con 0x7fa1e00043b0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e000c160 con 0x7fa1e00039b0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e000c360 con 0x7fa1e0005080 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 0x7fa1e0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00039b0 0x7fa1e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.346+0000 7fa1fe2f7640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1e0005080 0x7fa1e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1e0005080 msgr2=0x7fa1e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1e0005080 0x7fa1e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00039b0 msgr2=0x7fa1e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00039b0 0x7fa1e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/1158538586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e0003520 con 0x7fa1e00043b0 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fe2f7640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1e0005080 0x7fa1e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fe2f7640 1 -- 172.21.15.1:0/1158538586 reap_dead start 2024-05-09T06:28:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.347+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 0x7fa1e0004880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001830 tx=0x7fa1e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be74ac2941d5d10c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa1e00043b0 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec02aba0 con 0x7fa1e00043b0 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1fcaf4640 1 -- 172.21.15.1:0/1158538586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec001430 con 0x7fa1e00043b0 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 msgr2=0x7fa1e0004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 0x7fa1e0004880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001830 tx=0x7fa1e0005830 comp rx=0 tx=0).stop 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/1158538586 shutdown_connections 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/1158538586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1e00043b0 0x7fa1e0004880 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/1158538586 >> 172.21.15.1:0/1158538586 conn(0x7fa1f8073340 msgr2=0x7fa1f809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/1158538586 shutdown_connections 2024-05-09T06:28:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.348+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/1158538586 wait complete. 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.349+0000 7fa1ffd5a640 1 Processor -- start 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.349+0000 7fa1ffd5a640 1 -- start start 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 0x7fa1f80cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 0x7fa1f806be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f80b56d0 con 0x7fa1f81f7e70 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8161c60 con 0x7fa1e00043b0 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1ffd5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f80b5850 con 0x7fa1f81f46a0 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1fd2f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1fd2f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59074/0 (socket says 172.21.15.1:59074) 2024-05-09T06:28:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/2182436703 learned_addr learned my addr 172.21.15.1:0/2182436703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1fe2f7640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 0x7fa1f806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.350+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 0x7fa1f80cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 msgr2=0x7fa1f806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 0x7fa1f806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 msgr2=0x7fa1f80cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 0x7fa1f80cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 -- 172.21.15.1:0/2182436703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81f4b50 con 0x7fa1f81f7e70 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fe2f7640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 0x7fa1f806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 0x7fa1f80cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1fd2f5640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec035040 tx=0x7fa1ec001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec040020 con 0x7fa1f81f7e70 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec038040 con 0x7fa1f81f7e70 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa1f81f7e70 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f80cdd30 con 0x7fa1f81f7e70 2024-05-09T06:28:06.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.351+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81f7670 con 0x7fa1f81f7e70 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.352+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f80752b0 con 0x7fa1f81f7e70 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.353+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1ec043020 con 0x7fa1f81f7e70 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.353+0000 7fa1f67fc640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.353+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa1ec0ab4a0 con 0x7fa1f81f7e70 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.353+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.354+0000 7fa1fdaf6640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 0x7fa1e406e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ca640 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.357+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1ec04a030 con 0x7fa1f81f7e70 2024-05-09T06:28:06.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.538+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1f80b9f20 con 0x7fa1f81f7e70 2024-05-09T06:28:06.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.539+0000 7fa1f67fc640 1 -- 172.21.15.1:0/2182436703 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa1f81f4b50 con 0x7fa1f81f7e70 2024-05-09T06:28:06.521 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 msgr2=0x7fa1e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 0x7fa1e406e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ca640 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 msgr2=0x7fa1f814e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec035040 tx=0x7fa1ec001230 comp rx=0 tx=0).stop 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 shutdown_connections 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1f81f46a0 0x7fa1f806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f81f7e70 0x7fa1f814e670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 --2- 172.21.15.1:0/2182436703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1e00043b0 0x7fa1f80cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 >> 172.21.15.1:0/2182436703 conn(0x7fa1f8073340 msgr2=0x7fa1f806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 shutdown_connections 2024-05-09T06:28:06.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.541+0000 7fa1ffd5a640 1 -- 172.21.15.1:0/2182436703 wait complete. 2024-05-09T06:28:06.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:06.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.659+0000 7f113f61c640 1 Processor -- start 2024-05-09T06:28:06.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.659+0000 7f113f61c640 1 -- start start 2024-05-09T06:28:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113f61c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113f61c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f11380ceba0 0x7f11380ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113f61c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11380b1340 con 0x7f11380d08b0 2024-05-09T06:28:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113f61c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f11380cc260 con 0x7f11380ceba0 2024-05-09T06:28:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113f61c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f11380d1020 con 0x7f113815e270 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113d3b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113dbb9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f11380ceba0 0x7f11380ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113d3b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:53930/0 (socket says 172.21.15.1:53930) 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113d3b8640 1 -- 172.21.15.1:0/2929407993 learned_addr learned my addr 172.21.15.1:0/2929407993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.660+0000 7f113cbb7640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f11380d08b0 0x7f11380ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:38412/0 (socket says 172.21.15.1:38412) 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.661+0000 7f113d3b8640 1 -- 172.21.15.1:0/2929407993 >> v2:172.21.15.6:3300/0 conn(0x7f11380ceba0 msgr2=0x7f11380ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.661+0000 7f113d3b8640 1 --2- 172.21.15.1:0/2929407993 >> v2:172.21.15.6:3300/0 conn(0x7f11380ceba0 0x7f11380ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.661+0000 7f113d3b8640 1 -- 172.21.15.1:0/2929407993 >> v1:172.21.15.6:6789/0 conn(0x7f11380d08b0 legacy=0x7f11380ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.661+0000 7f113d3b8640 1 -- 172.21.15.1:0/2929407993 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11380cc4b0 con 0x7f113815e270 2024-05-09T06:28:06.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.661+0000 7f113d3b8640 1 --2- 172.21.15.1:0/2929407993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eba5160d981e2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f112ffff640 1 -- 172.21.15.1:0/2929407993 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134037070 con 0x7f113815e270 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f112ffff640 1 -- 172.21.15.1:0/2929407993 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113402ae50 con 0x7f113815e270 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f112ffff640 1 -- 172.21.15.1:0/2929407993 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134032860 con 0x7f113815e270 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 -- 172.21.15.1:0/2929407993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 msgr2=0x7f11380cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 --2- 172.21.15.1:0/2929407993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113402a300 comp rx=0 tx=0).stop 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 -- 172.21.15.1:0/2929407993 shutdown_connections 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 --2- 172.21.15.1:0/2929407993 >> v2:172.21.15.6:3300/0 conn(0x7f11380ceba0 0x7f11380ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 --2- 172.21.15.1:0/2929407993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f113815e270 0x7f11380cf180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 -- 172.21.15.1:0/2929407993 >> 172.21.15.1:0/2929407993 conn(0x7f11380d1c10 msgr2=0x7f11380d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 -- 172.21.15.1:0/2929407993 shutdown_connections 2024-05-09T06:28:06.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.662+0000 7f113f61c640 1 -- 172.21.15.1:0/2929407993 wait complete. 2024-05-09T06:28:06.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 Processor -- start 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 -- start start 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11380d08b0 0x7f11380c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f113815e650 con 0x7f113815e270 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f11380cca70 con 0x7f11380d08b0 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.664+0000 7f113f61c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f11380d0cf0 con 0x7f11380ceba0 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113dbb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113cbb7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11380d08b0 0x7f11380c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113dbb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59088/0 (socket says 172.21.15.1:59088) 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:53936/0 (socket says 172.21.15.1:53936) 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113dbb9640 1 -- 172.21.15.1:0/123245827 learned_addr learned my addr 172.21.15.1:0/123245827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 -- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11380d08b0 msgr2=0x7f11380c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11380d08b0 0x7f11380c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 -- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 msgr2=0x7f11380c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 -- 172.21.15.1:0/123245827 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11380ca670 con 0x7f11380ceba0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113dbb9640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113d3b8640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1134031000 tx=0x7f11340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134041070 con 0x7f11380ceba0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113403c660 con 0x7f11380ceba0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11380ca950 con 0x7f11380ceba0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.665+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134037040 con 0x7f11380ceba0 2024-05-09T06:28:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.666+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11380caf10 con 0x7f11380ceba0 2024-05-09T06:28:06.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.666+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11380752b0 con 0x7f11380ceba0 2024-05-09T06:28:06.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.667+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1134044070 con 0x7f11380ceba0 2024-05-09T06:28:06.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.668+0000 7f112dffb640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 0x7f111c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.668+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f11340abb40 con 0x7f11380ceba0 2024-05-09T06:28:06.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.668+0000 7f113cbb7640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 0x7f111c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.668+0000 7f113cbb7640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 0x7f111c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11380c8040 tx=0x7f1128007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.672+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1134079c30 con 0x7f11380ceba0 2024-05-09T06:28:06.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.855+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f11380d22a0 con 0x7f11380ceba0 2024-05-09T06:28:06.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.855+0000 7f112dffb640 1 -- 172.21.15.1:0/123245827 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f113407df30 con 0x7f11380ceba0 2024-05-09T06:28:06.837 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 msgr2=0x7f111c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 0x7f111c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11380c8040 tx=0x7f1128007e80 comp rx=0 tx=0).stop 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 msgr2=0x7f11381f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1134031000 tx=0x7f11340046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 shutdown_connections 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f111c06be30 0x7f111c06e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113815e270 0x7f11380c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11380d08b0 0x7f11380c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 --2- 172.21.15.1:0/123245827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11380ceba0 0x7f11381f4320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 >> 172.21.15.1:0/123245827 conn(0x7f11380d1c10 msgr2=0x7f11380aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 shutdown_connections 2024-05-09T06:28:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.858+0000 7f113f61c640 1 -- 172.21.15.1:0/123245827 wait complete. 2024-05-09T06:28:06.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:06.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.973+0000 7ff406031640 1 Processor -- start 2024-05-09T06:28:06.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.973+0000 7ff406031640 1 -- start start 2024-05-09T06:28:06.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff406031640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 0x7ff40006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff406031640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4000cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff406031640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4000ce0e0 con 0x7ff4000cdab0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff406031640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff4000ce3c0 con 0x7ff40006e6d0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff406031640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff4000ce6a0 con 0x7ff4000cf180 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3fffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff4000cdab0 0x7ff40015e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45738/0 (socket says 172.21.15.1:45738) 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3fffff640 1 -- 172.21.15.1:0/2557822215 learned_addr learned my addr 172.21.15.1:0/2557822215 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2557822215 >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 0x7ff40006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2557822215 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4000cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3feffd640 1 -- 172.21.15.1:0/2557822215 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984861916 0 0) 0x7ff4000ce0e0 con 0x7ff4000cdab0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3feffd640 1 -- 172.21.15.1:0/2557822215 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3ec003520 con 0x7ff4000cdab0 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2557822215 >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 msgr2=0x7ff40006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2557822215 >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 0x7ff40006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2557822215 >> v1:172.21.15.1:6789/0 conn(0x7ff4000cdab0 legacy=0x7ff40015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2557822215 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4000ce980 con 0x7ff4000cf180 2024-05-09T06:28:06.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.974+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2557822215 >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 0x7ff40006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.975+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2557822215 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4000cd570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80097a0 tx=0x7ff3e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1536129dcd289315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.975+0000 7ff3feffd640 1 -- 172.21.15.1:0/2557822215 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8037070 con 0x7ff4000cf180 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.975+0000 7ff3feffd640 1 -- 172.21.15.1:0/2557822215 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8032440 con 0x7ff4000cf180 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.975+0000 7ff3feffd640 1 -- 172.21.15.1:0/2557822215 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8032810 con 0x7ff4000cf180 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 -- 172.21.15.1:0/2557822215 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 msgr2=0x7ff4000cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 --2- 172.21.15.1:0/2557822215 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4000cd570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80097a0 tx=0x7ff3e802a180 comp rx=0 tx=0).stop 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 -- 172.21.15.1:0/2557822215 shutdown_connections 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 --2- 172.21.15.1:0/2557822215 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4000cd570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 --2- 172.21.15.1:0/2557822215 >> v2:172.21.15.6:3300/0 conn(0x7ff40006e6d0 0x7ff40006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 -- 172.21.15.1:0/2557822215 >> 172.21.15.1:0/2557822215 conn(0x7ff4000cc4b0 msgr2=0x7ff4000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 -- 172.21.15.1:0/2557822215 shutdown_connections 2024-05-09T06:28:06.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.976+0000 7ff406031640 1 -- 172.21.15.1:0/2557822215 wait complete. 2024-05-09T06:28:06.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.977+0000 7ff406031640 1 Processor -- start 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.977+0000 7ff406031640 1 -- start start 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 0x7ff40013afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4001ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff400070ed0 con 0x7ff4000cdab0 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff40006eff0 con 0x7ff40006e6d0 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff406031640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff4001447c0 con 0x7ff4000cf180 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59096/0 (socket says 172.21.15.1:59096) 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2848927683 learned_addr learned my addr 172.21.15.1:0/2848927683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3fffff640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4001ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 0x7ff40013afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.978+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 msgr2=0x7ff4001ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4001ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 msgr2=0x7ff40013afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 0x7ff40013afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3f7fff640 1 -- 172.21.15.1:0/2848927683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4001f0880 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 0x7ff40013afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3fffff640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4001ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3f7fff640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8000c00 tx=0x7ff3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8041020 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e803d640 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8037040 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001f0b20 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.979+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001f10e0 con 0x7ff4000cdab0 2024-05-09T06:28:06.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.980+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4000752b0 con 0x7ff4000cdab0 2024-05-09T06:28:06.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.980+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3e8044020 con 0x7ff4000cdab0 2024-05-09T06:28:06.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.981+0000 7ff3fcff9640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 0x7ff3e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:06.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.981+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff3e80ab9d0 con 0x7ff4000cdab0 2024-05-09T06:28:06.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.981+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 0x7ff3e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:06.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.981+0000 7ff3ff7fe640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 0x7ff3e006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009af0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:06.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:06.985+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3e804b030 con 0x7ff4000cdab0 2024-05-09T06:28:07.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.171+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff40006cf50 con 0x7ff4000cdab0 2024-05-09T06:28:07.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.173+0000 7ff3fcff9640 1 -- 172.21.15.1:0/2848927683 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7ff3e8079f90 con 0x7ff4000cdab0 2024-05-09T06:28:07.155 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:07.155 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:07.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 msgr2=0x7ff3e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 0x7ff3e006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009af0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2024-05-09T06:28:07.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 msgr2=0x7ff40013cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8000c00 tx=0x7ff3e80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 shutdown_connections 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff3e006be30 0x7ff3e006e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff4000cf180 0x7ff4001ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4000cdab0 0x7ff40013cd50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 --2- 172.21.15.1:0/2848927683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40006e6d0 0x7ff40013afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 >> 172.21.15.1:0/2848927683 conn(0x7ff4000cc4b0 msgr2=0x7ff400142190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 shutdown_connections 2024-05-09T06:28:07.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.175+0000 7ff406031640 1 -- 172.21.15.1:0/2848927683 wait complete. 2024-05-09T06:28:07.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:07.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:07.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.293+0000 7f8cd355a640 1 Processor -- start 2024-05-09T06:28:07.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.293+0000 7f8cd355a640 1 -- start start 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8ccc15e190 0x7f8ccc15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc0b56d0 0x7f8ccc0d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc161fe0 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc1622c0 con 0x7f8ccc15e190 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd355a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc1625a0 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd0af5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd0af5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59112/0 (socket says 172.21.15.1:59112) 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.294+0000 7f8cd0af5640 1 -- 172.21.15.1:0/3543560099 learned_addr learned my addr 172.21.15.1:0/3543560099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd1af7640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc0b56d0 0x7f8ccc0d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 -- 172.21.15.1:0/3543560099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc0b56d0 msgr2=0x7f8ccc0d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc0b56d0 0x7f8ccc0d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd12f6640 1 --2- 172.21.15.1:0/3543560099 >> v2:172.21.15.6:3300/0 conn(0x7f8ccc15e190 0x7f8ccc15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 -- 172.21.15.1:0/3543560099 >> v2:172.21.15.6:3300/0 conn(0x7f8ccc15e190 msgr2=0x7f8ccc15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/3543560099 >> v2:172.21.15.6:3300/0 conn(0x7f8ccc15e190 0x7f8ccc15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 -- 172.21.15.1:0/3543560099 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc162880 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.295+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0d1bf0 tx=0x7f8cc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4946b604dd967e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8ccbfff640 1 -- 172.21.15.1:0/3543560099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0037070 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8ccbfff640 1 -- 172.21.15.1:0/3543560099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0032440 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8ccbfff640 1 -- 172.21.15.1:0/3543560099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0032810 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 -- 172.21.15.1:0/3543560099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0d1c70 msgr2=0x7f8ccc0d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0d1bf0 tx=0x7f8cc002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 -- 172.21.15.1:0/3543560099 shutdown_connections 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc0b56d0 0x7f8ccc0d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 --2- 172.21.15.1:0/3543560099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0d3ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 --2- 172.21.15.1:0/3543560099 >> v2:172.21.15.6:3300/0 conn(0x7f8ccc15e190 0x7f8ccc15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 -- 172.21.15.1:0/3543560099 >> 172.21.15.1:0/3543560099 conn(0x7f8ccc073340 msgr2=0x7f8ccc09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 -- 172.21.15.1:0/3543560099 shutdown_connections 2024-05-09T06:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.296+0000 7f8cd355a640 1 -- 172.21.15.1:0/3543560099 wait complete. 2024-05-09T06:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.297+0000 7f8cd355a640 1 Processor -- start 2024-05-09T06:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.297+0000 7f8cd355a640 1 -- start start 2024-05-09T06:28:07.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 0x7f8ccc0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b3f60 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b3de0 con 0x7f8ccc0d1c70 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd355a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b40e0 con 0x7f8ccc15e190 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd12f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd12f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59118/0 (socket says 172.21.15.1:59118) 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.298+0000 7f8cd12f6640 1 -- 172.21.15.1:0/644110988 learned_addr learned my addr 172.21.15.1:0/644110988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd1af7640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 0x7f8ccc0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 -- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 msgr2=0x7f8ccc0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 0x7f8ccc0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 -- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 msgr2=0x7f8ccc0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 -- 172.21.15.1:0/644110988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc0cbc00 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd1af7640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 0x7f8ccc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd12f6640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c7f0 tx=0x7f8cc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4015070 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc4004220 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc0cbee0 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4007620 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.299+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc0cc780 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.300+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc0752b0 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.301+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cc40079d0 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.301+0000 7f8cc9ffb640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.301+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8cc408b580 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.301+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.302+0000 7f8cd0af5640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 0x7f8cb806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034000 tx=0x7f8cc002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.306+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8cc4059b30 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.488+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ccc0c7760 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.488+0000 7f8cc9ffb640 1 -- 172.21.15.1:0/644110988 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f8cc4059430 con 0x7f8ccc0b56d0 2024-05-09T06:28:07.470 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:07.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 msgr2=0x7f8cb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 0x7f8cb806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034000 tx=0x7f8cc002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:07.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 msgr2=0x7f8ccc06c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c7f0 tx=0x7f8cc400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 shutdown_connections 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ccc15e190 0x7f8ccc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ccc0d1c70 0x7f8ccc0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 --2- 172.21.15.1:0/644110988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc0b56d0 0x7f8ccc06c230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 >> 172.21.15.1:0/644110988 conn(0x7f8ccc073340 msgr2=0x7f8ccc15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 shutdown_connections 2024-05-09T06:28:07.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.490+0000 7f8cd355a640 1 -- 172.21.15.1:0/644110988 wait complete. 2024-05-09T06:28:07.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:07.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.607+0000 7f0773812640 1 Processor -- start 2024-05-09T06:28:07.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0773812640 1 -- start start 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0773812640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f076c15e190 0x7f076c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0773812640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c161fe0 con 0x7f076c0d1c70 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0773812640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c1622c0 con 0x7f076c0d45f0 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0773812640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f076c15f580 con 0x7f076c15e190 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0770dad640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f076c0d1c70 0x7f076c0d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45746/0 (socket says 172.21.15.1:45746) 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.608+0000 7f0770dad640 1 -- 172.21.15.1:0/1691661679 learned_addr learned my addr 172.21.15.1:0/1691661679 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:07.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f07715ae640 1 --2- 172.21.15.1:0/1691661679 >> v2:172.21.15.6:3300/0 conn(0x7f076c15e190 0x7f076c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914556024 0 0) 0x7f076c161fe0 con 0x7f076c0d1c70 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0754003520 con 0x7f076c0d1c70 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 34132160 0 0) 0x7f0754003520 con 0x7f076c0d1c70 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 >> v2:172.21.15.6:3300/0 conn(0x7f076c15e190 msgr2=0x7f076c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 --2- 172.21.15.1:0/1691661679 >> v2:172.21.15.6:3300/0 conn(0x7f076c15e190 0x7f076c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 >> v1:172.21.15.6:6789/0 conn(0x7f076c0d45f0 legacy=0x7f076c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c1625a0 con 0x7f076c0d1c70 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f0760001670 con 0x7f076c0d1c70 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f07715ae640 1 --2- 172.21.15.1:0/1691661679 >> v2:172.21.15.6:3300/0 conn(0x7f076c15e190 0x7f076c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 >> v1:172.21.15.1:6789/0 conn(0x7f076c0d1c70 legacy=0x7f076c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540039b0 0x7f0754003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0754004800 0x7f0754004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 0x7f075400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f075400c030 con 0x7f07540051c0 2024-05-09T06:28:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f075400c1e0 con 0x7f07540039b0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f075400c3e0 con 0x7f0754004800 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f0771daf640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 0x7f075400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f07715ae640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540039b0 0x7f0754003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.609+0000 7f0770dad640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0754004800 0x7f0754004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 -- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0754004800 msgr2=0x7f0754004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0754004800 0x7f0754004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 -- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540039b0 msgr2=0x7f0754003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540039b0 0x7f0754003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 -- 172.21.15.1:0/1691661679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0754003520 con 0x7f07540051c0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0770dad640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0754004800 0x7f0754004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 -- 172.21.15.1:0/1691661679 reap_dead start 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f0771daf640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 0x7f075400b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f075c001e20 tx=0x7f0754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9256b30aaadda45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c001a70 con 0x7f07540051c0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c02ae40 con 0x7f07540051c0 2024-05-09T06:28:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.610+0000 7f076bfff640 1 -- 172.21.15.1:0/1691661679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c003bb0 con 0x7f07540051c0 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 -- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 msgr2=0x7f075400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 0x7f075400b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f075c001e20 tx=0x7f0754005830 comp rx=0 tx=0).stop 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 -- 172.21.15.1:0/1691661679 shutdown_connections 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 --2- 172.21.15.1:0/1691661679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07540051c0 0x7f075400b9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 -- 172.21.15.1:0/1691661679 >> 172.21.15.1:0/1691661679 conn(0x7f076c073340 msgr2=0x7f076c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 -- 172.21.15.1:0/1691661679 shutdown_connections 2024-05-09T06:28:07.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.611+0000 7f0773812640 1 -- 172.21.15.1:0/1691661679 wait complete. 2024-05-09T06:28:07.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.612+0000 7f0773812640 1 Processor -- start 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.612+0000 7f0773812640 1 -- start start 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540051c0 0x7f076c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 0x7f076c141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0b56d0 con 0x7f076c0b24e0 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f076c161c60 con 0x7f07540051c0 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0773812640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f076c0b5850 con 0x7f076c141660 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59144/0 (socket says 172.21.15.1:59144) 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 -- 172.21.15.1:0/3213542179 learned_addr learned my addr 172.21.15.1:0/3213542179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f07715ae640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540051c0 0x7f076c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0771daf640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 0x7f076c141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 -- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 msgr2=0x7f076c141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 0x7f076c141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 -- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540051c0 msgr2=0x7f076c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540051c0 0x7f076c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 -- 172.21.15.1:0/3213542179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0b1720 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0771daf640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 0x7f076c141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.613+0000 7f0770dad640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07600017e0 tx=0x7f076000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.614+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076000ee00 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.614+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0760018070 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.614+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c0b1a00 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.614+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076000ca70 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.614+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c0b9b00 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.615+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f076c0752b0 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.615+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f076000cc10 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.616+0000 7f0769ffb640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 0x7f075806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.616+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f076008b010 con 0x7f076c0b24e0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.616+0000 7f07715ae640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 0x7f075806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.616+0000 7f07715ae640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 0x7f075806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0764009c10 tx=0x7f0764008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.620+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07600595b0 con 0x7f076c0b24e0 2024-05-09T06:28:07.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.805+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f076c08e1c0 con 0x7f076c0b24e0 2024-05-09T06:28:07.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.805+0000 7f0769ffb640 1 -- 172.21.15.1:0/3213542179 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0760058eb0 con 0x7f076c0b24e0 2024-05-09T06:28:07.787 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 msgr2=0x7f075806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 0x7f075806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0764009c10 tx=0x7f0764008040 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 msgr2=0x7f076c141120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07600017e0 tx=0x7f076000ec10 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 shutdown_connections 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f075806be30 0x7f075806e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f076c141660 0x7f076c141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f076c0b24e0 0x7f076c141120 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 --2- 172.21.15.1:0/3213542179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07540051c0 0x7f076c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 >> 172.21.15.1:0/3213542179 conn(0x7f076c073340 msgr2=0x7f076c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 shutdown_connections 2024-05-09T06:28:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.807+0000 7f0773812640 1 -- 172.21.15.1:0/3213542179 wait complete. 2024-05-09T06:28:07.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:07.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.925+0000 7fc5fb25d640 1 Processor -- start 2024-05-09T06:28:07.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.925+0000 7fc5fb25d640 1 -- start start 2024-05-09T06:28:07.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5fb25d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5f40b11f0 0x7fc5f40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5fb25d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 0x7fc5f40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5fb25d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5f415e900 con 0x7fc5f415e390 2024-05-09T06:28:07.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5fb25d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f415ebe0 con 0x7fc5f40b11f0 2024-05-09T06:28:07.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5fb25d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f415eec0 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5f40b11f0 0x7fc5f40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5f40b11f0 0x7fc5f40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35708/0 (socket says 172.21.15.1:35708) 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 -- 172.21.15.1:0/144671413 learned_addr learned my addr 172.21.15.1:0/144671413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f97fa640 1 --2- 172.21.15.1:0/144671413 >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 0x7fc5f40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 -- 172.21.15.1:0/144671413 >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 msgr2=0x7fc5f40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 --2- 172.21.15.1:0/144671413 >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 0x7fc5f40ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.926+0000 7fc5f3fff640 1 -- 172.21.15.1:0/144671413 >> v1:172.21.15.6:6789/0 conn(0x7fc5f415e390 legacy=0x7fc5f40cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f3fff640 1 -- 172.21.15.1:0/144671413 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f415f1a0 con 0x7fc5f40b11f0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f97fa640 1 --2- 172.21.15.1:0/144671413 >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 0x7fc5f40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f3fff640 1 --2- 172.21.15.1:0/144671413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b1680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40721e0 tx=0x7fc5e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff650eea8c4bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f37fe640 1 -- 172.21.15.1:0/144671413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8037070 con 0x7fc5f40b11f0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f37fe640 1 -- 172.21.15.1:0/144671413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e8032440 con 0x7fc5f40b11f0 2024-05-09T06:28:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.927+0000 7fc5f37fe640 1 -- 172.21.15.1:0/144671413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8032810 con 0x7fc5f40b11f0 2024-05-09T06:28:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 -- 172.21.15.1:0/144671413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40b11f0 msgr2=0x7fc5f40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/144671413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b1680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40721e0 tx=0x7fc5e802a230 comp rx=0 tx=0).stop 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 -- 172.21.15.1:0/144671413 shutdown_connections 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/144671413 >> v2:172.21.15.6:3300/0 conn(0x7fc5f40cc8e0 0x7fc5f40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/144671413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b1680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 -- 172.21.15.1:0/144671413 >> 172.21.15.1:0/144671413 conn(0x7fc5f40b1da0 msgr2=0x7fc5f40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 -- 172.21.15.1:0/144671413 shutdown_connections 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.928+0000 7fc5fb25d640 1 -- 172.21.15.1:0/144671413 wait complete. 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.929+0000 7fc5fb25d640 1 Processor -- start 2024-05-09T06:28:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.929+0000 7fc5fb25d640 1 -- start start 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5f415e390 0x7fc5f40b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f40cffb0 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4162ba0 con 0x7fc5f40b11f0 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5fb25d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f40cfb60 con 0x7fc5f415e390 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35716/0 (socket says 172.21.15.1:35716) 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5f3fff640 1 -- 172.21.15.1:0/2311170218 learned_addr learned my addr 172.21.15.1:0/2311170218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.930+0000 7fc5f97fa640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5f415e390 0x7fc5f40b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f8ff9640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 -- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5f415e390 msgr2=0x7fc5f40b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5f415e390 0x7fc5f40b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 -- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5f40b11f0 msgr2=0x7fc5f40b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 -- 172.21.15.1:0/2311170218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f40c50d0 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f3fff640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8041020 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e8030970 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f40c4850 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8037040 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.931+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f40c4df0 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.932+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f40752b0 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.932+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5e8044020 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.933+0000 7fc5f17fa640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.933+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc5e80abc00 con 0x7fc5f40cc8e0 2024-05-09T06:28:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.933+0000 7fc5f8ff9640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.933+0000 7fc5f8ff9640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 0x7fc5e006e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006fd0 tx=0x7fc5e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:07.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:07.937+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5e80b2080 con 0x7fc5f40cc8e0 2024-05-09T06:28:08.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.119+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5f40b2ae0 con 0x7fc5f40cc8e0 2024-05-09T06:28:08.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.120+0000 7fc5f17fa640 1 -- 172.21.15.1:0/2311170218 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc5e804b030 con 0x7fc5f40cc8e0 2024-05-09T06:28:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 msgr2=0x7fc5e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 0x7fc5e006e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006fd0 tx=0x7fc5e4008040 comp rx=0 tx=0).stop 2024-05-09T06:28:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 msgr2=0x7fc5f41f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 shutdown_connections 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5f415e390 0x7fc5f40b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5f40cc8e0 0x7fc5f41f72e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 --2- 172.21.15.1:0/2311170218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5f40b11f0 0x7fc5f40b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 >> 172.21.15.1:0/2311170218 conn(0x7fc5f40b1da0 msgr2=0x7fc5f4160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 shutdown_connections 2024-05-09T06:28:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.122+0000 7fc5fb25d640 1 -- 172.21.15.1:0/2311170218 wait complete. 2024-05-09T06:28:08.115 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:08.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.240+0000 7f03afc6b640 1 Processor -- start 2024-05-09T06:28:08.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.240+0000 7f03afc6b640 1 -- start start 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03afc6b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03a8147960 0x7f03a8071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03afc6b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a8147d40 con 0x7f03a80720a0 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03afc6b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a806d2f0 con 0x7f03a80ae8a0 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03afc6b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f03a8072820 con 0x7f03a8147960 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ada07640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03a8147960 0x7f03a8071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ad206640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f03a80720a0 0x7f03a806d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60640/0 (socket says 172.21.15.1:60640) 2024-05-09T06:28:08.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ada07640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03a8147960 0x7f03a8071b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35718/0 (socket says 172.21.15.1:35718) 2024-05-09T06:28:08.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ad206640 1 -- 172.21.15.1:0/2049094837 learned_addr learned my addr 172.21.15.1:0/2049094837 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:08.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ada07640 1 -- 172.21.15.1:0/2049094837 >> v1:172.21.15.6:6789/0 conn(0x7f03a80ae8a0 legacy=0x7f03a806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:08.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ada07640 1 -- 172.21.15.1:0/2049094837 >> v1:172.21.15.1:6789/0 conn(0x7f03a80720a0 legacy=0x7f03a806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03ada07640 1 -- 172.21.15.1:0/2049094837 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a8072b00 con 0x7f03a8147960 2024-05-09T06:28:08.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.241+0000 7f03aca05640 1 -- 172.21.15.1:0/2049094837 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592461573 0 0) 0x7f03a8147d40 con 0x7f03a80720a0 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.242+0000 7f03ada07640 1 --2- 172.21.15.1:0/2049094837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a8147960 0x7f03a8071b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03a40097d0 tx=0x7f03a402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae54fd9e625c2b54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.242+0000 7f03aca05640 1 -- 172.21.15.1:0/2049094837 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4037070 con 0x7f03a8147960 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.242+0000 7f03aca05640 1 -- 172.21.15.1:0/2049094837 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a4032440 con 0x7f03a8147960 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.242+0000 7f03aca05640 1 -- 172.21.15.1:0/2049094837 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4032810 con 0x7f03a8147960 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 -- 172.21.15.1:0/2049094837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a8147960 msgr2=0x7f03a8071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 --2- 172.21.15.1:0/2049094837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a8147960 0x7f03a8071b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03a40097d0 tx=0x7f03a402a210 comp rx=0 tx=0).stop 2024-05-09T06:28:08.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 -- 172.21.15.1:0/2049094837 shutdown_connections 2024-05-09T06:28:08.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 --2- 172.21.15.1:0/2049094837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a8147960 0x7f03a8071b60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 -- 172.21.15.1:0/2049094837 >> 172.21.15.1:0/2049094837 conn(0x7f03a80b1ba0 msgr2=0x7f03a80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 -- 172.21.15.1:0/2049094837 shutdown_connections 2024-05-09T06:28:08.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.243+0000 7f03afc6b640 1 -- 172.21.15.1:0/2049094837 wait complete. 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.244+0000 7f03afc6b640 1 Processor -- start 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.244+0000 7f03afc6b640 1 -- start start 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03a80720a0 0x7f03a81424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03a8147960 0x7f03a815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8072530 con 0x7f03a80ae8a0 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03a80730c0 con 0x7f03a8147960 2024-05-09T06:28:08.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03afc6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f03a80aed30 con 0x7f03a80720a0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35730/0 (socket says 172.21.15.1:35730) 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 -- 172.21.15.1:0/1663566532 learned_addr learned my addr 172.21.15.1:0/1663566532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ae208640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03a8147960 0x7f03a815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ada07640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03a80720a0 0x7f03a81424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 -- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03a80720a0 msgr2=0x7f03a81424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03a80720a0 0x7f03a81424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 -- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03a8147960 msgr2=0x7f03a815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03a8147960 0x7f03a815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.245+0000 7f03ad206640 1 -- 172.21.15.1:0/1663566532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a813b900 con 0x7f03a80ae8a0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f03ad206640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f039800c4d0 tx=0x7f039800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398018070 con 0x7f03a80ae8a0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398007500 con 0x7f03a80ae8a0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398004a20 con 0x7f03a80ae8a0 2024-05-09T06:28:08.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a815fde0 con 0x7f03a80ae8a0 2024-05-09T06:28:08.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.246+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a81603a0 con 0x7f03a80ae8a0 2024-05-09T06:28:08.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.247+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a80af3c0 con 0x7f03a80ae8a0 2024-05-09T06:28:08.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.247+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f039801b070 con 0x7f03a80ae8a0 2024-05-09T06:28:08.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.248+0000 7f039e7fc640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 0x7f038c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.248+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f039808a500 con 0x7f03a80ae8a0 2024-05-09T06:28:08.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.248+0000 7f03ada07640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 0x7f038c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.248+0000 7f03ada07640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 0x7f038c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.252+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03a80af3c0 con 0x7f03a80ae8a0 2024-05-09T06:28:08.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.438+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f03a806f500 con 0x7f03a80ae8a0 2024-05-09T06:28:08.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.439+0000 7f039e7fc640 1 -- 172.21.15.1:0/1663566532 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f03a813b900 con 0x7f03a80ae8a0 2024-05-09T06:28:08.421 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:08.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 msgr2=0x7f038c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 0x7f038c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:08.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 msgr2=0x7f03a813b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f039800c4d0 tx=0x7f039800c990 comp rx=0 tx=0).stop 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 shutdown_connections 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f038c06be30 0x7f038c06e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03a8147960 0x7f03a815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03a80ae8a0 0x7f03a813b3c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 --2- 172.21.15.1:0/1663566532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03a80720a0 0x7f03a81424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 >> 172.21.15.1:0/1663566532 conn(0x7f03a80b1ba0 msgr2=0x7f03a80b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 shutdown_connections 2024-05-09T06:28:08.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.441+0000 7f03afc6b640 1 -- 172.21.15.1:0/1663566532 wait complete. 2024-05-09T06:28:08.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:08.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.556+0000 7f02d6add640 1 Processor -- start 2024-05-09T06:28:08.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.556+0000 7f02d6add640 1 -- start start 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d6add640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f02d00cf180 0x7f02d00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d6add640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d00ce0e0 con 0x7f02d006e6d0 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d6add640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d00ce3c0 con 0x7f02d00cdab0 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d6add640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f02d00cdec0 con 0x7f02d00cf180 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02cffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f02d00cf180 0x7f02d00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d4879640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f02d006e6d0 0x7f02d006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60642/0 (socket says 172.21.15.1:60642) 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02d4879640 1 -- 172.21.15.1:0/3946955811 learned_addr learned my addr 172.21.15.1:0/3946955811 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:08.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02cffff640 1 -- 172.21.15.1:0/3946955811 >> v1:172.21.15.6:6789/0 conn(0x7f02d00cdab0 legacy=0x7f02d00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:08.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02cffff640 1 -- 172.21.15.1:0/3946955811 >> v1:172.21.15.1:6789/0 conn(0x7f02d006e6d0 legacy=0x7f02d006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02cffff640 1 -- 172.21.15.1:0/3946955811 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d00ce6a0 con 0x7f02d00cf180 2024-05-09T06:28:08.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.557+0000 7f02cf7fe640 1 -- 172.21.15.1:0/3946955811 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068191155 0 0) 0x7f02d00ce0e0 con 0x7f02d006e6d0 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02cffff640 1 --2- 172.21.15.1:0/3946955811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cf180 0x7f02d00cd570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0097a0 tx=0x7f02bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2cb644323382de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02cf7fe640 1 -- 172.21.15.1:0/3946955811 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc037070 con 0x7f02d00cf180 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02cf7fe640 1 -- 172.21.15.1:0/3946955811 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02bc032440 con 0x7f02d00cf180 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02cf7fe640 1 -- 172.21.15.1:0/3946955811 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc032810 con 0x7f02d00cf180 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02d6add640 1 -- 172.21.15.1:0/3946955811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cf180 msgr2=0x7f02d00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.558+0000 7f02d6add640 1 --2- 172.21.15.1:0/3946955811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cf180 0x7f02d00cd570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0097a0 tx=0x7f02bc02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.559+0000 7f02d6add640 1 -- 172.21.15.1:0/3946955811 shutdown_connections 2024-05-09T06:28:08.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.559+0000 7f02d6add640 1 --2- 172.21.15.1:0/3946955811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cf180 0x7f02d00cd570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.559+0000 7f02d6add640 1 -- 172.21.15.1:0/3946955811 >> 172.21.15.1:0/3946955811 conn(0x7f02d00cc4b0 msgr2=0x7f02d00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.559+0000 7f02d6add640 1 -- 172.21.15.1:0/3946955811 shutdown_connections 2024-05-09T06:28:08.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.559+0000 7f02d6add640 1 -- 172.21.15.1:0/3946955811 wait complete. 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.560+0000 7f02d6add640 1 Processor -- start 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.560+0000 7f02d6add640 1 -- start start 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02d006e6d0 0x7f02d00b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02d00cf180 0x7f02d013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0070ed0 con 0x7f02d00cdab0 2024-05-09T06:28:08.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00cec60 con 0x7f02d00cf180 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d6add640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f02d01447c0 con 0x7f02d006e6d0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35746/0 (socket says 172.21.15.1:35746) 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 -- 172.21.15.1:0/2133305340 learned_addr learned my addr 172.21.15.1:0/2133305340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d4879640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02d006e6d0 0x7f02d00b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02d507a640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02d00cf180 0x7f02d013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 -- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02d006e6d0 msgr2=0x7f02d00b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02d006e6d0 0x7f02d00b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 -- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02d00cf180 msgr2=0x7f02d013cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02d00cf180 0x7f02d013cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 -- 172.21.15.1:0/2133305340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d0142520 con 0x7f02d00cdab0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.561+0000 7f02cffff640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098c0 tx=0x7f02bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.562+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc041070 con 0x7f02d00cdab0 2024-05-09T06:28:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.562+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02bc03c870 con 0x7f02d00cdab0 2024-05-09T06:28:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.562+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc037040 con 0x7f02d00cdab0 2024-05-09T06:28:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.562+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d0142800 con 0x7f02d00cdab0 2024-05-09T06:28:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.562+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d0142dc0 con 0x7f02d00cdab0 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.563+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d00752b0 con 0x7f02d00cdab0 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.563+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02bc044070 con 0x7f02d00cdab0 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.564+0000 7f02cd7fa640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 0x7f02b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.564+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f02bc0abd90 con 0x7f02d00cdab0 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.564+0000 7f02d4879640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 0x7f02b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.564+0000 7f02d4879640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 0x7f02b406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02c4000f60 tx=0x7f02c4007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.568+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02bc07a4f0 con 0x7f02d00cdab0 2024-05-09T06:28:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.753+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02d006cf50 con 0x7f02d00cdab0 2024-05-09T06:28:08.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.754+0000 7f02cd7fa640 1 -- 172.21.15.1:0/2133305340 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f02bc079df0 con 0x7f02d00cdab0 2024-05-09T06:28:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:08.737 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 msgr2=0x7f02b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 0x7f02b406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02c4000f60 tx=0x7f02c4007c00 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 msgr2=0x7f02d0141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098c0 tx=0x7f02bc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 shutdown_connections 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f02b406be30 0x7f02b406e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02d00cf180 0x7f02d013cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02d00cdab0 0x7f02d0141db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 --2- 172.21.15.1:0/2133305340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02d006e6d0 0x7f02d00b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 >> 172.21.15.1:0/2133305340 conn(0x7f02d00cc4b0 msgr2=0x7f02d00b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 shutdown_connections 2024-05-09T06:28:08.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.757+0000 7f02d6add640 1 -- 172.21.15.1:0/2133305340 wait complete. 2024-05-09T06:28:08.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:08.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:08.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 Processor -- start 2024-05-09T06:28:08.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 -- start start 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcd0c06b1e0 0x7fcd0c0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0c072050 con 0x7fcd0c0d06f0 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0c072330 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.876+0000 7fcd12a3f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcd0c072610 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd10fdc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcd0c06b1e0 0x7fcd0c0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd0bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcd0c0d06f0 0x7fcd0c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60654/0 (socket says 172.21.15.1:60654) 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd0bfff640 1 -- 172.21.15.1:0/3588874590 learned_addr learned my addr 172.21.15.1:0/3588874590 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd10fdc640 1 -- 172.21.15.1:0/3588874590 >> v1:172.21.15.6:6789/0 conn(0x7fcd0c0c41e0 legacy=0x7fcd0c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd10fdc640 1 -- 172.21.15.1:0/3588874590 >> v1:172.21.15.1:6789/0 conn(0x7fcd0c0d06f0 legacy=0x7fcd0c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.877+0000 7fcd10fdc640 1 -- 172.21.15.1:0/3588874590 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c0728f0 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd10fdc640 1 --2- 172.21.15.1:0/3588874590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0cefe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009d90 tx=0x7fcd0402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7667abb146d744c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd0affd640 1 -- 172.21.15.1:0/3588874590 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007740 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd0affd640 1 -- 172.21.15.1:0/3588874590 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd040078e0 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd0affd640 1 -- 172.21.15.1:0/3588874590 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007cb0 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd12a3f640 1 -- 172.21.15.1:0/3588874590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c06b1e0 msgr2=0x7fcd0c0cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/3588874590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0cefe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009d90 tx=0x7fcd0402a5b0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd12a3f640 1 -- 172.21.15.1:0/3588874590 shutdown_connections 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/3588874590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0cefe0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.878+0000 7fcd12a3f640 1 -- 172.21.15.1:0/3588874590 >> 172.21.15.1:0/3588874590 conn(0x7fcd0c0d0b50 msgr2=0x7fcd0c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.879+0000 7fcd12a3f640 1 -- 172.21.15.1:0/3588874590 shutdown_connections 2024-05-09T06:28:08.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.879+0000 7fcd12a3f640 1 -- 172.21.15.1:0/3588874590 wait complete. 2024-05-09T06:28:08.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.880+0000 7fcd12a3f640 1 Processor -- start 2024-05-09T06:28:08.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.880+0000 7fcd12a3f640 1 -- start start 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 0x7fcd0c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0ccf70 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0ccdf0 con 0x7fcd0c06b1e0 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c0cd0f0 con 0x7fcd0c0d06f0 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35766/0 (socket says 172.21.15.1:35766) 2024-05-09T06:28:08.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 -- 172.21.15.1:0/1299830024 learned_addr learned my addr 172.21.15.1:0/1299830024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0bfff640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd10fdc640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 0x7fcd0c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 -- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 msgr2=0x7fcd0c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 0x7fcd0c15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 -- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 msgr2=0x7fcd0c0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 -- 172.21.15.1:0/1299830024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c0b4e50 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0bfff640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd10fdc640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 0x7fcd0c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd0b7fe640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c460 tx=0x7fcd0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00016020 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd00012870 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c0b3db0 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.881+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00012c60 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.882+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c0b4370 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.883+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0c0752b0 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.883+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd00019020 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.884+0000 7fcd08ff9640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.884+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcd0008ac80 con 0x7fcd0c0c41e0 2024-05-09T06:28:08.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.884+0000 7fcd0bfff640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.884+0000 7fcd0bfff640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 0x7fccf806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f60 tx=0x7fccfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:08.888+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd00020070 con 0x7fcd0c0c41e0 2024-05-09T06:28:09.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.073+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd0c072f30 con 0x7fcd0c0c41e0 2024-05-09T06:28:09.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.073+0000 7fcd08ff9640 1 -- 172.21.15.1:0/1299830024 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fcd000592c0 con 0x7fcd0c0c41e0 2024-05-09T06:28:09.055 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:09.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 msgr2=0x7fccf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 0x7fccf806e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f60 tx=0x7fccfc009040 comp rx=0 tx=0).stop 2024-05-09T06:28:09.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 msgr2=0x7fcd0c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c460 tx=0x7fcd0000c920 comp rx=0 tx=0).stop 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 shutdown_connections 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd0c0d06f0 0x7fcd0c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd0c0c41e0 0x7fcd0c0b3640 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 --2- 172.21.15.1:0/1299830024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c06b1e0 0x7fcd0c0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 >> 172.21.15.1:0/1299830024 conn(0x7fcd0c0d0b50 msgr2=0x7fcd0c15d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 shutdown_connections 2024-05-09T06:28:09.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.075+0000 7fcd12a3f640 1 -- 172.21.15.1:0/1299830024 wait complete. 2024-05-09T06:28:09.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.194+0000 7f9befb42640 1 Processor -- start 2024-05-09T06:28:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.194+0000 7f9befb42640 1 -- start start 2024-05-09T06:28:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9befb42640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9be815e390 0x7f9be80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9befb42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 0x7f9be80b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9befb42640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be815e900 con 0x7f9be80b1730 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9befb42640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9be80b1ba0 con 0x7f9be815e390 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9befb42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9be815ebe0 con 0x7f9be80cf990 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bee0df640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9be80b1730 0x7f9be8072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60658/0 (socket says 172.21.15.1:60658) 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bee0df640 1 -- 172.21.15.1:0/3360087260 learned_addr learned my addr 172.21.15.1:0/3360087260 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3360087260 >> v2:172.21.15.6:3300/0 conn(0x7f9be815e390 0x7f9be80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bed0dd640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 0x7f9be80b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873769179 0 0) 0x7f9be815e900 con 0x7f9be80b1730 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.195+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd4003520 con 0x7f9be80b1730 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2246464508 0 0) 0x7f9bd4003520 con 0x7f9be80b1730 2024-05-09T06:28:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed8de640 1 -- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 msgr2=0x7f9be80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 0x7f9be80b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed8de640 1 -- 172.21.15.1:0/3360087260 >> v1:172.21.15.1:6789/0 conn(0x7f9be80b1730 legacy=0x7f9be8072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed8de640 1 -- 172.21.15.1:0/3360087260 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be815eec0 con 0x7f9be815e390 2024-05-09T06:28:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed0dd640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 0x7f9be80b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.196+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be815e390 0x7f9be80cf450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db786d7890d7bff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4037070 con 0x7f9be815e390 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be402aba0 con 0x7f9be815e390 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9bec8dc640 1 -- 172.21.15.1:0/3360087260 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be00019f0 con 0x7f9be815e390 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 -- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be815e390 msgr2=0x7f9be80cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be815e390 0x7f9be80cf450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a110 comp rx=0 tx=0).stop 2024-05-09T06:28:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 -- 172.21.15.1:0/3360087260 shutdown_connections 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80cf990 0x7f9be80b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 --2- 172.21.15.1:0/3360087260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be815e390 0x7f9be80cf450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 -- 172.21.15.1:0/3360087260 >> 172.21.15.1:0/3360087260 conn(0x7f9be80b1da0 msgr2=0x7f9be80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 -- 172.21.15.1:0/3360087260 shutdown_connections 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.197+0000 7f9befb42640 1 -- 172.21.15.1:0/3360087260 wait complete. 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.198+0000 7f9befb42640 1 Processor -- start 2024-05-09T06:28:09.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.198+0000 7f9befb42640 1 -- start start 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80b1730 0x7f9be81616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 0x7f9be81f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9be80cd8f0 con 0x7f9be815e390 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9be815f6c0 con 0x7f9be80cf990 2024-05-09T06:28:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9befb42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9be8162ba0 con 0x7f9be80b1730 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9bee0df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9bee0df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35772/0 (socket says 172.21.15.1:35772) 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9bee0df640 1 -- 172.21.15.1:0/3497096629 learned_addr learned my addr 172.21.15.1:0/3497096629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9bed0dd640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 0x7f9be81f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.199+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80b1730 0x7f9be81616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 -- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80b1730 msgr2=0x7f9be81616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80b1730 0x7f9be81616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 -- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 msgr2=0x7f9be81f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 0x7f9be81f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 -- 172.21.15.1:0/3497096629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be80d1130 con 0x7f9be815e390 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bed0dd640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 0x7f9be81f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bee0df640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9be000cca0 tx=0x7f9be000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0017070 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0012400 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be80c4d30 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be00127f0 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.200+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be80c52f0 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.201+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be80c5ac0 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.201+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be001a070 con 0x7f9be815e390 2024-05-09T06:28:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.202+0000 7f9bde7fc640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 0x7f9bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.202+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9be008b230 con 0x7f9be815e390 2024-05-09T06:28:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.202+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 0x7f9bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.202+0000 7f9bed8de640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 0x7f9bcc06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.206+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9be00597e0 con 0x7f9be815e390 2024-05-09T06:28:09.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.389+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9be80b2ae0 con 0x7f9be815e390 2024-05-09T06:28:09.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.389+0000 7f9bde7fc640 1 -- 172.21.15.1:0/3497096629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f9be00590e0 con 0x7f9be815e390 2024-05-09T06:28:09.371 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.391+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 msgr2=0x7f9bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.391+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 0x7f9bcc06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.391+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 msgr2=0x7f9be80b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9be000cca0 tx=0x7f9be000a430 comp rx=0 tx=0).stop 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 shutdown_connections 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bcc06be30 0x7f9bcc06e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be815e390 0x7f9be80b30c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cf990 0x7f9be81f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 --2- 172.21.15.1:0/3497096629 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be80b1730 0x7f9be81616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 >> 172.21.15.1:0/3497096629 conn(0x7f9be80b1da0 msgr2=0x7f9be80cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 shutdown_connections 2024-05-09T06:28:09.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.392+0000 7f9befb42640 1 -- 172.21.15.1:0/3497096629 wait complete. 2024-05-09T06:28:09.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:09.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.509+0000 7f9e0608c640 1 Processor -- start 2024-05-09T06:28:09.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.509+0000 7f9e0608c640 1 -- start start 2024-05-09T06:28:09.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9e0608c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9e000d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9e0608c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9e0006ffb0 0x7f9e000d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9e0608c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0006c420 con 0x7f9e000ae9c0 2024-05-09T06:28:09.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9e0608c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0006f3f0 con 0x7f9e0006ffb0 2024-05-09T06:28:09.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9e0608c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0006c700 con 0x7f9e0015e300 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9dfeffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9e000ae9c0 0x7f9e0006f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60660/0 (socket says 172.21.15.1:60660) 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.510+0000 7f9dfeffd640 1 -- 172.21.15.1:0/3647157931 learned_addr learned my addr 172.21.15.1:0/3647157931 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfffff640 1 --2- 172.21.15.1:0/3647157931 >> v2:172.21.15.6:3300/0 conn(0x7f9e0006ffb0 0x7f9e000d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211602275 0 0) 0x7f9e0006c420 con 0x7f9e000ae9c0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9de8003520 con 0x7f9e000ae9c0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9e000d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2110494473 0 0) 0x7f9de8003520 con 0x7f9e000ae9c0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 msgr2=0x7f9e000d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9e000d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 >> v2:172.21.15.6:3300/0 conn(0x7f9e0006ffb0 msgr2=0x7f9e000d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/3647157931 >> v2:172.21.15.6:3300/0 conn(0x7f9e0006ffb0 0x7f9e000d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0006c9e0 con 0x7f9e000ae9c0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f9df4001670 con 0x7f9e000ae9c0 2024-05-09T06:28:09.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 >> v1:172.21.15.1:6789/0 conn(0x7f9e000ae9c0 legacy=0x7f9e0006f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80039b0 0x7f9de8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de8004800 0x7f9de8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0006c700 con 0x7f9de80039b0 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9de800b9c0 con 0x7f9de8004800 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9de800bbc0 con 0x7f9e0015e300 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80039b0 0x7f9de8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de8004800 0x7f9de8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dfffff640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 -- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 msgr2=0x7f9de80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e0015e300 0x7f9de80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 -- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de8004800 msgr2=0x7f9de8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de8004800 0x7f9de8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.511+0000 7f9dff7fe640 1 -- 172.21.15.1:0/3647157931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de8003520 con 0x7f9de80039b0 2024-05-09T06:28:09.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.512+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80039b0 0x7f9de8003dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009ac0 tx=0x7f9de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37137fcae6088d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.512+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0036070 con 0x7f9de80039b0 2024-05-09T06:28:09.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.512+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df0031400 con 0x7f9de80039b0 2024-05-09T06:28:09.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.512+0000 7f9dfe7fc640 1 -- 172.21.15.1:0/3647157931 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0009690 con 0x7f9de80039b0 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 -- 172.21.15.1:0/3647157931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80039b0 msgr2=0x7f9de8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 --2- 172.21.15.1:0/3647157931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80039b0 0x7f9de8003dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009ac0 tx=0x7f9de8005830 comp rx=0 tx=0).stop 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9dfffff640 1 -- 172.21.15.1:0/3647157931 reap_dead start 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 -- 172.21.15.1:0/3647157931 shutdown_connections 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 -- 172.21.15.1:0/3647157931 >> 172.21.15.1:0/3647157931 conn(0x7f9e000d1c50 msgr2=0x7f9e000d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 -- 172.21.15.1:0/3647157931 shutdown_connections 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.513+0000 7f9e0608c640 1 -- 172.21.15.1:0/3647157931 wait complete. 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.514+0000 7f9e0608c640 1 Processor -- start 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.514+0000 7f9e0608c640 1 -- start start 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 0x7f9e0014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 0x7f9e000c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e000d0930 con 0x7f9e000b5e20 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0006cfa0 con 0x7f9e000b6280 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9e0608c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9e000d0ab0 con 0x7f9de80039b0 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35786/0 (socket says 172.21.15.1:35786) 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 -- 172.21.15.1:0/2706234228 learned_addr learned my addr 172.21.15.1:0/2706234228 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 0x7f9e0014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfffff640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 0x7f9e000c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 -- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 msgr2=0x7f9e0014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 0x7f9e0014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 -- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 msgr2=0x7f9e000c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 0x7f9e000c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfeffd640 1 -- 172.21.15.1:0/2706234228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e000b4aa0 con 0x7f9e000b5e20 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dfffff640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 0x7f9e000c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.515+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 0x7f9e0014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9dfeffd640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007310 tx=0x7f9df40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df400f940 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df4018070 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e000b4d80 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df400fd40 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.516+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e000b5340 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.517+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e000752b0 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.517+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9df401d460 con 0x7f9e000b5e20 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.518+0000 7f9de7fff640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 0x7f9dec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.518+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 0x7f9dec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.518+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9df408b410 con 0x7f9e000b5e20 2024-05-09T06:28:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.518+0000 7f9dff7fe640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 0x7f9dec06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9df0033000 tx=0x7f9df0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.522+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9df4059890 con 0x7f9e000b5e20 2024-05-09T06:28:09.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.706+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e000d1090 con 0x7f9e000b5e20 2024-05-09T06:28:09.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.707+0000 7f9de7fff640 1 -- 172.21.15.1:0/2706234228 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f9df4059190 con 0x7f9e000b5e20 2024-05-09T06:28:09.689 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 msgr2=0x7f9dec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 0x7f9dec06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9df0033000 tx=0x7f9df0008030 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 msgr2=0x7f9e000c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007310 tx=0x7f9df40077d0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 shutdown_connections 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dec06be60 0x7f9dec06e320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e000b6280 0x7f9e000c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e000b5e20 0x7f9e000c6840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 --2- 172.21.15.1:0/2706234228 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80039b0 0x7f9e0014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 >> 172.21.15.1:0/2706234228 conn(0x7f9e000d1c50 msgr2=0x7f9e0015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 shutdown_connections 2024-05-09T06:28:09.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.709+0000 7f9e0608c640 1 -- 172.21.15.1:0/2706234228 wait complete. 2024-05-09T06:28:09.702 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.827+0000 7fa3cab9e640 1 Processor -- start 2024-05-09T06:28:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.827+0000 7fa3cab9e640 1 -- start start 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3cab9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 0x7fa3c406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3cab9e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 0x7fa3c415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3cab9e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c406cf40 con 0x7fa3c40b5260 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3cab9e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c4156980 con 0x7fa3c40b4de0 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3cab9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c406d870 con 0x7fa3c415c4c0 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c913b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa3c40b5260 0x7fa3c4070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60676/0 (socket says 172.21.15.1:60676) 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c913b640 1 -- 172.21.15.1:0/2674012708 learned_addr learned my addr 172.21.15.1:0/2674012708 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c3fff640 1 --2- 172.21.15.1:0/2674012708 >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 0x7fa3c415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c893a640 1 --2- 172.21.15.1:0/2674012708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 0x7fa3c406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344358923 0 0) 0x7fa3c406cf40 con 0x7fa3c40b5260 2024-05-09T06:28:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.828+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3b4003520 con 0x7fa3c40b5260 2024-05-09T06:28:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1827753932 0 0) 0x7fa3b4003520 con 0x7fa3c40b5260 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c893a640 1 -- 172.21.15.1:0/2674012708 >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 msgr2=0x7fa3c415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c893a640 1 --2- 172.21.15.1:0/2674012708 >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 0x7fa3c415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c893a640 1 -- 172.21.15.1:0/2674012708 >> v1:172.21.15.1:6789/0 conn(0x7fa3c40b5260 legacy=0x7fa3c4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c893a640 1 -- 172.21.15.1:0/2674012708 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c406d220 con 0x7fa3c415c4c0 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c3fff640 1 --2- 172.21.15.1:0/2674012708 >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 0x7fa3c415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.829+0000 7fa3c893a640 1 --2- 172.21.15.1:0/2674012708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 0x7fa3c406dd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80097a0 tx=0x7fa3b802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b428edea85968b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8037070 con 0x7fa3c415c4c0 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b802aba0 con 0x7fa3c415c4c0 2024-05-09T06:28:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3c37fe640 1 -- 172.21.15.1:0/2674012708 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0019f0 con 0x7fa3c415c4c0 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 -- 172.21.15.1:0/2674012708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 msgr2=0x7fa3c406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/2674012708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 0x7fa3c406dd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80097a0 tx=0x7fa3b802a110 comp rx=0 tx=0).stop 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 -- 172.21.15.1:0/2674012708 shutdown_connections 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/2674012708 >> v2:172.21.15.6:3300/0 conn(0x7fa3c40b4de0 0x7fa3c415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/2674012708 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c415c4c0 0x7fa3c406dd00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 -- 172.21.15.1:0/2674012708 >> 172.21.15.1:0/2674012708 conn(0x7fa3c40cc4d0 msgr2=0x7fa3c40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:09.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 -- 172.21.15.1:0/2674012708 shutdown_connections 2024-05-09T06:28:09.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.830+0000 7fa3cab9e640 1 -- 172.21.15.1:0/2674012708 wait complete. 2024-05-09T06:28:09.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.831+0000 7fa3cab9e640 1 Processor -- start 2024-05-09T06:28:09.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.831+0000 7fa3cab9e640 1 -- start start 2024-05-09T06:28:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c40b5260 0x7fa3c40ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c415c4c0 0x7fa3c40abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40b2dc0 con 0x7fa3c40b4de0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40b2c40 con 0x7fa3c415c4c0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3cab9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c415cde0 con 0x7fa3c40b5260 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3c893a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3c893a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35792/0 (socket says 172.21.15.1:35792) 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.832+0000 7fa3c893a640 1 -- 172.21.15.1:0/1058099671 learned_addr learned my addr 172.21.15.1:0/1058099671 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c913b640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c415c4c0 0x7fa3c40abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c3fff640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c40b5260 0x7fa3c40ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 -- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c40b5260 msgr2=0x7fa3c40ab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c40b5260 0x7fa3c40ab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 -- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c415c4c0 msgr2=0x7fa3c40abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c415c4c0 0x7fa3c40abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 -- 172.21.15.1:0/1058099671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c40aa920 con 0x7fa3c40b4de0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c893a640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80098c0 tx=0x7fa3b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8037050 con 0x7fa3c40b4de0 2024-05-09T06:28:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b8043070 con 0x7fa3c40b4de0 2024-05-09T06:28:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c40aac00 con 0x7fa3c40b4de0 2024-05-09T06:28:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8006e10 con 0x7fa3c40b4de0 2024-05-09T06:28:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.833+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c4159fd0 con 0x7fa3c40b4de0 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.834+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c40b2f70 con 0x7fa3c40b4de0 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.835+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3b803ca10 con 0x7fa3c40b4de0 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.835+0000 7fa3c17fa640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 0x7fa3a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.835+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa3b80abc60 con 0x7fa3c40b4de0 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.835+0000 7fa3c3fff640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 0x7fa3a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:09.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.836+0000 7fa3c3fff640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 0x7fa3a806e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009560 tx=0x7fa3b0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:09.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:09.840+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3c40b2f70 con 0x7fa3c40b4de0 2024-05-09T06:28:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.020+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa3c40752b0 con 0x7fa3c40b4de0 2024-05-09T06:28:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.020+0000 7fa3c17fa640 1 -- 172.21.15.1:0/1058099671 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fa3b80ab070 con 0x7fa3c40b4de0 2024-05-09T06:28:10.002 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:10.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 msgr2=0x7fa3a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 0x7fa3a806e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009560 tx=0x7fa3b0009190 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 msgr2=0x7fa3c413b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80098c0 tx=0x7fa3b80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 shutdown_connections 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3a806be30 0x7fa3a806e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3c415c4c0 0x7fa3c40abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3c40b5260 0x7fa3c40ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 --2- 172.21.15.1:0/1058099671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3c40b4de0 0x7fa3c413b200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 >> 172.21.15.1:0/1058099671 conn(0x7fa3c40cc4d0 msgr2=0x7fa3c40b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 shutdown_connections 2024-05-09T06:28:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.022+0000 7fa3cab9e640 1 -- 172.21.15.1:0/1058099671 wait complete. 2024-05-09T06:28:10.069 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:28:10.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.192+0000 7f6bdf129640 1 Processor -- start 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.192+0000 7f6bdf129640 1 -- start start 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bd815c320 0x7f6bd80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 0x7f6bd80d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 0x7f6bd80ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd80d2c80 con 0x7f6bd815c320 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd80d29a0 con 0x7f6bd80b0d00 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdf129640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd80cd430 con 0x7f6bd80d05e0 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdcec5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bd815c320 0x7f6bd80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdcec5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bd815c320 0x7f6bd80b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35806/0 (socket says 172.21.15.1:35806) 2024-05-09T06:28:10.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.193+0000 7f6bdcec5640 1 -- 172.21.15.1:0/129298822 learned_addr learned my addr 172.21.15.1:0/129298822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/129298822 >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 0x7f6bd80d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdd6c6640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 0x7f6bd80ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 -- 172.21.15.1:0/129298822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 msgr2=0x7f6bd80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 0x7f6bd80ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 -- 172.21.15.1:0/129298822 >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 msgr2=0x7f6bd80d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/129298822 >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 0x7f6bd80d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 -- 172.21.15.1:0/129298822 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd80cd710 con 0x7f6bd815c320 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/129298822 >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 0x7f6bd80d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:10.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdd6c6640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 0x7f6bd80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:10.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd815c320 0x7f6bd80b1250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806c1c0 tx=0x7f6bc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb5593dcd1a4cea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.194+0000 7f6bd77fe640 1 -- 172.21.15.1:0/129298822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8037070 con 0x7f6bd815c320 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bd77fe640 1 -- 172.21.15.1:0/129298822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc8032440 con 0x7f6bd815c320 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bd77fe640 1 -- 172.21.15.1:0/129298822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8032810 con 0x7f6bd815c320 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 -- 172.21.15.1:0/129298822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd815c320 msgr2=0x7f6bd80b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd815c320 0x7f6bd80b1250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806c1c0 tx=0x7f6bc802a270 comp rx=0 tx=0).stop 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 -- 172.21.15.1:0/129298822 shutdown_connections 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd80d05e0 0x7f6bd80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 --2- 172.21.15.1:0/129298822 >> v2:172.21.15.6:3300/0 conn(0x7f6bd80b0d00 0x7f6bd80d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 --2- 172.21.15.1:0/129298822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd815c320 0x7f6bd80b1250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 -- 172.21.15.1:0/129298822 >> 172.21.15.1:0/129298822 conn(0x7f6bd80d1d70 msgr2=0x7f6bd80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 -- 172.21.15.1:0/129298822 shutdown_connections 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.195+0000 7f6bdf129640 1 -- 172.21.15.1:0/129298822 wait complete. 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.196+0000 7f6bdf129640 1 Processor -- start 2024-05-09T06:28:10.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.196+0000 7f6bdf129640 1 -- start start 2024-05-09T06:28:10.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 0x7f6bd806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd815c320 0x7f6bd806c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80ce8e0 con 0x7f6bd80b0d00 2024-05-09T06:28:10.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80cfde0 con 0x7f6bd80d05e0 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdf129640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd80cea60 con 0x7f6bd815c320 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdcec5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdcec5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35818/0 (socket says 172.21.15.1:35818) 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdcec5640 1 -- 172.21.15.1:0/3270867395 learned_addr learned my addr 172.21.15.1:0/3270867395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 0x7f6bd806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.197+0000 7f6bdd6c6640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd815c320 0x7f6bd806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 -- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd815c320 msgr2=0x7f6bd806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd815c320 0x7f6bd806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 -- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 msgr2=0x7f6bd806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 0x7f6bd806c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 -- 172.21.15.1:0/3270867395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd806ce60 con 0x7f6bd80b0d00 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 0x7f6bd806c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdcec5640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8000c00 tx=0x7f6bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8037040 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc8042070 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd806d140 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8030bb0 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.198+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd806d700 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.199+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd80752b0 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.200+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bc8047460 con 0x7f6bd80b0d00 2024-05-09T06:28:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.200+0000 7f6bd57fa640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 0x7f6bc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.200+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6bc80abe50 con 0x7f6bd80b0d00 2024-05-09T06:28:10.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.200+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 0x7f6bc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.201+0000 7f6bd7fff640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 0x7f6bc406e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80b05b0 tx=0x7f6bcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.204+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6bc8045360 con 0x7f6bd80b0d00 2024-05-09T06:28:10.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.387+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bd80b2110 con 0x7f6bd80b0d00 2024-05-09T06:28:10.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.389+0000 7f6bd57fa640 1 -- 172.21.15.1:0/3270867395 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f6bc807a400 con 0x7f6bd80b0d00 2024-05-09T06:28:10.371 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:10.371 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 msgr2=0x7f6bc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 0x7f6bc406e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80b05b0 tx=0x7f6bcc008040 comp rx=0 tx=0).stop 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 msgr2=0x7f6bd814b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8000c00 tx=0x7f6bc80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 shutdown_connections 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bc406be30 0x7f6bc406e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bd815c320 0x7f6bd806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bd80d05e0 0x7f6bd806c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 --2- 172.21.15.1:0/3270867395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bd80b0d00 0x7f6bd814b6f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 >> 172.21.15.1:0/3270867395 conn(0x7f6bd80d1d70 msgr2=0x7f6bd80b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 shutdown_connections 2024-05-09T06:28:10.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.391+0000 7f6bdf129640 1 -- 172.21.15.1:0/3270867395 wait complete. 2024-05-09T06:28:10.384 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:10.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:10.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.510+0000 7f85bb0cd640 1 Processor -- start 2024-05-09T06:28:10.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.510+0000 7f85bb0cd640 1 -- start start 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.510+0000 7f85bb0cd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 0x7f85b40cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.510+0000 7f85bb0cd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f85b40d08b0 0x7f85b40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85bb0cd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b40b1340 con 0x7f85b40ce9d0 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85bb0cd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40cc260 con 0x7f85b40d08b0 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85bb0cd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40d1020 con 0x7f85b415e270 2024-05-09T06:28:10.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f85b40d08b0 0x7f85b40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b966a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f85b40ce9d0 0x7f85b406c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60688/0 (socket says 172.21.15.1:60688) 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b966a640 1 -- 172.21.15.1:0/2443092634 learned_addr learned my addr 172.21.15.1:0/2443092634 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b8e69640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 0x7f85b40cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b3fff640 1 -- 172.21.15.1:0/2443092634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 msgr2=0x7f85b40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b3fff640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 0x7f85b40cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b3fff640 1 -- 172.21.15.1:0/2443092634 >> v1:172.21.15.1:6789/0 conn(0x7f85b40ce9d0 legacy=0x7f85b406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b3fff640 1 -- 172.21.15.1:0/2443092634 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40cc4b0 con 0x7f85b40d08b0 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b37fe640 1 -- 172.21.15.1:0/2443092634 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285385457 0 0) 0x7f85ac001490 con 0x7f85b40ce9d0 2024-05-09T06:28:10.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.511+0000 7f85b8e69640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 0x7f85b40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85b3fff640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40ce490 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b446bf8dd07bcf28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85b37fe640 1 -- 172.21.15.1:0/2443092634 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0037070 con 0x7f85b40d08b0 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85b37fe640 1 -- 172.21.15.1:0/2443092634 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a0032440 con 0x7f85b40d08b0 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85b37fe640 1 -- 172.21.15.1:0/2443092634 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0032810 con 0x7f85b40d08b0 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 -- 172.21.15.1:0/2443092634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 msgr2=0x7f85b40ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40ce490 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002a210 comp rx=0 tx=0).stop 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 -- 172.21.15.1:0/2443092634 shutdown_connections 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40ce490 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/2443092634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b415e270 0x7f85b40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.512+0000 7f85bb0cd640 1 -- 172.21.15.1:0/2443092634 >> 172.21.15.1:0/2443092634 conn(0x7f85b40d1c10 msgr2=0x7f85b40d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.513+0000 7f85bb0cd640 1 -- 172.21.15.1:0/2443092634 shutdown_connections 2024-05-09T06:28:10.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.513+0000 7f85bb0cd640 1 -- 172.21.15.1:0/2443092634 wait complete. 2024-05-09T06:28:10.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.514+0000 7f85bb0cd640 1 Processor -- start 2024-05-09T06:28:10.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.514+0000 7f85bb0cd640 1 -- start start 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 0x7f85b41534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 0x7f85b41581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85b415e650 con 0x7f85b40d08b0 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cca70 con 0x7f85b415e270 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40d0cf0 con 0x7f85b40ce9d0 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35840/0 (socket says 172.21.15.1:35840) 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 -- 172.21.15.1:0/761651772 learned_addr learned my addr 172.21.15.1:0/761651772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b8e69640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 0x7f85b41534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b966a640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 0x7f85b41581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 -- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 msgr2=0x7f85b41534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 0x7f85b41534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 -- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 msgr2=0x7f85b41581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 0x7f85b41581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 -- 172.21.15.1:0/761651772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4157ac0 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b966a640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 0x7f85b41581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b8e69640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 0x7f85b41534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b3fff640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f85a0035040 tx=0x7f85a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0037040 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a0030770 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b4156e30 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0030b40 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.515+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b41573f0 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.516+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b40752b0 con 0x7f85b40d08b0 2024-05-09T06:28:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.517+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85a0044020 con 0x7f85b40d08b0 2024-05-09T06:28:10.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.517+0000 7f85b17fa640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 0x7f859806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.517+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f85a00ab7b0 con 0x7f85b40d08b0 2024-05-09T06:28:10.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.518+0000 7f85b8e69640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 0x7f859806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.518+0000 7f85b8e69640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 0x7f859806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c8520 tx=0x7f85a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.522+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f85a0032e20 con 0x7f85b40d08b0 2024-05-09T06:28:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.706+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85b40d29e0 con 0x7f85b40d08b0 2024-05-09T06:28:10.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.707+0000 7f85b17fa640 1 -- 172.21.15.1:0/761651772 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f85a0079d50 con 0x7f85b40d08b0 2024-05-09T06:28:10.689 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 msgr2=0x7f859806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 0x7f859806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f85b40c8520 tx=0x7f85a8008040 comp rx=0 tx=0).stop 2024-05-09T06:28:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 msgr2=0x7f85b40c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f85a0035040 tx=0x7f85a00046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 shutdown_connections 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f859806be30 0x7f859806e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b415e270 0x7f85b41581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85b40d08b0 0x7f85b40c8ce0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 --2- 172.21.15.1:0/761651772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b40ce9d0 0x7f85b41534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 >> 172.21.15.1:0/761651772 conn(0x7f85b40d1c10 msgr2=0x7f85b40af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 shutdown_connections 2024-05-09T06:28:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.709+0000 7f85bb0cd640 1 -- 172.21.15.1:0/761651772 wait complete. 2024-05-09T06:28:10.702 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:10.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.827+0000 7f0bac3f9640 1 Processor -- start 2024-05-09T06:28:10.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.827+0000 7f0bac3f9640 1 -- start start 2024-05-09T06:28:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0bac3f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ba415e390 0x7f0ba40cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0bac3f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0ba40b1900 0x7f0ba40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0bac3f9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba415e900 con 0x7f0ba40cf990 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0bac3f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba415ebe0 con 0x7f0ba415e390 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0bac3f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba415eec0 con 0x7f0ba40b1900 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ba415e390 0x7f0ba40cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ba415e390 0x7f0ba40cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35854/0 (socket says 172.21.15.1:35854) 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 -- 172.21.15.1:0/3015369249 learned_addr learned my addr 172.21.15.1:0/3015369249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 -- 172.21.15.1:0/3015369249 >> v2:172.21.15.6:3300/0 conn(0x7f0ba40b1900 msgr2=0x7f0ba40cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 --2- 172.21.15.1:0/3015369249 >> v2:172.21.15.6:3300/0 conn(0x7f0ba40b1900 0x7f0ba40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 -- 172.21.15.1:0/3015369249 >> v1:172.21.15.6:6789/0 conn(0x7f0ba40cf990 legacy=0x7f0ba40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:10.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.828+0000 7f0baa195640 1 -- 172.21.15.1:0/3015369249 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba415f1a0 con 0x7f0ba415e390 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.829+0000 7f0baa195640 1 --2- 172.21.15.1:0/3015369249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba40cf450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40721e0 tx=0x7f0ba002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9cd94c5e8aac3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.829+0000 7f0ba9193640 1 -- 172.21.15.1:0/3015369249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0037070 con 0x7f0ba415e390 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.829+0000 7f0ba9193640 1 -- 172.21.15.1:0/3015369249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0032440 con 0x7f0ba415e390 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.829+0000 7f0ba9193640 1 -- 172.21.15.1:0/3015369249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0032810 con 0x7f0ba415e390 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3015369249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 msgr2=0x7f0ba40cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3015369249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba40cf450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40721e0 tx=0x7f0ba002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3015369249 shutdown_connections 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3015369249 >> v2:172.21.15.6:3300/0 conn(0x7f0ba40b1900 0x7f0ba40cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3015369249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba40cf450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3015369249 >> 172.21.15.1:0/3015369249 conn(0x7f0ba40b1da0 msgr2=0x7f0ba40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3015369249 shutdown_connections 2024-05-09T06:28:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.830+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3015369249 wait complete. 2024-05-09T06:28:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.831+0000 7f0bac3f9640 1 Processor -- start 2024-05-09T06:28:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.831+0000 7f0bac3f9640 1 -- start start 2024-05-09T06:28:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ba40b1900 0x7f0ba40b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 0x7f0ba40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40cffb0 con 0x7f0ba415e390 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4162ba0 con 0x7f0ba40cf990 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba40cce20 con 0x7f0ba40b1900 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35868/0 (socket says 172.21.15.1:35868) 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 -- 172.21.15.1:0/3946978787 learned_addr learned my addr 172.21.15.1:0/3946978787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 -- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ba40b1900 msgr2=0x7f0ba40b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ba40b1900 0x7f0ba40b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 -- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 msgr2=0x7f0ba40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0ba9994640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 0x7f0ba40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 0x7f0ba40c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 -- 172.21.15.1:0/3946978787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40995c0 con 0x7f0ba415e390 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0ba9994640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 0x7f0ba40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0baa996640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00b970 tx=0x7f0b9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c019070 con 0x7f0ba415e390 2024-05-09T06:28:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c00cb60 con 0x7f0ba415e390 2024-05-09T06:28:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba40c6480 con 0x7f0ba415e390 2024-05-09T06:28:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c014440 con 0x7f0ba415e390 2024-05-09T06:28:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.832+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba40c6a40 con 0x7f0ba415e390 2024-05-09T06:28:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.833+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba40752b0 con 0x7f0ba415e390 2024-05-09T06:28:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.834+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b9c01c070 con 0x7f0ba415e390 2024-05-09T06:28:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.834+0000 7f0b9affd640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.835+0000 7f0baa195640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.835+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b9c08af20 con 0x7f0ba415e390 2024-05-09T06:28:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.835+0000 7f0baa195640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 0x7f0b8806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0034000 tx=0x7f0ba002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:10.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:10.839+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b9c059560 con 0x7f0ba415e390 2024-05-09T06:28:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.022+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ba40b2ae0 con 0x7f0ba415e390 2024-05-09T06:28:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.023+0000 7f0b9affd640 1 -- 172.21.15.1:0/3946978787 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0b9c058e60 con 0x7f0ba415e390 2024-05-09T06:28:11.005 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.024+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 msgr2=0x7f0b8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.024+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 0x7f0b8806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0034000 tx=0x7f0ba002c000 comp rx=0 tx=0).stop 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.024+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 msgr2=0x7f0ba4099080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.024+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00b970 tx=0x7f0b9c00be30 comp rx=0 tx=0).stop 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 shutdown_connections 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba415e390 0x7f0ba4099080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ba40cf990 0x7f0ba40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 --2- 172.21.15.1:0/3946978787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0ba40b1900 0x7f0ba40b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 >> 172.21.15.1:0/3946978787 conn(0x7f0ba40b1da0 msgr2=0x7f0ba40cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 shutdown_connections 2024-05-09T06:28:11.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.025+0000 7f0bac3f9640 1 -- 172.21.15.1:0/3946978787 wait complete. 2024-05-09T06:28:11.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:11.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.142+0000 7fcf48d61640 1 Processor -- start 2024-05-09T06:28:11.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.142+0000 7fcf48d61640 1 -- start start 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.143+0000 7fcf48d61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 0x7fcf440d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.143+0000 7fcf48d61640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.143+0000 7fcf48d61640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf440d2e20 con 0x7fcf440d0ce0 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.143+0000 7fcf48d61640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcf440d04a0 con 0x7fcf440cf690 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.143+0000 7fcf48d61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf440cfc00 con 0x7fcf4415e3d0 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf41d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf4259c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 0x7fcf440d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf41d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:51414/0 (socket says 172.21.15.1:51414) 2024-05-09T06:28:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf41d9b640 1 -- 172.21.15.1:0/2173270503 learned_addr learned my addr 172.21.15.1:0/2173270503 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf4259c640 1 -- 172.21.15.1:0/2173270503 >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 msgr2=0x7fcf440d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf4259c640 1 --2- 172.21.15.1:0/2173270503 >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf4259c640 1 -- 172.21.15.1:0/2173270503 >> v1:172.21.15.6:6789/0 conn(0x7fcf440d0ce0 legacy=0x7fcf440d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf4259c640 1 -- 172.21.15.1:0/2173270503 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf440cfee0 con 0x7fcf4415e3d0 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.144+0000 7fcf41d9b640 1 --2- 172.21.15.1:0/2173270503 >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.145+0000 7fcf4259c640 1 --2- 172.21.15.1:0/2173270503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 0x7fcf440d48b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009dc0 tx=0x7fcf3402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbdefab068bf966b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.145+0000 7fcf4159a640 1 -- 172.21.15.1:0/2173270503 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3402ae60 con 0x7fcf4415e3d0 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.145+0000 7fcf4159a640 1 -- 172.21.15.1:0/2173270503 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf34007740 con 0x7fcf4415e3d0 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.145+0000 7fcf4159a640 1 -- 172.21.15.1:0/2173270503 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34007b10 con 0x7fcf4415e3d0 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 -- 172.21.15.1:0/2173270503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 msgr2=0x7fcf440d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 --2- 172.21.15.1:0/2173270503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 0x7fcf440d48b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009dc0 tx=0x7fcf3402a680 comp rx=0 tx=0).stop 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 -- 172.21.15.1:0/2173270503 shutdown_connections 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 --2- 172.21.15.1:0/2173270503 >> v2:172.21.15.6:3300/0 conn(0x7fcf440cf690 0x7fcf440d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 --2- 172.21.15.1:0/2173270503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf4415e3d0 0x7fcf440d48b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 -- 172.21.15.1:0/2173270503 >> 172.21.15.1:0/2173270503 conn(0x7fcf440d3d90 msgr2=0x7fcf440d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 -- 172.21.15.1:0/2173270503 shutdown_connections 2024-05-09T06:28:11.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.146+0000 7fcf48d61640 1 -- 172.21.15.1:0/2173270503 wait complete. 2024-05-09T06:28:11.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.147+0000 7fcf48d61640 1 Processor -- start 2024-05-09T06:28:11.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.147+0000 7fcf48d61640 1 -- start start 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf440cf690 0x7fcf44155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf4415e3d0 0x7fcf44159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440b49d0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440b34a0 con 0x7fcf4415e3d0 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf48d61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf440b4b50 con 0x7fcf440cf690 2024-05-09T06:28:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35874/0 (socket says 172.21.15.1:35874) 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 -- 172.21.15.1:0/1513961053 learned_addr learned my addr 172.21.15.1:0/1513961053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf42d9d640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf4415e3d0 0x7fcf44159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf4259c640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf440cf690 0x7fcf44155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 -- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf440cf690 msgr2=0x7fcf44155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf440cf690 0x7fcf44155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 -- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf4415e3d0 msgr2=0x7fcf44159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf4415e3d0 0x7fcf44159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.148+0000 7fcf41d9b640 1 -- 172.21.15.1:0/1513961053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf440cdee0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf41d9b640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800e9b0 tx=0x7fcf3800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38017070 con 0x7fcf440d0ce0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3800c9a0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf441559a0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800cd50 con 0x7fcf440d0ce0 2024-05-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.149+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441f7850 con 0x7fcf440d0ce0 2024-05-09T06:28:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.150+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf440752b0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.150+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf3801a070 con 0x7fcf440d0ce0 2024-05-09T06:28:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.151+0000 7fcf32ffd640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 0x7fcf2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.151+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcf3808b140 con 0x7fcf440d0ce0 2024-05-09T06:28:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.151+0000 7fcf4259c640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 0x7fcf2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.151+0000 7fcf4259c640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 0x7fcf2406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf34034000 tx=0x7fcf3402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.155+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf38059700 con 0x7fcf440d0ce0 2024-05-09T06:28:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.338+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf440ca6f0 con 0x7fcf440d0ce0 2024-05-09T06:28:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.338+0000 7fcf32ffd640 1 -- 172.21.15.1:0/1513961053 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fcf38059000 con 0x7fcf440d0ce0 2024-05-09T06:28:11.320 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 msgr2=0x7fcf2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 0x7fcf2406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf34034000 tx=0x7fcf3402f000 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 msgr2=0x7fcf441606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800e9b0 tx=0x7fcf3800ee70 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 shutdown_connections 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf2406be30 0x7fcf2406e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf4415e3d0 0x7fcf44159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf440d0ce0 0x7fcf441606e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 --2- 172.21.15.1:0/1513961053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf440cf690 0x7fcf44155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 >> 172.21.15.1:0/1513961053 conn(0x7fcf440d3d90 msgr2=0x7fcf440d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 shutdown_connections 2024-05-09T06:28:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.340+0000 7fcf48d61640 1 -- 172.21.15.1:0/1513961053 wait complete. 2024-05-09T06:28:11.333 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.458+0000 7f8bb1f1d640 1 Processor -- start 2024-05-09T06:28:11.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.458+0000 7f8bb1f1d640 1 -- start start 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bb1f1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 0x7f8bac0d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bb1f1d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8bac0b75c0 0x7f8bac0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bb1f1d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bac0d2760 con 0x7f8bac0d0a10 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bb1f1d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8bac0b6e50 con 0x7f8bac0b75c0 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bb1f1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bac0d2a40 con 0x7f8bac15e0f0 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8babfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8bac0b75c0 0x7f8bac0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8baaffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8bac0d0a10 0x7f8bac0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60704/0 (socket says 172.21.15.1:60704) 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8baaffd640 1 -- 172.21.15.1:0/1435954995 learned_addr learned my addr 172.21.15.1:0/1435954995 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8bab7fe640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 0x7f8bac0d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.459+0000 7f8babfff640 1 -- 172.21.15.1:0/1435954995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 msgr2=0x7f8bac0d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8babfff640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 0x7f8bac0d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8babfff640 1 -- 172.21.15.1:0/1435954995 >> v1:172.21.15.1:6789/0 conn(0x7f8bac0d0a10 legacy=0x7f8bac0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8baa7fc640 1 -- 172.21.15.1:0/1435954995 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871799548 0 0) 0x7f8bac0d2760 con 0x7f8bac0d0a10 2024-05-09T06:28:11.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8babfff640 1 -- 172.21.15.1:0/1435954995 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac0d2d20 con 0x7f8bac0b75c0 2024-05-09T06:28:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8bab7fe640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 0x7f8bac0d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8babfff640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac0b75c0 0x7f8bac0b7a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d90 tx=0x7f8ba402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb31ea48875495d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8baa7fc640 1 -- 172.21.15.1:0/1435954995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4007740 con 0x7f8bac0b75c0 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8baa7fc640 1 -- 172.21.15.1:0/1435954995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba40078e0 con 0x7f8bac0b75c0 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.460+0000 7f8baa7fc640 1 -- 172.21.15.1:0/1435954995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4007cb0 con 0x7f8bac0b75c0 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/1435954995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac0b75c0 msgr2=0x7f8bac0b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac0b75c0 0x7f8bac0b7a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d90 tx=0x7f8ba402a660 comp rx=0 tx=0).stop 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/1435954995 shutdown_connections 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac0b75c0 0x7f8bac0b7a50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/1435954995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac15e0f0 0x7f8bac0d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/1435954995 >> 172.21.15.1:0/1435954995 conn(0x7f8bac09d3b0 msgr2=0x7f8bac0d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/1435954995 shutdown_connections 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.461+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/1435954995 wait complete. 2024-05-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.462+0000 7f8bb1f1d640 1 Processor -- start 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.462+0000 7f8bb1f1d640 1 -- start start 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac0b75c0 0x7f8bac14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bac0d0a10 0x7f8bac0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac15e650 con 0x7f8bac15e0f0 2024-05-09T06:28:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac15e4d0 con 0x7f8bac0d0a10 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bb1f1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bac0cfd70 con 0x7f8bac0b75c0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8babfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8babfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35900/0 (socket says 172.21.15.1:35900) 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8babfff640 1 -- 172.21.15.1:0/2003522265 learned_addr learned my addr 172.21.15.1:0/2003522265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8bab7fe640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac0b75c0 0x7f8bac14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.463+0000 7f8baaffd640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bac0d0a10 0x7f8bac0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 -- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac0b75c0 msgr2=0x7f8bac14cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac0b75c0 0x7f8bac14cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 -- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bac0d0a10 msgr2=0x7f8bac0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bac0d0a10 0x7f8bac0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 -- 172.21.15.1:0/2003522265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac0c94f0 con 0x7f8bac15e0f0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8babfff640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d90 tx=0x7f8ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4040070 con 0x7f8bac15e0f0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4031c40 con 0x7f8bac15e0f0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac0c97d0 con 0x7f8bac15e0f0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4003990 con 0x7f8bac15e0f0 2024-05-09T06:28:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.464+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac0c9d90 con 0x7f8bac15e0f0 2024-05-09T06:28:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.465+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac0752b0 con 0x7f8bac15e0f0 2024-05-09T06:28:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.465+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ba4039040 con 0x7f8bac15e0f0 2024-05-09T06:28:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.466+0000 7f8b93fff640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 0x7f8b9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.466+0000 7f8bab7fe640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 0x7f8b9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.466+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8ba40ac1e0 con 0x7f8bac15e0f0 2024-05-09T06:28:11.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.467+0000 7f8bab7fe640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 0x7f8b9806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00aac0 tx=0x7f8b9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.471+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ba407a140 con 0x7f8bac15e0f0 2024-05-09T06:28:11.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.652+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8bac06ce10 con 0x7f8bac15e0f0 2024-05-09T06:28:11.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.652+0000 7f8b93fff640 1 -- 172.21.15.1:0/2003522265 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f8ba407e440 con 0x7f8bac15e0f0 2024-05-09T06:28:11.634 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 msgr2=0x7f8b9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 0x7f8b9806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00aac0 tx=0x7f8b9c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 msgr2=0x7f8bac0cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d90 tx=0x7f8ba4004a40 comp rx=0 tx=0).stop 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 shutdown_connections 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b9806be30 0x7f8b9806e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bac15e0f0 0x7f8bac0cdea0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bac0d0a10 0x7f8bac0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 --2- 172.21.15.1:0/2003522265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bac0b75c0 0x7f8bac14cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 >> 172.21.15.1:0/2003522265 conn(0x7f8bac09d3b0 msgr2=0x7f8bac15d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 shutdown_connections 2024-05-09T06:28:11.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.654+0000 7f8bb1f1d640 1 -- 172.21.15.1:0/2003522265 wait complete. 2024-05-09T06:28:11.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-09T06:28:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.769+0000 7fc6ef722640 1 Processor -- start 2024-05-09T06:28:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.769+0000 7fc6ef722640 1 -- start start 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6ef722640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc6e815db10 0x7fc6e80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6ef722640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6e815e6d0 0x7fc6e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6ef722640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e8070f40 con 0x7fc6e80715d0 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6ef722640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e80cf170 con 0x7fc6e815e6d0 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6ef722640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e80cf450 con 0x7fc6e815db10 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6e815e6d0 0x7fc6e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6e815e6d0 0x7fc6e806e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35914/0 (socket says 172.21.15.1:35914) 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 -- 172.21.15.1:0/1560051500 learned_addr learned my addr 172.21.15.1:0/1560051500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 -- 172.21.15.1:0/1560051500 >> v2:172.21.15.6:3300/0 conn(0x7fc6e815db10 msgr2=0x7fc6e80b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 --2- 172.21.15.1:0/1560051500 >> v2:172.21.15.6:3300/0 conn(0x7fc6e815db10 0x7fc6e80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.770+0000 7fc6edcbf640 1 -- 172.21.15.1:0/1560051500 >> v1:172.21.15.6:6789/0 conn(0x7fc6e80715d0 legacy=0x7fc6e815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.771+0000 7fc6edcbf640 1 -- 172.21.15.1:0/1560051500 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8070c10 con 0x7fc6e815e6d0 2024-05-09T06:28:11.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.771+0000 7fc6edcbf640 1 --2- 172.21.15.1:0/1560051500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815e6d0 0x7fc6e806e340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80744b0 tx=0x7fc6e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00ae3cc15ba9272 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.771+0000 7fc6dffff640 1 -- 172.21.15.1:0/1560051500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4036070 con 0x7fc6e815e6d0 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.771+0000 7fc6dffff640 1 -- 172.21.15.1:0/1560051500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e4031600 con 0x7fc6e815e6d0 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.771+0000 7fc6dffff640 1 -- 172.21.15.1:0/1560051500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40319d0 con 0x7fc6e815e6d0 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 -- 172.21.15.1:0/1560051500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815e6d0 msgr2=0x7fc6e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1560051500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815e6d0 0x7fc6e806e340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80744b0 tx=0x7fc6e402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 -- 172.21.15.1:0/1560051500 shutdown_connections 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1560051500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815e6d0 0x7fc6e806e340 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1560051500 >> v2:172.21.15.6:3300/0 conn(0x7fc6e815db10 0x7fc6e80b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 -- 172.21.15.1:0/1560051500 >> 172.21.15.1:0/1560051500 conn(0x7fc6e80cc4b0 msgr2=0x7fc6e80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 -- 172.21.15.1:0/1560051500 shutdown_connections 2024-05-09T06:28:11.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.772+0000 7fc6ef722640 1 -- 172.21.15.1:0/1560051500 wait complete. 2024-05-09T06:28:11.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.773+0000 7fc6ef722640 1 Processor -- start 2024-05-09T06:28:11.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.773+0000 7fc6ef722640 1 -- start start 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80715d0 0x7fc6e8152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc6e815e6d0 0x7fc6e80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e815edc0 con 0x7fc6e815db10 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e815ec40 con 0x7fc6e80715d0 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ef722640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e806cd30 con 0x7fc6e815e6d0 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35920/0 (socket says 172.21.15.1:35920) 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 -- 172.21.15.1:0/1642208559 learned_addr learned my addr 172.21.15.1:0/1642208559 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6ed4be640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80715d0 0x7fc6e8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6edcbf640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc6e815e6d0 0x7fc6e80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 -- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc6e815e6d0 msgr2=0x7fc6e80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc6e815e6d0 0x7fc6e80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 -- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80715d0 msgr2=0x7fc6e8152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80715d0 0x7fc6e8152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.774+0000 7fc6eccbd640 1 -- 172.21.15.1:0/1642208559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e81f3b90 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6eccbd640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c410 tx=0x7fc6d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8017070 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d80035f0 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8003a20 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81f3e10 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.775+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81f43d0 con 0x7fc6e815db10 2024-05-09T06:28:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.776+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e80752b0 con 0x7fc6e815db10 2024-05-09T06:28:11.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.776+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc6d801a070 con 0x7fc6e815db10 2024-05-09T06:28:11.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.777+0000 7fc6ddffb640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 0x7fc6cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.777+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc6d808a740 con 0x7fc6e815db10 2024-05-09T06:28:11.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.777+0000 7fc6ed4be640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 0x7fc6cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.777+0000 7fc6ed4be640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 0x7fc6cc06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009890 tx=0x7fc6e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:11.782+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc6e81f3b90 con 0x7fc6e815db10 2024-05-09T06:28:12.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.035+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fc6e806db50 con 0x7fc6e815db10 2024-05-09T06:28:12.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.171+0000 7fc6ddffb640 1 -- 172.21.15.1:0/1642208559 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v70) ==== 145+0+0 (secure 0 0 0) 0x7fc6d8058cf0 con 0x7fc6e815db10 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 msgr2=0x7fc6cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 0x7fc6cc06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009890 tx=0x7fc6e0008040 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 msgr2=0x7fc6e8158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c410 tx=0x7fc6d800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 shutdown_connections 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc6cc06be30 0x7fc6cc06e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc6e815e6d0 0x7fc6e80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6e815db10 0x7fc6e8158830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 --2- 172.21.15.1:0/1642208559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80715d0 0x7fc6e8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 >> 172.21.15.1:0/1642208559 conn(0x7fc6e80cc4b0 msgr2=0x7fc6e80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 shutdown_connections 2024-05-09T06:28:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.173+0000 7fc6ef722640 1 -- 172.21.15.1:0/1642208559 wait complete. 2024-05-09T06:28:12.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:28:12.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.291+0000 7f9141922640 1 Processor -- start 2024-05-09T06:28:12.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.291+0000 7f9141922640 1 -- start start 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 0x7f913c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f913c0b5260 0x7f913c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f913c156980 con 0x7f913c0b5260 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f913c06d870 con 0x7f913c0b4de0 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f9141922640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f913c15d320 con 0x7f913c15c4c0 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.292+0000 7f913b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f913c0b5260 0x7f913c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f913c0b5260 0x7f913c15cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35928/0 (socket says 172.21.15.1:35928) 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 -- 172.21.15.1:0/1272287593 learned_addr learned my addr 172.21.15.1:0/1272287593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913a7fc640 1 --2- 172.21.15.1:0/1272287593 >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 0x7f913c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913affd640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 -- 172.21.15.1:0/1272287593 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 msgr2=0x7f913c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 -- 172.21.15.1:0/1272287593 >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 msgr2=0x7f913c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 --2- 172.21.15.1:0/1272287593 >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 0x7f913c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913b7fe640 1 -- 172.21.15.1:0/1272287593 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c06cf40 con 0x7f913c0b5260 2024-05-09T06:28:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.293+0000 7f913a7fc640 1 --2- 172.21.15.1:0/1272287593 >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 0x7f913c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f913b7fe640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b5260 0x7f913c15cde0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b27c0 tx=0x7f912c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e729efed9d04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9139ffb640 1 -- 172.21.15.1:0/1272287593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c037070 con 0x7f913c0b5260 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9139ffb640 1 -- 172.21.15.1:0/1272287593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c032440 con 0x7f913c0b5260 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9139ffb640 1 -- 172.21.15.1:0/1272287593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c032810 con 0x7f913c0b5260 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 -- 172.21.15.1:0/1272287593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b5260 msgr2=0x7f913c15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b5260 0x7f913c15cde0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b27c0 tx=0x7f912c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 -- 172.21.15.1:0/1272287593 shutdown_connections 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b5260 0x7f913c15cde0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 --2- 172.21.15.1:0/1272287593 >> v2:172.21.15.6:3300/0 conn(0x7f913c0b4de0 0x7f913c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 --2- 172.21.15.1:0/1272287593 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 -- 172.21.15.1:0/1272287593 >> 172.21.15.1:0/1272287593 conn(0x7f913c0cc4d0 msgr2=0x7f913c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 -- 172.21.15.1:0/1272287593 shutdown_connections 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.294+0000 7f9141922640 1 -- 172.21.15.1:0/1272287593 wait complete. 2024-05-09T06:28:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.295+0000 7f9141922640 1 Processor -- start 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 -- start start 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0b5260 0x7f913c0aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0b2c40 con 0x7f913c0b4de0 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f913c06d500 con 0x7f913c0b5260 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f9141922640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0b2dc0 con 0x7f913c15c4c0 2024-05-09T06:28:12.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.296+0000 7f913affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35944/0 (socket says 172.21.15.1:35944) 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 -- 172.21.15.1:0/262632223 learned_addr learned my addr 172.21.15.1:0/262632223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913a7fc640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0b5260 0x7f913c0aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913b7fe640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 -- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 msgr2=0x7f913c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 -- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0b5260 msgr2=0x7f913c0aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0b5260 0x7f913c0aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 -- 172.21.15.1:0/262632223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c0ada30 con 0x7f913c0b4de0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913b7fe640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f913affd640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f913400cd00 tx=0x7f913400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134003990 con 0x7f913c0b4de0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134003b30 con 0x7f913c0b4de0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c0add10 con 0x7f913c0b4de0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134004020 con 0x7f913c0b4de0 2024-05-09T06:28:12.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.297+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c0ae2d0 con 0x7f913c0b4de0 2024-05-09T06:28:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.298+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c0752b0 con 0x7f913c0b4de0 2024-05-09T06:28:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.299+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91340041c0 con 0x7f913c0b4de0 2024-05-09T06:28:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.299+0000 7f9133fff640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 0x7f911c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.299+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f913408b600 con 0x7f913c0b4de0 2024-05-09T06:28:12.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.299+0000 7f913a7fc640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 0x7f911c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.300+0000 7f913a7fc640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 0x7f911c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b2c10 tx=0x7f9124008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.304+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9134014030 con 0x7f913c0b4de0 2024-05-09T06:28:12.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.487+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f913c0b1170 con 0x7f913c0b4de0 2024-05-09T06:28:12.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.490+0000 7f9133fff640 1 -- 172.21.15.1:0/262632223 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f91340595f0 con 0x7f913c0b4de0 2024-05-09T06:28:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 msgr2=0x7f911c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 0x7f911c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b2c10 tx=0x7f9124008040 comp rx=0 tx=0).stop 2024-05-09T06:28:12.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 msgr2=0x7f913c144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f913400cd00 tx=0x7f913400a5c0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 shutdown_connections 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f911c06be30 0x7f911c06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f913c15c4c0 0x7f913c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f913c0b5260 0x7f913c0aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 --2- 172.21.15.1:0/262632223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913c0b4de0 0x7f913c144190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 >> 172.21.15.1:0/262632223 conn(0x7f913c0cc4d0 msgr2=0x7f913c0af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 shutdown_connections 2024-05-09T06:28:12.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.492+0000 7f9141922640 1 -- 172.21.15.1:0/262632223 wait complete. 2024-05-09T06:28:12.485 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:28:12.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:28:12.530 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:28:12.577 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:28:12.577 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:12.577 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:12.593 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:12.593 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:12.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 Processor -- start 2024-05-09T06:28:12.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 -- start start 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 0x7fabdc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabdc0b33c0 con 0x7fabdc0d0f80 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fabdc0d4dd0 con 0x7fabdc0d4940 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.813+0000 7fabe0b9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc0b3660 con 0x7fabdc161950 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabd3fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 0x7fabdc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:42650/0 (socket says 172.21.15.1:42650) 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabdad9d640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fabdc0d0f80 0x7fabdc0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:34060/0 (socket says 172.21.15.1:34060) 2024-05-09T06:28:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 -- 172.21.15.1:0/1149441250 learned_addr learned my addr 172.21.15.1:0/1149441250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 -- 172.21.15.1:0/1149441250 >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 msgr2=0x7fabdc0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 --2- 172.21.15.1:0/1149441250 >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 0x7fabdc0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 -- 172.21.15.1:0/1149441250 >> v1:172.21.15.6:6789/0 conn(0x7fabdc0d0f80 legacy=0x7fabdc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabda59c640 1 -- 172.21.15.1:0/1149441250 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc0b3940 con 0x7fabdc161950 2024-05-09T06:28:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.814+0000 7fabd3fff640 1 --2- 172.21.15.1:0/1149441250 >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 0x7fabdc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:12.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.815+0000 7fabda59c640 1 --2- 172.21.15.1:0/1149441250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b3590 tx=0x7fabcc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6922f454c7239203 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.815+0000 7fabd9d9b640 1 -- 172.21.15.1:0/1149441250 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc037070 con 0x7fabdc161950 2024-05-09T06:28:12.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.815+0000 7fabd9d9b640 1 -- 172.21.15.1:0/1149441250 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabcc032440 con 0x7fabdc161950 2024-05-09T06:28:12.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.815+0000 7fabd9d9b640 1 -- 172.21.15.1:0/1149441250 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc032810 con 0x7fabdc161950 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1149441250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 msgr2=0x7fabdc0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1149441250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b3590 tx=0x7fabcc02a250 comp rx=0 tx=0).stop 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1149441250 shutdown_connections 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1149441250 >> v2:172.21.15.6:3300/0 conn(0x7fabdc0d4940 0x7fabdc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1149441250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0d2400 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1149441250 >> 172.21.15.1:0/1149441250 conn(0x7fabdc0d3e20 msgr2=0x7fabdc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1149441250 shutdown_connections 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.816+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1149441250 wait complete. 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.817+0000 7fabe0b9e640 1 Processor -- start 2024-05-09T06:28:12.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.817+0000 7fabe0b9e640 1 -- start start 2024-05-09T06:28:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabdc0d0f80 0x7fabdc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc0b52d0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc0b5150 con 0x7fabdc0d0f80 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabe0b9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc0b5450 con 0x7fabdc161950 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabdad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35956/0 (socket says 172.21.15.1:35956) 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabdad9d640 1 -- 172.21.15.1:0/1586675155 learned_addr learned my addr 172.21.15.1:0/1586675155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabda59c640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabdc0d0f80 0x7fabdc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.818+0000 7fabd3fff640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 -- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 msgr2=0x7fabdc0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 -- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabdc0d0f80 msgr2=0x7fabdc14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabdc0d0f80 0x7fabdc14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 -- 172.21.15.1:0/1586675155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc0cc100 con 0x7fabdc0cdad0 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabd3fff640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabdad9d640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c7a0 tx=0x7fabd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4017070 con 0x7fabdc0cdad0 2024-05-09T06:28:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd400a640 con 0x7fabdc0cdad0 2024-05-09T06:28:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc0ccaf0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400a9f0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.819+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc0ccd90 con 0x7fabdc0cdad0 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.820+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc0752b0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.821+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabd401a070 con 0x7fabdc0cdad0 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.821+0000 7fabd2ffd640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 0x7fabbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.821+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fabd408a700 con 0x7fabdc0cdad0 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.821+0000 7fabda59c640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 0x7fabbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.821+0000 7fabda59c640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 0x7fabbc06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabcc034000 tx=0x7fabcc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:12.825+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fabd40586f0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.010+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabdc161150 con 0x7fabdc0cdad0 2024-05-09T06:28:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.011+0000 7fabd2ffd640 1 -- 172.21.15.1:0/1586675155 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7fabd405c9f0 con 0x7fabdc0cdad0 2024-05-09T06:28:12.993 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:12.993 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:12.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 msgr2=0x7fabbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 0x7fabbc06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabcc034000 tx=0x7fabcc02e000 comp rx=0 tx=0).stop 2024-05-09T06:28:12.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 msgr2=0x7fabdc06cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:12.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c7a0 tx=0x7fabd400cc60 comp rx=0 tx=0).stop 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 shutdown_connections 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fabbc06be30 0x7fabbc06e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabdc0cdad0 0x7fabdc06cfb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabdc161950 0x7fabdc0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 --2- 172.21.15.1:0/1586675155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabdc0d0f80 0x7fabdc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 >> 172.21.15.1:0/1586675155 conn(0x7fabdc0d3e20 msgr2=0x7fabdc15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 shutdown_connections 2024-05-09T06:28:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.013+0000 7fabe0b9e640 1 -- 172.21.15.1:0/1586675155 wait complete. 2024-05-09T06:28:13.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:13.007 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:13.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.132+0000 7f38cf09a640 1 Processor -- start 2024-05-09T06:28:13.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.132+0000 7f38cf09a640 1 -- start start 2024-05-09T06:28:13.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cf09a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38c80b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cf09a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f38c80cc8e0 0x7f38c80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cf09a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c815e900 con 0x7f38c815e390 2024-05-09T06:28:13.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cf09a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f38c80b1b90 con 0x7f38c80cc8e0 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cf09a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38c815ebe0 con 0x7f38c80b11f0 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cce36640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f38c815e390 0x7f38c80cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60706/0 (socket says 172.21.15.1:60706) 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cce36640 1 -- 172.21.15.1:0/948148325 learned_addr learned my addr 172.21.15.1:0/948148325 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3490929682 0 0) 0x7f38c815e900 con 0x7f38c815e390 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bffff640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38c80b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> v2:172.21.15.6:3300/0 conn(0x7f38c80cc8e0 0x7f38c80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b0003520 con 0x7f38c815e390 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1949433256 0 0) 0x7f38b0003520 con 0x7f38c815e390 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 msgr2=0x7f38c80b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38c80b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 >> v2:172.21.15.6:3300/0 conn(0x7f38c80cc8e0 msgr2=0x7f38c80ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 --2- 172.21.15.1:0/948148325 >> v2:172.21.15.6:3300/0 conn(0x7f38c80cc8e0 0x7f38c80ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c815eec0 con 0x7f38c815e390 2024-05-09T06:28:13.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f38c4001670 con 0x7f38c815e390 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.133+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> v2:172.21.15.6:3300/0 conn(0x7f38c80cc8e0 0x7f38c80ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 >> v1:172.21.15.1:6789/0 conn(0x7f38c815e390 legacy=0x7f38c80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38b0004d40 0x7f38b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38c815ebe0 con 0x7f38b0004d40 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38b000b9c0 con 0x7f38b00039b0 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38b000bb70 con 0x7f38c80b11f0 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38b0004d40 0x7f38b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cce36640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38bffff640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 -- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 msgr2=0x7f38b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38c80b11f0 0x7f38b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 -- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38b00039b0 msgr2=0x7f38b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.134+0000 7f38cd637640 1 -- 172.21.15.1:0/948148325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b0003520 con 0x7f38b0004d40 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cd637640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38b0004d40 0x7f38b00051c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f38c00073e0 tx=0x7f38b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a2ea964d2649ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0009c80 con 0x7f38b0004d40 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c002c680 con 0x7f38b0004d40 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38bf7fe640 1 -- 172.21.15.1:0/948148325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c002ca30 con 0x7f38b0004d40 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 -- 172.21.15.1:0/948148325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38b0004d40 msgr2=0x7f38b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 --2- 172.21.15.1:0/948148325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38b0004d40 0x7f38b00051c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f38c00073e0 tx=0x7f38b0005830 comp rx=0 tx=0).stop 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cd637640 1 -- 172.21.15.1:0/948148325 reap_dead start 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 -- 172.21.15.1:0/948148325 shutdown_connections 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 -- 172.21.15.1:0/948148325 >> 172.21.15.1:0/948148325 conn(0x7f38c80b1da0 msgr2=0x7f38c80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 -- 172.21.15.1:0/948148325 shutdown_connections 2024-05-09T06:28:13.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.135+0000 7f38cf09a640 1 -- 172.21.15.1:0/948148325 wait complete. 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.137+0000 7f38cf09a640 1 Processor -- start 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.137+0000 7f38cf09a640 1 -- start start 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.137+0000 7f38cf09a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 0x7f38c8153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 0x7f38c8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38c8162ba0 con 0x7f38c8099b00 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38c815f6c0 con 0x7f38c8072800 2024-05-09T06:28:13.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38c80cffb0 con 0x7f38b0004d40 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35972/0 (socket says 172.21.15.1:35972) 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 -- 172.21.15.1:0/143181645 learned_addr learned my addr 172.21.15.1:0/143181645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cce36640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 0x7f38c8153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cd637640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 0x7f38c8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 -- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 msgr2=0x7f38c8153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 0x7f38c8153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 -- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 msgr2=0x7f38c8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 0x7f38c8072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 -- 172.21.15.1:0/143181645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c80b2fc0 con 0x7f38c8099b00 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cce36640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 0x7f38c8153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cd637640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 0x7f38c8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bffff640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38b8011910 tx=0x7f38b8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800faf0 con 0x7f38c8099b00 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b800fc90 con 0x7f38c8099b00 2024-05-09T06:28:13.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c80b3260 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8014440 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.138+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c80b3820 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.139+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c80752b0 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.140+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38b8012040 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.140+0000 7f38bd7fa640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 0x7f38b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.140+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f38b808b820 con 0x7f38c8099b00 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.140+0000 7f38cce36640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 0x7f38b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.141+0000 7f38cce36640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 0x7f38b406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38c8153f40 tx=0x7f38c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.145+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38b8019030 con 0x7f38c8099b00 2024-05-09T06:28:13.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.326+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38c80b2ae0 con 0x7f38c8099b00 2024-05-09T06:28:13.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.327+0000 7f38bd7fa640 1 -- 172.21.15.1:0/143181645 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f38b8059dc0 con 0x7f38c8099b00 2024-05-09T06:28:13.309 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.328+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 msgr2=0x7f38b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.328+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 0x7f38b406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38c8153f40 tx=0x7f38c4009040 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.328+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 msgr2=0x7f38c80afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.328+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38b8011910 tx=0x7f38b8011dd0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 shutdown_connections 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f38b406be60 0x7f38b406e320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38c8072800 0x7f38c8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38c8099b00 0x7f38c80afac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 --2- 172.21.15.1:0/143181645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38b0004d40 0x7f38c8153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 >> 172.21.15.1:0/143181645 conn(0x7f38c80b1da0 msgr2=0x7f38c806e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 shutdown_connections 2024-05-09T06:28:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.329+0000 7f38cf09a640 1 -- 172.21.15.1:0/143181645 wait complete. 2024-05-09T06:28:13.322 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.446+0000 7f7d455c8640 1 Processor -- start 2024-05-09T06:28:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 -- start start 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d400b11f0 0x7f7d400b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7d400cc8e0 0x7f7d400ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d4015e900 con 0x7f7d4015e390 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7d400b1b90 con 0x7f7d400cc8e0 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d455c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4015ebe0 con 0x7f7d400b11f0 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.447+0000 7f7d3f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7d400cc8e0 0x7f7d400ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7d400cc8e0 0x7f7d400ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35982/0 (socket says 172.21.15.1:35982) 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 -- 172.21.15.1:0/1051697269 learned_addr learned my addr 172.21.15.1:0/1051697269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3e7fc640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d400b11f0 0x7f7d400b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 -- 172.21.15.1:0/1051697269 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d400b11f0 msgr2=0x7f7d400b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d400b11f0 0x7f7d400b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 -- 172.21.15.1:0/1051697269 >> v1:172.21.15.6:6789/0 conn(0x7f7d4015e390 legacy=0x7f7d400cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 -- 172.21.15.1:0/1051697269 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4015eec0 con 0x7f7d400cc8e0 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.448+0000 7f7d3f7fe640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400cc8e0 0x7f7d400ccd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009e50 tx=0x7f7d3002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f163859db7e380 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d3dffb640 1 -- 172.21.15.1:0/1051697269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30036070 con 0x7f7d400cc8e0 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d3dffb640 1 -- 172.21.15.1:0/1051697269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30031600 con 0x7f7d400cc8e0 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d3dffb640 1 -- 172.21.15.1:0/1051697269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300319d0 con 0x7f7d400cc8e0 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 -- 172.21.15.1:0/1051697269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400cc8e0 msgr2=0x7f7d400ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400cc8e0 0x7f7d400ccd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009e50 tx=0x7f7d3002a6c0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 -- 172.21.15.1:0/1051697269 shutdown_connections 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400cc8e0 0x7f7d400ccd50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1051697269 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d400b11f0 0x7f7d400b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 -- 172.21.15.1:0/1051697269 >> 172.21.15.1:0/1051697269 conn(0x7f7d400b1da0 msgr2=0x7f7d400b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 -- 172.21.15.1:0/1051697269 shutdown_connections 2024-05-09T06:28:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.449+0000 7f7d455c8640 1 -- 172.21.15.1:0/1051697269 wait complete. 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 Processor -- start 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 -- start start 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 0x7f7d4014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d4015e390 0x7f7d400b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40162ba0 con 0x7f7d400b11f0 2024-05-09T06:28:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015f6c0 con 0x7f7d400cc8e0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d455c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7d400cffb0 con 0x7f7d4015e390 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.451+0000 7f7d3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35990/0 (socket says 172.21.15.1:35990) 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 -- 172.21.15.1:0/1132938430 learned_addr learned my addr 172.21.15.1:0/1132938430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3e7fc640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 0x7f7d4014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3f7fe640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d4015e390 0x7f7d400b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 -- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d4015e390 msgr2=0x7f7d400b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d4015e390 0x7f7d400b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 -- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 msgr2=0x7f7d4014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 0x7f7d4014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 -- 172.21.15.1:0/1132938430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d400b46c0 con 0x7f7d400b11f0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3e7fc640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 0x7f7d4014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d3effd640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800b3f0 tx=0x7f7d3800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38015020 con 0x7f7d400b11f0 2024-05-09T06:28:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d38011830 con 0x7f7d400b11f0 2024-05-09T06:28:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d400b49a0 con 0x7f7d400b11f0 2024-05-09T06:28:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38011c20 con 0x7f7d400b11f0 2024-05-09T06:28:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.452+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d400b4f60 con 0x7f7d400b11f0 2024-05-09T06:28:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.453+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400752b0 con 0x7f7d400b11f0 2024-05-09T06:28:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.454+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d38018020 con 0x7f7d400b11f0 2024-05-09T06:28:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.454+0000 7f7d377fe640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 0x7f7d2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.455+0000 7f7d3e7fc640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 0x7f7d2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.455+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7d3808ae00 con 0x7f7d400b11f0 2024-05-09T06:28:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.455+0000 7f7d3e7fc640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 0x7f7d2006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d400c4210 tx=0x7f7d28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.459+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d3801f030 con 0x7f7d400b11f0 2024-05-09T06:28:13.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.641+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d400b61f0 con 0x7f7d400b11f0 2024-05-09T06:28:13.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.642+0000 7f7d377fe640 1 -- 172.21.15.1:0/1132938430 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7d380172c0 con 0x7f7d400b11f0 2024-05-09T06:28:13.624 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 msgr2=0x7f7d2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 0x7f7d2006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d400c4210 tx=0x7f7d28008040 comp rx=0 tx=0).stop 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 msgr2=0x7f7d400c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800b3f0 tx=0x7f7d3800b8b0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 shutdown_connections 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d2006be60 0x7f7d2006e320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7d4015e390 0x7f7d400b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d400cc8e0 0x7f7d4014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 --2- 172.21.15.1:0/1132938430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d400b11f0 0x7f7d400c6a70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 >> 172.21.15.1:0/1132938430 conn(0x7f7d400b1da0 msgr2=0x7f7d40072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 shutdown_connections 2024-05-09T06:28:13.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.644+0000 7f7d455c8640 1 -- 172.21.15.1:0/1132938430 wait complete. 2024-05-09T06:28:13.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.763+0000 7f59b9d4f640 1 Processor -- start 2024-05-09T06:28:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.763+0000 7f59b9d4f640 1 -- start start 2024-05-09T06:28:13.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b9d4f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f59b4158a50 0x7f59b40718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b9d4f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 0x7f59b4071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b9d4f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b415b420 con 0x7f59b406e6d0 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b9d4f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f59b4158e30 con 0x7f59b4158a50 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b9d4f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f59b4072480 con 0x7f59b415c4a0 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f59b4158a50 0x7f59b40718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f59b406e6d0 0x7f59b40cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60708/0 (socket says 172.21.15.1:60708) 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.764+0000 7f59b3fff640 1 -- 172.21.15.1:0/4074674041 learned_addr learned my addr 172.21.15.1:0/4074674041 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/4074674041 >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 0x7f59b4071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b37fe640 1 -- 172.21.15.1:0/4074674041 >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 msgr2=0x7f59b4071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b37fe640 1 --2- 172.21.15.1:0/4074674041 >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 0x7f59b4071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b37fe640 1 -- 172.21.15.1:0/4074674041 >> v1:172.21.15.1:6789/0 conn(0x7f59b406e6d0 legacy=0x7f59b40cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b37fe640 1 -- 172.21.15.1:0/4074674041 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b415b6e0 con 0x7f59b4158a50 2024-05-09T06:28:13.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/4074674041 >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 0x7f59b4071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:13.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.765+0000 7f59b37fe640 1 --2- 172.21.15.1:0/4074674041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b4158a50 0x7f59b40718c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f59b4068a10 tx=0x7f59a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35599a65c5e98d82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b27fc640 1 -- 172.21.15.1:0/4074674041 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4037070 con 0x7f59b4158a50 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b27fc640 1 -- 172.21.15.1:0/4074674041 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4032440 con 0x7f59b4158a50 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b27fc640 1 -- 172.21.15.1:0/4074674041 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4032810 con 0x7f59b4158a50 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 -- 172.21.15.1:0/4074674041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b4158a50 msgr2=0x7f59b40718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/4074674041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b4158a50 0x7f59b40718c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f59b4068a10 tx=0x7f59a402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 -- 172.21.15.1:0/4074674041 shutdown_connections 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/4074674041 >> v2:172.21.15.6:3300/0 conn(0x7f59b415c4a0 0x7f59b4071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/4074674041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b4158a50 0x7f59b40718c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 -- 172.21.15.1:0/4074674041 >> 172.21.15.1:0/4074674041 conn(0x7f59b40cc4d0 msgr2=0x7f59b40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 -- 172.21.15.1:0/4074674041 shutdown_connections 2024-05-09T06:28:13.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.766+0000 7f59b9d4f640 1 -- 172.21.15.1:0/4074674041 wait complete. 2024-05-09T06:28:13.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.767+0000 7f59b9d4f640 1 Processor -- start 2024-05-09T06:28:13.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 -- start start 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 0x7f59b40cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 0x7f59b40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59b415bf80 con 0x7f59b406e6d0 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59b415bca0 con 0x7f59b415c4a0 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b9d4f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f59b4070ea0 con 0x7f59b4158a50 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36008/0 (socket says 172.21.15.1:36008) 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.768+0000 7f59b37fe640 1 -- 172.21.15.1:0/548603049 learned_addr learned my addr 172.21.15.1:0/548603049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b3fff640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 0x7f59b40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 0x7f59b40cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 -- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 msgr2=0x7f59b40cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 0x7f59b40cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 -- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 msgr2=0x7f59b40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 0x7f59b40ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 -- 172.21.15.1:0/548603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40b3b70 con 0x7f59b406e6d0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b3fff640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 0x7f59b40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 0x7f59b40cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b37fe640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4037040 con 0x7f59b406e6d0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4042070 con 0x7f59b406e6d0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b40b3e50 con 0x7f59b406e6d0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4030a50 con 0x7f59b406e6d0 2024-05-09T06:28:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.769+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b40b4410 con 0x7f59b406e6d0 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.770+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b40752b0 con 0x7f59b406e6d0 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.771+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59a4030bf0 con 0x7f59b406e6d0 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.771+0000 7f599bfff640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 0x7f59a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.771+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f59a40abc80 con 0x7f59b406e6d0 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.771+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 0x7f59a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:13.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.772+0000 7f59b2ffd640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 0x7f59a006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:13.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.776+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59a407a2c0 con 0x7f59b406e6d0 2024-05-09T06:28:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.959+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59b4153ff0 con 0x7f59b406e6d0 2024-05-09T06:28:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.960+0000 7f599bfff640 1 -- 172.21.15.1:0/548603049 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f59a4079bc0 con 0x7f59b406e6d0 2024-05-09T06:28:13.942 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 msgr2=0x7f59a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 0x7f59a006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8008040 comp rx=0 tx=0).stop 2024-05-09T06:28:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 msgr2=0x7f59b40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a40046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 shutdown_connections 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59a006be30 0x7f59a006e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59b415c4a0 0x7f59b40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59b4158a50 0x7f59b40cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 --2- 172.21.15.1:0/548603049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b406e6d0 0x7f59b40b3190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 >> 172.21.15.1:0/548603049 conn(0x7f59b40cc4d0 msgr2=0x7f59b40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 shutdown_connections 2024-05-09T06:28:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:13.962+0000 7f59b9d4f640 1 -- 172.21.15.1:0/548603049 wait complete. 2024-05-09T06:28:13.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.079+0000 7fe264e83640 1 Processor -- start 2024-05-09T06:28:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.079+0000 7fe264e83640 1 -- start start 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe264e83640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe2600cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe264e83640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe2600cf8e0 0x7fe2600ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe264e83640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe260072050 con 0x7fe2600c41e0 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe264e83640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600cd160 con 0x7fe2600cf8e0 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe264e83640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe260072330 con 0x7fe2600d06f0 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe257fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe2600c41e0 0x7fe260161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60716/0 (socket says 172.21.15.1:60716) 2024-05-09T06:28:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe257fff640 1 -- 172.21.15.1:0/2862323178 learned_addr learned my addr 172.21.15.1:0/2862323178 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> v2:172.21.15.6:3300/0 conn(0x7fe2600cf8e0 0x7fe2600ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25e59c640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe2600cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748049643 0 0) 0x7fe260072050 con 0x7fe2600c41e0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe24c003520 con 0x7fe2600c41e0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4216408734 0 0) 0x7fe24c003520 con 0x7fe2600c41e0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 msgr2=0x7fe2600cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe2600cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 >> v2:172.21.15.6:3300/0 conn(0x7fe2600cf8e0 msgr2=0x7fe2600ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 --2- 172.21.15.1:0/2862323178 >> v2:172.21.15.6:3300/0 conn(0x7fe2600cf8e0 0x7fe2600ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe260072610 con 0x7fe2600c41e0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fe248001670 con 0x7fe2600c41e0 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> v2:172.21.15.6:3300/0 conn(0x7fe2600cf8e0 0x7fe2600ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.080+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 >> v1:172.21.15.1:6789/0 conn(0x7fe2600c41e0 legacy=0x7fe260161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe24c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe24c003f30 0x7fe24c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c004d40 0x7fe24c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2600cd160 con 0x7fe24c004d40 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe260072330 con 0x7fe24c003f30 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c00b9c0 con 0x7fe2600d06f0 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c004d40 0x7fe24c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe257fff640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe24c003f30 0x7fe24c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25e59c640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe24c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 -- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 msgr2=0x7fe24c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2600d06f0 0x7fe24c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 -- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe24c003f30 msgr2=0x7fe24c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe24c003f30 0x7fe24c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.081+0000 7fe25ed9d640 1 -- 172.21.15.1:0/2862323178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c003520 con 0x7fe24c004d40 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c004d40 0x7fe24c0051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e350 tx=0x7fe24c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cd6302f309cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258009de0 con 0x7fe24c004d40 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25802e650 con 0x7fe24c004d40 2024-05-09T06:28:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe25dd9b640 1 -- 172.21.15.1:0/2862323178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25802ea00 con 0x7fe24c004d40 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 -- 172.21.15.1:0/2862323178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c004d40 msgr2=0x7fe24c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 --2- 172.21.15.1:0/2862323178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c004d40 0x7fe24c0051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e350 tx=0x7fe24c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe25ed9d640 1 -- 172.21.15.1:0/2862323178 reap_dead start 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 -- 172.21.15.1:0/2862323178 shutdown_connections 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 -- 172.21.15.1:0/2862323178 >> 172.21.15.1:0/2862323178 conn(0x7fe2600d0b50 msgr2=0x7fe2600d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 -- 172.21.15.1:0/2862323178 shutdown_connections 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.082+0000 7fe264e83640 1 -- 172.21.15.1:0/2862323178 wait complete. 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.084+0000 7fe264e83640 1 Processor -- start 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.084+0000 7fe264e83640 1 -- start start 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe24c004d40 0x7fe26015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26015d000 0x7fe2600a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe26006b1e0 con 0x7fe2600c5190 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe260071ce0 con 0x7fe26015d000 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe26006b360 con 0x7fe24c004d40 2024-05-09T06:28:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36026/0 (socket says 172.21.15.1:36026) 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 -- 172.21.15.1:0/1471389265 learned_addr learned my addr 172.21.15.1:0/1471389265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe25e59c640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe24c004d40 0x7fe26015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe25ed9d640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26015d000 0x7fe2600a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 -- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe24c004d40 msgr2=0x7fe26015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe24c004d40 0x7fe26015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 -- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26015d000 msgr2=0x7fe2600a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26015d000 0x7fe2600a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 -- 172.21.15.1:0/1471389265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2600c5620 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe257fff640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe248007340 tx=0x7fe248007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800f940 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe248018070 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2600c6530 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800fd40 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.085+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2600c6af0 con 0x7fe2600c5190 2024-05-09T06:28:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.086+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26006b6f0 con 0x7fe2600c5190 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.087+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe24801d460 con 0x7fe2600c5190 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.087+0000 7fe256ffd640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 0x7fe24006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.087+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe24808b3f0 con 0x7fe2600c5190 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.087+0000 7fe25e59c640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 0x7fe24006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.088+0000 7fe25e59c640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 0x7fe24006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2500099e0 tx=0x7fe250008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.092+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe26006b6f0 con 0x7fe2600c5190 2024-05-09T06:28:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.276+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe2600c5900 con 0x7fe2600c5190 2024-05-09T06:28:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.276+0000 7fe256ffd640 1 -- 172.21.15.1:0/1471389265 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe248059a30 con 0x7fe2600c5190 2024-05-09T06:28:14.258 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 msgr2=0x7fe24006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 0x7fe24006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2500099e0 tx=0x7fe250008040 comp rx=0 tx=0).stop 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 msgr2=0x7fe2600a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe248007340 tx=0x7fe248007800 comp rx=0 tx=0).stop 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 shutdown_connections 2024-05-09T06:28:14.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24006be30 0x7fe24006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26015d000 0x7fe2600a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2600c5190 0x7fe2600a2e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 --2- 172.21.15.1:0/1471389265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe24c004d40 0x7fe26015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 >> 172.21.15.1:0/1471389265 conn(0x7fe2600d0b50 msgr2=0x7fe26015d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 shutdown_connections 2024-05-09T06:28:14.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.278+0000 7fe264e83640 1 -- 172.21.15.1:0/1471389265 wait complete. 2024-05-09T06:28:14.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:14.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.394+0000 7fdbfb27d640 1 Processor -- start 2024-05-09T06:28:14.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.394+0000 7fdbfb27d640 1 -- start start 2024-05-09T06:28:14.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbfb27d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbf415e610 0x7fdbf4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbfb27d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e9f0 0x7fdbf4161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbfb27d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf40cdfd0 con 0x7fdbf41620b0 2024-05-09T06:28:14.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbfb27d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf40ceb50 con 0x7fdbf415e610 2024-05-09T06:28:14.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbfb27d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf40ce2b0 con 0x7fdbf415e9f0 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf9019640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbf415e610 0x7fdbf4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf981a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdbf41620b0 0x7fdbf40adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60730/0 (socket says 172.21.15.1:60730) 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf981a640 1 -- 172.21.15.1:0/3522326306 learned_addr learned my addr 172.21.15.1:0/3522326306 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf9019640 1 -- 172.21.15.1:0/3522326306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e9f0 msgr2=0x7fdbf4161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf9019640 1 --2- 172.21.15.1:0/3522326306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e9f0 0x7fdbf4161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf9019640 1 -- 172.21.15.1:0/3522326306 >> v1:172.21.15.1:6789/0 conn(0x7fdbf41620b0 legacy=0x7fdbf40adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbf9019640 1 -- 172.21.15.1:0/3522326306 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40ce590 con 0x7fdbf415e610 2024-05-09T06:28:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.395+0000 7fdbebfff640 1 -- 172.21.15.1:0/3522326306 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349699541 0 0) 0x7fdbf40cdfd0 con 0x7fdbf41620b0 2024-05-09T06:28:14.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.396+0000 7fdbf9019640 1 --2- 172.21.15.1:0/3522326306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf415e610 0x7fdbf4070c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00097a0 tx=0x7fdbf002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec63336697fe13bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.396+0000 7fdbebfff640 1 -- 172.21.15.1:0/3522326306 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0037070 con 0x7fdbf415e610 2024-05-09T06:28:14.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.396+0000 7fdbebfff640 1 -- 172.21.15.1:0/3522326306 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf002ae50 con 0x7fdbf415e610 2024-05-09T06:28:14.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.396+0000 7fdbebfff640 1 -- 172.21.15.1:0/3522326306 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0032660 con 0x7fdbf415e610 2024-05-09T06:28:14.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 -- 172.21.15.1:0/3522326306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf415e610 msgr2=0x7fdbf4070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/3522326306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf415e610 0x7fdbf4070c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00097a0 tx=0x7fdbf002a100 comp rx=0 tx=0).stop 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 -- 172.21.15.1:0/3522326306 shutdown_connections 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/3522326306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e9f0 0x7fdbf4161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/3522326306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf415e610 0x7fdbf4070c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 -- 172.21.15.1:0/3522326306 >> 172.21.15.1:0/3522326306 conn(0x7fdbf40cc4b0 msgr2=0x7fdbf40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 -- 172.21.15.1:0/3522326306 shutdown_connections 2024-05-09T06:28:14.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.397+0000 7fdbfb27d640 1 -- 172.21.15.1:0/3522326306 wait complete. 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.398+0000 7fdbfb27d640 1 Processor -- start 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.398+0000 7fdbfb27d640 1 -- start start 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 0x7fdbf415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf415e9f0 0x7fdbf40b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf415ee30 con 0x7fdbf41620b0 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40718c0 con 0x7fdbf415e9f0 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbfb27d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf40b3240 con 0x7fdbf415e610 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36036/0 (socket says 172.21.15.1:36036) 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 -- 172.21.15.1:0/545261850 learned_addr learned my addr 172.21.15.1:0/545261850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:14.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf8818640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf415e9f0 0x7fdbf40b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf9019640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 0x7fdbf415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 -- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 msgr2=0x7fdbf415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 0x7fdbf415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 -- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf415e9f0 msgr2=0x7fdbf40b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf415e9f0 0x7fdbf40b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 -- 172.21.15.1:0/545261850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40b3520 con 0x7fdbf41620b0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf9019640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 0x7fdbf415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.399+0000 7fdbf981a640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c7b0 tx=0x7fdbec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.400+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec018070 con 0x7fdbf41620b0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.400+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec00a640 con 0x7fdbf41620b0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.400+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00aa70 con 0x7fdbf41620b0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.400+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf40b3800 con 0x7fdbf41620b0 2024-05-09T06:28:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.400+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf4147480 con 0x7fdbf41620b0 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.401+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf40752b0 con 0x7fdbf41620b0 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.401+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbec00ac10 con 0x7fdbf41620b0 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.402+0000 7fdbe9ffb640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 0x7fdbd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.402+0000 7fdbf9019640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 0x7fdbd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.402+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdbec08a930 con 0x7fdbf41620b0 2024-05-09T06:28:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.402+0000 7fdbf9019640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 0x7fdbd806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0034000 tx=0x7fdbf002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.406+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbec0587f0 con 0x7fdbf41620b0 2024-05-09T06:28:14.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.600+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdbf40af250 con 0x7fdbf41620b0 2024-05-09T06:28:14.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.600+0000 7fdbe9ffb640 1 -- 172.21.15.1:0/545261850 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fdbec05caf0 con 0x7fdbf41620b0 2024-05-09T06:28:14.582 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 msgr2=0x7fdbd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 0x7fdbd806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0034000 tx=0x7fdbf002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 msgr2=0x7fdbf406dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c7b0 tx=0x7fdbec00cc70 comp rx=0 tx=0).stop 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 shutdown_connections 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbd806be30 0x7fdbd806e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf41620b0 0x7fdbf406dd80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf415e9f0 0x7fdbf40b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 --2- 172.21.15.1:0/545261850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbf415e610 0x7fdbf415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 >> 172.21.15.1:0/545261850 conn(0x7fdbf40cc4b0 msgr2=0x7fdbf40b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 shutdown_connections 2024-05-09T06:28:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.602+0000 7fdbfb27d640 1 -- 172.21.15.1:0/545261850 wait complete. 2024-05-09T06:28:14.595 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:14.595 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:14.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.773+0000 7f77ac2d4640 1 Processor -- start 2024-05-09T06:28:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.773+0000 7f77ac2d4640 1 -- start start 2024-05-09T06:28:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77ac2d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a40ce4d0 0x7f77a40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77ac2d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f77a415e340 0x7f77a40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77ac2d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a406d070 con 0x7f77a40ccb40 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77ac2d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f77a40cf330 con 0x7f77a40ce4d0 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77ac2d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f77a406d350 con 0x7f77a415e340 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77aa070640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a40ce4d0 0x7f77a40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77aa070640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a40ce4d0 0x7f77a40ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36042/0 (socket says 172.21.15.1:36042) 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.774+0000 7f77aa070640 1 -- 172.21.15.1:0/2574978984 learned_addr learned my addr 172.21.15.1:0/2574978984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77aa070640 1 -- 172.21.15.1:0/2574978984 >> v2:172.21.15.6:3300/0 conn(0x7f77a415e340 msgr2=0x7f77a40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77aa070640 1 --2- 172.21.15.1:0/2574978984 >> v2:172.21.15.6:3300/0 conn(0x7f77a415e340 0x7f77a40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77aa070640 1 -- 172.21.15.1:0/2574978984 >> v1:172.21.15.6:6789/0 conn(0x7f77a40ccb40 legacy=0x7f77a40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77aa070640 1 -- 172.21.15.1:0/2574978984 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a40cc340 con 0x7f77a40ce4d0 2024-05-09T06:28:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77aa070640 1 --2- 172.21.15.1:0/2574978984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a40ce4d0 0x7f77a40ce8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77a40722f0 tx=0x7f77a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca90a22490dfb1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.775+0000 7f77a906e640 1 -- 172.21.15.1:0/2574978984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037070 con 0x7f77a40ce4d0 2024-05-09T06:28:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77a906e640 1 -- 172.21.15.1:0/2574978984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a0032440 con 0x7f77a40ce4d0 2024-05-09T06:28:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77a906e640 1 -- 172.21.15.1:0/2574978984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032810 con 0x7f77a40ce4d0 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 -- 172.21.15.1:0/2574978984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a40ce4d0 msgr2=0x7f77a40ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/2574978984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a40ce4d0 0x7f77a40ce8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77a40722f0 tx=0x7f77a002a230 comp rx=0 tx=0).stop 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 -- 172.21.15.1:0/2574978984 shutdown_connections 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/2574978984 >> v2:172.21.15.6:3300/0 conn(0x7f77a415e340 0x7f77a40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/2574978984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a40ce4d0 0x7f77a40ce8b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 -- 172.21.15.1:0/2574978984 >> 172.21.15.1:0/2574978984 conn(0x7f77a40d1c50 msgr2=0x7f77a40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 -- 172.21.15.1:0/2574978984 shutdown_connections 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.776+0000 7f77ac2d4640 1 -- 172.21.15.1:0/2574978984 wait complete. 2024-05-09T06:28:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.777+0000 7f77ac2d4640 1 Processor -- start 2024-05-09T06:28:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.777+0000 7f77ac2d4640 1 -- start start 2024-05-09T06:28:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a40ccb40 0x7f77a4159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a40ce4d0 0x7f77a415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77a406cd00 con 0x7f77a4160510 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77a406d630 con 0x7f77a40ccb40 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77ac2d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77a40d0ea0 con 0x7f77a40ce4d0 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77aa871640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77aa871640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36056/0 (socket says 172.21.15.1:36056) 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77aa871640 1 -- 172.21.15.1:0/3830027302 learned_addr learned my addr 172.21.15.1:0/3830027302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.778+0000 7f77aa070640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a40ccb40 0x7f77a4159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77a986f640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a40ce4d0 0x7f77a415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 -- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a40ce4d0 msgr2=0x7f77a415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a40ce4d0 0x7f77a415ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 -- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a40ccb40 msgr2=0x7f77a4159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a40ccb40 0x7f77a4159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 -- 172.21.15.1:0/3830027302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a415a380 con 0x7f77a4160510 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77aa871640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c780 tx=0x7f779c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x7f77a4160510 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c003da0 con 0x7f77a4160510 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0052b0 con 0x7f77a4160510 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a40c8d20 con 0x7f77a4160510 2024-05-09T06:28:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.779+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a40c92e0 con 0x7f77a4160510 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.780+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a40752b0 con 0x7f77a4160510 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.781+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f779c01a070 con 0x7f77a4160510 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.781+0000 7f779affd640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 0x7f778806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.781+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f779c08a730 con 0x7f77a4160510 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.781+0000 7f77aa070640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 0x7f778806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.782+0000 7f77aa070640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 0x7f778806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.786+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f779c058720 con 0x7f77a4160510 2024-05-09T06:28:14.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.969+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77a40cbff0 con 0x7f77a4160510 2024-05-09T06:28:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.970+0000 7f779affd640 1 -- 172.21.15.1:0/3830027302 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f779c05ca20 con 0x7f77a4160510 2024-05-09T06:28:14.953 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:14.953 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 msgr2=0x7f778806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 0x7f778806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 msgr2=0x7f77a40c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c780 tx=0x7f779c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 shutdown_connections 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f778806be60 0x7f778806e320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a4160510 0x7f77a40c9f90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a40ce4d0 0x7f77a415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 --2- 172.21.15.1:0/3830027302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a40ccb40 0x7f77a4159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.972+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 >> 172.21.15.1:0/3830027302 conn(0x7f77a40d1c50 msgr2=0x7f77a406ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.973+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 shutdown_connections 2024-05-09T06:28:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:14.973+0000 7f77ac2d4640 1 -- 172.21.15.1:0/3830027302 wait complete. 2024-05-09T06:28:14.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:14.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:15.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.091+0000 7ffb72641640 1 Processor -- start 2024-05-09T06:28:15.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.091+0000 7ffb72641640 1 -- start start 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb72641640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 0x7ffb6c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb72641640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb72641640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb6c15b420 con 0x7ffb6c158a50 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb72641640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6c158e30 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb72641640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6c072480 con 0x7ffb6c071fd0 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36062/0 (socket says 172.21.15.1:36062) 2024-05-09T06:28:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 -- 172.21.15.1:0/340572570 learned_addr learned my addr 172.21.15.1:0/340572570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb6b7fe640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 0x7ffb6c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 -- 172.21.15.1:0/340572570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 msgr2=0x7ffb6c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 0x7ffb6c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 -- 172.21.15.1:0/340572570 >> v1:172.21.15.6:6789/0 conn(0x7ffb6c158a50 legacy=0x7ffb6c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb70bde640 1 -- 172.21.15.1:0/340572570 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c15b6e0 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.092+0000 7ffb6b7fe640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 0x7ffb6c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.093+0000 7ffb70bde640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e50 tx=0x7ffb6402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf31a2e50add7d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.093+0000 7ffb6affd640 1 -- 172.21.15.1:0/340572570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64036070 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.093+0000 7ffb6affd640 1 -- 172.21.15.1:0/340572570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb64031600 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.093+0000 7ffb6affd640 1 -- 172.21.15.1:0/340572570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640319d0 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 -- 172.21.15.1:0/340572570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c15c4a0 msgr2=0x7ffb6c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e50 tx=0x7ffb6402a710 comp rx=0 tx=0).stop 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 -- 172.21.15.1:0/340572570 shutdown_connections 2024-05-09T06:28:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c15c4a0 0x7ffb6c06ec10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 --2- 172.21.15.1:0/340572570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c071fd0 0x7ffb6c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 -- 172.21.15.1:0/340572570 >> 172.21.15.1:0/340572570 conn(0x7ffb6c0cc4d0 msgr2=0x7ffb6c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 -- 172.21.15.1:0/340572570 shutdown_connections 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.094+0000 7ffb72641640 1 -- 172.21.15.1:0/340572570 wait complete. 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.095+0000 7ffb72641640 1 Processor -- start 2024-05-09T06:28:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.095+0000 7ffb72641640 1 -- start start 2024-05-09T06:28:15.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb6c071fd0 0x7ffb6c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c15c4a0 0x7ffb6c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c15bf80 con 0x7ffb6c158a50 2024-05-09T06:28:15.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c15bca0 con 0x7ffb6c071fd0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb72641640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6c070ea0 con 0x7ffb6c15c4a0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36068/0 (socket says 172.21.15.1:36068) 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 -- 172.21.15.1:0/4095607797 learned_addr learned my addr 172.21.15.1:0/4095607797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb70bde640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c15c4a0 0x7ffb6c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6bfff640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb6c071fd0 0x7ffb6c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 -- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c15c4a0 msgr2=0x7ffb6c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c15c4a0 0x7ffb6c06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 -- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb6c071fd0 msgr2=0x7ffb6c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb6c071fd0 0x7ffb6c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.096+0000 7ffb6b7fe640 1 -- 172.21.15.1:0/4095607797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c0b3c40 con 0x7ffb6c158a50 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb6b7fe640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e840 tx=0x7ffb6000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000f040 con 0x7ffb6c158a50 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb60004030 con 0x7ffb6c158a50 2024-05-09T06:28:15.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c0b3f20 con 0x7ffb6c158a50 2024-05-09T06:28:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004400 con 0x7ffb6c158a50 2024-05-09T06:28:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.097+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c0b44e0 con 0x7ffb6c158a50 2024-05-09T06:28:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.098+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c0752b0 con 0x7ffb6c158a50 2024-05-09T06:28:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.098+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb6000c6d0 con 0x7ffb6c158a50 2024-05-09T06:28:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.099+0000 7ffb68ff9640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 0x7ffb5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.099+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ffb6008ae50 con 0x7ffb6c158a50 2024-05-09T06:28:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.099+0000 7ffb6bfff640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 0x7ffb5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.099+0000 7ffb6bfff640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 0x7ffb5806e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c15ed40 tx=0x7ffb5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.103+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb60017020 con 0x7ffb6c158a50 2024-05-09T06:28:15.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.288+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb6c072cb0 con 0x7ffb6c158a50 2024-05-09T06:28:15.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.289+0000 7ffb68ff9640 1 -- 172.21.15.1:0/4095607797 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ffb60058e40 con 0x7ffb6c158a50 2024-05-09T06:28:15.271 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:15.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 msgr2=0x7ffb5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 0x7ffb5806e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c15ed40 tx=0x7ffb5c009040 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 msgr2=0x7ffb6c151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e840 tx=0x7ffb6000ed00 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 shutdown_connections 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb5806be30 0x7ffb5806e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb6c15c4a0 0x7ffb6c06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb6c158a50 0x7ffb6c151cb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 --2- 172.21.15.1:0/4095607797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb6c071fd0 0x7ffb6c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 >> 172.21.15.1:0/4095607797 conn(0x7ffb6c0cc4d0 msgr2=0x7ffb6c0cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 shutdown_connections 2024-05-09T06:28:15.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.291+0000 7ffb72641640 1 -- 172.21.15.1:0/4095607797 wait complete. 2024-05-09T06:28:15.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.409+0000 7fa47f95a640 1 Processor -- start 2024-05-09T06:28:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.409+0000 7fa47f95a640 1 -- start start 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47f95a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa4780b4ff0 0x7fa478071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47f95a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4780cd590 0x7fa4780cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47f95a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa47806d160 con 0x7fa478162a20 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47f95a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa47806eb50 con 0x7fa4780b4ff0 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47f95a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa47806ee30 con 0x7fa4780cd590 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47d6f6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa4780b4ff0 0x7fa478071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47def7640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa478162a20 0x7fa478162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60738/0 (socket says 172.21.15.1:60738) 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47def7640 1 -- 172.21.15.1:0/580051181 learned_addr learned my addr 172.21.15.1:0/580051181 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.410+0000 7fa47cef5640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4780cd590 0x7fa4780cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa47d6f6640 1 -- 172.21.15.1:0/580051181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4780cd590 msgr2=0x7fa4780cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa47d6f6640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4780cd590 0x7fa4780cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa47d6f6640 1 -- 172.21.15.1:0/580051181 >> v1:172.21.15.1:6789/0 conn(0x7fa478162a20 legacy=0x7fa478162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa47d6f6640 1 -- 172.21.15.1:0/580051181 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47806e820 con 0x7fa4780b4ff0 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa477fff640 1 -- 172.21.15.1:0/580051181 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079687812 0 0) 0x7fa47806d160 con 0x7fa478162a20 2024-05-09T06:28:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.411+0000 7fa47d6f6640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478071a00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097a0 tx=0x7fa47002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc371511d7de36e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa477fff640 1 -- 172.21.15.1:0/580051181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470037070 con 0x7fa4780b4ff0 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa477fff640 1 -- 172.21.15.1:0/580051181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47002ae50 con 0x7fa4780b4ff0 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa477fff640 1 -- 172.21.15.1:0/580051181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470032660 con 0x7fa4780b4ff0 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 -- 172.21.15.1:0/580051181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 msgr2=0x7fa478071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478071a00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097a0 tx=0x7fa47002a100 comp rx=0 tx=0).stop 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 -- 172.21.15.1:0/580051181 shutdown_connections 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4780cd590 0x7fa4780cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 --2- 172.21.15.1:0/580051181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478071a00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 -- 172.21.15.1:0/580051181 >> 172.21.15.1:0/580051181 conn(0x7fa4780cc4d0 msgr2=0x7fa4780cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 -- 172.21.15.1:0/580051181 shutdown_connections 2024-05-09T06:28:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.412+0000 7fa47f95a640 1 -- 172.21.15.1:0/580051181 wait complete. 2024-05-09T06:28:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.413+0000 7fa47f95a640 1 Processor -- start 2024-05-09T06:28:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 -- start start 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 0x7fa4781fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 0x7fa4781fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780720c0 con 0x7fa4780b4ff0 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa478071f40 con 0x7fa4780cd590 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47f95a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780b53d0 con 0x7fa478162a20 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47d6f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47d6f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36092/0 (socket says 172.21.15.1:36092) 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.414+0000 7fa47d6f6640 1 -- 172.21.15.1:0/134073242 learned_addr learned my addr 172.21.15.1:0/134073242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:15.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47def7640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 0x7fa4781fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47cef5640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 0x7fa4781fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 -- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 msgr2=0x7fa4781fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 0x7fa4781fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 -- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 msgr2=0x7fa4781fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 0x7fa4781fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 -- 172.21.15.1:0/134073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa478165670 con 0x7fa4780b4ff0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47def7640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 0x7fa4781fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47cef5640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 0x7fa4781fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47d6f6640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa470035040 tx=0x7fa4700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470041070 con 0x7fa4780b4ff0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47003c670 con 0x7fa4780b4ff0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa478165950 con 0x7fa4780b4ff0 2024-05-09T06:28:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470037040 con 0x7fa4780b4ff0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.415+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa478165f10 con 0x7fa4780b4ff0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.416+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4780752b0 con 0x7fa4780b4ff0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.417+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa47003a070 con 0x7fa4780b4ff0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.417+0000 7fa475ffb640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 0x7fa46406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.417+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa4700ab590 con 0x7fa4780b4ff0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.417+0000 7fa47cef5640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 0x7fa46406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.418+0000 7fa47cef5640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 0x7fa46406e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006fd0 tx=0x7fa46c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.422+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa470079c60 con 0x7fa4780b4ff0 2024-05-09T06:28:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.602+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4780cf280 con 0x7fa4780b4ff0 2024-05-09T06:28:15.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.603+0000 7fa475ffb640 1 -- 172.21.15.1:0/134073242 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa470079560 con 0x7fa4780b4ff0 2024-05-09T06:28:15.585 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 msgr2=0x7fa46406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 0x7fa46406e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006fd0 tx=0x7fa46c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 msgr2=0x7fa478153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa470035040 tx=0x7fa4700046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 shutdown_connections 2024-05-09T06:28:15.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa46406be60 0x7fa46406e320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa478162a20 0x7fa4781fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4780cd590 0x7fa4781fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 --2- 172.21.15.1:0/134073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4780b4ff0 0x7fa478153b10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 >> 172.21.15.1:0/134073242 conn(0x7fa4780cc4d0 msgr2=0x7fa4780b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 shutdown_connections 2024-05-09T06:28:15.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.605+0000 7fa47f95a640 1 -- 172.21.15.1:0/134073242 wait complete. 2024-05-09T06:28:15.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:15.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.722+0000 7f3400b42640 1 Processor -- start 2024-05-09T06:28:15.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.722+0000 7f3400b42640 1 -- start start 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f3400b42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc0b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f3400b42640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33fc0d2760 con 0x7f33fc15e0f0 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f3400b42640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33fc0d2a40 con 0x7f33fc0d0a10 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f3400b42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc0d4f70 con 0x7f33fc0d0e40 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f33fa59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f33fc15e0f0 0x7f33fc0d4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60750/0 (socket says 172.21.15.1:60750) 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f33fa59c640 1 -- 172.21.15.1:0/524029540 learned_addr learned my addr 172.21.15.1:0/524029540 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.723+0000 7f33fad9d640 1 --2- 172.21.15.1:0/524029540 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc0b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 255157992 0 0) 0x7f33fc0d2760 con 0x7f33fc15e0f0 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33e4003520 con 0x7f33fc15e0f0 2024-05-09T06:28:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861449428 0 0) 0x7f33fc0d2a40 con 0x7f33fc0d0a10 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33fc0d2760 con 0x7f33fc0d0a10 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33fad9d640 1 -- 172.21.15.1:0/524029540 >> v1:172.21.15.6:6789/0 conn(0x7f33fc0d0a10 legacy=0x7f33fc0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33fad9d640 1 -- 172.21.15.1:0/524029540 >> v1:172.21.15.1:6789/0 conn(0x7f33fc15e0f0 legacy=0x7f33fc0d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 855137026 0 0) 0x7f33e4003520 con 0x7f33fc15e0f0 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.724+0000 7f33fad9d640 1 -- 172.21.15.1:0/524029540 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc0d2d20 con 0x7f33fc0d0e40 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f33fad9d640 1 --2- 172.21.15.1:0/524029540 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc0b7cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009dc0 tx=0x7f33f402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cddf26ceb36046da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec001670 con 0x7f33fc0d0e40 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f4007540 con 0x7f33fc0d0e40 2024-05-09T06:28:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f33f959a640 1 -- 172.21.15.1:0/524029540 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f40078f0 con 0x7f33fc0d0e40 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 -- 172.21.15.1:0/524029540 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 msgr2=0x7f33fc0b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 --2- 172.21.15.1:0/524029540 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc0b7cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009dc0 tx=0x7f33f402a660 comp rx=0 tx=0).stop 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 -- 172.21.15.1:0/524029540 shutdown_connections 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 --2- 172.21.15.1:0/524029540 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc0b7cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 -- 172.21.15.1:0/524029540 >> 172.21.15.1:0/524029540 conn(0x7f33fc09d3b0 msgr2=0x7f33fc0d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 -- 172.21.15.1:0/524029540 shutdown_connections 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.725+0000 7f3400b42640 1 -- 172.21.15.1:0/524029540 wait complete. 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.727+0000 7f3400b42640 1 Processor -- start 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.727+0000 7f3400b42640 1 -- start start 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33fc15e0f0 0x7f33fc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc0d1260 con 0x7f33fc0d0a10 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc0b6e50 con 0x7f33fc15e0f0 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc15e4d0 con 0x7f33fc0d0e40 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36098/0 (socket says 172.21.15.1:36098) 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 -- 172.21.15.1:0/1836875550 learned_addr learned my addr 172.21.15.1:0/1836875550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33f9d9b640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fad9d640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33fc15e0f0 0x7f33fc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 -- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 msgr2=0x7f33fc15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 -- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33fc15e0f0 msgr2=0x7f33fc157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33fc15e0f0 0x7f33fc157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 -- 172.21.15.1:0/1836875550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc0cc620 con 0x7f33fc0d0a10 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33fa59c640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c4f0 tx=0x7f33ec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec017070 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ec005ce0 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc0cc900 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec007290 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.728+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc0ccec0 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.730+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc0752b0 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.730+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33ec01a070 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.730+0000 7f33eaffd640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 0x7f33dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.731+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f33ec08a6c0 con 0x7f33fc0d0a10 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.731+0000 7f33f9d9b640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 0x7f33dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.731+0000 7f33f9d9b640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 0x7f33dc06e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33fc15cfb0 tx=0x7f33f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.735+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33ec058620 con 0x7f33fc0d0a10 2024-05-09T06:28:15.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.918+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33fc0d4c10 con 0x7f33fc0d0a10 2024-05-09T06:28:15.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.918+0000 7f33eaffd640 1 -- 172.21.15.1:0/1836875550 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f33ec05c920 con 0x7f33fc0d0a10 2024-05-09T06:28:15.901 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 msgr2=0x7f33dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 0x7f33dc06e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33fc15cfb0 tx=0x7f33f0009040 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 msgr2=0x7f33fc0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c4f0 tx=0x7f33ec00c9b0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 shutdown_connections 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33dc06be30 0x7f33dc06e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33fc15e0f0 0x7f33fc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33fc0d0e40 0x7f33fc15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 --2- 172.21.15.1:0/1836875550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33fc0d0a10 0x7f33fc0cd6c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 >> 172.21.15.1:0/1836875550 conn(0x7f33fc09d3b0 msgr2=0x7f33fc0b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.920+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 shutdown_connections 2024-05-09T06:28:15.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:15.921+0000 7f3400b42640 1 -- 172.21.15.1:0/1836875550 wait complete. 2024-05-09T06:28:15.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:16.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 Processor -- start 2024-05-09T06:28:16.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 -- start start 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fad5415e270 0x7fad540cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad540b1340 con 0x7fad540d08b0 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad540d1020 con 0x7fad540ceba0 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.039+0000 7fad5afc8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fad540cc260 con 0x7fad5415e270 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad58d64640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fad5415e270 0x7fad540cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad53fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fad540d08b0 0x7fad540ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60760/0 (socket says 172.21.15.1:60760) 2024-05-09T06:28:16.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad53fff640 1 -- 172.21.15.1:0/2808524595 learned_addr learned my addr 172.21.15.1:0/2808524595 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:16.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad58d64640 1 -- 172.21.15.1:0/2808524595 >> v1:172.21.15.6:6789/0 conn(0x7fad540ceba0 legacy=0x7fad540ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad58d64640 1 -- 172.21.15.1:0/2808524595 >> v1:172.21.15.1:6789/0 conn(0x7fad540d08b0 legacy=0x7fad540ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad58d64640 1 -- 172.21.15.1:0/2808524595 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad540cc4b0 con 0x7fad5415e270 2024-05-09T06:28:16.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.040+0000 7fad537fe640 1 -- 172.21.15.1:0/2808524595 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623881677 0 0) 0x7fad40001110 con 0x7fad540d08b0 2024-05-09T06:28:16.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad58d64640 1 --2- 172.21.15.1:0/2808524595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415e270 0x7fad540cf180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad480097a0 tx=0x7fad4802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cbd3a21c3a6b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad537fe640 1 -- 172.21.15.1:0/2808524595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037070 con 0x7fad5415e270 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad537fe640 1 -- 172.21.15.1:0/2808524595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48032640 con 0x7fad5415e270 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad537fe640 1 -- 172.21.15.1:0/2808524595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48032a10 con 0x7fad5415e270 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 -- 172.21.15.1:0/2808524595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415e270 msgr2=0x7fad540cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 --2- 172.21.15.1:0/2808524595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415e270 0x7fad540cf180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad480097a0 tx=0x7fad4802a2c0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 -- 172.21.15.1:0/2808524595 shutdown_connections 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 --2- 172.21.15.1:0/2808524595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415e270 0x7fad540cf180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 -- 172.21.15.1:0/2808524595 >> 172.21.15.1:0/2808524595 conn(0x7fad540d1c10 msgr2=0x7fad540d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 -- 172.21.15.1:0/2808524595 shutdown_connections 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.041+0000 7fad5afc8640 1 -- 172.21.15.1:0/2808524595 wait complete. 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 Processor -- start 2024-05-09T06:28:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 -- start start 2024-05-09T06:28:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad540ceba0 0x7fad540cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5415e270 0x7fad54149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad540d0d20 con 0x7fad540d08b0 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad540cca70 con 0x7fad540ceba0 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.043+0000 7fad5afc8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad5415e650 con 0x7fad5415e270 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36108/0 (socket says 172.21.15.1:36108) 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 -- 172.21.15.1:0/1042985542 learned_addr learned my addr 172.21.15.1:0/1042985542 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad58d64640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad540ceba0 0x7fad540cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad59565640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5415e270 0x7fad54149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 -- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5415e270 msgr2=0x7fad54149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5415e270 0x7fad54149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 -- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad540ceba0 msgr2=0x7fad540cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad540ceba0 0x7fad540cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 -- 172.21.15.1:0/1042985542 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad540ca630 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad53fff640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c7b0 tx=0x7fad4000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40018070 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad400072d0 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad540ca910 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400047e0 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.044+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad540caed0 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.045+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad540752b0 con 0x7fad540d08b0 2024-05-09T06:28:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.046+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad4001b070 con 0x7fad540d08b0 2024-05-09T06:28:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.046+0000 7fad517fa640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.046+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fad4008a540 con 0x7fad540d08b0 2024-05-09T06:28:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.046+0000 7fad58d64640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.047+0000 7fad58d64640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 0x7fad3806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.050+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad540b1340 con 0x7fad540d08b0 2024-05-09T06:28:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.233+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fad540b7920 con 0x7fad540d08b0 2024-05-09T06:28:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.233+0000 7fad517fa640 1 -- 172.21.15.1:0/1042985542 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fad40058af0 con 0x7fad540d08b0 2024-05-09T06:28:16.216 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.235+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 msgr2=0x7fad3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.235+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 0x7fad3806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad4802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 msgr2=0x7fad540c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c7b0 tx=0x7fad4000cc70 comp rx=0 tx=0).stop 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 shutdown_connections 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5415e270 0x7fad54149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad540d08b0 0x7fad540c96f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 --2- 172.21.15.1:0/1042985542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad540ceba0 0x7fad540cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 >> 172.21.15.1:0/1042985542 conn(0x7fad540d1c10 msgr2=0x7fad540ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:16.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 shutdown_connections 2024-05-09T06:28:16.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.236+0000 7fad5afc8640 1 -- 172.21.15.1:0/1042985542 wait complete. 2024-05-09T06:28:16.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:16.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.352+0000 7f3831209640 1 Processor -- start 2024-05-09T06:28:16.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.352+0000 7f3831209640 1 -- start start 2024-05-09T06:28:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3831209640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f382c161950 0x7f382c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3831209640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f382c0d4940 0x7f382c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3831209640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382c0b33c0 con 0x7f382c0d0f80 2024-05-09T06:28:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3831209640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f382c0d4dd0 con 0x7f382c0d4940 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3831209640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f382c0b3660 con 0x7f382c161950 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f382c0d4940 0x7f382c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f382c0d4940 0x7f382c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36114/0 (socket says 172.21.15.1:36114) 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 -- 172.21.15.1:0/1470005254 learned_addr learned my addr 172.21.15.1:0/1470005254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f382ad9d640 1 --2- 172.21.15.1:0/1470005254 >> v2:172.21.15.6:3300/0 conn(0x7f382c161950 0x7f382c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 -- 172.21.15.1:0/1470005254 >> v2:172.21.15.6:3300/0 conn(0x7f382c161950 msgr2=0x7f382c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 --2- 172.21.15.1:0/1470005254 >> v2:172.21.15.6:3300/0 conn(0x7f382c161950 0x7f382c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 -- 172.21.15.1:0/1470005254 >> v1:172.21.15.6:6789/0 conn(0x7f382c0d0f80 legacy=0x7f382c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.353+0000 7f3823fff640 1 -- 172.21.15.1:0/1470005254 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c0b3940 con 0x7f382c0d4940 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.354+0000 7f3823fff640 1 --2- 172.21.15.1:0/1470005254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c0d4940 0x7f382c0d2940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f382c06e3d0 tx=0x7f381402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cc91218566d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.354+0000 7f382a59c640 1 -- 172.21.15.1:0/1470005254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814037070 con 0x7f382c0d4940 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.354+0000 7f382a59c640 1 -- 172.21.15.1:0/1470005254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3814032440 con 0x7f382c0d4940 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.354+0000 7f382a59c640 1 -- 172.21.15.1:0/1470005254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814032810 con 0x7f382c0d4940 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 -- 172.21.15.1:0/1470005254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c0d4940 msgr2=0x7f382c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 --2- 172.21.15.1:0/1470005254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c0d4940 0x7f382c0d2940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f382c06e3d0 tx=0x7f381402a200 comp rx=0 tx=0).stop 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 -- 172.21.15.1:0/1470005254 shutdown_connections 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 --2- 172.21.15.1:0/1470005254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c0d4940 0x7f382c0d2940 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 --2- 172.21.15.1:0/1470005254 >> v2:172.21.15.6:3300/0 conn(0x7f382c161950 0x7f382c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 -- 172.21.15.1:0/1470005254 >> 172.21.15.1:0/1470005254 conn(0x7f382c0d3e20 msgr2=0x7f382c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 -- 172.21.15.1:0/1470005254 shutdown_connections 2024-05-09T06:28:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.355+0000 7f3831209640 1 -- 172.21.15.1:0/1470005254 wait complete. 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.356+0000 7f3831209640 1 Processor -- start 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.356+0000 7f3831209640 1 -- start start 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 0x7f382c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 0x7f382c15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f382c0b52d0 con 0x7f382c161950 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f382c0b5150 con 0x7f382c0d0f80 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3831209640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f382c0b5450 con 0x7f382c15dc70 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36120/0 (socket says 172.21.15.1:36120) 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 -- 172.21.15.1:0/2813731208 learned_addr learned my addr 172.21.15.1:0/2813731208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f382b59e640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 0x7f382c15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 -- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 msgr2=0x7f382c15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f382ad9d640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 0x7f382c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 0x7f382c15aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.357+0000 7f3823fff640 1 -- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 msgr2=0x7f382c1f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f3823fff640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 0x7f382c1f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f3823fff640 1 -- 172.21.15.1:0/2813731208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c0cc070 con 0x7f382c161950 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f382b59e640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 0x7f382c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f382ad9d640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 0x7f382c1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:16.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f3823fff640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f38140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814041020 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38140306c0 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c0cc350 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814037040 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.358+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c0cc910 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.359+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382c0b3190 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.359+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3814044020 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.360+0000 7f38237fe640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 0x7f380c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.360+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f38140aba60 con 0x7f382c161950 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.360+0000 7f382ad9d640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 0x7f380c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.360+0000 7f382ad9d640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 0x7f380c06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.364+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f382c0b3190 con 0x7f382c161950 2024-05-09T06:28:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.564+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f382c0752b0 con 0x7f382c161950 2024-05-09T06:28:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.564+0000 7f38237fe640 1 -- 172.21.15.1:0/2813731208 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f381404b030 con 0x7f382c161950 2024-05-09T06:28:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 msgr2=0x7f380c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 0x7f380c06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f381c006fd0 tx=0x7f381c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 msgr2=0x7f382c1f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f38140046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 shutdown_connections 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f380c06be30 0x7f380c06e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f382c15dc70 0x7f382c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f382c161950 0x7f382c1f50a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 --2- 172.21.15.1:0/2813731208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c0d0f80 0x7f382c1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 >> 172.21.15.1:0/2813731208 conn(0x7f382c0d3e20 msgr2=0x7f382c160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:16.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.566+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 shutdown_connections 2024-05-09T06:28:16.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.567+0000 7f3831209640 1 -- 172.21.15.1:0/2813731208 wait complete. 2024-05-09T06:28:16.560 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:28:16.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:16.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:16.586 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:16.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:28:16.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:16.587 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:28:16.643 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:16.643 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:16.664 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:16.664 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:16.723 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:16.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.845+0000 7fcfda69e640 1 Processor -- start 2024-05-09T06:28:16.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 -- start start 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcfd40d4940 0x7fcfd40b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcfd40d2b10 0x7fcfd40b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd40b33c0 con 0x7fcfd4161950 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd40d4dd0 con 0x7fcfd40d2b10 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfda69e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd40b3660 con 0x7fcfd40d4940 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfd8c3b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcfd40d2b10 0x7fcfd40b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.846+0000 7fcfd3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcfd4161950 0x7fcfd40d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60764/0 (socket says 172.21.15.1:60764) 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd3fff640 1 -- 172.21.15.1:0/3627066788 learned_addr learned my addr 172.21.15.1:0/3627066788 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd8c3b640 1 -- 172.21.15.1:0/3627066788 >> v2:172.21.15.6:3300/0 conn(0x7fcfd40d4940 msgr2=0x7fcfd40b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd8c3b640 1 --2- 172.21.15.1:0/3627066788 >> v2:172.21.15.6:3300/0 conn(0x7fcfd40d4940 0x7fcfd40b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd8c3b640 1 -- 172.21.15.1:0/3627066788 >> v1:172.21.15.1:6789/0 conn(0x7fcfd4161950 legacy=0x7fcfd40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd8c3b640 1 -- 172.21.15.1:0/3627066788 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40b3940 con 0x7fcfd40d2b10 2024-05-09T06:28:16.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.847+0000 7fcfd2ffd640 1 -- 172.21.15.1:0/3627066788 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242735245 0 0) 0x7fcfd40b33c0 con 0x7fcfd4161950 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfd8c3b640 1 --2- 172.21.15.1:0/3627066788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b4680 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcfd406e3d0 tx=0x7fcfcc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6b2768b827f25b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfd2ffd640 1 -- 172.21.15.1:0/3627066788 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc007740 con 0x7fcfd40d2b10 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfd2ffd640 1 -- 172.21.15.1:0/3627066788 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfcc0078e0 con 0x7fcfd40d2b10 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfd2ffd640 1 -- 172.21.15.1:0/3627066788 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc007cb0 con 0x7fcfd40d2b10 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 -- 172.21.15.1:0/3627066788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd40d2b10 msgr2=0x7fcfd40b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 --2- 172.21.15.1:0/3627066788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b4680 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcfd406e3d0 tx=0x7fcfcc02a680 comp rx=0 tx=0).stop 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 -- 172.21.15.1:0/3627066788 shutdown_connections 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 --2- 172.21.15.1:0/3627066788 >> v2:172.21.15.6:3300/0 conn(0x7fcfd40d4940 0x7fcfd40b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 --2- 172.21.15.1:0/3627066788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b4680 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 -- 172.21.15.1:0/3627066788 >> 172.21.15.1:0/3627066788 conn(0x7fcfd40d3e20 msgr2=0x7fcfd40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:16.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 -- 172.21.15.1:0/3627066788 shutdown_connections 2024-05-09T06:28:16.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.848+0000 7fcfda69e640 1 -- 172.21.15.1:0/3627066788 wait complete. 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.850+0000 7fcfda69e640 1 Processor -- start 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.850+0000 7fcfda69e640 1 -- start start 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.850+0000 7fcfda69e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.850+0000 7fcfda69e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 0x7fcfd41f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40d1100 con 0x7fcfd4161950 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40d0f80 con 0x7fcfd40d2b10 2024-05-09T06:28:16.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd40d1c00 con 0x7fcfd40b6f10 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36140/0 (socket says 172.21.15.1:36140) 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 -- 172.21.15.1:0/1485522761 learned_addr learned my addr 172.21.15.1:0/1485522761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd3fff640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd8c3b640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 0x7fcfd41f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 -- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 msgr2=0x7fcfd41f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 0x7fcfd41f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 -- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 msgr2=0x7fcfd40b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 -- 172.21.15.1:0/1485522761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40b8a50 con 0x7fcfd4161950 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd3fff640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd8c3b640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 0x7fcfd41f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd37fe640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b4a0 tx=0x7fcfc800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8014070 con 0x7fcfd4161950 2024-05-09T06:28:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc8010610 con 0x7fcfd4161950 2024-05-09T06:28:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd40b7470 con 0x7fcfd4161950 2024-05-09T06:28:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8010a00 con 0x7fcfd4161950 2024-05-09T06:28:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.851+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd40b7d10 con 0x7fcfd4161950 2024-05-09T06:28:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.852+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd40752b0 con 0x7fcfd4161950 2024-05-09T06:28:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.853+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcfc8017070 con 0x7fcfd4161950 2024-05-09T06:28:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.853+0000 7fcfd0ff9640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 0x7fcfc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.853+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcfc808ac80 con 0x7fcfd4161950 2024-05-09T06:28:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.853+0000 7fcfd3fff640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 0x7fcfc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.854+0000 7fcfd3fff640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 0x7fcfc006e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000f60 tx=0x7fcfc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:16.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:16.858+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcfc8059230 con 0x7fcfd4161950 2024-05-09T06:28:17.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.042+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfd40d21e0 con 0x7fcfd4161950 2024-05-09T06:28:17.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.043+0000 7fcfd0ff9640 1 -- 172.21.15.1:0/1485522761 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7fcfc8058b30 con 0x7fcfd4161950 2024-05-09T06:28:17.026 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:17.026 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 msgr2=0x7fcfc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 0x7fcfc006e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000f60 tx=0x7fcfc4009040 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 msgr2=0x7fcfd40b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b4a0 tx=0x7fcfc800b960 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 shutdown_connections 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcfc006be30 0x7fcfc006e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcfd40b6f10 0x7fcfd41f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd4161950 0x7fcfd40b69d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 --2- 172.21.15.1:0/1485522761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd40d2b10 0x7fcfd40b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.045+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 >> 172.21.15.1:0/1485522761 conn(0x7fcfd40d3e20 msgr2=0x7fcfd40b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.046+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 shutdown_connections 2024-05-09T06:28:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.046+0000 7fcfda69e640 1 -- 172.21.15.1:0/1485522761 wait complete. 2024-05-09T06:28:17.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:17.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:17.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.165+0000 7fe4e15a5640 1 Processor -- start 2024-05-09T06:28:17.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.165+0000 7fe4e15a5640 1 -- start start 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4e15a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 0x7fe4dc0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4e15a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe4dc0ccbf0 0x7fe4dc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4e15a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc0cd590 con 0x7fe4dc0ce600 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4e15a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0d2ba0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4e15a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc0ae9a0 con 0x7fe4dc15c000 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4d27fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe4dc0ccbf0 0x7fe4dc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4db7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe4dc0ce600 0x7fe4dc0cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60770/0 (socket says 172.21.15.1:60770) 2024-05-09T06:28:17.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4db7fe640 1 -- 172.21.15.1:0/635352588 learned_addr learned my addr 172.21.15.1:0/635352588 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4daffd640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 0x7fe4dc0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4d27fc640 1 -- 172.21.15.1:0/635352588 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 msgr2=0x7fe4dc0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4d27fc640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 0x7fe4dc0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4d27fc640 1 -- 172.21.15.1:0/635352588 >> v1:172.21.15.1:6789/0 conn(0x7fe4dc0ce600 legacy=0x7fe4dc0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4d27fc640 1 -- 172.21.15.1:0/635352588 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0d0390 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4daffd640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 0x7fe4dc0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.166+0000 7fe4da7fc640 1 -- 172.21.15.1:0/635352588 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202981809 0 0) 0x7fe4dc0cd590 con 0x7fe4dc0ce600 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.167+0000 7fe4d27fc640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc0cd050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40097a0 tx=0x7fe4c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbd4b3e6c397d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.167+0000 7fe4da7fc640 1 -- 172.21.15.1:0/635352588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4037070 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.167+0000 7fe4da7fc640 1 -- 172.21.15.1:0/635352588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4032440 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.167+0000 7fe4da7fc640 1 -- 172.21.15.1:0/635352588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4032810 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 -- 172.21.15.1:0/635352588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 msgr2=0x7fe4dc0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc0cd050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40097a0 tx=0x7fe4c402a180 comp rx=0 tx=0).stop 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 -- 172.21.15.1:0/635352588 shutdown_connections 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc0cd050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/635352588 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc15c000 0x7fe4dc0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 -- 172.21.15.1:0/635352588 >> 172.21.15.1:0/635352588 conn(0x7fe4dc0d1c10 msgr2=0x7fe4dc0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 -- 172.21.15.1:0/635352588 shutdown_connections 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.168+0000 7fe4e15a5640 1 -- 172.21.15.1:0/635352588 wait complete. 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.169+0000 7fe4e15a5640 1 Processor -- start 2024-05-09T06:28:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.169+0000 7fe4e15a5640 1 -- start start 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc0ce600 0x7fe4dc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4dc15c000 0x7fe4dc06f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc15c3e0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc0d0950 con 0x7fe4dc15c000 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4e15a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc0b0de0 con 0x7fe4dc0ce600 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36152/0 (socket says 172.21.15.1:36152) 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 -- 172.21.15.1:0/301332800 learned_addr learned my addr 172.21.15.1:0/301332800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4d27fc640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc0ce600 0x7fe4dc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4db7fe640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4dc15c000 0x7fe4dc06f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 -- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc0ce600 msgr2=0x7fe4dc06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc0ce600 0x7fe4dc06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 -- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4dc15c000 msgr2=0x7fe4dc06f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4dc15c000 0x7fe4dc06f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.170+0000 7fe4daffd640 1 -- 172.21.15.1:0/301332800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc149da0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4daffd640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d7d0 tx=0x7fe4cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004030 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0041d0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc149a20 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0096a0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.171+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc14a2f0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.172+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc0752b0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.172+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4cc009840 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.173+0000 7fe4d3fff640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 0x7fe4bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.173+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe4cc08b360 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.173+0000 7fe4d27fc640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 0x7fe4bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.173+0000 7fe4d27fc640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 0x7fe4bc06e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4034000 tx=0x7fe4c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.177+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4dc149a20 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.360+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4dc0b04b0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.360+0000 7fe4d3fff640 1 -- 172.21.15.1:0/301332800 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe4cc0599a0 con 0x7fe4dc0ccbf0 2024-05-09T06:28:17.342 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:17.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 msgr2=0x7fe4bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 0x7fe4bc06e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4034000 tx=0x7fe4c402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 msgr2=0x7fe4dc1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d7d0 tx=0x7fe4cc00dc90 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 shutdown_connections 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe4bc06be60 0x7fe4bc06e320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4dc15c000 0x7fe4dc06f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4dc0ce600 0x7fe4dc06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 --2- 172.21.15.1:0/301332800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4dc0ccbf0 0x7fe4dc1494e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 >> 172.21.15.1:0/301332800 conn(0x7fe4dc0d1c10 msgr2=0x7fe4dc15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 shutdown_connections 2024-05-09T06:28:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.362+0000 7fe4e15a5640 1 -- 172.21.15.1:0/301332800 wait complete. 2024-05-09T06:28:17.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.480+0000 7fcef7645640 1 Processor -- start 2024-05-09T06:28:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.480+0000 7fcef7645640 1 -- start start 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef7645640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcef00cf180 0x7fcef00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef7645640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef00b1340 con 0x7fcef00ceba0 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef7645640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef00d1020 con 0x7fcef015e270 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef7645640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcef00d0df0 con 0x7fcef00cf180 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef5be2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcef00cf180 0x7fcef00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef4be0640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcef00ceba0 0x7fcef00ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60780/0 (socket says 172.21.15.1:60780) 2024-05-09T06:28:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef4be0640 1 -- 172.21.15.1:0/278799956 learned_addr learned my addr 172.21.15.1:0/278799956 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef5be2640 1 -- 172.21.15.1:0/278799956 >> v1:172.21.15.6:6789/0 conn(0x7fcef015e270 legacy=0x7fcef00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef5be2640 1 -- 172.21.15.1:0/278799956 >> v1:172.21.15.1:6789/0 conn(0x7fcef00ceba0 legacy=0x7fcef00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcee7fff640 1 -- 172.21.15.1:0/278799956 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3799866546 0 0) 0x7fcee0001110 con 0x7fcef00ceba0 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.481+0000 7fcef5be2640 1 -- 172.21.15.1:0/278799956 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef00cc4b0 con 0x7fcef00cf180 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.482+0000 7fcef5be2640 1 --2- 172.21.15.1:0/278799956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00cf180 0x7fcef00d08b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009d90 tx=0x7fcee802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77defb8bc166402 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.482+0000 7fcee7fff640 1 -- 172.21.15.1:0/278799956 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8037070 con 0x7fcef00cf180 2024-05-09T06:28:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.482+0000 7fcee7fff640 1 -- 172.21.15.1:0/278799956 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee8007740 con 0x7fcef00cf180 2024-05-09T06:28:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.482+0000 7fcee7fff640 1 -- 172.21.15.1:0/278799956 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8007b10 con 0x7fcef00cf180 2024-05-09T06:28:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 -- 172.21.15.1:0/278799956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00cf180 msgr2=0x7fcef00d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 --2- 172.21.15.1:0/278799956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00cf180 0x7fcef00d08b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009d90 tx=0x7fcee802a780 comp rx=0 tx=0).stop 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 -- 172.21.15.1:0/278799956 shutdown_connections 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 --2- 172.21.15.1:0/278799956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00cf180 0x7fcef00d08b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 -- 172.21.15.1:0/278799956 >> 172.21.15.1:0/278799956 conn(0x7fcef00d1c10 msgr2=0x7fcef00d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 -- 172.21.15.1:0/278799956 shutdown_connections 2024-05-09T06:28:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.483+0000 7fcef7645640 1 -- 172.21.15.1:0/278799956 wait complete. 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.484+0000 7fcef7645640 1 Processor -- start 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.484+0000 7fcef7645640 1 -- start start 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef00ceba0 0x7fcef01f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcef00cf180 0x7fcef00c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcef00cf590 con 0x7fcef015e270 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcef00cc260 con 0x7fcef00ceba0 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef7645640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcef00cca70 con 0x7fcef00cf180 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef53e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef00ceba0 0x7fcef01f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36178/0 (socket says 172.21.15.1:36178) 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 -- 172.21.15.1:0/3824439013 learned_addr learned my addr 172.21.15.1:0/3824439013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef4be0640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcef00cf180 0x7fcef00c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 -- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcef00cf180 msgr2=0x7fcef00c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcef00cf180 0x7fcef00c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 -- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef00ceba0 msgr2=0x7fcef01f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef00ceba0 0x7fcef01f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.485+0000 7fcef5be2640 1 -- 172.21.15.1:0/3824439013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef01f4f90 con 0x7fcef015e270 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcef5be2640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcee8035040 tx=0x7fcee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8042020 con 0x7fcef015e270 2024-05-09T06:28:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee8032990 con 0x7fcef015e270 2024-05-09T06:28:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8037040 con 0x7fcef015e270 2024-05-09T06:28:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef006fd80 con 0x7fcef015e270 2024-05-09T06:28:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.486+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef015be30 con 0x7fcef015e270 2024-05-09T06:28:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.487+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef00752b0 con 0x7fcef015e270 2024-05-09T06:28:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.487+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcee8045020 con 0x7fcef015e270 2024-05-09T06:28:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.488+0000 7fcee5ffb640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 0x7fced406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.488+0000 7fcef53e1640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 0x7fced406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.488+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcee80ac0b0 con 0x7fcef015e270 2024-05-09T06:28:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.489+0000 7fcef53e1640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 0x7fced406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcef01ea770 tx=0x7fceec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.493+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcee804c070 con 0x7fcef015e270 2024-05-09T06:28:17.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:28:17.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:28:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.677+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcef00cd010 con 0x7fcef015e270 2024-05-09T06:28:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.677+0000 7fcee5ffb640 1 -- 172.21.15.1:0/3824439013 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fcef01f4f90 con 0x7fcef015e270 2024-05-09T06:28:17.659 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:17.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 msgr2=0x7fced406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 0x7fced406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcef01ea770 tx=0x7fceec009040 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 msgr2=0x7fcef0140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcee8035040 tx=0x7fcee8004a40 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 shutdown_connections 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fced406be30 0x7fced406e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef015e270 0x7fcef0140720 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcef00cf180 0x7fcef00c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 --2- 172.21.15.1:0/3824439013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcef00ceba0 0x7fcef01f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 >> 172.21.15.1:0/3824439013 conn(0x7fcef00d1c10 msgr2=0x7fcef00ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 shutdown_connections 2024-05-09T06:28:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.680+0000 7fcef7645640 1 -- 172.21.15.1:0/3824439013 wait complete. 2024-05-09T06:28:17.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:17.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.838+0000 7ff6948d4640 1 Processor -- start 2024-05-09T06:28:17.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.838+0000 7ff6948d4640 1 -- start start 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.839+0000 7ff6948d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff69015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.839+0000 7ff6948d4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff690161fe0 con 0x7ff6900d45f0 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.839+0000 7ff6948d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6901622c0 con 0x7ff6900d1c70 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.839+0000 7ff6948d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff69015f580 con 0x7ff69015e190 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68ed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff6900d45f0 0x7ff6900b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42396/0 (socket says 172.21.15.1:42396) 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68ed9d640 1 -- 172.21.15.1:0/2755167213 learned_addr learned my addr 172.21.15.1:0/2755167213 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68e59c640 1 --2- 172.21.15.1:0/2755167213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff69015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139427054 0 0) 0x7ff690161fe0 con 0x7ff6900d45f0 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff67c003520 con 0x7ff6900d45f0 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155095215 0 0) 0x7ff6901622c0 con 0x7ff6900d1c70 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff690161fe0 con 0x7ff6900d1c70 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68e59c640 1 -- 172.21.15.1:0/2755167213 >> v1:172.21.15.6:6789/0 conn(0x7ff6900d1c70 legacy=0x7ff6900d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68e59c640 1 -- 172.21.15.1:0/2755167213 >> v1:172.21.15.1:6789/0 conn(0x7ff6900d45f0 legacy=0x7ff6900b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.840+0000 7ff68e59c640 1 -- 172.21.15.1:0/2755167213 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6901625a0 con 0x7ff69015e190 2024-05-09T06:28:17.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff68e59c640 1 --2- 172.21.15.1:0/2755167213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff69015f780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff680009dc0 tx=0x7ff68002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba49948a804a9bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680007740 con 0x7ff69015e190 2024-05-09T06:28:17.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6800078e0 con 0x7ff69015e190 2024-05-09T06:28:17.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff68d59a640 1 -- 172.21.15.1:0/2755167213 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680007c90 con 0x7ff69015e190 2024-05-09T06:28:17.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff6948d4640 1 -- 172.21.15.1:0/2755167213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 msgr2=0x7ff69015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.841+0000 7ff6948d4640 1 --2- 172.21.15.1:0/2755167213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff69015f780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff680009dc0 tx=0x7ff68002a6d0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.842+0000 7ff6948d4640 1 -- 172.21.15.1:0/2755167213 shutdown_connections 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.842+0000 7ff6948d4640 1 --2- 172.21.15.1:0/2755167213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff69015f780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.842+0000 7ff6948d4640 1 -- 172.21.15.1:0/2755167213 >> 172.21.15.1:0/2755167213 conn(0x7ff690073340 msgr2=0x7ff69009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.842+0000 7ff6948d4640 1 -- 172.21.15.1:0/2755167213 shutdown_connections 2024-05-09T06:28:17.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.842+0000 7ff6948d4640 1 -- 172.21.15.1:0/2755167213 wait complete. 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.843+0000 7ff6948d4640 1 Processor -- start 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.843+0000 7ff6948d4640 1 -- start start 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6900d1c70 0x7ff6900b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff6900cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6900b56d0 con 0x7ff6900d45f0 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff690161c60 con 0x7ff6900d1c70 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff6900b5850 con 0x7ff69015e190 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68e59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6900d1c70 0x7ff6900b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51038/0 (socket says 172.21.15.1:51038) 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 -- 172.21.15.1:0/3464022403 learned_addr learned my addr 172.21.15.1:0/3464022403 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68ed9d640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff6900cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 -- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 msgr2=0x7ff6900cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff6900cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 -- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6900d1c70 msgr2=0x7ff6900b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6900d1c70 0x7ff6900b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 -- 172.21.15.1:0/3464022403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6900cbd80 con 0x7ff6900d45f0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff68dd9b640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c400 tx=0x7ff67800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678017070 con 0x7ff6900d45f0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6780052d0 con 0x7ff6900d45f0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6900cc060 con 0x7ff6900d45f0 2024-05-09T06:28:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.844+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678003810 con 0x7ff6900d45f0 2024-05-09T06:28:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.845+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6900cc620 con 0x7ff6900d45f0 2024-05-09T06:28:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.846+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6900752b0 con 0x7ff6900d45f0 2024-05-09T06:28:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.846+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff67801a070 con 0x7ff6900d45f0 2024-05-09T06:28:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.847+0000 7ff686ffd640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 0x7ff67006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.847+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff67808a5d0 con 0x7ff6900d45f0 2024-05-09T06:28:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.847+0000 7ff68e59c640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 0x7ff67006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.847+0000 7ff68e59c640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 0x7ff67006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff680034000 tx=0x7ff68002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:17.851+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff678058530 con 0x7ff6900d45f0 2024-05-09T06:28:18.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.037+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6900cdf80 con 0x7ff6900d45f0 2024-05-09T06:28:18.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.037+0000 7ff686ffd640 1 -- 172.21.15.1:0/3464022403 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7ff67805c830 con 0x7ff6900d45f0 2024-05-09T06:28:18.019 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:18.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 msgr2=0x7ff67006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 0x7ff67006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff680034000 tx=0x7ff68002f000 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 msgr2=0x7ff6900b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c400 tx=0x7ff67800c8c0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 shutdown_connections 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff67006be60 0x7ff67006e320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff69015e190 0x7ff6900cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6900d45f0 0x7ff6900b2200 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 --2- 172.21.15.1:0/3464022403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6900d1c70 0x7ff6900b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 >> 172.21.15.1:0/3464022403 conn(0x7ff690073340 msgr2=0x7ff6900b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 shutdown_connections 2024-05-09T06:28:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.039+0000 7ff6948d4640 1 -- 172.21.15.1:0/3464022403 wait complete. 2024-05-09T06:28:18.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.157+0000 7fe27cdc1640 1 Processor -- start 2024-05-09T06:28:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.157+0000 7fe27cdc1640 1 -- start start 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.158+0000 7fe27cdc1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 0x7fe2780cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.158+0000 7fe27cdc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 0x7fe2780ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.158+0000 7fe27cdc1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe278072050 con 0x7fe27806b1e0 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.158+0000 7fe27cdc1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe2780cd160 con 0x7fe2780d06f0 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.158+0000 7fe27cdc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe278072330 con 0x7fe2780c41e0 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe276d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe27806b1e0 0x7fe278161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42408/0 (socket says 172.21.15.1:42408) 2024-05-09T06:28:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe276d9d640 1 -- 172.21.15.1:0/1007137573 learned_addr learned my addr 172.21.15.1:0/1007137573 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe275d9b640 1 --2- 172.21.15.1:0/1007137573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 0x7fe2780ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe27659c640 1 --2- 172.21.15.1:0/1007137573 >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 0x7fe2780cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe27559a640 1 -- 172.21.15.1:0/1007137573 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053815360 0 0) 0x7fe278072050 con 0x7fe27806b1e0 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe27559a640 1 -- 172.21.15.1:0/1007137573 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe260003520 con 0x7fe27806b1e0 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe275d9b640 1 -- 172.21.15.1:0/1007137573 >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 msgr2=0x7fe2780cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe275d9b640 1 --2- 172.21.15.1:0/1007137573 >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 0x7fe2780cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe275d9b640 1 -- 172.21.15.1:0/1007137573 >> v1:172.21.15.1:6789/0 conn(0x7fe27806b1e0 legacy=0x7fe278161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe275d9b640 1 -- 172.21.15.1:0/1007137573 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe278072610 con 0x7fe2780c41e0 2024-05-09T06:28:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.159+0000 7fe27659c640 1 --2- 172.21.15.1:0/1007137573 >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 0x7fe2780cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe275d9b640 1 --2- 172.21.15.1:0/1007137573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 0x7fe2780ccc20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f07d85bfbfc0013 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe27559a640 1 -- 172.21.15.1:0/1007137573 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c037070 con 0x7fe2780c41e0 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe27559a640 1 -- 172.21.15.1:0/1007137573 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c032440 con 0x7fe2780c41e0 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe27559a640 1 -- 172.21.15.1:0/1007137573 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0327f0 con 0x7fe2780c41e0 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe27cdc1640 1 -- 172.21.15.1:0/1007137573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 msgr2=0x7fe2780ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.160+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/1007137573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 0x7fe2780ccc20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02a180 comp rx=0 tx=0).stop 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 -- 172.21.15.1:0/1007137573 shutdown_connections 2024-05-09T06:28:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/1007137573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe2780c41e0 0x7fe2780ccc20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/1007137573 >> v2:172.21.15.6:3300/0 conn(0x7fe2780d06f0 0x7fe2780cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 -- 172.21.15.1:0/1007137573 >> 172.21.15.1:0/1007137573 conn(0x7fe2780d0b50 msgr2=0x7fe2780d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 -- 172.21.15.1:0/1007137573 shutdown_connections 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.161+0000 7fe27cdc1640 1 -- 172.21.15.1:0/1007137573 wait complete. 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.162+0000 7fe27cdc1640 1 Processor -- start 2024-05-09T06:28:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.162+0000 7fe27cdc1640 1 -- start start 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe27806b1e0 0x7fe2781f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 0x7fe2780c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2780cf8e0 con 0x7fe2780d06f0 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe278071ce0 con 0x7fe2780c41e0 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27cdc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe2780cfa60 con 0x7fe27806b1e0 2024-05-09T06:28:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51052/0 (socket says 172.21.15.1:51052) 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 -- 172.21.15.1:0/5294811 learned_addr learned my addr 172.21.15.1:0/5294811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe27659c640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe27806b1e0 0x7fe2781f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe275d9b640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 0x7fe2780c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 -- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe27806b1e0 msgr2=0x7fe2781f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe27806b1e0 0x7fe2781f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 -- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 msgr2=0x7fe2780c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 0x7fe2780c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 -- 172.21.15.1:0/5294811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27809f840 con 0x7fe2780d06f0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe275d9b640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 0x7fe2780c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe276d9d640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe27000cb40 tx=0x7fe27000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.163+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270017070 con 0x7fe2780d06f0 2024-05-09T06:28:18.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.164+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27000ac40 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.164+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270012400 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.164+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27809fb20 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.164+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2780a00e0 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.164+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2780752b0 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.165+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe27001a070 con 0x7fe2780d06f0 2024-05-09T06:28:18.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.166+0000 7fe266ffd640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 0x7fe25806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.166+0000 7fe27659c640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 0x7fe25806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.166+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe27008af20 con 0x7fe2780d06f0 2024-05-09T06:28:18.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.166+0000 7fe27659c640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 0x7fe25806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe268009560 tx=0x7fe268009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.170+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe270058e80 con 0x7fe2780d06f0 2024-05-09T06:28:18.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.354+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe278072f30 con 0x7fe2780d06f0 2024-05-09T06:28:18.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.355+0000 7fe266ffd640 1 -- 172.21.15.1:0/5294811 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe27005d180 con 0x7fe2780d06f0 2024-05-09T06:28:18.337 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 msgr2=0x7fe25806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 0x7fe25806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe268009560 tx=0x7fe268009190 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 msgr2=0x7fe2780a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe27000cb40 tx=0x7fe27000a060 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 shutdown_connections 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe25806be30 0x7fe25806e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2780d06f0 0x7fe2780a1180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2780c41e0 0x7fe2780c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 --2- 172.21.15.1:0/5294811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe27806b1e0 0x7fe2781f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 >> 172.21.15.1:0/5294811 conn(0x7fe2780d0b50 msgr2=0x7fe2780cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 shutdown_connections 2024-05-09T06:28:18.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.357+0000 7fe27cdc1640 1 -- 172.21.15.1:0/5294811 wait complete. 2024-05-09T06:28:18.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.473+0000 7f5a49570640 1 Processor -- start 2024-05-09T06:28:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.473+0000 7f5a49570640 1 -- start start 2024-05-09T06:28:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a49570640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5a440b0080 0x7f5a44071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a49570640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 0x7f5a4415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a49570640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4415f630 con 0x7f5a440cbb60 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a49570640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5a440cbfe0 con 0x7f5a440b0080 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a49570640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5a440d04a0 con 0x7f5a4415ece0 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a42ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5a440b0080 0x7f5a44071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a3bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5a440cbb60 0x7f5a4415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42420/0 (socket says 172.21.15.1:42420) 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a3bfff640 1 -- 172.21.15.1:0/709177771 learned_addr learned my addr 172.21.15.1:0/709177771 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a437fe640 1 --2- 172.21.15.1:0/709177771 >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 0x7f5a4415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a42ffd640 1 -- 172.21.15.1:0/709177771 >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 msgr2=0x7f5a4415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/709177771 >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 0x7f5a4415f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a42ffd640 1 -- 172.21.15.1:0/709177771 >> v1:172.21.15.1:6789/0 conn(0x7f5a440cbb60 legacy=0x7f5a4415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a427fc640 1 -- 172.21.15.1:0/709177771 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4070691302 0 0) 0x7f5a2c001110 con 0x7f5a440cbb60 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a42ffd640 1 -- 172.21.15.1:0/709177771 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a440722c0 con 0x7f5a440b0080 2024-05-09T06:28:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.474+0000 7f5a437fe640 1 --2- 172.21.15.1:0/709177771 >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 0x7f5a4415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.475+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/709177771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440b0080 0x7f5a44071ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a440c7bb0 tx=0x7f5a3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75dc67e5db199e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.475+0000 7f5a427fc640 1 -- 172.21.15.1:0/709177771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037070 con 0x7f5a440b0080 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.475+0000 7f5a427fc640 1 -- 172.21.15.1:0/709177771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34032440 con 0x7f5a440b0080 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.475+0000 7f5a427fc640 1 -- 172.21.15.1:0/709177771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34032810 con 0x7f5a440b0080 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 -- 172.21.15.1:0/709177771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440b0080 msgr2=0x7f5a44071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 --2- 172.21.15.1:0/709177771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440b0080 0x7f5a44071ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a440c7bb0 tx=0x7f5a3402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 -- 172.21.15.1:0/709177771 shutdown_connections 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 --2- 172.21.15.1:0/709177771 >> v2:172.21.15.6:3300/0 conn(0x7f5a4415ece0 0x7f5a4415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 --2- 172.21.15.1:0/709177771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440b0080 0x7f5a44071ce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 -- 172.21.15.1:0/709177771 >> 172.21.15.1:0/709177771 conn(0x7f5a440d0b50 msgr2=0x7f5a440d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 -- 172.21.15.1:0/709177771 shutdown_connections 2024-05-09T06:28:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.476+0000 7f5a49570640 1 -- 172.21.15.1:0/709177771 wait complete. 2024-05-09T06:28:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.477+0000 7f5a49570640 1 Processor -- start 2024-05-09T06:28:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.477+0000 7f5a49570640 1 -- start start 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 0x7f5a440ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a4415ece0 0x7f5a440c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44072b60 con 0x7f5a440cbb60 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44072880 con 0x7f5a4415ece0 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4415e7c0 con 0x7f5a440b0080 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51056/0 (socket says 172.21.15.1:51056) 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 -- 172.21.15.1:0/1752197756 learned_addr learned my addr 172.21.15.1:0/1752197756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a437fe640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a4415ece0 0x7f5a440c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 0x7f5a440ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 -- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 msgr2=0x7f5a440ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 0x7f5a440ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 -- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a4415ece0 msgr2=0x7f5a440c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a4415ece0 0x7f5a440c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 -- 172.21.15.1:0/1752197756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a440c4e20 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 0x7f5a440ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3bfff640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00c780 tx=0x7f5a2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c018070 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0072d0 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.478+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a440c5100 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.479+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0047e0 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.479+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a440c56c0 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.479+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a440752b0 con 0x7f5a440cbb60 2024-05-09T06:28:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.480+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a2c01b070 con 0x7f5a440cbb60 2024-05-09T06:28:18.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.481+0000 7f5a3b7fe640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.481+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.481+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5a2c08a660 con 0x7f5a440cbb60 2024-05-09T06:28:18.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.481+0000 7f5a42ffd640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 0x7f5a2406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a3402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.485+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a4415f630 con 0x7f5a440cbb60 2024-05-09T06:28:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.681+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a440b22e0 con 0x7f5a440cbb60 2024-05-09T06:28:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.681+0000 7f5a3b7fe640 1 -- 172.21.15.1:0/1752197756 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5a2c058b70 con 0x7f5a440cbb60 2024-05-09T06:28:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 msgr2=0x7f5a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 0x7f5a2406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5a34034000 tx=0x7f5a3402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 msgr2=0x7f5a44155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00c780 tx=0x7f5a2c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 shutdown_connections 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a4415ece0 0x7f5a440c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a440cbb60 0x7f5a44155e80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 --2- 172.21.15.1:0/1752197756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a440b0080 0x7f5a440ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.683+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 >> 172.21.15.1:0/1752197756 conn(0x7f5a440d0b50 msgr2=0x7f5a440d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.684+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 shutdown_connections 2024-05-09T06:28:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.684+0000 7f5a49570640 1 -- 172.21.15.1:0/1752197756 wait complete. 2024-05-09T06:28:18.676 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:18.677 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:18.695 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:18.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.816+0000 7f1e7f177640 1 Processor -- start 2024-05-09T06:28:18.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.816+0000 7f1e7f177640 1 -- start start 2024-05-09T06:28:18.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7f177640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1e78161950 0x7f1e780d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7f177640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1e780d0f80 0x7f1e780d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7f177640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780b33c0 con 0x7f1e780d4940 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7f177640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1e780d4dd0 con 0x7f1e78161950 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7f177640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1e780b3660 con 0x7f1e780d0f80 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7cf13640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1e78161950 0x7f1e780d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e77fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1e780d4940 0x7f1e780cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42436/0 (socket says 172.21.15.1:42436) 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e77fff640 1 -- 172.21.15.1:0/865827962 learned_addr learned my addr 172.21.15.1:0/865827962 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7d714640 1 --2- 172.21.15.1:0/865827962 >> v2:172.21.15.6:3300/0 conn(0x7f1e780d0f80 0x7f1e780d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7cf13640 1 -- 172.21.15.1:0/865827962 >> v2:172.21.15.6:3300/0 conn(0x7f1e780d0f80 msgr2=0x7f1e780d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7cf13640 1 --2- 172.21.15.1:0/865827962 >> v2:172.21.15.6:3300/0 conn(0x7f1e780d0f80 0x7f1e780d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.817+0000 7f1e7cf13640 1 -- 172.21.15.1:0/865827962 >> v1:172.21.15.1:6789/0 conn(0x7f1e780d4940 legacy=0x7f1e780cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e7cf13640 1 -- 172.21.15.1:0/865827962 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780b3940 con 0x7f1e78161950 2024-05-09T06:28:18.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e777fe640 1 -- 172.21.15.1:0/865827962 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515775604 0 0) 0x7f1e780b33c0 con 0x7f1e780d4940 2024-05-09T06:28:18.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e7cf13640 1 --2- 172.21.15.1:0/865827962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780d2400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e7808e540 tx=0x7f1e6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d2eda828996977 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e777fe640 1 -- 172.21.15.1:0/865827962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68037070 con 0x7f1e78161950 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e777fe640 1 -- 172.21.15.1:0/865827962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e68032440 con 0x7f1e78161950 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.818+0000 7f1e777fe640 1 -- 172.21.15.1:0/865827962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68032810 con 0x7f1e78161950 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 -- 172.21.15.1:0/865827962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 msgr2=0x7f1e780d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 --2- 172.21.15.1:0/865827962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780d2400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e7808e540 tx=0x7f1e6802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 -- 172.21.15.1:0/865827962 shutdown_connections 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 --2- 172.21.15.1:0/865827962 >> v2:172.21.15.6:3300/0 conn(0x7f1e780d0f80 0x7f1e780d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 --2- 172.21.15.1:0/865827962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780d2400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 -- 172.21.15.1:0/865827962 >> 172.21.15.1:0/865827962 conn(0x7f1e780d3e20 msgr2=0x7f1e780d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 -- 172.21.15.1:0/865827962 shutdown_connections 2024-05-09T06:28:18.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.819+0000 7f1e7f177640 1 -- 172.21.15.1:0/865827962 wait complete. 2024-05-09T06:28:18.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.820+0000 7f1e7f177640 1 Processor -- start 2024-05-09T06:28:18.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.820+0000 7f1e7f177640 1 -- start start 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e780d0f80 0x7f1e78154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 0x7f1e780cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780b52d0 con 0x7f1e78161950 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780b5150 con 0x7f1e780b99f0 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7f177640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1e780b5450 con 0x7f1e780d0f80 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e77fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51074/0 (socket says 172.21.15.1:51074) 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e77fff640 1 -- 172.21.15.1:0/890029704 learned_addr learned my addr 172.21.15.1:0/890029704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7d714640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 0x7f1e780cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.821+0000 7f1e7cf13640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e780d0f80 0x7f1e78154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 -- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e780d0f80 msgr2=0x7f1e78154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e780d0f80 0x7f1e78154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 -- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 msgr2=0x7f1e780cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 0x7f1e780cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 -- 172.21.15.1:0/890029704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780b7d20 con 0x7f1e78161950 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e7d714640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 0x7f1e780cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e77fff640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4d0 tx=0x7f1e6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c017070 con 0x7f1e78161950 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005d00 con 0x7f1e78161950 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e780b8000 con 0x7f1e78161950 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0072b0 con 0x7f1e78161950 2024-05-09T06:28:18.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.822+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e780b85c0 con 0x7f1e78161950 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.823+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780752b0 con 0x7f1e78161950 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.823+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6c01a070 con 0x7f1e78161950 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.824+0000 7f1e757fa640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.824+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1e6c08a5a0 con 0x7f1e78161950 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.824+0000 7f1e7cf13640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:18.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.824+0000 7f1e7cf13640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 0x7f1e6406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e68034000 tx=0x7f1e6802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:18.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:18.828+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e6c058be0 con 0x7f1e78161950 2024-05-09T06:28:18.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.013+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e780d21e0 con 0x7f1e78161950 2024-05-09T06:28:18.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.014+0000 7f1e757fa640 1 -- 172.21.15.1:0/890029704 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f1e6c0584e0 con 0x7f1e78161950 2024-05-09T06:28:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:18.997 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 msgr2=0x7f1e6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 0x7f1e6406e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e68034000 tx=0x7f1e6802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 msgr2=0x7f1e780b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4d0 tx=0x7f1e6c00c990 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 shutdown_connections 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1e6406be30 0x7f1e6406e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e780b99f0 0x7f1e780cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e78161950 0x7f1e780b7420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 --2- 172.21.15.1:0/890029704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1e780d0f80 0x7f1e78154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 >> 172.21.15.1:0/890029704 conn(0x7f1e780d3e20 msgr2=0x7f1e780d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 shutdown_connections 2024-05-09T06:28:19.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.017+0000 7f1e7f177640 1 -- 172.21.15.1:0/890029704 wait complete. 2024-05-09T06:28:19.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:19.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.138+0000 7f05f1038640 1 Processor -- start 2024-05-09T06:28:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.138+0000 7f05f1038640 1 -- start start 2024-05-09T06:28:19.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05f1038640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05ec0d1c70 0x7f05ec15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05f1038640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d45f0 0x7f05ec0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05f1038640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05ec161fe0 con 0x7f05ec15e190 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05f1038640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f05ec15f580 con 0x7f05ec0d1c70 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05f1038640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05ec1622c0 con 0x7f05ec0d45f0 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05ea59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05ec0d1c70 0x7f05ec15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05ea59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05ec0d1c70 0x7f05ec15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51080/0 (socket says 172.21.15.1:51080) 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05ea59c640 1 -- 172.21.15.1:0/1312103735 learned_addr learned my addr 172.21.15.1:0/1312103735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.139+0000 7f05eb59e640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d45f0 0x7f05ec0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.140+0000 7f05ea59c640 1 -- 172.21.15.1:0/1312103735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d45f0 msgr2=0x7f05ec0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.140+0000 7f05ea59c640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d45f0 0x7f05ec0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.140+0000 7f05ea59c640 1 -- 172.21.15.1:0/1312103735 >> v1:172.21.15.6:6789/0 conn(0x7f05ec15e190 legacy=0x7f05ec0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.140+0000 7f05ea59c640 1 -- 172.21.15.1:0/1312103735 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1625a0 con 0x7f05ec0d1c70 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.140+0000 7f05ea59c640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d1c70 0x7f05ec15f780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009870 tx=0x7f05e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02b99c3fc4d24ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05e9d9b640 1 -- 172.21.15.1:0/1312103735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0037070 con 0x7f05ec0d1c70 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05e9d9b640 1 -- 172.21.15.1:0/1312103735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e0032440 con 0x7f05ec0d1c70 2024-05-09T06:28:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05e9d9b640 1 -- 172.21.15.1:0/1312103735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0032810 con 0x7f05ec0d1c70 2024-05-09T06:28:19.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 -- 172.21.15.1:0/1312103735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d1c70 msgr2=0x7f05ec15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d1c70 0x7f05ec15f780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009870 tx=0x7f05e002a270 comp rx=0 tx=0).stop 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 -- 172.21.15.1:0/1312103735 shutdown_connections 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d45f0 0x7f05ec0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 --2- 172.21.15.1:0/1312103735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d1c70 0x7f05ec15f780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 -- 172.21.15.1:0/1312103735 >> 172.21.15.1:0/1312103735 conn(0x7f05ec073340 msgr2=0x7f05ec09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 -- 172.21.15.1:0/1312103735 shutdown_connections 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.141+0000 7f05f1038640 1 -- 172.21.15.1:0/1312103735 wait complete. 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.143+0000 7f05f1038640 1 Processor -- start 2024-05-09T06:28:19.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.143+0000 7f05f1038640 1 -- start start 2024-05-09T06:28:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d1c70 0x7f05ec156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 0x7f05ec0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec0b56d0 con 0x7f05ec0d45f0 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec161c60 con 0x7f05ec15e190 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05ec0b5850 con 0x7f05ec0d1c70 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51086/0 (socket says 172.21.15.1:51086) 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 -- 172.21.15.1:0/4074926559 learned_addr learned my addr 172.21.15.1:0/4074926559 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ead9d640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d1c70 0x7f05ec156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05eb59e640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 0x7f05ec0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 -- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d1c70 msgr2=0x7f05ec156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d1c70 0x7f05ec156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 -- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 msgr2=0x7f05ec0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 0x7f05ec0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 -- 172.21.15.1:0/4074926559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec0c7c70 con 0x7f05ec0d45f0 2024-05-09T06:28:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05eb59e640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 0x7f05ec0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05ea59c640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0037040 con 0x7f05ec0d45f0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e0042070 con 0x7f05ec0d45f0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec0cc3f0 con 0x7f05ec0d45f0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.144+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0030860 con 0x7f05ec0d45f0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.145+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec0cc690 con 0x7f05ec0d45f0 2024-05-09T06:28:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.146+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec156e20 con 0x7f05ec0d45f0 2024-05-09T06:28:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.146+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05e0030a00 con 0x7f05ec0d45f0 2024-05-09T06:28:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.147+0000 7f05d37fe640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 0x7f05d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.147+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f05e00abb70 con 0x7f05ec0d45f0 2024-05-09T06:28:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.147+0000 7f05ead9d640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 0x7f05d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.147+0000 7f05ead9d640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 0x7f05d806e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007fd0 tx=0x7f05dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.151+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05e0045360 con 0x7f05ec0d45f0 2024-05-09T06:28:19.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.335+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05ec0752b0 con 0x7f05ec0d45f0 2024-05-09T06:28:19.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.335+0000 7f05d37fe640 1 -- 172.21.15.1:0/4074926559 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f05e007a080 con 0x7f05ec0d45f0 2024-05-09T06:28:19.317 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.337+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 msgr2=0x7f05d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.337+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 0x7f05d806e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007fd0 tx=0x7f05dc009040 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.337+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 msgr2=0x7f05ec0d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e00046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 shutdown_connections 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f05d806be60 0x7f05d806e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05ec15e190 0x7f05ec0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05ec0d45f0 0x7f05ec0d1290 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 --2- 172.21.15.1:0/4074926559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05ec0d1c70 0x7f05ec156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 >> 172.21.15.1:0/4074926559 conn(0x7f05ec073340 msgr2=0x7f05ec15f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 shutdown_connections 2024-05-09T06:28:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.338+0000 7f05f1038640 1 -- 172.21.15.1:0/4074926559 wait complete. 2024-05-09T06:28:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:19.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.459+0000 7f6837b90640 1 Processor -- start 2024-05-09T06:28:19.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.459+0000 7f6837b90640 1 -- start start 2024-05-09T06:28:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f6837b90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6830158a50 0x7f68300718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f6837b90640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f683015c4a0 0x7f6830071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f6837b90640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f683015b420 con 0x7f683006e6d0 2024-05-09T06:28:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f6837b90640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6830158e30 con 0x7f683015c4a0 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f6837b90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6830072480 con 0x7f6830158a50 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683512b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f683015c4a0 0x7f6830071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683612d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f683006e6d0 0x7f68300cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42448/0 (socket says 172.21.15.1:42448) 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683612d640 1 -- 172.21.15.1:0/3198460416 learned_addr learned my addr 172.21.15.1:0/3198460416 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683592c640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6830158a50 0x7f68300718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683512b640 1 -- 172.21.15.1:0/3198460416 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6830158a50 msgr2=0x7f68300718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683512b640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6830158a50 0x7f68300718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683512b640 1 -- 172.21.15.1:0/3198460416 >> v1:172.21.15.1:6789/0 conn(0x7f683006e6d0 legacy=0x7f68300cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.460+0000 7f683512b640 1 -- 172.21.15.1:0/3198460416 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683015b6e0 con 0x7f683015c4a0 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.461+0000 7f683512b640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f6830071e00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68200097a0 tx=0x7f682002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f499666733cb158 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.461+0000 7f683492a640 1 -- 172.21.15.1:0/3198460416 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820037070 con 0x7f683015c4a0 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.461+0000 7f683492a640 1 -- 172.21.15.1:0/3198460416 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6820032440 con 0x7f683015c4a0 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.461+0000 7f683492a640 1 -- 172.21.15.1:0/3198460416 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820032810 con 0x7f683015c4a0 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 -- 172.21.15.1:0/3198460416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 msgr2=0x7f6830071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f6830071e00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68200097a0 tx=0x7f682002a180 comp rx=0 tx=0).stop 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 -- 172.21.15.1:0/3198460416 shutdown_connections 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f6830071e00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 --2- 172.21.15.1:0/3198460416 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6830158a50 0x7f68300718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 -- 172.21.15.1:0/3198460416 >> 172.21.15.1:0/3198460416 conn(0x7f68300cc4d0 msgr2=0x7f68300cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 -- 172.21.15.1:0/3198460416 shutdown_connections 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.462+0000 7f6837b90640 1 -- 172.21.15.1:0/3198460416 wait complete. 2024-05-09T06:28:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.463+0000 7f6837b90640 1 Processor -- start 2024-05-09T06:28:19.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.463+0000 7f6837b90640 1 -- start start 2024-05-09T06:28:19.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f683006e6d0 0x7f68301f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 0x7f6830071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f683015bf80 con 0x7f683015c4a0 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f683015bca0 con 0x7f6830158a50 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6830070ea0 con 0x7f683006e6d0 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51102/0 (socket says 172.21.15.1:51102) 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 -- 172.21.15.1:0/1917539789 learned_addr learned my addr 172.21.15.1:0/1917539789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683512b640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 0x7f6830071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683592c640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f683006e6d0 0x7f68301f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 -- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f683006e6d0 msgr2=0x7f68301f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f683006e6d0 0x7f68301f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 -- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 msgr2=0x7f6830071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 0x7f6830071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 -- 172.21.15.1:0/1917539789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683015d100 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683512b640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 0x7f6830071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f683612d640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f682800ca80 tx=0x7f682800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828017070 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682800ac30 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828012600 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.464+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683015d3a0 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.465+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68300a9ed0 con 0x7f683015c4a0 2024-05-09T06:28:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.466+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68300752b0 con 0x7f683015c4a0 2024-05-09T06:28:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.466+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f682801a070 con 0x7f683015c4a0 2024-05-09T06:28:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.467+0000 7f68267fc640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.467+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f682808ada0 con 0x7f683015c4a0 2024-05-09T06:28:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.467+0000 7f683592c640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.467+0000 7f683592c640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 0x7f681406e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f682c006fd0 tx=0x7f682c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.471+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f682800ae20 con 0x7f683015c4a0 2024-05-09T06:28:19.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.657+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6830072d50 con 0x7f683015c4a0 2024-05-09T06:28:19.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.657+0000 7f68267fc640 1 -- 172.21.15.1:0/1917539789 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f6828058d90 con 0x7f683015c4a0 2024-05-09T06:28:19.639 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:19.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 msgr2=0x7f681406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 0x7f681406e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f682c006fd0 tx=0x7f682c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:19.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 msgr2=0x7f68300a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f682800ca80 tx=0x7f682800cf40 comp rx=0 tx=0).stop 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 shutdown_connections 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683015c4a0 0x7f68300a9480 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6830158a50 0x7f6830071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 --2- 172.21.15.1:0/1917539789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f683006e6d0 0x7f68301f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 >> 172.21.15.1:0/1917539789 conn(0x7f68300cc4d0 msgr2=0x7f68300afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 shutdown_connections 2024-05-09T06:28:19.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.659+0000 7f6837b90640 1 -- 172.21.15.1:0/1917539789 wait complete. 2024-05-09T06:28:19.653 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.795+0000 7f6e4636a640 1 Processor -- start 2024-05-09T06:28:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.795+0000 7f6e4636a640 1 -- start start 2024-05-09T06:28:19.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e4636a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e4006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e4636a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4006cf40 con 0x7f6e4015cfb0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e4636a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4006d220 con 0x7f6e400b4de0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e4636a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e40156980 con 0x7f6e4015c4c0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e44907640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6e4015cfb0 0x7f6e40070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42456/0 (socket says 172.21.15.1:42456) 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e44907640 1 -- 172.21.15.1:0/236061334 learned_addr learned my addr 172.21.15.1:0/236061334 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e4006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453967634 0 0) 0x7f6e4006cf40 con 0x7f6e4015cfb0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.796+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e2c003520 con 0x7f6e4015cfb0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1997026919 0 0) 0x7f6e2c003520 con 0x7f6e4015cfb0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 msgr2=0x7f6e4006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e4006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 >> v1:172.21.15.6:6789/0 conn(0x7f6e400b4de0 legacy=0x7f6e4015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4006d500 con 0x7f6e4015cfb0 2024-05-09T06:28:19.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6e380019f0 con 0x7f6e4015cfb0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 >> v1:172.21.15.1:6789/0 conn(0x7f6e4015cfb0 legacy=0x7f6e40070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2c0039e0 0x7f6e2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c004d40 0x7f6e2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c00bac0 con 0x7f6e2c0039e0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c00bc70 con 0x7f6e2c004d40 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c00be70 con 0x7f6e4015c4c0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2c0039e0 0x7f6e2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e44907640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c004d40 0x7f6e2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3f7fe640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 -- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 msgr2=0x7f6e2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 -- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c004d40 msgr2=0x7f6e2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c004d40 0x7f6e2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3ffff640 1 -- 172.21.15.1:0/236061334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c003520 con 0x7f6e2c0039e0 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.797+0000 7f6e3f7fe640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e4015c4c0 0x7f6e2c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2c0039e0 0x7f6e2c003e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009ac0 tx=0x7f6e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237a513b58ed53be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30036070 con 0x7f6e2c0039e0 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e30031400 con 0x7f6e2c0039e0 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e3effd640 1 -- 172.21.15.1:0/236061334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30009690 con 0x7f6e2c0039e0 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e4636a640 1 -- 172.21.15.1:0/236061334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2c0039e0 msgr2=0x7f6e2c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.798+0000 7f6e4636a640 1 --2- 172.21.15.1:0/236061334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2c0039e0 0x7f6e2c003e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009ac0 tx=0x7f6e2c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.799+0000 7f6e44907640 1 -- 172.21.15.1:0/236061334 reap_dead start 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.799+0000 7f6e4636a640 1 -- 172.21.15.1:0/236061334 shutdown_connections 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.799+0000 7f6e4636a640 1 -- 172.21.15.1:0/236061334 >> 172.21.15.1:0/236061334 conn(0x7f6e400cc4d0 msgr2=0x7f6e400cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.799+0000 7f6e4636a640 1 -- 172.21.15.1:0/236061334 shutdown_connections 2024-05-09T06:28:19.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.799+0000 7f6e4636a640 1 -- 172.21.15.1:0/236061334 wait complete. 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.800+0000 7f6e4636a640 1 Processor -- start 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.800+0000 7f6e4636a640 1 -- start start 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c0039e0 0x7f6e4015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e401e49a0 0x7f6e401e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e400b5260 con 0x7f6e401f40f0 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4006d870 con 0x7f6e2c0039e0 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e4636a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e400b53e0 con 0x7f6e401e49a0 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51114/0 (socket says 172.21.15.1:51114) 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 -- 172.21.15.1:0/1243938558 learned_addr learned my addr 172.21.15.1:0/1243938558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e3f7fe640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e401e49a0 0x7f6e401e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c0039e0 0x7f6e4015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 -- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e401e49a0 msgr2=0x7f6e401e4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e401e49a0 0x7f6e401e4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 -- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c0039e0 msgr2=0x7f6e4015b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c0039e0 0x7f6e4015b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 -- 172.21.15.1:0/1243938558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401e5360 con 0x7f6e401f40f0 2024-05-09T06:28:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.801+0000 7f6e44907640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e38000f80 tx=0x7f6e38007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.802+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800ac20 con 0x7f6e401f40f0 2024-05-09T06:28:19.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.802+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e38019070 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.802+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401f7670 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.802+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e380133f0 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.802+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401f7c30 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.803+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e400752b0 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.803+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e380135b0 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.804+0000 7f6e3cff9640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 0x7f6e2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.804+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e3808b760 con 0x7f6e401f40f0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.804+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 0x7f6e2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:19.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.804+0000 7f6e3ffff640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 0x7f6e2006e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e30033000 tx=0x7f6e30008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:19.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.808+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e380596c0 con 0x7f6e401f40f0 2024-05-09T06:28:19.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.994+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e4015cc20 con 0x7f6e401f40f0 2024-05-09T06:28:19.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.994+0000 7f6e3cff9640 1 -- 172.21.15.1:0/1243938558 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6e3805d9c0 con 0x7f6e401f40f0 2024-05-09T06:28:19.976 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.996+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 msgr2=0x7f6e2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.996+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 0x7f6e2006e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e30033000 tx=0x7f6e30008030 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 msgr2=0x7f6e401f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e38000f80 tx=0x7f6e38007bd0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 shutdown_connections 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e2006be30 0x7f6e2006e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401f40f0 0x7f6e401f45a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e401e49a0 0x7f6e401e4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 --2- 172.21.15.1:0/1243938558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e2c0039e0 0x7f6e4015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 >> 172.21.15.1:0/1243938558 conn(0x7f6e400cc4d0 msgr2=0x7f6e40070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 shutdown_connections 2024-05-09T06:28:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:19.997+0000 7f6e4636a640 1 -- 172.21.15.1:0/1243938558 wait complete. 2024-05-09T06:28:19.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:20.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.114+0000 7f4f62353640 1 Processor -- start 2024-05-09T06:28:20.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.114+0000 7f4f62353640 1 -- start start 2024-05-09T06:28:20.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f62353640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c15e190 0x7f4f5c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f62353640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f62353640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c161fe0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f62353640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5c15f580 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f62353640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c1622c0 con 0x7f4f5c15e190 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f608f0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f608f0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51118/0 (socket says 172.21.15.1:51118) 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f608f0640 1 -- 172.21.15.1:0/151208092 learned_addr learned my addr 172.21.15.1:0/151208092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.115+0000 7f4f5bfff640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c15e190 0x7f4f5c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f608f0640 1 -- 172.21.15.1:0/151208092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c15e190 msgr2=0x7f4f5c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f608f0640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c15e190 0x7f4f5c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f608f0640 1 -- 172.21.15.1:0/151208092 >> v1:172.21.15.6:6789/0 conn(0x7f4f5c0d1c70 legacy=0x7f4f5c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f608f0640 1 -- 172.21.15.1:0/151208092 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c1625a0 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f608f0640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009e50 tx=0x7f4f5402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617bb7ea7ce1d586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f5affd640 1 -- 172.21.15.1:0/151208092 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54036070 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f5affd640 1 -- 172.21.15.1:0/151208092 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f54031600 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.116+0000 7f4f5affd640 1 -- 172.21.15.1:0/151208092 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f540319d0 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 -- 172.21.15.1:0/151208092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d45f0 msgr2=0x7f4f5c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009e50 tx=0x7f4f5402a710 comp rx=0 tx=0).stop 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 -- 172.21.15.1:0/151208092 shutdown_connections 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b3de0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 --2- 172.21.15.1:0/151208092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c15e190 0x7f4f5c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 -- 172.21.15.1:0/151208092 >> 172.21.15.1:0/151208092 conn(0x7f4f5c073340 msgr2=0x7f4f5c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 -- 172.21.15.1:0/151208092 shutdown_connections 2024-05-09T06:28:20.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.117+0000 7f4f62353640 1 -- 172.21.15.1:0/151208092 wait complete. 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.118+0000 7f4f62353640 1 Processor -- start 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 -- start start 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f5c15e190 0x7f4f5c0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0b56d0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c161c60 con 0x7f4f5c15e190 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f62353640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c0b5850 con 0x7f4f5c0d45f0 2024-05-09T06:28:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51128/0 (socket says 172.21.15.1:51128) 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.119+0000 7f4f5bfff640 1 -- 172.21.15.1:0/542458027 learned_addr learned my addr 172.21.15.1:0/542458027 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f608f0640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f5c15e190 0x7f4f5c0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5b7fe640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 -- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 msgr2=0x7f4f5c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 -- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f5c15e190 msgr2=0x7f4f5c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f5c15e190 0x7f4f5c0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 -- 172.21.15.1:0/542458027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0b32b0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5b7fe640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f5bfff640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d5f0 tx=0x7f4f4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0125f0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c019070 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c0c7bd0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0129d0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.120+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c0c8190 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.121+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5c0752b0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.122+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f4c012b70 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.122+0000 7f4f58ff9640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 0x7f4f4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.122+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4f4c08b530 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.122+0000 7f4f5b7fe640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 0x7f4f4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.123+0000 7f4f5b7fe640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 0x7f4f4806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c15ff90 tx=0x7f4f50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.126+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f4c059b70 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.310+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4f5c08e1c0 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.311+0000 7f4f58ff9640 1 -- 172.21.15.1:0/542458027 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4f4c059470 con 0x7f4f5c0d1c70 2024-05-09T06:28:20.340 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 msgr2=0x7f4f4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 0x7f4f4806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c15ff90 tx=0x7f4f50009040 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 msgr2=0x7f4f5c14f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d5f0 tx=0x7f4f4c00dab0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 shutdown_connections 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f4806be30 0x7f4f4806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f5c15e190 0x7f4f5c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4f5c0d45f0 0x7f4f5c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 --2- 172.21.15.1:0/542458027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f5c0d1c70 0x7f4f5c14f5f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 >> 172.21.15.1:0/542458027 conn(0x7f4f5c073340 msgr2=0x7f4f5c0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 shutdown_connections 2024-05-09T06:28:20.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.313+0000 7f4f62353640 1 -- 172.21.15.1:0/542458027 wait complete. 2024-05-09T06:28:20.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:20.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.466+0000 7fc5e2f10640 1 Processor -- start 2024-05-09T06:28:20.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.466+0000 7fc5e2f10640 1 -- start start 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.467+0000 7fc5e2f10640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5dc161950 0x7fc5dc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.467+0000 7fc5e2f10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 0x7fc5dc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.467+0000 7fc5e2f10640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc0b33c0 con 0x7fc5dc0d0f80 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.467+0000 7fc5e2f10640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0d4dd0 con 0x7fc5dc161950 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.467+0000 7fc5e2f10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc0b3660 con 0x7fc5dc0d4940 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5dc161950 0x7fc5dc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 0x7fc5dc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5dc161950 0x7fc5dc0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33720/0 (socket says 172.21.15.1:33720) 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e14ad640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc5dc0d0f80 0x7fc5dc0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:52912/0 (socket says 172.21.15.1:52912) 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 -- 172.21.15.1:0/4068980484 learned_addr learned my addr 172.21.15.1:0/4068980484 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 -- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 msgr2=0x7fc5dc0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 0x7fc5dc0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 -- 172.21.15.1:0/4068980484 >> v1:172.21.15.6:6789/0 conn(0x7fc5dc0d0f80 legacy=0x7fc5dc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5e0cac640 1 -- 172.21.15.1:0/4068980484 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0b3940 con 0x7fc5dc161950 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5d37fe640 1 -- 172.21.15.1:0/4068980484 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633278936 0 0) 0x7fc5dc0b33c0 con 0x7fc5dc0d0f80 2024-05-09T06:28:20.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.468+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 0x7fc5dc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:20.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5e0cac640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc0d2400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0b3590 tx=0x7fc5d802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb72c368c92eb95a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5d37fe640 1 -- 172.21.15.1:0/4068980484 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8037070 con 0x7fc5dc161950 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5d37fe640 1 -- 172.21.15.1:0/4068980484 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d8032440 con 0x7fc5dc161950 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5d37fe640 1 -- 172.21.15.1:0/4068980484 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8032810 con 0x7fc5dc161950 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5e2f10640 1 -- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 msgr2=0x7fc5dc0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.469+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc0d2400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0b3590 tx=0x7fc5d802a250 comp rx=0 tx=0).stop 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 -- 172.21.15.1:0/4068980484 shutdown_connections 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d4940 0x7fc5dc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/4068980484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc0d2400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 -- 172.21.15.1:0/4068980484 >> 172.21.15.1:0/4068980484 conn(0x7fc5dc0d3e20 msgr2=0x7fc5dc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 -- 172.21.15.1:0/4068980484 shutdown_connections 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.470+0000 7fc5e2f10640 1 -- 172.21.15.1:0/4068980484 wait complete. 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.471+0000 7fc5e2f10640 1 Processor -- start 2024-05-09T06:28:20.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.471+0000 7fc5e2f10640 1 -- start start 2024-05-09T06:28:20.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d0f80 0x7fc5dc14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0b52d0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0b5150 con 0x7fc5dc161950 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e2f10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc0b5450 con 0x7fc5dc0d0f80 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51140/0 (socket says 172.21.15.1:51140) 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 -- 172.21.15.1:0/181441565 learned_addr learned my addr 172.21.15.1:0/181441565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e0cac640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d0f80 0x7fc5dc14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 -- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d0f80 msgr2=0x7fc5dc14e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d0f80 0x7fc5dc14e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 -- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 msgr2=0x7fc5dc14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 -- 172.21.15.1:0/181441565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0cac80 con 0x7fc5dc14fc60 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.472+0000 7fc5e14ad640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c7b0 tx=0x7fc5d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.473+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018070 con 0x7fc5dc14fc60 2024-05-09T06:28:20.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.473+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d400a640 con 0x7fc5dc14fc60 2024-05-09T06:28:20.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.473+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400aa70 con 0x7fc5dc14fc60 2024-05-09T06:28:20.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.473+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc14ebc0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.473+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc14f180 con 0x7fc5dc14fc60 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.474+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0752b0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.474+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5d401b070 con 0x7fc5dc14fc60 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.475+0000 7fc5d17fa640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 0x7fc5c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.475+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc5d408a830 con 0x7fc5dc14fc60 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.475+0000 7fc5e0cac640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 0x7fc5c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.475+0000 7fc5e0cac640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 0x7fc5c806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8034000 tx=0x7fc5d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.479+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5d4058de0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.675+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc5dc0d0bf0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.675+0000 7fc5d17fa640 1 -- 172.21.15.1:0/181441565 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc5d40586e0 con 0x7fc5dc14fc60 2024-05-09T06:28:20.657 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:20.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 msgr2=0x7fc5c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 0x7fc5c806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8034000 tx=0x7fc5d802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 msgr2=0x7fc5dc151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c7b0 tx=0x7fc5d400cc70 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 shutdown_connections 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c806be30 0x7fc5c806e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5dc14fc60 0x7fc5dc151a20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5dc161950 0x7fc5dc14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 --2- 172.21.15.1:0/181441565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5dc0d0f80 0x7fc5dc14e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 >> 172.21.15.1:0/181441565 conn(0x7fc5dc0d3e20 msgr2=0x7fc5dc06bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 shutdown_connections 2024-05-09T06:28:20.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:20.677+0000 7fc5e2f10640 1 -- 172.21.15.1:0/181441565 wait complete. 2024-05-09T06:28:20.670 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2024-05-09T06:28:20.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:20.670 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-09T06:28:20.704 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:20.704 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:28:20.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:20.704 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:28:20.774 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:20.774 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:20.791 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:20.792 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:20.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:20.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.010+0000 7f7a0ea29640 1 Processor -- start 2024-05-09T06:28:20.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.010+0000 7f7a0ea29640 1 -- start start 2024-05-09T06:28:20.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0ea29640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7a080b0080 0x7f7a08071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0ea29640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a080cbb60 0x7f7a0815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0ea29640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a080d04a0 con 0x7f7a0815fd70 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0ea29640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7a080cbfe0 con 0x7f7a080b0080 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0ea29640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a080722c0 con 0x7f7a080cbb60 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a07fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7a080b0080 0x7f7a08071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0cfc6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7a0815fd70 0x7f7a0815ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42470/0 (socket says 172.21.15.1:42470) 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a0cfc6640 1 -- 172.21.15.1:0/4040572318 learned_addr learned my addr 172.21.15.1:0/4040572318 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a077fe640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a080cbb60 0x7f7a0815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a07fff640 1 -- 172.21.15.1:0/4040572318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a080cbb60 msgr2=0x7f7a0815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a07fff640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a080cbb60 0x7f7a0815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.011+0000 7f7a07fff640 1 -- 172.21.15.1:0/4040572318 >> v1:172.21.15.1:6789/0 conn(0x7f7a0815fd70 legacy=0x7f7a0815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a06ffd640 1 -- 172.21.15.1:0/4040572318 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865869001 0 0) 0x7f7a080d04a0 con 0x7f7a0815fd70 2024-05-09T06:28:20.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a07fff640 1 -- 172.21.15.1:0/4040572318 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a080725a0 con 0x7f7a080b0080 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a07fff640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a08071ce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582170be9673c30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a06ffd640 1 -- 172.21.15.1:0/4040572318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8037070 con 0x7f7a080b0080 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a06ffd640 1 -- 172.21.15.1:0/4040572318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f8032440 con 0x7f7a080b0080 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.012+0000 7f7a06ffd640 1 -- 172.21.15.1:0/4040572318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8032810 con 0x7f7a080b0080 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 -- 172.21.15.1:0/4040572318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 msgr2=0x7f7a08071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a08071ce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802a180 comp rx=0 tx=0).stop 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 -- 172.21.15.1:0/4040572318 shutdown_connections 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a080cbb60 0x7f7a0815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/4040572318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a08071ce0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 -- 172.21.15.1:0/4040572318 >> 172.21.15.1:0/4040572318 conn(0x7f7a080d0b50 msgr2=0x7f7a080d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 -- 172.21.15.1:0/4040572318 shutdown_connections 2024-05-09T06:28:20.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.013+0000 7f7a0ea29640 1 -- 172.21.15.1:0/4040572318 wait complete. 2024-05-09T06:28:20.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.014+0000 7f7a0ea29640 1 Processor -- start 2024-05-09T06:28:20.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.014+0000 7f7a0ea29640 1 -- start start 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 0x7f7a081f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 0x7f7a081f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0815e7c0 con 0x7f7a080b0080 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08072b60 con 0x7f7a080cbb60 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0ea29640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0815f3f0 con 0x7f7a0815fd70 2024-05-09T06:28:20.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51152/0 (socket says 172.21.15.1:51152) 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a07fff640 1 -- 172.21.15.1:0/370013521 learned_addr learned my addr 172.21.15.1:0/370013521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a07fff640 1 -- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 msgr2=0x7f7a081f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a077fe640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 0x7f7a081f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.015+0000 7f7a0cfc6640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 0x7f7a081f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a07fff640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 0x7f7a081f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a07fff640 1 -- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 msgr2=0x7f7a081f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a07fff640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 0x7f7a081f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a07fff640 1 -- 172.21.15.1:0/370013521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a080b35b0 con 0x7f7a080b0080 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a077fe640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 0x7f7a081f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a0cfc6640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 0x7f7a081f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a07fff640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f79f80098c0 tx=0x7f79f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8037040 con 0x7f7a080b0080 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f8042070 con 0x7f7a080b0080 2024-05-09T06:28:21.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a080c8e80 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8030bb0 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.016+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a080c9440 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.017+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a080752b0 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.017+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79f8047460 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.018+0000 7f7a04ff9640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 0x7f79f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.018+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f79f80abd80 con 0x7f7a080b0080 2024-05-09T06:28:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.018+0000 7f7a077fe640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 0x7f79f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.018+0000 7f7a077fe640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 0x7f79f406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006fd0 tx=0x7f79fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.022+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79f8032e20 con 0x7f7a080b0080 2024-05-09T06:28:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.207+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a080c8b20 con 0x7f7a080b0080 2024-05-09T06:28:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.208+0000 7f7a04ff9640 1 -- 172.21.15.1:0/370013521 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f79f807a330 con 0x7f7a080b0080 2024-05-09T06:28:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:21.190 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 msgr2=0x7f79f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 0x7f79f406e320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006fd0 tx=0x7f79fc008040 comp rx=0 tx=0).stop 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 msgr2=0x7f7a080c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f79f80098c0 tx=0x7f79f80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 shutdown_connections 2024-05-09T06:28:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79f406be60 0x7f79f406e320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a0815fd70 0x7f7a081f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a080cbb60 0x7f7a081f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 --2- 172.21.15.1:0/370013521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a080b0080 0x7f7a080c85e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 >> 172.21.15.1:0/370013521 conn(0x7f7a080d0b50 msgr2=0x7f7a080d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 shutdown_connections 2024-05-09T06:28:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.210+0000 7f7a0ea29640 1 -- 172.21.15.1:0/370013521 wait complete. 2024-05-09T06:28:21.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:21.204 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:21.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 Processor -- start 2024-05-09T06:28:21.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 -- start start 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0b9815e270 0x7f0b980cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 0x7f0b980ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b980b1340 con 0x7f0b980d08b0 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0b980cc260 con 0x7f0b9815e270 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.329+0000 7f0b9db61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b980d1020 con 0x7f0b980ceba0 2024-05-09T06:28:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b977fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0b9815e270 0x7f0b980cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b96ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0b980d08b0 0x7f0b980ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42472/0 (socket says 172.21.15.1:42472) 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b96ffd640 1 -- 172.21.15.1:0/797549691 learned_addr learned my addr 172.21.15.1:0/797549691 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b97fff640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 0x7f0b980ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b977fe640 1 -- 172.21.15.1:0/797549691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 msgr2=0x7f0b980ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b977fe640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 0x7f0b980ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b977fe640 1 -- 172.21.15.1:0/797549691 >> v1:172.21.15.1:6789/0 conn(0x7f0b980d08b0 legacy=0x7f0b980ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b967fc640 1 -- 172.21.15.1:0/797549691 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479072948 0 0) 0x7f0b8c001110 con 0x7f0b980d08b0 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b977fe640 1 -- 172.21.15.1:0/797549691 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b980cc4b0 con 0x7f0b9815e270 2024-05-09T06:28:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.330+0000 7f0b97fff640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 0x7f0b980ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:21.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b977fe640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9815e270 0x7f0b980cf180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009d90 tx=0x7f0b8802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275f29aa93d4f678 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b967fc640 1 -- 172.21.15.1:0/797549691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007740 con 0x7f0b9815e270 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b967fc640 1 -- 172.21.15.1:0/797549691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b880078e0 con 0x7f0b9815e270 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b967fc640 1 -- 172.21.15.1:0/797549691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007cb0 con 0x7f0b9815e270 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b9db61640 1 -- 172.21.15.1:0/797549691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9815e270 msgr2=0x7f0b980cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.331+0000 7f0b9db61640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9815e270 0x7f0b980cf180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009d90 tx=0x7f0b8802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 -- 172.21.15.1:0/797549691 shutdown_connections 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b980ceba0 0x7f0b980ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 --2- 172.21.15.1:0/797549691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9815e270 0x7f0b980cf180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 -- 172.21.15.1:0/797549691 >> 172.21.15.1:0/797549691 conn(0x7f0b980d1c10 msgr2=0x7f0b980d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 -- 172.21.15.1:0/797549691 shutdown_connections 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.332+0000 7f0b9db61640 1 -- 172.21.15.1:0/797549691 wait complete. 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.333+0000 7f0b9db61640 1 Processor -- start 2024-05-09T06:28:21.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.333+0000 7f0b9db61640 1 -- start start 2024-05-09T06:28:21.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 0x7f0b980c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b9815e270 0x7f0b98152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d0d20 con 0x7f0b980d08b0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980cca70 con 0x7f0b980ceba0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9815e650 con 0x7f0b9815e270 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51168/0 (socket says 172.21.15.1:51168) 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 -- 172.21.15.1:0/4177643700 learned_addr learned my addr 172.21.15.1:0/4177643700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b97fff640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b9815e270 0x7f0b98152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b977fe640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 0x7f0b980c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 -- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b9815e270 msgr2=0x7f0b98152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b9815e270 0x7f0b98152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 -- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 msgr2=0x7f0b980c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 0x7f0b980c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 -- 172.21.15.1:0/4177643700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b980c7bc0 con 0x7f0b980d08b0 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b977fe640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 0x7f0b980c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:21.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b96ffd640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c430 tx=0x7f0b8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0074e0 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c0047e0 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c004bd0 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.334+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b98156f50 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.335+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b98156990 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.335+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b980752b0 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.336+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b8c011070 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.336+0000 7f0b7ffff640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 0x7f0b8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.336+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b8c08ae40 con 0x7f0b980d08b0 2024-05-09T06:28:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.336+0000 7f0b977fe640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 0x7f0b8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.337+0000 7f0b977fe640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 0x7f0b8406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0b88034000 tx=0x7f0b8802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.341+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b980b1340 con 0x7f0b980d08b0 2024-05-09T06:28:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.523+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b980d29e0 con 0x7f0b980d08b0 2024-05-09T06:28:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.523+0000 7f0b7ffff640 1 -- 172.21.15.1:0/4177643700 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0b8c059480 con 0x7f0b980d08b0 2024-05-09T06:28:21.505 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 msgr2=0x7f0b8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 0x7f0b8406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0b88034000 tx=0x7f0b8802f000 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 msgr2=0x7f0b98155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c430 tx=0x7f0b8c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 shutdown_connections 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b8406be60 0x7f0b8406e320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b9815e270 0x7f0b98152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b980d08b0 0x7f0b98155f40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 --2- 172.21.15.1:0/4177643700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b980ceba0 0x7f0b980c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 >> 172.21.15.1:0/4177643700 conn(0x7f0b980d1c10 msgr2=0x7f0b980af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 shutdown_connections 2024-05-09T06:28:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.525+0000 7f0b9db61640 1 -- 172.21.15.1:0/4177643700 wait complete. 2024-05-09T06:28:21.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.643+0000 7f4fc0b63640 1 Processor -- start 2024-05-09T06:28:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.643+0000 7f4fc0b63640 1 -- start start 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fc0b63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4fbc06b8f0 0x7f4fbc15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fc0b63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fc0b63640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fbc072030 con 0x7f4fbc159b90 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fc0b63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4fbc072cb0 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fc0b63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc06c150 con 0x7f4fbc0d1640 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fba59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4fbc159b90 0x7f4fbc06b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42474/0 (socket says 172.21.15.1:42474) 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fba59c640 1 -- 172.21.15.1:0/1903825558 learned_addr learned my addr 172.21.15.1:0/1903825558 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/1903825558 >> v2:172.21.15.6:3300/0 conn(0x7f4fbc06b8f0 0x7f4fbc15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fbad9d640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099027584 0 0) 0x7f4fbc072030 con 0x7f4fbc159b90 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.644+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fa8003520 con 0x7f4fbc159b90 2024-05-09T06:28:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 119362960 0 0) 0x7f4fa8003520 con 0x7f4fbc159b90 2024-05-09T06:28:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb9d9b640 1 -- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 msgr2=0x7f4fbc06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc06f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb9d9b640 1 -- 172.21.15.1:0/1903825558 >> v1:172.21.15.1:6789/0 conn(0x7f4fbc159b90 legacy=0x7f4fbc06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb9d9b640 1 -- 172.21.15.1:0/1903825558 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc072310 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fbad9d640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc06f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.645+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc15c3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097a0 tx=0x7f4fa402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfb70a92b55bd4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4037070 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa402ab10 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fb959a640 1 -- 172.21.15.1:0/1903825558 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0019f0 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 -- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc06b8f0 msgr2=0x7f4fbc15c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc15c3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097a0 tx=0x7f4fa402a080 comp rx=0 tx=0).stop 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 -- 172.21.15.1:0/1903825558 shutdown_connections 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc06f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/1903825558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc15c3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 -- 172.21.15.1:0/1903825558 >> 172.21.15.1:0/1903825558 conn(0x7f4fbc0d0b50 msgr2=0x7f4fbc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 -- 172.21.15.1:0/1903825558 shutdown_connections 2024-05-09T06:28:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.646+0000 7f4fc0b63640 1 -- 172.21.15.1:0/1903825558 wait complete. 2024-05-09T06:28:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 Processor -- start 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 -- start start 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc159b90 0x7f4fbc0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc15d580 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc0728d0 con 0x7f4fbc159b90 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.648+0000 7f4fc0b63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc071ce0 con 0x7f4fbc0d1640 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51170/0 (socket says 172.21.15.1:51170) 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 -- 172.21.15.1:0/3911961630 learned_addr learned my addr 172.21.15.1:0/3911961630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fbad9d640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc159b90 0x7f4fbc0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 -- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 msgr2=0x7f4fbc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 -- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc159b90 msgr2=0x7f4fbc0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc159b90 0x7f4fbc0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 -- 172.21.15.1:0/3911961630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc0cdc90 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fba59c640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0018d0 tx=0x7f4fac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac017070 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac012400 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac012830 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc0cdf70 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.649+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc0c3780 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.650+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc0c4bc0 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.651+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fac01a070 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.651+0000 7f4fb2ffd640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 0x7f4f9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.652+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 0x7f4f9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.652+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4fac08b1f0 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.652+0000 7f4fb9d9b640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 0x7f4f9c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4034000 tx=0x7f4fa402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.656+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4fac059830 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.839+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4fbc072e30 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.840+0000 7f4fb2ffd640 1 -- 172.21.15.1:0/3911961630 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4fac059130 con 0x7f4fbc06b8f0 2024-05-09T06:28:21.822 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 msgr2=0x7f4f9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 0x7f4f9c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4034000 tx=0x7f4fa402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 msgr2=0x7f4fbc156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0018d0 tx=0x7f4fac00a430 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 shutdown_connections 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4f9c06be30 0x7f4f9c06e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fbc159b90 0x7f4fbc0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fbc0d1640 0x7f4fbc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 --2- 172.21.15.1:0/3911961630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fbc06b8f0 0x7f4fbc156810 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 >> 172.21.15.1:0/3911961630 conn(0x7f4fbc0d0b50 msgr2=0x7f4fbc15bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.841+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 shutdown_connections 2024-05-09T06:28:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.842+0000 7f4fc0b63640 1 -- 172.21.15.1:0/3911961630 wait complete. 2024-05-09T06:28:21.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:21.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.960+0000 7f48eb82c640 1 Processor -- start 2024-05-09T06:28:21.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.960+0000 7f48eb82c640 1 -- start start 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48eb82c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f48e4074660 0x7f48e4160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48eb82c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 0x7f48e415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48eb82c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e415cba0 con 0x7f48e40cd450 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48eb82c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f48e415ce80 con 0x7f48e4074660 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48eb82c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f48e415d160 con 0x7f48e41605d0 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f48e4074660 0x7f48e4160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f48e4074660 0x7f48e4160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51172/0 (socket says 172.21.15.1:51172) 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 -- 172.21.15.1:0/3206513247 learned_addr learned my addr 172.21.15.1:0/3206513247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:21.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e9dc9640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 0x7f48e415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 -- 172.21.15.1:0/3206513247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 msgr2=0x7f48e415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 0x7f48e415d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 -- 172.21.15.1:0/3206513247 >> v1:172.21.15.6:6789/0 conn(0x7f48e40cd450 legacy=0x7f48e415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e8dc7640 1 -- 172.21.15.1:0/3206513247 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e415fb70 con 0x7f48e4074660 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.961+0000 7f48e9dc9640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 0x7f48e415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.962+0000 7f48e8dc7640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e4074660 0x7f48e4160090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e23b5003bf6b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.962+0000 7f48dbfff640 1 -- 172.21.15.1:0/3206513247 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037070 con 0x7f48e4074660 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.962+0000 7f48dbfff640 1 -- 172.21.15.1:0/3206513247 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d4032440 con 0x7f48e4074660 2024-05-09T06:28:21.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.962+0000 7f48dbfff640 1 -- 172.21.15.1:0/3206513247 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4032810 con 0x7f48e4074660 2024-05-09T06:28:21.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 -- 172.21.15.1:0/3206513247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e4074660 msgr2=0x7f48e4160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e4074660 0x7f48e4160090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009870 tx=0x7f48d402a210 comp rx=0 tx=0).stop 2024-05-09T06:28:21.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 -- 172.21.15.1:0/3206513247 shutdown_connections 2024-05-09T06:28:21.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e41605d0 0x7f48e415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 --2- 172.21.15.1:0/3206513247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e4074660 0x7f48e4160090 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 -- 172.21.15.1:0/3206513247 >> 172.21.15.1:0/3206513247 conn(0x7f48e40b1da0 msgr2=0x7f48e40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 -- 172.21.15.1:0/3206513247 shutdown_connections 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.963+0000 7f48eb82c640 1 -- 172.21.15.1:0/3206513247 wait complete. 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.964+0000 7f48eb82c640 1 Processor -- start 2024-05-09T06:28:21.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.964+0000 7f48eb82c640 1 -- start start 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e4074660 0x7f48e40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48e40cd450 0x7f48e4099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f48e415dc60 con 0x7f48e41605d0 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f48e415dae0 con 0x7f48e40cd450 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48eb82c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f48e415dde0 con 0x7f48e4074660 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51174/0 (socket says 172.21.15.1:51174) 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 -- 172.21.15.1:0/567322514 learned_addr learned my addr 172.21.15.1:0/567322514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e8dc7640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48e40cd450 0x7f48e4099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e95c8640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e4074660 0x7f48e40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 -- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e4074660 msgr2=0x7f48e40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e4074660 0x7f48e40b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 -- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48e40cd450 msgr2=0x7f48e4099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48e40cd450 0x7f48e4099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 -- 172.21.15.1:0/567322514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e41f63e0 con 0x7f48e41605d0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.965+0000 7f48e9dc9640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00cef0 tx=0x7f48dc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.966+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc01a070 con 0x7f48e41605d0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.966+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc00d860 con 0x7f48e41605d0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.966+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e40c4310 con 0x7f48e41605d0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.966+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00dc50 con 0x7f48e41605d0 2024-05-09T06:28:21.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.966+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e40c48d0 con 0x7f48e41605d0 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.967+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e40752b0 con 0x7f48e41605d0 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.967+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48dc01d070 con 0x7f48e41605d0 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.968+0000 7f48d9ffb640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 0x7f48c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.968+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f48dc08bb70 con 0x7f48e41605d0 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.968+0000 7f48e95c8640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 0x7f48c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.969+0000 7f48e95c8640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 0x7f48c806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0004ea0 tx=0x7f48e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:21.972+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f48dc05a080 con 0x7f48e41605d0 2024-05-09T06:28:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.156+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48e40c4050 con 0x7f48e41605d0 2024-05-09T06:28:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.157+0000 7f48d9ffb640 1 -- 172.21.15.1:0/567322514 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f48dc059980 con 0x7f48e41605d0 2024-05-09T06:28:22.140 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 msgr2=0x7f48c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 0x7f48c806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0004ea0 tx=0x7f48e0009040 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 msgr2=0x7f48e41f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00cef0 tx=0x7f48dc00a710 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 shutdown_connections 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f48c806be30 0x7f48c806e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48e41605d0 0x7f48e41f55d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48e40cd450 0x7f48e4099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 --2- 172.21.15.1:0/567322514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f48e4074660 0x7f48e40b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 >> 172.21.15.1:0/567322514 conn(0x7f48e40b1da0 msgr2=0x7f48e415a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 shutdown_connections 2024-05-09T06:28:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.159+0000 7f48eb82c640 1 -- 172.21.15.1:0/567322514 wait complete. 2024-05-09T06:28:22.152 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.277+0000 7f58db7b7640 1 Processor -- start 2024-05-09T06:28:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.277+0000 7f58db7b7640 1 -- start start 2024-05-09T06:28:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58db7b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58d415bfd0 0x7f58d40cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58db7b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d40cf050 0x7f58d40cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58db7b7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d40cfee0 con 0x7f58d40d0230 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58db7b7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f58d40ccb40 con 0x7f58d415bfd0 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58db7b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f58d40cce20 con 0x7f58d40cf050 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9553640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58d415bfd0 0x7f58d40cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9d54640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f58d40d0230 0x7f58d406ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42480/0 (socket says 172.21.15.1:42480) 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9d54640 1 -- 172.21.15.1:0/1507917031 learned_addr learned my addr 172.21.15.1:0/1507917031 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d8d52640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d40cf050 0x7f58d40cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9553640 1 -- 172.21.15.1:0/1507917031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d40cf050 msgr2=0x7f58d40cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9553640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d40cf050 0x7f58d40cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9553640 1 -- 172.21.15.1:0/1507917031 >> v1:172.21.15.1:6789/0 conn(0x7f58d40d0230 legacy=0x7f58d406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58cbfff640 1 -- 172.21.15.1:0/1507917031 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114358569 0 0) 0x7f58d40cfee0 con 0x7f58d40d0230 2024-05-09T06:28:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.278+0000 7f58d9553640 1 -- 172.21.15.1:0/1507917031 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d40cd100 con 0x7f58d415bfd0 2024-05-09T06:28:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.279+0000 7f58d9553640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d415bfd0 0x7f58d40cf820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097a0 tx=0x7f58d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f9786444b9a6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.279+0000 7f58cbfff640 1 -- 172.21.15.1:0/1507917031 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037070 con 0x7f58d415bfd0 2024-05-09T06:28:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.279+0000 7f58cbfff640 1 -- 172.21.15.1:0/1507917031 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0032440 con 0x7f58d415bfd0 2024-05-09T06:28:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.279+0000 7f58cbfff640 1 -- 172.21.15.1:0/1507917031 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0032810 con 0x7f58d415bfd0 2024-05-09T06:28:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 -- 172.21.15.1:0/1507917031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d415bfd0 msgr2=0x7f58d40cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d415bfd0 0x7f58d40cf820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097a0 tx=0x7f58d002a180 comp rx=0 tx=0).stop 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 -- 172.21.15.1:0/1507917031 shutdown_connections 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d40cf050 0x7f58d40cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 --2- 172.21.15.1:0/1507917031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d415bfd0 0x7f58d40cf820 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 -- 172.21.15.1:0/1507917031 >> 172.21.15.1:0/1507917031 conn(0x7f58d40d1c50 msgr2=0x7f58d40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 -- 172.21.15.1:0/1507917031 shutdown_connections 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.280+0000 7f58db7b7640 1 -- 172.21.15.1:0/1507917031 wait complete. 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.281+0000 7f58db7b7640 1 Processor -- start 2024-05-09T06:28:22.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.281+0000 7f58db7b7640 1 -- start start 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40d0230 0x7f58d414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d415bfd0 0x7f58d415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40cf4b0 con 0x7f58d40cf050 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f58d415c3b0 con 0x7f58d40d0230 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58db7b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f58d40cc760 con 0x7f58d415bfd0 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51200/0 (socket says 172.21.15.1:51200) 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 -- 172.21.15.1:0/3260956025 learned_addr learned my addr 172.21.15.1:0/3260956025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 -- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d415bfd0 msgr2=0x7f58d415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d415bfd0 0x7f58d415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 -- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40d0230 msgr2=0x7f58d414f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40d0230 0x7f58d414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.282+0000 7f58d9553640 1 -- 172.21.15.1:0/3260956025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d415dca0 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58d9553640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f58d0035040 tx=0x7f58d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037040 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0030770 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0030b40 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d415df80 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.283+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d40c8a20 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.284+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d40752b0 con 0x7f58d40cf050 2024-05-09T06:28:22.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.284+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58d0044020 con 0x7f58d40cf050 2024-05-09T06:28:22.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.285+0000 7f58c9ffb640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.285+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f58d00ab870 con 0x7f58d40cf050 2024-05-09T06:28:22.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.285+0000 7f58d8d52640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.285+0000 7f58d8d52640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 0x7f58b806e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ca9e0 tx=0x7f58c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.289+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58d0032e20 con 0x7f58d40cf050 2024-05-09T06:28:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.473+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f58d40b7bd0 con 0x7f58d40cf050 2024-05-09T06:28:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.474+0000 7f58c9ffb640 1 -- 172.21.15.1:0/3260956025 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f58d0079e20 con 0x7f58d40cf050 2024-05-09T06:28:22.456 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 msgr2=0x7f58b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 0x7f58b806e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ca9e0 tx=0x7f58c4008040 comp rx=0 tx=0).stop 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 msgr2=0x7f58d40c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f58d0035040 tx=0x7f58d00046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 shutdown_connections 2024-05-09T06:28:22.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f58b806be30 0x7f58b806e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58d415bfd0 0x7f58d415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40d0230 0x7f58d414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 --2- 172.21.15.1:0/3260956025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58d40cf050 0x7f58d40c84e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 >> 172.21.15.1:0/3260956025 conn(0x7f58d40d1c50 msgr2=0x7f58d40b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 shutdown_connections 2024-05-09T06:28:22.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.476+0000 7f58db7b7640 1 -- 172.21.15.1:0/3260956025 wait complete. 2024-05-09T06:28:22.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:22.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.591+0000 7fe980742640 1 Processor -- start 2024-05-09T06:28:22.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.591+0000 7fe980742640 1 -- start start 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.592+0000 7fe980742640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 0x7fe9780b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.592+0000 7fe980742640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9780b2a60 0x7fe9780b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.592+0000 7fe980742640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9780b5230 con 0x7fe9780b4da0 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.592+0000 7fe980742640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe978071c20 con 0x7fe9780b2a60 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.592+0000 7fe980742640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe97806eb40 con 0x7fe9781454b0 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9780b2a60 0x7fe9780b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9780b2a60 0x7fe9780b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51216/0 (socket says 172.21.15.1:51216) 2024-05-09T06:28:22.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 -- 172.21.15.1:0/2152326571 learned_addr learned my addr 172.21.15.1:0/2152326571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97e4de640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 0x7fe9780b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 -- 172.21.15.1:0/2152326571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 msgr2=0x7fe9780b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 0x7fe9780b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 -- 172.21.15.1:0/2152326571 >> v1:172.21.15.6:6789/0 conn(0x7fe9780b4da0 legacy=0x7fe97815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97dcdd640 1 -- 172.21.15.1:0/2152326571 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97806ee20 con 0x7fe9780b2a60 2024-05-09T06:28:22.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.593+0000 7fe97e4de640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 0x7fe9780b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:22.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe97dcdd640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b2a60 0x7fe9780b4860 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc8ca663ba0170e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe97d4dc640 1 -- 172.21.15.1:0/2152326571 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037070 con 0x7fe9780b2a60 2024-05-09T06:28:22.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe97d4dc640 1 -- 172.21.15.1:0/2152326571 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c032440 con 0x7fe9780b2a60 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe97d4dc640 1 -- 172.21.15.1:0/2152326571 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c032810 con 0x7fe9780b2a60 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 -- 172.21.15.1:0/2152326571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b2a60 msgr2=0x7fe9780b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b2a60 0x7fe9780b4860 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02a210 comp rx=0 tx=0).stop 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 -- 172.21.15.1:0/2152326571 shutdown_connections 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b2a60 0x7fe9780b4860 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 --2- 172.21.15.1:0/2152326571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9781454b0 0x7fe9780b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 -- 172.21.15.1:0/2152326571 >> 172.21.15.1:0/2152326571 conn(0x7fe9780cc4b0 msgr2=0x7fe9780cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 -- 172.21.15.1:0/2152326571 shutdown_connections 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.594+0000 7fe980742640 1 -- 172.21.15.1:0/2152326571 wait complete. 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 Processor -- start 2024-05-09T06:28:22.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 -- start start 2024-05-09T06:28:22.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9780b2a60 0x7fe97813f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9781454b0 0x7fe9780b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe978145890 con 0x7fe9780b4da0 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe97806dbc0 con 0x7fe9781454b0 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.596+0000 7fe980742640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe97806e0e0 con 0x7fe9780b2a60 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51230/0 (socket says 172.21.15.1:51230) 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 -- 172.21.15.1:0/3421656223 learned_addr learned my addr 172.21.15.1:0/3421656223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97e4de640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9780b2a60 0x7fe97813f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97ecdf640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9781454b0 0x7fe9780b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 -- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9780b2a60 msgr2=0x7fe97813f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9780b2a60 0x7fe97813f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 -- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9781454b0 msgr2=0x7fe9780b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9781454b0 0x7fe9780b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 -- 172.21.15.1:0/3421656223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9780c92e0 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe97dcdd640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe96c000c00 tx=0x7fe96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037040 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c042070 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9780c95c0 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c030bd0 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.597+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9780c9b80 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.598+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9780aca30 con 0x7fe9780b4da0 2024-05-09T06:28:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.599+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe96c047460 con 0x7fe9780b4da0 2024-05-09T06:28:22.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.599+0000 7fe976ffd640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 0x7fe96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.599+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe96c0abda0 con 0x7fe9780b4da0 2024-05-09T06:28:22.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.599+0000 7fe97e4de640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 0x7fe96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.600+0000 7fe97e4de640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 0x7fe96406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe97000ab60 tx=0x7fe970008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.603+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe96c079d00 con 0x7fe9780b4da0 2024-05-09T06:28:22.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.798+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe97804fe90 con 0x7fe9780b4da0 2024-05-09T06:28:22.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.798+0000 7fe976ffd640 1 -- 172.21.15.1:0/3421656223 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fe97804fe90 con 0x7fe9780b4da0 2024-05-09T06:28:22.781 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:22.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 msgr2=0x7fe96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 0x7fe96406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe97000ab60 tx=0x7fe970008040 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 msgr2=0x7fe9780ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe96c000c00 tx=0x7fe96c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 shutdown_connections 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe96406be30 0x7fe96406e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9781454b0 0x7fe9780b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9780b4da0 0x7fe9780ceec0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 --2- 172.21.15.1:0/3421656223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9780b2a60 0x7fe97813f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 >> 172.21.15.1:0/3421656223 conn(0x7fe9780cc4b0 msgr2=0x7fe978071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 shutdown_connections 2024-05-09T06:28:22.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.800+0000 7fe980742640 1 -- 172.21.15.1:0/3421656223 wait complete. 2024-05-09T06:28:22.793 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:22.793 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:22.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.971+0000 7fe47184d640 1 Processor -- start 2024-05-09T06:28:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 -- start start 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe46c1454b0 0x7fe46c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 0x7fe46c15f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe46c0b5230 con 0x7fe46c1454b0 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe46c06eb40 con 0x7fe46c0b4da0 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.972+0000 7fe47184d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe46c06ee20 con 0x7fe46c0b2a60 2024-05-09T06:28:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe46c1454b0 0x7fe46c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe46c1454b0 0x7fe46c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51236/0 (socket says 172.21.15.1:51236) 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 -- 172.21.15.1:0/3134100551 learned_addr learned my addr 172.21.15.1:0/3134100551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46b7fe640 1 --2- 172.21.15.1:0/3134100551 >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 0x7fe46c15f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46a7fc640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 -- 172.21.15.1:0/3134100551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 msgr2=0x7fe46c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 -- 172.21.15.1:0/3134100551 >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 msgr2=0x7fe46c15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 --2- 172.21.15.1:0/3134100551 >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 0x7fe46c15f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46affd640 1 -- 172.21.15.1:0/3134100551 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c06d8e0 con 0x7fe46c1454b0 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46b7fe640 1 --2- 172.21.15.1:0/3134100551 >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 0x7fe46c15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.973+0000 7fe46a7fc640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe46affd640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c1454b0 0x7fe46c0b4320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe46c06c210 tx=0x7fe46402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0edaec6055b0b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe469ffb640 1 -- 172.21.15.1:0/3134100551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464036070 con 0x7fe46c1454b0 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe469ffb640 1 -- 172.21.15.1:0/3134100551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe464031600 con 0x7fe46c1454b0 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe469ffb640 1 -- 172.21.15.1:0/3134100551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4640319d0 con 0x7fe46c1454b0 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 -- 172.21.15.1:0/3134100551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c1454b0 msgr2=0x7fe46c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c1454b0 0x7fe46c0b4320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe46c06c210 tx=0x7fe46402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 -- 172.21.15.1:0/3134100551 shutdown_connections 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 --2- 172.21.15.1:0/3134100551 >> v2:172.21.15.6:3300/0 conn(0x7fe46c0b4da0 0x7fe46c15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 --2- 172.21.15.1:0/3134100551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c1454b0 0x7fe46c0b4320 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.974+0000 7fe47184d640 1 -- 172.21.15.1:0/3134100551 >> 172.21.15.1:0/3134100551 conn(0x7fe46c0cc4b0 msgr2=0x7fe46c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.975+0000 7fe47184d640 1 -- 172.21.15.1:0/3134100551 shutdown_connections 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.975+0000 7fe47184d640 1 -- 172.21.15.1:0/3134100551 wait complete. 2024-05-09T06:28:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.976+0000 7fe47184d640 1 Processor -- start 2024-05-09T06:28:22.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.976+0000 7fe47184d640 1 -- start start 2024-05-09T06:28:22.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.976+0000 7fe47184d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 0x7fe46c12f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c06e0e0 con 0x7fe46c0b4da0 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c145890 con 0x7fe46c1454b0 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe46c06e260 con 0x7fe46c0b2a60 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51248/0 (socket says 172.21.15.1:51248) 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 -- 172.21.15.1:0/2870581794 learned_addr learned my addr 172.21.15.1:0/2870581794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46b7fe640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 0x7fe46c12f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46affd640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 -- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 msgr2=0x7fe46c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 -- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 msgr2=0x7fe46c12f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 0x7fe46c12f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 -- 172.21.15.1:0/2870581794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c12f7c0 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46b7fe640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 0x7fe46c12f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe46a7fc640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c820 tx=0x7fe45400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454015070 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe454007a30 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c12faa0 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454007de0 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.977+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c0a9d20 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.978+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c0752b0 con 0x7fe46c0b4da0 2024-05-09T06:28:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.979+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe454018070 con 0x7fe46c0b4da0 2024-05-09T06:28:22.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.979+0000 7fe463fff640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 0x7fe44c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:22.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.980+0000 7fe46affd640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 0x7fe44c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:22.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.980+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe45408b180 con 0x7fe46c0b4da0 2024-05-09T06:28:22.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.980+0000 7fe46affd640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 0x7fe44c06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe464033000 tx=0x7fe464009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:22.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:22.984+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe454059730 con 0x7fe46c0b4da0 2024-05-09T06:28:23.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.169+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe46c072fa0 con 0x7fe46c0b4da0 2024-05-09T06:28:23.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.170+0000 7fe463fff640 1 -- 172.21.15.1:0/2870581794 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7fe454059030 con 0x7fe46c0b4da0 2024-05-09T06:28:23.152 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:23.152 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:23.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 msgr2=0x7fe44c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 0x7fe44c06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe464033000 tx=0x7fe464009000 comp rx=0 tx=0).stop 2024-05-09T06:28:23.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 msgr2=0x7fe46c12ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c820 tx=0x7fe45400cce0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 shutdown_connections 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe44c06be30 0x7fe44c06e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46c1454b0 0x7fe46c12f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46c0b4da0 0x7fe46c12ec00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 --2- 172.21.15.1:0/2870581794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe46c0b2a60 0x7fe46c15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 >> 172.21.15.1:0/2870581794 conn(0x7fe46c0cc4b0 msgr2=0x7fe46c0aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 shutdown_connections 2024-05-09T06:28:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.172+0000 7fe47184d640 1 -- 172.21.15.1:0/2870581794 wait complete. 2024-05-09T06:28:23.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:23.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.290+0000 7f5500b61640 1 Processor -- start 2024-05-09T06:28:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.291+0000 7f5500b61640 1 -- start start 2024-05-09T06:28:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f5500b61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 0x7f54fc06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f5500b61640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 0x7f54fc15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f5500b61640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc06cf40 con 0x7f54fc0b5260 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f5500b61640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f54fc156980 con 0x7f54fc0b4de0 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f5500b61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc06d870 con 0x7f54fc15c4c0 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 0x7f54fc06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54f9d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 0x7f54fc15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54fad9d640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f54fc0b5260 0x7f54fc070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:52938/0 (socket says 172.21.15.1:52938) 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54fad9d640 1 -- 172.21.15.1:0/1253346780 learned_addr learned my addr 172.21.15.1:0/1253346780 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54f959a640 1 -- 172.21.15.1:0/1253346780 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063610087 0 0) 0x7f54fc06cf40 con 0x7f54fc0b5260 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.292+0000 7f54f959a640 1 -- 172.21.15.1:0/1253346780 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54e4003520 con 0x7f54fc0b5260 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54fa59c640 1 -- 172.21.15.1:0/1253346780 >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 msgr2=0x7f54fc15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54fa59c640 1 --2- 172.21.15.1:0/1253346780 >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 0x7f54fc15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54fa59c640 1 -- 172.21.15.1:0/1253346780 >> v1:172.21.15.6:6789/0 conn(0x7f54fc0b5260 legacy=0x7f54fc070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54fa59c640 1 -- 172.21.15.1:0/1253346780 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc06d220 con 0x7f54fc15c4c0 2024-05-09T06:28:23.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54f9d9b640 1 --2- 172.21.15.1:0/1253346780 >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 0x7f54fc15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54fa59c640 1 --2- 172.21.15.1:0/1253346780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 0x7f54fc06dd00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0097a0 tx=0x7f54ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb55291a9aaa34b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54f959a640 1 -- 172.21.15.1:0/1253346780 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec037070 con 0x7f54fc15c4c0 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.293+0000 7f54f959a640 1 -- 172.21.15.1:0/1253346780 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec032440 con 0x7f54fc15c4c0 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f54f959a640 1 -- 172.21.15.1:0/1253346780 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0327f0 con 0x7f54fc15c4c0 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 -- 172.21.15.1:0/1253346780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 msgr2=0x7f54fc06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 --2- 172.21.15.1:0/1253346780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 0x7f54fc06dd00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0097a0 tx=0x7f54ec02a180 comp rx=0 tx=0).stop 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 -- 172.21.15.1:0/1253346780 shutdown_connections 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 --2- 172.21.15.1:0/1253346780 >> v2:172.21.15.6:3300/0 conn(0x7f54fc0b4de0 0x7f54fc15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 --2- 172.21.15.1:0/1253346780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc15c4c0 0x7f54fc06dd00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 -- 172.21.15.1:0/1253346780 >> 172.21.15.1:0/1253346780 conn(0x7f54fc0cc4d0 msgr2=0x7f54fc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 -- 172.21.15.1:0/1253346780 shutdown_connections 2024-05-09T06:28:23.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.294+0000 7f5500b61640 1 -- 172.21.15.1:0/1253346780 wait complete. 2024-05-09T06:28:23.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.295+0000 7f5500b61640 1 Processor -- start 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 -- start start 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc0b5260 0x7f54fc1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54fc15c4c0 0x7f54fc13d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc0b2dc0 con 0x7f54fc0b4de0 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc0b2c40 con 0x7f54fc15c4c0 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f5500b61640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc15cde0 con 0x7f54fc0b5260 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51260/0 (socket says 172.21.15.1:51260) 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.296+0000 7f54fa59c640 1 -- 172.21.15.1:0/2699798106 learned_addr learned my addr 172.21.15.1:0/2699798106 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fad9d640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54fc15c4c0 0x7f54fc13d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54f9d9b640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc0b5260 0x7f54fc1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 -- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc0b5260 msgr2=0x7f54fc1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc0b5260 0x7f54fc1f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 -- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54fc15c4c0 msgr2=0x7f54fc13d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54fc15c4c0 0x7f54fc13d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 -- 172.21.15.1:0/2699798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc0b0740 con 0x7f54fc0b4de0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54fa59c640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec041020 con 0x7f54fc0b4de0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec03d620 con 0x7f54fc0b4de0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc0b0a20 con 0x7f54fc0b4de0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec037040 con 0x7f54fc0b4de0 2024-05-09T06:28:23.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.297+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc13fa30 con 0x7f54fc0b4de0 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.298+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54fc13fc30 con 0x7f54fc0b4de0 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.299+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54ec044020 con 0x7f54fc0b4de0 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.299+0000 7f54eaffd640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 0x7f54dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.299+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f54ec0ab920 con 0x7f54fc0b4de0 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.299+0000 7f54f9d9b640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 0x7f54dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.300+0000 7f54f9d9b640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 0x7f54dc06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009560 tx=0x7f54f0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.304+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54fc13fc30 con 0x7f54fc0b4de0 2024-05-09T06:28:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.489+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54fc0752b0 con 0x7f54fc0b4de0 2024-05-09T06:28:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.490+0000 7f54eaffd640 1 -- 172.21.15.1:0/2699798106 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f54ec04b030 con 0x7f54fc0b4de0 2024-05-09T06:28:23.472 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 msgr2=0x7f54dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 0x7f54dc06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009560 tx=0x7f54f0009190 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 msgr2=0x7f54fc0b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec0046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 shutdown_connections 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54dc06be30 0x7f54dc06e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54fc15c4c0 0x7f54fc13d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54fc0b5260 0x7f54fc1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 --2- 172.21.15.1:0/2699798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54fc0b4de0 0x7f54fc0b00a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 >> 172.21.15.1:0/2699798106 conn(0x7f54fc0cc4d0 msgr2=0x7f54fc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 shutdown_connections 2024-05-09T06:28:23.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.492+0000 7f5500b61640 1 -- 172.21.15.1:0/2699798106 wait complete. 2024-05-09T06:28:23.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:23.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.610+0000 7f0cf295a640 1 Processor -- start 2024-05-09T06:28:23.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.610+0000 7f0cf295a640 1 -- start start 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf295a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf295a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf295a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec06cf40 con 0x7f0cec15c4c0 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf295a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec156980 con 0x7f0cec15cde0 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf295a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec06e180 con 0x7f0cec06dd00 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf0ef7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf0ef7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:33818/0 (socket says 172.21.15.1:33818) 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cebfff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f0cec15c4c0 0x7f0cec0b4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:52950/0 (socket says 172.21.15.1:52950) 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0cf0ef7640 1 -- 172.21.15.1:0/1256059925 learned_addr learned my addr 172.21.15.1:0/1256059925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0ceaffd640 1 -- 172.21.15.1:0/1256059925 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182865375 0 0) 0x7f0cec06cf40 con 0x7f0cec15c4c0 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0ceaffd640 1 -- 172.21.15.1:0/1256059925 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd4003520 con 0x7f0cec15c4c0 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.611+0000 7f0ceb7fe640 1 -- 172.21.15.1:0/1256059925 >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 msgr2=0x7f0cec070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.612+0000 7f0ceb7fe640 1 --2- 172.21.15.1:0/1256059925 >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.612+0000 7f0ceb7fe640 1 -- 172.21.15.1:0/1256059925 >> v1:172.21.15.6:6789/0 conn(0x7f0cec15c4c0 legacy=0x7f0cec0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.612+0000 7f0ceb7fe640 1 -- 172.21.15.1:0/1256059925 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec06d220 con 0x7f0cec06dd00 2024-05-09T06:28:23.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.612+0000 7f0cf0ef7640 1 --2- 172.21.15.1:0/1256059925 >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.612+0000 7f0ceb7fe640 1 --2- 172.21.15.1:0/1256059925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec15c8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd4fa99b1cf58df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0ceaffd640 1 -- 172.21.15.1:0/1256059925 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0037070 con 0x7f0cec06dd00 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0ceaffd640 1 -- 172.21.15.1:0/1256059925 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0032440 con 0x7f0cec06dd00 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0ceaffd640 1 -- 172.21.15.1:0/1256059925 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0032810 con 0x7f0cec06dd00 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 -- 172.21.15.1:0/1256059925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 msgr2=0x7f0cec15c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 --2- 172.21.15.1:0/1256059925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec15c8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00097a0 tx=0x7f0ce002a180 comp rx=0 tx=0).stop 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 -- 172.21.15.1:0/1256059925 shutdown_connections 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 --2- 172.21.15.1:0/1256059925 >> v2:172.21.15.6:3300/0 conn(0x7f0cec15cde0 0x7f0cec070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 --2- 172.21.15.1:0/1256059925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec15c8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 -- 172.21.15.1:0/1256059925 >> 172.21.15.1:0/1256059925 conn(0x7f0cec0cc4d0 msgr2=0x7f0cec0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 -- 172.21.15.1:0/1256059925 shutdown_connections 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.613+0000 7f0cf295a640 1 -- 172.21.15.1:0/1256059925 wait complete. 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.614+0000 7f0cf295a640 1 Processor -- start 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.614+0000 7f0cf295a640 1 -- start start 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cec15c4c0 0x7f0cec0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec15d270 con 0x7f0cec15cde0 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec06d870 con 0x7f0cec15c4c0 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf295a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0b54f0 con 0x7f0cec06dd00 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf0ef7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf0ef7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51272/0 (socket says 172.21.15.1:51272) 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cf0ef7640 1 -- 172.21.15.1:0/590295818 learned_addr learned my addr 172.21.15.1:0/590295818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0ceb7fe640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cec15c4c0 0x7f0cec0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.615+0000 7f0cebfff640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 -- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 msgr2=0x7f0cec1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 -- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cec15c4c0 msgr2=0x7f0cec0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cec15c4c0 0x7f0cec0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 -- 172.21.15.1:0/590295818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec0addd0 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf0ef7640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400efc0 tx=0x7f0ce400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400cbe0 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce400cd80 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4003bb0 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec0ae0d0 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.616+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec0ae910 con 0x7f0cec15cde0 2024-05-09T06:28:23.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.617+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec0752b0 con 0x7f0cec15cde0 2024-05-09T06:28:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.618+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ce401b030 con 0x7f0cec15cde0 2024-05-09T06:28:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.618+0000 7f0ce8ff9640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.618+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0ce408b470 con 0x7f0cec15cde0 2024-05-09T06:28:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.618+0000 7f0cebfff640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.619+0000 7f0cebfff640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 0x7f0cd806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000f60 tx=0x7f0cdc007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.623+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ce4022020 con 0x7f0cec15cde0 2024-05-09T06:28:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.805+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cec15ec80 con 0x7f0cec15cde0 2024-05-09T06:28:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.805+0000 7f0ce8ff9640 1 -- 172.21.15.1:0/590295818 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0cec0ae0d0 con 0x7f0cec15cde0 2024-05-09T06:28:23.788 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 msgr2=0x7f0cd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 0x7f0cd806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000f60 tx=0x7f0cdc007cf0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 msgr2=0x7f0cec15ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400efc0 tx=0x7f0ce400c490 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 shutdown_connections 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cec15cde0 0x7f0cec15ab00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cec15c4c0 0x7f0cec0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 --2- 172.21.15.1:0/590295818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cec06dd00 0x7f0cec1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.807+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 >> 172.21.15.1:0/590295818 conn(0x7f0cec0cc4d0 msgr2=0x7f0cec0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.808+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 shutdown_connections 2024-05-09T06:28:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.808+0000 7f0cf295a640 1 -- 172.21.15.1:0/590295818 wait complete. 2024-05-09T06:28:23.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:23.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.925+0000 7f0fb806b640 1 Processor -- start 2024-05-09T06:28:23.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.925+0000 7f0fb806b640 1 -- start start 2024-05-09T06:28:23.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0fb015c4c0 0x7f0fb006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0fb00b4de0 0x7f0fb015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b5260 0x7f0fb015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb0156980 con 0x7f0fb00b4de0 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb006d870 con 0x7f0fb015c4c0 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb806b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb015d320 con 0x7f0fb00b5260 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0fb00b4de0 0x7f0fb015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0fb00b4de0 0x7f0fb015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51280/0 (socket says 172.21.15.1:51280) 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 -- 172.21.15.1:0/14623317 learned_addr learned my addr 172.21.15.1:0/14623317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb6608640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b5260 0x7f0fb015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5e07640 1 --2- 172.21.15.1:0/14623317 >> v2:172.21.15.6:3300/0 conn(0x7f0fb015c4c0 0x7f0fb006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 -- 172.21.15.1:0/14623317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b5260 msgr2=0x7f0fb015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b5260 0x7f0fb015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 -- 172.21.15.1:0/14623317 >> v2:172.21.15.6:3300/0 conn(0x7f0fb015c4c0 msgr2=0x7f0fb006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 --2- 172.21.15.1:0/14623317 >> v2:172.21.15.6:3300/0 conn(0x7f0fb015c4c0 0x7f0fb006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.926+0000 7f0fb5606640 1 -- 172.21.15.1:0/14623317 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb006cf40 con 0x7f0fb00b4de0 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.927+0000 7f0fb5606640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b4de0 0x7f0fb015c8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0fb004fb80 tx=0x7f0fa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a19e5652be636c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.927+0000 7f0fb4e05640 1 -- 172.21.15.1:0/14623317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0037070 con 0x7f0fb00b4de0 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.927+0000 7f0fb4e05640 1 -- 172.21.15.1:0/14623317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa0032440 con 0x7f0fb00b4de0 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.927+0000 7f0fb4e05640 1 -- 172.21.15.1:0/14623317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0032810 con 0x7f0fb00b4de0 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 -- 172.21.15.1:0/14623317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b4de0 msgr2=0x7f0fb015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b4de0 0x7f0fb015c8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0fb004fb80 tx=0x7f0fa002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 -- 172.21.15.1:0/14623317 shutdown_connections 2024-05-09T06:28:23.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b5260 0x7f0fb015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 --2- 172.21.15.1:0/14623317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b4de0 0x7f0fb015c8a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 --2- 172.21.15.1:0/14623317 >> v2:172.21.15.6:3300/0 conn(0x7f0fb015c4c0 0x7f0fb006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 -- 172.21.15.1:0/14623317 >> 172.21.15.1:0/14623317 conn(0x7f0fb00cc4d0 msgr2=0x7f0fb00cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 -- 172.21.15.1:0/14623317 shutdown_connections 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.928+0000 7f0fb806b640 1 -- 172.21.15.1:0/14623317 wait complete. 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.929+0000 7f0fb806b640 1 Processor -- start 2024-05-09T06:28:23.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.929+0000 7f0fb806b640 1 -- start start 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 0x7f0fb00aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb015c4c0 0x7f0fb0141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb00b2c40 con 0x7f0fb00b5260 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb006d500 con 0x7f0fb015c4c0 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb806b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb00b2dc0 con 0x7f0fb00b4de0 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51292/0 (socket says 172.21.15.1:51292) 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 -- 172.21.15.1:0/757840994 learned_addr learned my addr 172.21.15.1:0/757840994 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5e07640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 0x7f0fb00aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb6608640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb015c4c0 0x7f0fb0141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 -- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 msgr2=0x7f0fb00aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 0x7f0fb00aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 -- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb015c4c0 msgr2=0x7f0fb0141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb015c4c0 0x7f0fb0141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 -- 172.21.15.1:0/757840994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb015b370 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5e07640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 0x7f0fb00aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.930+0000 7f0fb5606640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fa002a220 tx=0x7f0fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.931+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0041020 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.931+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa00306c0 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.931+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb015b650 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.931+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0037040 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.931+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb0157b60 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.932+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb00752b0 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.932+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fa0044020 con 0x7f0fb00b5260 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.932+0000 7f0fa67fc640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 0x7f0f9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.933+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0fa00ab8e0 con 0x7f0fb00b5260 2024-05-09T06:28:23.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.933+0000 7f0fb5e07640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 0x7f0f9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.933+0000 7f0fb5e07640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 0x7f0f9406e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006fd0 tx=0x7f0fac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:23.937+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fa004b030 con 0x7f0fb00b5260 2024-05-09T06:28:24.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.118+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0fb0142360 con 0x7f0fb00b5260 2024-05-09T06:28:24.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.119+0000 7f0fa67fc640 1 -- 172.21.15.1:0/757840994 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0fa0079f20 con 0x7f0fb00b5260 2024-05-09T06:28:24.101 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 msgr2=0x7f0f9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 0x7f0f9406e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006fd0 tx=0x7f0fac008040 comp rx=0 tx=0).stop 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 msgr2=0x7f0fb015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fa002a220 tx=0x7f0fa00046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 shutdown_connections 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f9406be60 0x7f0f9406e320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fb015c4c0 0x7f0fb0141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb00b5260 0x7f0fb015ac20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 --2- 172.21.15.1:0/757840994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fb00b4de0 0x7f0fb00aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 >> 172.21.15.1:0/757840994 conn(0x7f0fb00cc4d0 msgr2=0x7f0fb015eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 shutdown_connections 2024-05-09T06:28:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.121+0000 7f0fb806b640 1 -- 172.21.15.1:0/757840994 wait complete. 2024-05-09T06:28:24.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.239+0000 7f93ef229640 1 Processor -- start 2024-05-09T06:28:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.239+0000 7f93ef229640 1 -- start start 2024-05-09T06:28:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ef229640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f93e80c41e0 0x7f93e80ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ef229640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f93e806b1e0 0x7f93e80cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ef229640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8072050 con 0x7f93e80d06f0 2024-05-09T06:28:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ef229640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f93e8072330 con 0x7f93e80c41e0 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ef229640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f93e8072610 con 0x7f93e806b1e0 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f93e80c41e0 0x7f93e80ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ecfc5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f93e80d06f0 0x7f93e80cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42484/0 (socket says 172.21.15.1:42484) 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ecfc5640 1 -- 172.21.15.1:0/1302834445 learned_addr learned my addr 172.21.15.1:0/1302834445 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93ed7c6640 1 --2- 172.21.15.1:0/1302834445 >> v2:172.21.15.6:3300/0 conn(0x7f93e806b1e0 0x7f93e80cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e7fff640 1 -- 172.21.15.1:0/1302834445 >> v2:172.21.15.6:3300/0 conn(0x7f93e806b1e0 msgr2=0x7f93e80cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e7fff640 1 --2- 172.21.15.1:0/1302834445 >> v2:172.21.15.6:3300/0 conn(0x7f93e806b1e0 0x7f93e80cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e7fff640 1 -- 172.21.15.1:0/1302834445 >> v1:172.21.15.1:6789/0 conn(0x7f93e80d06f0 legacy=0x7f93e80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e7fff640 1 -- 172.21.15.1:0/1302834445 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e80728f0 con 0x7f93e80c41e0 2024-05-09T06:28:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.240+0000 7f93e77fe640 1 -- 172.21.15.1:0/1302834445 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134880874 0 0) 0x7f93e8072050 con 0x7f93e80d06f0 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.241+0000 7f93e7fff640 1 --2- 172.21.15.1:0/1302834445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80c41e0 0x7f93e80ce8d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3780 tx=0x7f93d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6d7b9b3be8ccf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.241+0000 7f93e77fe640 1 -- 172.21.15.1:0/1302834445 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037070 con 0x7f93e80c41e0 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.241+0000 7f93e77fe640 1 -- 172.21.15.1:0/1302834445 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4032440 con 0x7f93e80c41e0 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.241+0000 7f93e77fe640 1 -- 172.21.15.1:0/1302834445 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4032810 con 0x7f93e80c41e0 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 -- 172.21.15.1:0/1302834445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80c41e0 msgr2=0x7f93e80ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 --2- 172.21.15.1:0/1302834445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80c41e0 0x7f93e80ce8d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3780 tx=0x7f93d402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 -- 172.21.15.1:0/1302834445 shutdown_connections 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 --2- 172.21.15.1:0/1302834445 >> v2:172.21.15.6:3300/0 conn(0x7f93e806b1e0 0x7f93e80cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 --2- 172.21.15.1:0/1302834445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80c41e0 0x7f93e80ce8d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 -- 172.21.15.1:0/1302834445 >> 172.21.15.1:0/1302834445 conn(0x7f93e80d0b50 msgr2=0x7f93e80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 -- 172.21.15.1:0/1302834445 shutdown_connections 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.242+0000 7f93ef229640 1 -- 172.21.15.1:0/1302834445 wait complete. 2024-05-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.243+0000 7f93ef229640 1 Processor -- start 2024-05-09T06:28:24.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.243+0000 7f93ef229640 1 -- start start 2024-05-09T06:28:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e806b1e0 0x7f93e81f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93e80c41e0 0x7f93e809f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80ccf70 con 0x7f93e80d06f0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80ccdf0 con 0x7f93e806b1e0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ef229640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f93e80cd0f0 con 0x7f93e80c41e0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51322/0 (socket says 172.21.15.1:51322) 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 -- 172.21.15.1:0/2249232212 learned_addr learned my addr 172.21.15.1:0/2249232212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ecfc5640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e806b1e0 0x7f93e81f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93e7fff640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93e80c41e0 0x7f93e809f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 -- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93e80c41e0 msgr2=0x7f93e809f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93e80c41e0 0x7f93e809f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 -- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e806b1e0 msgr2=0x7f93e81f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e806b1e0 0x7f93e81f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 -- 172.21.15.1:0/2249232212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81f4b00 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.244+0000 7f93ed7c6640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f93e000cd70 tx=0x7f93e000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.245+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003b90 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.245+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0003d30 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.245+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81f3d40 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.245+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004240 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.245+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81f4300 con 0x7f93e80d06f0 2024-05-09T06:28:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.246+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e80c7790 con 0x7f93e80d06f0 2024-05-09T06:28:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.246+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93e00043e0 con 0x7f93e80d06f0 2024-05-09T06:28:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.247+0000 7f93e57fa640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 0x7f93cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.247+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f93e008bcd0 con 0x7f93e80d06f0 2024-05-09T06:28:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.247+0000 7f93ecfc5640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 0x7f93cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.247+0000 7f93ecfc5640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 0x7f93cc06e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93dc000f60 tx=0x7f93dc007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.251+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93e80c7790 con 0x7f93e80d06f0 2024-05-09T06:28:24.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.433+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f93e80752b0 con 0x7f93e80d06f0 2024-05-09T06:28:24.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.433+0000 7f93e57fa640 1 -- 172.21.15.1:0/2249232212 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f93e005a310 con 0x7f93e80d06f0 2024-05-09T06:28:24.415 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:24.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 msgr2=0x7f93cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 0x7f93cc06e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93dc000f60 tx=0x7f93dc007c00 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 msgr2=0x7f93e809faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f93e000cd70 tx=0x7f93e000a660 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 shutdown_connections 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f93cc06be30 0x7f93cc06e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e80d06f0 0x7f93e809faf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f93e80c41e0 0x7f93e809f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 --2- 172.21.15.1:0/2249232212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e806b1e0 0x7f93e81f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 >> 172.21.15.1:0/2249232212 conn(0x7f93e80d0b50 msgr2=0x7f93e806ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 shutdown_connections 2024-05-09T06:28:24.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.435+0000 7f93ef229640 1 -- 172.21.15.1:0/2249232212 wait complete. 2024-05-09T06:28:24.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:24.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.551+0000 7f6e2bdf9640 1 Processor -- start 2024-05-09T06:28:24.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.551+0000 7f6e2bdf9640 1 -- start start 2024-05-09T06:28:24.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2bdf9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e240b2a60 0x7f6e2415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2bdf9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 0x7f6e2406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2bdf9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e24070f40 con 0x7f6e2415db10 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2bdf9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2415e950 con 0x7f6e240b2a60 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2bdf9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6e240cf170 con 0x7f6e2415e500 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29394640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e240b2a60 0x7f6e2415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29b95640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6e2415db10 0x7f6e240715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42498/0 (socket says 172.21.15.1:42498) 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29b95640 1 -- 172.21.15.1:0/3091304575 learned_addr learned my addr 172.21.15.1:0/3091304575 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2a396640 1 --2- 172.21.15.1:0/3091304575 >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 0x7f6e2406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29394640 1 -- 172.21.15.1:0/3091304575 >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 msgr2=0x7f6e2406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29394640 1 --2- 172.21.15.1:0/3091304575 >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 0x7f6e2406cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29394640 1 -- 172.21.15.1:0/3091304575 >> v1:172.21.15.1:6789/0 conn(0x7f6e2415db10 legacy=0x7f6e240715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e29394640 1 -- 172.21.15.1:0/3091304575 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240cf450 con 0x7f6e240b2a60 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e2a396640 1 --2- 172.21.15.1:0/3091304575 >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 0x7f6e2406cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:24.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.552+0000 7f6e28b93640 1 -- 172.21.15.1:0/3091304575 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92521950 0 0) 0x7f6e24070f40 con 0x7f6e2415db10 2024-05-09T06:28:24.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.553+0000 7f6e29394640 1 --2- 172.21.15.1:0/3091304575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e240b2a60 0x7f6e2415dfc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e240b2770 tx=0x7f6e1802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8d773b04eb0a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.553+0000 7f6e28b93640 1 -- 172.21.15.1:0/3091304575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18037070 con 0x7f6e240b2a60 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.553+0000 7f6e28b93640 1 -- 172.21.15.1:0/3091304575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18032440 con 0x7f6e240b2a60 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.553+0000 7f6e28b93640 1 -- 172.21.15.1:0/3091304575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1802ae60 con 0x7f6e240b2a60 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/3091304575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e240b2a60 msgr2=0x7f6e2415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/3091304575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e240b2a60 0x7f6e2415dfc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e240b2770 tx=0x7f6e1802a110 comp rx=0 tx=0).stop 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/3091304575 shutdown_connections 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/3091304575 >> v2:172.21.15.6:3300/0 conn(0x7f6e2415e500 0x7f6e2406cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/3091304575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e240b2a60 0x7f6e2415dfc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/3091304575 >> 172.21.15.1:0/3091304575 conn(0x7f6e240cc4b0 msgr2=0x7f6e240cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/3091304575 shutdown_connections 2024-05-09T06:28:24.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.554+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/3091304575 wait complete. 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.555+0000 7f6e2bdf9640 1 Processor -- start 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.555+0000 7f6e2bdf9640 1 -- start start 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e240b2a60 0x7f6e24153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 0x7f6e241f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2406d3f0 con 0x7f6e2415db10 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2406d270 con 0x7f6e240b2a60 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2bdf9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2415ec40 con 0x7f6e2415e500 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51338/0 (socket says 172.21.15.1:51338) 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 -- 172.21.15.1:0/657495295 learned_addr learned my addr 172.21.15.1:0/657495295 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29b95640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e240b2a60 0x7f6e24153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2a396640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 0x7f6e241f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 -- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 msgr2=0x7f6e241f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 0x7f6e241f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 -- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e240b2a60 msgr2=0x7f6e24153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e240b2a60 0x7f6e24153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e29394640 1 -- 172.21.15.1:0/657495295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e24154340 con 0x7f6e2415db10 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.556+0000 7f6e2a396640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 0x7f6e241f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e29394640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18037040 con 0x7f6e2415db10 2024-05-09T06:28:24.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18042070 con 0x7f6e2415db10 2024-05-09T06:28:24.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241590e0 con 0x7f6e2415db10 2024-05-09T06:28:24.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18030a50 con 0x7f6e2415db10 2024-05-09T06:28:24.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.557+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e24153600 con 0x7f6e2415db10 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.558+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e24158c90 con 0x7f6e2415db10 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.558+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e18030bf0 con 0x7f6e2415db10 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.559+0000 7f6e227fc640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 0x7f6e1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.559+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e180abce0 con 0x7f6e2415db10 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.559+0000 7f6e29b95640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 0x7f6e1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.560+0000 7f6e29b95640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 0x7f6e1006e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.564+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e24158c90 con 0x7f6e2415db10 2024-05-09T06:28:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.761+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e24159600 con 0x7f6e2415db10 2024-05-09T06:28:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.761+0000 7f6e227fc640 1 -- 172.21.15.1:0/657495295 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f6e18045360 con 0x7f6e2415db10 2024-05-09T06:28:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 msgr2=0x7f6e1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 0x7f6e1006e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c007b50 comp rx=0 tx=0).stop 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 msgr2=0x7f6e24151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e180046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 shutdown_connections 2024-05-09T06:28:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e1006be30 0x7f6e1006e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e2415e500 0x7f6e241f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e2415db10 0x7f6e24151d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 --2- 172.21.15.1:0/657495295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e240b2a60 0x7f6e24153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 >> 172.21.15.1:0/657495295 conn(0x7f6e240cc4b0 msgr2=0x7f6e2406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 shutdown_connections 2024-05-09T06:28:24.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:24.763+0000 7f6e2bdf9640 1 -- 172.21.15.1:0/657495295 wait complete. 2024-05-09T06:28:24.756 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:28:24.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:24.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-09T06:28:24.782 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:24.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:28:24.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:24.783 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:28:24.839 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:24.839 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:24.859 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:24.859 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:24.919 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:25.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.042+0000 7fccfa6e7640 1 Processor -- start 2024-05-09T06:28:25.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.042+0000 7fccfa6e7640 1 -- start start 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccfa6e7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fccf41454b0 0x7fccf40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccfa6e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 0x7fccf40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccfa6e7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf40b5230 con 0x7fccf40b4da0 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccfa6e7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fccf4071c20 con 0x7fccf41454b0 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccfa6e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fccf406eb40 con 0x7fccf40b2a60 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccf3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fccf41454b0 0x7fccf40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccf3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fccf41454b0 0x7fccf40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51344/0 (socket says 172.21.15.1:51344) 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.043+0000 7fccf3fff640 1 -- 172.21.15.1:0/3457246151 learned_addr learned my addr 172.21.15.1:0/3457246151 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccebfff640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 0x7fccf40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf3fff640 1 -- 172.21.15.1:0/3457246151 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 msgr2=0x7fccf40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf3fff640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 0x7fccf40b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf3fff640 1 -- 172.21.15.1:0/3457246151 >> v1:172.21.15.6:6789/0 conn(0x7fccf40b4da0 legacy=0x7fccf415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf3fff640 1 -- 172.21.15.1:0/3457246151 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf406ee20 con 0x7fccf41454b0 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccebfff640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 0x7fccf40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:25.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf3fff640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf40b4320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009870 tx=0x7fcce402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d660ab1d6f8df52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.044+0000 7fccf37fe640 1 -- 172.21.15.1:0/3457246151 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4037070 con 0x7fccf41454b0 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccf37fe640 1 -- 172.21.15.1:0/3457246151 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce4032440 con 0x7fccf41454b0 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccf37fe640 1 -- 172.21.15.1:0/3457246151 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4032810 con 0x7fccf41454b0 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 -- 172.21.15.1:0/3457246151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 msgr2=0x7fccf40b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf40b4320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009870 tx=0x7fcce402a200 comp rx=0 tx=0).stop 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 -- 172.21.15.1:0/3457246151 shutdown_connections 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b2a60 0x7fccf40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/3457246151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf40b4320 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 -- 172.21.15.1:0/3457246151 >> 172.21.15.1:0/3457246151 conn(0x7fccf40cc4b0 msgr2=0x7fccf40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 -- 172.21.15.1:0/3457246151 shutdown_connections 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.045+0000 7fccfa6e7640 1 -- 172.21.15.1:0/3457246151 wait complete. 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.046+0000 7fccfa6e7640 1 Processor -- start 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.046+0000 7fccfa6e7640 1 -- start start 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf40b2a60 0x7fccf413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 0x7fccf41e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4145890 con 0x7fccf41454b0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fccf406dbc0 con 0x7fccf40b2a60 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccfa6e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fccf406e0e0 con 0x7fccf40b4da0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccf8c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccf8c84640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51348/0 (socket says 172.21.15.1:51348) 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccf8c84640 1 -- 172.21.15.1:0/2744141102 learned_addr learned my addr 172.21.15.1:0/2744141102 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccf3fff640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf40b2a60 0x7fccf413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.047+0000 7fccebfff640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 0x7fccf41e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 -- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 msgr2=0x7fccf41e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 0x7fccf41e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 -- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf40b2a60 msgr2=0x7fccf413c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf40b2a60 0x7fccf413c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 -- 172.21.15.1:0/2744141102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf413f090 con 0x7fccf41454b0 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccebfff640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 0x7fccf41e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf8c84640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c780 tx=0x7fccec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec004b60 con 0x7fccf41454b0 2024-05-09T06:28:25.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec004d00 con 0x7fccf41454b0 2024-05-09T06:28:25.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf413f330 con 0x7fccf41454b0 2024-05-09T06:28:25.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec00aa90 con 0x7fccf41454b0 2024-05-09T06:28:25.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.048+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf413f8f0 con 0x7fccf41454b0 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.049+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf40752b0 con 0x7fccf41454b0 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.050+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccec00ac30 con 0x7fccf41454b0 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.050+0000 7fccf17fa640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 0x7fccd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.050+0000 7fccf3fff640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 0x7fccd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.050+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fccec08b420 con 0x7fccf41454b0 2024-05-09T06:28:25.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.051+0000 7fccf3fff640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 0x7fccd406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcce4034000 tx=0x7fcce402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.055+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fccec020070 con 0x7fccf41454b0 2024-05-09T06:28:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.241+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccf4072fa0 con 0x7fccf41454b0 2024-05-09T06:28:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.242+0000 7fccf17fa640 1 -- 172.21.15.1:0/2744141102 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7fccec0599d0 con 0x7fccf41454b0 2024-05-09T06:28:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:25.225 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 msgr2=0x7fccd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 0x7fccd406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcce4034000 tx=0x7fcce402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 msgr2=0x7fccf41f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c780 tx=0x7fccec00cc40 comp rx=0 tx=0).stop 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 shutdown_connections 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fccd406be30 0x7fccd406e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccf41454b0 0x7fccf41f1c00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fccf40b4da0 0x7fccf41e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 --2- 172.21.15.1:0/2744141102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fccf40b2a60 0x7fccf413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.244+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 >> 172.21.15.1:0/2744141102 conn(0x7fccf40cc4b0 msgr2=0x7fccf40b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.245+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 shutdown_connections 2024-05-09T06:28:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.245+0000 7fccfa6e7640 1 -- 172.21.15.1:0/2744141102 wait complete. 2024-05-09T06:28:25.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:25.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:25.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 Processor -- start 2024-05-09T06:28:25.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 -- start start 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c7c0d1c70 0x7f6c7c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c7c161fe0 con 0x7f6c7c15e190 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6c7c15f580 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.365+0000 7f6c820b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c1622c0 con 0x7f6c7c0d45f0 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c7c0d1c70 0x7f6c7c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6c7c15e190 0x7f6c7c0d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42508/0 (socket says 172.21.15.1:42508) 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7b7fe640 1 -- 172.21.15.1:0/350565367 learned_addr learned my addr 172.21.15.1:0/350565367 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7bfff640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7affd640 1 -- 172.21.15.1:0/350565367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 msgr2=0x7f6c7c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7affd640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7affd640 1 -- 172.21.15.1:0/350565367 >> v1:172.21.15.1:6789/0 conn(0x7f6c7c15e190 legacy=0x7f6c7c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7affd640 1 -- 172.21.15.1:0/350565367 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1625a0 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.366+0000 7f6c7a7fc640 1 -- 172.21.15.1:0/350565367 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913879652 0 0) 0x7f6c7c161fe0 con 0x7f6c7c15e190 2024-05-09T06:28:25.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c7affd640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c15f780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097a0 tx=0x7f6c7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd47c8e0f2c7330 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c7a7fc640 1 -- 172.21.15.1:0/350565367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70037070 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c7a7fc640 1 -- 172.21.15.1:0/350565367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c70032440 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c7a7fc640 1 -- 172.21.15.1:0/350565367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70032810 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c820b3640 1 -- 172.21.15.1:0/350565367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c0d1c70 msgr2=0x7f6c7c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c820b3640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c15f780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097a0 tx=0x7f6c7002a180 comp rx=0 tx=0).stop 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c820b3640 1 -- 172.21.15.1:0/350565367 shutdown_connections 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.367+0000 7f6c820b3640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.368+0000 7f6c820b3640 1 --2- 172.21.15.1:0/350565367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c15f780 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.368+0000 7f6c820b3640 1 -- 172.21.15.1:0/350565367 >> 172.21.15.1:0/350565367 conn(0x7f6c7c073340 msgr2=0x7f6c7c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.368+0000 7f6c820b3640 1 -- 172.21.15.1:0/350565367 shutdown_connections 2024-05-09T06:28:25.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.368+0000 7f6c820b3640 1 -- 172.21.15.1:0/350565367 wait complete. 2024-05-09T06:28:25.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.369+0000 7f6c820b3640 1 Processor -- start 2024-05-09T06:28:25.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.369+0000 7f6c820b3640 1 -- start start 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c0b56d0 con 0x7f6c7c15e190 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c161c60 con 0x7f6c7c0d1c70 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c820b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c0b5850 con 0x7f6c7c0d45f0 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51372/0 (socket says 172.21.15.1:51372) 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 -- 172.21.15.1:0/2753089437 learned_addr learned my addr 172.21.15.1:0/2753089437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 -- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 msgr2=0x7f6c7c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7affd640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7b7fe640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 -- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c7c0d1c70 msgr2=0x7f6c7c0c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c0c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 -- 172.21.15.1:0/2753089437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1f7280 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c7bfff640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009e30 tx=0x7f6c7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74018020 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.370+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c74014480 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.371+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1f7560 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.371+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74014870 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.371+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0ccba0 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.372+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c0752b0 con 0x7f6c7c15e190 2024-05-09T06:28:25.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.372+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c74005d10 con 0x7f6c7c15e190 2024-05-09T06:28:25.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.372+0000 7f6c63fff640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 0x7f6c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.373+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6c7408c1c0 con 0x7f6c7c15e190 2024-05-09T06:28:25.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.373+0000 7f6c7b7fe640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 0x7f6c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.373+0000 7f6c7b7fe640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 0x7f6c6806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c000f60 tx=0x7f6c6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.377+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c74022030 con 0x7f6c7c15e190 2024-05-09T06:28:25.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.558+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c7c08e1c0 con 0x7f6c7c15e190 2024-05-09T06:28:25.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.558+0000 7f6c63fff640 1 -- 172.21.15.1:0/2753089437 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6c74010040 con 0x7f6c7c15e190 2024-05-09T06:28:25.540 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 msgr2=0x7f6c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 0x7f6c6806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c000f60 tx=0x7f6c6c009040 comp rx=0 tx=0).stop 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 msgr2=0x7f6c7c0b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009e30 tx=0x7f6c7400c490 comp rx=0 tx=0).stop 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 shutdown_connections 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c6806be30 0x7f6c6806e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c7c15e190 0x7f6c7c0b5b60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c7c0d45f0 0x7f6c7c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 --2- 172.21.15.1:0/2753089437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c7c0d1c70 0x7f6c7c0c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 >> 172.21.15.1:0/2753089437 conn(0x7f6c7c073340 msgr2=0x7f6c7c15f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 shutdown_connections 2024-05-09T06:28:25.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.560+0000 7f6c820b3640 1 -- 172.21.15.1:0/2753089437 wait complete. 2024-05-09T06:28:25.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:25.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.680+0000 7f28856da640 1 Processor -- start 2024-05-09T06:28:25.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.680+0000 7f28856da640 1 -- start start 2024-05-09T06:28:25.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f28856da640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f288015e190 0x7f288015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f28856da640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f28800d45f0 0x7f28800b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f28856da640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2880161fe0 con 0x7f28800d1c70 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f28856da640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f288015f580 con 0x7f28800d45f0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f28856da640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f28801622c0 con 0x7f288015e190 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f287f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f28800d45f0 0x7f28800b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f287e7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f28800d1c70 0x7f28800d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42522/0 (socket says 172.21.15.1:42522) 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.681+0000 7f287e7fc640 1 -- 172.21.15.1:0/718213167 learned_addr learned my addr 172.21.15.1:0/718213167 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287effd640 1 --2- 172.21.15.1:0/718213167 >> v2:172.21.15.6:3300/0 conn(0x7f288015e190 0x7f288015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287f7fe640 1 -- 172.21.15.1:0/718213167 >> v2:172.21.15.6:3300/0 conn(0x7f288015e190 msgr2=0x7f288015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287f7fe640 1 --2- 172.21.15.1:0/718213167 >> v2:172.21.15.6:3300/0 conn(0x7f288015e190 0x7f288015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287f7fe640 1 -- 172.21.15.1:0/718213167 >> v1:172.21.15.1:6789/0 conn(0x7f28800d1c70 legacy=0x7f28800d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287f7fe640 1 -- 172.21.15.1:0/718213167 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28801625a0 con 0x7f28800d45f0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.682+0000 7f287dffb640 1 -- 172.21.15.1:0/718213167 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189425631 0 0) 0x7f2880161fe0 con 0x7f28800d1c70 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f287f7fe640 1 --2- 172.21.15.1:0/718213167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d45f0 0x7f28800b3de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28800d1bf0 tx=0x7f287802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620a4a1b1ae6e99d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f287dffb640 1 -- 172.21.15.1:0/718213167 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878037070 con 0x7f28800d45f0 2024-05-09T06:28:25.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f287dffb640 1 -- 172.21.15.1:0/718213167 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2878032440 con 0x7f28800d45f0 2024-05-09T06:28:25.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f287dffb640 1 -- 172.21.15.1:0/718213167 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878032810 con 0x7f28800d45f0 2024-05-09T06:28:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 -- 172.21.15.1:0/718213167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d45f0 msgr2=0x7f28800b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 --2- 172.21.15.1:0/718213167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d45f0 0x7f28800b3de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28800d1bf0 tx=0x7f287802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 -- 172.21.15.1:0/718213167 shutdown_connections 2024-05-09T06:28:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 --2- 172.21.15.1:0/718213167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d45f0 0x7f28800b3de0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 --2- 172.21.15.1:0/718213167 >> v2:172.21.15.6:3300/0 conn(0x7f288015e190 0x7f288015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 -- 172.21.15.1:0/718213167 >> 172.21.15.1:0/718213167 conn(0x7f2880073340 msgr2=0x7f288009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 -- 172.21.15.1:0/718213167 shutdown_connections 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.683+0000 7f28856da640 1 -- 172.21.15.1:0/718213167 wait complete. 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 Processor -- start 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 -- start start 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28800d45f0 0x7f28800b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f288015e190 0x7f28800b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f28800b56d0 con 0x7f28800d1c70 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2880161c60 con 0x7f28800d45f0 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f28856da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f28800b5850 con 0x7f288015e190 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f287effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f287effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51394/0 (socket says 172.21.15.1:51394) 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.685+0000 7f287effd640 1 -- 172.21.15.1:0/309434500 learned_addr learned my addr 172.21.15.1:0/309434500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287e7fc640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28800d45f0 0x7f28800b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287f7fe640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f288015e190 0x7f28800b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 -- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f288015e190 msgr2=0x7f28800b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f288015e190 0x7f28800b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 -- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28800d45f0 msgr2=0x7f28800b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28800d45f0 0x7f28800b8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 -- 172.21.15.1:0/309434500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28800cc130 con 0x7f28800d1c70 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f287effd640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f287000efc0 tx=0x7f287000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870018020 con 0x7f28800d1c70 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2870014480 con 0x7f28800d1c70 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28800cc410 con 0x7f28800d1c70 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870014850 con 0x7f28800d1c70 2024-05-09T06:28:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.686+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28800cc9d0 con 0x7f28800d1c70 2024-05-09T06:28:25.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.687+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28800cce10 con 0x7f28800d1c70 2024-05-09T06:28:25.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.688+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2870005d10 con 0x7f28800d1c70 2024-05-09T06:28:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.688+0000 7f28677fe640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 0x7f286c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.688+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f287008bbd0 con 0x7f28800d1c70 2024-05-09T06:28:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.688+0000 7f287e7fc640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 0x7f286c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.689+0000 7f287e7fc640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 0x7f286c06e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28800a1fc0 tx=0x7f2874007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.693+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2870022070 con 0x7f28800d1c70 2024-05-09T06:28:25.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.877+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28800752b0 con 0x7f28800d1c70 2024-05-09T06:28:25.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.878+0000 7f28677fe640 1 -- 172.21.15.1:0/309434500 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f2870010040 con 0x7f28800d1c70 2024-05-09T06:28:25.860 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 msgr2=0x7f286c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 0x7f286c06e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28800a1fc0 tx=0x7f2874007d50 comp rx=0 tx=0).stop 2024-05-09T06:28:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 msgr2=0x7f28800ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f287000efc0 tx=0x7f287000c490 comp rx=0 tx=0).stop 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 shutdown_connections 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f286c06be30 0x7f286c06e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f288015e190 0x7f28800b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28800d45f0 0x7f28800b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 --2- 172.21.15.1:0/309434500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28800d1c70 0x7f28800ca0d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 >> 172.21.15.1:0/309434500 conn(0x7f2880073340 msgr2=0x7f288006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 shutdown_connections 2024-05-09T06:28:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.880+0000 7f28856da640 1 -- 172.21.15.1:0/309434500 wait complete. 2024-05-09T06:28:25.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.998+0000 7f305e4ca640 1 Processor -- start 2024-05-09T06:28:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 -- start start 2024-05-09T06:28:25.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f305806dd00 0x7f305815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f305815cde0 0x7f3058070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305806cf40 con 0x7f305815c4c0 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3058156980 con 0x7f305806dd00 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:25.999+0000 7f305e4ca640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f305806e180 con 0x7f305815cde0 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f305806dd00 0x7f305815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f305806dd00 0x7f305815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51406/0 (socket says 172.21.15.1:51406) 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 -- 172.21.15.1:0/1847400101 learned_addr learned my addr 172.21.15.1:0/1847400101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f305ca67640 1 --2- 172.21.15.1:0/1847400101 >> v2:172.21.15.6:3300/0 conn(0x7f305815cde0 0x7f3058070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 -- 172.21.15.1:0/1847400101 >> v2:172.21.15.6:3300/0 conn(0x7f305815cde0 msgr2=0x7f3058070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 --2- 172.21.15.1:0/1847400101 >> v2:172.21.15.6:3300/0 conn(0x7f305815cde0 0x7f3058070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 -- 172.21.15.1:0/1847400101 >> v1:172.21.15.6:6789/0 conn(0x7f305815c4c0 legacy=0x7f30580b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.000+0000 7f30577fe640 1 -- 172.21.15.1:0/1847400101 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305806d220 con 0x7f305806dd00 2024-05-09T06:28:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f30577fe640 1 --2- 172.21.15.1:0/1847400101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305806dd00 0x7f305815c8a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f305815c450 tx=0x7f304c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3674d89e08a032d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f3056ffd640 1 -- 172.21.15.1:0/1847400101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c037070 con 0x7f305806dd00 2024-05-09T06:28:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f3056ffd640 1 -- 172.21.15.1:0/1847400101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304c032440 con 0x7f305806dd00 2024-05-09T06:28:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f3056ffd640 1 -- 172.21.15.1:0/1847400101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c032810 con 0x7f305806dd00 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 -- 172.21.15.1:0/1847400101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305806dd00 msgr2=0x7f305815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 --2- 172.21.15.1:0/1847400101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305806dd00 0x7f305815c8a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f305815c450 tx=0x7f304c02a270 comp rx=0 tx=0).stop 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 -- 172.21.15.1:0/1847400101 shutdown_connections 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 --2- 172.21.15.1:0/1847400101 >> v2:172.21.15.6:3300/0 conn(0x7f305815cde0 0x7f3058070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 --2- 172.21.15.1:0/1847400101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305806dd00 0x7f305815c8a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 -- 172.21.15.1:0/1847400101 >> 172.21.15.1:0/1847400101 conn(0x7f30580cc4d0 msgr2=0x7f30580cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 -- 172.21.15.1:0/1847400101 shutdown_connections 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.001+0000 7f305e4ca640 1 -- 172.21.15.1:0/1847400101 wait complete. 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.003+0000 7f305e4ca640 1 Processor -- start 2024-05-09T06:28:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.003+0000 7f305e4ca640 1 -- start start 2024-05-09T06:28:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.003+0000 7f305e4ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 0x7f30580cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.003+0000 7f305e4ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 0x7f30580cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f305815d270 con 0x7f305815cde0 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f305806d870 con 0x7f305815c4c0 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f30580b54f0 con 0x7f305806dd00 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51410/0 (socket says 172.21.15.1:51410) 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 -- 172.21.15.1:0/488731603 learned_addr learned my addr 172.21.15.1:0/488731603 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f3057fff640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 0x7f30580cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f30577fe640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 0x7f30580cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 -- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 msgr2=0x7f30580cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 0x7f30580cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 -- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 msgr2=0x7f30580cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 0x7f30580cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 -- 172.21.15.1:0/488731603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305815b540 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f30577fe640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 0x7f30580cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f3057fff640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 0x7f30580cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305ca67640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f305000efc0 tx=0x7f305000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050018020 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050014480 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305815b880 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050014870 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.004+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305815c0c0 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.005+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30580752b0 con 0x7f305815cde0 2024-05-09T06:28:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.006+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3050005d10 con 0x7f305815cde0 2024-05-09T06:28:25.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.006+0000 7f3054ff9640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 0x7f304406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:25.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.007+0000 7f3057fff640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 0x7f304406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:25.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.007+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f305008bd50 con 0x7f305815cde0 2024-05-09T06:28:25.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.007+0000 7f3057fff640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 0x7f304406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3048001680 tx=0x7f3048009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:25.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.011+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3050022070 con 0x7f305815cde0 2024-05-09T06:28:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.195+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f305806e5e0 con 0x7f305815cde0 2024-05-09T06:28:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.195+0000 7f3054ff9640 1 -- 172.21.15.1:0/488731603 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f305815b880 con 0x7f305815cde0 2024-05-09T06:28:26.177 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:26.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 msgr2=0x7f304406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 0x7f304406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3048001680 tx=0x7f3048009040 comp rx=0 tx=0).stop 2024-05-09T06:28:26.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 msgr2=0x7f3058142070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f305000efc0 tx=0x7f305000c490 comp rx=0 tx=0).stop 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 shutdown_connections 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f304406be30 0x7f304406e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305815cde0 0x7f3058142070 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f305815c4c0 0x7f30580cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 --2- 172.21.15.1:0/488731603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f305806dd00 0x7f30580cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 >> 172.21.15.1:0/488731603 conn(0x7f30580cc4d0 msgr2=0x7f30580af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 shutdown_connections 2024-05-09T06:28:26.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.197+0000 7f305e4ca640 1 -- 172.21.15.1:0/488731603 wait complete. 2024-05-09T06:28:26.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:26.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 Processor -- start 2024-05-09T06:28:26.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 -- start start 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d7006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1d700b4de0 0x7f1d7015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7006cf40 con 0x7f1d700b5260 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1d70156980 con 0x7f1d700b4de0 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.315+0000 7f1d7598b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7006d870 con 0x7f1d7015c4c0 2024-05-09T06:28:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1d700b5260 0x7f1d70070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42528/0 (socket says 172.21.15.1:42528) 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/189874902 learned_addr learned my addr 172.21.15.1:0/189874902 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778611046 0 0) 0x7f1d7006cf40 con 0x7f1d700b5260 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d5c003520 con 0x7f1d700b5260 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/189874902 >> v2:172.21.15.6:3300/0 conn(0x7f1d700b4de0 0x7f1d7015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6effd640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d7006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3323337015 0 0) 0x7f1d5c003520 con 0x7f1d700b5260 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 msgr2=0x7f1d7006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d7006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 >> v2:172.21.15.6:3300/0 conn(0x7f1d700b4de0 msgr2=0x7f1d7015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 --2- 172.21.15.1:0/189874902 >> v2:172.21.15.6:3300/0 conn(0x7f1d700b4de0 0x7f1d7015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7006d220 con 0x7f1d700b5260 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f1d600019f0 con 0x7f1d700b5260 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/189874902 >> v2:172.21.15.6:3300/0 conn(0x7f1d700b4de0 0x7f1d7015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 >> v1:172.21.15.1:6789/0 conn(0x7f1d700b5260 legacy=0x7f1d70070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d5c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d5c003fe0 0x7f1d5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d5c004d40 0x7f1d5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7006d870 con 0x7f1d5c004d40 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c00ba70 con 0x7f1d5c003fe0 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c00bc40 con 0x7f1d7015c4c0 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d5c004d40 0x7f1d5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6effd640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d5c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.316+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d5c003fe0 0x7f1d5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 msgr2=0x7f1d5c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d7015c4c0 0x7f1d5c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d5c003fe0 msgr2=0x7f1d5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d5c003fe0 0x7f1d5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/189874902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c003520 con 0x7f1d5c004d40 2024-05-09T06:28:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.317+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d5c004d40 0x7f1d5c0051c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000a870 tx=0x7f1d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c7e6d882e9da06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000a4c0 con 0x7f1d5c004d40 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6002f070 con 0x7f1d5c004d40 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d6dffb640 1 -- 172.21.15.1:0/189874902 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6002bcc0 con 0x7f1d5c004d40 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 -- 172.21.15.1:0/189874902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d5c004d40 msgr2=0x7f1d5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 --2- 172.21.15.1:0/189874902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d5c004d40 0x7f1d5c0051c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000a870 tx=0x7f1d5c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/189874902 reap_dead start 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 -- 172.21.15.1:0/189874902 shutdown_connections 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 -- 172.21.15.1:0/189874902 >> 172.21.15.1:0/189874902 conn(0x7f1d700cc4d0 msgr2=0x7f1d700cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 -- 172.21.15.1:0/189874902 shutdown_connections 2024-05-09T06:28:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.318+0000 7f1d7598b640 1 -- 172.21.15.1:0/189874902 wait complete. 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.319+0000 7f1d7598b640 1 Processor -- start 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 -- start start 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 0x7f1d7015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 0x7f1d700ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700b2dc0 con 0x7f1d701f6640 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700b2c40 con 0x7f1d700cdbb0 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d7598b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7006dd00 con 0x7f1d5c004d40 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d6e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d6e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51430/0 (socket says 172.21.15.1:51430) 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.320+0000 7f1d6e7fc640 1 -- 172.21.15.1:0/3575074931 learned_addr learned my addr 172.21.15.1:0/3575074931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6effd640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 0x7f1d7015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 0x7f1d700ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 -- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 msgr2=0x7f1d7015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 0x7f1d7015eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 -- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 msgr2=0x7f1d700ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 0x7f1d700ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 -- 172.21.15.1:0/3575074931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700ce8e0 con 0x7f1d701f6640 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6effd640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 0x7f1d7015eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6f7fe640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 0x7f1d700ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d6e7fc640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1d58011730 tx=0x7f1d58011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58019070 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58004020 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d700cebc0 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800f040 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.321+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d700cf180 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.322+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d701f3c20 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.323+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d5800f970 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.323+0000 7f1d67fff640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 0x7f1d5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.323+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1d5808b690 con 0x7f1d701f6640 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.323+0000 7f1d6effd640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 0x7f1d5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.324+0000 7f1d6effd640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 0x7f1d5006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006fd0 tx=0x7f1d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.327+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d58059bb0 con 0x7f1d701f6640 2024-05-09T06:28:26.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.512+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d700752b0 con 0x7f1d701f6640 2024-05-09T06:28:26.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.512+0000 7f1d67fff640 1 -- 172.21.15.1:0/3575074931 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f1d580594b0 con 0x7f1d701f6640 2024-05-09T06:28:26.494 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 msgr2=0x7f1d5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 0x7f1d5006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006fd0 tx=0x7f1d68008040 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 msgr2=0x7f1d701f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1d58011730 tx=0x7f1d58011bf0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 shutdown_connections 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d5006be30 0x7f1d5006e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d700cdbb0 0x7f1d700ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d701f6640 0x7f1d701f6aa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 --2- 172.21.15.1:0/3575074931 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d5c004d40 0x7f1d7015eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.514+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 >> 172.21.15.1:0/3575074931 conn(0x7f1d700cc4d0 msgr2=0x7f1d700af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.515+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 shutdown_connections 2024-05-09T06:28:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.515+0000 7f1d7598b640 1 -- 172.21.15.1:0/3575074931 wait complete. 2024-05-09T06:28:26.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:26.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.630+0000 7fce23ea5640 1 Processor -- start 2024-05-09T06:28:26.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.631+0000 7fce23ea5640 1 -- start start 2024-05-09T06:28:26.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.631+0000 7fce23ea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c06e6d0 0x7fce1c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.631+0000 7fce23ea5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fce1c0cf180 0x7fce1c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.631+0000 7fce23ea5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fce1c0cdab0 0x7fce1c15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.631+0000 7fce23ea5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fce1c0ce0e0 con 0x7fce1c0cf180 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce23ea5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fce1c0ce3c0 con 0x7fce1c0cdab0 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce23ea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce1c0ce6a0 con 0x7fce1c06e6d0 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fce1c0cf180 0x7fce1c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fce1c0cf180 0x7fce1c0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51436/0 (socket says 172.21.15.1:51436) 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 -- 172.21.15.1:0/1752126208 learned_addr learned my addr 172.21.15.1:0/1752126208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce22442640 1 --2- 172.21.15.1:0/1752126208 >> v2:172.21.15.6:3300/0 conn(0x7fce1c0cdab0 0x7fce1c15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21c41640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c06e6d0 0x7fce1c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 -- 172.21.15.1:0/1752126208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c06e6d0 msgr2=0x7fce1c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c06e6d0 0x7fce1c06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 -- 172.21.15.1:0/1752126208 >> v2:172.21.15.6:3300/0 conn(0x7fce1c0cdab0 msgr2=0x7fce1c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 --2- 172.21.15.1:0/1752126208 >> v2:172.21.15.6:3300/0 conn(0x7fce1c0cdab0 0x7fce1c15e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.632+0000 7fce21440640 1 -- 172.21.15.1:0/1752126208 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c0ce980 con 0x7fce1c0cf180 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce21440640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0cd570 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0b16f0 tx=0x7fce0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76907ce8b5b11837 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce20c3f640 1 -- 172.21.15.1:0/1752126208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c037070 con 0x7fce1c0cf180 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce20c3f640 1 -- 172.21.15.1:0/1752126208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0c032440 con 0x7fce1c0cf180 2024-05-09T06:28:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce20c3f640 1 -- 172.21.15.1:0/1752126208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c032810 con 0x7fce1c0cf180 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 -- 172.21.15.1:0/1752126208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 msgr2=0x7fce1c0cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0cd570 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0b16f0 tx=0x7fce0c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 -- 172.21.15.1:0/1752126208 shutdown_connections 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 --2- 172.21.15.1:0/1752126208 >> v2:172.21.15.6:3300/0 conn(0x7fce1c0cdab0 0x7fce1c15e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0cd570 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 --2- 172.21.15.1:0/1752126208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c06e6d0 0x7fce1c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.633+0000 7fce23ea5640 1 -- 172.21.15.1:0/1752126208 >> 172.21.15.1:0/1752126208 conn(0x7fce1c0cc4b0 msgr2=0x7fce1c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.634+0000 7fce23ea5640 1 -- 172.21.15.1:0/1752126208 shutdown_connections 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.634+0000 7fce23ea5640 1 -- 172.21.15.1:0/1752126208 wait complete. 2024-05-09T06:28:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 Processor -- start 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 -- start start 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 0x7fce1c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 0x7fce1c1f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c070ed0 con 0x7fce1c0cf180 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c06eff0 con 0x7fce1c06e6d0 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.635+0000 7fce23ea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce1c1447c0 con 0x7fce1c0cdab0 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51448/0 (socket says 172.21.15.1:51448) 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 -- 172.21.15.1:0/949426643 learned_addr learned my addr 172.21.15.1:0/949426643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce21440640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 0x7fce1c1f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce21c41640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 0x7fce1c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 -- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 msgr2=0x7fce1c1f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 0x7fce1c1f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 -- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 msgr2=0x7fce1c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 0x7fce1c0a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 -- 172.21.15.1:0/949426643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c0a99c0 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce21c41640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 0x7fce1c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce21440640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 0x7fce1c1f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce22442640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce14007e40 tx=0x7fce1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14014620 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce14019070 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1c0a9c90 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14014a20 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.636+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1c0aa250 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.637+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1c0752b0 con 0x7fce1c0cf180 2024-05-09T06:28:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.638+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce14014bc0 con 0x7fce1c0cf180 2024-05-09T06:28:26.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.638+0000 7fce127fc640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 0x7fce0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.638+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fce1408bc10 con 0x7fce1c0cf180 2024-05-09T06:28:26.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.638+0000 7fce21c41640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 0x7fce0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.639+0000 7fce21c41640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 0x7fce0006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce18006fd0 tx=0x7fce18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.643+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce1405a120 con 0x7fce1c0cf180 2024-05-09T06:28:26.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.839+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce1c06cf50 con 0x7fce1c0cf180 2024-05-09T06:28:26.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.839+0000 7fce127fc640 1 -- 172.21.15.1:0/949426643 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fce14059a20 con 0x7fce1c0cf180 2024-05-09T06:28:26.821 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 msgr2=0x7fce0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 0x7fce0006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce18006fd0 tx=0x7fce18008040 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 msgr2=0x7fce1c0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce14007e40 tx=0x7fce1400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 shutdown_connections 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0006be30 0x7fce0006e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1c0cf180 0x7fce1c0b3fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce1c0cdab0 0x7fce1c1f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.841+0000 7fce23ea5640 1 --2- 172.21.15.1:0/949426643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce1c06e6d0 0x7fce1c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.842+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 >> 172.21.15.1:0/949426643 conn(0x7fce1c0cc4b0 msgr2=0x7fce1c0b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.842+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 shutdown_connections 2024-05-09T06:28:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.842+0000 7fce23ea5640 1 -- 172.21.15.1:0/949426643 wait complete. 2024-05-09T06:28:26.835 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:28:26.835 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:26.856 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:26.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.978+0000 7f375ce38640 1 Processor -- start 2024-05-09T06:28:26.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.978+0000 7f375ce38640 1 -- start start 2024-05-09T06:28:26.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f375ce38640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3758163c70 0x7f3758166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f375ce38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580ce0e0 con 0x7f37580cf180 2024-05-09T06:28:26.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f375ce38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580ce3c0 con 0x7f375806e6d0 2024-05-09T06:28:26.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f375ce38640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f37580ce6a0 con 0x7f3758163c70 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f3755d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f37580cf180 0x7f37580cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42536/0 (socket says 172.21.15.1:42536) 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.979+0000 7f3755d9b640 1 -- 172.21.15.1:0/549137511 learned_addr learned my addr 172.21.15.1:0/549137511 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642439221 0 0) 0x7f37580ce0e0 con 0x7f37580cf180 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f3756d9d640 1 --2- 172.21.15.1:0/549137511 >> v2:172.21.15.6:3300/0 conn(0x7f3758163c70 0x7f3758166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3744003520 con 0x7f37580cf180 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544627187 0 0) 0x7f3744003520 con 0x7f37580cf180 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 >> v2:172.21.15.6:3300/0 conn(0x7f3758163c70 msgr2=0x7f3758166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 --2- 172.21.15.1:0/549137511 >> v2:172.21.15.6:3300/0 conn(0x7f3758163c70 0x7f3758166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 >> v1:172.21.15.6:6789/0 conn(0x7f375806e6d0 legacy=0x7f375806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37580ce980 con 0x7f37580cf180 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3740001670 con 0x7f37580cf180 2024-05-09T06:28:26.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 >> v1:172.21.15.1:6789/0 conn(0x7f37580cf180 legacy=0x7f37580cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 0x7f3744003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3744004800 0x7f3744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440051c0 0x7f374400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37580ce6a0 con 0x7f37440039b0 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f374400c060 con 0x7f37440051c0 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f374400c260 con 0x7f3744004800 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375659c640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 0x7f3744003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f3756d9d640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440051c0 0x7f374400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f3755d9b640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3744004800 0x7f3744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375659c640 1 -- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3744004800 msgr2=0x7f3744004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.980+0000 7f375659c640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3744004800 0x7f3744004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375659c640 1 -- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440051c0 msgr2=0x7f374400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375659c640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440051c0 0x7f374400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375659c640 1 -- 172.21.15.1:0/549137511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3744003520 con 0x7f37440039b0 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f3756d9d640 1 -- 172.21.15.1:0/549137511 reap_dead start 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375659c640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 0x7f3744003dd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3750001e50 tx=0x7f3744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99862d480a6248 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750001a70 con 0x7f37440039b0 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375002ae40 con 0x7f37440039b0 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.981+0000 7f375559a640 1 -- 172.21.15.1:0/549137511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750003bb0 con 0x7f37440039b0 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 -- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 msgr2=0x7f3744003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 0x7f3744003dd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3750001e50 tx=0x7f3744005830 comp rx=0 tx=0).stop 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 -- 172.21.15.1:0/549137511 shutdown_connections 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 --2- 172.21.15.1:0/549137511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37440039b0 0x7f3744003dd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 -- 172.21.15.1:0/549137511 >> 172.21.15.1:0/549137511 conn(0x7f37580cc4b0 msgr2=0x7f37580cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 -- 172.21.15.1:0/549137511 shutdown_connections 2024-05-09T06:28:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.982+0000 7f375ce38640 1 -- 172.21.15.1:0/549137511 wait complete. 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.983+0000 7f375ce38640 1 Processor -- start 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.983+0000 7f375ce38640 1 -- start start 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440039b0 0x7f375815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f375814a390 0x7f375814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37581447c0 con 0x7f3758134050 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3758070ed0 con 0x7f37440039b0 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375ce38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3758144940 con 0x7f375814a390 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f3755d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f3755d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51478/0 (socket says 172.21.15.1:51478) 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f3755d9b640 1 -- 172.21.15.1:0/3939397858 learned_addr learned my addr 172.21.15.1:0/3939397858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f375659c640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440039b0 0x7f375815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.984+0000 7f3756d9d640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f375814a390 0x7f375814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 -- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f375814a390 msgr2=0x7f375814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f375814a390 0x7f375814a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 -- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440039b0 msgr2=0x7f375815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440039b0 0x7f375815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 -- 172.21.15.1:0/3939397858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37581361e0 con 0x7f3758134050 2024-05-09T06:28:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f3755d9b640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f374000e700 tx=0x7f374000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374000c6c0 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740018070 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374000caa0 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37581364c0 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.985+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3758136a80 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.986+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37580752b0 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.987+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f374000cc40 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.987+0000 7f374effd640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.987+0000 7f375659c640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.987+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f374008ae60 con 0x7f3758134050 2024-05-09T06:28:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.988+0000 7f375659c640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 0x7f373806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3750032000 tx=0x7f375002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:26.992+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3740058e50 con 0x7f3758134050 2024-05-09T06:28:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.180+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f375806cf50 con 0x7f3758134050 2024-05-09T06:28:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.181+0000 7f374effd640 1 -- 172.21.15.1:0/3939397858 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f374005d150 con 0x7f3758134050 2024-05-09T06:28:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:27.163 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 msgr2=0x7f373806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 0x7f373806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3750032000 tx=0x7f375002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 msgr2=0x7f37581344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f374000e700 tx=0x7f374000ebc0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 shutdown_connections 2024-05-09T06:28:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373806be30 0x7f373806e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f375814a390 0x7f375814a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3758134050 0x7f37581344d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 --2- 172.21.15.1:0/3939397858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37440039b0 0x7f375815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 >> 172.21.15.1:0/3939397858 conn(0x7f37580cc4b0 msgr2=0x7f375813a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 shutdown_connections 2024-05-09T06:28:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.183+0000 7f375ce38640 1 -- 172.21.15.1:0/3939397858 wait complete. 2024-05-09T06:28:27.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:27.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:27.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.303+0000 7fad5aa12640 1 Processor -- start 2024-05-09T06:28:27.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.303+0000 7fad5aa12640 1 -- start start 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad5aa12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 0x7fad540718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad5aa12640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad5415c4a0 0x7fad54071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad5aa12640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad5415b420 con 0x7fad5406e6d0 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad5aa12640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad54158e30 con 0x7fad5415c4a0 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad5aa12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad54072480 con 0x7fad54158a50 2024-05-09T06:28:27.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad537fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad5415c4a0 0x7fad54071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad53fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 0x7fad540718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad537fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad5415c4a0 0x7fad54071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:34038/0 (socket says 172.21.15.1:34038) 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad53fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 0x7fad540718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:40312/0 (socket says 172.21.15.1:40312) 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.304+0000 7fad537fe640 1 -- 172.21.15.1:0/2111031746 learned_addr learned my addr 172.21.15.1:0/2111031746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.305+0000 7fad537fe640 1 -- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 msgr2=0x7fad540718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.305+0000 7fad537fe640 1 --2- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 0x7fad540718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.305+0000 7fad537fe640 1 -- 172.21.15.1:0/2111031746 >> v1:172.21.15.6:6789/0 conn(0x7fad5406e6d0 legacy=0x7fad540cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.305+0000 7fad537fe640 1 -- 172.21.15.1:0/2111031746 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5415b6e0 con 0x7fad5415c4a0 2024-05-09T06:28:27.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.305+0000 7fad537fe640 1 --2- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad5415c4a0 0x7fad54071e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757ab3a23f65d539 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad52ffd640 1 -- 172.21.15.1:0/2111031746 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037070 con 0x7fad5415c4a0 2024-05-09T06:28:27.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad52ffd640 1 -- 172.21.15.1:0/2111031746 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48032440 con 0x7fad5415c4a0 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad52ffd640 1 -- 172.21.15.1:0/2111031746 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48032810 con 0x7fad5415c4a0 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 -- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad5415c4a0 msgr2=0x7fad54071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 --2- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad5415c4a0 0x7fad54071e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802a270 comp rx=0 tx=0).stop 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 -- 172.21.15.1:0/2111031746 shutdown_connections 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 --2- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad5415c4a0 0x7fad54071e00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 --2- 172.21.15.1:0/2111031746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad54158a50 0x7fad540718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 -- 172.21.15.1:0/2111031746 >> 172.21.15.1:0/2111031746 conn(0x7fad540cc4d0 msgr2=0x7fad540cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 -- 172.21.15.1:0/2111031746 shutdown_connections 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.306+0000 7fad5aa12640 1 -- 172.21.15.1:0/2111031746 wait complete. 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.307+0000 7fad5aa12640 1 Processor -- start 2024-05-09T06:28:27.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 -- start start 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5406e6d0 0x7fad540b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 0x7fad540b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad5415bf80 con 0x7fad5415c4a0 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad5415bca0 con 0x7fad54158a50 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.308+0000 7fad5aa12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad54070ea0 con 0x7fad5406e6d0 2024-05-09T06:28:27.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad53fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5406e6d0 0x7fad540b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51484/0 (socket says 172.21.15.1:51484) 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad537fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 0x7fad540b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 -- 172.21.15.1:0/94962437 learned_addr learned my addr 172.21.15.1:0/94962437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 -- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5406e6d0 msgr2=0x7fad540b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5406e6d0 0x7fad540b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 -- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 msgr2=0x7fad540b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 0x7fad540b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 -- 172.21.15.1:0/94962437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad540b38c0 con 0x7fad5415c4a0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad537fe640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 0x7fad540b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad58faf640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c6e0 tx=0x7fad4c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c003b70 con 0x7fad5415c4a0 2024-05-09T06:28:27.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad540b3ba0 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4c003d30 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c005310 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.309+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5415c9d0 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.310+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad540752b0 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.311+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad4c0054b0 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.311+0000 7fad50ff9640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 0x7fad4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.311+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fad4c08afc0 con 0x7fad5415c4a0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.311+0000 7fad53fff640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 0x7fad4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.312+0000 7fad53fff640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 0x7fad4006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad540b11f0 tx=0x7fad44009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.316+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad4c058f30 con 0x7fad5415c4a0 2024-05-09T06:28:27.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.498+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad5415ab80 con 0x7fad5415c4a0 2024-05-09T06:28:27.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.499+0000 7fad50ff9640 1 -- 172.21.15.1:0/94962437 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fad4c05d230 con 0x7fad5415c4a0 2024-05-09T06:28:27.481 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:27.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 msgr2=0x7fad4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 0x7fad4006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad540b11f0 tx=0x7fad44009190 comp rx=0 tx=0).stop 2024-05-09T06:28:27.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 msgr2=0x7fad5414d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c6e0 tx=0x7fad4c00cba0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 shutdown_connections 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad4006be30 0x7fad4006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad5415c4a0 0x7fad5414d5e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad54158a50 0x7fad540b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 --2- 172.21.15.1:0/94962437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad5406e6d0 0x7fad540b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 >> 172.21.15.1:0/94962437 conn(0x7fad540cc4d0 msgr2=0x7fad540aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 shutdown_connections 2024-05-09T06:28:27.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.501+0000 7fad5aa12640 1 -- 172.21.15.1:0/94962437 wait complete. 2024-05-09T06:28:27.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:27.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.619+0000 7f5ec2789640 1 Processor -- start 2024-05-09T06:28:27.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.619+0000 7f5ec2789640 1 -- start start 2024-05-09T06:28:27.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ec2789640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ec2789640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc15d5a0 0x7f5ebc15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ec2789640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc15c920 con 0x7f5ebc074660 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ec2789640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc15cba0 con 0x7f5ebc0cd450 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ec2789640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc15ce80 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ebbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5eb3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5ebc074660 0x7f5ebc160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42552/0 (socket says 172.21.15.1:42552) 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5eb3fff640 1 -- 172.21.15.1:0/1847659724 learned_addr learned my addr 172.21.15.1:0/1847659724 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.620+0000 7f5ebbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51486/0 (socket says 172.21.15.1:51486) 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ec0d26640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc15d5a0 0x7f5ebc15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebbfff640 1 -- 172.21.15.1:0/1847659724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc15d5a0 msgr2=0x7f5ebc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebbfff640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc15d5a0 0x7f5ebc15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebbfff640 1 -- 172.21.15.1:0/1847659724 >> v1:172.21.15.1:6789/0 conn(0x7f5ebc074660 legacy=0x7f5ebc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebbfff640 1 -- 172.21.15.1:0/1847659724 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc15d160 con 0x7f5ebc0cd450 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebb7fe640 1 -- 172.21.15.1:0/1847659724 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118566725 0 0) 0x7f5ea4001110 con 0x7f5ebc074660 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.621+0000 7f5ebbfff640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0097a0 tx=0x7f5eac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211ac9ea4714517 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ebb7fe640 1 -- 172.21.15.1:0/1847659724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037070 con 0x7f5ebc0cd450 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ebb7fe640 1 -- 172.21.15.1:0/1847659724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac032440 con 0x7f5ebc0cd450 2024-05-09T06:28:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ebb7fe640 1 -- 172.21.15.1:0/1847659724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac032810 con 0x7f5ebc0cd450 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 -- 172.21.15.1:0/1847659724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc0cd450 msgr2=0x7f5ebc15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0097a0 tx=0x7f5eac02a180 comp rx=0 tx=0).stop 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 -- 172.21.15.1:0/1847659724 shutdown_connections 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc15d5a0 0x7f5ebc15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 --2- 172.21.15.1:0/1847659724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc0cd450 0x7f5ebc15c3e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 -- 172.21.15.1:0/1847659724 >> 172.21.15.1:0/1847659724 conn(0x7f5ebc0b1da0 msgr2=0x7f5ebc0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 -- 172.21.15.1:0/1847659724 shutdown_connections 2024-05-09T06:28:27.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.622+0000 7f5ec2789640 1 -- 172.21.15.1:0/1847659724 wait complete. 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.623+0000 7f5ec2789640 1 Processor -- start 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.623+0000 7f5ec2789640 1 -- start start 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc074660 0x7f5ebc159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc0cd450 0x7f5ebc099740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0cd9b0 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0cd830 con 0x7f5ebc074660 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec2789640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc1607a0 con 0x7f5ebc0cd450 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec0d26640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec0d26640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51492/0 (socket says 172.21.15.1:51492) 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ec0d26640 1 -- 172.21.15.1:0/3325583570 learned_addr learned my addr 172.21.15.1:0/3325583570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5eb3fff640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc0cd450 0x7f5ebc099740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.624+0000 7f5ebbfff640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc074660 0x7f5ebc159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 -- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc0cd450 msgr2=0x7f5ebc099740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc0cd450 0x7f5ebc099740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 -- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc074660 msgr2=0x7f5ebc159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc074660 0x7f5ebc159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 -- 172.21.15.1:0/3325583570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0c59a0 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec0d26640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4007e90 tx=0x7f5eb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4014420 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4019070 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc0c5c80 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4014800 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.625+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc0c6240 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.626+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0752b0 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.627+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5eb40149a0 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.627+0000 7f5eb97fa640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 0x7f5e9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.627+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5eb408ba70 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.627+0000 7f5ebbfff640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 0x7f5e9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.627+0000 7f5ebbfff640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 0x7f5e9c06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.631+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5eb405a020 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.818+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5ebc0c6410 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.818+0000 7f5eb97fa640 1 -- 172.21.15.1:0/3325583570 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f5eb4059920 con 0x7f5ebc15d5a0 2024-05-09T06:28:27.801 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.820+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 msgr2=0x7f5e9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.820+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 0x7f5e9c06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.820+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 msgr2=0x7f5ebc1e7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.820+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4007e90 tx=0x7f5eb400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 shutdown_connections 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e9c06be30 0x7f5e9c06e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ebc15d5a0 0x7f5ebc1e7120 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ebc0cd450 0x7f5ebc099740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 --2- 172.21.15.1:0/3325583570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ebc074660 0x7f5ebc159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 >> 172.21.15.1:0/3325583570 conn(0x7f5ebc0b1da0 msgr2=0x7f5ebc15a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 shutdown_connections 2024-05-09T06:28:27.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.821+0000 7f5ec2789640 1 -- 172.21.15.1:0/3325583570 wait complete. 2024-05-09T06:28:27.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.939+0000 7f62e1c12640 1 Processor -- start 2024-05-09T06:28:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.939+0000 7f62e1c12640 1 -- start start 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc158a50 0x7f62dc0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62dc15c4a0 0x7f62dc071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f62dc06e6d0 0x7f62dc06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f62dc158e30 con 0x7f62dc15c4a0 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f62dc072480 con 0x7f62dc06e6d0 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62e1c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f62dc15b420 con 0x7f62dc158a50 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62dc15c4a0 0x7f62dc071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62dc15c4a0 0x7f62dc071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48012/0 (socket says 172.21.15.1:48012) 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 -- 172.21.15.1:0/4162566818 learned_addr learned my addr 172.21.15.1:0/4162566818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62db7fe640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc158a50 0x7f62dc0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62dbfff640 1 --2- 172.21.15.1:0/4162566818 >> v2:172.21.15.6:3300/0 conn(0x7f62dc06e6d0 0x7f62dc06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 -- 172.21.15.1:0/4162566818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc158a50 msgr2=0x7f62dc0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc158a50 0x7f62dc0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 -- 172.21.15.1:0/4162566818 >> v2:172.21.15.6:3300/0 conn(0x7f62dc06e6d0 msgr2=0x7f62dc06eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 --2- 172.21.15.1:0/4162566818 >> v2:172.21.15.6:3300/0 conn(0x7f62dc06e6d0 0x7f62dc06eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.940+0000 7f62daffd640 1 -- 172.21.15.1:0/4162566818 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc15b6e0 con 0x7f62dc15c4a0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.941+0000 7f62daffd640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc15c4a0 0x7f62dc071e00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f62dc06c2a0 tx=0x7f62d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7efaf0971ef015 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.941+0000 7f62da7fc640 1 -- 172.21.15.1:0/4162566818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0037070 con 0x7f62dc15c4a0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.941+0000 7f62da7fc640 1 -- 172.21.15.1:0/4162566818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d0032440 con 0x7f62dc15c4a0 2024-05-09T06:28:27.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.941+0000 7f62da7fc640 1 -- 172.21.15.1:0/4162566818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0032810 con 0x7f62dc15c4a0 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 -- 172.21.15.1:0/4162566818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc15c4a0 msgr2=0x7f62dc071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc15c4a0 0x7f62dc071e00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f62dc06c2a0 tx=0x7f62d002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 -- 172.21.15.1:0/4162566818 shutdown_connections 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 --2- 172.21.15.1:0/4162566818 >> v2:172.21.15.6:3300/0 conn(0x7f62dc06e6d0 0x7f62dc06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc15c4a0 0x7f62dc071e00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 --2- 172.21.15.1:0/4162566818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc158a50 0x7f62dc0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 -- 172.21.15.1:0/4162566818 >> 172.21.15.1:0/4162566818 conn(0x7f62dc0cc4d0 msgr2=0x7f62dc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 -- 172.21.15.1:0/4162566818 shutdown_connections 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.942+0000 7f62e1c12640 1 -- 172.21.15.1:0/4162566818 wait complete. 2024-05-09T06:28:27.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.943+0000 7f62e1c12640 1 Processor -- start 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.943+0000 7f62e1c12640 1 -- start start 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62dc06e6d0 0x7f62dc150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc15c4a0 0x7f62dc1f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc15bf80 con 0x7f62dc158a50 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc15bca0 con 0x7f62dc06e6d0 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62e1c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f62dc070ea0 con 0x7f62dc15c4a0 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48020/0 (socket says 172.21.15.1:48020) 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 -- 172.21.15.1:0/2923983009 learned_addr learned my addr 172.21.15.1:0/2923983009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 -- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc15c4a0 msgr2=0x7f62dc1f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc15c4a0 0x7f62dc1f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 -- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62dc06e6d0 msgr2=0x7f62dc150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62dc06e6d0 0x7f62dc150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:27.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 -- 172.21.15.1:0/2923983009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc155d70 con 0x7f62dc158a50 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62daffd640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0041070 con 0x7f62dc158a50 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d003c870 con 0x7f62dc158a50 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.944+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0037040 con 0x7f62dc158a50 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.945+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc0ae440 con 0x7f62dc158a50 2024-05-09T06:28:27.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.945+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc0aea00 con 0x7f62dc158a50 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.946+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc0ae270 con 0x7f62dc158a50 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.946+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62d0044070 con 0x7f62dc158a50 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.946+0000 7f62c3fff640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.947+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f62d00abf70 con 0x7f62dc158a50 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.947+0000 7f62db7fe640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:27.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.947+0000 7f62db7fe640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 0x7f62c806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b940 tx=0x7f62cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:27.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:27.951+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f62d007a630 con 0x7f62dc158a50 2024-05-09T06:28:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.133+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f62dc0752b0 con 0x7f62dc158a50 2024-05-09T06:28:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.133+0000 7f62c3fff640 1 -- 172.21.15.1:0/2923983009 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f62d0079f30 con 0x7f62dc158a50 2024-05-09T06:28:28.115 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 msgr2=0x7f62c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 0x7f62c806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b940 tx=0x7f62cc008040 comp rx=0 tx=0).stop 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 msgr2=0x7f62dc153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d00046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 shutdown_connections 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f62c806be30 0x7f62c806e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62dc15c4a0 0x7f62dc1f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62dc158a50 0x7f62dc153dc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.135+0000 7f62e1c12640 1 --2- 172.21.15.1:0/2923983009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62dc06e6d0 0x7f62dc150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.136+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 >> 172.21.15.1:0/2923983009 conn(0x7f62dc0cc4d0 msgr2=0x7f62dc071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.136+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 shutdown_connections 2024-05-09T06:28:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.136+0000 7f62e1c12640 1 -- 172.21.15.1:0/2923983009 wait complete. 2024-05-09T06:28:28.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:28.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.254+0000 7fea758c3640 1 Processor -- start 2024-05-09T06:28:28.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.254+0000 7fea758c3640 1 -- start start 2024-05-09T06:28:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea758c3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea7015e270 0x7fea700cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea758c3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea700b1340 con 0x7fea700ceba0 2024-05-09T06:28:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea758c3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea700d1020 con 0x7fea700d08b0 2024-05-09T06:28:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea758c3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fea700cc260 con 0x7fea7015e270 2024-05-09T06:28:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea6f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fea700ceba0 0x7fea700ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51108/0 (socket says 172.21.15.1:51108) 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea6f7fe640 1 -- 172.21.15.1:0/1725248536 learned_addr learned my addr 172.21.15.1:0/1725248536 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.255+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637148609 0 0) 0x7fea68001490 con 0x7fea700ceba0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea58003520 con 0x7fea700ceba0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6effd640 1 --2- 172.21.15.1:0/1725248536 >> v2:172.21.15.6:3300/0 conn(0x7fea7015e270 0x7fea700cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3001633546 0 0) 0x7fea58003520 con 0x7fea700ceba0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 >> v2:172.21.15.6:3300/0 conn(0x7fea7015e270 msgr2=0x7fea700cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 --2- 172.21.15.1:0/1725248536 >> v2:172.21.15.6:3300/0 conn(0x7fea7015e270 0x7fea700cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 >> v1:172.21.15.6:6789/0 conn(0x7fea700d08b0 legacy=0x7fea700ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea700cc4b0 con 0x7fea700ceba0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fea68001bc0 con 0x7fea700ceba0 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 >> v1:172.21.15.1:6789/0 conn(0x7fea700ceba0 legacy=0x7fea700ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea580039b0 0x7fea58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 0x7fea58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea580051c0 0x7fea5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea700cc260 con 0x7fea58004800 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fea5800c070 con 0x7fea580039b0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fea5800c270 con 0x7fea580051c0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 0x7fea58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6effd640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea580039b0 0x7fea58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6f7fe640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea580051c0 0x7fea5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 -- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea580051c0 msgr2=0x7fea5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea580051c0 0x7fea5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 -- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea580039b0 msgr2=0x7fea58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea580039b0 0x7fea58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6e7fc640 1 -- 172.21.15.1:0/1725248536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea58003520 con 0x7fea58004800 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.256+0000 7fea6f7fe640 1 -- 172.21.15.1:0/1725248536 reap_dead start 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.257+0000 7fea6e7fc640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 0x7fea58004c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fea64001750 tx=0x7fea58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e7ce309b881d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.257+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64001370 con 0x7fea58004800 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.257+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea640052d0 con 0x7fea58004800 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.257+0000 7fea6dffb640 1 -- 172.21.15.1:0/1725248536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6402abc0 con 0x7fea58004800 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 -- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 msgr2=0x7fea58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 0x7fea58004c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fea64001750 tx=0x7fea58005830 comp rx=0 tx=0).stop 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 -- 172.21.15.1:0/1725248536 shutdown_connections 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 --2- 172.21.15.1:0/1725248536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea58004800 0x7fea58004c80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 -- 172.21.15.1:0/1725248536 >> 172.21.15.1:0/1725248536 conn(0x7fea700d1c10 msgr2=0x7fea700d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 -- 172.21.15.1:0/1725248536 shutdown_connections 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.258+0000 7fea758c3640 1 -- 172.21.15.1:0/1725248536 wait complete. 2024-05-09T06:28:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.259+0000 7fea758c3640 1 Processor -- start 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.259+0000 7fea758c3640 1 -- start start 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea58004800 0x7fea70153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7009bbb0 0x7fea7015f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea700d0d20 con 0x7fea7015fdc0 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fea700cca70 con 0x7fea7009bbb0 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea758c3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fea7015e650 con 0x7fea58004800 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48030/0 (socket says 172.21.15.1:48030) 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 -- 172.21.15.1:0/809939459 learned_addr learned my addr 172.21.15.1:0/809939459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 -- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea58004800 msgr2=0x7fea70153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6effd640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea58004800 0x7fea70153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6e7fc640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7009bbb0 0x7fea7015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea58004800 0x7fea70153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 -- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7009bbb0 msgr2=0x7fea7015f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7009bbb0 0x7fea7015f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 -- 172.21.15.1:0/809939459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea700cbdb0 con 0x7fea7015fdc0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea6f7fe640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea7006c1c0 tx=0x7fea68007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.260+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800ea80 con 0x7fea7015fdc0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.261+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68019070 con 0x7fea7015fdc0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.261+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea700c9280 con 0x7fea7015fdc0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.261+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800ee80 con 0x7fea7015fdc0 2024-05-09T06:28:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.261+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea700cb9d0 con 0x7fea7015fdc0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.262+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea700cc090 con 0x7fea7015fdc0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.262+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea6801e460 con 0x7fea7015fdc0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.263+0000 7fea57fff640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 0x7fea5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.263+0000 7fea6effd640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 0x7fea5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.263+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fea6808b4d0 con 0x7fea7015fdc0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.263+0000 7fea6effd640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 0x7fea5c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea60009a60 tx=0x7fea60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.267+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea700b1340 con 0x7fea7015fdc0 2024-05-09T06:28:28.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.450+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fea700752b0 con 0x7fea7015fdc0 2024-05-09T06:28:28.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.450+0000 7fea57fff640 1 -- 172.21.15.1:0/809939459 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fea68059a70 con 0x7fea7015fdc0 2024-05-09T06:28:28.432 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:28.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 msgr2=0x7fea5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 0x7fea5c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea60009a60 tx=0x7fea60008040 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 msgr2=0x7fea70160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea7006c1c0 tx=0x7fea68007bc0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 shutdown_connections 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea5c06be30 0x7fea5c06e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea7015fdc0 0x7fea70160240 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7009bbb0 0x7fea7015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 --2- 172.21.15.1:0/809939459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea58004800 0x7fea70153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 >> 172.21.15.1:0/809939459 conn(0x7fea700d1c10 msgr2=0x7fea700cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 shutdown_connections 2024-05-09T06:28:28.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.452+0000 7fea758c3640 1 -- 172.21.15.1:0/809939459 wait complete. 2024-05-09T06:28:28.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:28.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.568+0000 7fb29eaa5640 1 Processor -- start 2024-05-09T06:28:28.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 -- start start 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb2980c6860 0x7fb298099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb2980b2aa0 0x7fb29815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb298099930 con 0x7fb2980c6c40 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb2980b1e20 con 0x7fb2980c6860 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29eaa5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb29815ee90 con 0x7fb2980b2aa0 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.569+0000 7fb29c841640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb2980c6860 0x7fb298099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb2980c6860 0x7fb298099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48042/0 (socket says 172.21.15.1:48042) 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 -- 172.21.15.1:0/3576627491 learned_addr learned my addr 172.21.15.1:0/3576627491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 -- 172.21.15.1:0/3576627491 >> v2:172.21.15.6:3300/0 conn(0x7fb2980b2aa0 msgr2=0x7fb29815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 --2- 172.21.15.1:0/3576627491 >> v2:172.21.15.6:3300/0 conn(0x7fb2980b2aa0 0x7fb29815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 -- 172.21.15.1:0/3576627491 >> v1:172.21.15.6:6789/0 conn(0x7fb2980c6c40 legacy=0x7fb29815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.570+0000 7fb29c841640 1 -- 172.21.15.1:0/3576627491 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb298162930 con 0x7fb2980c6860 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29c841640 1 --2- 172.21.15.1:0/3576627491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980c6860 0x7fb298099b30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb29809e570 tx=0x7fb28802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaa5bec4c4b553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb2977fe640 1 -- 172.21.15.1:0/3576627491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288037070 con 0x7fb2980c6860 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb2977fe640 1 -- 172.21.15.1:0/3576627491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb288032440 con 0x7fb2980c6860 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb2977fe640 1 -- 172.21.15.1:0/3576627491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288032810 con 0x7fb2980c6860 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 -- 172.21.15.1:0/3576627491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980c6860 msgr2=0x7fb298099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/3576627491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980c6860 0x7fb298099b30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb29809e570 tx=0x7fb28802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 -- 172.21.15.1:0/3576627491 shutdown_connections 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/3576627491 >> v2:172.21.15.6:3300/0 conn(0x7fb2980b2aa0 0x7fb29815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/3576627491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980c6860 0x7fb298099b30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 -- 172.21.15.1:0/3576627491 >> 172.21.15.1:0/3576627491 conn(0x7fb2980b1fb0 msgr2=0x7fb2980b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 -- 172.21.15.1:0/3576627491 shutdown_connections 2024-05-09T06:28:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.571+0000 7fb29eaa5640 1 -- 172.21.15.1:0/3576627491 wait complete. 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.572+0000 7fb29eaa5640 1 Processor -- start 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 -- start start 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2980c6860 0x7fb2980c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2980c6c40 0x7fb2981f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb29815eb20 con 0x7fb2980b2aa0 2024-05-09T06:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb29815fa30 con 0x7fb2980c6c40 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.573+0000 7fb29eaa5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980c70d0 con 0x7fb2980c6860 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48044/0 (socket says 172.21.15.1:48044) 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 -- 172.21.15.1:0/2661336293 learned_addr learned my addr 172.21.15.1:0/2661336293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29d042640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2980c6c40 0x7fb2981f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb297fff640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2980c6860 0x7fb2980c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 -- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2980c6860 msgr2=0x7fb2980c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2980c6860 0x7fb2980c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 -- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2980c6c40 msgr2=0x7fb2981f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2980c6c40 0x7fb2981f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 -- 172.21.15.1:0/2661336293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980c7b50 con 0x7fb2980b2aa0 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29c841640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288041070 con 0x7fb2980b2aa0 2024-05-09T06:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28803c870 con 0x7fb2980b2aa0 2024-05-09T06:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288037040 con 0x7fb2980b2aa0 2024-05-09T06:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2980c7e30 con 0x7fb2980b2aa0 2024-05-09T06:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.574+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981f4580 con 0x7fb2980b2aa0 2024-05-09T06:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.575+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2981f3e50 con 0x7fb2980b2aa0 2024-05-09T06:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.576+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb288044070 con 0x7fb2980b2aa0 2024-05-09T06:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.576+0000 7fb2957fa640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 0x7fb28406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.577+0000 7fb297fff640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 0x7fb28406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.577+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb2880ac090 con 0x7fb2980b2aa0 2024-05-09T06:28:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.577+0000 7fb297fff640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 0x7fb28406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb29809e570 tx=0x7fb28c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.581+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb298099930 con 0x7fb2980b2aa0 2024-05-09T06:28:28.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.778+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2980752b0 con 0x7fb2980b2aa0 2024-05-09T06:28:28.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.778+0000 7fb2957fa640 1 -- 172.21.15.1:0/2661336293 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fb28807a750 con 0x7fb2980b2aa0 2024-05-09T06:28:28.760 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.780+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 msgr2=0x7fb28406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.780+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 0x7fb28406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb29809e570 tx=0x7fb28c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.780+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 msgr2=0x7fb2980c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.780+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 shutdown_connections 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb28406be30 0x7fb28406e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2980c6c40 0x7fb2981f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2980c6860 0x7fb2980c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 --2- 172.21.15.1:0/2661336293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2980b2aa0 0x7fb2980c9610 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 >> 172.21.15.1:0/2661336293 conn(0x7fb2980b1fb0 msgr2=0x7fb29806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 shutdown_connections 2024-05-09T06:28:28.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.781+0000 7fb29eaa5640 1 -- 172.21.15.1:0/2661336293 wait complete. 2024-05-09T06:28:28.773 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2024-05-09T06:28:28.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:28.774 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-09T06:28:28.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:28.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:28:28.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:28.805 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:28:28.864 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:28:28.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:28.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.986+0000 7f9d2319e640 1 Processor -- start 2024-05-09T06:28:28.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.986+0000 7f9d2319e640 1 -- start start 2024-05-09T06:28:28.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2319e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2319e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 0x7f9d1c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2319e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d1c0b48a0 con 0x7f9d1c0cfa00 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2319e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1c0b4b80 con 0x7f9d1c0d0630 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2319e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c0d4b10 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.987+0000 7f9d2219c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48050/0 (socket says 172.21.15.1:48050) 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 -- 172.21.15.1:0/550363612 learned_addr learned my addr 172.21.15.1:0/550363612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2299d640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 0x7f9d1c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 -- 172.21.15.1:0/550363612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 msgr2=0x7f9d1c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 0x7f9d1c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 -- 172.21.15.1:0/550363612 >> v1:172.21.15.6:6789/0 conn(0x7f9d1c0cfa00 legacy=0x7f9d1c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2219c640 1 -- 172.21.15.1:0/550363612 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c0d4830 con 0x7f9d1c0d0630 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.988+0000 7f9d2299d640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 0x7f9d1c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:28.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2219c640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009870 tx=0x7f9d1802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8972579b3de3fc55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2119a640 1 -- 172.21.15.1:0/550363612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18037070 con 0x7f9d1c0d0630 2024-05-09T06:28:28.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2119a640 1 -- 172.21.15.1:0/550363612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1802ae10 con 0x7f9d1c0d0630 2024-05-09T06:28:28.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2119a640 1 -- 172.21.15.1:0/550363612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18032660 con 0x7f9d1c0d0630 2024-05-09T06:28:28.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 -- 172.21.15.1:0/550363612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d0630 msgr2=0x7f9d1c0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009870 tx=0x7f9d1802a100 comp rx=0 tx=0).stop 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 -- 172.21.15.1:0/550363612 shutdown_connections 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0d29d0 0x7f9d1c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 --2- 172.21.15.1:0/550363612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c0b3c30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 -- 172.21.15.1:0/550363612 >> 172.21.15.1:0/550363612 conn(0x7f9d1c0d3be0 msgr2=0x7f9d1c0d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 -- 172.21.15.1:0/550363612 shutdown_connections 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.989+0000 7f9d2319e640 1 -- 172.21.15.1:0/550363612 wait complete. 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 Processor -- start 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 -- start start 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0cfa00 0x7f9d1c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c1f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0d0b90 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0d0a10 con 0x7f9d1c0d0630 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.991+0000 7f9d2319e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c0d0d10 con 0x7f9d1c0cfa00 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48054/0 (socket says 172.21.15.1:48054) 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 -- 172.21.15.1:0/3321755769 learned_addr learned my addr 172.21.15.1:0/3321755769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2219c640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0cfa00 0x7f9d1c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2199b640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c1f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 -- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0cfa00 msgr2=0x7f9d1c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0cfa00 0x7f9d1c0cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 -- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 msgr2=0x7f9d1c1f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c1f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 -- 172.21.15.1:0/3321755769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c0b0850 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2199b640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c1f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2299d640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400cef0 tx=0x7f9d1400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1401a070 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1400d860 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c0b0b30 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400dc50 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.992+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c0b10f0 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.993+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c0752b0 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.994+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d1401d070 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.994+0000 7f9d12ffd640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 0x7f9d0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.994+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9d1408baf0 con 0x7f9d1c0d29d0 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.994+0000 7f9d2219c640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 0x7f9d0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.995+0000 7f9d2219c640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 0x7f9d0006e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d18034000 tx=0x7f9d1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:28.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:28.999+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d14059a90 con 0x7f9d1c0d29d0 2024-05-09T06:28:29.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.222+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d1c1f3780 con 0x7f9d1c0d29d0 2024-05-09T06:28:29.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.222+0000 7f9d12ffd640 1 -- 172.21.15.1:0/3321755769 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f9d1405dd90 con 0x7f9d1c0d29d0 2024-05-09T06:28:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:29.205 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:28:29.205 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 msgr2=0x7f9d0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 0x7f9d0006e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d18034000 tx=0x7f9d1802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 msgr2=0x7f9d1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400cef0 tx=0x7f9d1400a710 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 shutdown_connections 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9d0006be30 0x7f9d0006e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d1c0d29d0 0x7f9d1c06c510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d1c0d0630 0x7f9d1c1f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 --2- 172.21.15.1:0/3321755769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9d1c0cfa00 0x7f9d1c0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 >> 172.21.15.1:0/3321755769 conn(0x7f9d1c0d3be0 msgr2=0x7f9d1c161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 shutdown_connections 2024-05-09T06:28:29.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.224+0000 7f9d2319e640 1 -- 172.21.15.1:0/3321755769 wait complete. 2024-05-09T06:28:29.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:29.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.339+0000 7f69256ba640 1 Processor -- start 2024-05-09T06:28:29.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.340+0000 7f69256ba640 1 -- start start 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.340+0000 7f69256ba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69200d2b10 0x7f69200b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.340+0000 7f69256ba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200b33c0 con 0x7f69200d4940 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.340+0000 7f69256ba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200b3660 con 0x7f6920161950 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.340+0000 7f69256ba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69200d4dd0 con 0x7f69200d2b10 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69200d2b10 0x7f69200b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691e7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69200d4940 0x7f69200b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51118/0 (socket says 172.21.15.1:51118) 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691e7fc640 1 -- 172.21.15.1:0/921134346 learned_addr learned my addr 172.21.15.1:0/921134346 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691f7fe640 1 -- 172.21.15.1:0/921134346 >> v1:172.21.15.6:6789/0 conn(0x7f6920161950 legacy=0x7f69200d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691f7fe640 1 -- 172.21.15.1:0/921134346 >> v1:172.21.15.1:6789/0 conn(0x7f69200d4940 legacy=0x7f69200b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691f7fe640 1 -- 172.21.15.1:0/921134346 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69200b3940 con 0x7f69200d2b10 2024-05-09T06:28:29.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.341+0000 7f691dffb640 1 -- 172.21.15.1:0/921134346 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578480052 0 0) 0x7f69200b33c0 con 0x7f69200d4940 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f691f7fe640 1 --2- 172.21.15.1:0/921134346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200d2b10 0x7f69200b4850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f692008e540 tx=0x7f691002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff2ce3a7bb8f56d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f691dffb640 1 -- 172.21.15.1:0/921134346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910037070 con 0x7f69200d2b10 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f691dffb640 1 -- 172.21.15.1:0/921134346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6910032440 con 0x7f69200d2b10 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f691dffb640 1 -- 172.21.15.1:0/921134346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910032810 con 0x7f69200d2b10 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 -- 172.21.15.1:0/921134346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200d2b10 msgr2=0x7f69200b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 --2- 172.21.15.1:0/921134346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200d2b10 0x7f69200b4850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f692008e540 tx=0x7f691002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 -- 172.21.15.1:0/921134346 shutdown_connections 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 --2- 172.21.15.1:0/921134346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200d2b10 0x7f69200b4850 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 -- 172.21.15.1:0/921134346 >> 172.21.15.1:0/921134346 conn(0x7f69200d3e20 msgr2=0x7f69200d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 -- 172.21.15.1:0/921134346 shutdown_connections 2024-05-09T06:28:29.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.342+0000 7f69256ba640 1 -- 172.21.15.1:0/921134346 wait complete. 2024-05-09T06:28:29.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 Processor -- start 2024-05-09T06:28:29.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 -- start start 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69200d2b10 0x7f6920152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69200cd090 0x7f69200cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.344+0000 7f69256ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69200d1100 con 0x7f6920161950 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69256ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69200d0f80 con 0x7f69200d2b10 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69256ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69200d1c00 con 0x7f69200cd090 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48064/0 (socket says 172.21.15.1:48064) 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 -- 172.21.15.1:0/3684580949 learned_addr learned my addr 172.21.15.1:0/3684580949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691effd640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69200d2b10 0x7f6920152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691f7fe640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69200cd090 0x7f69200cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 -- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69200cd090 msgr2=0x7f69200cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69200cd090 0x7f69200cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 -- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69200d2b10 msgr2=0x7f6920152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69200d2b10 0x7f6920152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 -- 172.21.15.1:0/3684580949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69200b6a10 con 0x7f6920161950 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f691e7fc640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f690800c4d0 tx=0x7f690800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908018070 con 0x7f6920161950 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6908007500 con 0x7f6920161950 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908004a20 con 0x7f6920161950 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69200b6cb0 con 0x7f6920161950 2024-05-09T06:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.345+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69200b7270 con 0x7f6920161950 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.346+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692015fd50 con 0x7f6920161950 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.347+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6908004bc0 con 0x7f6920161950 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.347+0000 7f69177fe640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 0x7f690006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.348+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f690808a4e0 con 0x7f6920161950 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.348+0000 7f691effd640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 0x7f690006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.348+0000 7f691effd640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 0x7f690006e320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f692008d7a0 tx=0x7f691800d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.352+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f692015fd50 con 0x7f6920161950 2024-05-09T06:28:29.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.537+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69200cd4a0 con 0x7f6920161950 2024-05-09T06:28:29.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.538+0000 7f69177fe640 1 -- 172.21.15.1:0/3684580949 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f69200cd4a0 con 0x7f6920161950 2024-05-09T06:28:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:29.521 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:29.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 msgr2=0x7f690006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 0x7f690006e320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f692008d7a0 tx=0x7f691800d040 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 msgr2=0x7f69200ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f690800c4d0 tx=0x7f690800c990 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 shutdown_connections 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f690006be60 0x7f690006e320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69200cd090 0x7f69200cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920161950 0x7f69200ccb50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 --2- 172.21.15.1:0/3684580949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69200d2b10 0x7f6920152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 >> 172.21.15.1:0/3684580949 conn(0x7f69200d3e20 msgr2=0x7f69200b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 shutdown_connections 2024-05-09T06:28:29.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.540+0000 7f69256ba640 1 -- 172.21.15.1:0/3684580949 wait complete. 2024-05-09T06:28:29.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:27:19.164805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:27:19.974104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T06:27:23.679732+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:28:29.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:29.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.659+0000 7fa8d497d640 1 Processor -- start 2024-05-09T06:28:29.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.659+0000 7fa8d497d640 1 -- start start 2024-05-09T06:28:29.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8d497d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa8d015e270 0x7fa8d00cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8d497d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d00b1340 con 0x7fa8d00d08b0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8d497d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d00d1020 con 0x7fa8d00ceba0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8d497d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa8d00cc260 con 0x7fa8d015e270 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cdd9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa8d00d08b0 0x7fa8d00ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51120/0 (socket says 172.21.15.1:51120) 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cdd9b640 1 -- 172.21.15.1:0/4262440367 learned_addr learned my addr 172.21.15.1:0/4262440367 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2272384484 0 0) 0x7fa8c4001110 con 0x7fa8d00d08b0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8ce59c640 1 --2- 172.21.15.1:0/4262440367 >> v2:172.21.15.6:3300/0 conn(0x7fa8d015e270 0x7fa8d00cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8b8003520 con 0x7fa8d00d08b0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3298064729 0 0) 0x7fa8b8003520 con 0x7fa8d00d08b0 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 >> v2:172.21.15.6:3300/0 conn(0x7fa8d015e270 msgr2=0x7fa8d00cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 --2- 172.21.15.1:0/4262440367 >> v2:172.21.15.6:3300/0 conn(0x7fa8d015e270 0x7fa8d00cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 >> v1:172.21.15.6:6789/0 conn(0x7fa8d00ceba0 legacy=0x7fa8d00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d00cc4b0 con 0x7fa8d00d08b0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fa8c4001840 con 0x7fa8d00d08b0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 >> v1:172.21.15.1:6789/0 conn(0x7fa8d00d08b0 legacy=0x7fa8d00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8b80051c0 0x7fa8b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d00cc260 con 0x7fa8b8004800 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b800c070 con 0x7fa8b80039b0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.660+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b800c270 con 0x7fa8b80051c0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8ce59c640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8ced9d640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8b80051c0 0x7fa8b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 -- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8b80051c0 msgr2=0x7fa8b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8b80051c0 0x7fa8b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 -- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8b80039b0 msgr2=0x7fa8b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8cdd9b640 1 -- 172.21.15.1:0/4262440367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b8003520 con 0x7fa8b8004800 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8ced9d640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8b80051c0 0x7fa8b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.661+0000 7fa8ced9d640 1 -- 172.21.15.1:0/4262440367 reap_dead start 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c4c0 tx=0x7fa8b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22b29bef713765 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400c0e0 con 0x7fa8b8004800 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c4034070 con 0x7fa8b8004800 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8cd59a640 1 -- 172.21.15.1:0/4262440367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4005d70 con 0x7fa8b8004800 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 -- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 msgr2=0x7fa8b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c4c0 tx=0x7fa8b8005830 comp rx=0 tx=0).stop 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 -- 172.21.15.1:0/4262440367 shutdown_connections 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 --2- 172.21.15.1:0/4262440367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 -- 172.21.15.1:0/4262440367 >> 172.21.15.1:0/4262440367 conn(0x7fa8d00d1c10 msgr2=0x7fa8d00d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 -- 172.21.15.1:0/4262440367 shutdown_connections 2024-05-09T06:28:29.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.662+0000 7fa8d497d640 1 -- 172.21.15.1:0/4262440367 wait complete. 2024-05-09T06:28:29.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.664+0000 7fa8d497d640 1 Processor -- start 2024-05-09T06:28:29.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.664+0000 7fa8d497d640 1 -- start start 2024-05-09T06:28:29.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 0x7fa8d01527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 0x7fa8d0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d00d0d20 con 0x7fa8b8004800 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d00cca70 con 0x7fa8d01551f0 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d015e650 con 0x7fa8d0153020 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48082/0 (socket says 172.21.15.1:48082) 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 -- 172.21.15.1:0/3339151150 learned_addr learned my addr 172.21.15.1:0/3339151150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 0x7fa8d01527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ced9d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 0x7fa8d0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 -- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 msgr2=0x7fa8d0159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 0x7fa8d0159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 -- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 msgr2=0x7fa8d01527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 0x7fa8d01527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 -- 172.21.15.1:0/3339151150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d01584f0 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ced9d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 0x7fa8d0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 0x7fa8d01527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8ce59c640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0011aa0 tx=0x7fa8c0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000fb80 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c000fd20 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d00cb910 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0014660 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.665+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d0152d10 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.667+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d006eac0 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.667+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8c0012040 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.667+0000 7fa8beffd640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 0x7fa8b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.668+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa8c008b980 con 0x7fa8b8004800 2024-05-09T06:28:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.668+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 0x7fa8b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.668+0000 7fa8cdd9b640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 0x7fa8b006e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c490 tx=0x7fa8c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.672+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8d006eac0 con 0x7fa8b8004800 2024-05-09T06:28:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.858+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8d00752b0 con 0x7fa8b8004800 2024-05-09T06:28:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.859+0000 7fa8beffd640 1 -- 172.21.15.1:0/3339151150 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa8c0019030 con 0x7fa8b8004800 2024-05-09T06:28:29.841 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 msgr2=0x7fa8b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 0x7fa8b006e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c490 tx=0x7fa8c402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 msgr2=0x7fa8d0157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0011aa0 tx=0x7fa8c0011f60 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 shutdown_connections 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8b006be30 0x7fa8b006e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8d0153020 0x7fa8d0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8d01551f0 0x7fa8d01527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 --2- 172.21.15.1:0/3339151150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8b8004800 0x7fa8d0157700 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 >> 172.21.15.1:0/3339151150 conn(0x7fa8d00d1c10 msgr2=0x7fa8d00aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 shutdown_connections 2024-05-09T06:28:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.861+0000 7fa8d497d640 1 -- 172.21.15.1:0/3339151150 wait complete. 2024-05-09T06:28:29.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:29.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 Processor -- start 2024-05-09T06:28:29.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 -- start start 2024-05-09T06:28:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc84006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc8400b4de0 0x7fc84015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc84006cf40 con 0x7fc8400b5260 2024-05-09T06:28:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc840156980 con 0x7fc8400b4de0 2024-05-09T06:28:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.980+0000 7fc8458da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc84006d870 con 0x7fc84015c4c0 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc8400b5260 0x7fc840070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51126/0 (socket says 172.21.15.1:51126) 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83f7fe640 1 -- 172.21.15.1:0/1488806564 learned_addr learned my addr 172.21.15.1:0/1488806564 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83e7fc640 1 --2- 172.21.15.1:0/1488806564 >> v2:172.21.15.6:3300/0 conn(0x7fc8400b4de0 0x7fc84015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83effd640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc84006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747588479 0 0) 0x7fc84006cf40 con 0x7fc8400b5260 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc82c003520 con 0x7fc8400b5260 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831216545 0 0) 0x7fc82c003520 con 0x7fc8400b5260 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 msgr2=0x7fc84006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc84006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 >> v2:172.21.15.6:3300/0 conn(0x7fc8400b4de0 msgr2=0x7fc84015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 --2- 172.21.15.1:0/1488806564 >> v2:172.21.15.6:3300/0 conn(0x7fc8400b4de0 0x7fc84015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84006d220 con 0x7fc8400b5260 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fc8380019f0 con 0x7fc8400b5260 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 >> v1:172.21.15.1:6789/0 conn(0x7fc8400b5260 legacy=0x7fc840070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc82c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc82c003f30 0x7fc82c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc82c004d40 0x7fc82c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc84006d870 con 0x7fc82c004d40 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c00b9c0 con 0x7fc82c003f30 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c00bb40 con 0x7fc84015c4c0 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc82c004d40 0x7fc82c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83effd640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc82c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.981+0000 7fc83e7fc640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc82c003f30 0x7fc82c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 -- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 msgr2=0x7fc82c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc84015c4c0 0x7fc82c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 -- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc82c003f30 msgr2=0x7fc82c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc82c003f30 0x7fc82c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 -- 172.21.15.1:0/1488806564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c003520 con 0x7fc82c004d40 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.982+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc82c004d40 0x7fc82c0051c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc83800a920 tx=0x7fc82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4313949f68b79e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83800a4c0 con 0x7fc82c004d40 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83802f070 con 0x7fc82c004d40 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc83dffb640 1 -- 172.21.15.1:0/1488806564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83802bd40 con 0x7fc82c004d40 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 -- 172.21.15.1:0/1488806564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc82c004d40 msgr2=0x7fc82c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 --2- 172.21.15.1:0/1488806564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc82c004d40 0x7fc82c0051c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc83800a920 tx=0x7fc82c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc83f7fe640 1 -- 172.21.15.1:0/1488806564 reap_dead start 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 -- 172.21.15.1:0/1488806564 shutdown_connections 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 -- 172.21.15.1:0/1488806564 >> 172.21.15.1:0/1488806564 conn(0x7fc8400cc4d0 msgr2=0x7fc8400cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 -- 172.21.15.1:0/1488806564 shutdown_connections 2024-05-09T06:28:29.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.983+0000 7fc8458da640 1 -- 172.21.15.1:0/1488806564 wait complete. 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.984+0000 7fc8458da640 1 Processor -- start 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 -- start start 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc82c004d40 0x7fc8401f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 0x7fc84015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8400b2dc0 con 0x7fc8400ad890 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc8400b2c40 con 0x7fc84015aba0 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc8458da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc84006dd00 con 0x7fc82c004d40 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc83e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc83e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48108/0 (socket says 172.21.15.1:48108) 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.985+0000 7fc83e7fc640 1 -- 172.21.15.1:0/2026512514 learned_addr learned my addr 172.21.15.1:0/2026512514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83effd640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc82c004d40 0x7fc8401f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 0x7fc84015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 -- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc82c004d40 msgr2=0x7fc8401f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc82c004d40 0x7fc8401f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 -- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 msgr2=0x7fc84015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 0x7fc84015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 -- 172.21.15.1:0/2026512514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8400ae350 con 0x7fc8400ad890 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83f7fe640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 0x7fc84015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:29.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc83e7fc640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc828011960 tx=0x7fc828011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828009590 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82800f9d0 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8400ae630 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82800fdc0 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.986+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8400aebf0 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.987+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8400752b0 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.988+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc828012040 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.988+0000 7fc837fff640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 0x7fc82006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.988+0000 7fc83effd640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 0x7fc82006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.988+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc82808b800 con 0x7fc8400ad890 2024-05-09T06:28:29.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.989+0000 7fc83effd640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 0x7fc82006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc830009bc0 tx=0x7fc830009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:29.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:29.993+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc828019030 con 0x7fc8400ad890 2024-05-09T06:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.174+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8400b0c60 con 0x7fc8400ad890 2024-05-09T06:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.174+0000 7fc837fff640 1 -- 172.21.15.1:0/2026512514 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc828059db0 con 0x7fc8400ad890 2024-05-09T06:28:30.157 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.176+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 msgr2=0x7fc82006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.176+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 0x7fc82006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc830009bc0 tx=0x7fc830009190 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 msgr2=0x7fc8400add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc828011960 tx=0x7fc828011d60 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 shutdown_connections 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc82006be30 0x7fc82006e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc84015aba0 0x7fc84015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8400ad890 0x7fc8400add10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 --2- 172.21.15.1:0/2026512514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc82c004d40 0x7fc8401f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 >> 172.21.15.1:0/2026512514 conn(0x7fc8400cc4d0 msgr2=0x7fc8400af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 shutdown_connections 2024-05-09T06:28:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.177+0000 7fc8458da640 1 -- 172.21.15.1:0/2026512514 wait complete. 2024-05-09T06:28:30.170 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.295+0000 7f8a0b093640 1 Processor -- start 2024-05-09T06:28:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.295+0000 7f8a0b093640 1 -- start start 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a0b093640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a040cd450 0x7f8a0415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a0b093640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a04074660 0x7f8a04160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a0b093640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0415c920 con 0x7f8a041605d0 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a0b093640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0415fc00 con 0x7f8a04074660 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a0b093640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0415cba0 con 0x7f8a040cd450 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a04074660 0x7f8a04160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a04074660 0x7f8a04160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48122/0 (socket says 172.21.15.1:48122) 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 -- 172.21.15.1:0/2058304255 learned_addr learned my addr 172.21.15.1:0/2058304255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a08e2f640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a040cd450 0x7f8a0415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 -- 172.21.15.1:0/2058304255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a040cd450 msgr2=0x7f8a0415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a040cd450 0x7f8a0415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 -- 172.21.15.1:0/2058304255 >> v1:172.21.15.6:6789/0 conn(0x7f8a041605d0 legacy=0x7f8a0415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.296+0000 7f8a03fff640 1 -- 172.21.15.1:0/2058304255 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0415ce80 con 0x7f8a04074660 2024-05-09T06:28:30.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a03fff640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04074660 0x7f8a04160090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c14b13134ab40c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a037fe640 1 -- 172.21.15.1:0/2058304255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0037070 con 0x7f8a04074660 2024-05-09T06:28:30.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a037fe640 1 -- 172.21.15.1:0/2058304255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f0032440 con 0x7f8a04074660 2024-05-09T06:28:30.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a037fe640 1 -- 172.21.15.1:0/2058304255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0032810 con 0x7f8a04074660 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a0b093640 1 -- 172.21.15.1:0/2058304255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04074660 msgr2=0x7f8a04160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.297+0000 7f8a0b093640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04074660 0x7f8a04160090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a210 comp rx=0 tx=0).stop 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 -- 172.21.15.1:0/2058304255 shutdown_connections 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04074660 0x7f8a04160090 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 --2- 172.21.15.1:0/2058304255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a040cd450 0x7f8a0415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 -- 172.21.15.1:0/2058304255 >> 172.21.15.1:0/2058304255 conn(0x7f8a040b1da0 msgr2=0x7f8a040b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 -- 172.21.15.1:0/2058304255 shutdown_connections 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.298+0000 7f8a0b093640 1 -- 172.21.15.1:0/2058304255 wait complete. 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.299+0000 7f8a0b093640 1 Processor -- start 2024-05-09T06:28:30.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.299+0000 7f8a0b093640 1 -- start start 2024-05-09T06:28:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a04074660 0x7f8a04156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a040cd450 0x7f8a040c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0415dcb0 con 0x7f8a041605d0 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8a040cd830 con 0x7f8a040cd450 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a0b093640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0415de30 con 0x7f8a04074660 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48136/0 (socket says 172.21.15.1:48136) 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 -- 172.21.15.1:0/4210554941 learned_addr learned my addr 172.21.15.1:0/4210554941 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a03fff640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a040cd450 0x7f8a040c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a08e2f640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a04074660 0x7f8a04156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 -- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a04074660 msgr2=0x7f8a04156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a04074660 0x7f8a04156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 -- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a040cd450 msgr2=0x7f8a040c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a040cd450 0x7f8a040c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 -- 172.21.15.1:0/4210554941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a041f4140 con 0x7f8a041605d0 2024-05-09T06:28:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.300+0000 7f8a09630640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9a0 tx=0x7f89fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.301+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019070 con 0x7f8a041605d0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.301+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc00cba0 con 0x7f8a041605d0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.301+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a04156f90 con 0x7f8a041605d0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.301+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc014440 con 0x7f8a041605d0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.301+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a04099d20 con 0x7f8a041605d0 2024-05-09T06:28:30.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.302+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a040752b0 con 0x7f8a041605d0 2024-05-09T06:28:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.302+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89fc01c070 con 0x7f8a041605d0 2024-05-09T06:28:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.303+0000 7f8a017fa640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 0x7f89e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.303+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f89fc08ae50 con 0x7f8a041605d0 2024-05-09T06:28:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.303+0000 7f8a08e2f640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 0x7f89e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.303+0000 7f8a08e2f640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 0x7f89e806e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f800a970 tx=0x7f89f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.307+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89fc058e40 con 0x7f8a041605d0 2024-05-09T06:28:30.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.491+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a040b61f0 con 0x7f8a041605d0 2024-05-09T06:28:30.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.492+0000 7f8a017fa640 1 -- 172.21.15.1:0/4210554941 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f89fc05d140 con 0x7f8a041605d0 2024-05-09T06:28:30.474 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 msgr2=0x7f89e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 0x7f89e806e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f800a970 tx=0x7f89f8008040 comp rx=0 tx=0).stop 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 msgr2=0x7f8a040d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9a0 tx=0x7f89fc00be60 comp rx=0 tx=0).stop 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 shutdown_connections 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f89e806be30 0x7f89e806e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041605d0 0x7f8a040d01c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a040cd450 0x7f8a040c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 --2- 172.21.15.1:0/4210554941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a04074660 0x7f8a04156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 >> 172.21.15.1:0/4210554941 conn(0x7f8a040b1da0 msgr2=0x7f8a040ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 shutdown_connections 2024-05-09T06:28:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.494+0000 7f8a0b093640 1 -- 172.21.15.1:0/4210554941 wait complete. 2024-05-09T06:28:30.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:28:30.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.612+0000 7f1250246640 1 Processor -- start 2024-05-09T06:28:30.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.612+0000 7f1250246640 1 -- start start 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f1250246640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1248071fd0 0x7f124806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f1250246640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 0x7f124806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f1250246640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124815b420 con 0x7f1248158a50 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f1250246640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1248158e30 con 0x7f1248071fd0 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f1250246640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1248072480 con 0x7f124815c4a0 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124d7e1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1248071fd0 0x7f124806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124dfe2640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1248158a50 0x7f12480718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51128/0 (socket says 172.21.15.1:51128) 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124dfe2640 1 -- 172.21.15.1:0/2505165916 learned_addr learned my addr 172.21.15.1:0/2505165916 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124e7e3640 1 --2- 172.21.15.1:0/2505165916 >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 0x7f124806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124d7e1640 1 -- 172.21.15.1:0/2505165916 >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 msgr2=0x7f124806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124d7e1640 1 --2- 172.21.15.1:0/2505165916 >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 0x7f124806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124d7e1640 1 -- 172.21.15.1:0/2505165916 >> v1:172.21.15.1:6789/0 conn(0x7f1248158a50 legacy=0x7f12480718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124d7e1640 1 -- 172.21.15.1:0/2505165916 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124815b6e0 con 0x7f1248071fd0 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124cfe0640 1 -- 172.21.15.1:0/2505165916 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949480388 0 0) 0x7f124815b420 con 0x7f1248158a50 2024-05-09T06:28:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.613+0000 7f124e7e3640 1 --2- 172.21.15.1:0/2505165916 >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 0x7f124806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.614+0000 7f124d7e1640 1 --2- 172.21.15.1:0/2505165916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f124806e6d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f124806d050 tx=0x7f123c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e047789f6765f326 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.614+0000 7f124cfe0640 1 -- 172.21.15.1:0/2505165916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c037070 con 0x7f1248071fd0 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.614+0000 7f124cfe0640 1 -- 172.21.15.1:0/2505165916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c032440 con 0x7f1248071fd0 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.614+0000 7f124cfe0640 1 -- 172.21.15.1:0/2505165916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c032810 con 0x7f1248071fd0 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 -- 172.21.15.1:0/2505165916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 msgr2=0x7f124806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 --2- 172.21.15.1:0/2505165916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f124806e6d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f124806d050 tx=0x7f123c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 -- 172.21.15.1:0/2505165916 shutdown_connections 2024-05-09T06:28:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 --2- 172.21.15.1:0/2505165916 >> v2:172.21.15.6:3300/0 conn(0x7f124815c4a0 0x7f124806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 --2- 172.21.15.1:0/2505165916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f124806e6d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 -- 172.21.15.1:0/2505165916 >> 172.21.15.1:0/2505165916 conn(0x7f12480cc4d0 msgr2=0x7f12480cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 -- 172.21.15.1:0/2505165916 shutdown_connections 2024-05-09T06:28:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.615+0000 7f1250246640 1 -- 172.21.15.1:0/2505165916 wait complete. 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.616+0000 7f1250246640 1 Processor -- start 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.616+0000 7f1250246640 1 -- start start 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248158a50 0x7f1248146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f124815c4a0 0x7f1248157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f124815bf80 con 0x7f1248071fd0 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f124815bca0 con 0x7f1248158a50 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f1250246640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1248070ea0 con 0x7f124815c4a0 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48148/0 (socket says 172.21.15.1:48148) 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 -- 172.21.15.1:0/1668843299 learned_addr learned my addr 172.21.15.1:0/1668843299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 -- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f124815c4a0 msgr2=0x7f1248157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f124815c4a0 0x7f1248157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 -- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248158a50 msgr2=0x7f1248146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248158a50 0x7f1248146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 -- 172.21.15.1:0/1668843299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12480add30 con 0x7f1248071fd0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.617+0000 7f124dfe2640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f124000c4d0 tx=0x7f124000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.618+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240018070 con 0x7f1248071fd0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.618+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240007500 con 0x7f1248071fd0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.618+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12480ae010 con 0x7f1248071fd0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.618+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240004a20 con 0x7f1248071fd0 2024-05-09T06:28:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.618+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12480ae5d0 con 0x7f1248071fd0 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.619+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12480752b0 con 0x7f1248071fd0 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.619+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f124001b070 con 0x7f1248071fd0 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.620+0000 7f12467fc640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 0x7f123406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.620+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f124008a5e0 con 0x7f1248071fd0 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.620+0000 7f124d7e1640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 0x7f123406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.620+0000 7f124d7e1640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 0x7f123406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f123c034000 tx=0x7f123c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.624+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1240058b90 con 0x7f1248071fd0 2024-05-09T06:28:30.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.806+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f124806e4f0 con 0x7f1248071fd0 2024-05-09T06:28:30.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.807+0000 7f12467fc640 1 -- 172.21.15.1:0/1668843299 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f1240058490 con 0x7f1248071fd0 2024-05-09T06:28:30.789 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 msgr2=0x7f123406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 0x7f123406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f123c034000 tx=0x7f123c02c000 comp rx=0 tx=0).stop 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 msgr2=0x7f12480aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f124000c4d0 tx=0x7f124000c990 comp rx=0 tx=0).stop 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 shutdown_connections 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.808+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f123406be30 0x7f123406e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f124815c4a0 0x7f1248157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1248158a50 0x7f1248146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 --2- 172.21.15.1:0/1668843299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248071fd0 0x7f12480aedd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 >> 172.21.15.1:0/1668843299 conn(0x7f12480cc4d0 msgr2=0x7f1248069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 shutdown_connections 2024-05-09T06:28:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.809+0000 7f1250246640 1 -- 172.21.15.1:0/1668843299 wait complete. 2024-05-09T06:28:30.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.923+0000 7f1623316640 1 Processor -- start 2024-05-09T06:28:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 -- start start 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f161c161950 0x7f161c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f161c0d4940 0x7f161c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f161c0d4dd0 con 0x7f161c161950 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f161c0b33c0 con 0x7f161c0d4940 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.924+0000 7f1623316640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f161c0b3660 con 0x7f161c0d0f80 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f161c161950 0x7f161c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f161c161950 0x7f161c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48156/0 (socket says 172.21.15.1:48156) 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 -- 172.21.15.1:0/345473205 learned_addr learned my addr 172.21.15.1:0/345473205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16218b3640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16208b1640 1 --2- 172.21.15.1:0/345473205 >> v2:172.21.15.6:3300/0 conn(0x7f161c0d4940 0x7f161c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 -- 172.21.15.1:0/345473205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 msgr2=0x7f161c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 -- 172.21.15.1:0/345473205 >> v2:172.21.15.6:3300/0 conn(0x7f161c0d4940 msgr2=0x7f161c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 --2- 172.21.15.1:0/345473205 >> v2:172.21.15.6:3300/0 conn(0x7f161c0d4940 0x7f161c0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.925+0000 7f16210b2640 1 -- 172.21.15.1:0/345473205 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c0b3940 con 0x7f161c161950 2024-05-09T06:28:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f16210b2640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0d2400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f161c06e3d0 tx=0x7f161802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1242d5a97d51f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1613fff640 1 -- 172.21.15.1:0/345473205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618037070 con 0x7f161c161950 2024-05-09T06:28:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1613fff640 1 -- 172.21.15.1:0/345473205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1618032440 con 0x7f161c161950 2024-05-09T06:28:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1613fff640 1 -- 172.21.15.1:0/345473205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618032810 con 0x7f161c161950 2024-05-09T06:28:30.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 -- 172.21.15.1:0/345473205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 msgr2=0x7f161c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0d2400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f161c06e3d0 tx=0x7f161802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 -- 172.21.15.1:0/345473205 shutdown_connections 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 --2- 172.21.15.1:0/345473205 >> v2:172.21.15.6:3300/0 conn(0x7f161c0d4940 0x7f161c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 --2- 172.21.15.1:0/345473205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0d2400 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.926+0000 7f1623316640 1 -- 172.21.15.1:0/345473205 >> 172.21.15.1:0/345473205 conn(0x7f161c0d3e20 msgr2=0x7f161c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.927+0000 7f1623316640 1 -- 172.21.15.1:0/345473205 shutdown_connections 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.927+0000 7f1623316640 1 -- 172.21.15.1:0/345473205 wait complete. 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 Processor -- start 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 -- start start 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f161c0cda30 0x7f161c0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f161c0d1d80 con 0x7f161c161950 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f161c0d1c00 con 0x7f161c0cda30 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.928+0000 7f1623316640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f161c0d1f00 con 0x7f161c0d0f80 2024-05-09T06:28:30.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48160/0 (socket says 172.21.15.1:48160) 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 -- 172.21.15.1:0/1183062865 learned_addr learned my addr 172.21.15.1:0/1183062865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16210b2640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16218b3640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f161c0cda30 0x7f161c0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 -- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 msgr2=0x7f161c15c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c15c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 -- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f161c0cda30 msgr2=0x7f161c0b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f161c0cda30 0x7f161c0b8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 -- 172.21.15.1:0/1183062865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c0cbb50 con 0x7f161c161950 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f16208b1640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f160c00d800 tx=0x7f160c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c00b6c0 con 0x7f161c161950 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c019070 con 0x7f161c161950 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c0cbe30 con 0x7f161c161950 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.929+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c00ba80 con 0x7f161c161950 2024-05-09T06:28:30.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.930+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c0cc3f0 con 0x7f161c161950 2024-05-09T06:28:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.931+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c0752b0 con 0x7f161c161950 2024-05-09T06:28:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.931+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f160c00bc20 con 0x7f161c161950 2024-05-09T06:28:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.931+0000 7f1611ffb640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 0x7f160006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.932+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f160c08b530 con 0x7f161c161950 2024-05-09T06:28:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.932+0000 7f16210b2640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 0x7f160006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.932+0000 7f16210b2640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 0x7f160006e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1618034000 tx=0x7f161802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:30.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:30.936+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f160c059490 con 0x7f161c161950 2024-05-09T06:28:31.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.159+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f161c0d21e0 con 0x7f161c161950 2024-05-09T06:28:31.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.160+0000 7f1611ffb640 1 -- 172.21.15.1:0/1183062865 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f160c05d790 con 0x7f161c161950 2024-05-09T06:28:31.142 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:31.142 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:28:31.142 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 msgr2=0x7f160006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 0x7f160006e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1618034000 tx=0x7f161802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 msgr2=0x7f161c0b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f160c00d800 tx=0x7f160c00dcc0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 shutdown_connections 2024-05-09T06:28:31.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f160006be30 0x7f160006e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f161c0cda30 0x7f161c0b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f161c161950 0x7f161c0b83d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 --2- 172.21.15.1:0/1183062865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f161c0d0f80 0x7f161c15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 >> 172.21.15.1:0/1183062865 conn(0x7f161c0d3e20 msgr2=0x7f161c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 shutdown_connections 2024-05-09T06:28:31.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.162+0000 7f1623316640 1 -- 172.21.15.1:0/1183062865 wait complete. 2024-05-09T06:28:31.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:31.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.277+0000 7f0b403ee640 1 Processor -- start 2024-05-09T06:28:31.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.277+0000 7f0b403ee640 1 -- start start 2024-05-09T06:28:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b403ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0b38161950 0x7f0b380d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b403ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b380d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b403ee640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b380b33c0 con 0x7f0b380d4940 2024-05-09T06:28:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b403ee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0b380d4dd0 con 0x7f0b38161950 2024-05-09T06:28:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b403ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b380b3660 con 0x7f0b380d0f80 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0b38161950 0x7f0b380d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0b38161950 0x7f0b380d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48172/0 (socket says 172.21.15.1:48172) 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 -- 172.21.15.1:0/3687439415 learned_addr learned my addr 172.21.15.1:0/3687439415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 -- 172.21.15.1:0/3687439415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 msgr2=0x7f0b380d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 --2- 172.21.15.1:0/3687439415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b380d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 -- 172.21.15.1:0/3687439415 >> v1:172.21.15.6:6789/0 conn(0x7f0b380d4940 legacy=0x7f0b380cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.278+0000 7f0b3e18a640 1 -- 172.21.15.1:0/3687439415 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b380b3940 con 0x7f0b38161950 2024-05-09T06:28:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.279+0000 7f0b3e18a640 1 --2- 172.21.15.1:0/3687439415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380d2400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009870 tx=0x7f0b3402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d971fbcc314b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.279+0000 7f0b3d188640 1 -- 172.21.15.1:0/3687439415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34037070 con 0x7f0b38161950 2024-05-09T06:28:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.279+0000 7f0b3d188640 1 -- 172.21.15.1:0/3687439415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34032440 con 0x7f0b38161950 2024-05-09T06:28:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.279+0000 7f0b3d188640 1 -- 172.21.15.1:0/3687439415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032810 con 0x7f0b38161950 2024-05-09T06:28:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 -- 172.21.15.1:0/3687439415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 msgr2=0x7f0b380d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 --2- 172.21.15.1:0/3687439415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380d2400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009870 tx=0x7f0b3402a210 comp rx=0 tx=0).stop 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 -- 172.21.15.1:0/3687439415 shutdown_connections 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 --2- 172.21.15.1:0/3687439415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b380d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 --2- 172.21.15.1:0/3687439415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380d2400 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 -- 172.21.15.1:0/3687439415 >> 172.21.15.1:0/3687439415 conn(0x7f0b380d3e20 msgr2=0x7f0b380d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 -- 172.21.15.1:0/3687439415 shutdown_connections 2024-05-09T06:28:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.280+0000 7f0b403ee640 1 -- 172.21.15.1:0/3687439415 wait complete. 2024-05-09T06:28:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.281+0000 7f0b403ee640 1 Processor -- start 2024-05-09T06:28:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.281+0000 7f0b403ee640 1 -- start start 2024-05-09T06:28:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b38150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 0x7f0b3815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380b52d0 con 0x7f0b38161950 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380b5150 con 0x7f0b380b80e0 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b403ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0b380b5450 con 0x7f0b380d0f80 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48182/0 (socket says 172.21.15.1:48182) 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 -- 172.21.15.1:0/2530757886 learned_addr learned my addr 172.21.15.1:0/2530757886 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 -- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 msgr2=0x7f0b38150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3e18a640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b38150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3e98b640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 0x7f0b3815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b38150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 -- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 msgr2=0x7f0b3815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 0x7f0b3815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3d989640 1 -- 172.21.15.1:0/2530757886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b380b6b00 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.282+0000 7f0b3e98b640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 0x7f0b3815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b3d989640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c410 tx=0x7f0b2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28017070 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b280035f0 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003a20 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b380b6de0 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.283+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b380b73a0 con 0x7f0b38161950 2024-05-09T06:28:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.284+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b380752b0 con 0x7f0b38161950 2024-05-09T06:28:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.284+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b28003bc0 con 0x7f0b38161950 2024-05-09T06:28:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.285+0000 7f0b2effd640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.285+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b2808a580 con 0x7f0b38161950 2024-05-09T06:28:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.285+0000 7f0b3e18a640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.285+0000 7f0b3e18a640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b34034000 tx=0x7f0b3402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.289+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b28058bc0 con 0x7f0b38161950 2024-05-09T06:28:31.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.474+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b380d21e0 con 0x7f0b38161950 2024-05-09T06:28:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.475+0000 7f0b2effd640 1 -- 172.21.15.1:0/2530757886 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f0b280010c0 con 0x7f0b38161950 2024-05-09T06:28:31.457 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:31.458 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 msgr2=0x7f0b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b34034000 tx=0x7f0b3402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 msgr2=0x7f0b380b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c410 tx=0x7f0b2800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 shutdown_connections 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0b1c06be30 0x7f0b1c06e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b380b80e0 0x7f0b3815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38161950 0x7f0b380b7ba0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 --2- 172.21.15.1:0/2530757886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0b380d0f80 0x7f0b38150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 >> 172.21.15.1:0/2530757886 conn(0x7f0b380d3e20 msgr2=0x7f0b380cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 shutdown_connections 2024-05-09T06:28:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.478+0000 7f0b403ee640 1 -- 172.21.15.1:0/2530757886 wait complete. 2024-05-09T06:28:31.471 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:28:31.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:31.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.593+0000 7f1bb16d4640 1 Processor -- start 2024-05-09T06:28:31.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.593+0000 7f1bb16d4640 1 -- start start 2024-05-09T06:28:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1bac1454b0 0x7f1bac0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bac0b2a60 0x7f1bac0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1bac0b5230 con 0x7f1bac1454b0 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1bac06eb40 con 0x7f1bac0b2a60 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bb16d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1bac06ee20 con 0x7f1bac0b4da0 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1bac1454b0 0x7f1bac0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1bac1454b0 0x7f1bac0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48188/0 (socket says 172.21.15.1:48188) 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 -- 172.21.15.1:0/3320448234 learned_addr learned my addr 172.21.15.1:0/3320448234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bab7fe640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baa7fc640 1 --2- 172.21.15.1:0/3320448234 >> v2:172.21.15.6:3300/0 conn(0x7f1bac0b2a60 0x7f1bac0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 -- 172.21.15.1:0/3320448234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 msgr2=0x7f1bac15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 -- 172.21.15.1:0/3320448234 >> v2:172.21.15.6:3300/0 conn(0x7f1bac0b2a60 msgr2=0x7f1bac0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 --2- 172.21.15.1:0/3320448234 >> v2:172.21.15.6:3300/0 conn(0x7f1bac0b2a60 0x7f1bac0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1baaffd640 1 -- 172.21.15.1:0/3320448234 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac06d8e0 con 0x7f1bac1454b0 2024-05-09T06:28:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.594+0000 7f1bab7fe640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.595+0000 7f1baaffd640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac0b4320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1bac04f830 tx=0x7f1ba402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c61a487ab6b8614 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.595+0000 7f1ba9ffb640 1 -- 172.21.15.1:0/3320448234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4037070 con 0x7f1bac1454b0 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.595+0000 7f1ba9ffb640 1 -- 172.21.15.1:0/3320448234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba4032440 con 0x7f1bac1454b0 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.595+0000 7f1ba9ffb640 1 -- 172.21.15.1:0/3320448234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4032810 con 0x7f1bac1454b0 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 -- 172.21.15.1:0/3320448234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 msgr2=0x7f1bac0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac0b4320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1bac04f830 tx=0x7f1ba402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 -- 172.21.15.1:0/3320448234 shutdown_connections 2024-05-09T06:28:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/3320448234 >> v2:172.21.15.6:3300/0 conn(0x7f1bac0b2a60 0x7f1bac0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/3320448234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac0b4320 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 -- 172.21.15.1:0/3320448234 >> 172.21.15.1:0/3320448234 conn(0x7f1bac0cc4b0 msgr2=0x7f1bac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 -- 172.21.15.1:0/3320448234 shutdown_connections 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.596+0000 7f1bb16d4640 1 -- 172.21.15.1:0/3320448234 wait complete. 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.597+0000 7f1bb16d4640 1 Processor -- start 2024-05-09T06:28:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.597+0000 7f1bb16d4640 1 -- start start 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 0x7f1bac0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac06e0e0 con 0x7f1bac1454b0 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac145890 con 0x7f1bac0b2a60 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bb16d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1bac06e260 con 0x7f1bac0b4da0 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48204/0 (socket says 172.21.15.1:48204) 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 -- 172.21.15.1:0/1375290970 learned_addr learned my addr 172.21.15.1:0/1375290970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1baaffd640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 0x7f1bac0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1baa7fc640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 -- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 msgr2=0x7f1bac15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 -- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 msgr2=0x7f1bac0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 0x7f1bac0ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1bab7fe640 1 -- 172.21.15.1:0/1375290970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac0ab0f0 con 0x7f1bac1454b0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.598+0000 7f1baaffd640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 0x7f1bac0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1bab7fe640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fa0 tx=0x7f1b9c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c01a070 con 0x7f1bac1454b0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c003bb0 con 0x7f1bac1454b0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004030 con 0x7f1bac1454b0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac0ab3d0 con 0x7f1bac1454b0 2024-05-09T06:28:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.599+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac0ab990 con 0x7f1bac1454b0 2024-05-09T06:28:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.600+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac0752b0 con 0x7f1bac1454b0 2024-05-09T06:28:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.600+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b9c01d070 con 0x7f1bac1454b0 2024-05-09T06:28:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.601+0000 7f1ba37fe640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 0x7f1b8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.601+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1b9c08b4e0 con 0x7f1bac1454b0 2024-05-09T06:28:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.601+0000 7f1baaffd640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 0x7f1b8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.601+0000 7f1baaffd640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 0x7f1b8c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4034000 tx=0x7f1ba402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.605+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b9c010070 con 0x7f1bac1454b0 2024-05-09T06:28:31.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.830+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bac0b5b00 con 0x7f1bac1454b0 2024-05-09T06:28:31.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.830+0000 7f1ba37fe640 1 -- 172.21.15.1:0/1375290970 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7f1b9c059b20 con 0x7f1bac1454b0 2024-05-09T06:28:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:31.812 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:28:31.812 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 msgr2=0x7f1b8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 0x7f1b8c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4034000 tx=0x7f1ba402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 msgr2=0x7f1bac15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fa0 tx=0x7f1b9c00d560 comp rx=0 tx=0).stop 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 shutdown_connections 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b8c06be30 0x7f1b8c06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bac1454b0 0x7f1bac15b820 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bac0b4da0 0x7f1bac15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 --2- 172.21.15.1:0/1375290970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac0b2a60 0x7f1bac0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 >> 172.21.15.1:0/1375290970 conn(0x7f1bac0cc4b0 msgr2=0x7f1bac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 shutdown_connections 2024-05-09T06:28:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.832+0000 7f1bb16d4640 1 -- 172.21.15.1:0/1375290970 wait complete. 2024-05-09T06:28:31.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:31.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.947+0000 7efd018bd640 1 Processor -- start 2024-05-09T06:28:31.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.948+0000 7efd018bd640 1 -- start start 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.948+0000 7efd018bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 0x7efcfc15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.948+0000 7efd018bd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc099930 con 0x7efcfc099b30 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.948+0000 7efd018bd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc162930 con 0x7efcfc0c6860 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.948+0000 7efd018bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc0b1e20 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfa7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7efcfc099b30 0x7efcfc15f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51132/0 (socket says 172.21.15.1:51132) 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfa7fc640 1 -- 172.21.15.1:0/3800212802 learned_addr learned my addr 172.21.15.1:0/3800212802 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfb7fe640 1 --2- 172.21.15.1:0/3800212802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 0x7efcfc15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239967869 0 0) 0x7efcf0001110 con 0x7efcfc099b30 2024-05-09T06:28:31.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efce4003520 con 0x7efcfc099b30 2024-05-09T06:28:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282377658 0 0) 0x7efcfc162930 con 0x7efcfc0c6860 2024-05-09T06:28:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfb7fe640 1 -- 172.21.15.1:0/3800212802 >> v1:172.21.15.6:6789/0 conn(0x7efcfc0c6860 legacy=0x7efcfc0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfb7fe640 1 -- 172.21.15.1:0/3800212802 >> v1:172.21.15.1:6789/0 conn(0x7efcfc099b30 legacy=0x7efcfc15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.949+0000 7efcfb7fe640 1 -- 172.21.15.1:0/3800212802 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc162c10 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efcfb7fe640 1 --2- 172.21.15.1:0/3800212802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 0x7efcfc15e4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009dc0 tx=0x7efcf402a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959e39b73747fa8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4035020 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf4007540 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efcf9ffb640 1 -- 172.21.15.1:0/3800212802 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00017c0 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efd018bd640 1 -- 172.21.15.1:0/3800212802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 msgr2=0x7efcfc15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.950+0000 7efd018bd640 1 --2- 172.21.15.1:0/3800212802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 0x7efcfc15e4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009dc0 tx=0x7efcf402a5b0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.951+0000 7efd018bd640 1 -- 172.21.15.1:0/3800212802 shutdown_connections 2024-05-09T06:28:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.951+0000 7efd018bd640 1 --2- 172.21.15.1:0/3800212802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0b2aa0 0x7efcfc15e4b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.951+0000 7efd018bd640 1 -- 172.21.15.1:0/3800212802 >> 172.21.15.1:0/3800212802 conn(0x7efcfc0b1fb0 msgr2=0x7efcfc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.951+0000 7efd018bd640 1 -- 172.21.15.1:0/3800212802 shutdown_connections 2024-05-09T06:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.951+0000 7efd018bd640 1 -- 172.21.15.1:0/3800212802 wait complete. 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.952+0000 7efd018bd640 1 Processor -- start 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.952+0000 7efd018bd640 1 -- start start 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0b2aa0 0x7efcfc1f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0c6860 0x7efcfc0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc15fa30 con 0x7efcfc099b30 2024-05-09T06:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc15ee90 con 0x7efcfc0b2aa0 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efd018bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc0cdf50 con 0x7efcfc0c6860 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48206/0 (socket says 172.21.15.1:48206) 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 -- 172.21.15.1:0/3382748154 learned_addr learned my addr 172.21.15.1:0/3382748154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfa7fc640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0b2aa0 0x7efcfc1f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfb7fe640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0c6860 0x7efcfc0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 -- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0c6860 msgr2=0x7efcfc0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0c6860 0x7efcfc0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 -- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0b2aa0 msgr2=0x7efcfc1f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0b2aa0 0x7efcfc1f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 -- 172.21.15.1:0/3382748154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc1f8040 con 0x7efcfc099b30 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efcfaffd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c520 tx=0x7efcec00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.953+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec017070 con 0x7efcfc099b30 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.954+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc1f8320 con 0x7efcfc099b30 2024-05-09T06:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.954+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec0035f0 con 0x7efcfc099b30 2024-05-09T06:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.954+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec003a00 con 0x7efcfc099b30 2024-05-09T06:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.954+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc0b3260 con 0x7efcfc099b30 2024-05-09T06:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.954+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc0b3460 con 0x7efcfc099b30 2024-05-09T06:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.955+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcec01a070 con 0x7efcfc099b30 2024-05-09T06:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.955+0000 7efce3fff640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 0x7efce806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.956+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efcec08a5d0 con 0x7efcfc099b30 2024-05-09T06:28:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.956+0000 7efcfa7fc640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 0x7efce806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.956+0000 7efcfa7fc640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 0x7efce806e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcfc162010 tx=0x7efcf0001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:31.960+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcfc0b3460 con 0x7efcfc099b30 2024-05-09T06:28:32.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.185+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcfc0752b0 con 0x7efcfc099b30 2024-05-09T06:28:32.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.186+0000 7efce3fff640 1 -- 172.21.15.1:0/3382748154 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5156 (secure 0 0 0) 0x7efcec058540 con 0x7efcfc099b30 2024-05-09T06:28:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:32.168 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/31988955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":31988955},{"type":"v1","addr":"172.21.15.6:6835","nonce":31988955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1586292254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1586292254},{"type":"v1","addr":"172.21.15.1:6836","nonce":1586292254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1992701426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":1992701426},{"type":"v1","addr":"172.21.15.1:6837","nonce":1992701426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:27:20.997818+0000","modified":"2024-05-09T06:27:43.117905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/278895687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":278895687},{"type":"v1","addr":"172.21.15.6:6834","nonce":278895687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-05-09T06:28:32.168 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-05-09T06:28:32.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 msgr2=0x7efce806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 0x7efce806e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcfc162010 tx=0x7efcf0001e00 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 msgr2=0x7efcfc0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c520 tx=0x7efcec00c9e0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 shutdown_connections 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efce806be30 0x7efce806e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcfc0c6860 0x7efcfc0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0b2aa0 0x7efcfc1f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 --2- 172.21.15.1:0/3382748154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcfc099b30 0x7efcfc0cd430 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 >> 172.21.15.1:0/3382748154 conn(0x7efcfc0b1fb0 msgr2=0x7efcfc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 shutdown_connections 2024-05-09T06:28:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.188+0000 7efd018bd640 1 -- 172.21.15.1:0/3382748154 wait complete. 2024-05-09T06:28:32.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.304+0000 7f53f3aba640 1 Processor -- start 2024-05-09T06:28:32.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.304+0000 7f53f3aba640 1 -- start start 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f3aba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f53ec071ce0 0x7f53ec0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f3aba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53ec15ece0 0x7f53ec15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f3aba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53ec15f630 con 0x7f53ec0b0080 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f3aba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f53ec0d04a0 con 0x7f53ec15ece0 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f3aba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f53ec0722c0 con 0x7f53ec071ce0 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53ec15ece0 0x7f53ec15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f53ec15ece0 0x7f53ec15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48222/0 (socket says 172.21.15.1:48222) 2024-05-09T06:28:32.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 -- 172.21.15.1:0/717937334 learned_addr learned my addr 172.21.15.1:0/717937334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f1055640 1 --2- 172.21.15.1:0/717937334 >> v2:172.21.15.6:3300/0 conn(0x7f53ec071ce0 0x7f53ec0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 -- 172.21.15.1:0/717937334 >> v2:172.21.15.6:3300/0 conn(0x7f53ec071ce0 msgr2=0x7f53ec0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 --2- 172.21.15.1:0/717937334 >> v2:172.21.15.6:3300/0 conn(0x7f53ec071ce0 0x7f53ec0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 -- 172.21.15.1:0/717937334 >> v1:172.21.15.6:6789/0 conn(0x7f53ec0b0080 legacy=0x7f53ec15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.305+0000 7f53f2057640 1 -- 172.21.15.1:0/717937334 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec0725a0 con 0x7f53ec15ece0 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f2057640 1 --2- 172.21.15.1:0/717937334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec15ece0 0x7f53ec15f0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53ec074b60 tx=0x7f53e402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7419b438b6e3bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f0854640 1 -- 172.21.15.1:0/717937334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4036070 con 0x7f53ec15ece0 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f0854640 1 -- 172.21.15.1:0/717937334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4031600 con 0x7f53ec15ece0 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f0854640 1 -- 172.21.15.1:0/717937334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40319d0 con 0x7f53ec15ece0 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f3aba640 1 -- 172.21.15.1:0/717937334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec15ece0 msgr2=0x7f53ec15f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.306+0000 7f53f3aba640 1 --2- 172.21.15.1:0/717937334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec15ece0 0x7f53ec15f0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53ec074b60 tx=0x7f53e402a730 comp rx=0 tx=0).stop 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 -- 172.21.15.1:0/717937334 shutdown_connections 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 --2- 172.21.15.1:0/717937334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec15ece0 0x7f53ec15f0f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 --2- 172.21.15.1:0/717937334 >> v2:172.21.15.6:3300/0 conn(0x7f53ec071ce0 0x7f53ec0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 -- 172.21.15.1:0/717937334 >> 172.21.15.1:0/717937334 conn(0x7f53ec0d0b50 msgr2=0x7f53ec0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 -- 172.21.15.1:0/717937334 shutdown_connections 2024-05-09T06:28:32.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.307+0000 7f53f3aba640 1 -- 172.21.15.1:0/717937334 wait complete. 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.308+0000 7f53f3aba640 1 Processor -- start 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.308+0000 7f53f3aba640 1 -- start start 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53ec071ce0 0x7f53ec1e9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 0x7f53ec1f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec15e7c0 con 0x7f53ec0b0080 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec072b60 con 0x7f53ec15ece0 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f3aba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f53ec15ff40 con 0x7f53ec071ce0 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48224/0 (socket says 172.21.15.1:48224) 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 -- 172.21.15.1:0/2179199907 learned_addr learned my addr 172.21.15.1:0/2179199907 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f2057640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 0x7f53ec1f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1856640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53ec071ce0 0x7f53ec1e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 -- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53ec071ce0 msgr2=0x7f53ec1e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53ec071ce0 0x7f53ec1e9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 -- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 msgr2=0x7f53ec1f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 0x7f53ec1f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 -- 172.21.15.1:0/2179199907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1f73b0 con 0x7f53ec0b0080 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f2057640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 0x7f53ec1f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53f1055640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e9b0 tx=0x7f53dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc017070 con 0x7f53ec0b0080 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.309+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc00c9a0 con 0x7f53ec0b0080 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.310+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00cd90 con 0x7f53ec0b0080 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.310+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1f48f0 con 0x7f53ec0b0080 2024-05-09T06:28:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.310+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1f7670 con 0x7f53ec0b0080 2024-05-09T06:28:32.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.311+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec1f4360 con 0x7f53ec0b0080 2024-05-09T06:28:32.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.311+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53dc01a070 con 0x7f53ec0b0080 2024-05-09T06:28:32.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.312+0000 7f53e27fc640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 0x7f53d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.312+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f53dc08b1a0 con 0x7f53ec0b0080 2024-05-09T06:28:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.312+0000 7f53f1856640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 0x7f53d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.312+0000 7f53f1856640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 0x7f53d006e320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53e8001780 tx=0x7f53e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.316+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53dc059100 con 0x7f53ec0b0080 2024-05-09T06:28:32.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.498+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53ec0752b0 con 0x7f53ec0b0080 2024-05-09T06:28:32.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.499+0000 7f53e27fc640 1 -- 172.21.15.1:0/2179199907 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18770 (secure 0 0 0) 0x7f53dc05d400 con 0x7f53ec0b0080 2024-05-09T06:28:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:32.482 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:02.136372+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:27:19.164805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:27:19.974104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T06:27:23.679732+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:32.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 msgr2=0x7f53d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 0x7f53d006e320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53e8001780 tx=0x7f53e8009040 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 msgr2=0x7f53ec1f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e9b0 tx=0x7f53dc00ee70 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 shutdown_connections 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f53d006be60 0x7f53d006e320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec15ece0 0x7f53ec1f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53ec0b0080 0x7f53ec1f3dc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 --2- 172.21.15.1:0/2179199907 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f53ec071ce0 0x7f53ec1e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.501+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 >> 172.21.15.1:0/2179199907 conn(0x7f53ec0d0b50 msgr2=0x7f53ec0d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.502+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 shutdown_connections 2024-05-09T06:28:32.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.502+0000 7f53f3aba640 1 -- 172.21.15.1:0/2179199907 wait complete. 2024-05-09T06:28:32.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:28:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.617+0000 7ff1eca4c640 1 Processor -- start 2024-05-09T06:28:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.617+0000 7ff1eca4c640 1 -- start start 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.618+0000 7ff1eca4c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1e80d2b10 0x7ff1e80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.618+0000 7ff1eca4c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e80b33c0 con 0x7ff1e80d4940 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.618+0000 7ff1eca4c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e80b3660 con 0x7ff1e8161950 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.618+0000 7ff1eca4c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff1e80d4dd0 con 0x7ff1e80d2b10 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1e80d2b10 0x7ff1e80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e5d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff1e80d4940 0x7ff1e80b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51136/0 (socket says 172.21.15.1:51136) 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e5d9b640 1 -- 172.21.15.1:0/1377431981 learned_addr learned my addr 172.21.15.1:0/1377431981 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e6d9d640 1 -- 172.21.15.1:0/1377431981 >> v1:172.21.15.6:6789/0 conn(0x7ff1e8161950 legacy=0x7ff1e80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e6d9d640 1 -- 172.21.15.1:0/1377431981 >> v1:172.21.15.1:6789/0 conn(0x7ff1e80d4940 legacy=0x7ff1e80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e6d9d640 1 -- 172.21.15.1:0/1377431981 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e80b3940 con 0x7ff1e80d2b10 2024-05-09T06:28:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.619+0000 7ff1e559a640 1 -- 172.21.15.1:0/1377431981 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158800799 0 0) 0x7ff1e80b33c0 con 0x7ff1e80d4940 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1e6d9d640 1 --2- 172.21.15.1:0/1377431981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e80b4850 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80097a0 tx=0x7ff1d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6de3b8874b6c750 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1e559a640 1 -- 172.21.15.1:0/1377431981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8037070 con 0x7ff1e80d2b10 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1e559a640 1 -- 172.21.15.1:0/1377431981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d8032440 con 0x7ff1e80d2b10 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1e559a640 1 -- 172.21.15.1:0/1377431981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8032810 con 0x7ff1e80d2b10 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1eca4c640 1 -- 172.21.15.1:0/1377431981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 msgr2=0x7ff1e80b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/1377431981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e80b4850 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80097a0 tx=0x7ff1d802a230 comp rx=0 tx=0).stop 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1eca4c640 1 -- 172.21.15.1:0/1377431981 shutdown_connections 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/1377431981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e80b4850 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.620+0000 7ff1eca4c640 1 -- 172.21.15.1:0/1377431981 >> 172.21.15.1:0/1377431981 conn(0x7ff1e80d3e20 msgr2=0x7ff1e80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.621+0000 7ff1eca4c640 1 -- 172.21.15.1:0/1377431981 shutdown_connections 2024-05-09T06:28:32.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.621+0000 7ff1eca4c640 1 -- 172.21.15.1:0/1377431981 wait complete. 2024-05-09T06:28:32.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.622+0000 7ff1eca4c640 1 Processor -- start 2024-05-09T06:28:32.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.622+0000 7ff1eca4c640 1 -- start start 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 0x7ff1e809e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e80b86d0 0x7ff1e80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e80d1100 con 0x7ff1e80d2b10 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e80d0f80 con 0x7ff1e80b86d0 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e80d1c00 con 0x7ff1e8161950 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48234/0 (socket says 172.21.15.1:48234) 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 -- 172.21.15.1:0/3207458699 learned_addr learned my addr 172.21.15.1:0/3207458699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:32.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e5d9b640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 0x7ff1e809e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e6d9d640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e80b86d0 0x7ff1e80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 -- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 msgr2=0x7ff1e809e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 0x7ff1e809e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 -- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e80b86d0 msgr2=0x7ff1e80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e80b86d0 0x7ff1e80b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 -- 172.21.15.1:0/3207458699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e80b9270 con 0x7ff1e80d2b10 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e5d9b640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 0x7ff1e809e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1e659c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c710 tx=0x7ff1e000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0003b90 con 0x7ff1e80d2b10 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0003d30 con 0x7ff1e80d2b10 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e80cb090 con 0x7ff1e80d2b10 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000aa90 con 0x7ff1e80d2b10 2024-05-09T06:28:32.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.623+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e80cb650 con 0x7ff1e80d2b10 2024-05-09T06:28:32.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.625+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e80752b0 con 0x7ff1e80d2b10 2024-05-09T06:28:32.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.625+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1e000ac30 con 0x7ff1e80d2b10 2024-05-09T06:28:32.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.625+0000 7ff1deffd640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 0x7ff1c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.625+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff1e008b0a0 con 0x7ff1e80d2b10 2024-05-09T06:28:32.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.625+0000 7ff1e5d9b640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 0x7ff1c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.626+0000 7ff1e5d9b640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 0x7ff1c806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80cf4d0 tx=0x7ff1d0007c50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.630+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1e000ce20 con 0x7ff1e80d2b10 2024-05-09T06:28:32.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.858+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1e80ca970 con 0x7ff1e80d2b10 2024-05-09T06:28:32.840 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:28:32.858+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:28:32.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.860+0000 7ff1deffd640 1 -- 172.21.15.1:0/3207458699 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7ff1e00596e0 con 0x7ff1e80d2b10 2024-05-09T06:28:32.842 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 msgr2=0x7ff1c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 0x7ff1c806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80cf4d0 tx=0x7ff1d0007c50 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 msgr2=0x7ff1e815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c710 tx=0x7ff1e000cbd0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 shutdown_connections 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1c806be30 0x7ff1c806e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1e80b86d0 0x7ff1e80b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.862+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1e8161950 0x7ff1e809e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.863+0000 7ff1eca4c640 1 --2- 172.21.15.1:0/3207458699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1e80d2b10 0x7ff1e815c560 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.863+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 >> 172.21.15.1:0/3207458699 conn(0x7ff1e80d3e20 msgr2=0x7ff1e80b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.863+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 shutdown_connections 2024-05-09T06:28:32.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.863+0000 7ff1eca4c640 1 -- 172.21.15.1:0/3207458699 wait complete. 2024-05-09T06:28:32.848 INFO:tasks.ceph.mds.b.smithi006.stderr: -4755> 2024-05-09T06:27:25.613+0000 7fd8a620e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:28:32.856 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:28:32.881 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:28:32.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.980+0000 7fea6f721640 1 Processor -- start 2024-05-09T06:28:32.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.980+0000 7fea6f721640 1 -- start start 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6f721640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea68159b90 0x7fea6815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6f721640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea68072030 con 0x7fea680d1640 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6f721640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea68072310 con 0x7fea6806b8f0 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6f721640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fea68072cb0 con 0x7fea68159b90 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6ccbc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fea680d1640 0x7fea6806b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51140/0 (socket says 172.21.15.1:51140) 2024-05-09T06:28:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6ccbc640 1 -- 172.21.15.1:0/420557774 learned_addr learned my addr 172.21.15.1:0/420557774 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> v2:172.21.15.6:3300/0 conn(0x7fea68159b90 0x7fea6815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252275269 0 0) 0x7fea68072030 con 0x7fea680d1640 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea54003520 con 0x7fea680d1640 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525796006 0 0) 0x7fea68072310 con 0x7fea6806b8f0 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea68072030 con 0x7fea6806b8f0 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3532234140 0 0) 0x7fea54003520 con 0x7fea680d1640 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 >> v2:172.21.15.6:3300/0 conn(0x7fea68159b90 msgr2=0x7fea6815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 --2- 172.21.15.1:0/420557774 >> v2:172.21.15.6:3300/0 conn(0x7fea68159b90 0x7fea6815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 >> v1:172.21.15.6:6789/0 conn(0x7fea6806b8f0 legacy=0x7fea6806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.981+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea680725f0 con 0x7fea680d1640 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fea58001670 con 0x7fea680d1640 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> v2:172.21.15.6:3300/0 conn(0x7fea68159b90 0x7fea6815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 >> v1:172.21.15.1:6789/0 conn(0x7fea680d1640 legacy=0x7fea6806b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 0x7fea54003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54004800 0x7fea54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea54005210 0x7fea5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea68072cb0 con 0x7fea54003b90 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fea5400c140 con 0x7fea54005210 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fea5400c340 con 0x7fea54004800 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 0x7fea54003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6ccbc640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54004800 0x7fea54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6dcbe640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea54005210 0x7fea5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 -- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54004800 msgr2=0x7fea54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54004800 0x7fea54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 -- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea54005210 msgr2=0x7fea5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea54005210 0x7fea5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6dcbe640 1 -- 172.21.15.1:0/420557774 reap_dead start 2024-05-09T06:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.982+0000 7fea6d4bd640 1 -- 172.21.15.1:0/420557774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea54003520 con 0x7fea54003b90 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 0x7fea54003f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea64009c90 tx=0x7fea54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531403e841ca7686 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64036070 con 0x7fea54003b90 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea64031400 con 0x7fea54003b90 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea5ffff640 1 -- 172.21.15.1:0/420557774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64009860 con 0x7fea54003b90 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6f721640 1 -- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 msgr2=0x7fea54003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6f721640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 0x7fea54003f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea64009c90 tx=0x7fea54005830 comp rx=0 tx=0).stop 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6f721640 1 -- 172.21.15.1:0/420557774 shutdown_connections 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6f721640 1 --2- 172.21.15.1:0/420557774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea54003b90 0x7fea54003f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.983+0000 7fea6f721640 1 -- 172.21.15.1:0/420557774 >> 172.21.15.1:0/420557774 conn(0x7fea680d0b50 msgr2=0x7fea680d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.984+0000 7fea6f721640 1 -- 172.21.15.1:0/420557774 shutdown_connections 2024-05-09T06:28:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.984+0000 7fea6f721640 1 -- 172.21.15.1:0/420557774 wait complete. 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.985+0000 7fea6f721640 1 Processor -- start 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.985+0000 7fea6f721640 1 -- start start 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54003b90 0x7fea680c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea680c38d0 0x7fea680cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea6815d580 con 0x7fea680a1520 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fea6806c150 con 0x7fea680c38d0 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fea68071ce0 con 0x7fea54003b90 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48248/0 (socket says 172.21.15.1:48248) 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 -- 172.21.15.1:0/4225066897 learned_addr learned my addr 172.21.15.1:0/4225066897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54003b90 0x7fea680c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6dcbe640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea680c38d0 0x7fea680cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 -- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54003b90 msgr2=0x7fea680c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54003b90 0x7fea680c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 -- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea680c38d0 msgr2=0x7fea680cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea680c38d0 0x7fea680cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 -- 172.21.15.1:0/4225066897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea680ce1e0 con 0x7fea680a1520 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6ccbc640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fea58004e30 tx=0x7fea58007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800f7b0 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea58018070 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800fb70 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.986+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea680cf260 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.987+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6806b1e0 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.988+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea680752b0 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.988+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea5800fd10 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.989+0000 7fea5dffb640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 0x7fea4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.989+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fea5808af40 con 0x7fea680a1520 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.989+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 0x7fea4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.989+0000 7fea6d4bd640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 0x7fea4c06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea64033000 tx=0x7fea64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:32.993+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea58058ed0 con 0x7fea680a1520 2024-05-09T06:28:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.216+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fea68072e30 con 0x7fea680a1520 2024-05-09T06:28:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.865+0000 7fea5dffb640 1 -- 172.21.15.1:0/4225066897 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fea5805d1d0 con 0x7fea680a1520 2024-05-09T06:28:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 msgr2=0x7fea4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 0x7fea4c06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea64033000 tx=0x7fea64008030 comp rx=0 tx=0).stop 2024-05-09T06:28:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 msgr2=0x7fea680c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fea58004e30 tx=0x7fea58007430 comp rx=0 tx=0).stop 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 shutdown_connections 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fea4c06be30 0x7fea4c06e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea680c38d0 0x7fea680cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea680a1520 0x7fea680c3390 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 --2- 172.21.15.1:0/4225066897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea54003b90 0x7fea680c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 >> 172.21.15.1:0/4225066897 conn(0x7fea680d0b50 msgr2=0x7fea680c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 shutdown_connections 2024-05-09T06:28:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.867+0000 7fea6f721640 1 -- 172.21.15.1:0/4225066897 wait complete. 2024-05-09T06:28:33.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:28:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.989+0000 7ff832b42640 1 Processor -- start 2024-05-09T06:28:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.989+0000 7ff832b42640 1 -- start start 2024-05-09T06:28:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff832b42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c15e5f0 0x7ff82c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff832b42640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff82c1628f0 0x7ff82c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff832b42640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82c0cb0e0 con 0x7ff82c15e210 2024-05-09T06:28:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff832b42640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c0cca50 con 0x7ff82c1628f0 2024-05-09T06:28:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff832b42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0b17c0 con 0x7ff82c15e5f0 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff82c1628f0 0x7ff82c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff82c1628f0 0x7ff82c0ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48256/0 (socket says 172.21.15.1:48256) 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 -- 172.21.15.1:0/1104587679 learned_addr learned my addr 172.21.15.1:0/1104587679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff823fff640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c15e5f0 0x7ff82c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 -- 172.21.15.1:0/1104587679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c15e5f0 msgr2=0x7ff82c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c15e5f0 0x7ff82c0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 -- 172.21.15.1:0/1104587679 >> v1:172.21.15.6:6789/0 conn(0x7ff82c15e210 legacy=0x7ff82c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.990+0000 7ff8310df640 1 -- 172.21.15.1:0/1104587679 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0b1aa0 con 0x7ff82c1628f0 2024-05-09T06:28:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.991+0000 7ff8310df640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c1628f0 0x7ff82c0ccee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff824009e50 tx=0x7ff82402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902dfd0bb5ca6742 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.991+0000 7ff8237fe640 1 -- 172.21.15.1:0/1104587679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824036070 con 0x7ff82c1628f0 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.991+0000 7ff8237fe640 1 -- 172.21.15.1:0/1104587679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824031600 con 0x7ff82c1628f0 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.991+0000 7ff8237fe640 1 -- 172.21.15.1:0/1104587679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8240319d0 con 0x7ff82c1628f0 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 -- 172.21.15.1:0/1104587679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c1628f0 msgr2=0x7ff82c0ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c1628f0 0x7ff82c0ccee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff824009e50 tx=0x7ff82402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 -- 172.21.15.1:0/1104587679 shutdown_connections 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c1628f0 0x7ff82c0ccee0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 --2- 172.21.15.1:0/1104587679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c15e5f0 0x7ff82c0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 -- 172.21.15.1:0/1104587679 >> 172.21.15.1:0/1104587679 conn(0x7ff82c071a50 msgr2=0x7ff82c0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 -- 172.21.15.1:0/1104587679 shutdown_connections 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.992+0000 7ff832b42640 1 -- 172.21.15.1:0/1104587679 wait complete. 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.993+0000 7ff832b42640 1 Processor -- start 2024-05-09T06:28:33.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.993+0000 7ff832b42640 1 -- start start 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff82c1628f0 0x7ff82c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c0c64c0 0x7ff82c0ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0cd420 con 0x7ff82c15e210 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c162d70 con 0x7ff82c1628f0 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff832b42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0cd5a0 con 0x7ff82c0c64c0 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48270/0 (socket says 172.21.15.1:48270) 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 -- 172.21.15.1:0/3376377506 learned_addr learned my addr 172.21.15.1:0/3376377506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 -- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c0c64c0 msgr2=0x7ff82c0ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c0c64c0 0x7ff82c0ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 -- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff82c1628f0 msgr2=0x7ff82c0c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff82c1628f0 0x7ff82c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 -- 172.21.15.1:0/3376377506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c161370 con 0x7ff82c15e210 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8308de640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c410 tx=0x7ff82800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff82c15e210 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.994+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8280035f0 con 0x7ff82c15e210 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.995+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c0c9010 con 0x7ff82c15e210 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.995+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828003a20 con 0x7ff82c15e210 2024-05-09T06:28:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.995+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c15d630 con 0x7ff82c15e210 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.995+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c075320 con 0x7ff82c15e210 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.996+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff82801a070 con 0x7ff82c15e210 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.996+0000 7ff8217fa640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 0x7ff81806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.997+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7ff82808a740 con 0x7ff82c15e210 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.997+0000 7ff823fff640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 0x7ff81806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:33.997+0000 7ff823fff640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 0x7ff81806e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff81c006fd0 tx=0x7ff81c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.001+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff828058c90 con 0x7ff82c15e210 2024-05-09T06:28:34.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.183+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff82c0b6300 con 0x7ff82c15e210 2024-05-09T06:28:34.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.866+0000 7ff8217fa640 1 -- 172.21.15.1:0/3376377506 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7ff828058590 con 0x7ff82c15e210 2024-05-09T06:28:34.849 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 msgr2=0x7ff81806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 0x7ff81806e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff81c006fd0 tx=0x7ff81c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 msgr2=0x7ff82c1f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c410 tx=0x7ff82800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 shutdown_connections 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff81806be30 0x7ff81806e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff82c0c64c0 0x7ff82c0ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff82c1628f0 0x7ff82c0c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 --2- 172.21.15.1:0/3376377506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82c15e210 0x7ff82c1f6c20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.870+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 >> 172.21.15.1:0/3376377506 conn(0x7ff82c071a50 msgr2=0x7ff82c06d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:34.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.871+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 shutdown_connections 2024-05-09T06:28:34.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.871+0000 7ff832b42640 1 -- 172.21.15.1:0/3376377506 wait complete. 2024-05-09T06:28:34.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:28:34.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.999+0000 7f34b5c10640 1 Processor -- start 2024-05-09T06:28:34.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:34.999+0000 7f34b5c10640 1 -- start start 2024-05-09T06:28:34.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34b5c10640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34b006f920 0x7f34b00d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34b5c10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34b5c10640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b00d0110 con 0x7f34b0162770 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34b5c10640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f34b00cfee0 con 0x7f34b006f920 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34b5c10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f34b0162c50 con 0x7f34b006f120 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34b006f920 0x7f34b00d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34b006f920 0x7f34b00d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48278/0 (socket says 172.21.15.1:48278) 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 -- 172.21.15.1:0/2013063342 learned_addr learned my addr 172.21.15.1:0/2013063342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 -- 172.21.15.1:0/2013063342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 msgr2=0x7f34b00cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 --2- 172.21.15.1:0/2013063342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 -- 172.21.15.1:0/2013063342 >> v1:172.21.15.6:6789/0 conn(0x7f34b0162770 legacy=0x7f34b006fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:34.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.000+0000 7f34af7fe640 1 -- 172.21.15.1:0/2013063342 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b00d03f0 con 0x7f34b006f920 2024-05-09T06:28:34.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.001+0000 7f34af7fe640 1 --2- 172.21.15.1:0/2013063342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00d0ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009870 tx=0x7f34a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12049ca3117d0440 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.001+0000 7f34aeffd640 1 -- 172.21.15.1:0/2013063342 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0037070 con 0x7f34b006f920 2024-05-09T06:28:34.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.001+0000 7f34aeffd640 1 -- 172.21.15.1:0/2013063342 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0032440 con 0x7f34b006f920 2024-05-09T06:28:34.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.001+0000 7f34aeffd640 1 -- 172.21.15.1:0/2013063342 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0032810 con 0x7f34b006f920 2024-05-09T06:28:34.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 -- 172.21.15.1:0/2013063342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 msgr2=0x7f34b00d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 --2- 172.21.15.1:0/2013063342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00d0ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009870 tx=0x7f34a002a210 comp rx=0 tx=0).stop 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 -- 172.21.15.1:0/2013063342 shutdown_connections 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 --2- 172.21.15.1:0/2013063342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b00cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 --2- 172.21.15.1:0/2013063342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00d0ed0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 -- 172.21.15.1:0/2013063342 >> 172.21.15.1:0/2013063342 conn(0x7f34b00d1d10 msgr2=0x7f34b00d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 -- 172.21.15.1:0/2013063342 shutdown_connections 2024-05-09T06:28:34.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.002+0000 7f34b5c10640 1 -- 172.21.15.1:0/2013063342 wait complete. 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.003+0000 7f34b5c10640 1 Processor -- start 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.003+0000 7f34b5c10640 1 -- start start 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34b0162770 0x7f34b00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34b015e590 con 0x7f34b006f920 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00d09b0 con 0x7f34b0162770 2024-05-09T06:28:34.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34b5c10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f34b00d2980 con 0x7f34b006f120 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48280/0 (socket says 172.21.15.1:48280) 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 -- 172.21.15.1:0/3138633775 learned_addr learned my addr 172.21.15.1:0/3138633775 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34af7fe640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 -- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 msgr2=0x7f34b014f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b014f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 -- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34b0162770 msgr2=0x7f34b00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34b0162770 0x7f34b00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 -- 172.21.15.1:0/3138633775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b01573b0 con 0x7f34b006f920 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34af7fe640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b014f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.004+0000 7f34a7fff640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f349800c410 tx=0x7f349800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.005+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498017070 con 0x7f34b006f920 2024-05-09T06:28:34.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.005+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34980035f0 con 0x7f34b006f920 2024-05-09T06:28:34.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.005+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b0157690 con 0x7f34b006f920 2024-05-09T06:28:34.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.005+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498003a20 con 0x7f34b006f920 2024-05-09T06:28:34.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.005+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b0157c50 con 0x7f34b006f920 2024-05-09T06:28:34.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.006+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b0075320 con 0x7f34b006f920 2024-05-09T06:28:34.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.006+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f349801a070 con 0x7f34b006f920 2024-05-09T06:28:34.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.007+0000 7f34acff9640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 0x7f349006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.007+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f349808a660 con 0x7f34b006f920 2024-05-09T06:28:34.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.007+0000 7f34af7fe640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 0x7f349006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.007+0000 7f34af7fe640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 0x7f349006e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.012+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3498058cd0 con 0x7f34b006f920 2024-05-09T06:28:35.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.199+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f34b006f700 con 0x7f34b006f920 2024-05-09T06:28:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.874+0000 7f34acff9640 1 -- 172.21.15.1:0/3138633775 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f34980585d0 con 0x7f34b006f920 2024-05-09T06:28:35.856 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:28:35.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 msgr2=0x7f349006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 0x7f349006e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:35.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 msgr2=0x7f34b00cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f349800c410 tx=0x7f349800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 shutdown_connections 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f349006be30 0x7f349006e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34b0162770 0x7f34b00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34b006f920 0x7f34b00cb1e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 --2- 172.21.15.1:0/3138633775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34b006f120 0x7f34b014f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 >> 172.21.15.1:0/3138633775 conn(0x7f34b00d1d10 msgr2=0x7f34b015a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.877+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 shutdown_connections 2024-05-09T06:28:35.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:35.878+0000 7f34b5c10640 1 -- 172.21.15.1:0/3138633775 wait complete. 2024-05-09T06:28:35.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T06:28:35.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.005+0000 7f8f065e3640 1 Processor -- start 2024-05-09T06:28:35.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.005+0000 7f8f065e3640 1 -- start start 2024-05-09T06:28:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f065e3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8f0015e210 0x7f8f000b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f065e3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8f0015e5f0 0x7f8f001628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f065e3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f000b17c0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f065e3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8f00162e30 con 0x7f8f0015e5f0 2024-05-09T06:28:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f065e3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8f000cca50 con 0x7f8f0015e210 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8eff7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8f0015e5f0 0x7f8f001628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f04b80640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8f000ca9d0 0x7f8f000ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51156/0 (socket says 172.21.15.1:51156) 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8f04b80640 1 -- 172.21.15.1:0/2597261465 learned_addr learned my addr 172.21.15.1:0/2597261465 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8efffff640 1 --2- 172.21.15.1:0/2597261465 >> v2:172.21.15.6:3300/0 conn(0x7f8f0015e210 0x7f8f000b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8eff7fe640 1 -- 172.21.15.1:0/2597261465 >> v2:172.21.15.6:3300/0 conn(0x7f8f0015e210 msgr2=0x7f8f000b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8eff7fe640 1 --2- 172.21.15.1:0/2597261465 >> v2:172.21.15.6:3300/0 conn(0x7f8f0015e210 0x7f8f000b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8eff7fe640 1 -- 172.21.15.1:0/2597261465 >> v1:172.21.15.1:6789/0 conn(0x7f8f000ca9d0 legacy=0x7f8f000ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8eff7fe640 1 -- 172.21.15.1:0/2597261465 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f000b1aa0 con 0x7f8f0015e5f0 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.006+0000 7f8efeffd640 1 -- 172.21.15.1:0/2597261465 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251977514 0 0) 0x7f8f000b17c0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.007+0000 7f8eff7fe640 1 --2- 172.21.15.1:0/2597261465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f0015e5f0 0x7f8f001628f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f000b27c0 tx=0x7f8ef402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69fe6f4fd6ad9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.007+0000 7f8efeffd640 1 -- 172.21.15.1:0/2597261465 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4037070 con 0x7f8f0015e5f0 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.007+0000 7f8efeffd640 1 -- 172.21.15.1:0/2597261465 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef4032440 con 0x7f8f0015e5f0 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.007+0000 7f8efeffd640 1 -- 172.21.15.1:0/2597261465 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4032810 con 0x7f8f0015e5f0 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 -- 172.21.15.1:0/2597261465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f0015e5f0 msgr2=0x7f8f001628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 --2- 172.21.15.1:0/2597261465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f0015e5f0 0x7f8f001628f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f000b27c0 tx=0x7f8ef402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 -- 172.21.15.1:0/2597261465 shutdown_connections 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 --2- 172.21.15.1:0/2597261465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f0015e5f0 0x7f8f001628f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 --2- 172.21.15.1:0/2597261465 >> v2:172.21.15.6:3300/0 conn(0x7f8f0015e210 0x7f8f000b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 -- 172.21.15.1:0/2597261465 >> 172.21.15.1:0/2597261465 conn(0x7f8f00071a50 msgr2=0x7f8f000b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 -- 172.21.15.1:0/2597261465 shutdown_connections 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.008+0000 7f8f065e3640 1 -- 172.21.15.1:0/2597261465 wait complete. 2024-05-09T06:28:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.009+0000 7f8f065e3640 1 Processor -- start 2024-05-09T06:28:35.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.009+0000 7f8f065e3640 1 -- start start 2024-05-09T06:28:35.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8f0015e210 0x7f8f00160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 0x7f8f001f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f000cb000 con 0x7f8f000ca9d0 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8f000cae80 con 0x7f8f001f6a40 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8f000cb180 con 0x7f8f0015e210 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48296/0 (socket says 172.21.15.1:48296) 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 -- 172.21.15.1:0/1711183181 learned_addr learned my addr 172.21.15.1:0/1711183181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 -- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8f0015e210 msgr2=0x7f8f00160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f04b80640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 0x7f8f001f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8f0015e210 0x7f8f00160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 -- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 msgr2=0x7f8f001f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 0x7f8f001f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 -- 172.21.15.1:0/1711183181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f00160940 con 0x7f8f000ca9d0 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f04b80640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 0x7f8f001f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:35.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efffff640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000d830 tx=0x7f8ef000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000b6c0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef0019070 con 0x7f8f000ca9d0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000ba80 con 0x7f8f000ca9d0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f000c77f0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.011+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f000c7db0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.012+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f00075320 con 0x7f8f000ca9d0 2024-05-09T06:28:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.012+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ef001e460 con 0x7f8f000ca9d0 2024-05-09T06:28:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.013+0000 7f8efcff9640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 0x7f8eec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.013+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f8ef008b5b0 con 0x7f8f000ca9d0 2024-05-09T06:28:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.013+0000 7f8eff7fe640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 0x7f8eec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.013+0000 7f8eff7fe640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 0x7f8eec06e320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034000 tx=0x7f8ef402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.018+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ef0059ed0 con 0x7f8f000ca9d0 2024-05-09T06:28:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.200+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f000b6300 con 0x7f8f000ca9d0 2024-05-09T06:28:36.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.876+0000 7f8efcff9640 1 -- 172.21.15.1:0/1711183181 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f8ef00597d0 con 0x7f8f000ca9d0 2024-05-09T06:28:36.857 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' removed 2024-05-09T06:28:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.879+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 msgr2=0x7f8eec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.879+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 0x7f8eec06e320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4034000 tx=0x7f8ef402c000 comp rx=0 tx=0).stop 2024-05-09T06:28:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.880+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 msgr2=0x7f8f000c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.880+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000d830 tx=0x7f8ef000dcf0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 shutdown_connections 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8eec06be60 0x7f8eec06e320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f001f6a40 0x7f8f001f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8f0015e210 0x7f8f00160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 --2- 172.21.15.1:0/1711183181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f000ca9d0 0x7f8f000c40a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 >> 172.21.15.1:0/1711183181 conn(0x7f8f00071a50 msgr2=0x7f8f000b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 shutdown_connections 2024-05-09T06:28:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:36.882+0000 7f8f065e3640 1 -- 172.21.15.1:0/1711183181 wait complete. 2024-05-09T06:28:36.880 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:28:36.881 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:28:36.881 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:28:36.881 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:36.881 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:28:36.881 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:28:36.882 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:28:36.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:36.882 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:28:36.882 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:28:36.882 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:28:36.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:36.882 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:28:36.882 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:28:36.883 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:28:36.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:28:36.891 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:28:36.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:28:36.892 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:28:36.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:28:36.898 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:28:36.898 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:28:36.899 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:28:36.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:28:36.934 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:28:36.935 DEBUG:teuthology.parallel:result is None 2024-05-09T06:28:36.935 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:28:36.936 DEBUG:teuthology.parallel:result is None 2024-05-09T06:28:36.941 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:28:36.941 DEBUG:teuthology.parallel:result is None 2024-05-09T06:28:36.942 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:28:36.942 DEBUG:teuthology.parallel:result is None 2024-05-09T06:28:36.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:28:37.016 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:28:37.016 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:28:37.029 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:28:37.029 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:28:37.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.072+0000 7f81c4ba2640 1 Processor -- start 2024-05-09T06:28:37.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.073+0000 7f81c4ba2640 1 -- start start 2024-05-09T06:28:37.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81c4ba2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f81c00d0d90 0x7f81c00b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:37.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81c4ba2640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c00b54b0 con 0x7f81c01611e0 2024-05-09T06:28:37.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81c4ba2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c00cf730 con 0x7f81c00d09b0 2024-05-09T06:28:37.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81c4ba2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00d1a40 con 0x7f81c00d0d90 2024-05-09T06:28:37.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f81c01611e0 0x7f81c0161630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51172/0 (socket says 172.21.15.1:51172) 2024-05-09T06:28:37.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bdd9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f81c00d0d90 0x7f81c00b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:37.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bed9d640 1 -- 172.21.15.1:0/3655261707 learned_addr learned my addr 172.21.15.1:0/3655261707 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3655261707 >> v1:172.21.15.6:6789/0 conn(0x7f81c00d09b0 legacy=0x7f81c00b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3655261707 >> v1:172.21.15.1:6789/0 conn(0x7f81c01611e0 legacy=0x7f81c0161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.074+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3655261707 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00cfa10 con 0x7f81c00d0d90 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81bdd9b640 1 --2- 172.21.15.1:0/3655261707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c00b6cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f81c00d1bc0 tx=0x7f81b402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23233d21b0787786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81bd59a640 1 -- 172.21.15.1:0/3655261707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4037070 con 0x7f81c00d0d90 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81bd59a640 1 -- 172.21.15.1:0/3655261707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b402ae40 con 0x7f81c00d0d90 2024-05-09T06:28:37.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81bd59a640 1 -- 172.21.15.1:0/3655261707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4032660 con 0x7f81c00d0d90 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3655261707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 msgr2=0x7f81c00b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.075+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3655261707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c00b6cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f81c00d1bc0 tx=0x7f81b402a130 comp rx=0 tx=0).stop 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.076+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3655261707 shutdown_connections 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.076+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3655261707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c00b6cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.076+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3655261707 >> 172.21.15.1:0/3655261707 conn(0x7f81c00d3e70 msgr2=0x7f81c00d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.076+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3655261707 shutdown_connections 2024-05-09T06:28:37.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.076+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3655261707 wait complete. 2024-05-09T06:28:37.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.077+0000 7f81c4ba2640 1 Processor -- start 2024-05-09T06:28:37.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.077+0000 7f81c4ba2640 1 -- start start 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 0x7f81c01568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 0x7f81c01e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00d4bc0 con 0x7f81c00d0d90 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00d1c90 con 0x7f81c00d09b0 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81c4ba2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00d21b0 con 0x7f81c01611e0 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48350/0 (socket says 172.21.15.1:48350) 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3726350108 learned_addr learned my addr 172.21.15.1:0/3726350108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bed9d640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 0x7f81c01e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81be59c640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 0x7f81c01568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 msgr2=0x7f81c01e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 0x7f81c01e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 msgr2=0x7f81c01568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 0x7f81c01568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 -- 172.21.15.1:0/3726350108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01f4320 con 0x7f81c00d0d90 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bed9d640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 0x7f81c01e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81be59c640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 0x7f81c01568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.078+0000 7f81bdd9b640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009890 tx=0x7f81b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.079+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4041020 con 0x7f81c00d0d90 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.079+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b403c670 con 0x7f81c00d0d90 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.079+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01f4600 con 0x7f81c00d0d90 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.079+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4037040 con 0x7f81c00d0d90 2024-05-09T06:28:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.079+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01f4bc0 con 0x7f81c00d0d90 2024-05-09T06:28:37.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.080+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c00752b0 con 0x7f81c00d0d90 2024-05-09T06:28:37.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.080+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81b4044020 con 0x7f81c00d0d90 2024-05-09T06:28:37.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.081+0000 7f81aeffd640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 0x7f81a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:37.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.081+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f81b40ab450 con 0x7f81c00d0d90 2024-05-09T06:28:37.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.081+0000 7f81be59c640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 0x7f81a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:37.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.081+0000 7f81be59c640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 0x7f81a006e320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81c01e9370 tx=0x7f81b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:37.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.085+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81b404b030 con 0x7f81c00d0d90 2024-05-09T06:28:37.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.281+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f81c006b390 con 0x7f81c00d0d90 2024-05-09T06:28:37.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.896+0000 7f81aeffd640 1 -- 172.21.15.1:0/3726350108 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f81b407a000 con 0x7f81c00d0d90 2024-05-09T06:28:37.877 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.899+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 msgr2=0x7f81a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.899+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 0x7f81a006e320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81c01e9370 tx=0x7f81b0009040 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.899+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 msgr2=0x7f81c01e77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.899+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009890 tx=0x7f81b40046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 shutdown_connections 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f81a006be60 0x7f81a006e320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81c01611e0 0x7f81c01e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81c00d0d90 0x7f81c01e77d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 --2- 172.21.15.1:0/3726350108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81c00d09b0 0x7f81c01568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 >> 172.21.15.1:0/3726350108 conn(0x7f81c00d3e70 msgr2=0x7f81c00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 shutdown_connections 2024-05-09T06:28:37.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:37.900+0000 7f81c4ba2640 1 -- 172.21.15.1:0/3726350108 wait complete. 2024-05-09T06:28:37.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:28:38.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.023+0000 7fcb60a58640 1 Processor -- start 2024-05-09T06:28:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.023+0000 7fcb60a58640 1 -- start start 2024-05-09T06:28:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb60a58640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcb5c06e6d0 0x7fcb5c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb60a58640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcb5c0b3300 0x7fcb5c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb60a58640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c0ce0e0 con 0x7fcb5c0cf180 2024-05-09T06:28:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb60a58640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5c0ce3c0 con 0x7fcb5c0b3300 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb60a58640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5c0ce6a0 con 0x7fcb5c06e6d0 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb59d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcb5c0cf180 0x7fcb5c0cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44634/0 (socket says 172.21.15.1:44634) 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb59d9b640 1 -- 172.21.15.1:0/32417300 learned_addr learned my addr 172.21.15.1:0/32417300 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5ad9d640 1 --2- 172.21.15.1:0/32417300 >> v2:172.21.15.1:3300/0 conn(0x7fcb5c0b3300 0x7fcb5c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5a59c640 1 --2- 172.21.15.1:0/32417300 >> v2:172.21.15.6:3300/0 conn(0x7fcb5c06e6d0 0x7fcb5c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5959a640 1 -- 172.21.15.1:0/32417300 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1288133899 0 0) 0x7fcb5c0ce0e0 con 0x7fcb5c0cf180 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5959a640 1 -- 172.21.15.1:0/32417300 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb48003520 con 0x7fcb5c0cf180 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5ad9d640 1 -- 172.21.15.1:0/32417300 >> v2:172.21.15.6:3300/0 conn(0x7fcb5c06e6d0 msgr2=0x7fcb5c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5ad9d640 1 --2- 172.21.15.1:0/32417300 >> v2:172.21.15.6:3300/0 conn(0x7fcb5c06e6d0 0x7fcb5c06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5ad9d640 1 -- 172.21.15.1:0/32417300 >> v1:172.21.15.1:6789/0 conn(0x7fcb5c0cf180 legacy=0x7fcb5c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.024+0000 7fcb5ad9d640 1 -- 172.21.15.1:0/32417300 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c0ce980 con 0x7fcb5c0b3300 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.025+0000 7fcb5ad9d640 1 --2- 172.21.15.1:0/32417300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0b3710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c074650 tx=0x7fcb5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2929cce549ef16b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.025+0000 7fcb5959a640 1 -- 172.21.15.1:0/32417300 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54037070 con 0x7fcb5c0b3300 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.025+0000 7fcb5959a640 1 -- 172.21.15.1:0/32417300 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54032440 con 0x7fcb5c0b3300 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.025+0000 7fcb5959a640 1 -- 172.21.15.1:0/32417300 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54032810 con 0x7fcb5c0b3300 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 -- 172.21.15.1:0/32417300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 msgr2=0x7fcb5c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 --2- 172.21.15.1:0/32417300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0b3710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c074650 tx=0x7fcb5402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 -- 172.21.15.1:0/32417300 shutdown_connections 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 --2- 172.21.15.1:0/32417300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0b3710 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 --2- 172.21.15.1:0/32417300 >> v2:172.21.15.6:3300/0 conn(0x7fcb5c06e6d0 0x7fcb5c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 -- 172.21.15.1:0/32417300 >> 172.21.15.1:0/32417300 conn(0x7fcb5c0cc4b0 msgr2=0x7fcb5c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 -- 172.21.15.1:0/32417300 shutdown_connections 2024-05-09T06:28:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.026+0000 7fcb60a58640 1 -- 172.21.15.1:0/32417300 wait complete. 2024-05-09T06:28:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.027+0000 7fcb60a58640 1 Processor -- start 2024-05-09T06:28:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.027+0000 7fcb60a58640 1 -- start start 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb5c06e6d0 0x7fcb5c1363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb5c0cf180 0x7fcb5c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c070ed0 con 0x7fcb5c0b3300 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c06eff0 con 0x7fcb5c06e6d0 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb60a58640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c1447c0 con 0x7fcb5c0cf180 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49586/0 (socket says 172.21.15.1:49586) 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 -- 172.21.15.1:0/3983882452 learned_addr learned my addr 172.21.15.1:0/3983882452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 -- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb5c0cf180 msgr2=0x7fcb5c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb5c0cf180 0x7fcb5c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 -- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb5c06e6d0 msgr2=0x7fcb5c1363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb5a59c640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb5c06e6d0 0x7fcb5c1363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb5c06e6d0 0x7fcb5c1363c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.028+0000 7fcb59d9b640 1 -- 172.21.15.1:0/3983882452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c15d780 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb59d9b640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400c400 tx=0x7fcb4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb44005d00 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c15da60 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44007310 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.029+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c15e020 con 0x7fcb5c0b3300 2024-05-09T06:28:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.030+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c0752b0 con 0x7fcb5c0b3300 2024-05-09T06:28:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.030+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb4401a070 con 0x7fcb5c0b3300 2024-05-09T06:28:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.031+0000 7fcb52ffd640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 0x7fcb3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.031+0000 7fcb5a59c640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 0x7fcb3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.031+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fcb4408a070 con 0x7fcb5c0b3300 2024-05-09T06:28:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.031+0000 7fcb5a59c640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 0x7fcb3c06e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0abf10 tx=0x7fcb4c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.035+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb44058c70 con 0x7fcb5c0b3300 2024-05-09T06:28:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.288+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcb5c0cb920 con 0x7fcb5c0b3300 2024-05-09T06:28:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.289+0000 7fcb52ffd640 1 -- 172.21.15.1:0/3983882452 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fcb44058570 con 0x7fcb5c0b3300 2024-05-09T06:28:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:38.270 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:28:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:28:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:28:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:28:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:28:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:28:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:28:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-mds", 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxmF2vJKhAAvGCw4KlGDHfMc4RV4LYqYw==", 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-mds" 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-mgr", 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxmA3bJKhAAdLaUU+TDrQ3eV4TdJbYu5w==", 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-osd", 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxmboDJKhAAqgQeUsXC7eyPfytvg8mKzA==", 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-osd" 2024-05-09T06:28:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-rbd", 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxmUIjJKhAAFNK544+cnxQREnU/uzYovw==", 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxm84/JKhAAadvWx/ZRl+DVr3CoZtsDRQ==", 2024-05-09T06:28:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.bootstrap-rgw", 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCwbDxm5ZfJKhAA96yneMA+pYPcN6JrzvPqCg==", 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:28:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:28:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:28:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:28:38.289 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 msgr2=0x7fcb3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 0x7fcb3c06e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0abf10 tx=0x7fcb4c00b040 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 msgr2=0x7fcb5c0a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400c400 tx=0x7fcb4400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 shutdown_connections 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcb3c06be30 0x7fcb3c06e2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb5c0cf180 0x7fcb5c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb5c0b3300 0x7fcb5c0a9d30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 --2- 172.21.15.1:0/3983882452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb5c06e6d0 0x7fcb5c1363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 >> 172.21.15.1:0/3983882452 conn(0x7fcb5c0cc4b0 msgr2=0x7fcb5c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 shutdown_connections 2024-05-09T06:28:38.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.291+0000 7fcb60a58640 1 -- 172.21.15.1:0/3983882452 wait complete. 2024-05-09T06:28:38.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-09T06:28:38.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.416+0000 7f5757458640 1 Processor -- start 2024-05-09T06:28:38.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.416+0000 7f5757458640 1 -- start start 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f575015e1d0 0x7f575006f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f575006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57500d2890 0x7f57500cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f57500cc3c0 con 0x7f57500d2890 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f57500ccc60 con 0x7f575015e1d0 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.417+0000 7f5757458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f575006c2f0 con 0x7f575015e5b0 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57500d2890 0x7f57500cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57500d2890 0x7f57500cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49600/0 (socket says 172.21.15.1:49600) 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 -- 172.21.15.1:0/3243980205 learned_addr learned my addr 172.21.15.1:0/3243980205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:38.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57551f4640 1 --2- 172.21.15.1:0/3243980205 >> v2:172.21.15.6:3300/0 conn(0x7f575015e1d0 0x7f575006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57549f3640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f575006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 -- 172.21.15.1:0/3243980205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 msgr2=0x7f575006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f575006fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 -- 172.21.15.1:0/3243980205 >> v2:172.21.15.6:3300/0 conn(0x7f575015e1d0 msgr2=0x7f575006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 --2- 172.21.15.1:0/3243980205 >> v2:172.21.15.6:3300/0 conn(0x7f575015e1d0 0x7f575006f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.418+0000 7f57559f5640 1 -- 172.21.15.1:0/3243980205 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575006f500 con 0x7f57500d2890 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.419+0000 7f57559f5640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500cd0f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5748009e50 tx=0x7f574802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6c96ea74a1724b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.419+0000 7f5747fff640 1 -- 172.21.15.1:0/3243980205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748036070 con 0x7f57500d2890 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.419+0000 7f5747fff640 1 -- 172.21.15.1:0/3243980205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5748031600 con 0x7f57500d2890 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.419+0000 7f5747fff640 1 -- 172.21.15.1:0/3243980205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57480319d0 con 0x7f57500d2890 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 -- 172.21.15.1:0/3243980205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 msgr2=0x7f57500cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500cd0f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5748009e50 tx=0x7f574802a630 comp rx=0 tx=0).stop 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 -- 172.21.15.1:0/3243980205 shutdown_connections 2024-05-09T06:28:38.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500cd0f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 --2- 172.21.15.1:0/3243980205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f575006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 --2- 172.21.15.1:0/3243980205 >> v2:172.21.15.6:3300/0 conn(0x7f575015e1d0 0x7f575006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 -- 172.21.15.1:0/3243980205 >> 172.21.15.1:0/3243980205 conn(0x7f57500d1d70 msgr2=0x7f57500d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 -- 172.21.15.1:0/3243980205 shutdown_connections 2024-05-09T06:28:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.420+0000 7f5757458640 1 -- 172.21.15.1:0/3243980205 wait complete. 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.421+0000 7f5757458640 1 Processor -- start 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.421+0000 7f5757458640 1 -- start start 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 0x7f575015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f5750146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f575006f1b0 con 0x7f57500d2890 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f57500cc8f0 con 0x7f575015e1d0 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5757458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f57500cd630 con 0x7f575015e5b0 2024-05-09T06:28:38.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49604/0 (socket says 172.21.15.1:49604) 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 -- 172.21.15.1:0/1463888915 learned_addr learned my addr 172.21.15.1:0/1463888915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57559f5640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f5750146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57549f3640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 0x7f575015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 -- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 msgr2=0x7f5750146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f5750146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 -- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 msgr2=0x7f575015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 0x7f575015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 -- 172.21.15.1:0/1463888915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57500cabd0 con 0x7f57500d2890 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57559f5640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f5750146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57549f3640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 0x7f575015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f57551f4640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e890 tx=0x7f574c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.422+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c017070 con 0x7f57500d2890 2024-05-09T06:28:38.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.423+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574c00c940 con 0x7f57500d2890 2024-05-09T06:28:38.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.423+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57500caeb0 con 0x7f57500d2890 2024-05-09T06:28:38.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.423+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c00cd10 con 0x7f57500d2890 2024-05-09T06:28:38.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.423+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57500cb750 con 0x7f57500d2890 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.424+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57500cb080 con 0x7f57500d2890 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.424+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f574c01a070 con 0x7f57500d2890 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.425+0000 7f5745ffb640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 0x7f573406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.425+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f574c08af90 con 0x7f57500d2890 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.425+0000 7f57549f3640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 0x7f573406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.425+0000 7f57549f3640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 0x7f573406e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57501f7090 tx=0x7f5740008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.429+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f574c059b90 con 0x7f57500d2890 2024-05-09T06:28:38.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.680+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f57500752b0 con 0x7f57500d2890 2024-05-09T06:28:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.682+0000 7f5745ffb640 1 -- 172.21.15.1:0/1463888915 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f574c059490 con 0x7f57500d2890 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 msgr2=0x7f573406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 0x7f573406e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57501f7090 tx=0x7f5740008040 comp rx=0 tx=0).stop 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 msgr2=0x7f57500c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e890 tx=0x7f574c00ed50 comp rx=0 tx=0).stop 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 shutdown_connections 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f573406be30 0x7f573406e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f575015e5b0 0x7f5750146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f575015e1d0 0x7f575015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 --2- 172.21.15.1:0/1463888915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57500d2890 0x7f57500c8160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 >> 172.21.15.1:0/1463888915 conn(0x7f57500d1d70 msgr2=0x7f57500d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 shutdown_connections 2024-05-09T06:28:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.684+0000 7f5757458640 1 -- 172.21.15.1:0/1463888915 wait complete. 2024-05-09T06:28:38.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-09T06:28:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 Processor -- start 2024-05-09T06:28:38.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 -- start start 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f21a806e6d0 0x7f21a806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a80ce0e0 con 0x7f21a80cf180 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a80ce3c0 con 0x7f21a80b3300 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.800+0000 7f21ad8c4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f21a80ce6a0 con 0x7f21a806e6d0 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a6ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f21a806e6d0 0x7f21a806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a67fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f21a80cf180 0x7f21a80cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44644/0 (socket says 172.21.15.1:44644) 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a67fc640 1 -- 172.21.15.1:0/3209880893 learned_addr learned my addr 172.21.15.1:0/3209880893 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a6ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f21a806e6d0 0x7f21a806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49612/0 (socket says 172.21.15.1:49612) 2024-05-09T06:28:38.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a6ffd640 1 -- 172.21.15.1:0/3209880893 >> v1:172.21.15.6:6789/0 conn(0x7f21a80b3300 legacy=0x7f21a80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:38.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a6ffd640 1 -- 172.21.15.1:0/3209880893 >> v1:172.21.15.1:6789/0 conn(0x7f21a80cf180 legacy=0x7f21a80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a5ffb640 1 -- 172.21.15.1:0/3209880893 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827182732 0 0) 0x7f21a80ce0e0 con 0x7f21a80cf180 2024-05-09T06:28:38.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.801+0000 7f21a6ffd640 1 -- 172.21.15.1:0/3209880893 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a80ce980 con 0x7f21a806e6d0 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21a6ffd640 1 --2- 172.21.15.1:0/3209880893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a806e6d0 0x7f21a806eab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f219800ba40 tx=0x7f219802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133994630593e2fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21a5ffb640 1 -- 172.21.15.1:0/3209880893 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198032770 con 0x7f21a806e6d0 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21a5ffb640 1 -- 172.21.15.1:0/3209880893 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198038070 con 0x7f21a806e6d0 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21a5ffb640 1 -- 172.21.15.1:0/3209880893 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198032b30 con 0x7f21a806e6d0 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21ad8c4640 1 -- 172.21.15.1:0/3209880893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a806e6d0 msgr2=0x7f21a806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/3209880893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a806e6d0 0x7f21a806eab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f219800ba40 tx=0x7f219802c8b0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21ad8c4640 1 -- 172.21.15.1:0/3209880893 shutdown_connections 2024-05-09T06:28:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/3209880893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a806e6d0 0x7f21a806eab0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.802+0000 7f21ad8c4640 1 -- 172.21.15.1:0/3209880893 >> 172.21.15.1:0/3209880893 conn(0x7f21a80cc4b0 msgr2=0x7f21a80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.803+0000 7f21ad8c4640 1 -- 172.21.15.1:0/3209880893 shutdown_connections 2024-05-09T06:28:38.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.803+0000 7f21ad8c4640 1 -- 172.21.15.1:0/3209880893 wait complete. 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.804+0000 7f21ad8c4640 1 Processor -- start 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.804+0000 7f21ad8c4640 1 -- start start 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 0x7f21a815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21a80b3300 0x7f21a815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8070ed0 con 0x7f21a80cf180 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f21a806eff0 con 0x7f21a806e6d0 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f21a81447c0 con 0x7f21a80b3300 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49616/0 (socket says 172.21.15.1:49616) 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 -- 172.21.15.1:0/683121856 learned_addr learned my addr 172.21.15.1:0/683121856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:38.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a6ffd640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 0x7f21a815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 -- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21a80b3300 msgr2=0x7f21a815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21a80b3300 0x7f21a815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 -- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 msgr2=0x7f21a815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 0x7f21a815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 -- 172.21.15.1:0/683121856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a81416e0 con 0x7f21a80cf180 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a6ffd640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 0x7f21a815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21a77fe640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c7b0 tx=0x7f21a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004b60 con 0x7f21a80cf180 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a0004d00 con 0x7f21a80cf180 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a815d660 con 0x7f21a80cf180 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a000aa90 con 0x7f21a80cf180 2024-05-09T06:28:38.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.805+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a81419a0 con 0x7f21a80cf180 2024-05-09T06:28:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.806+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a80752b0 con 0x7f21a80cf180 2024-05-09T06:28:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.807+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21a000ac30 con 0x7f21a80cf180 2024-05-09T06:28:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.808+0000 7f218ffff640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.808+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f21a008aed0 con 0x7f21a80cf180 2024-05-09T06:28:38.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.808+0000 7f21a6ffd640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:38.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.808+0000 7f21a6ffd640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 0x7f219406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2198037000 tx=0x7f2198008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:38.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:38.812+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21a0020070 con 0x7f21a80cf180 2024-05-09T06:28:39.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.067+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f21a806cf50 con 0x7f21a80cf180 2024-05-09T06:28:39.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.069+0000 7f218ffff640 1 -- 172.21.15.1:0/683121856 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f21a0059ad0 con 0x7f21a80cf180 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 msgr2=0x7f219406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 0x7f219406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2198037000 tx=0x7f2198008040 comp rx=0 tx=0).stop 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 msgr2=0x7f21a815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c7b0 tx=0x7f21a000cc70 comp rx=0 tx=0).stop 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 shutdown_connections 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a80cf180 0x7f21a815e530 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21a80b3300 0x7f21a815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 --2- 172.21.15.1:0/683121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f21a806e6d0 0x7f21a815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 >> 172.21.15.1:0/683121856 conn(0x7f21a80cc4b0 msgr2=0x7f21a806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 shutdown_connections 2024-05-09T06:28:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.071+0000 7f21ad8c4640 1 -- 172.21.15.1:0/683121856 wait complete. 2024-05-09T06:28:39.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-09T06:28:39.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.187+0000 7f86d407d640 1 Processor -- start 2024-05-09T06:28:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 -- start start 2024-05-09T06:28:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86cc0d0f80 0x7f86cc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86cc0b33c0 con 0x7f86cc0d4940 2024-05-09T06:28:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f86cc0d4dd0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.188+0000 7f86d407d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f86cc0b3660 con 0x7f86cc161950 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86cc0d0f80 0x7f86cc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86cc0d0f80 0x7f86cc0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49622/0 (socket says 172.21.15.1:49622) 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 -- 172.21.15.1:0/2443099890 learned_addr learned my addr 172.21.15.1:0/2443099890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d1e19640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 -- 172.21.15.1:0/2443099890 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 msgr2=0x7f86cc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 -- 172.21.15.1:0/2443099890 >> v1:172.21.15.6:6789/0 conn(0x7f86cc0d4940 legacy=0x7f86cc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.189+0000 7f86d261a640 1 -- 172.21.15.1:0/2443099890 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc0b3940 con 0x7f86cc0d0f80 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d261a640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0d2940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b3590 tx=0x7f86c402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a49d10e5f8ec1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d0e17640 1 -- 172.21.15.1:0/2443099890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4036070 con 0x7f86cc0d0f80 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d0e17640 1 -- 172.21.15.1:0/2443099890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4031600 con 0x7f86cc0d0f80 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d0e17640 1 -- 172.21.15.1:0/2443099890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40319d0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 -- 172.21.15.1:0/2443099890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 msgr2=0x7f86cc0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0d2940 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b3590 tx=0x7f86c402a710 comp rx=0 tx=0).stop 2024-05-09T06:28:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 -- 172.21.15.1:0/2443099890 shutdown_connections 2024-05-09T06:28:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0d2940 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 --2- 172.21.15.1:0/2443099890 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.190+0000 7f86d407d640 1 -- 172.21.15.1:0/2443099890 >> 172.21.15.1:0/2443099890 conn(0x7f86cc0d3e20 msgr2=0x7f86cc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.191+0000 7f86d407d640 1 -- 172.21.15.1:0/2443099890 shutdown_connections 2024-05-09T06:28:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.191+0000 7f86d407d640 1 -- 172.21.15.1:0/2443099890 wait complete. 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 Processor -- start 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 -- start start 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 0x7f86cc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc0b52d0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc0b5150 con 0x7f86cc0a14f0 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.192+0000 7f86d407d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f86cc0b5450 con 0x7f86cc161950 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49628/0 (socket says 172.21.15.1:49628) 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 -- 172.21.15.1:0/2157079264 learned_addr learned my addr 172.21.15.1:0/2157079264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1618640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d261a640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 0x7f86cc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 -- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 msgr2=0x7f86cc0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 -- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 msgr2=0x7f86cc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 0x7f86cc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 -- 172.21.15.1:0/2157079264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc0a1980 con 0x7f86cc0d0f80 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d261a640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 0x7f86cc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:39.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1618640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d1e19640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d5f0 tx=0x7f86c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c80125f0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c8019070 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc1f7320 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c80129d0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.193+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1f78e0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.194+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86cc0752b0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.195+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86c8012b70 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.195+0000 7f86c27fc640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 0x7f86b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.195+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f86c808b050 con 0x7f86cc0d0f80 2024-05-09T06:28:39.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.195+0000 7f86d1618640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 0x7f86b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.196+0000 7f86d1618640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 0x7f86b006e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86bc007fd0 tx=0x7f86bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.200+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86c8059bc0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.451+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f86cc0d21e0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.453+0000 7f86c27fc640 1 -- 172.21.15.1:0/2157079264 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f86c80594c0 con 0x7f86cc0d0f80 2024-05-09T06:28:39.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 msgr2=0x7f86b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 0x7f86b006e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86bc007fd0 tx=0x7f86bc009040 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 msgr2=0x7f86cc0b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d5f0 tx=0x7f86c800dab0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 shutdown_connections 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f86b006be60 0x7f86b006e320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86cc0a14f0 0x7f86cc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86cc161950 0x7f86cc0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 --2- 172.21.15.1:0/2157079264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86cc0d0f80 0x7f86cc0b9900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 >> 172.21.15.1:0/2157079264 conn(0x7f86cc0d3e20 msgr2=0x7f86cc15ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.455+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 shutdown_connections 2024-05-09T06:28:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.456+0000 7f86d407d640 1 -- 172.21.15.1:0/2157079264 wait complete. 2024-05-09T06:28:39.447 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-09T06:28:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.571+0000 7fdd5e3f4640 1 Processor -- start 2024-05-09T06:28:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.571+0000 7fdd5e3f4640 1 -- start start 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5e3f4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 0x7fdd580b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5e3f4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd580b4f70 0x7fdd58070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5e3f4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5806eb40 con 0x7fdd580b2a60 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5e3f4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5806ee20 con 0x7fdd580b4f70 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5e3f4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5806d8e0 con 0x7fdd581454b0 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd580b4f70 0x7fdd58070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd580b4f70 0x7fdd58070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49630/0 (socket says 172.21.15.1:49630) 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd577fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdd580b2a60 0x7fdd580b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44648/0 (socket says 172.21.15.1:44648) 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 -- 172.21.15.1:0/1805519238 learned_addr learned my addr 172.21.15.1:0/1805519238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 -- 172.21.15.1:0/1805519238 >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 msgr2=0x7fdd580b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd57fff640 1 --2- 172.21.15.1:0/1805519238 >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 0x7fdd580b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 --2- 172.21.15.1:0/1805519238 >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 0x7fdd580b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 -- 172.21.15.1:0/1805519238 >> v1:172.21.15.1:6789/0 conn(0x7fdd580b2a60 legacy=0x7fdd580b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd56ffd640 1 -- 172.21.15.1:0/1805519238 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747077481 0 0) 0x7fdd5806eb40 con 0x7fdd580b2a60 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd5c991640 1 -- 172.21.15.1:0/1805519238 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5806dbc0 con 0x7fdd580b4f70 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.572+0000 7fdd57fff640 1 --2- 172.21.15.1:0/1805519238 >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 0x7fdd580b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.573+0000 7fdd5c991640 1 --2- 172.21.15.1:0/1805519238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b4f70 0x7fdd58070c10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdd580689d0 tx=0x7fdd5002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b7b95be2f86c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.573+0000 7fdd56ffd640 1 -- 172.21.15.1:0/1805519238 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50036070 con 0x7fdd580b4f70 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.573+0000 7fdd56ffd640 1 -- 172.21.15.1:0/1805519238 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd50031600 con 0x7fdd580b4f70 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.573+0000 7fdd56ffd640 1 -- 172.21.15.1:0/1805519238 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd500319d0 con 0x7fdd580b4f70 2024-05-09T06:28:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/1805519238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b4f70 msgr2=0x7fdd58070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/1805519238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b4f70 0x7fdd58070c10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdd580689d0 tx=0x7fdd5002a730 comp rx=0 tx=0).stop 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/1805519238 shutdown_connections 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/1805519238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b4f70 0x7fdd58070c10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/1805519238 >> v2:172.21.15.6:3300/0 conn(0x7fdd581454b0 0x7fdd580b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/1805519238 >> 172.21.15.1:0/1805519238 conn(0x7fdd580cc4b0 msgr2=0x7fdd580cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/1805519238 shutdown_connections 2024-05-09T06:28:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.574+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/1805519238 wait complete. 2024-05-09T06:28:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.575+0000 7fdd5e3f4640 1 Processor -- start 2024-05-09T06:28:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.575+0000 7fdd5e3f4640 1 -- start start 2024-05-09T06:28:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580b4f70 0x7fdd581e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd581454b0 0x7fdd581e1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5806e260 con 0x7fdd580b2a60 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5806e0e0 con 0x7fdd581454b0 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5e3f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdd58071150 con 0x7fdd580b4f70 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49646/0 (socket says 172.21.15.1:49646) 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd57fff640 1 -- 172.21.15.1:0/2773485687 learned_addr learned my addr 172.21.15.1:0/2773485687 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd577fe640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580b4f70 0x7fdd581e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.576+0000 7fdd5c991640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd581454b0 0x7fdd581e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 -- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580b4f70 msgr2=0x7fdd581e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580b4f70 0x7fdd581e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 -- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd581454b0 msgr2=0x7fdd581e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd581454b0 0x7fdd581e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 -- 172.21.15.1:0/2773485687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581f6100 con 0x7fdd580b2a60 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd57fff640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c7f0 tx=0x7fdd4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48015070 con 0x7fdd580b2a60 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd48007820 con 0x7fdd580b2a60 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581e3a10 con 0x7fdd580b2a60 2024-05-09T06:28:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48007bd0 con 0x7fdd580b2a60 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.577+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581e3fd0 con 0x7fdd580b2a60 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.578+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580752b0 con 0x7fdd580b2a60 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.579+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd48018070 con 0x7fdd580b2a60 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.579+0000 7fdd54ff9640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.579+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fdd4808acf0 con 0x7fdd580b2a60 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.579+0000 7fdd577fe640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.580+0000 7fdd577fe640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd581e16d0 tx=0x7fdd40007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.584+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd4808f070 con 0x7fdd580b2a60 2024-05-09T06:28:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.838+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fdd58072fa0 con 0x7fdd580b2a60 2024-05-09T06:28:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.840+0000 7fdd54ff9640 1 -- 172.21.15.1:0/2773485687 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fdd48059210 con 0x7fdd580b2a60 2024-05-09T06:28:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 msgr2=0x7fdd3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd581e16d0 tx=0x7fdd40007d50 comp rx=0 tx=0).stop 2024-05-09T06:28:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 msgr2=0x7fdd5813c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c7f0 tx=0x7fdd4800ccb0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 shutdown_connections 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd581454b0 0x7fdd581e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580b4f70 0x7fdd581e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 --2- 172.21.15.1:0/2773485687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd580b2a60 0x7fdd5813c6d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 >> 172.21.15.1:0/2773485687 conn(0x7fdd580cc4b0 msgr2=0x7fdd580b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 shutdown_connections 2024-05-09T06:28:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.842+0000 7fdd5e3f4640 1 -- 172.21.15.1:0/2773485687 wait complete. 2024-05-09T06:28:39.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-09T06:28:39.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.959+0000 7fb9a1da2640 1 Processor -- start 2024-05-09T06:28:39.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.959+0000 7fb9a1da2640 1 -- start start 2024-05-09T06:28:39.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb9a1da2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb99c1565e0 0x7fb99c074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb9a1da2640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c074a70 con 0x7fb99c15c680 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb9a1da2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c15cc40 con 0x7fb99c1601c0 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb9a1da2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb99c15fd30 con 0x7fb99c1565e0 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb99affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb99c15c680 0x7fb99c0afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44662/0 (socket says 172.21.15.1:44662) 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb99affd640 1 -- 172.21.15.1:0/3572748828 learned_addr learned my addr 172.21.15.1:0/3572748828 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb99b7fe640 1 --2- 172.21.15.1:0/3572748828 >> v2:172.21.15.6:3300/0 conn(0x7fb99c1565e0 0x7fb99c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601310338 0 0) 0x7fb99c074a70 con 0x7fb99c15c680 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.960+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb984003520 con 0x7fb99c15c680 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3913301422 0 0) 0x7fb984003520 con 0x7fb99c15c680 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 >> v2:172.21.15.6:3300/0 conn(0x7fb99c1565e0 msgr2=0x7fb99c074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 --2- 172.21.15.1:0/3572748828 >> v2:172.21.15.6:3300/0 conn(0x7fb99c1565e0 0x7fb99c074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 >> v1:172.21.15.6:6789/0 conn(0x7fb99c1601c0 legacy=0x7fb99c0b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c15cf20 con 0x7fb99c15c680 2024-05-09T06:28:39.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb990001670 con 0x7fb99c15c680 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 >> v1:172.21.15.1:6789/0 conn(0x7fb99c15c680 legacy=0x7fb99c0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb984004800 0x7fb984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb98400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c15fd30 con 0x7fb9840051c0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb98400c060 con 0x7fb984004800 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb98400c260 con 0x7fb9840039b0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb98400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99b7fe640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99affd640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb984004800 0x7fb984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 -- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9840039b0 msgr2=0x7fb984003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9840039b0 0x7fb984003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 -- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb984004800 msgr2=0x7fb984004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb984004800 0x7fb984004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.961+0000 7fb99bfff640 1 -- 172.21.15.1:0/3572748828 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb984003520 con 0x7fb9840051c0 2024-05-09T06:28:39.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.962+0000 7fb99bfff640 1 -- 172.21.15.1:0/3572748828 reap_dead start 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.962+0000 7fb99bfff640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb98400b9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb994001e50 tx=0x7fb984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b681b46120f0ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.962+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994001a70 con 0x7fb9840051c0 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.962+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb994003970 con 0x7fb9840051c0 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.962+0000 7fb99a7fc640 1 -- 172.21.15.1:0/3572748828 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994003d60 con 0x7fb9840051c0 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 -- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 msgr2=0x7fb98400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb98400b9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb994001e50 tx=0x7fb984005830 comp rx=0 tx=0).stop 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 -- 172.21.15.1:0/3572748828 shutdown_connections 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/3572748828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb98400b9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 -- 172.21.15.1:0/3572748828 >> 172.21.15.1:0/3572748828 conn(0x7fb99c0b1fb0 msgr2=0x7fb99c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 -- 172.21.15.1:0/3572748828 shutdown_connections 2024-05-09T06:28:39.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.963+0000 7fb9a1da2640 1 -- 172.21.15.1:0/3572748828 wait complete. 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.964+0000 7fb9a1da2640 1 Processor -- start 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.964+0000 7fb9a1da2640 1 -- start start 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb99c158870 0x7fb99c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb99c152a40 0x7fb99c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c1569c0 con 0x7fb9840051c0 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c15d4e0 con 0x7fb99c158870 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb9a1da2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb99c074530 con 0x7fb99c152a40 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49672/0 (socket says 172.21.15.1:49672) 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 -- 172.21.15.1:0/2224903414 learned_addr learned my addr 172.21.15.1:0/2224903414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:39.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99bfff640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb99c152a40 0x7fb99c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99affd640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb99c158870 0x7fb99c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 -- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb99c152a40 msgr2=0x7fb99c156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb99c152a40 0x7fb99c156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 -- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb99c158870 msgr2=0x7fb99c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb99c158870 0x7fb99c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 -- 172.21.15.1:0/2224903414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c14fdc0 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.965+0000 7fb99b7fe640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0118e0 tx=0x7fb98c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.966+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00faf0 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.966+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c00fc90 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.966+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c014440 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.966+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99c1500a0 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.966+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99c150660 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.967+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99c0752b0 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.967+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb98c012040 con 0x7fb9840051c0 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.968+0000 7fb983fff640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:39.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.968+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb98c08b2c0 con 0x7fb9840051c0 2024-05-09T06:28:39.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.968+0000 7fb99affd640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:39.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.968+0000 7fb99affd640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 0x7fb98806e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb99c1f5310 tx=0x7fb990001c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:39.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:39.972+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb98c019030 con 0x7fb9840051c0 2024-05-09T06:28:40.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.232+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb99c152e20 con 0x7fb9840051c0 2024-05-09T06:28:40.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.235+0000 7fb983fff640 1 -- 172.21.15.1:0/2224903414 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fb98c0597e0 con 0x7fb9840051c0 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 msgr2=0x7fb98806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 0x7fb98806e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb99c1f5310 tx=0x7fb990001c60 comp rx=0 tx=0).stop 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 msgr2=0x7fb99c148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0118e0 tx=0x7fb98c011da0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 shutdown_connections 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb98806be30 0x7fb98806e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb99c152a40 0x7fb99c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb99c158870 0x7fb99c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 --2- 172.21.15.1:0/2224903414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9840051c0 0x7fb99c148670 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 >> 172.21.15.1:0/2224903414 conn(0x7fb99c0b1fb0 msgr2=0x7fb99c0ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 shutdown_connections 2024-05-09T06:28:40.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.237+0000 7fb9a1da2640 1 -- 172.21.15.1:0/2224903414 wait complete. 2024-05-09T06:28:40.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-09T06:28:40.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.352+0000 7f063c5c1640 1 Processor -- start 2024-05-09T06:28:40.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.352+0000 7f063c5c1640 1 -- start start 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063c5c1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06340b0080 0x7f0634071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063c5c1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06340d04a0 con 0x7f063415ece0 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063c5c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06340722c0 con 0x7f06340cbb60 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063c5c1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f06340cbfe0 con 0x7f06340b0080 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063ab5e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f063415ece0 0x7f063415f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44664/0 (socket says 172.21.15.1:44664) 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.353+0000 7f063ab5e640 1 -- 172.21.15.1:0/3455101019 learned_addr learned my addr 172.21.15.1:0/3455101019 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063a35d640 1 --2- 172.21.15.1:0/3455101019 >> v2:172.21.15.6:3300/0 conn(0x7f06340b0080 0x7f0634071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774417891 0 0) 0x7f06340d04a0 con 0x7f063415ece0 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f061c003520 con 0x7f063415ece0 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 529356789 0 0) 0x7f061c003520 con 0x7f063415ece0 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 >> v2:172.21.15.6:3300/0 conn(0x7f06340b0080 msgr2=0x7f0634071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 --2- 172.21.15.1:0/3455101019 >> v2:172.21.15.6:3300/0 conn(0x7f06340b0080 0x7f0634071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 >> v1:172.21.15.6:6789/0 conn(0x7f06340cbb60 legacy=0x7f063415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06340725a0 con 0x7f063415ece0 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f06240019f0 con 0x7f063415ece0 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 >> v1:172.21.15.1:6789/0 conn(0x7f063415ece0 legacy=0x7f063415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f061c0039b0 0x7f061c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 0x7f061c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c0051c0 0x7f061c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06340cbfe0 con 0x7f061c004800 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f061c00c060 con 0x7f061c0051c0 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f061c00c260 con 0x7f061c0039b0 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 0x7f061c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063a35d640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f061c0039b0 0x7f061c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f063ab5e640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c0051c0 0x7f061c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 -- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f061c0039b0 msgr2=0x7f061c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f061c0039b0 0x7f061c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 -- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c0051c0 msgr2=0x7f061c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c0051c0 0x7f061c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.354+0000 7f0639b5c640 1 -- 172.21.15.1:0/3455101019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c003520 con 0x7f061c004800 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063ab5e640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c0051c0 0x7f061c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063a35d640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f061c0039b0 0x7f061c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063ab5e640 1 -- 172.21.15.1:0/3455101019 reap_dead start 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f0639b5c640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 0x7f061c004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0628001750 tx=0x7f061c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c2c62c6bb6c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628001370 con 0x7f061c004800 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06280052d0 con 0x7f061c004800 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.355+0000 7f063935b640 1 -- 172.21.15.1:0/3455101019 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062802abc0 con 0x7f061c004800 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 -- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 msgr2=0x7f061c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 0x7f061c004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0628001750 tx=0x7f061c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 -- 172.21.15.1:0/3455101019 shutdown_connections 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 --2- 172.21.15.1:0/3455101019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f061c004800 0x7f061c004c80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 -- 172.21.15.1:0/3455101019 >> 172.21.15.1:0/3455101019 conn(0x7f06340d0b50 msgr2=0x7f06340d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 -- 172.21.15.1:0/3455101019 shutdown_connections 2024-05-09T06:28:40.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.356+0000 7f063c5c1640 1 -- 172.21.15.1:0/3455101019 wait complete. 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.357+0000 7f063c5c1640 1 Processor -- start 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.357+0000 7f063c5c1640 1 -- start start 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 0x7f06341f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 0x7f063406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f063415e7c0 con 0x7f063415a0f0 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0634072b60 con 0x7f061c004800 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063c5c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0634071ce0 con 0x7f06340c4680 2024-05-09T06:28:40.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49684/0 (socket says 172.21.15.1:49684) 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 -- 172.21.15.1:0/2885794512 learned_addr learned my addr 172.21.15.1:0/2885794512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063ab5e640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 0x7f063406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063a35d640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 0x7f06341f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 -- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 msgr2=0x7f063406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 0x7f063406b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 -- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 msgr2=0x7f06341f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 0x7f06341f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f0639b5c640 1 -- 172.21.15.1:0/2885794512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341f4ff0 con 0x7f063415a0f0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063a35d640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 0x7f06341f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.358+0000 7f063ab5e640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 0x7f063406b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f0639b5c640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0628001870 tx=0x7f0628001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062803f030 con 0x7f063415a0f0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0628005480 con 0x7f063415a0f0 2024-05-09T06:28:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06340c63f0 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628037040 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.359+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341f7f20 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.360+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06340752b0 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.360+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0628042070 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.361+0000 7f0632ffd640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 0x7f062006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.361+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f06280aa200 con 0x7f063415a0f0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.361+0000 7f063a35d640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 0x7f062006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.361+0000 7f063a35d640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 0x7f062006e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0634050860 tx=0x7f062c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.365+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0628078ce0 con 0x7f063415a0f0 2024-05-09T06:28:40.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.622+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f06340b1280 con 0x7f063415a0f0 2024-05-09T06:28:40.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.624+0000 7f0632ffd640 1 -- 172.21.15.1:0/2885794512 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f06280785e0 con 0x7f063415a0f0 2024-05-09T06:28:40.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 msgr2=0x7f062006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 0x7f062006e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0634050860 tx=0x7f062c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:40.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 msgr2=0x7f06341f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0628001870 tx=0x7f0628001120 comp rx=0 tx=0).stop 2024-05-09T06:28:40.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 shutdown_connections 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f062006be60 0x7f062006e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06340c4680 0x7f063406b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063415a0f0 0x7f06341f4ab0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 --2- 172.21.15.1:0/2885794512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f061c004800 0x7f06341f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 >> 172.21.15.1:0/2885794512 conn(0x7f06340d0b50 msgr2=0x7f063415b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 shutdown_connections 2024-05-09T06:28:40.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.626+0000 7f063c5c1640 1 -- 172.21.15.1:0/2885794512 wait complete. 2024-05-09T06:28:40.618 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:40.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.741+0000 7fa8f4676640 1 Processor -- start 2024-05-09T06:28:40.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.741+0000 7fa8f4676640 1 -- start start 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f4676640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec15e050 0x7fa8ec15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f4676640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f4676640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8ec0aef90 con 0x7fa8ec06e4f0 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f4676640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec06ee90 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f4676640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec0b3b80 con 0x7fa8ec15e050 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49698/0 (socket says 172.21.15.1:49698) 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 -- 172.21.15.1:0/4033117855 learned_addr learned my addr 172.21.15.1:0/4033117855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2412640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec15e050 0x7fa8ec15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 -- 172.21.15.1:0/4033117855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec15e050 msgr2=0x7fa8ec15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec15e050 0x7fa8ec15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 -- 172.21.15.1:0/4033117855 >> v1:172.21.15.6:6789/0 conn(0x7fa8ec06e4f0 legacy=0x7fa8ec0ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.742+0000 7fa8f2c13640 1 -- 172.21.15.1:0/4033117855 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec0b3530 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.743+0000 7fa8f2c13640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009870 tx=0x7fa8e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa14cec64fd5d2e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.743+0000 7fa8f1410640 1 -- 172.21.15.1:0/4033117855 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4037070 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.743+0000 7fa8f1410640 1 -- 172.21.15.1:0/4033117855 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e4032440 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.743+0000 7fa8f1410640 1 -- 172.21.15.1:0/4033117855 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4032810 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 -- 172.21.15.1:0/4033117855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 msgr2=0x7fa8ec06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009870 tx=0x7fa8e402a270 comp rx=0 tx=0).stop 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 -- 172.21.15.1:0/4033117855 shutdown_connections 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec06e950 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4033117855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec15e050 0x7fa8ec15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 -- 172.21.15.1:0/4033117855 >> 172.21.15.1:0/4033117855 conn(0x7fa8ec06d270 msgr2=0x7fa8ec06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 -- 172.21.15.1:0/4033117855 shutdown_connections 2024-05-09T06:28:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.744+0000 7fa8f4676640 1 -- 172.21.15.1:0/4033117855 wait complete. 2024-05-09T06:28:40.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.745+0000 7fa8f4676640 1 Processor -- start 2024-05-09T06:28:40.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.745+0000 7fa8f4676640 1 -- start start 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec06e4f0 0x7fa8ec0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8ec15e050 0x7fa8ec159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec15eaf0 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec15e970 con 0x7fa8ec15e050 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f4676640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec0b4010 con 0x7fa8ec06e4f0 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49712/0 (socket says 172.21.15.1:49712) 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 -- 172.21.15.1:0/4222291634 learned_addr learned my addr 172.21.15.1:0/4222291634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f2412640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec06e4f0 0x7fa8ec0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f2c13640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8ec15e050 0x7fa8ec159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 -- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec06e4f0 msgr2=0x7fa8ec0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec06e4f0 0x7fa8ec0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 -- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8ec15e050 msgr2=0x7fa8ec159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8ec15e050 0x7fa8ec159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 -- 172.21.15.1:0/4222291634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec0b04b0 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8f1c11640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c430 tx=0x7fa8dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.746+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0052d0 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.747+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc005490 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.747+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc007260 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.747+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec0b0790 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.747+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec0b0d50 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.748+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec0b1130 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.748+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8dc007400 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.748+0000 7fa8e2ffd640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 0x7fa8d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.749+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fa8dc08a1c0 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.749+0000 7fa8f2412640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 0x7fa8d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.749+0000 7fa8f2412640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 0x7fa8d006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80099c0 tx=0x7fa8e8003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.753+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8ec0b1130 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.937+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8ec0752b0 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.938+0000 7fa8e2ffd640 1 -- 172.21.15.1:0/4222291634 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13032 (secure 0 0 0) 0x7fa8dc058d30 con 0x7fa8ec0ae9b0 2024-05-09T06:28:40.919 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:40.919 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":34,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:37.907375+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:40.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 msgr2=0x7fa8d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 0x7fa8d006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80099c0 tx=0x7fa8e8003990 comp rx=0 tx=0).stop 2024-05-09T06:28:40.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 msgr2=0x7fa8ec154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c430 tx=0x7fa8dc00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 shutdown_connections 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8d006be30 0x7fa8d006e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8ec15e050 0x7fa8ec159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec0ae9b0 0x7fa8ec154840 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 --2- 172.21.15.1:0/4222291634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8ec06e4f0 0x7fa8ec0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 >> 172.21.15.1:0/4222291634 conn(0x7fa8ec06d270 msgr2=0x7fa8ec15fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 shutdown_connections 2024-05-09T06:28:40.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:40.940+0000 7fa8f4676640 1 -- 172.21.15.1:0/4222291634 wait complete. 2024-05-09T06:28:40.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:28:40.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:41.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.058+0000 7f6989e7b640 1 Processor -- start 2024-05-09T06:28:41.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.058+0000 7f6989e7b640 1 -- start start 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6989e7b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f698406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6989e7b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698406cf40 con 0x7f698415cfb0 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6989e7b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698406d220 con 0x7f69840b4de0 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6989e7b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6984156980 con 0x7f698415c4c0 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6983fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f698415cfb0 0x7f6984070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44670/0 (socket says 172.21.15.1:44670) 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.059+0000 7f6983fff640 1 -- 172.21.15.1:0/14891866 learned_addr learned my addr 172.21.15.1:0/14891866 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f698406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679336101 0 0) 0x7f698406cf40 con 0x7f698415cfb0 2024-05-09T06:28:41.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f696c003520 con 0x7f698415cfb0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586847587 0 0) 0x7f698406d220 con 0x7f69840b4de0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f698406cf40 con 0x7f69840b4de0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 7089513 0 0) 0x7f696c003520 con 0x7f698415cfb0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 msgr2=0x7f698406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f698406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 >> v1:172.21.15.6:6789/0 conn(0x7f69840b4de0 legacy=0x7f698415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698406d500 con 0x7f698415cfb0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f697c0019f0 con 0x7f698415cfb0 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 >> v1:172.21.15.1:6789/0 conn(0x7f698415cfb0 legacy=0x7f6984070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f696c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f696c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f696c00bb50 con 0x7f696c003b70 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f696c00bcd0 con 0x7f696c004800 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f696c00bea0 con 0x7f698415c4c0 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f696c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f6983fff640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f696c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f6982ffd640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 -- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 msgr2=0x7f696c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f696c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 -- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c004800 msgr2=0x7f696c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f69837fe640 1 -- 172.21.15.1:0/14891866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c003520 con 0x7f696c003b70 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.060+0000 7f6983fff640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415c4c0 0x7f696c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.061+0000 7f69837fe640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f696c003f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6974009af0 tx=0x7f696c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fd89dfc2d88885 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.061+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974036070 con 0x7f696c003b70 2024-05-09T06:28:41.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.061+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6974031400 con 0x7f696c003b70 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.061+0000 7f69827fc640 1 -- 172.21.15.1:0/14891866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974009690 con 0x7f696c003b70 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 -- 172.21.15.1:0/14891866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 msgr2=0x7f696c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 --2- 172.21.15.1:0/14891866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f696c003f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6974009af0 tx=0x7f696c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6983fff640 1 -- 172.21.15.1:0/14891866 reap_dead start 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 -- 172.21.15.1:0/14891866 shutdown_connections 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 -- 172.21.15.1:0/14891866 >> 172.21.15.1:0/14891866 conn(0x7f69840cc4d0 msgr2=0x7f69840cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 -- 172.21.15.1:0/14891866 shutdown_connections 2024-05-09T06:28:41.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.062+0000 7f6989e7b640 1 -- 172.21.15.1:0/14891866 wait complete. 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.063+0000 7f6989e7b640 1 Processor -- start 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.063+0000 7f6989e7b640 1 -- start start 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6984155c20 0x7f69840ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b8c0 0x7f698415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69840b5260 con 0x7f696c003b70 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f698406d870 con 0x7f698415b8c0 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f6989e7b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69840b53e0 con 0x7f6984155c20 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49722/0 (socket says 172.21.15.1:49722) 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 -- 172.21.15.1:0/824947410 learned_addr learned my addr 172.21.15.1:0/824947410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 -- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6984155c20 msgr2=0x7f69840ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6984155c20 0x7f69840ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 -- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b8c0 msgr2=0x7f698415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b8c0 0x7f698415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.064+0000 7f69837fe640 1 -- 172.21.15.1:0/824947410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69840ae3d0 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f69837fe640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6974034040 tx=0x7f6974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974041020 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697403c470 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69840ae670 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974036040 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.065+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69840aec30 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.066+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6984156060 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.066+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6974044020 con 0x7f696c003b70 2024-05-09T06:28:41.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.067+0000 7f696bfff640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 0x7f697006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.067+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f69740aab10 con 0x7f696c003b70 2024-05-09T06:28:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.067+0000 7f6982ffd640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 0x7f697006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.067+0000 7f6982ffd640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 0x7f697006e320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.071+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6974079710 con 0x7f696c003b70 2024-05-09T06:28:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.253+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69840752b0 con 0x7f696c003b70 2024-05-09T06:28:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.254+0000 7f696bfff640 1 -- 172.21.15.1:0/824947410 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f6974079010 con 0x7f696c003b70 2024-05-09T06:28:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 msgr2=0x7f697006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 0x7f697006e320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978009040 comp rx=0 tx=0).stop 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 msgr2=0x7f6984144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6974034040 tx=0x7f6974004730 comp rx=0 tx=0).stop 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 shutdown_connections 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697006be60 0x7f697006e320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b8c0 0x7f698415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6984155c20 0x7f69840ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 --2- 172.21.15.1:0/824947410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c003b70 0x7f6984144700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 >> 172.21.15.1:0/824947410 conn(0x7f69840cc4d0 msgr2=0x7f698415c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 shutdown_connections 2024-05-09T06:28:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.256+0000 7f6989e7b640 1 -- 172.21.15.1:0/824947410 wait complete. 2024-05-09T06:28:41.248 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:28:41.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.371+0000 7f961c0f5640 1 Processor -- start 2024-05-09T06:28:41.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.371+0000 7f961c0f5640 1 -- start start 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.372+0000 7f961c0f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f96140d4940 0x7f96140b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.372+0000 7f961c0f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140d2b10 0x7f96140b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.372+0000 7f961c0f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96140b33c0 con 0x7f9614161950 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.372+0000 7f961c0f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f96140d4dd0 con 0x7f96140d4940 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.372+0000 7f961c0f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96140b3660 con 0x7f96140d2b10 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f961a692640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140d2b10 0x7f96140b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619690640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f96140d4940 0x7f96140b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619e91640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9614161950 0x7f96140d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:42496/0 (socket says 172.21.15.1:42496) 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619e91640 1 -- 172.21.15.1:0/3382961872 learned_addr learned my addr 172.21.15.1:0/3382961872 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:41.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619690640 1 -- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140d2b10 msgr2=0x7f96140b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619690640 1 --2- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140d2b10 0x7f96140b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619690640 1 -- 172.21.15.1:0/3382961872 >> v1:172.21.15.6:6789/0 conn(0x7f9614161950 legacy=0x7f96140d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9619690640 1 -- 172.21.15.1:0/3382961872 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96140b3940 con 0x7f96140d4940 2024-05-09T06:28:41.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.373+0000 7f9618e8f640 1 -- 172.21.15.1:0/3382961872 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437610686 0 0) 0x7f96140b33c0 con 0x7f9614161950 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.374+0000 7f9619690640 1 --2- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d4940 0x7f96140b4140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96080097a0 tx=0x7f960802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28b089fb99a4423c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.374+0000 7f9618e8f640 1 -- 172.21.15.1:0/3382961872 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608037070 con 0x7f96140d4940 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.374+0000 7f9618e8f640 1 -- 172.21.15.1:0/3382961872 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9608032440 con 0x7f96140d4940 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.374+0000 7f9618e8f640 1 -- 172.21.15.1:0/3382961872 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608032810 con 0x7f96140d4940 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 -- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d4940 msgr2=0x7f96140b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 --2- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d4940 0x7f96140b4140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96080097a0 tx=0x7f960802a180 comp rx=0 tx=0).stop 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 -- 172.21.15.1:0/3382961872 shutdown_connections 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 --2- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140d2b10 0x7f96140b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 --2- 172.21.15.1:0/3382961872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d4940 0x7f96140b4140 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 -- 172.21.15.1:0/3382961872 >> 172.21.15.1:0/3382961872 conn(0x7f96140d3e20 msgr2=0x7f96140d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 -- 172.21.15.1:0/3382961872 shutdown_connections 2024-05-09T06:28:41.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.375+0000 7f961c0f5640 1 -- 172.21.15.1:0/3382961872 wait complete. 2024-05-09T06:28:41.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.376+0000 7f961c0f5640 1 Processor -- start 2024-05-09T06:28:41.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.376+0000 7f961c0f5640 1 -- start start 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d2b10 0x7f9614159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140ccfd0 0x7f9614160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96140d1100 con 0x7f9614161950 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96140d0f80 con 0x7f96140d2b10 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96140d1c00 con 0x7f96140ccfd0 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49726/0 (socket says 172.21.15.1:49726) 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 -- 172.21.15.1:0/2574422500 learned_addr learned my addr 172.21.15.1:0/2574422500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619e91640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d2b10 0x7f9614159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961a692640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140ccfd0 0x7f9614160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 -- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140ccfd0 msgr2=0x7f9614160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140ccfd0 0x7f9614160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 -- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d2b10 msgr2=0x7f9614159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d2b10 0x7f9614159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 -- 172.21.15.1:0/2574422500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614159690 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f9619690640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9608000c00 tx=0x7f96080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608037040 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9608042070 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96140cd550 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.377+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608030940 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.378+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96140cdb10 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.379+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96140752b0 con 0x7f9614161950 2024-05-09T06:28:41.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.379+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9608030ae0 con 0x7f9614161950 2024-05-09T06:28:41.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.379+0000 7f96127fc640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 0x7f960006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.379+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f96080ab480 con 0x7f9614161950 2024-05-09T06:28:41.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.379+0000 7f9619e91640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 0x7f960006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.380+0000 7f9619e91640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 0x7f960006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96040012e0 tx=0x7f9604009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.384+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96080799a0 con 0x7f9614161950 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.568+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9614160330 con 0x7f9614161950 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.568+0000 7f96127fc640 1 -- 172.21.15.1:0/2574422500 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f960807dca0 con 0x7f9614161950 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:28:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:28:41.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:41.550 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:28:41.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 msgr2=0x7f960006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 0x7f960006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96040012e0 tx=0x7f9604009040 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 msgr2=0x7f96140cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9608000c00 tx=0x7f96080046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 shutdown_connections 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f960006be30 0x7f960006e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96140ccfd0 0x7f9614160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9614161950 0x7f96140cca90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 --2- 172.21.15.1:0/2574422500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96140d2b10 0x7f9614159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 >> 172.21.15.1:0/2574422500 conn(0x7f96140d3e20 msgr2=0x7f96140ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 shutdown_connections 2024-05-09T06:28:41.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.570+0000 7f961c0f5640 1 -- 172.21.15.1:0/2574422500 wait complete. 2024-05-09T06:28:41.562 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:28:41.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:28:41.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.689+0000 7f696f321640 1 Processor -- start 2024-05-09T06:28:41.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.689+0000 7f696f321640 1 -- start start 2024-05-09T06:28:41.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696f321640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69681446d0 0x7f696806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696f321640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69680cdd50 con 0x7f69680d03c0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696f321640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69680ce030 con 0x7f6968143bb0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696f321640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f69680ce8d0 con 0x7f69681446d0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696c8bc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69680d03c0 0x7f696806d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44676/0 (socket says 172.21.15.1:44676) 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.690+0000 7f696c8bc640 1 -- 172.21.15.1:0/1802068810 learned_addr learned my addr 172.21.15.1:0/1802068810 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696d8be640 1 --2- 172.21.15.1:0/1802068810 >> v2:172.21.15.6:3300/0 conn(0x7f69681446d0 0x7f696806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929183088 0 0) 0x7f69680cdd50 con 0x7f69680d03c0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6950003520 con 0x7f69680d03c0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 861030760 0 0) 0x7f6950003520 con 0x7f69680d03c0 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 >> v2:172.21.15.6:3300/0 conn(0x7f69681446d0 msgr2=0x7f696806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 --2- 172.21.15.1:0/1802068810 >> v2:172.21.15.6:3300/0 conn(0x7f69681446d0 0x7f696806da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 >> v1:172.21.15.6:6789/0 conn(0x7f6968143bb0 legacy=0x7f69680cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69680ce310 con 0x7f69680d03c0 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f695c001670 con 0x7f69680d03c0 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 >> v1:172.21.15.1:6789/0 conn(0x7f69680d03c0 legacy=0x7f696806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500039b0 0x7f6950003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 0x7f6950004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6950005120 0x7f695000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69680ce8d0 con 0x7f69500043b0 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f695000c060 con 0x7f6950005120 2024-05-09T06:28:41.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f695000c260 con 0x7f69500039b0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 0x7f6950004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696d8be640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6950005120 0x7f695000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696d0bd640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500039b0 0x7f6950003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 -- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500039b0 msgr2=0x7f6950003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500039b0 0x7f6950003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 -- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6950005120 msgr2=0x7f695000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6950005120 0x7f695000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696d8be640 1 -- 172.21.15.1:0/1802068810 reap_dead start 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.691+0000 7f696c8bc640 1 -- 172.21.15.1:0/1802068810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6950003520 con 0x7f69500043b0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.692+0000 7f696d0bd640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500039b0 0x7f6950003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.692+0000 7f696c8bc640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 0x7f6950004880 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c240 tx=0x7f6950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e5fdcf735fa130 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.692+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c007d10 con 0x7f69500043b0 2024-05-09T06:28:41.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.692+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c005ad0 con 0x7f69500043b0 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.692+0000 7f6967fff640 1 -- 172.21.15.1:0/1802068810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c007070 con 0x7f69500043b0 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 -- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 msgr2=0x7f6950004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 0x7f6950004880 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c240 tx=0x7f6950005830 comp rx=0 tx=0).stop 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 -- 172.21.15.1:0/1802068810 shutdown_connections 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 --2- 172.21.15.1:0/1802068810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500043b0 0x7f6950004880 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 -- 172.21.15.1:0/1802068810 >> 172.21.15.1:0/1802068810 conn(0x7f696806b3c0 msgr2=0x7f696806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:41.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 -- 172.21.15.1:0/1802068810 shutdown_connections 2024-05-09T06:28:41.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.693+0000 7f696f321640 1 -- 172.21.15.1:0/1802068810 wait complete. 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.694+0000 7f696f321640 1 Processor -- start 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.694+0000 7f696f321640 1 -- start start 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 0x7f6968150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 0x7f69680b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69680cdaf0 con 0x7f696814f100 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69680cd970 con 0x7f69500043b0 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696f321640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6968144110 con 0x7f69680b38a0 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49734/0 (socket says 172.21.15.1:49734) 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 -- 172.21.15.1:0/3586073623 learned_addr learned my addr 172.21.15.1:0/3586073623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696d8be640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 0x7f69680b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696d0bd640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 0x7f6968150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 -- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 msgr2=0x7f69680b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 0x7f69680b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 -- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 msgr2=0x7f6968150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 0x7f6968150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.695+0000 7f696c8bc640 1 -- 172.21.15.1:0/3586073623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681511c0 con 0x7f696814f100 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f696d8be640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 0x7f69680b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f696d0bd640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 0x7f6968150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f696c8bc640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c360 tx=0x7f69680ce200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c037040 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c001470 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0049e0 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6968150920 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.696+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6968150ee0 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.697+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69680b4260 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.698+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f695c004b80 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.698+0000 7f6965ffb640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 0x7f695406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.698+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f695c0aa610 con 0x7f696814f100 2024-05-09T06:28:41.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.698+0000 7f696d0bd640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 0x7f695406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.699+0000 7f696d0bd640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 0x7f695406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6968050860 tx=0x7f6958009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:41.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.703+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f695c079210 con 0x7f696814f100 2024-05-09T06:28:41.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:41.884+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f69680cf6b0 con 0x7f696814f100 2024-05-09T06:28:42.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.077+0000 7f6965ffb640 1 -- 172.21.15.1:0/3586073623 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f695c078b10 con 0x7f696814f100 2024-05-09T06:28:42.059 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:28:42.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 msgr2=0x7f695406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 0x7f695406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6968050860 tx=0x7f6958009040 comp rx=0 tx=0).stop 2024-05-09T06:28:42.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 msgr2=0x7f6968136b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c360 tx=0x7f69680ce200 comp rx=0 tx=0).stop 2024-05-09T06:28:42.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 shutdown_connections 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f695406be30 0x7f695406e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69680b38a0 0x7f69680b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696814f100 0x7f6968136b20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 --2- 172.21.15.1:0/3586073623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69500043b0 0x7f6968150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 >> 172.21.15.1:0/3586073623 conn(0x7f696806b3c0 msgr2=0x7f69680cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 shutdown_connections 2024-05-09T06:28:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.080+0000 7f696f321640 1 -- 172.21.15.1:0/3586073623 wait complete. 2024-05-09T06:28:42.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:28:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.208+0000 7f54d9658640 1 Processor -- start 2024-05-09T06:28:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.208+0000 7f54d9658640 1 -- start start 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.208+0000 7f54d9658640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54d415e210 0x7f54d40d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.208+0000 7f54d9658640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 0x7f54d40d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d9658640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d40d2060 con 0x7f54d40d0fb0 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d9658640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f54d40b3170 con 0x7f54d415e210 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d9658640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f54d40d4da0 con 0x7f54d40cfa90 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54d415e210 0x7f54d40d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54d415e210 0x7f54d40d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49750/0 (socket says 172.21.15.1:49750) 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54cbfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f54d40d0fb0 0x7f54d40b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44684/0 (socket says 172.21.15.1:44684) 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 -- 172.21.15.1:0/2188163875 learned_addr learned my addr 172.21.15.1:0/2188163875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d37fe640 1 --2- 172.21.15.1:0/2188163875 >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 0x7f54d40d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 -- 172.21.15.1:0/2188163875 >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 msgr2=0x7f54d40d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/2188163875 >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 0x7f54d40d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 -- 172.21.15.1:0/2188163875 >> v1:172.21.15.1:6789/0 conn(0x7f54d40d0fb0 legacy=0x7f54d40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d27fc640 1 -- 172.21.15.1:0/2188163875 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533236424 0 0) 0x7f54d40d2060 con 0x7f54d40d0fb0 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d2ffd640 1 -- 172.21.15.1:0/2188163875 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d40d4ac0 con 0x7f54d415e210 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.209+0000 7f54d37fe640 1 --2- 172.21.15.1:0/2188163875 >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 0x7f54d40d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.210+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/2188163875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d40d22a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f54d408c690 tx=0x7f54c402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be827b1356b55ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.210+0000 7f54d27fc640 1 -- 172.21.15.1:0/2188163875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4037070 con 0x7f54d415e210 2024-05-09T06:28:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.210+0000 7f54d27fc640 1 -- 172.21.15.1:0/2188163875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4032440 con 0x7f54d415e210 2024-05-09T06:28:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.210+0000 7f54d27fc640 1 -- 172.21.15.1:0/2188163875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4032810 con 0x7f54d415e210 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 -- 172.21.15.1:0/2188163875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 msgr2=0x7f54d40d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 --2- 172.21.15.1:0/2188163875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d40d22a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f54d408c690 tx=0x7f54c402a150 comp rx=0 tx=0).stop 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 -- 172.21.15.1:0/2188163875 shutdown_connections 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 --2- 172.21.15.1:0/2188163875 >> v2:172.21.15.6:3300/0 conn(0x7f54d40cfa90 0x7f54d40d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 --2- 172.21.15.1:0/2188163875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d40d22a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 -- 172.21.15.1:0/2188163875 >> 172.21.15.1:0/2188163875 conn(0x7f54d40d3e70 msgr2=0x7f54d40d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 -- 172.21.15.1:0/2188163875 shutdown_connections 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.211+0000 7f54d9658640 1 -- 172.21.15.1:0/2188163875 wait complete. 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.212+0000 7f54d9658640 1 Processor -- start 2024-05-09T06:28:42.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.212+0000 7f54d9658640 1 -- start start 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 0x7f54d40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 0x7f54d41f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54d415e5f0 con 0x7f54d415e210 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f54d40b2e40 con 0x7f54d40d0fb0 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54d415e770 con 0x7f54d40cfa90 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49752/0 (socket says 172.21.15.1:49752) 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 -- 172.21.15.1:0/3260076556 learned_addr learned my addr 172.21.15.1:0/3260076556 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54cbfff640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 0x7f54d41f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 0x7f54d40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 -- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 msgr2=0x7f54d40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 0x7f54d40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 -- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 msgr2=0x7f54d41f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 0x7f54d41f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 -- 172.21.15.1:0/3260076556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d40ca6f0 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54cbfff640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 0x7f54d41f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 0x7f54d40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d37fe640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00cd50 tx=0x7f54cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00f680 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc00f840 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d40cd430 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.213+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00fc30 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.214+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d40cd9f0 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.214+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d40752b0 con 0x7f54d415e210 2024-05-09T06:28:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.215+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54cc0213f0 con 0x7f54d415e210 2024-05-09T06:28:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.216+0000 7f54cb7fe640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.216+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f54cc08bba0 con 0x7f54d415e210 2024-05-09T06:28:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.216+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.216+0000 7f54d2ffd640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 0x7f54b406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c4034000 tx=0x7f54c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.220+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54cc014030 con 0x7f54d415e210 2024-05-09T06:28:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:42.405+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f54d40d2060 con 0x7f54d415e210 2024-05-09T06:28:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.083+0000 7f54cb7fe640 1 -- 172.21.15.1:0/3260076556 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f54cc05a630 con 0x7f54d415e210 2024-05-09T06:28:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:28:43.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.087+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 msgr2=0x7f54b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.087+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 0x7f54b406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c4034000 tx=0x7f54c402e000 comp rx=0 tx=0).stop 2024-05-09T06:28:43.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.087+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 msgr2=0x7f54d41f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.087+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00cd50 tx=0x7f54cc00a650 comp rx=0 tx=0).stop 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 shutdown_connections 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54b406be30 0x7f54b406e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d415e210 0x7f54d41f3bc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54d40d0fb0 0x7f54d41f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 --2- 172.21.15.1:0/3260076556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54d40cfa90 0x7f54d40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.089+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 >> 172.21.15.1:0/3260076556 conn(0x7f54d40d3e70 msgr2=0x7f54d40d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:43.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.092+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 shutdown_connections 2024-05-09T06:28:43.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.093+0000 7f54d9658640 1 -- 172.21.15.1:0/3260076556 wait complete. 2024-05-09T06:28:43.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:28:43.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.228+0000 7f4bccad4640 1 Processor -- start 2024-05-09T06:28:43.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.228+0000 7f4bccad4640 1 -- start start 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bccad4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4bc806dd00 0x7f4bc815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bccad4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4bc815cde0 0x7f4bc8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bccad4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc806cf40 con 0x7f4bc815c4c0 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bccad4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc8156980 con 0x7f4bc815cde0 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bccad4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc806e180 con 0x7f4bc806dd00 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4bc815cde0 0x7f4bc8070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4bc815cde0 0x7f4bc8070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49762/0 (socket says 172.21.15.1:49762) 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 -- 172.21.15.1:0/4198203863 learned_addr learned my addr 172.21.15.1:0/4198203863 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bbffff640 1 --2- 172.21.15.1:0/4198203863 >> v2:172.21.15.6:3300/0 conn(0x7f4bc806dd00 0x7f4bc815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 -- 172.21.15.1:0/4198203863 >> v2:172.21.15.6:3300/0 conn(0x7f4bc806dd00 msgr2=0x7f4bc815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 --2- 172.21.15.1:0/4198203863 >> v2:172.21.15.6:3300/0 conn(0x7f4bc806dd00 0x7f4bc815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 -- 172.21.15.1:0/4198203863 >> v1:172.21.15.6:6789/0 conn(0x7f4bc815c4c0 legacy=0x7f4bc80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.229+0000 7f4bc6d9d640 1 -- 172.21.15.1:0/4198203863 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc806d220 con 0x7f4bc815cde0 2024-05-09T06:28:43.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.230+0000 7f4bc6d9d640 1 --2- 172.21.15.1:0/4198203863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815cde0 0x7f4bc8070c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80cd190 tx=0x7f4bc002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ed6fe798e6768b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.230+0000 7f4bc5d9b640 1 -- 172.21.15.1:0/4198203863 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0036070 con 0x7f4bc815cde0 2024-05-09T06:28:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.230+0000 7f4bc5d9b640 1 -- 172.21.15.1:0/4198203863 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc0031600 con 0x7f4bc815cde0 2024-05-09T06:28:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.230+0000 7f4bc5d9b640 1 -- 172.21.15.1:0/4198203863 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00319d0 con 0x7f4bc815cde0 2024-05-09T06:28:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 -- 172.21.15.1:0/4198203863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815cde0 msgr2=0x7f4bc8070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 --2- 172.21.15.1:0/4198203863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815cde0 0x7f4bc8070c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80cd190 tx=0x7f4bc002a730 comp rx=0 tx=0).stop 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 -- 172.21.15.1:0/4198203863 shutdown_connections 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 --2- 172.21.15.1:0/4198203863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815cde0 0x7f4bc8070c10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 --2- 172.21.15.1:0/4198203863 >> v2:172.21.15.6:3300/0 conn(0x7f4bc806dd00 0x7f4bc815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 -- 172.21.15.1:0/4198203863 >> 172.21.15.1:0/4198203863 conn(0x7f4bc80cc4d0 msgr2=0x7f4bc80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 -- 172.21.15.1:0/4198203863 shutdown_connections 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.231+0000 7f4bccad4640 1 -- 172.21.15.1:0/4198203863 wait complete. 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.232+0000 7f4bccad4640 1 Processor -- start 2024-05-09T06:28:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.232+0000 7f4bccad4640 1 -- start start 2024-05-09T06:28:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc806dd00 0x7f4bc80acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bc815cde0 0x7f4bc81436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc815d270 con 0x7f4bc815c4c0 2024-05-09T06:28:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc806d870 con 0x7f4bc806dd00 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bccad4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc80b54f0 con 0x7f4bc815cde0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49766/0 (socket says 172.21.15.1:49766) 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bbffff640 1 -- 172.21.15.1:0/2739613969 learned_addr learned my addr 172.21.15.1:0/2739613969 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bc6d9d640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bc815cde0 0x7f4bc81436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.233+0000 7f4bc659c640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc806dd00 0x7f4bc80acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 -- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bc815cde0 msgr2=0x7f4bc81436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bc815cde0 0x7f4bc81436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 -- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc806dd00 msgr2=0x7f4bc80acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc806dd00 0x7f4bc80acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 -- 172.21.15.1:0/2739613969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc8144040 con 0x7f4bc815c4c0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbffff640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e9e0 tx=0x7f4bb000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0017070 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb000c9d0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc80abe20 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000cdc0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.234+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc80ac3e0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.235+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc8067ac0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.235+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.236+0000 7f4bbeffd640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 0x7f4ba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.236+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f4bb008afc0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.236+0000 7f4bc659c640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 0x7f4ba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.236+0000 7f4bc659c640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 0x7f4ba806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007fd0 tx=0x7f4bb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.241+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4bc8067ac0 con 0x7f4bc815c4c0 2024-05-09T06:28:43.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:43.461+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4bc80b3330 con 0x7f4bc815c4c0 2024-05-09T06:28:44.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.085+0000 7f4bbeffd640 1 -- 172.21.15.1:0/2739613969 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4bc80b3330 con 0x7f4bc815c4c0 2024-05-09T06:28:44.066 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 5 and data pool 6 2024-05-09T06:28:44.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.089+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 msgr2=0x7f4ba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.089+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 0x7f4ba806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007fd0 tx=0x7f4bb8009040 comp rx=0 tx=0).stop 2024-05-09T06:28:44.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.089+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 msgr2=0x7f4bc80aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.089+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e9e0 tx=0x7f4bb000eea0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 shutdown_connections 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ba806be30 0x7f4ba806e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bc815cde0 0x7f4bc81436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bc815c4c0 0x7f4bc80aeb80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 --2- 172.21.15.1:0/2739613969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bc806dd00 0x7f4bc80acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 >> 172.21.15.1:0/2739613969 conn(0x7f4bc80cc4d0 msgr2=0x7f4bc80af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 shutdown_connections 2024-05-09T06:28:44.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.090+0000 7f4bccad4640 1 -- 172.21.15.1:0/2739613969 wait complete. 2024-05-09T06:28:44.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:44.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.234+0000 7f05437c1640 1 Processor -- start 2024-05-09T06:28:44.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.234+0000 7f05437c1640 1 -- start start 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f05437c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f053c099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f05437c1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f053c15e4b0 0x7f053c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f05437c1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f053c099930 con 0x7f053c0b2aa0 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f05437c1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f053c0b1e20 con 0x7f053c15e4b0 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f05437c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f053c15ee90 con 0x7f053c0c6860 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0540d5c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f053c0b2aa0 0x7f053c0c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44686/0 (socket says 172.21.15.1:44686) 2024-05-09T06:28:44.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0540d5c640 1 -- 172.21.15.1:0/3987346268 learned_addr learned my addr 172.21.15.1:0/3987346268 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944589927 0 0) 0x7f052c001110 con 0x7f053c0b2aa0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0528003520 con 0x7f053c0b2aa0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0541d5e640 1 --2- 172.21.15.1:0/3987346268 >> v2:172.21.15.6:3300/0 conn(0x7f053c15e4b0 0x7f053c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f054155d640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f053c099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2337148805 0 0) 0x7f0528003520 con 0x7f053c0b2aa0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 msgr2=0x7f053c099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f053c099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 >> v2:172.21.15.6:3300/0 conn(0x7f053c15e4b0 msgr2=0x7f053c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 --2- 172.21.15.1:0/3987346268 >> v2:172.21.15.6:3300/0 conn(0x7f053c15e4b0 0x7f053c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c162930 con 0x7f053c0b2aa0 2024-05-09T06:28:44.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f052c001840 con 0x7f053c0b2aa0 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 >> v1:172.21.15.1:6789/0 conn(0x7f053c0b2aa0 legacy=0x7f053c0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.235+0000 7f0533fff640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f0528004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0533fff640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0528004d40 0x7f05280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f053c15ee90 con 0x7f0528004d40 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f052800b9c0 con 0x7f05280039b0 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f052800bb40 con 0x7f053c0c6860 2024-05-09T06:28:44.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0528004d40 0x7f05280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f054155d640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0540d5c640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f0528004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 -- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 msgr2=0x7f0528004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0c6860 0x7f0528004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 -- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05280039b0 msgr2=0x7f0528003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.236+0000 7f0541d5e640 1 -- 172.21.15.1:0/3987346268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0528003520 con 0x7f0528004d40 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f0541d5e640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0528004d40 0x7f05280051c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0534009fc0 tx=0x7f0528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9070d24cfc3c90f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534009c10 con 0x7f0528004d40 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0534036070 con 0x7f0528004d40 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f0533fff640 1 -- 172.21.15.1:0/3987346268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053402d070 con 0x7f0528004d40 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 -- 172.21.15.1:0/3987346268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0528004d40 msgr2=0x7f05280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 --2- 172.21.15.1:0/3987346268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0528004d40 0x7f05280051c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0534009fc0 tx=0x7f0528005830 comp rx=0 tx=0).stop 2024-05-09T06:28:44.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f0541d5e640 1 -- 172.21.15.1:0/3987346268 reap_dead start 2024-05-09T06:28:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 -- 172.21.15.1:0/3987346268 shutdown_connections 2024-05-09T06:28:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 -- 172.21.15.1:0/3987346268 >> 172.21.15.1:0/3987346268 conn(0x7f053c0b1fb0 msgr2=0x7f053c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 -- 172.21.15.1:0/3987346268 shutdown_connections 2024-05-09T06:28:44.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.237+0000 7f05437c1640 1 -- 172.21.15.1:0/3987346268 wait complete. 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 Processor -- start 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 -- start start 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528004d40 0x7f053c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0b35e0 0x7f053c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f053c15f860 con 0x7f053c1594f0 2024-05-09T06:28:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f053c15eb20 con 0x7f0528004d40 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.239+0000 7f05437c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f053c15f9e0 con 0x7f053c0b35e0 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49778/0 (socket says 172.21.15.1:49778) 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 -- 172.21.15.1:0/51738604 learned_addr learned my addr 172.21.15.1:0/51738604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0541d5e640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0b35e0 0x7f053c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f054155d640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528004d40 0x7f053c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 -- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0b35e0 msgr2=0x7f053c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0b35e0 0x7f053c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 -- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528004d40 msgr2=0x7f053c0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528004d40 0x7f053c0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 -- 172.21.15.1:0/51738604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c0b3fb0 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0540d5c640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f052c007310 tx=0x7f052c007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c00f960 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c018070 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c0b4290 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c00fd40 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.240+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c0cb680 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.241+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c074010 con 0x7f053c1594f0 2024-05-09T06:28:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.242+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f052c01d460 con 0x7f053c1594f0 2024-05-09T06:28:44.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.242+0000 7f0531ffb640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 0x7f052006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.243+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f052c08afe0 con 0x7f053c1594f0 2024-05-09T06:28:44.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.243+0000 7f054155d640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 0x7f052006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.243+0000 7f054155d640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 0x7f052006e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0538009b40 tx=0x7f0538009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.247+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f053c074010 con 0x7f053c1594f0 2024-05-09T06:28:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.428+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f053c0752b0 con 0x7f053c1594f0 2024-05-09T06:28:44.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.430+0000 7f0531ffb640 1 -- 172.21.15.1:0/51738604 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16823 (secure 0 0 0) 0x7f053c099930 con 0x7f053c1594f0 2024-05-09T06:28:44.411 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:44.411 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:43.461949+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 msgr2=0x7f052006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 0x7f052006e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0538009b40 tx=0x7f0538009190 comp rx=0 tx=0).stop 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 msgr2=0x7f053c0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f052c007310 tx=0x7f052c007710 comp rx=0 tx=0).stop 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 shutdown_connections 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f052006be60 0x7f052006e320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f053c0b35e0 0x7f053c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1594f0 0x7f053c0b30a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 --2- 172.21.15.1:0/51738604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528004d40 0x7f053c0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 >> 172.21.15.1:0/51738604 conn(0x7f053c0b1fb0 msgr2=0x7f053c0c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 shutdown_connections 2024-05-09T06:28:44.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.432+0000 7f05437c1640 1 -- 172.21.15.1:0/51738604 wait complete. 2024-05-09T06:28:44.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.547+0000 7fa8c3ebd640 1 Processor -- start 2024-05-09T06:28:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.547+0000 7fa8c3ebd640 1 -- start start 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c3ebd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8bc0c6860 0x7fa8bc099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c3ebd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc099930 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c3ebd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc162930 con 0x7fa8bc0b2aa0 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c3ebd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0b1e20 con 0x7fa8bc0c6860 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c1c59640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8bc0c6860 0x7fa8bc099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c245a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa8bc15e4b0 0x7fa8bc15f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44702/0 (socket says 172.21.15.1:44702) 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c245a640 1 -- 172.21.15.1:0/2964274101 learned_addr learned my addr 172.21.15.1:0/2964274101 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c1c59640 1 -- 172.21.15.1:0/2964274101 >> v1:172.21.15.6:6789/0 conn(0x7fa8bc0b2aa0 legacy=0x7fa8bc0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c1c59640 1 -- 172.21.15.1:0/2964274101 >> v1:172.21.15.1:6789/0 conn(0x7fa8bc15e4b0 legacy=0x7fa8bc15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.548+0000 7fa8c1c59640 1 -- 172.21.15.1:0/2964274101 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc162c10 con 0x7fa8bc0c6860 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c1c59640 1 --2- 172.21.15.1:0/2964274101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc0c6860 0x7fa8bc099b30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e657f59d99bf42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c0c57640 1 -- 172.21.15.1:0/2964274101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8037070 con 0x7fa8bc0c6860 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c0c57640 1 -- 172.21.15.1:0/2964274101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8032440 con 0x7fa8bc0c6860 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c0c57640 1 -- 172.21.15.1:0/2964274101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8032810 con 0x7fa8bc0c6860 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/2964274101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc0c6860 msgr2=0x7fa8bc099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.549+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/2964274101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc0c6860 0x7fa8bc099b30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80097a0 tx=0x7fa8b802a230 comp rx=0 tx=0).stop 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.550+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/2964274101 shutdown_connections 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.550+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/2964274101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc0c6860 0x7fa8bc099b30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.550+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/2964274101 >> 172.21.15.1:0/2964274101 conn(0x7fa8bc0b1fb0 msgr2=0x7fa8bc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.550+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/2964274101 shutdown_connections 2024-05-09T06:28:44.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.550+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/2964274101 wait complete. 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.551+0000 7fa8c3ebd640 1 Processor -- start 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.551+0000 7fa8c3ebd640 1 -- start start 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.551+0000 7fa8c3ebd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8bc0b2aa0 0x7fa8bc0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 0x7fa8bc156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc15fa30 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc15ee90 con 0x7fa8bc0b2aa0 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0cdf50 con 0x7fa8bc0c6860 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49796/0 (socket says 172.21.15.1:49796) 2024-05-09T06:28:44.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 -- 172.21.15.1:0/4066650064 learned_addr learned my addr 172.21.15.1:0/4066650064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c1c59640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8bc0b2aa0 0x7fa8bc0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c1458640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 0x7fa8bc156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 -- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 msgr2=0x7fa8bc156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 0x7fa8bc156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 -- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8bc0b2aa0 msgr2=0x7fa8bc0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8bc0b2aa0 0x7fa8bc0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 -- 172.21.15.1:0/4066650064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1566b0 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c1458640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 0x7fa8bc156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c245a640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400ca80 tx=0x7fa8b400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4017070 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b400ac30 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4012600 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc0b3640 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.552+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc0b3c00 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.554+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc0752b0 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.554+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8b401a070 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.555+0000 7fa8b27fc640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.555+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fa8b408abc0 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.555+0000 7fa8c1c59640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.555+0000 7fa8c1c59640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 0x7fa8a006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8034000 tx=0x7fa8b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.559+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8bc099930 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.742+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8bc099ba0 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.743+0000 7fa8b27fc640 1 -- 172.21.15.1:0/4066650064 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16823 (secure 0 0 0) 0x7fa8b400ae20 con 0x7fa8bc15e4b0 2024-05-09T06:28:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:44.725 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:43.461949+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:44.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 msgr2=0x7fa8a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 0x7fa8a006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8034000 tx=0x7fa8b802e000 comp rx=0 tx=0).stop 2024-05-09T06:28:44.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 msgr2=0x7fa8bc1616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400ca80 tx=0x7fa8b400cf40 comp rx=0 tx=0).stop 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 shutdown_connections 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8bc15e4b0 0x7fa8bc1616d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8bc0c6860 0x7fa8bc156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 --2- 172.21.15.1:0/4066650064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8bc0b2aa0 0x7fa8bc0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.745+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 >> 172.21.15.1:0/4066650064 conn(0x7fa8bc0b1fb0 msgr2=0x7fa8bc0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.746+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 shutdown_connections 2024-05-09T06:28:44.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.746+0000 7fa8c3ebd640 1 -- 172.21.15.1:0/4066650064 wait complete. 2024-05-09T06:28:44.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:28:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.867+0000 7f3f204bd640 1 Processor -- start 2024-05-09T06:28:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.867+0000 7f3f204bd640 1 -- start start 2024-05-09T06:28:44.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f204bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f180cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f204bd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3f180cf990 0x7f3f180b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f204bd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1815e900 con 0x7f3f180b1730 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f204bd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3f180b1ba0 con 0x7f3f180cf990 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f204bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1815ebe0 con 0x7f3f1815e390 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1ea5a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3f180b1730 0x7f3f18072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44706/0 (socket says 172.21.15.1:44706) 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1ea5a640 1 -- 172.21.15.1:0/3041455268 learned_addr learned my addr 172.21.15.1:0/3041455268 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1e259640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f180cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1da58640 1 --2- 172.21.15.1:0/3041455268 >> v2:172.21.15.6:3300/0 conn(0x7f3f180cf990 0x7f3f180b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822183500 0 0) 0x7f3f1815e900 con 0x7f3f180b1730 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f04003520 con 0x7f3f180b1730 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1452176120 0 0) 0x7f3f04003520 con 0x7f3f180b1730 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 msgr2=0x7f3f180cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f180cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 >> v2:172.21.15.6:3300/0 conn(0x7f3f180cf990 msgr2=0x7f3f180b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 --2- 172.21.15.1:0/3041455268 >> v2:172.21.15.6:3300/0 conn(0x7f3f180cf990 0x7f3f180b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1815eec0 con 0x7f3f180b1730 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3f100019f0 con 0x7f3f180b1730 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 >> v1:172.21.15.1:6789/0 conn(0x7f3f180b1730 legacy=0x7f3f18072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f04004800 0x7f3f04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180b1ba0 con 0x7f3f040039b0 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815ebe0 con 0x7f3f04004800 2024-05-09T06:28:44.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.868+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0400b9c0 con 0x7f3f1815e390 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1da58640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f04004800 0x7f3f04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1ea5a640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 -- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 msgr2=0x7f3f040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f1815e390 0x7f3f040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 -- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f04004800 msgr2=0x7f3f04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f04004800 0x7f3f04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.869+0000 7f3f1e259640 1 -- 172.21.15.1:0/3041455268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f04003520 con 0x7f3f040039b0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f1e259640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f04003dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009ac0 tx=0x7f3f04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95b14790fd090e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14036070 con 0x7f3f040039b0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f14031400 con 0x7f3f040039b0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f1d257640 1 -- 172.21.15.1:0/3041455268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14009690 con 0x7f3f040039b0 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 -- 172.21.15.1:0/3041455268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 msgr2=0x7f3f04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 --2- 172.21.15.1:0/3041455268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f04003dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009ac0 tx=0x7f3f04005830 comp rx=0 tx=0).stop 2024-05-09T06:28:44.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f1ea5a640 1 -- 172.21.15.1:0/3041455268 reap_dead start 2024-05-09T06:28:44.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 -- 172.21.15.1:0/3041455268 shutdown_connections 2024-05-09T06:28:44.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 -- 172.21.15.1:0/3041455268 >> 172.21.15.1:0/3041455268 conn(0x7f3f180b1da0 msgr2=0x7f3f180b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:44.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 -- 172.21.15.1:0/3041455268 shutdown_connections 2024-05-09T06:28:44.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.870+0000 7f3f204bd640 1 -- 172.21.15.1:0/3041455268 wait complete. 2024-05-09T06:28:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 Processor -- start 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 -- start start 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f180c5ca0 0x7f3f181e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f180b3fd0 0x7f3f180b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180cd8f0 con 0x7f3f040039b0 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815f6c0 con 0x7f3f180b3fd0 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f204bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f18162ba0 con 0x7f3f180c5ca0 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f1e259640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f1e259640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49820/0 (socket says 172.21.15.1:49820) 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.872+0000 7f3f1e259640 1 -- 172.21.15.1:0/2593538531 learned_addr learned my addr 172.21.15.1:0/2593538531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1da58640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f180c5ca0 0x7f3f181e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1ea5a640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f180b3fd0 0x7f3f180b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 -- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f180c5ca0 msgr2=0x7f3f181e7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f180c5ca0 0x7f3f181e7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 -- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f180b3fd0 msgr2=0x7f3f180b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f180b3fd0 0x7f3f180b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 -- 172.21.15.1:0/2593538531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180c4c00 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f1e259640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009be0 tx=0x7f3f14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14040020 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f14003bc0 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14036040 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f180c4ee0 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.873+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f180c54a0 con 0x7f3f040039b0 2024-05-09T06:28:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.874+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f180752b0 con 0x7f3f040039b0 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.875+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f14043020 con 0x7f3f040039b0 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.875+0000 7f3f0effd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.875+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f3f140ab140 con 0x7f3f040039b0 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.875+0000 7f3f1da58640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.875+0000 7f3f1da58640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 0x7f3efc06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3f181f7b10 tx=0x7f3f08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:44.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:44.879+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f1404a070 con 0x7f3f040039b0 2024-05-09T06:28:45.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:45.104+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3f180b2ae0 con 0x7f3f040039b0 2024-05-09T06:28:46.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.107+0000 7f3f0effd640 1 -- 172.21.15.1:0/2593538531 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f3f140044e0 con 0x7f3f040039b0 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 msgr2=0x7f3efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 0x7f3efc06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3f181f7b10 tx=0x7f3f08008040 comp rx=0 tx=0).stop 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 msgr2=0x7f3f1814bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009be0 tx=0x7f3f14004730 comp rx=0 tx=0).stop 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 shutdown_connections 2024-05-09T06:28:46.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f180b3fd0 0x7f3f180b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f180c5ca0 0x7f3f181e7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 --2- 172.21.15.1:0/2593538531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f040039b0 0x7f3f1814bb90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 >> 172.21.15.1:0/2593538531 conn(0x7f3f180b1da0 msgr2=0x7f3f18161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 shutdown_connections 2024-05-09T06:28:46.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.111+0000 7f3f204bd640 1 -- 172.21.15.1:0/2593538531 wait complete. 2024-05-09T06:28:46.107 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:28:46.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:28:46.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.239+0000 7fc3d8bce640 1 Processor -- start 2024-05-09T06:28:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.240+0000 7fc3d8bce640 1 -- start start 2024-05-09T06:28:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.240+0000 7fc3d8bce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc3d4140e10 0x7fc3d4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d8bce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3d40ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d8bce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d40ae7f0 con 0x7fc3d4071440 2024-05-09T06:28:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d8bce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d40aead0 con 0x7fc3d4140e10 2024-05-09T06:28:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d8bce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d40aedb0 con 0x7fc3d415f1d0 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d1d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc3d4071440 0x7fc3d415eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44710/0 (socket says 172.21.15.1:44710) 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d1d9b640 1 -- 172.21.15.1:0/588898073 learned_addr learned my addr 172.21.15.1:0/588898073 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3d40ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d259c640 1 --2- 172.21.15.1:0/588898073 >> v2:172.21.15.6:3300/0 conn(0x7fc3d4140e10 0x7fc3d4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769095563 0 0) 0x7fc3d40ae7f0 con 0x7fc3d4071440 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c0003520 con 0x7fc3d4071440 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3495507705 0 0) 0x7fc3c0003520 con 0x7fc3d4071440 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 msgr2=0x7fc3d40ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3d40ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 >> v2:172.21.15.6:3300/0 conn(0x7fc3d4140e10 msgr2=0x7fc3d4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 --2- 172.21.15.1:0/588898073 >> v2:172.21.15.6:3300/0 conn(0x7fc3d4140e10 0x7fc3d4070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d40af090 con 0x7fc3d4071440 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fc3bc001670 con 0x7fc3d4071440 2024-05-09T06:28:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 >> v1:172.21.15.1:6789/0 conn(0x7fc3d4071440 legacy=0x7fc3d415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3c00039b0 0x7fc3c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d40aead0 con 0x7fc3c0004d40 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d40aedb0 con 0x7fc3c00039b0 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.241+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c000b9c0 con 0x7fc3d415f1d0 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d1d9b640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d259c640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3c00039b0 0x7fc3c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 -- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 msgr2=0x7fc3c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d415f1d0 0x7fc3c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 -- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3c00039b0 msgr2=0x7fc3c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3c00039b0 0x7fc3c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.242+0000 7fc3d2d9d640 1 -- 172.21.15.1:0/588898073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0003520 con 0x7fc3c0004d40 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3c00051c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009fc0 tx=0x7fc3c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e04202269a4aa91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc009c10 con 0x7fc3c0004d40 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3cc036070 con 0x7fc3c0004d40 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d159a640 1 -- 172.21.15.1:0/588898073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc031880 con 0x7fc3c0004d40 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 -- 172.21.15.1:0/588898073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 msgr2=0x7fc3c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/588898073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3c00051c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009fc0 tx=0x7fc3c0005830 comp rx=0 tx=0).stop 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d2d9d640 1 -- 172.21.15.1:0/588898073 reap_dead start 2024-05-09T06:28:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 -- 172.21.15.1:0/588898073 shutdown_connections 2024-05-09T06:28:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 -- 172.21.15.1:0/588898073 >> 172.21.15.1:0/588898073 conn(0x7fc3d40cc4d0 msgr2=0x7fc3d40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 -- 172.21.15.1:0/588898073 shutdown_connections 2024-05-09T06:28:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.243+0000 7fc3d8bce640 1 -- 172.21.15.1:0/588898073 wait complete. 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.245+0000 7fc3d8bce640 1 Processor -- start 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.245+0000 7fc3d8bce640 1 -- start start 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3d40b3260 0x7fc3d41f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 0x7fc3d40b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4141370 con 0x7fc3c0004d40 2024-05-09T06:28:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d41411f0 con 0x7fc3d40b3260 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d40ada60 con 0x7fc3d40b4da0 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49838/0 (socket says 172.21.15.1:49838) 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 -- 172.21.15.1:0/1285295497 learned_addr learned my addr 172.21.15.1:0/1285295497 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 0x7fc3d40b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d1d9b640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3d40b3260 0x7fc3d41f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 -- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 msgr2=0x7fc3d40b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 0x7fc3d40b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 -- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3d40b3260 msgr2=0x7fc3d41f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3d40b3260 0x7fc3d41f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 -- 172.21.15.1:0/1285295497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d406dd60 con 0x7fc3c0004d40 2024-05-09T06:28:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d2d9d640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 0x7fc3d40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d259c640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4011960 tx=0x7fc3c4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400faf0 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c400fc90 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d406e040 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4014440 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.246+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d406e600 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.248+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d40752b0 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.248+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3c4012040 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.249+0000 7fc3caffd640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.249+0000 7fc3d1d9b640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.249+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc3c408b570 con 0x7fc3c0004d40 2024-05-09T06:28:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.249+0000 7fc3d1d9b640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 0x7fc3b406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc001d30 tx=0x7fc3bc007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.253+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3c4019030 con 0x7fc3c0004d40 2024-05-09T06:28:46.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:46.478+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc3d40b0330 con 0x7fc3c0004d40 2024-05-09T06:28:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.112+0000 7fc3caffd640 1 -- 172.21.15.1:0/1285295497 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fc3c4059e20 con 0x7fc3c0004d40 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 msgr2=0x7fc3b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 0x7fc3b406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc001d30 tx=0x7fc3bc007970 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 msgr2=0x7fc3d40b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4011960 tx=0x7fc3c4011e20 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 shutdown_connections 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3b406be30 0x7fc3b406e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3d40b4da0 0x7fc3d40b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3d40b3260 0x7fc3d41f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 --2- 172.21.15.1:0/1285295497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3c0004d40 0x7fc3d40b2850 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.116+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 >> 172.21.15.1:0/1285295497 conn(0x7fc3d40cc4d0 msgr2=0x7fc3d40af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.117+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 shutdown_connections 2024-05-09T06:28:47.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.117+0000 7fc3d8bce640 1 -- 172.21.15.1:0/1285295497 wait complete. 2024-05-09T06:28:47.112 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:28:47.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:47.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 Processor -- start 2024-05-09T06:28:47.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 -- start start 2024-05-09T06:28:47.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f334c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c0b33c0 con 0x7f334c0d0f80 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c0b3660 con 0x7f334c0d4940 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.242+0000 7f335309a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0d4dd0 con 0x7f334c161950 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f3351637640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f334c0d0f80 0x7f334c0cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44724/0 (socket says 172.21.15.1:44724) 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f3351637640 1 -- 172.21.15.1:0/1420977552 learned_addr learned my addr 172.21.15.1:0/1420977552 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f3350e36640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f334c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804039948 0 0) 0x7f334c0b33c0 con 0x7f334c0d0f80 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3334003520 con 0x7f334c0d0f80 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3018523156 0 0) 0x7f3334003520 con 0x7f334c0d0f80 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 msgr2=0x7f334c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f334c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 >> v1:172.21.15.6:6789/0 conn(0x7f334c0d4940 legacy=0x7f334c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c0b3940 con 0x7f334c0d0f80 2024-05-09T06:28:47.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f33440019f0 con 0x7f334c0d0f80 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 >> v1:172.21.15.1:6789/0 conn(0x7f334c0d0f80 legacy=0x7f334c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340039e0 0x7f3334003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f3334004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3334004d40 0x7f33340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f334c0d4dd0 con 0x7f3334004d40 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f333400ba70 con 0x7f33340039e0 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f333400bc70 con 0x7f334c161950 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f3351637640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3334004d40 0x7f33340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f3350e36640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340039e0 0x7f3334003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.243+0000 7f334bfff640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f3334004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 -- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 msgr2=0x7f3334004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c161950 0x7f3334004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 -- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340039e0 msgr2=0x7f3334003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33340039e0 0x7f3334003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 -- 172.21.15.1:0/1420977552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3334003520 con 0x7f3334004d40 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.244+0000 7f3351637640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3334004d40 0x7f33340051c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f334c08d0b0 tx=0x7f3334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a564a1f6cdfaab98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334400a460 con 0x7f3334004d40 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334402f070 con 0x7f3334004d40 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f334b7fe640 1 -- 172.21.15.1:0/1420977552 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334402bc50 con 0x7f3334004d40 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 -- 172.21.15.1:0/1420977552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3334004d40 msgr2=0x7f33340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 --2- 172.21.15.1:0/1420977552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3334004d40 0x7f33340051c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f334c08d0b0 tx=0x7f3334005830 comp rx=0 tx=0).stop 2024-05-09T06:28:47.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f3351637640 1 -- 172.21.15.1:0/1420977552 reap_dead start 2024-05-09T06:28:47.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 -- 172.21.15.1:0/1420977552 shutdown_connections 2024-05-09T06:28:47.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 -- 172.21.15.1:0/1420977552 >> 172.21.15.1:0/1420977552 conn(0x7f334c0d3e20 msgr2=0x7f334c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 -- 172.21.15.1:0/1420977552 shutdown_connections 2024-05-09T06:28:47.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.245+0000 7f335309a640 1 -- 172.21.15.1:0/1420977552 wait complete. 2024-05-09T06:28:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.246+0000 7f335309a640 1 Processor -- start 2024-05-09T06:28:47.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 -- start start 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f334c0a1670 0x7f334c09e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 0x7f334c0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f334c0b5150 con 0x7f334c15c180 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f334c0cf850 con 0x7f334c0a1670 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.247+0000 7f335309a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0b52d0 con 0x7f334c0b5730 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49848/0 (socket says 172.21.15.1:49848) 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 -- 172.21.15.1:0/3172343696 learned_addr learned my addr 172.21.15.1:0/3172343696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3351637640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 0x7f334c0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f334bfff640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f334c0a1670 0x7f334c09e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 -- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 msgr2=0x7f334c0b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 0x7f334c0b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 -- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f334c0a1670 msgr2=0x7f334c09e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f334c0a1670 0x7f334c09e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 -- 172.21.15.1:0/3172343696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c0a1af0 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3351637640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 0x7f334c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f3350e36640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f333c0119a0 tx=0x7f333c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c00fa90 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c00fc30 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c0b8e70 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c014440 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.248+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c0b6a10 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.249+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c0d3380 con 0x7f334c15c180 2024-05-09T06:28:47.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.250+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f333c012040 con 0x7f334c15c180 2024-05-09T06:28:47.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.250+0000 7f33497fa640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 0x7f333806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.250+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f333c08b5e0 con 0x7f334c15c180 2024-05-09T06:28:47.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.250+0000 7f334bfff640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 0x7f333806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.251+0000 7f334bfff640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 0x7f333806e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f334c0b9a10 tx=0x7f3340009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.255+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f334c0d3380 con 0x7f334c15c180 2024-05-09T06:28:47.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.473+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f334c0752b0 con 0x7f334c15c180 2024-05-09T06:28:47.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.474+0000 7f33497fa640 1 -- 172.21.15.1:0/3172343696 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7f333c019030 con 0x7f334c15c180 2024-05-09T06:28:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:47.456 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:28:47.456 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 msgr2=0x7f333806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 0x7f333806e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f334c0b9a10 tx=0x7f3340009040 comp rx=0 tx=0).stop 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 msgr2=0x7f334c0b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f333c0119a0 tx=0x7f333c011e60 comp rx=0 tx=0).stop 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 shutdown_connections 2024-05-09T06:28:47.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f333806be60 0x7f333806e320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f334c0b5730 0x7f334c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f334c0a1670 0x7f334c09e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 --2- 172.21.15.1:0/3172343696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c15c180 0x7f334c0b8770 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 >> 172.21.15.1:0/3172343696 conn(0x7f334c0d3e20 msgr2=0x7f334c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 shutdown_connections 2024-05-09T06:28:47.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.476+0000 7f335309a640 1 -- 172.21.15.1:0/3172343696 wait complete. 2024-05-09T06:28:47.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.592+0000 7f3222258640 1 Processor -- start 2024-05-09T06:28:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.592+0000 7f3222258640 1 -- start start 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f3222258640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f321c15c320 0x7f321c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f3222258640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0b0d00 0x7f321c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f3222258640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c0d2c80 con 0x7f321c0d05e0 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f3222258640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0d29a0 con 0x7f321c15c320 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f3222258640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0cd430 con 0x7f321c0b0d00 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f321bfff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f321c0d05e0 0x7f321c0ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:42540/0 (socket says 172.21.15.1:42540) 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f321bfff640 1 -- 172.21.15.1:0/192557355 learned_addr learned my addr 172.21.15.1:0/192557355 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f321affd640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0b0d00 0x7f321c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.593+0000 7f321b7fe640 1 --2- 172.21.15.1:0/192557355 >> v2:172.21.15.6:3300/0 conn(0x7f321c15c320 0x7f321c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321a7fc640 1 -- 172.21.15.1:0/192557355 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146383956 0 0) 0x7f321c0d2c80 con 0x7f321c0d05e0 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321a7fc640 1 -- 172.21.15.1:0/192557355 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3208003520 con 0x7f321c0d05e0 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321b7fe640 1 -- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0b0d00 msgr2=0x7f321c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321b7fe640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0b0d00 0x7f321c0d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321b7fe640 1 -- 172.21.15.1:0/192557355 >> v1:172.21.15.6:6789/0 conn(0x7f321c0d05e0 legacy=0x7f321c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.594+0000 7f321b7fe640 1 -- 172.21.15.1:0/192557355 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0cd710 con 0x7f321c15c320 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f321b7fe640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c15c320 0x7f321c0b1250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097a0 tx=0x7f320c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72b42dbbdb7e2a3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f321a7fc640 1 -- 172.21.15.1:0/192557355 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c037070 con 0x7f321c15c320 2024-05-09T06:28:47.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f321a7fc640 1 -- 172.21.15.1:0/192557355 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c032440 con 0x7f321c15c320 2024-05-09T06:28:47.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f321a7fc640 1 -- 172.21.15.1:0/192557355 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0327f0 con 0x7f321c15c320 2024-05-09T06:28:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 -- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c15c320 msgr2=0x7f321c0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c15c320 0x7f321c0b1250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097a0 tx=0x7f320c02a210 comp rx=0 tx=0).stop 2024-05-09T06:28:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 -- 172.21.15.1:0/192557355 shutdown_connections 2024-05-09T06:28:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0b0d00 0x7f321c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 --2- 172.21.15.1:0/192557355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c15c320 0x7f321c0b1250 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 -- 172.21.15.1:0/192557355 >> 172.21.15.1:0/192557355 conn(0x7f321c0d1d70 msgr2=0x7f321c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 -- 172.21.15.1:0/192557355 shutdown_connections 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.595+0000 7f3222258640 1 -- 172.21.15.1:0/192557355 wait complete. 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 Processor -- start 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 -- start start 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c0b0d00 0x7f321c1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0d05e0 0x7f321c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f321c070020 con 0x7f321c15c320 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cfde0 con 0x7f321c0b0d00 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.597+0000 7f3222258640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0701a0 con 0x7f321c0d05e0 2024-05-09T06:28:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49852/0 (socket says 172.21.15.1:49852) 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 -- 172.21.15.1:0/3719799148 learned_addr learned my addr 172.21.15.1:0/3719799148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321affd640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0d05e0 0x7f321c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321b7fe640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c0b0d00 0x7f321c1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 -- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0d05e0 msgr2=0x7f321c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0d05e0 0x7f321c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 -- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c0b0d00 msgr2=0x7f321c1f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c0b0d00 0x7f321c1f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 -- 172.21.15.1:0/3719799148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c15e300 con 0x7f321c15c320 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f321bfff640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f321400cb40 tx=0x7f321400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214017070 con 0x7f321c15c320 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c15e5e0 con 0x7f321c15c320 2024-05-09T06:28:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321400aca0 con 0x7f321c15c320 2024-05-09T06:28:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214012620 con 0x7f321c15c320 2024-05-09T06:28:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.598+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c15eba0 con 0x7f321c15c320 2024-05-09T06:28:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.600+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c0752b0 con 0x7f321c15c320 2024-05-09T06:28:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.600+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f321401a070 con 0x7f321c15c320 2024-05-09T06:28:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.601+0000 7f3213fff640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.601+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f321408af40 con 0x7f321c15c320 2024-05-09T06:28:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.601+0000 7f321b7fe640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.601+0000 7f321b7fe640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f320c034000 tx=0x7f320c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.605+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f32140596e0 con 0x7f321c15c320 2024-05-09T06:28:47.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:28:47.676 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:28:47.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.791+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f321c09fb60 con 0x7f321c15c320 2024-05-09T06:28:47.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.792+0000 7f3213fff640 1 -- 172.21.15.1:0/3719799148 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16810 (secure 0 0 0) 0x7f3214058fe0 con 0x7f321c15c320 2024-05-09T06:28:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:47.774 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:45.085064+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 msgr2=0x7f31fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f320c034000 tx=0x7f320c02e000 comp rx=0 tx=0).stop 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 msgr2=0x7f321c09f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f321400cb40 tx=0x7f321400a060 comp rx=0 tx=0).stop 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 shutdown_connections 2024-05-09T06:28:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f321c15c320 0x7f321c09f080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f321c0d05e0 0x7f321c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 --2- 172.21.15.1:0/3719799148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f321c0b0d00 0x7f321c1f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 >> 172.21.15.1:0/3719799148 conn(0x7f321c0d1d70 msgr2=0x7f321c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 shutdown_connections 2024-05-09T06:28:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.794+0000 7f3222258640 1 -- 172.21.15.1:0/3719799148 wait complete. 2024-05-09T06:28:47.788 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:28:47.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:28:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.910+0000 7f63b202a640 1 Processor -- start 2024-05-09T06:28:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.910+0000 7f63b202a640 1 -- start start 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63b202a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b2a60 0x7f63ac0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63b202a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f63ac0b4f70 0x7f63ac070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63b202a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ac06eb40 con 0x7f63ac1454b0 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63b202a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac06ee20 con 0x7f63ac0b4f70 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63b202a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac06d8e0 con 0x7f63ac0b2a60 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63abfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f63ac0b4f70 0x7f63ac070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63abfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f63ac0b4f70 0x7f63ac070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41822/0 (socket says 172.21.15.1:41822) 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63abfff640 1 -- 172.21.15.1:0/1452149645 learned_addr learned my addr 172.21.15.1:0/1452149645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.911+0000 7f63aaffd640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b2a60 0x7f63ac0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63abfff640 1 -- 172.21.15.1:0/1452149645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b2a60 msgr2=0x7f63ac0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63abfff640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b2a60 0x7f63ac0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63abfff640 1 -- 172.21.15.1:0/1452149645 >> v1:172.21.15.6:6789/0 conn(0x7f63ac1454b0 legacy=0x7f63ac0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63abfff640 1 -- 172.21.15.1:0/1452149645 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac06dbc0 con 0x7f63ac0b4f70 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63abfff640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac0b4f70 0x7f63ac070c10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009e50 tx=0x7f63a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd68165183f33d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63aa7fc640 1 -- 172.21.15.1:0/1452149645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4036070 con 0x7f63ac0b4f70 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63aa7fc640 1 -- 172.21.15.1:0/1452149645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4031600 con 0x7f63ac0b4f70 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.912+0000 7f63aa7fc640 1 -- 172.21.15.1:0/1452149645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a40319d0 con 0x7f63ac0b4f70 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 -- 172.21.15.1:0/1452149645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac0b4f70 msgr2=0x7f63ac070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac0b4f70 0x7f63ac070c10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009e50 tx=0x7f63a402a710 comp rx=0 tx=0).stop 2024-05-09T06:28:47.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 -- 172.21.15.1:0/1452149645 shutdown_connections 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac0b4f70 0x7f63ac070c10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 --2- 172.21.15.1:0/1452149645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b2a60 0x7f63ac0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 -- 172.21.15.1:0/1452149645 >> 172.21.15.1:0/1452149645 conn(0x7f63ac0cc4b0 msgr2=0x7f63ac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 -- 172.21.15.1:0/1452149645 shutdown_connections 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.913+0000 7f63b202a640 1 -- 172.21.15.1:0/1452149645 wait complete. 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.914+0000 7f63b202a640 1 Processor -- start 2024-05-09T06:28:47.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.914+0000 7f63b202a640 1 -- start start 2024-05-09T06:28:47.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 0x7f63ac1f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 0x7f63ac0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac06e260 con 0x7f63ac1454b0 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac06e0e0 con 0x7f63ac0b2a60 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63b202a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac071150 con 0x7f63ac0b4f70 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41826/0 (socket says 172.21.15.1:41826) 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.915+0000 7f63abfff640 1 -- 172.21.15.1:0/3457830456 learned_addr learned my addr 172.21.15.1:0/3457830456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63aaffd640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 0x7f63ac0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 -- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 msgr2=0x7f63ac0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63ab7fe640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 0x7f63ac1f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 0x7f63ac0ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 -- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 msgr2=0x7f63ac1f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 0x7f63ac1f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 -- 172.21.15.1:0/3457830456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac1f76d0 con 0x7f63ac1454b0 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63ab7fe640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 0x7f63ac1f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63aaffd640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 0x7f63ac0ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63abfff640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009e50 tx=0x7f63a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4041020 con 0x7f63ac1454b0 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4007780 con 0x7f63ac1454b0 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4036040 con 0x7f63ac1454b0 2024-05-09T06:28:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac1f3f10 con 0x7f63ac1454b0 2024-05-09T06:28:47.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.916+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac1f7c20 con 0x7f63ac1454b0 2024-05-09T06:28:47.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.917+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac0af430 con 0x7f63ac1454b0 2024-05-09T06:28:47.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.918+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63a4044020 con 0x7f63ac1454b0 2024-05-09T06:28:47.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.918+0000 7f6393fff640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 0x7f639806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.919+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f63a40abc80 con 0x7f63ac1454b0 2024-05-09T06:28:47.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.919+0000 7f63ab7fe640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 0x7f639806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.919+0000 7f63ab7fe640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 0x7f639806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f639c0015e0 tx=0x7f639c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:47.923+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f63ac0af430 con 0x7f63ac1454b0 2024-05-09T06:28:48.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.102+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63ac068770 con 0x7f639806be30 2024-05-09T06:28:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.108+0000 7f6393fff640 1 -- 172.21.15.1:0/3457830456 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245559 (secure 0 0 0) 0x7f639c00d4f0 con 0x7f639806be30 2024-05-09T06:28:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:48.095 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:28:48.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 msgr2=0x7f639806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 0x7f639806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f639c0015e0 tx=0x7f639c009040 comp rx=0 tx=0).stop 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 msgr2=0x7f63ac0ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009e50 tx=0x7f63a4004a40 comp rx=0 tx=0).stop 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 shutdown_connections 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f639806be30 0x7f639806e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63ac1454b0 0x7f63ac0ae390 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63ac0b4f70 0x7f63ac0ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 --2- 172.21.15.1:0/3457830456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac0b2a60 0x7f63ac1f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 >> 172.21.15.1:0/3457830456 conn(0x7f63ac0cc4b0 msgr2=0x7f63ac141790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 shutdown_connections 2024-05-09T06:28:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.111+0000 7f63b202a640 1 -- 172.21.15.1:0/3457830456 wait complete. 2024-05-09T06:28:48.104 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-05-09T06:28:47.085680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.007337"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091760+0000","last_change":"2024-05-09T06:28:43.091760+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091760+0000","last_undegraded":"2024-05-09T06:28:43.091760+0000","last_fullsized":"2024-05-09T06:28:43.091760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092634+0000","last_change":"2024-05-09T06:28:43.092634+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092634+0000","last_undegraded":"2024-05-09T06:28:43.092634+0000","last_fullsized":"2024-05-09T06:28:43.092634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090871+0000","last_change":"2024-05-09T06:28:43.090871+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090871+0000","last_undegraded":"2024-05-09T06:28:43.090871+0000","last_fullsized":"2024-05-09T06:28:43.090871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090337+0000","last_change":"2024-05-09T06:28:43.090337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090337+0000","last_undegraded":"2024-05-09T06:28:43.090337+0000","last_fullsized":"2024-05-09T06:28:43.090337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088992+0000","last_change":"2024-05-09T06:28:43.088992+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088992+0000","last_undegraded":"2024-05-09T06:28:43.088992+0000","last_fullsized":"2024-05-09T06:28:43.088992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095168+0000","last_change":"2024-05-09T06:28:43.095168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095168+0000","last_undegraded":"2024-05-09T06:28:43.095168+0000","last_fullsized":"2024-05-09T06:28:43.095168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092833+0000","last_change":"2024-05-09T06:28:43.092833+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092833+0000","last_undegraded":"2024-05-09T06:28:43.092833+0000","last_fullsized":"2024-05-09T06:28:43.092833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091596+0000","last_change":"2024-05-09T06:28:43.091596+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091596+0000","last_undegraded":"2024-05-09T06:28:43.091596+0000","last_fullsized":"2024-05-09T06:28:43.091596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089477+0000","last_change":"2024-05-09T06:28:43.089477+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089477+0000","last_undegraded":"2024-05-09T06:28:43.089477+0000","last_fullsized":"2024-05-09T06:28:43.089477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095941+0000","last_change":"2024-05-09T06:28:43.095941+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095941+0000","last_undegraded":"2024-05-09T06:28:43.095941+0000","last_fullsized":"2024-05-09T06:28:43.095941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092798+0000","last_change":"2024-05-09T06:28:43.092798+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092798+0000","last_undegraded":"2024-05-09T06:28:43.092798+0000","last_fullsized":"2024-05-09T06:28:43.092798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089455+0000","last_change":"2024-05-09T06:28:43.089455+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089455+0000","last_undegraded":"2024-05-09T06:28:43.089455+0000","last_fullsized":"2024-05-09T06:28:43.089455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095544+0000","last_change":"2024-05-09T06:28:43.095544+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095544+0000","last_undegraded":"2024-05-09T06:28:43.095544+0000","last_fullsized":"2024-05-09T06:28:43.095544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095591+0000","last_change":"2024-05-09T06:28:43.095591+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095591+0000","last_undegraded":"2024-05-09T06:28:43.095591+0000","last_fullsized":"2024-05-09T06:28:43.095591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092337+0000","last_change":"2024-05-09T06:28:43.092337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092337+0000","last_undegraded":"2024-05-09T06:28:43.092337+0000","last_fullsized":"2024-05-09T06:28:43.092337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094761+0000","last_change":"2024-05-09T06:28:43.094761+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094761+0000","last_undegraded":"2024-05-09T06:28:43.094761+0000","last_fullsized":"2024-05-09T06:28:43.094761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097884+0000","last_change":"2024-05-09T06:28:43.097884+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097884+0000","last_undegraded":"2024-05-09T06:28:43.097884+0000","last_fullsized":"2024-05-09T06:28:43.097884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088822+0000","last_change":"2024-05-09T06:28:43.088822+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088822+0000","last_undegraded":"2024-05-09T06:28:43.088822+0000","last_fullsized":"2024-05-09T06:28:43.088822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090190+0000","last_change":"2024-05-09T06:28:43.090190+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090190+0000","last_undegraded":"2024-05-09T06:28:43.090190+0000","last_fullsized":"2024-05-09T06:28:43.090190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088332+0000","last_change":"2024-05-09T06:28:43.088332+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088332+0000","last_undegraded":"2024-05-09T06:28:43.088332+0000","last_fullsized":"2024-05-09T06:28:43.088332+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093271+0000","last_change":"2024-05-09T06:28:43.093271+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093271+0000","last_undegraded":"2024-05-09T06:28:43.093271+0000","last_fullsized":"2024-05-09T06:28:43.093271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094338+0000","last_change":"2024-05-09T06:28:43.094338+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094338+0000","last_undegraded":"2024-05-09T06:28:43.094338+0000","last_fullsized":"2024-05-09T06:28:43.094338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095708+0000","last_change":"2024-05-09T06:28:43.095708+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095708+0000","last_undegraded":"2024-05-09T06:28:43.095708+0000","last_fullsized":"2024-05-09T06:28:43.095708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089315+0000","last_change":"2024-05-09T06:28:43.089315+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089315+0000","last_undegraded":"2024-05-09T06:28:43.089315+0000","last_fullsized":"2024-05-09T06:28:43.089315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088440+0000","last_change":"2024-05-09T06:28:43.088440+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088440+0000","last_undegraded":"2024-05-09T06:28:43.088440+0000","last_fullsized":"2024-05-09T06:28:43.088440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094956+0000","last_change":"2024-05-09T06:28:43.094956+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094956+0000","last_undegraded":"2024-05-09T06:28:43.094956+0000","last_fullsized":"2024-05-09T06:28:43.094956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088696+0000","last_change":"2024-05-09T06:28:43.088696+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088696+0000","last_undegraded":"2024-05-09T06:28:43.088696+0000","last_fullsized":"2024-05-09T06:28:43.088696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.096282+0000","last_change":"2024-05-09T06:28:43.096282+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.096282+0000","last_undegraded":"2024-05-09T06:28:43.096282+0000","last_fullsized":"2024-05-09T06:28:43.096282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088202+0000","last_change":"2024-05-09T06:28:43.088202+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088202+0000","last_undegraded":"2024-05-09T06:28:43.088202+0000","last_fullsized":"2024-05-09T06:28:43.088202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088378+0000","last_change":"2024-05-09T06:28:43.088378+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088378+0000","last_undegraded":"2024-05-09T06:28:43.088378+0000","last_fullsized":"2024-05-09T06:28:43.088378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095226+0000","last_change":"2024-05-09T06:28:43.095226+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095226+0000","last_undegraded":"2024-05-09T06:28:43.095226+0000","last_fullsized":"2024-05-09T06:28:43.095226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095399+0000","last_change":"2024-05-09T06:28:43.095399+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095399+0000","last_undegraded":"2024-05-09T06:28:43.095399+0000","last_fullsized":"2024-05-09T06:28:43.095399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092559+0000","last_change":"2024-05-09T06:28:43.092559+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092559+0000","last_undegraded":"2024-05-09T06:28:43.092559+0000","last_fullsized":"2024-05-09T06:28:43.092559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100473+0000","last_change":"2024-05-09T06:28:43.097628+0000","last_active":"2024-05-09T06:28:43.100473+0000","last_peered":"2024-05-09T06:28:43.100473+0000","last_clean":"2024-05-09T06:28:43.100473+0000","last_became_active":"2024-05-09T06:28:43.097348+0000","last_became_peered":"2024-05-09T06:28:43.097348+0000","last_unstale":"2024-05-09T06:28:43.100473+0000","last_undegraded":"2024-05-09T06:28:43.100473+0000","last_fullsized":"2024-05-09T06:28:43.100473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095959+0000","last_change":"2024-05-09T06:28:43.095959+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095959+0000","last_undegraded":"2024-05-09T06:28:43.095959+0000","last_fullsized":"2024-05-09T06:28:43.095959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091246+0000","last_change":"2024-05-09T06:28:43.091246+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091246+0000","last_undegraded":"2024-05-09T06:28:43.091246+0000","last_fullsized":"2024-05-09T06:28:43.091246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093492+0000","last_change":"2024-05-09T06:28:43.093492+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093492+0000","last_undegraded":"2024-05-09T06:28:43.093492+0000","last_fullsized":"2024-05-09T06:28:43.093492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.095553+0000","last_change":"2024-05-09T06:28:43.095553+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095553+0000","last_undegraded":"2024-05-09T06:28:43.095553+0000","last_fullsized":"2024-05-09T06:28:43.095553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092622+0000","last_change":"2024-05-09T06:28:43.092622+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092622+0000","last_undegraded":"2024-05-09T06:28:43.092622+0000","last_fullsized":"2024-05-09T06:28:43.092622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094727+0000","last_change":"2024-05-09T06:28:43.094727+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094727+0000","last_undegraded":"2024-05-09T06:28:43.094727+0000","last_fullsized":"2024-05-09T06:28:43.094727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092001+0000","last_change":"2024-05-09T06:28:43.092001+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092001+0000","last_undegraded":"2024-05-09T06:28:43.092001+0000","last_fullsized":"2024-05-09T06:28:43.092001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091798+0000","last_change":"2024-05-09T06:28:43.091798+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091798+0000","last_undegraded":"2024-05-09T06:28:43.091798+0000","last_fullsized":"2024-05-09T06:28:43.091798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091744+0000","last_change":"2024-05-09T06:28:43.091744+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091744+0000","last_undegraded":"2024-05-09T06:28:43.091744+0000","last_fullsized":"2024-05-09T06:28:43.091744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108921+0000","last_change":"2024-05-09T06:28:43.107479+0000","last_active":"2024-05-09T06:28:43.108921+0000","last_peered":"2024-05-09T06:28:43.108921+0000","last_clean":"2024-05-09T06:28:43.108921+0000","last_became_active":"2024-05-09T06:28:43.107175+0000","last_became_peered":"2024-05-09T06:28:43.107175+0000","last_unstale":"2024-05-09T06:28:43.108921+0000","last_undegraded":"2024-05-09T06:28:43.108921+0000","last_fullsized":"2024-05-09T06:28:43.108921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093881+0000","last_change":"2024-05-09T06:28:43.093881+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093881+0000","last_undegraded":"2024-05-09T06:28:43.093881+0000","last_fullsized":"2024-05-09T06:28:43.093881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094244+0000","last_change":"2024-05-09T06:28:43.094244+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094244+0000","last_undegraded":"2024-05-09T06:28:43.094244+0000","last_fullsized":"2024-05-09T06:28:43.094244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094270+0000","last_change":"2024-05-09T06:28:43.094270+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094270+0000","last_undegraded":"2024-05-09T06:28:43.094270+0000","last_fullsized":"2024-05-09T06:28:43.094270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.102412+0000","last_change":"2024-05-09T06:28:43.099299+0000","last_active":"2024-05-09T06:28:43.102412+0000","last_peered":"2024-05-09T06:28:43.102412+0000","last_clean":"2024-05-09T06:28:43.102412+0000","last_became_active":"2024-05-09T06:28:43.099001+0000","last_became_peered":"2024-05-09T06:28:43.099001+0000","last_unstale":"2024-05-09T06:28:43.102412+0000","last_undegraded":"2024-05-09T06:28:43.102412+0000","last_fullsized":"2024-05-09T06:28:43.102412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:59.842925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088924+0000","last_change":"2024-05-09T06:28:43.088924+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088924+0000","last_undegraded":"2024-05-09T06:28:43.088924+0000","last_fullsized":"2024-05-09T06:28:43.088924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108201+0000","last_change":"2024-05-09T06:28:43.101397+0000","last_active":"2024-05-09T06:28:43.108201+0000","last_peered":"2024-05-09T06:28:43.108201+0000","last_clean":"2024-05-09T06:28:43.108201+0000","last_became_active":"2024-05-09T06:28:43.101067+0000","last_became_peered":"2024-05-09T06:28:43.101067+0000","last_unstale":"2024-05-09T06:28:43.108201+0000","last_undegraded":"2024-05-09T06:28:43.108201+0000","last_fullsized":"2024-05-09T06:28:43.108201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:09.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099075+0000","last_change":"2024-05-09T06:28:43.099075+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099075+0000","last_undegraded":"2024-05-09T06:28:43.099075+0000","last_fullsized":"2024-05-09T06:28:43.099075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.092728+0000","last_change":"2024-05-09T06:28:43.092728+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092728+0000","last_undegraded":"2024-05-09T06:28:43.092728+0000","last_fullsized":"2024-05-09T06:28:43.092728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092422+0000","last_change":"2024-05-09T06:28:43.092422+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092422+0000","last_undegraded":"2024-05-09T06:28:43.092422+0000","last_fullsized":"2024-05-09T06:28:43.092422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":6,"reported_epoch":36,"state":"active","last_fresh":"2024-05-09T06:28:43.105390+0000","last_change":"2024-05-09T06:28:43.105323+0000","last_active":"2024-05-09T06:28:43.105390+0000","last_peered":"2024-05-09T06:28:43.105390+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"2024-05-09T06:28:43.105323+0000","last_became_peered":"2024-05-09T06:28:43.105323+0000","last_unstale":"2024-05-09T06:28:43.105390+0000","last_undegraded":"2024-05-09T06:28:43.105390+0000","last_fullsized":"2024-05-09T06:28:43.105390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093177+0000","last_change":"2024-05-09T06:28:43.093177+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093177+0000","last_undegraded":"2024-05-09T06:28:43.093177+0000","last_fullsized":"2024-05-09T06:28:43.093177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108647+0000","last_change":"2024-05-09T06:28:43.105704+0000","last_active":"2024-05-09T06:28:43.108647+0000","last_peered":"2024-05-09T06:28:43.108647+0000","last_clean":"2024-05-09T06:28:43.108647+0000","last_became_active":"2024-05-09T06:28:43.105384+0000","last_became_peered":"2024-05-09T06:28:43.105384+0000","last_unstale":"2024-05-09T06:28:43.108647+0000","last_undegraded":"2024-05-09T06:28:43.108647+0000","last_fullsized":"2024-05-09T06:28:43.108647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:30.961461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091614+0000","last_change":"2024-05-09T06:28:43.091614+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091614+0000","last_undegraded":"2024-05-09T06:28:43.091614+0000","last_fullsized":"2024-05-09T06:28:43.091614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099059+0000","last_change":"2024-05-09T06:28:43.097679+0000","last_active":"2024-05-09T06:28:43.099059+0000","last_peered":"2024-05-09T06:28:43.099059+0000","last_clean":"2024-05-09T06:28:43.099059+0000","last_became_active":"2024-05-09T06:28:43.097373+0000","last_became_peered":"2024-05-09T06:28:43.097373+0000","last_unstale":"2024-05-09T06:28:43.099059+0000","last_undegraded":"2024-05-09T06:28:43.099059+0000","last_fullsized":"2024-05-09T06:28:43.099059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:37.004402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091149+0000","last_change":"2024-05-09T06:28:43.091149+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091149+0000","last_undegraded":"2024-05-09T06:28:43.091149+0000","last_fullsized":"2024-05-09T06:28:43.091149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094410+0000","last_change":"2024-05-09T06:28:43.094410+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094410+0000","last_undegraded":"2024-05-09T06:28:43.094410+0000","last_fullsized":"2024-05-09T06:28:43.094410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091620+0000","last_change":"2024-05-09T06:28:43.091620+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091620+0000","last_undegraded":"2024-05-09T06:28:43.091620+0000","last_fullsized":"2024-05-09T06:28:43.091620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097906+0000","last_change":"2024-05-09T06:28:43.096518+0000","last_active":"2024-05-09T06:28:43.097906+0000","last_peered":"2024-05-09T06:28:43.097906+0000","last_clean":"2024-05-09T06:28:43.097906+0000","last_became_active":"2024-05-09T06:28:43.096235+0000","last_became_peered":"2024-05-09T06:28:43.096235+0000","last_unstale":"2024-05-09T06:28:43.097906+0000","last_undegraded":"2024-05-09T06:28:43.097906+0000","last_fullsized":"2024-05-09T06:28:43.097906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:02.760035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093556+0000","last_change":"2024-05-09T06:28:43.093556+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093556+0000","last_undegraded":"2024-05-09T06:28:43.093556+0000","last_fullsized":"2024-05-09T06:28:43.093556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108183+0000","last_change":"2024-05-09T06:28:43.108183+0000","last_active":"2024-05-09T06:28:43.108183+0000","last_peered":"2024-05-09T06:28:43.108183+0000","last_clean":"2024-05-09T06:28:43.108183+0000","last_became_active":"2024-05-09T06:28:43.107072+0000","last_became_peered":"2024-05-09T06:28:43.107072+0000","last_unstale":"2024-05-09T06:28:43.108183+0000","last_undegraded":"2024-05-09T06:28:43.108183+0000","last_fullsized":"2024-05-09T06:28:43.108183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092413+0000","last_change":"2024-05-09T06:28:43.092413+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092413+0000","last_undegraded":"2024-05-09T06:28:43.092413+0000","last_fullsized":"2024-05-09T06:28:43.092413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098259+0000","last_change":"2024-05-09T06:28:43.098259+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098259+0000","last_undegraded":"2024-05-09T06:28:43.098259+0000","last_fullsized":"2024-05-09T06:28:43.098259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092320+0000","last_change":"2024-05-09T06:28:43.092320+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092320+0000","last_undegraded":"2024-05-09T06:28:43.092320+0000","last_fullsized":"2024-05-09T06:28:43.092320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091605+0000","last_change":"2024-05-09T06:28:43.091605+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091605+0000","last_undegraded":"2024-05-09T06:28:43.091605+0000","last_fullsized":"2024-05-09T06:28:43.091605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094168+0000","last_change":"2024-05-09T06:28:43.094168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094168+0000","last_undegraded":"2024-05-09T06:28:43.094168+0000","last_fullsized":"2024-05-09T06:28:43.094168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.086055+0000","last_change":"2024-05-09T06:28:43.086055+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.086055+0000","last_undegraded":"2024-05-09T06:28:43.086055+0000","last_fullsized":"2024-05-09T06:28:43.086055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088918+0000","last_change":"2024-05-09T06:28:43.088918+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088918+0000","last_undegraded":"2024-05-09T06:28:43.088918+0000","last_fullsized":"2024-05-09T06:28:43.088918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094171+0000","last_change":"2024-05-09T06:28:43.094171+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094171+0000","last_undegraded":"2024-05-09T06:28:43.094171+0000","last_fullsized":"2024-05-09T06:28:43.094171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094312+0000","last_change":"2024-05-09T06:28:43.094312+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094312+0000","last_undegraded":"2024-05-09T06:28:43.094312+0000","last_fullsized":"2024-05-09T06:28:43.094312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085920+0000","last_change":"2024-05-09T06:28:43.085920+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085920+0000","last_undegraded":"2024-05-09T06:28:43.085920+0000","last_fullsized":"2024-05-09T06:28:43.085920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090013+0000","last_change":"2024-05-09T06:28:43.090013+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090013+0000","last_undegraded":"2024-05-09T06:28:43.090013+0000","last_fullsized":"2024-05-09T06:28:43.090013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100163+0000","last_change":"2024-05-09T06:28:43.100163+0000","last_active":"2024-05-09T06:28:43.100163+0000","last_peered":"2024-05-09T06:28:43.100163+0000","last_clean":"2024-05-09T06:28:43.100163+0000","last_became_active":"2024-05-09T06:28:43.099826+0000","last_became_peered":"2024-05-09T06:28:43.099826+0000","last_unstale":"2024-05-09T06:28:43.100163+0000","last_undegraded":"2024-05-09T06:28:43.100163+0000","last_fullsized":"2024-05-09T06:28:43.100163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:11.395053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097458+0000","last_change":"2024-05-09T06:28:43.097458+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097458+0000","last_undegraded":"2024-05-09T06:28:43.097458+0000","last_fullsized":"2024-05-09T06:28:43.097458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.107115+0000","last_change":"2024-05-09T06:28:43.105547+0000","last_active":"2024-05-09T06:28:43.107115+0000","last_peered":"2024-05-09T06:28:43.107115+0000","last_clean":"2024-05-09T06:28:43.107115+0000","last_became_active":"2024-05-09T06:28:43.104806+0000","last_became_peered":"2024-05-09T06:28:43.104806+0000","last_unstale":"2024-05-09T06:28:43.107115+0000","last_undegraded":"2024-05-09T06:28:43.107115+0000","last_fullsized":"2024-05-09T06:28:43.107115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:12.906616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092042+0000","last_change":"2024-05-09T06:28:43.092042+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092042+0000","last_undegraded":"2024-05-09T06:28:43.092042+0000","last_fullsized":"2024-05-09T06:28:43.092042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099690+0000","last_change":"2024-05-09T06:28:43.099690+0000","last_active":"2024-05-09T06:28:43.099690+0000","last_peered":"2024-05-09T06:28:43.099690+0000","last_clean":"2024-05-09T06:28:43.099690+0000","last_became_active":"2024-05-09T06:28:43.099173+0000","last_became_peered":"2024-05-09T06:28:43.099173+0000","last_unstale":"2024-05-09T06:28:43.099690+0000","last_undegraded":"2024-05-09T06:28:43.099690+0000","last_fullsized":"2024-05-09T06:28:43.099690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:36.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094579+0000","last_change":"2024-05-09T06:28:43.094579+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094579+0000","last_undegraded":"2024-05-09T06:28:43.094579+0000","last_fullsized":"2024-05-09T06:28:43.094579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098565+0000","last_change":"2024-05-09T06:28:43.098565+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098565+0000","last_undegraded":"2024-05-09T06:28:43.098565+0000","last_fullsized":"2024-05-09T06:28:43.098565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094618+0000","last_change":"2024-05-09T06:28:43.094618+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094618+0000","last_undegraded":"2024-05-09T06:28:43.094618+0000","last_fullsized":"2024-05-09T06:28:43.094618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085912+0000","last_change":"2024-05-09T06:28:43.085912+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085912+0000","last_undegraded":"2024-05-09T06:28:43.085912+0000","last_fullsized":"2024-05-09T06:28:43.085912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092156+0000","last_change":"2024-05-09T06:28:43.092156+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092156+0000","last_undegraded":"2024-05-09T06:28:43.092156+0000","last_fullsized":"2024-05-09T06:28:43.092156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.090449+0000","last_change":"2024-05-09T06:28:43.090449+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090449+0000","last_undegraded":"2024-05-09T06:28:43.090449+0000","last_fullsized":"2024-05-09T06:28:43.090449+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091192+0000","last_change":"2024-05-09T06:28:43.091192+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091192+0000","last_undegraded":"2024-05-09T06:28:43.091192+0000","last_fullsized":"2024-05-09T06:28:43.091192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097963+0000","last_change":"2024-05-09T06:28:43.097963+0000","last_active":"2024-05-09T06:28:43.097963+0000","last_peered":"2024-05-09T06:28:43.097963+0000","last_clean":"2024-05-09T06:28:43.097963+0000","last_became_active":"2024-05-09T06:28:43.097670+0000","last_became_peered":"2024-05-09T06:28:43.097670+0000","last_unstale":"2024-05-09T06:28:43.097963+0000","last_undegraded":"2024-05-09T06:28:43.097963+0000","last_fullsized":"2024-05-09T06:28:43.097963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095027+0000","last_change":"2024-05-09T06:28:43.095027+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095027+0000","last_undegraded":"2024-05-09T06:28:43.095027+0000","last_fullsized":"2024-05-09T06:28:43.095027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.094929+0000","last_change":"2024-05-09T06:28:43.094133+0000","last_active":"2024-05-09T06:28:43.094929+0000","last_peered":"2024-05-09T06:28:43.094929+0000","last_clean":"2024-05-09T06:28:43.094929+0000","last_became_active":"2024-05-09T06:28:43.093802+0000","last_became_peered":"2024-05-09T06:28:43.093802+0000","last_unstale":"2024-05-09T06:28:43.094929+0000","last_undegraded":"2024-05-09T06:28:43.094929+0000","last_fullsized":"2024-05-09T06:28:43.094929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:21.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095109+0000","last_change":"2024-05-09T06:28:43.095109+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095109+0000","last_undegraded":"2024-05-09T06:28:43.095109+0000","last_fullsized":"2024-05-09T06:28:43.095109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100580+0000","last_change":"2024-05-09T06:28:43.096894+0000","last_active":"2024-05-09T06:28:43.100580+0000","last_peered":"2024-05-09T06:28:43.100580+0000","last_clean":"2024-05-09T06:28:43.100580+0000","last_became_active":"2024-05-09T06:28:43.096592+0000","last_became_peered":"2024-05-09T06:28:43.096592+0000","last_unstale":"2024-05-09T06:28:43.100580+0000","last_undegraded":"2024-05-09T06:28:43.100580+0000","last_fullsized":"2024-05-09T06:28:43.100580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:18:01.363601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093078+0000","last_change":"2024-05-09T06:28:43.093078+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093078+0000","last_undegraded":"2024-05-09T06:28:43.093078+0000","last_fullsized":"2024-05-09T06:28:43.093078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108318+0000","last_change":"2024-05-09T06:28:43.101342+0000","last_active":"2024-05-09T06:28:43.108318+0000","last_peered":"2024-05-09T06:28:43.108318+0000","last_clean":"2024-05-09T06:28:43.108318+0000","last_became_active":"2024-05-09T06:28:43.101027+0000","last_became_peered":"2024-05-09T06:28:43.101027+0000","last_unstale":"2024-05-09T06:28:43.108318+0000","last_undegraded":"2024-05-09T06:28:43.108318+0000","last_fullsized":"2024-05-09T06:28:43.108318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:14.687508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099148+0000","last_change":"2024-05-09T06:28:43.099148+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099148+0000","last_undegraded":"2024-05-09T06:28:43.099148+0000","last_fullsized":"2024-05-09T06:28:43.099148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108722+0000","last_change":"2024-05-09T06:28:43.107403+0000","last_active":"2024-05-09T06:28:43.108722+0000","last_peered":"2024-05-09T06:28:43.108722+0000","last_clean":"2024-05-09T06:28:43.108722+0000","last_became_active":"2024-05-09T06:28:43.107064+0000","last_became_peered":"2024-05-09T06:28:43.107064+0000","last_unstale":"2024-05-09T06:28:43.108722+0000","last_undegraded":"2024-05-09T06:28:43.108722+0000","last_fullsized":"2024-05-09T06:28:43.108722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.161063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100583+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:28:43.100583+0000","last_peered":"2024-05-09T06:28:43.100583+0000","last_clean":"2024-05-09T06:28:43.100583+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:28:43.100583+0000","last_undegraded":"2024-05-09T06:28:43.100583+0000","last_fullsized":"2024-05-09T06:28:43.100583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:28:48.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:28:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.266+0000 7f994d07d640 1 Processor -- start 2024-05-09T06:28:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.266+0000 7f994d07d640 1 -- start start 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.267+0000 7f994d07d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9948161950 0x7f99480d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.267+0000 7f994d07d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f99480d4940 0x7f99480d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.267+0000 7f994d07d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99480b33c0 con 0x7f99480d0f80 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.267+0000 7f994d07d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f99480d4dd0 con 0x7f99480d4940 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.267+0000 7f994d07d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f99480b3660 con 0x7f9948161950 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994659c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f99480d4940 0x7f99480d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f9946d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9948161950 0x7f99480d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994759e640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f99480d0f80 0x7f99480cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:38098/0 (socket says 172.21.15.1:38098) 2024-05-09T06:28:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994759e640 1 -- 172.21.15.1:0/115803085 learned_addr learned my addr 172.21.15.1:0/115803085 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994659c640 1 -- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9948161950 msgr2=0x7f99480d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994659c640 1 --2- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9948161950 0x7f99480d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994659c640 1 -- 172.21.15.1:0/115803085 >> v1:172.21.15.6:6789/0 conn(0x7f99480d0f80 legacy=0x7f99480cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f994659c640 1 -- 172.21.15.1:0/115803085 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99480b3940 con 0x7f99480d4940 2024-05-09T06:28:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.268+0000 7f9945d9b640 1 -- 172.21.15.1:0/115803085 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899308898 0 0) 0x7f99480b33c0 con 0x7f99480d0f80 2024-05-09T06:28:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994659c640 1 --2- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99480d4940 0x7f99480d2940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99480b3590 tx=0x7f993c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2598ee0c20059d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f9945d9b640 1 -- 172.21.15.1:0/115803085 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c037070 con 0x7f99480d4940 2024-05-09T06:28:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f9945d9b640 1 -- 172.21.15.1:0/115803085 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c032440 con 0x7f99480d4940 2024-05-09T06:28:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f9945d9b640 1 -- 172.21.15.1:0/115803085 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c032810 con 0x7f99480d4940 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 -- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99480d4940 msgr2=0x7f99480d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 --2- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99480d4940 0x7f99480d2940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99480b3590 tx=0x7f993c02a180 comp rx=0 tx=0).stop 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 -- 172.21.15.1:0/115803085 shutdown_connections 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 --2- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99480d4940 0x7f99480d2940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 --2- 172.21.15.1:0/115803085 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9948161950 0x7f99480d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 -- 172.21.15.1:0/115803085 >> 172.21.15.1:0/115803085 conn(0x7f99480d3e20 msgr2=0x7f99480d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 -- 172.21.15.1:0/115803085 shutdown_connections 2024-05-09T06:28:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.269+0000 7f994d07d640 1 -- 172.21.15.1:0/115803085 wait complete. 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.271+0000 7f994d07d640 1 Processor -- start 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.271+0000 7f994d07d640 1 -- start start 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99480d0f80 0x7f99480cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994809efe0 0x7f994809f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99480b52d0 con 0x7f9948161950 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99480b5150 con 0x7f994809efe0 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994d07d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f99480b5450 con 0x7f99480d0f80 2024-05-09T06:28:48.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41832/0 (socket says 172.21.15.1:41832) 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 -- 172.21.15.1:0/3429528173 learned_addr learned my addr 172.21.15.1:0/3429528173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994759e640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994809efe0 0x7f994809f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f9946d9d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99480d0f80 0x7f99480cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 -- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99480d0f80 msgr2=0x7f99480cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99480d0f80 0x7f99480cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 -- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994809efe0 msgr2=0x7f994809f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994809efe0 0x7f994809f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 -- 172.21.15.1:0/3429528173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994815db50 con 0x7f9948161950 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.272+0000 7f994659c640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f993c000c00 tx=0x7f993c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.273+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c037050 con 0x7f9948161950 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.273+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c038040 con 0x7f9948161950 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.273+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994814e5f0 con 0x7f9948161950 2024-05-09T06:28:48.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.273+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c004470 con 0x7f9948161950 2024-05-09T06:28:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.273+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994814ee90 con 0x7f9948161950 2024-05-09T06:28:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.274+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99480752b0 con 0x7f9948161950 2024-05-09T06:28:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.274+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f993c003610 con 0x7f9948161950 2024-05-09T06:28:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.275+0000 7f992f7fe640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 0x7f993406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.275+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f993c0abac0 con 0x7f9948161950 2024-05-09T06:28:48.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.275+0000 7f9946d9d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 0x7f993406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.275+0000 7f9946d9d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 0x7f993406e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9938009a30 tx=0x7f9938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.279+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f994815db50 con 0x7f9948161950 2024-05-09T06:28:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.460+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99480d21e0 con 0x7f993406be30 2024-05-09T06:28:48.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.466+0000 7f992f7fe640 1 -- 172.21.15.1:0/3429528173 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245559 (secure 0 0 0) 0x7f99480d21e0 con 0x7f993406be30 2024-05-09T06:28:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:48.452 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 msgr2=0x7f993406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 0x7f993406e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9938009a30 tx=0x7f9938008040 comp rx=0 tx=0).stop 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 msgr2=0x7f99481601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f993c000c00 tx=0x7f993c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 shutdown_connections 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993406be30 0x7f993406e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994809efe0 0x7f994809f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948161950 0x7f99481601c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 --2- 172.21.15.1:0/3429528173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99480d0f80 0x7f99480cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 >> 172.21.15.1:0/3429528173 conn(0x7f99480d3e20 msgr2=0x7f994815e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 shutdown_connections 2024-05-09T06:28:48.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.469+0000 7f994d07d640 1 -- 172.21.15.1:0/3429528173 wait complete. 2024-05-09T06:28:48.462 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-05-09T06:28:47.085680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.007337"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091760+0000","last_change":"2024-05-09T06:28:43.091760+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091760+0000","last_undegraded":"2024-05-09T06:28:43.091760+0000","last_fullsized":"2024-05-09T06:28:43.091760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092634+0000","last_change":"2024-05-09T06:28:43.092634+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092634+0000","last_undegraded":"2024-05-09T06:28:43.092634+0000","last_fullsized":"2024-05-09T06:28:43.092634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090871+0000","last_change":"2024-05-09T06:28:43.090871+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090871+0000","last_undegraded":"2024-05-09T06:28:43.090871+0000","last_fullsized":"2024-05-09T06:28:43.090871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090337+0000","last_change":"2024-05-09T06:28:43.090337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090337+0000","last_undegraded":"2024-05-09T06:28:43.090337+0000","last_fullsized":"2024-05-09T06:28:43.090337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088992+0000","last_change":"2024-05-09T06:28:43.088992+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088992+0000","last_undegraded":"2024-05-09T06:28:43.088992+0000","last_fullsized":"2024-05-09T06:28:43.088992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095168+0000","last_change":"2024-05-09T06:28:43.095168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095168+0000","last_undegraded":"2024-05-09T06:28:43.095168+0000","last_fullsized":"2024-05-09T06:28:43.095168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092833+0000","last_change":"2024-05-09T06:28:43.092833+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092833+0000","last_undegraded":"2024-05-09T06:28:43.092833+0000","last_fullsized":"2024-05-09T06:28:43.092833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091596+0000","last_change":"2024-05-09T06:28:43.091596+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091596+0000","last_undegraded":"2024-05-09T06:28:43.091596+0000","last_fullsized":"2024-05-09T06:28:43.091596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089477+0000","last_change":"2024-05-09T06:28:43.089477+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089477+0000","last_undegraded":"2024-05-09T06:28:43.089477+0000","last_fullsized":"2024-05-09T06:28:43.089477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095941+0000","last_change":"2024-05-09T06:28:43.095941+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095941+0000","last_undegraded":"2024-05-09T06:28:43.095941+0000","last_fullsized":"2024-05-09T06:28:43.095941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092798+0000","last_change":"2024-05-09T06:28:43.092798+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092798+0000","last_undegraded":"2024-05-09T06:28:43.092798+0000","last_fullsized":"2024-05-09T06:28:43.092798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089455+0000","last_change":"2024-05-09T06:28:43.089455+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089455+0000","last_undegraded":"2024-05-09T06:28:43.089455+0000","last_fullsized":"2024-05-09T06:28:43.089455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095544+0000","last_change":"2024-05-09T06:28:43.095544+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095544+0000","last_undegraded":"2024-05-09T06:28:43.095544+0000","last_fullsized":"2024-05-09T06:28:43.095544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095591+0000","last_change":"2024-05-09T06:28:43.095591+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095591+0000","last_undegraded":"2024-05-09T06:28:43.095591+0000","last_fullsized":"2024-05-09T06:28:43.095591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092337+0000","last_change":"2024-05-09T06:28:43.092337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092337+0000","last_undegraded":"2024-05-09T06:28:43.092337+0000","last_fullsized":"2024-05-09T06:28:43.092337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094761+0000","last_change":"2024-05-09T06:28:43.094761+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094761+0000","last_undegraded":"2024-05-09T06:28:43.094761+0000","last_fullsized":"2024-05-09T06:28:43.094761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097884+0000","last_change":"2024-05-09T06:28:43.097884+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097884+0000","last_undegraded":"2024-05-09T06:28:43.097884+0000","last_fullsized":"2024-05-09T06:28:43.097884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088822+0000","last_change":"2024-05-09T06:28:43.088822+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088822+0000","last_undegraded":"2024-05-09T06:28:43.088822+0000","last_fullsized":"2024-05-09T06:28:43.088822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090190+0000","last_change":"2024-05-09T06:28:43.090190+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090190+0000","last_undegraded":"2024-05-09T06:28:43.090190+0000","last_fullsized":"2024-05-09T06:28:43.090190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088332+0000","last_change":"2024-05-09T06:28:43.088332+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088332+0000","last_undegraded":"2024-05-09T06:28:43.088332+0000","last_fullsized":"2024-05-09T06:28:43.088332+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093271+0000","last_change":"2024-05-09T06:28:43.093271+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093271+0000","last_undegraded":"2024-05-09T06:28:43.093271+0000","last_fullsized":"2024-05-09T06:28:43.093271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094338+0000","last_change":"2024-05-09T06:28:43.094338+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094338+0000","last_undegraded":"2024-05-09T06:28:43.094338+0000","last_fullsized":"2024-05-09T06:28:43.094338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095708+0000","last_change":"2024-05-09T06:28:43.095708+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095708+0000","last_undegraded":"2024-05-09T06:28:43.095708+0000","last_fullsized":"2024-05-09T06:28:43.095708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089315+0000","last_change":"2024-05-09T06:28:43.089315+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089315+0000","last_undegraded":"2024-05-09T06:28:43.089315+0000","last_fullsized":"2024-05-09T06:28:43.089315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088440+0000","last_change":"2024-05-09T06:28:43.088440+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088440+0000","last_undegraded":"2024-05-09T06:28:43.088440+0000","last_fullsized":"2024-05-09T06:28:43.088440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094956+0000","last_change":"2024-05-09T06:28:43.094956+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094956+0000","last_undegraded":"2024-05-09T06:28:43.094956+0000","last_fullsized":"2024-05-09T06:28:43.094956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088696+0000","last_change":"2024-05-09T06:28:43.088696+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088696+0000","last_undegraded":"2024-05-09T06:28:43.088696+0000","last_fullsized":"2024-05-09T06:28:43.088696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.096282+0000","last_change":"2024-05-09T06:28:43.096282+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.096282+0000","last_undegraded":"2024-05-09T06:28:43.096282+0000","last_fullsized":"2024-05-09T06:28:43.096282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088202+0000","last_change":"2024-05-09T06:28:43.088202+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088202+0000","last_undegraded":"2024-05-09T06:28:43.088202+0000","last_fullsized":"2024-05-09T06:28:43.088202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088378+0000","last_change":"2024-05-09T06:28:43.088378+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088378+0000","last_undegraded":"2024-05-09T06:28:43.088378+0000","last_fullsized":"2024-05-09T06:28:43.088378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095226+0000","last_change":"2024-05-09T06:28:43.095226+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095226+0000","last_undegraded":"2024-05-09T06:28:43.095226+0000","last_fullsized":"2024-05-09T06:28:43.095226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095399+0000","last_change":"2024-05-09T06:28:43.095399+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095399+0000","last_undegraded":"2024-05-09T06:28:43.095399+0000","last_fullsized":"2024-05-09T06:28:43.095399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092559+0000","last_change":"2024-05-09T06:28:43.092559+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092559+0000","last_undegraded":"2024-05-09T06:28:43.092559+0000","last_fullsized":"2024-05-09T06:28:43.092559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100473+0000","last_change":"2024-05-09T06:28:43.097628+0000","last_active":"2024-05-09T06:28:43.100473+0000","last_peered":"2024-05-09T06:28:43.100473+0000","last_clean":"2024-05-09T06:28:43.100473+0000","last_became_active":"2024-05-09T06:28:43.097348+0000","last_became_peered":"2024-05-09T06:28:43.097348+0000","last_unstale":"2024-05-09T06:28:43.100473+0000","last_undegraded":"2024-05-09T06:28:43.100473+0000","last_fullsized":"2024-05-09T06:28:43.100473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095959+0000","last_change":"2024-05-09T06:28:43.095959+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095959+0000","last_undegraded":"2024-05-09T06:28:43.095959+0000","last_fullsized":"2024-05-09T06:28:43.095959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091246+0000","last_change":"2024-05-09T06:28:43.091246+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091246+0000","last_undegraded":"2024-05-09T06:28:43.091246+0000","last_fullsized":"2024-05-09T06:28:43.091246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093492+0000","last_change":"2024-05-09T06:28:43.093492+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093492+0000","last_undegraded":"2024-05-09T06:28:43.093492+0000","last_fullsized":"2024-05-09T06:28:43.093492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.095553+0000","last_change":"2024-05-09T06:28:43.095553+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095553+0000","last_undegraded":"2024-05-09T06:28:43.095553+0000","last_fullsized":"2024-05-09T06:28:43.095553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092622+0000","last_change":"2024-05-09T06:28:43.092622+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092622+0000","last_undegraded":"2024-05-09T06:28:43.092622+0000","last_fullsized":"2024-05-09T06:28:43.092622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094727+0000","last_change":"2024-05-09T06:28:43.094727+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094727+0000","last_undegraded":"2024-05-09T06:28:43.094727+0000","last_fullsized":"2024-05-09T06:28:43.094727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092001+0000","last_change":"2024-05-09T06:28:43.092001+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092001+0000","last_undegraded":"2024-05-09T06:28:43.092001+0000","last_fullsized":"2024-05-09T06:28:43.092001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091798+0000","last_change":"2024-05-09T06:28:43.091798+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091798+0000","last_undegraded":"2024-05-09T06:28:43.091798+0000","last_fullsized":"2024-05-09T06:28:43.091798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091744+0000","last_change":"2024-05-09T06:28:43.091744+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091744+0000","last_undegraded":"2024-05-09T06:28:43.091744+0000","last_fullsized":"2024-05-09T06:28:43.091744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108921+0000","last_change":"2024-05-09T06:28:43.107479+0000","last_active":"2024-05-09T06:28:43.108921+0000","last_peered":"2024-05-09T06:28:43.108921+0000","last_clean":"2024-05-09T06:28:43.108921+0000","last_became_active":"2024-05-09T06:28:43.107175+0000","last_became_peered":"2024-05-09T06:28:43.107175+0000","last_unstale":"2024-05-09T06:28:43.108921+0000","last_undegraded":"2024-05-09T06:28:43.108921+0000","last_fullsized":"2024-05-09T06:28:43.108921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093881+0000","last_change":"2024-05-09T06:28:43.093881+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093881+0000","last_undegraded":"2024-05-09T06:28:43.093881+0000","last_fullsized":"2024-05-09T06:28:43.093881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094244+0000","last_change":"2024-05-09T06:28:43.094244+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094244+0000","last_undegraded":"2024-05-09T06:28:43.094244+0000","last_fullsized":"2024-05-09T06:28:43.094244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094270+0000","last_change":"2024-05-09T06:28:43.094270+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094270+0000","last_undegraded":"2024-05-09T06:28:43.094270+0000","last_fullsized":"2024-05-09T06:28:43.094270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.102412+0000","last_change":"2024-05-09T06:28:43.099299+0000","last_active":"2024-05-09T06:28:43.102412+0000","last_peered":"2024-05-09T06:28:43.102412+0000","last_clean":"2024-05-09T06:28:43.102412+0000","last_became_active":"2024-05-09T06:28:43.099001+0000","last_became_peered":"2024-05-09T06:28:43.099001+0000","last_unstale":"2024-05-09T06:28:43.102412+0000","last_undegraded":"2024-05-09T06:28:43.102412+0000","last_fullsized":"2024-05-09T06:28:43.102412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:59.842925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088924+0000","last_change":"2024-05-09T06:28:43.088924+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088924+0000","last_undegraded":"2024-05-09T06:28:43.088924+0000","last_fullsized":"2024-05-09T06:28:43.088924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108201+0000","last_change":"2024-05-09T06:28:43.101397+0000","last_active":"2024-05-09T06:28:43.108201+0000","last_peered":"2024-05-09T06:28:43.108201+0000","last_clean":"2024-05-09T06:28:43.108201+0000","last_became_active":"2024-05-09T06:28:43.101067+0000","last_became_peered":"2024-05-09T06:28:43.101067+0000","last_unstale":"2024-05-09T06:28:43.108201+0000","last_undegraded":"2024-05-09T06:28:43.108201+0000","last_fullsized":"2024-05-09T06:28:43.108201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:09.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099075+0000","last_change":"2024-05-09T06:28:43.099075+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099075+0000","last_undegraded":"2024-05-09T06:28:43.099075+0000","last_fullsized":"2024-05-09T06:28:43.099075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.092728+0000","last_change":"2024-05-09T06:28:43.092728+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092728+0000","last_undegraded":"2024-05-09T06:28:43.092728+0000","last_fullsized":"2024-05-09T06:28:43.092728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092422+0000","last_change":"2024-05-09T06:28:43.092422+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092422+0000","last_undegraded":"2024-05-09T06:28:43.092422+0000","last_fullsized":"2024-05-09T06:28:43.092422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":6,"reported_epoch":36,"state":"active","last_fresh":"2024-05-09T06:28:43.105390+0000","last_change":"2024-05-09T06:28:43.105323+0000","last_active":"2024-05-09T06:28:43.105390+0000","last_peered":"2024-05-09T06:28:43.105390+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"2024-05-09T06:28:43.105323+0000","last_became_peered":"2024-05-09T06:28:43.105323+0000","last_unstale":"2024-05-09T06:28:43.105390+0000","last_undegraded":"2024-05-09T06:28:43.105390+0000","last_fullsized":"2024-05-09T06:28:43.105390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093177+0000","last_change":"2024-05-09T06:28:43.093177+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093177+0000","last_undegraded":"2024-05-09T06:28:43.093177+0000","last_fullsized":"2024-05-09T06:28:43.093177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108647+0000","last_change":"2024-05-09T06:28:43.105704+0000","last_active":"2024-05-09T06:28:43.108647+0000","last_peered":"2024-05-09T06:28:43.108647+0000","last_clean":"2024-05-09T06:28:43.108647+0000","last_became_active":"2024-05-09T06:28:43.105384+0000","last_became_peered":"2024-05-09T06:28:43.105384+0000","last_unstale":"2024-05-09T06:28:43.108647+0000","last_undegraded":"2024-05-09T06:28:43.108647+0000","last_fullsized":"2024-05-09T06:28:43.108647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:30.961461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091614+0000","last_change":"2024-05-09T06:28:43.091614+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091614+0000","last_undegraded":"2024-05-09T06:28:43.091614+0000","last_fullsized":"2024-05-09T06:28:43.091614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099059+0000","last_change":"2024-05-09T06:28:43.097679+0000","last_active":"2024-05-09T06:28:43.099059+0000","last_peered":"2024-05-09T06:28:43.099059+0000","last_clean":"2024-05-09T06:28:43.099059+0000","last_became_active":"2024-05-09T06:28:43.097373+0000","last_became_peered":"2024-05-09T06:28:43.097373+0000","last_unstale":"2024-05-09T06:28:43.099059+0000","last_undegraded":"2024-05-09T06:28:43.099059+0000","last_fullsized":"2024-05-09T06:28:43.099059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:37.004402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091149+0000","last_change":"2024-05-09T06:28:43.091149+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091149+0000","last_undegraded":"2024-05-09T06:28:43.091149+0000","last_fullsized":"2024-05-09T06:28:43.091149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094410+0000","last_change":"2024-05-09T06:28:43.094410+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094410+0000","last_undegraded":"2024-05-09T06:28:43.094410+0000","last_fullsized":"2024-05-09T06:28:43.094410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091620+0000","last_change":"2024-05-09T06:28:43.091620+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091620+0000","last_undegraded":"2024-05-09T06:28:43.091620+0000","last_fullsized":"2024-05-09T06:28:43.091620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097906+0000","last_change":"2024-05-09T06:28:43.096518+0000","last_active":"2024-05-09T06:28:43.097906+0000","last_peered":"2024-05-09T06:28:43.097906+0000","last_clean":"2024-05-09T06:28:43.097906+0000","last_became_active":"2024-05-09T06:28:43.096235+0000","last_became_peered":"2024-05-09T06:28:43.096235+0000","last_unstale":"2024-05-09T06:28:43.097906+0000","last_undegraded":"2024-05-09T06:28:43.097906+0000","last_fullsized":"2024-05-09T06:28:43.097906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:02.760035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093556+0000","last_change":"2024-05-09T06:28:43.093556+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093556+0000","last_undegraded":"2024-05-09T06:28:43.093556+0000","last_fullsized":"2024-05-09T06:28:43.093556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108183+0000","last_change":"2024-05-09T06:28:43.108183+0000","last_active":"2024-05-09T06:28:43.108183+0000","last_peered":"2024-05-09T06:28:43.108183+0000","last_clean":"2024-05-09T06:28:43.108183+0000","last_became_active":"2024-05-09T06:28:43.107072+0000","last_became_peered":"2024-05-09T06:28:43.107072+0000","last_unstale":"2024-05-09T06:28:43.108183+0000","last_undegraded":"2024-05-09T06:28:43.108183+0000","last_fullsized":"2024-05-09T06:28:43.108183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092413+0000","last_change":"2024-05-09T06:28:43.092413+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092413+0000","last_undegraded":"2024-05-09T06:28:43.092413+0000","last_fullsized":"2024-05-09T06:28:43.092413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098259+0000","last_change":"2024-05-09T06:28:43.098259+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098259+0000","last_undegraded":"2024-05-09T06:28:43.098259+0000","last_fullsized":"2024-05-09T06:28:43.098259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092320+0000","last_change":"2024-05-09T06:28:43.092320+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092320+0000","last_undegraded":"2024-05-09T06:28:43.092320+0000","last_fullsized":"2024-05-09T06:28:43.092320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091605+0000","last_change":"2024-05-09T06:28:43.091605+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091605+0000","last_undegraded":"2024-05-09T06:28:43.091605+0000","last_fullsized":"2024-05-09T06:28:43.091605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094168+0000","last_change":"2024-05-09T06:28:43.094168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094168+0000","last_undegraded":"2024-05-09T06:28:43.094168+0000","last_fullsized":"2024-05-09T06:28:43.094168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.086055+0000","last_change":"2024-05-09T06:28:43.086055+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.086055+0000","last_undegraded":"2024-05-09T06:28:43.086055+0000","last_fullsized":"2024-05-09T06:28:43.086055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088918+0000","last_change":"2024-05-09T06:28:43.088918+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088918+0000","last_undegraded":"2024-05-09T06:28:43.088918+0000","last_fullsized":"2024-05-09T06:28:43.088918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094171+0000","last_change":"2024-05-09T06:28:43.094171+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094171+0000","last_undegraded":"2024-05-09T06:28:43.094171+0000","last_fullsized":"2024-05-09T06:28:43.094171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094312+0000","last_change":"2024-05-09T06:28:43.094312+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094312+0000","last_undegraded":"2024-05-09T06:28:43.094312+0000","last_fullsized":"2024-05-09T06:28:43.094312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085920+0000","last_change":"2024-05-09T06:28:43.085920+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085920+0000","last_undegraded":"2024-05-09T06:28:43.085920+0000","last_fullsized":"2024-05-09T06:28:43.085920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090013+0000","last_change":"2024-05-09T06:28:43.090013+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090013+0000","last_undegraded":"2024-05-09T06:28:43.090013+0000","last_fullsized":"2024-05-09T06:28:43.090013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100163+0000","last_change":"2024-05-09T06:28:43.100163+0000","last_active":"2024-05-09T06:28:43.100163+0000","last_peered":"2024-05-09T06:28:43.100163+0000","last_clean":"2024-05-09T06:28:43.100163+0000","last_became_active":"2024-05-09T06:28:43.099826+0000","last_became_peered":"2024-05-09T06:28:43.099826+0000","last_unstale":"2024-05-09T06:28:43.100163+0000","last_undegraded":"2024-05-09T06:28:43.100163+0000","last_fullsized":"2024-05-09T06:28:43.100163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:11.395053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097458+0000","last_change":"2024-05-09T06:28:43.097458+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097458+0000","last_undegraded":"2024-05-09T06:28:43.097458+0000","last_fullsized":"2024-05-09T06:28:43.097458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.107115+0000","last_change":"2024-05-09T06:28:43.105547+0000","last_active":"2024-05-09T06:28:43.107115+0000","last_peered":"2024-05-09T06:28:43.107115+0000","last_clean":"2024-05-09T06:28:43.107115+0000","last_became_active":"2024-05-09T06:28:43.104806+0000","last_became_peered":"2024-05-09T06:28:43.104806+0000","last_unstale":"2024-05-09T06:28:43.107115+0000","last_undegraded":"2024-05-09T06:28:43.107115+0000","last_fullsized":"2024-05-09T06:28:43.107115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:12.906616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092042+0000","last_change":"2024-05-09T06:28:43.092042+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092042+0000","last_undegraded":"2024-05-09T06:28:43.092042+0000","last_fullsized":"2024-05-09T06:28:43.092042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099690+0000","last_change":"2024-05-09T06:28:43.099690+0000","last_active":"2024-05-09T06:28:43.099690+0000","last_peered":"2024-05-09T06:28:43.099690+0000","last_clean":"2024-05-09T06:28:43.099690+0000","last_became_active":"2024-05-09T06:28:43.099173+0000","last_became_peered":"2024-05-09T06:28:43.099173+0000","last_unstale":"2024-05-09T06:28:43.099690+0000","last_undegraded":"2024-05-09T06:28:43.099690+0000","last_fullsized":"2024-05-09T06:28:43.099690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:36.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094579+0000","last_change":"2024-05-09T06:28:43.094579+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094579+0000","last_undegraded":"2024-05-09T06:28:43.094579+0000","last_fullsized":"2024-05-09T06:28:43.094579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098565+0000","last_change":"2024-05-09T06:28:43.098565+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098565+0000","last_undegraded":"2024-05-09T06:28:43.098565+0000","last_fullsized":"2024-05-09T06:28:43.098565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094618+0000","last_change":"2024-05-09T06:28:43.094618+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094618+0000","last_undegraded":"2024-05-09T06:28:43.094618+0000","last_fullsized":"2024-05-09T06:28:43.094618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085912+0000","last_change":"2024-05-09T06:28:43.085912+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085912+0000","last_undegraded":"2024-05-09T06:28:43.085912+0000","last_fullsized":"2024-05-09T06:28:43.085912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092156+0000","last_change":"2024-05-09T06:28:43.092156+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092156+0000","last_undegraded":"2024-05-09T06:28:43.092156+0000","last_fullsized":"2024-05-09T06:28:43.092156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.090449+0000","last_change":"2024-05-09T06:28:43.090449+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090449+0000","last_undegraded":"2024-05-09T06:28:43.090449+0000","last_fullsized":"2024-05-09T06:28:43.090449+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091192+0000","last_change":"2024-05-09T06:28:43.091192+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091192+0000","last_undegraded":"2024-05-09T06:28:43.091192+0000","last_fullsized":"2024-05-09T06:28:43.091192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097963+0000","last_change":"2024-05-09T06:28:43.097963+0000","last_active":"2024-05-09T06:28:43.097963+0000","last_peered":"2024-05-09T06:28:43.097963+0000","last_clean":"2024-05-09T06:28:43.097963+0000","last_became_active":"2024-05-09T06:28:43.097670+0000","last_became_peered":"2024-05-09T06:28:43.097670+0000","last_unstale":"2024-05-09T06:28:43.097963+0000","last_undegraded":"2024-05-09T06:28:43.097963+0000","last_fullsized":"2024-05-09T06:28:43.097963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095027+0000","last_change":"2024-05-09T06:28:43.095027+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095027+0000","last_undegraded":"2024-05-09T06:28:43.095027+0000","last_fullsized":"2024-05-09T06:28:43.095027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.094929+0000","last_change":"2024-05-09T06:28:43.094133+0000","last_active":"2024-05-09T06:28:43.094929+0000","last_peered":"2024-05-09T06:28:43.094929+0000","last_clean":"2024-05-09T06:28:43.094929+0000","last_became_active":"2024-05-09T06:28:43.093802+0000","last_became_peered":"2024-05-09T06:28:43.093802+0000","last_unstale":"2024-05-09T06:28:43.094929+0000","last_undegraded":"2024-05-09T06:28:43.094929+0000","last_fullsized":"2024-05-09T06:28:43.094929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:21.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095109+0000","last_change":"2024-05-09T06:28:43.095109+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095109+0000","last_undegraded":"2024-05-09T06:28:43.095109+0000","last_fullsized":"2024-05-09T06:28:43.095109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100580+0000","last_change":"2024-05-09T06:28:43.096894+0000","last_active":"2024-05-09T06:28:43.100580+0000","last_peered":"2024-05-09T06:28:43.100580+0000","last_clean":"2024-05-09T06:28:43.100580+0000","last_became_active":"2024-05-09T06:28:43.096592+0000","last_became_peered":"2024-05-09T06:28:43.096592+0000","last_unstale":"2024-05-09T06:28:43.100580+0000","last_undegraded":"2024-05-09T06:28:43.100580+0000","last_fullsized":"2024-05-09T06:28:43.100580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:18:01.363601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093078+0000","last_change":"2024-05-09T06:28:43.093078+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093078+0000","last_undegraded":"2024-05-09T06:28:43.093078+0000","last_fullsized":"2024-05-09T06:28:43.093078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108318+0000","last_change":"2024-05-09T06:28:43.101342+0000","last_active":"2024-05-09T06:28:43.108318+0000","last_peered":"2024-05-09T06:28:43.108318+0000","last_clean":"2024-05-09T06:28:43.108318+0000","last_became_active":"2024-05-09T06:28:43.101027+0000","last_became_peered":"2024-05-09T06:28:43.101027+0000","last_unstale":"2024-05-09T06:28:43.108318+0000","last_undegraded":"2024-05-09T06:28:43.108318+0000","last_fullsized":"2024-05-09T06:28:43.108318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:14.687508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099148+0000","last_change":"2024-05-09T06:28:43.099148+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099148+0000","last_undegraded":"2024-05-09T06:28:43.099148+0000","last_fullsized":"2024-05-09T06:28:43.099148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108722+0000","last_change":"2024-05-09T06:28:43.107403+0000","last_active":"2024-05-09T06:28:43.108722+0000","last_peered":"2024-05-09T06:28:43.108722+0000","last_clean":"2024-05-09T06:28:43.108722+0000","last_became_active":"2024-05-09T06:28:43.107064+0000","last_became_peered":"2024-05-09T06:28:43.107064+0000","last_unstale":"2024-05-09T06:28:43.108722+0000","last_undegraded":"2024-05-09T06:28:43.108722+0000","last_fullsized":"2024-05-09T06:28:43.108722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.161063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100583+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:28:43.100583+0000","last_peered":"2024-05-09T06:28:43.100583+0000","last_clean":"2024-05-09T06:28:43.100583+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:28:43.100583+0000","last_undegraded":"2024-05-09T06:28:43.100583+0000","last_fullsized":"2024-05-09T06:28:43.100583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:28:48.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:28:48.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.625+0000 7f49e359e640 1 Processor -- start 2024-05-09T06:28:48.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.625+0000 7f49e359e640 1 -- start start 2024-05-09T06:28:48.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e359e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f49e4155240 0x7f49e40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e359e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f49e40b3860 0x7f49e415e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e359e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e40b3ee0 con 0x7f49e415eb90 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e359e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f49e415f4d0 con 0x7f49e40b3860 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e359e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f49e415efe0 con 0x7f49e4155240 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e1d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f49e40b3860 0x7f49e415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.626+0000 7f49e1d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f49e40b3860 0x7f49e415e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41848/0 (socket says 172.21.15.1:41848) 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e1d9b640 1 -- 172.21.15.1:0/222971916 learned_addr learned my addr 172.21.15.1:0/222971916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e259c640 1 --2- 172.21.15.1:0/222971916 >> v2:172.21.15.6:3300/0 conn(0x7f49e4155240 0x7f49e40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e1d9b640 1 -- 172.21.15.1:0/222971916 >> v2:172.21.15.6:3300/0 conn(0x7f49e4155240 msgr2=0x7f49e40b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e1d9b640 1 --2- 172.21.15.1:0/222971916 >> v2:172.21.15.6:3300/0 conn(0x7f49e4155240 0x7f49e40b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e1d9b640 1 -- 172.21.15.1:0/222971916 >> v1:172.21.15.6:6789/0 conn(0x7f49e415eb90 legacy=0x7f49e415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.627+0000 7f49e1d9b640 1 -- 172.21.15.1:0/222971916 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e40b41c0 con 0x7f49e40b3860 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e1d9b640 1 --2- 172.21.15.1:0/222971916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e40b3860 0x7f49e415e650 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49e415f770 tx=0x7f49d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e693ad5138644b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e159a640 1 -- 172.21.15.1:0/222971916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8037070 con 0x7f49e40b3860 2024-05-09T06:28:48.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e159a640 1 -- 172.21.15.1:0/222971916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8032440 con 0x7f49e40b3860 2024-05-09T06:28:48.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e159a640 1 -- 172.21.15.1:0/222971916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8032810 con 0x7f49e40b3860 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 -- 172.21.15.1:0/222971916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e40b3860 msgr2=0x7f49e415e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 --2- 172.21.15.1:0/222971916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e40b3860 0x7f49e415e650 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49e415f770 tx=0x7f49d802a270 comp rx=0 tx=0).stop 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 -- 172.21.15.1:0/222971916 shutdown_connections 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 --2- 172.21.15.1:0/222971916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e40b3860 0x7f49e415e650 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 --2- 172.21.15.1:0/222971916 >> v2:172.21.15.6:3300/0 conn(0x7f49e4155240 0x7f49e40b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 -- 172.21.15.1:0/222971916 >> 172.21.15.1:0/222971916 conn(0x7f49e40b1610 msgr2=0x7f49e40b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 -- 172.21.15.1:0/222971916 shutdown_connections 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.628+0000 7f49e359e640 1 -- 172.21.15.1:0/222971916 wait complete. 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.630+0000 7f49e359e640 1 Processor -- start 2024-05-09T06:28:48.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.630+0000 7f49e359e640 1 -- start start 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.630+0000 7f49e359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49e40b3860 0x7f49e414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.630+0000 7f49e359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49e4155240 0x7f49e414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.630+0000 7f49e359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e359e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40b4900 con 0x7f49e415eb90 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40b4780 con 0x7f49e4155240 2024-05-09T06:28:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f49e415d9c0 con 0x7f49e40b3860 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41862/0 (socket says 172.21.15.1:41862) 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 -- 172.21.15.1:0/4119761069 learned_addr learned my addr 172.21.15.1:0/4119761069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e259c640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49e40b3860 0x7f49e414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e1d9b640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49e4155240 0x7f49e414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 -- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49e40b3860 msgr2=0x7f49e414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49e40b3860 0x7f49e414aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 -- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49e4155240 msgr2=0x7f49e414ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49e4155240 0x7f49e414ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 -- 172.21.15.1:0/4119761069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e414de80 con 0x7f49e415eb90 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e2d9d640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c6b0 tx=0x7f49dc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc003b70 con 0x7f49e415eb90 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc003d30 con 0x7f49e415eb90 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e41542e0 con 0x7f49e415eb90 2024-05-09T06:28:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc005310 con 0x7f49e415eb90 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.631+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e4147370 con 0x7f49e415eb90 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.632+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e40b5c00 con 0x7f49e415eb90 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.633+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49dc0054b0 con 0x7f49e415eb90 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.633+0000 7f49d2ffd640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 0x7f49c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.633+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f49dc08ad00 con 0x7f49e415eb90 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.633+0000 7f49e259c640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 0x7f49c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.634+0000 7f49e259c640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 0x7f49c406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006fd0 tx=0x7f49d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.638+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49e40b5c00 con 0x7f49e415eb90 2024-05-09T06:28:48.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.818+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49e40752b0 con 0x7f49c406be30 2024-05-09T06:28:48.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.823+0000 7f49d2ffd640 1 -- 172.21.15.1:0/4119761069 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245559 (secure 0 0 0) 0x7f49e40752b0 con 0x7f49c406be30 2024-05-09T06:28:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:48.809 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:28:48.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.826+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 msgr2=0x7f49c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.826+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 0x7f49c406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006fd0 tx=0x7f49d4008040 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.826+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 msgr2=0x7f49e414b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.826+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c6b0 tx=0x7f49dc00cb70 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 shutdown_connections 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49c406be30 0x7f49c406e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49e415eb90 0x7f49e414b010 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49e4155240 0x7f49e414ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 --2- 172.21.15.1:0/4119761069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49e40b3860 0x7f49e414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 >> 172.21.15.1:0/4119761069 conn(0x7f49e40b1610 msgr2=0x7f49e40cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 shutdown_connections 2024-05-09T06:28:48.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.827+0000 7f49e359e640 1 -- 172.21.15.1:0/4119761069 wait complete. 2024-05-09T06:28:48.820 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-05-09T06:28:47.085680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.007337"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091760+0000","last_change":"2024-05-09T06:28:43.091760+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091760+0000","last_undegraded":"2024-05-09T06:28:43.091760+0000","last_fullsized":"2024-05-09T06:28:43.091760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092634+0000","last_change":"2024-05-09T06:28:43.092634+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092634+0000","last_undegraded":"2024-05-09T06:28:43.092634+0000","last_fullsized":"2024-05-09T06:28:43.092634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090871+0000","last_change":"2024-05-09T06:28:43.090871+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090871+0000","last_undegraded":"2024-05-09T06:28:43.090871+0000","last_fullsized":"2024-05-09T06:28:43.090871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090337+0000","last_change":"2024-05-09T06:28:43.090337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090337+0000","last_undegraded":"2024-05-09T06:28:43.090337+0000","last_fullsized":"2024-05-09T06:28:43.090337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088992+0000","last_change":"2024-05-09T06:28:43.088992+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088992+0000","last_undegraded":"2024-05-09T06:28:43.088992+0000","last_fullsized":"2024-05-09T06:28:43.088992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095168+0000","last_change":"2024-05-09T06:28:43.095168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095168+0000","last_undegraded":"2024-05-09T06:28:43.095168+0000","last_fullsized":"2024-05-09T06:28:43.095168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092833+0000","last_change":"2024-05-09T06:28:43.092833+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092833+0000","last_undegraded":"2024-05-09T06:28:43.092833+0000","last_fullsized":"2024-05-09T06:28:43.092833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091596+0000","last_change":"2024-05-09T06:28:43.091596+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091596+0000","last_undegraded":"2024-05-09T06:28:43.091596+0000","last_fullsized":"2024-05-09T06:28:43.091596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089477+0000","last_change":"2024-05-09T06:28:43.089477+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089477+0000","last_undegraded":"2024-05-09T06:28:43.089477+0000","last_fullsized":"2024-05-09T06:28:43.089477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095941+0000","last_change":"2024-05-09T06:28:43.095941+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095941+0000","last_undegraded":"2024-05-09T06:28:43.095941+0000","last_fullsized":"2024-05-09T06:28:43.095941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092798+0000","last_change":"2024-05-09T06:28:43.092798+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092798+0000","last_undegraded":"2024-05-09T06:28:43.092798+0000","last_fullsized":"2024-05-09T06:28:43.092798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089455+0000","last_change":"2024-05-09T06:28:43.089455+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089455+0000","last_undegraded":"2024-05-09T06:28:43.089455+0000","last_fullsized":"2024-05-09T06:28:43.089455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095544+0000","last_change":"2024-05-09T06:28:43.095544+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095544+0000","last_undegraded":"2024-05-09T06:28:43.095544+0000","last_fullsized":"2024-05-09T06:28:43.095544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095591+0000","last_change":"2024-05-09T06:28:43.095591+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095591+0000","last_undegraded":"2024-05-09T06:28:43.095591+0000","last_fullsized":"2024-05-09T06:28:43.095591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092337+0000","last_change":"2024-05-09T06:28:43.092337+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092337+0000","last_undegraded":"2024-05-09T06:28:43.092337+0000","last_fullsized":"2024-05-09T06:28:43.092337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094761+0000","last_change":"2024-05-09T06:28:43.094761+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094761+0000","last_undegraded":"2024-05-09T06:28:43.094761+0000","last_fullsized":"2024-05-09T06:28:43.094761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097884+0000","last_change":"2024-05-09T06:28:43.097884+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097884+0000","last_undegraded":"2024-05-09T06:28:43.097884+0000","last_fullsized":"2024-05-09T06:28:43.097884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088822+0000","last_change":"2024-05-09T06:28:43.088822+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088822+0000","last_undegraded":"2024-05-09T06:28:43.088822+0000","last_fullsized":"2024-05-09T06:28:43.088822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090190+0000","last_change":"2024-05-09T06:28:43.090190+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090190+0000","last_undegraded":"2024-05-09T06:28:43.090190+0000","last_fullsized":"2024-05-09T06:28:43.090190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088332+0000","last_change":"2024-05-09T06:28:43.088332+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088332+0000","last_undegraded":"2024-05-09T06:28:43.088332+0000","last_fullsized":"2024-05-09T06:28:43.088332+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093271+0000","last_change":"2024-05-09T06:28:43.093271+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093271+0000","last_undegraded":"2024-05-09T06:28:43.093271+0000","last_fullsized":"2024-05-09T06:28:43.093271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094338+0000","last_change":"2024-05-09T06:28:43.094338+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094338+0000","last_undegraded":"2024-05-09T06:28:43.094338+0000","last_fullsized":"2024-05-09T06:28:43.094338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095708+0000","last_change":"2024-05-09T06:28:43.095708+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095708+0000","last_undegraded":"2024-05-09T06:28:43.095708+0000","last_fullsized":"2024-05-09T06:28:43.095708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.089315+0000","last_change":"2024-05-09T06:28:43.089315+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.089315+0000","last_undegraded":"2024-05-09T06:28:43.089315+0000","last_fullsized":"2024-05-09T06:28:43.089315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088440+0000","last_change":"2024-05-09T06:28:43.088440+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088440+0000","last_undegraded":"2024-05-09T06:28:43.088440+0000","last_fullsized":"2024-05-09T06:28:43.088440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094956+0000","last_change":"2024-05-09T06:28:43.094956+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094956+0000","last_undegraded":"2024-05-09T06:28:43.094956+0000","last_fullsized":"2024-05-09T06:28:43.094956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088696+0000","last_change":"2024-05-09T06:28:43.088696+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088696+0000","last_undegraded":"2024-05-09T06:28:43.088696+0000","last_fullsized":"2024-05-09T06:28:43.088696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.096282+0000","last_change":"2024-05-09T06:28:43.096282+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.096282+0000","last_undegraded":"2024-05-09T06:28:43.096282+0000","last_fullsized":"2024-05-09T06:28:43.096282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088202+0000","last_change":"2024-05-09T06:28:43.088202+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088202+0000","last_undegraded":"2024-05-09T06:28:43.088202+0000","last_fullsized":"2024-05-09T06:28:43.088202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088378+0000","last_change":"2024-05-09T06:28:43.088378+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088378+0000","last_undegraded":"2024-05-09T06:28:43.088378+0000","last_fullsized":"2024-05-09T06:28:43.088378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095226+0000","last_change":"2024-05-09T06:28:43.095226+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095226+0000","last_undegraded":"2024-05-09T06:28:43.095226+0000","last_fullsized":"2024-05-09T06:28:43.095226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095399+0000","last_change":"2024-05-09T06:28:43.095399+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095399+0000","last_undegraded":"2024-05-09T06:28:43.095399+0000","last_fullsized":"2024-05-09T06:28:43.095399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092559+0000","last_change":"2024-05-09T06:28:43.092559+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092559+0000","last_undegraded":"2024-05-09T06:28:43.092559+0000","last_fullsized":"2024-05-09T06:28:43.092559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100473+0000","last_change":"2024-05-09T06:28:43.097628+0000","last_active":"2024-05-09T06:28:43.100473+0000","last_peered":"2024-05-09T06:28:43.100473+0000","last_clean":"2024-05-09T06:28:43.100473+0000","last_became_active":"2024-05-09T06:28:43.097348+0000","last_became_peered":"2024-05-09T06:28:43.097348+0000","last_unstale":"2024-05-09T06:28:43.100473+0000","last_undegraded":"2024-05-09T06:28:43.100473+0000","last_fullsized":"2024-05-09T06:28:43.100473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095959+0000","last_change":"2024-05-09T06:28:43.095959+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095959+0000","last_undegraded":"2024-05-09T06:28:43.095959+0000","last_fullsized":"2024-05-09T06:28:43.095959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091246+0000","last_change":"2024-05-09T06:28:43.091246+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091246+0000","last_undegraded":"2024-05-09T06:28:43.091246+0000","last_fullsized":"2024-05-09T06:28:43.091246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093492+0000","last_change":"2024-05-09T06:28:43.093492+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093492+0000","last_undegraded":"2024-05-09T06:28:43.093492+0000","last_fullsized":"2024-05-09T06:28:43.093492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.095553+0000","last_change":"2024-05-09T06:28:43.095553+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095553+0000","last_undegraded":"2024-05-09T06:28:43.095553+0000","last_fullsized":"2024-05-09T06:28:43.095553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092622+0000","last_change":"2024-05-09T06:28:43.092622+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092622+0000","last_undegraded":"2024-05-09T06:28:43.092622+0000","last_fullsized":"2024-05-09T06:28:43.092622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094727+0000","last_change":"2024-05-09T06:28:43.094727+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094727+0000","last_undegraded":"2024-05-09T06:28:43.094727+0000","last_fullsized":"2024-05-09T06:28:43.094727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092001+0000","last_change":"2024-05-09T06:28:43.092001+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092001+0000","last_undegraded":"2024-05-09T06:28:43.092001+0000","last_fullsized":"2024-05-09T06:28:43.092001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091798+0000","last_change":"2024-05-09T06:28:43.091798+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091798+0000","last_undegraded":"2024-05-09T06:28:43.091798+0000","last_fullsized":"2024-05-09T06:28:43.091798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091744+0000","last_change":"2024-05-09T06:28:43.091744+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091744+0000","last_undegraded":"2024-05-09T06:28:43.091744+0000","last_fullsized":"2024-05-09T06:28:43.091744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108921+0000","last_change":"2024-05-09T06:28:43.107479+0000","last_active":"2024-05-09T06:28:43.108921+0000","last_peered":"2024-05-09T06:28:43.108921+0000","last_clean":"2024-05-09T06:28:43.108921+0000","last_became_active":"2024-05-09T06:28:43.107175+0000","last_became_peered":"2024-05-09T06:28:43.107175+0000","last_unstale":"2024-05-09T06:28:43.108921+0000","last_undegraded":"2024-05-09T06:28:43.108921+0000","last_fullsized":"2024-05-09T06:28:43.108921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093881+0000","last_change":"2024-05-09T06:28:43.093881+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093881+0000","last_undegraded":"2024-05-09T06:28:43.093881+0000","last_fullsized":"2024-05-09T06:28:43.093881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094244+0000","last_change":"2024-05-09T06:28:43.094244+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094244+0000","last_undegraded":"2024-05-09T06:28:43.094244+0000","last_fullsized":"2024-05-09T06:28:43.094244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094270+0000","last_change":"2024-05-09T06:28:43.094270+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094270+0000","last_undegraded":"2024-05-09T06:28:43.094270+0000","last_fullsized":"2024-05-09T06:28:43.094270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.102412+0000","last_change":"2024-05-09T06:28:43.099299+0000","last_active":"2024-05-09T06:28:43.102412+0000","last_peered":"2024-05-09T06:28:43.102412+0000","last_clean":"2024-05-09T06:28:43.102412+0000","last_became_active":"2024-05-09T06:28:43.099001+0000","last_became_peered":"2024-05-09T06:28:43.099001+0000","last_unstale":"2024-05-09T06:28:43.102412+0000","last_undegraded":"2024-05-09T06:28:43.102412+0000","last_fullsized":"2024-05-09T06:28:43.102412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:59.842925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088924+0000","last_change":"2024-05-09T06:28:43.088924+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088924+0000","last_undegraded":"2024-05-09T06:28:43.088924+0000","last_fullsized":"2024-05-09T06:28:43.088924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108201+0000","last_change":"2024-05-09T06:28:43.101397+0000","last_active":"2024-05-09T06:28:43.108201+0000","last_peered":"2024-05-09T06:28:43.108201+0000","last_clean":"2024-05-09T06:28:43.108201+0000","last_became_active":"2024-05-09T06:28:43.101067+0000","last_became_peered":"2024-05-09T06:28:43.101067+0000","last_unstale":"2024-05-09T06:28:43.108201+0000","last_undegraded":"2024-05-09T06:28:43.108201+0000","last_fullsized":"2024-05-09T06:28:43.108201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:09.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099075+0000","last_change":"2024-05-09T06:28:43.099075+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099075+0000","last_undegraded":"2024-05-09T06:28:43.099075+0000","last_fullsized":"2024-05-09T06:28:43.099075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.092728+0000","last_change":"2024-05-09T06:28:43.092728+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092728+0000","last_undegraded":"2024-05-09T06:28:43.092728+0000","last_fullsized":"2024-05-09T06:28:43.092728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092422+0000","last_change":"2024-05-09T06:28:43.092422+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092422+0000","last_undegraded":"2024-05-09T06:28:43.092422+0000","last_fullsized":"2024-05-09T06:28:43.092422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":6,"reported_epoch":36,"state":"active","last_fresh":"2024-05-09T06:28:43.105390+0000","last_change":"2024-05-09T06:28:43.105323+0000","last_active":"2024-05-09T06:28:43.105390+0000","last_peered":"2024-05-09T06:28:43.105390+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"2024-05-09T06:28:43.105323+0000","last_became_peered":"2024-05-09T06:28:43.105323+0000","last_unstale":"2024-05-09T06:28:43.105390+0000","last_undegraded":"2024-05-09T06:28:43.105390+0000","last_fullsized":"2024-05-09T06:28:43.105390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093177+0000","last_change":"2024-05-09T06:28:43.093177+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093177+0000","last_undegraded":"2024-05-09T06:28:43.093177+0000","last_fullsized":"2024-05-09T06:28:43.093177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108647+0000","last_change":"2024-05-09T06:28:43.105704+0000","last_active":"2024-05-09T06:28:43.108647+0000","last_peered":"2024-05-09T06:28:43.108647+0000","last_clean":"2024-05-09T06:28:43.108647+0000","last_became_active":"2024-05-09T06:28:43.105384+0000","last_became_peered":"2024-05-09T06:28:43.105384+0000","last_unstale":"2024-05-09T06:28:43.108647+0000","last_undegraded":"2024-05-09T06:28:43.108647+0000","last_fullsized":"2024-05-09T06:28:43.108647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:30.961461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091614+0000","last_change":"2024-05-09T06:28:43.091614+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091614+0000","last_undegraded":"2024-05-09T06:28:43.091614+0000","last_fullsized":"2024-05-09T06:28:43.091614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099059+0000","last_change":"2024-05-09T06:28:43.097679+0000","last_active":"2024-05-09T06:28:43.099059+0000","last_peered":"2024-05-09T06:28:43.099059+0000","last_clean":"2024-05-09T06:28:43.099059+0000","last_became_active":"2024-05-09T06:28:43.097373+0000","last_became_peered":"2024-05-09T06:28:43.097373+0000","last_unstale":"2024-05-09T06:28:43.099059+0000","last_undegraded":"2024-05-09T06:28:43.099059+0000","last_fullsized":"2024-05-09T06:28:43.099059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:37.004402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091149+0000","last_change":"2024-05-09T06:28:43.091149+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091149+0000","last_undegraded":"2024-05-09T06:28:43.091149+0000","last_fullsized":"2024-05-09T06:28:43.091149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094410+0000","last_change":"2024-05-09T06:28:43.094410+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094410+0000","last_undegraded":"2024-05-09T06:28:43.094410+0000","last_fullsized":"2024-05-09T06:28:43.094410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091620+0000","last_change":"2024-05-09T06:28:43.091620+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091620+0000","last_undegraded":"2024-05-09T06:28:43.091620+0000","last_fullsized":"2024-05-09T06:28:43.091620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097906+0000","last_change":"2024-05-09T06:28:43.096518+0000","last_active":"2024-05-09T06:28:43.097906+0000","last_peered":"2024-05-09T06:28:43.097906+0000","last_clean":"2024-05-09T06:28:43.097906+0000","last_became_active":"2024-05-09T06:28:43.096235+0000","last_became_peered":"2024-05-09T06:28:43.096235+0000","last_unstale":"2024-05-09T06:28:43.097906+0000","last_undegraded":"2024-05-09T06:28:43.097906+0000","last_fullsized":"2024-05-09T06:28:43.097906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:02.760035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093556+0000","last_change":"2024-05-09T06:28:43.093556+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093556+0000","last_undegraded":"2024-05-09T06:28:43.093556+0000","last_fullsized":"2024-05-09T06:28:43.093556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108183+0000","last_change":"2024-05-09T06:28:43.108183+0000","last_active":"2024-05-09T06:28:43.108183+0000","last_peered":"2024-05-09T06:28:43.108183+0000","last_clean":"2024-05-09T06:28:43.108183+0000","last_became_active":"2024-05-09T06:28:43.107072+0000","last_became_peered":"2024-05-09T06:28:43.107072+0000","last_unstale":"2024-05-09T06:28:43.108183+0000","last_undegraded":"2024-05-09T06:28:43.108183+0000","last_fullsized":"2024-05-09T06:28:43.108183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092413+0000","last_change":"2024-05-09T06:28:43.092413+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092413+0000","last_undegraded":"2024-05-09T06:28:43.092413+0000","last_fullsized":"2024-05-09T06:28:43.092413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098259+0000","last_change":"2024-05-09T06:28:43.098259+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098259+0000","last_undegraded":"2024-05-09T06:28:43.098259+0000","last_fullsized":"2024-05-09T06:28:43.098259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092320+0000","last_change":"2024-05-09T06:28:43.092320+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092320+0000","last_undegraded":"2024-05-09T06:28:43.092320+0000","last_fullsized":"2024-05-09T06:28:43.092320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.091605+0000","last_change":"2024-05-09T06:28:43.091605+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091605+0000","last_undegraded":"2024-05-09T06:28:43.091605+0000","last_fullsized":"2024-05-09T06:28:43.091605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094168+0000","last_change":"2024-05-09T06:28:43.094168+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094168+0000","last_undegraded":"2024-05-09T06:28:43.094168+0000","last_fullsized":"2024-05-09T06:28:43.094168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.086055+0000","last_change":"2024-05-09T06:28:43.086055+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.086055+0000","last_undegraded":"2024-05-09T06:28:43.086055+0000","last_fullsized":"2024-05-09T06:28:43.086055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.088918+0000","last_change":"2024-05-09T06:28:43.088918+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.088918+0000","last_undegraded":"2024-05-09T06:28:43.088918+0000","last_fullsized":"2024-05-09T06:28:43.088918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.094171+0000","last_change":"2024-05-09T06:28:43.094171+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094171+0000","last_undegraded":"2024-05-09T06:28:43.094171+0000","last_fullsized":"2024-05-09T06:28:43.094171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094312+0000","last_change":"2024-05-09T06:28:43.094312+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094312+0000","last_undegraded":"2024-05-09T06:28:43.094312+0000","last_fullsized":"2024-05-09T06:28:43.094312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085920+0000","last_change":"2024-05-09T06:28:43.085920+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085920+0000","last_undegraded":"2024-05-09T06:28:43.085920+0000","last_fullsized":"2024-05-09T06:28:43.085920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.090013+0000","last_change":"2024-05-09T06:28:43.090013+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090013+0000","last_undegraded":"2024-05-09T06:28:43.090013+0000","last_fullsized":"2024-05-09T06:28:43.090013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100163+0000","last_change":"2024-05-09T06:28:43.100163+0000","last_active":"2024-05-09T06:28:43.100163+0000","last_peered":"2024-05-09T06:28:43.100163+0000","last_clean":"2024-05-09T06:28:43.100163+0000","last_became_active":"2024-05-09T06:28:43.099826+0000","last_became_peered":"2024-05-09T06:28:43.099826+0000","last_unstale":"2024-05-09T06:28:43.100163+0000","last_undegraded":"2024-05-09T06:28:43.100163+0000","last_fullsized":"2024-05-09T06:28:43.100163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:11.395053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.097458+0000","last_change":"2024-05-09T06:28:43.097458+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.097458+0000","last_undegraded":"2024-05-09T06:28:43.097458+0000","last_fullsized":"2024-05-09T06:28:43.097458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.107115+0000","last_change":"2024-05-09T06:28:43.105547+0000","last_active":"2024-05-09T06:28:43.107115+0000","last_peered":"2024-05-09T06:28:43.107115+0000","last_clean":"2024-05-09T06:28:43.107115+0000","last_became_active":"2024-05-09T06:28:43.104806+0000","last_became_peered":"2024-05-09T06:28:43.104806+0000","last_unstale":"2024-05-09T06:28:43.107115+0000","last_undegraded":"2024-05-09T06:28:43.107115+0000","last_fullsized":"2024-05-09T06:28:43.107115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:12.906616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092042+0000","last_change":"2024-05-09T06:28:43.092042+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092042+0000","last_undegraded":"2024-05-09T06:28:43.092042+0000","last_fullsized":"2024-05-09T06:28:43.092042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.099690+0000","last_change":"2024-05-09T06:28:43.099690+0000","last_active":"2024-05-09T06:28:43.099690+0000","last_peered":"2024-05-09T06:28:43.099690+0000","last_clean":"2024-05-09T06:28:43.099690+0000","last_became_active":"2024-05-09T06:28:43.099173+0000","last_became_peered":"2024-05-09T06:28:43.099173+0000","last_unstale":"2024-05-09T06:28:43.099690+0000","last_undegraded":"2024-05-09T06:28:43.099690+0000","last_fullsized":"2024-05-09T06:28:43.099690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:36.029999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094579+0000","last_change":"2024-05-09T06:28:43.094579+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094579+0000","last_undegraded":"2024-05-09T06:28:43.094579+0000","last_fullsized":"2024-05-09T06:28:43.094579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.098565+0000","last_change":"2024-05-09T06:28:43.098565+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.098565+0000","last_undegraded":"2024-05-09T06:28:43.098565+0000","last_fullsized":"2024-05-09T06:28:43.098565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.094618+0000","last_change":"2024-05-09T06:28:43.094618+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.094618+0000","last_undegraded":"2024-05-09T06:28:43.094618+0000","last_fullsized":"2024-05-09T06:28:43.094618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.085912+0000","last_change":"2024-05-09T06:28:43.085912+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.085912+0000","last_undegraded":"2024-05-09T06:28:43.085912+0000","last_fullsized":"2024-05-09T06:28:43.085912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.092156+0000","last_change":"2024-05-09T06:28:43.092156+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.092156+0000","last_undegraded":"2024-05-09T06:28:43.092156+0000","last_fullsized":"2024-05-09T06:28:43.092156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":36,"state":"creating+activating","last_fresh":"2024-05-09T06:28:43.090449+0000","last_change":"2024-05-09T06:28:43.090449+0000","last_active":"2024-05-09T06:28:42.072940+0000","last_peered":"2024-05-09T06:28:42.072940+0000","last_clean":"2024-05-09T06:28:42.072940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.090449+0000","last_undegraded":"2024-05-09T06:28:43.090449+0000","last_fullsized":"2024-05-09T06:28:43.090449+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.091192+0000","last_change":"2024-05-09T06:28:43.091192+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.091192+0000","last_undegraded":"2024-05-09T06:28:43.091192+0000","last_fullsized":"2024-05-09T06:28:43.091192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.097963+0000","last_change":"2024-05-09T06:28:43.097963+0000","last_active":"2024-05-09T06:28:43.097963+0000","last_peered":"2024-05-09T06:28:43.097963+0000","last_clean":"2024-05-09T06:28:43.097963+0000","last_became_active":"2024-05-09T06:28:43.097670+0000","last_became_peered":"2024-05-09T06:28:43.097670+0000","last_unstale":"2024-05-09T06:28:43.097963+0000","last_undegraded":"2024-05-09T06:28:43.097963+0000","last_fullsized":"2024-05-09T06:28:43.097963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095027+0000","last_change":"2024-05-09T06:28:43.095027+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095027+0000","last_undegraded":"2024-05-09T06:28:43.095027+0000","last_fullsized":"2024-05-09T06:28:43.095027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.094929+0000","last_change":"2024-05-09T06:28:43.094133+0000","last_active":"2024-05-09T06:28:43.094929+0000","last_peered":"2024-05-09T06:28:43.094929+0000","last_clean":"2024-05-09T06:28:43.094929+0000","last_became_active":"2024-05-09T06:28:43.093802+0000","last_became_peered":"2024-05-09T06:28:43.093802+0000","last_unstale":"2024-05-09T06:28:43.094929+0000","last_undegraded":"2024-05-09T06:28:43.094929+0000","last_fullsized":"2024-05-09T06:28:43.094929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:21.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.095109+0000","last_change":"2024-05-09T06:28:43.095109+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.095109+0000","last_undegraded":"2024-05-09T06:28:43.095109+0000","last_fullsized":"2024-05-09T06:28:43.095109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100580+0000","last_change":"2024-05-09T06:28:43.096894+0000","last_active":"2024-05-09T06:28:43.100580+0000","last_peered":"2024-05-09T06:28:43.100580+0000","last_clean":"2024-05-09T06:28:43.100580+0000","last_became_active":"2024-05-09T06:28:43.096592+0000","last_became_peered":"2024-05-09T06:28:43.096592+0000","last_unstale":"2024-05-09T06:28:43.100580+0000","last_undegraded":"2024-05-09T06:28:43.100580+0000","last_fullsized":"2024-05-09T06:28:43.100580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:18:01.363601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.093078+0000","last_change":"2024-05-09T06:28:43.093078+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.093078+0000","last_undegraded":"2024-05-09T06:28:43.093078+0000","last_fullsized":"2024-05-09T06:28:43.093078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108318+0000","last_change":"2024-05-09T06:28:43.101342+0000","last_active":"2024-05-09T06:28:43.108318+0000","last_peered":"2024-05-09T06:28:43.108318+0000","last_clean":"2024-05-09T06:28:43.108318+0000","last_became_active":"2024-05-09T06:28:43.101027+0000","last_became_peered":"2024-05-09T06:28:43.101027+0000","last_unstale":"2024-05-09T06:28:43.108318+0000","last_undegraded":"2024-05-09T06:28:43.108318+0000","last_fullsized":"2024-05-09T06:28:43.108318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:14.687508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-09T06:28:43.099148+0000","last_change":"2024-05-09T06:28:43.099148+0000","last_active":"2024-05-09T06:28:43.078294+0000","last_peered":"2024-05-09T06:28:43.078294+0000","last_clean":"2024-05-09T06:28:43.078294+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:28:43.099148+0000","last_undegraded":"2024-05-09T06:28:43.099148+0000","last_fullsized":"2024-05-09T06:28:43.099148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.108722+0000","last_change":"2024-05-09T06:28:43.107403+0000","last_active":"2024-05-09T06:28:43.108722+0000","last_peered":"2024-05-09T06:28:43.108722+0000","last_clean":"2024-05-09T06:28:43.108722+0000","last_became_active":"2024-05-09T06:28:43.107064+0000","last_became_peered":"2024-05-09T06:28:43.107064+0000","last_unstale":"2024-05-09T06:28:43.108722+0000","last_undegraded":"2024-05-09T06:28:43.108722+0000","last_fullsized":"2024-05-09T06:28:43.108722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.161063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:28:43.100583+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:28:43.100583+0000","last_peered":"2024-05-09T06:28:43.100583+0000","last_clean":"2024-05-09T06:28:43.100583+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:28:43.100583+0000","last_undegraded":"2024-05-09T06:28:43.100583+0000","last_fullsized":"2024-05-09T06:28:43.100583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:28:48.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-05-09T06:28:48.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091760+0000', 'last_change': '2024-05-09T06:28:43.091760+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091760+0000', 'last_undegraded': '2024-05-09T06:28:43.091760+0000', 'last_fullsized': '2024-05-09T06:28:43.091760+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092634+0000', 'last_change': '2024-05-09T06:28:43.092634+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092634+0000', 'last_undegraded': '2024-05-09T06:28:43.092634+0000', 'last_fullsized': '2024-05-09T06:28:43.092634+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.090871+0000', 'last_change': '2024-05-09T06:28:43.090871+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.090871+0000', 'last_undegraded': '2024-05-09T06:28:43.090871+0000', 'last_fullsized': '2024-05-09T06:28:43.090871+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.090337+0000', 'last_change': '2024-05-09T06:28:43.090337+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.090337+0000', 'last_undegraded': '2024-05-09T06:28:43.090337+0000', 'last_fullsized': '2024-05-09T06:28:43.090337+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088992+0000', 'last_change': '2024-05-09T06:28:43.088992+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088992+0000', 'last_undegraded': '2024-05-09T06:28:43.088992+0000', 'last_fullsized': '2024-05-09T06:28:43.088992+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-05-09T06:28:48.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095168+0000', 'last_change': '2024-05-09T06:28:43.095168+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095168+0000', 'last_undegraded': '2024-05-09T06:28:43.095168+0000', 'last_fullsized': '2024-05-09T06:28:43.095168+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092833+0000', 'last_change': '2024-05-09T06:28:43.092833+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092833+0000', 'last_undegraded': '2024-05-09T06:28:43.092833+0000', 'last_fullsized': '2024-05-09T06:28:43.092833+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091596+0000', 'last_change': '2024-05-09T06:28:43.091596+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091596+0000', 'last_undegraded': '2024-05-09T06:28:43.091596+0000', 'last_fullsized': '2024-05-09T06:28:43.091596+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.089477+0000', 'last_change': '2024-05-09T06:28:43.089477+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.089477+0000', 'last_undegraded': '2024-05-09T06:28:43.089477+0000', 'last_fullsized': '2024-05-09T06:28:43.089477+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095941+0000', 'last_change': '2024-05-09T06:28:43.095941+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095941+0000', 'last_undegraded': '2024-05-09T06:28:43.095941+0000', 'last_fullsized': '2024-05-09T06:28:43.095941+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-05-09T06:28:48.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092798+0000', 'last_change': '2024-05-09T06:28:43.092798+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092798+0000', 'last_undegraded': '2024-05-09T06:28:43.092798+0000', 'last_fullsized': '2024-05-09T06:28:43.092798+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.089455+0000', 'last_change': '2024-05-09T06:28:43.089455+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.089455+0000', 'last_undegraded': '2024-05-09T06:28:43.089455+0000', 'last_fullsized': '2024-05-09T06:28:43.089455+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095544+0000', 'last_change': '2024-05-09T06:28:43.095544+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095544+0000', 'last_undegraded': '2024-05-09T06:28:43.095544+0000', 'last_fullsized': '2024-05-09T06:28:43.095544+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095591+0000', 'last_change': '2024-05-09T06:28:43.095591+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095591+0000', 'last_undegraded': '2024-05-09T06:28:43.095591+0000', 'last_fullsized': '2024-05-09T06:28:43.095591+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092337+0000', 'last_change': '2024-05-09T06:28:43.092337+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092337+0000', 'last_undegraded': '2024-05-09T06:28:43.092337+0000', 'last_fullsized': '2024-05-09T06:28:43.092337+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-05-09T06:28:48.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094761+0000', 'last_change': '2024-05-09T06:28:43.094761+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094761+0000', 'last_undegraded': '2024-05-09T06:28:43.094761+0000', 'last_fullsized': '2024-05-09T06:28:43.094761+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.097884+0000', 'last_change': '2024-05-09T06:28:43.097884+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.097884+0000', 'last_undegraded': '2024-05-09T06:28:43.097884+0000', 'last_fullsized': '2024-05-09T06:28:43.097884+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088822+0000', 'last_change': '2024-05-09T06:28:43.088822+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088822+0000', 'last_undegraded': '2024-05-09T06:28:43.088822+0000', 'last_fullsized': '2024-05-09T06:28:43.088822+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.090190+0000', 'last_change': '2024-05-09T06:28:43.090190+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.090190+0000', 'last_undegraded': '2024-05-09T06:28:43.090190+0000', 'last_fullsized': '2024-05-09T06:28:43.090190+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088332+0000', 'last_change': '2024-05-09T06:28:43.088332+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088332+0000', 'last_undegraded': '2024-05-09T06:28:43.088332+0000', 'last_fullsized': '2024-05-09T06:28:43.088332+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093271+0000', 'last_change': '2024-05-09T06:28:43.093271+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093271+0000', 'last_undegraded': '2024-05-09T06:28:43.093271+0000', 'last_fullsized': '2024-05-09T06:28:43.093271+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-05-09T06:28:48.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094338+0000', 'last_change': '2024-05-09T06:28:43.094338+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094338+0000', 'last_undegraded': '2024-05-09T06:28:43.094338+0000', 'last_fullsized': '2024-05-09T06:28:43.094338+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095708+0000', 'last_change': '2024-05-09T06:28:43.095708+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095708+0000', 'last_undegraded': '2024-05-09T06:28:43.095708+0000', 'last_fullsized': '2024-05-09T06:28:43.095708+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.089315+0000', 'last_change': '2024-05-09T06:28:43.089315+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.089315+0000', 'last_undegraded': '2024-05-09T06:28:43.089315+0000', 'last_fullsized': '2024-05-09T06:28:43.089315+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088440+0000', 'last_change': '2024-05-09T06:28:43.088440+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088440+0000', 'last_undegraded': '2024-05-09T06:28:43.088440+0000', 'last_fullsized': '2024-05-09T06:28:43.088440+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094956+0000', 'last_change': '2024-05-09T06:28:43.094956+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094956+0000', 'last_undegraded': '2024-05-09T06:28:43.094956+0000', 'last_fullsized': '2024-05-09T06:28:43.094956+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088696+0000', 'last_change': '2024-05-09T06:28:43.088696+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088696+0000', 'last_undegraded': '2024-05-09T06:28:43.088696+0000', 'last_fullsized': '2024-05-09T06:28:43.088696+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-05-09T06:28:48.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.096282+0000', 'last_change': '2024-05-09T06:28:43.096282+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.096282+0000', 'last_undegraded': '2024-05-09T06:28:43.096282+0000', 'last_fullsized': '2024-05-09T06:28:43.096282+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088202+0000', 'last_change': '2024-05-09T06:28:43.088202+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088202+0000', 'last_undegraded': '2024-05-09T06:28:43.088202+0000', 'last_fullsized': '2024-05-09T06:28:43.088202+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088378+0000', 'last_change': '2024-05-09T06:28:43.088378+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088378+0000', 'last_undegraded': '2024-05-09T06:28:43.088378+0000', 'last_fullsized': '2024-05-09T06:28:43.088378+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095226+0000', 'last_change': '2024-05-09T06:28:43.095226+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095226+0000', 'last_undegraded': '2024-05-09T06:28:43.095226+0000', 'last_fullsized': '2024-05-09T06:28:43.095226+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095399+0000', 'last_change': '2024-05-09T06:28:43.095399+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095399+0000', 'last_undegraded': '2024-05-09T06:28:43.095399+0000', 'last_fullsized': '2024-05-09T06:28:43.095399+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092559+0000', 'last_change': '2024-05-09T06:28:43.092559+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092559+0000', 'last_undegraded': '2024-05-09T06:28:43.092559+0000', 'last_fullsized': '2024-05-09T06:28:43.092559+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095959+0000', 'last_change': '2024-05-09T06:28:43.095959+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095959+0000', 'last_undegraded': '2024-05-09T06:28:43.095959+0000', 'last_fullsized': '2024-05-09T06:28:43.095959+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.091246+0000', 'last_change': '2024-05-09T06:28:43.091246+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091246+0000', 'last_undegraded': '2024-05-09T06:28:43.091246+0000', 'last_fullsized': '2024-05-09T06:28:43.091246+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093492+0000', 'last_change': '2024-05-09T06:28:43.093492+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093492+0000', 'last_undegraded': '2024-05-09T06:28:43.093492+0000', 'last_fullsized': '2024-05-09T06:28:43.093492+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.095553+0000', 'last_change': '2024-05-09T06:28:43.095553+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095553+0000', 'last_undegraded': '2024-05-09T06:28:43.095553+0000', 'last_fullsized': '2024-05-09T06:28:43.095553+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': ['1', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092622+0000', 'last_change': '2024-05-09T06:28:43.092622+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092622+0000', 'last_undegraded': '2024-05-09T06:28:43.092622+0000', 'last_fullsized': '2024-05-09T06:28:43.092622+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-05-09T06:28:48.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.094727+0000', 'last_change': '2024-05-09T06:28:43.094727+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094727+0000', 'last_undegraded': '2024-05-09T06:28:43.094727+0000', 'last_fullsized': '2024-05-09T06:28:43.094727+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092001+0000', 'last_change': '2024-05-09T06:28:43.092001+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092001+0000', 'last_undegraded': '2024-05-09T06:28:43.092001+0000', 'last_fullsized': '2024-05-09T06:28:43.092001+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.091798+0000', 'last_change': '2024-05-09T06:28:43.091798+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091798+0000', 'last_undegraded': '2024-05-09T06:28:43.091798+0000', 'last_fullsized': '2024-05-09T06:28:43.091798+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-05-09T06:28:48.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091744+0000', 'last_change': '2024-05-09T06:28:43.091744+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091744+0000', 'last_undegraded': '2024-05-09T06:28:43.091744+0000', 'last_fullsized': '2024-05-09T06:28:43.091744+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093881+0000', 'last_change': '2024-05-09T06:28:43.093881+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093881+0000', 'last_undegraded': '2024-05-09T06:28:43.093881+0000', 'last_fullsized': '2024-05-09T06:28:43.093881+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.094244+0000', 'last_change': '2024-05-09T06:28:43.094244+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094244+0000', 'last_undegraded': '2024-05-09T06:28:43.094244+0000', 'last_fullsized': '2024-05-09T06:28:43.094244+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094270+0000', 'last_change': '2024-05-09T06:28:43.094270+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094270+0000', 'last_undegraded': '2024-05-09T06:28:43.094270+0000', 'last_fullsized': '2024-05-09T06:28:43.094270+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088924+0000', 'last_change': '2024-05-09T06:28:43.088924+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088924+0000', 'last_undegraded': '2024-05-09T06:28:43.088924+0000', 'last_fullsized': '2024-05-09T06:28:43.088924+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.099075+0000', 'last_change': '2024-05-09T06:28:43.099075+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.099075+0000', 'last_undegraded': '2024-05-09T06:28:43.099075+0000', 'last_fullsized': '2024-05-09T06:28:43.099075+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.092728+0000', 'last_change': '2024-05-09T06:28:43.092728+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092728+0000', 'last_undegraded': '2024-05-09T06:28:43.092728+0000', 'last_fullsized': '2024-05-09T06:28:43.092728+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092422+0000', 'last_change': '2024-05-09T06:28:43.092422+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092422+0000', 'last_undegraded': '2024-05-09T06:28:43.092422+0000', 'last_fullsized': '2024-05-09T06:28:43.092422+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 36, 'state': 'active', 'last_fresh': '2024-05-09T06:28:43.105390+0000', 'last_change': '2024-05-09T06:28:43.105323+0000', 'last_active': '2024-05-09T06:28:43.105390+0000', 'last_peered': '2024-05-09T06:28:43.105390+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '2024-05-09T06:28:43.105323+0000', 'last_became_peered': '2024-05-09T06:28:43.105323+0000', 'last_unstale': '2024-05-09T06:28:43.105390+0000', 'last_undegraded': '2024-05-09T06:28:43.105390+0000', 'last_fullsized': '2024-05-09T06:28:43.105390+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093177+0000', 'last_change': '2024-05-09T06:28:43.093177+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093177+0000', 'last_undegraded': '2024-05-09T06:28:43.093177+0000', 'last_fullsized': '2024-05-09T06:28:43.093177+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091614+0000', 'last_change': '2024-05-09T06:28:43.091614+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091614+0000', 'last_undegraded': '2024-05-09T06:28:43.091614+0000', 'last_fullsized': '2024-05-09T06:28:43.091614+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091149+0000', 'last_change': '2024-05-09T06:28:43.091149+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091149+0000', 'last_undegraded': '2024-05-09T06:28:43.091149+0000', 'last_fullsized': '2024-05-09T06:28:43.091149+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.094410+0000', 'last_change': '2024-05-09T06:28:43.094410+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094410+0000', 'last_undegraded': '2024-05-09T06:28:43.094410+0000', 'last_fullsized': '2024-05-09T06:28:43.094410+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091620+0000', 'last_change': '2024-05-09T06:28:43.091620+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091620+0000', 'last_undegraded': '2024-05-09T06:28:43.091620+0000', 'last_fullsized': '2024-05-09T06:28:43.091620+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-05-09T06:28:48.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093556+0000', 'last_change': '2024-05-09T06:28:43.093556+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093556+0000', 'last_undegraded': '2024-05-09T06:28:43.093556+0000', 'last_fullsized': '2024-05-09T06:28:43.093556+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092413+0000', 'last_change': '2024-05-09T06:28:43.092413+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092413+0000', 'last_undegraded': '2024-05-09T06:28:43.092413+0000', 'last_fullsized': '2024-05-09T06:28:43.092413+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.098259+0000', 'last_change': '2024-05-09T06:28:43.098259+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.098259+0000', 'last_undegraded': '2024-05-09T06:28:43.098259+0000', 'last_fullsized': '2024-05-09T06:28:43.098259+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092320+0000', 'last_change': '2024-05-09T06:28:43.092320+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092320+0000', 'last_undegraded': '2024-05-09T06:28:43.092320+0000', 'last_fullsized': '2024-05-09T06:28:43.092320+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.091605+0000', 'last_change': '2024-05-09T06:28:43.091605+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091605+0000', 'last_undegraded': '2024-05-09T06:28:43.091605+0000', 'last_fullsized': '2024-05-09T06:28:43.091605+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094168+0000', 'last_change': '2024-05-09T06:28:43.094168+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094168+0000', 'last_undegraded': '2024-05-09T06:28:43.094168+0000', 'last_fullsized': '2024-05-09T06:28:43.094168+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.086055+0000', 'last_change': '2024-05-09T06:28:43.086055+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.086055+0000', 'last_undegraded': '2024-05-09T06:28:43.086055+0000', 'last_fullsized': '2024-05-09T06:28:43.086055+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-05-09T06:28:48.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.088918+0000', 'last_change': '2024-05-09T06:28:43.088918+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.088918+0000', 'last_undegraded': '2024-05-09T06:28:43.088918+0000', 'last_fullsized': '2024-05-09T06:28:43.088918+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.094171+0000', 'last_change': '2024-05-09T06:28:43.094171+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094171+0000', 'last_undegraded': '2024-05-09T06:28:43.094171+0000', 'last_fullsized': '2024-05-09T06:28:43.094171+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': ['1', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094312+0000', 'last_change': '2024-05-09T06:28:43.094312+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094312+0000', 'last_undegraded': '2024-05-09T06:28:43.094312+0000', 'last_fullsized': '2024-05-09T06:28:43.094312+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-05-09T06:28:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.085920+0000', 'last_change': '2024-05-09T06:28:43.085920+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.085920+0000', 'last_undegraded': '2024-05-09T06:28:43.085920+0000', 'last_fullsized': '2024-05-09T06:28:43.085920+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.090013+0000', 'last_change': '2024-05-09T06:28:43.090013+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.090013+0000', 'last_undegraded': '2024-05-09T06:28:43.090013+0000', 'last_fullsized': '2024-05-09T06:28:43.090013+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.097458+0000', 'last_change': '2024-05-09T06:28:43.097458+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.097458+0000', 'last_undegraded': '2024-05-09T06:28:43.097458+0000', 'last_fullsized': '2024-05-09T06:28:43.097458+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-05-09T06:28:48.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092042+0000', 'last_change': '2024-05-09T06:28:43.092042+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092042+0000', 'last_undegraded': '2024-05-09T06:28:43.092042+0000', 'last_fullsized': '2024-05-09T06:28:43.092042+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094579+0000', 'last_change': '2024-05-09T06:28:43.094579+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094579+0000', 'last_undegraded': '2024-05-09T06:28:43.094579+0000', 'last_fullsized': '2024-05-09T06:28:43.094579+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.098565+0000', 'last_change': '2024-05-09T06:28:43.098565+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.098565+0000', 'last_undegraded': '2024-05-09T06:28:43.098565+0000', 'last_fullsized': '2024-05-09T06:28:43.098565+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.094618+0000', 'last_change': '2024-05-09T06:28:43.094618+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.094618+0000', 'last_undegraded': '2024-05-09T06:28:43.094618+0000', 'last_fullsized': '2024-05-09T06:28:43.094618+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.085912+0000', 'last_change': '2024-05-09T06:28:43.085912+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.085912+0000', 'last_undegraded': '2024-05-09T06:28:43.085912+0000', 'last_fullsized': '2024-05-09T06:28:43.085912+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.092156+0000', 'last_change': '2024-05-09T06:28:43.092156+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.092156+0000', 'last_undegraded': '2024-05-09T06:28:43.092156+0000', 'last_fullsized': '2024-05-09T06:28:43.092156+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 36, 'state': 'creating+activating', 'last_fresh': '2024-05-09T06:28:43.090449+0000', 'last_change': '2024-05-09T06:28:43.090449+0000', 'last_active': '2024-05-09T06:28:42.072940+0000', 'last_peered': '2024-05-09T06:28:42.072940+0000', 'last_clean': '2024-05-09T06:28:42.072940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.090449+0000', 'last_undegraded': '2024-05-09T06:28:43.090449+0000', 'last_fullsized': '2024-05-09T06:28:43.090449+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:42.072940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.091192+0000', 'last_change': '2024-05-09T06:28:43.091192+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.091192+0000', 'last_undegraded': '2024-05-09T06:28:43.091192+0000', 'last_fullsized': '2024-05-09T06:28:43.091192+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095027+0000', 'last_change': '2024-05-09T06:28:43.095027+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095027+0000', 'last_undegraded': '2024-05-09T06:28:43.095027+0000', 'last_fullsized': '2024-05-09T06:28:43.095027+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.095109+0000', 'last_change': '2024-05-09T06:28:43.095109+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.095109+0000', 'last_undegraded': '2024-05-09T06:28:43.095109+0000', 'last_fullsized': '2024-05-09T06:28:43.095109+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.093078+0000', 'last_change': '2024-05-09T06:28:43.093078+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.093078+0000', 'last_undegraded': '2024-05-09T06:28:43.093078+0000', 'last_fullsized': '2024-05-09T06:28:43.093078+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-05-09T06:28:48.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:28:43.099148+0000', 'last_change': '2024-05-09T06:28:43.099148+0000', 'last_active': '2024-05-09T06:28:43.078294+0000', 'last_peered': '2024-05-09T06:28:43.078294+0000', 'last_clean': '2024-05-09T06:28:43.078294+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:28:43.099148+0000', 'last_undegraded': '2024-05-09T06:28:43.099148+0000', 'last_fullsized': '2024-05-09T06:28:43.099148+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'last_clean_scrub_stamp': '2024-05-09T06:28:43.078294+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:28:48.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-09T06:28:48.881 INFO:tasks.ceph.mds.d.smithi006.stderr:2024-05-09T06:28:48.899+0000 7f92c8edc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:28:48.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.981+0000 7fda7203f640 1 Processor -- start 2024-05-09T06:28:48.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.981+0000 7fda7203f640 1 -- start start 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fda6c0b0080 0x7fda6c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c0cbb60 0x7fda6c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda6c15fd70 0x7fda6c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c0cbfe0 con 0x7fda6c0b0080 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c0d04a0 con 0x7fda6c15fd70 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda7203f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c0722c0 con 0x7fda6c0cbb60 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fda6c0b0080 0x7fda6c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fda6c0b0080 0x7fda6c071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41868/0 (socket says 172.21.15.1:41868) 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 -- 172.21.15.1:0/1286438060 learned_addr learned my addr 172.21.15.1:0/1286438060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6affd640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c0cbb60 0x7fda6c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6bfff640 1 --2- 172.21.15.1:0/1286438060 >> v2:172.21.15.6:3300/0 conn(0x7fda6c15fd70 0x7fda6c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 -- 172.21.15.1:0/1286438060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c0cbb60 msgr2=0x7fda6c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c0cbb60 0x7fda6c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 -- 172.21.15.1:0/1286438060 >> v2:172.21.15.6:3300/0 conn(0x7fda6c15fd70 msgr2=0x7fda6c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/1286438060 >> v2:172.21.15.6:3300/0 conn(0x7fda6c15fd70 0x7fda6c15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.982+0000 7fda6b7fe640 1 -- 172.21.15.1:0/1286438060 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c0725a0 con 0x7fda6c0b0080 2024-05-09T06:28:48.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.983+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0b0080 0x7fda6c071ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0b2690 tx=0x7fda5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff47500e489867ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.983+0000 7fda6a7fc640 1 -- 172.21.15.1:0/1286438060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c037070 con 0x7fda6c0b0080 2024-05-09T06:28:48.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.983+0000 7fda6a7fc640 1 -- 172.21.15.1:0/1286438060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c032440 con 0x7fda6c0b0080 2024-05-09T06:28:48.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.983+0000 7fda6a7fc640 1 -- 172.21.15.1:0/1286438060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c032810 con 0x7fda6c0b0080 2024-05-09T06:28:48.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 -- 172.21.15.1:0/1286438060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0b0080 msgr2=0x7fda6c071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0b0080 0x7fda6c071ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0b2690 tx=0x7fda5c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 -- 172.21.15.1:0/1286438060 shutdown_connections 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 --2- 172.21.15.1:0/1286438060 >> v2:172.21.15.6:3300/0 conn(0x7fda6c15fd70 0x7fda6c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c0cbb60 0x7fda6c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 --2- 172.21.15.1:0/1286438060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0b0080 0x7fda6c071ce0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 -- 172.21.15.1:0/1286438060 >> 172.21.15.1:0/1286438060 conn(0x7fda6c0d0b50 msgr2=0x7fda6c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 -- 172.21.15.1:0/1286438060 shutdown_connections 2024-05-09T06:28:48.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.984+0000 7fda7203f640 1 -- 172.21.15.1:0/1286438060 wait complete. 2024-05-09T06:28:48.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.985+0000 7fda7203f640 1 Processor -- start 2024-05-09T06:28:48.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.985+0000 7fda7203f640 1 -- start start 2024-05-09T06:28:48.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 0x7fda6c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 0x7fda6c0b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c15e7c0 con 0x7fda6c0cbb60 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c072b60 con 0x7fda6c0b0080 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda7203f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c15f220 con 0x7fda6c15fd70 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41878/0 (socket says 172.21.15.1:41878) 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 -- 172.21.15.1:0/3170058783 learned_addr learned my addr 172.21.15.1:0/3170058783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6bfff640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 0x7fda6c0b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 0x7fda6c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 -- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 msgr2=0x7fda6c0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 0x7fda6c0b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 -- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 msgr2=0x7fda6c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 0x7fda6c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:48.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 -- 172.21.15.1:0/3170058783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c0c8190 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6bfff640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 0x7fda6c0b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 0x7fda6c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.986+0000 7fda6affd640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c820 tx=0x7fda6000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.987+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60015070 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.987+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60004220 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.987+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c0c8470 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.987+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6000ce10 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.987+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c0c8d10 con 0x7fda6c0cbb60 2024-05-09T06:28:48.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.988+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c0c8f10 con 0x7fda6c0cbb60 2024-05-09T06:28:48.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.988+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda60007830 con 0x7fda6c0cbb60 2024-05-09T06:28:48.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.989+0000 7fda53fff640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 0x7fda5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:48.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.989+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fda6008aed0 con 0x7fda6c0cbb60 2024-05-09T06:28:48.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.989+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 0x7fda5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:48.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.989+0000 7fda6b7fe640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 0x7fda5806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda5c034000 tx=0x7fda5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:48.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:48.993+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda6c0c8f10 con 0x7fda6c0cbb60 2024-05-09T06:28:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.251+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fda6c0b1160 con 0x7fda6c0cbb60 2024-05-09T06:28:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.251+0000 7fda53fff640 1 -- 172.21.15.1:0/3170058783 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1013 (secure 0 0 0) 0x7fda60059650 con 0x7fda6c0cbb60 2024-05-09T06:28:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:49.234 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":112,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1715236027,"num_in_osds":8,"osd_in_since":1715236019,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":243163136,"bytes_avail":772850950144,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":5550}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-09T06:28:27.082020+0000","services":{}},"progress_events":{}} 2024-05-09T06:28:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 msgr2=0x7fda5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 0x7fda5806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda5c034000 tx=0x7fda5c02e000 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 msgr2=0x7fda6c1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c820 tx=0x7fda6000cce0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 shutdown_connections 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fda5806be30 0x7fda5806e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fda6c15fd70 0x7fda6c0b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda6c0cbb60 0x7fda6c1589f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 --2- 172.21.15.1:0/3170058783 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda6c0b0080 0x7fda6c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.253+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 >> 172.21.15.1:0/3170058783 conn(0x7fda6c0d0b50 msgr2=0x7fda6c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.254+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 shutdown_connections 2024-05-09T06:28:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.254+0000 7fda7203f640 1 -- 172.21.15.1:0/3170058783 wait complete. 2024-05-09T06:28:49.246 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-09T06:28:49.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:28:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.369+0000 7f37b2af5640 1 Processor -- start 2024-05-09T06:28:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.369+0000 7f37b2af5640 1 -- start start 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b2af5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f37ac0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b2af5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f37ac0b0d00 0x7f37ac0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b2af5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac0d2c80 con 0x7f37ac0d05e0 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b2af5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f37ac0d29a0 con 0x7f37ac0b0d00 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b2af5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac0cd430 con 0x7f37ac15c320 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b1092640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f37ac0d05e0 0x7f37ac0ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58596/0 (socket says 172.21.15.1:58596) 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b1092640 1 -- 172.21.15.1:0/631800506 learned_addr learned my addr 172.21.15.1:0/631800506 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37abfff640 1 --2- 172.21.15.1:0/631800506 >> v2:172.21.15.6:3300/0 conn(0x7f37ac0b0d00 0x7f37ac0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37b0891640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f37ac0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626497598 0 0) 0x7f37ac0d2c80 con 0x7f37ac0d05e0 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f379c003520 con 0x7f37ac0d05e0 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2826601029 0 0) 0x7f379c003520 con 0x7f37ac0d05e0 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 msgr2=0x7f37ac0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f37ac0b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 >> v2:172.21.15.6:3300/0 conn(0x7f37ac0b0d00 msgr2=0x7f37ac0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 --2- 172.21.15.1:0/631800506 >> v2:172.21.15.6:3300/0 conn(0x7f37ac0b0d00 0x7f37ac0d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac0cd710 con 0x7f37ac0d05e0 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f37a40019f0 con 0x7f37ac0d05e0 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 >> v1:172.21.15.1:6789/0 conn(0x7f37ac0d05e0 legacy=0x7f37ac0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f379c0039b0 0x7f379c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.370+0000 7f37ab7fe640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c004800 0x7f379c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37ab7fe640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f379c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac0d29a0 con 0x7f379c0039b0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac0cd430 con 0x7f379c004800 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f379c00b9c0 con 0x7f37ac15c320 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f379c0039b0 0x7f379c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b1092640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f379c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37abfff640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c004800 0x7f379c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 -- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 msgr2=0x7f379c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37ac15c320 0x7f379c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 -- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c004800 msgr2=0x7f379c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f379c004800 0x7f379c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.371+0000 7f37b0891640 1 -- 172.21.15.1:0/631800506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c003520 con 0x7f379c0039b0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b0891640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f379c0039b0 0x7f379c003dd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009ac0 tx=0x7f379c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd6fa44ae361591 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0036070 con 0x7f379c0039b0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a0031400 con 0x7f379c0039b0 2024-05-09T06:28:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37ab7fe640 1 -- 172.21.15.1:0/631800506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0009690 con 0x7f379c0039b0 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 -- 172.21.15.1:0/631800506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f379c0039b0 msgr2=0x7f379c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 --2- 172.21.15.1:0/631800506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f379c0039b0 0x7f379c003dd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009ac0 tx=0x7f379c005830 comp rx=0 tx=0).stop 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b1092640 1 -- 172.21.15.1:0/631800506 reap_dead start 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 -- 172.21.15.1:0/631800506 shutdown_connections 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 -- 172.21.15.1:0/631800506 >> 172.21.15.1:0/631800506 conn(0x7f37ac0d1d70 msgr2=0x7f37ac0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 -- 172.21.15.1:0/631800506 shutdown_connections 2024-05-09T06:28:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.372+0000 7f37b2af5640 1 -- 172.21.15.1:0/631800506 wait complete. 2024-05-09T06:28:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.374+0000 7f37b2af5640 1 Processor -- start 2024-05-09T06:28:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.374+0000 7f37b2af5640 1 -- start start 2024-05-09T06:28:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f379c0039b0 0x7f37ac14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 0x7f37ac14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac070020 con 0x7f37ac09c950 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac0cfde0 con 0x7f37ac09f520 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac0701a0 con 0x7f379c0039b0 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41904/0 (socket says 172.21.15.1:41904) 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 -- 172.21.15.1:0/163736689 learned_addr learned my addr 172.21.15.1:0/163736689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b0891640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f379c0039b0 0x7f37ac14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b1092640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 0x7f37ac14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 -- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f379c0039b0 msgr2=0x7f37ac14d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f379c0039b0 0x7f37ac14d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 -- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 msgr2=0x7f37ac14fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 0x7f37ac14fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 -- 172.21.15.1:0/163736689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac09bbb0 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b1092640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 0x7f37ac14fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37abfff640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3798011960 tx=0x7f3798011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379800faf0 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379800fc90 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac150930 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798014440 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.375+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac09c150 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.377+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac0752b0 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.377+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3798012040 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.377+0000 7f37a97fa640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 0x7f379006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.378+0000 7f37b0891640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 0x7f379006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.378+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f379808b600 con 0x7f37ac09c950 2024-05-09T06:28:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.378+0000 7f37b0891640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 0x7f379006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37a0033000 tx=0x7f37a0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.382+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3798019030 con 0x7f37ac09c950 2024-05-09T06:28:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.560+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37ac0d2c80 con 0x7f379006be30 2024-05-09T06:28:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.566+0000 7f37a97fa640 1 -- 172.21.15.1:0/163736689 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f37ac0d2c80 con 0x7f379006be30 2024-05-09T06:28:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:28:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 msgr2=0x7f379006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 0x7f379006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37a0033000 tx=0x7f37a0008030 comp rx=0 tx=0).stop 2024-05-09T06:28:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 msgr2=0x7f37ac09efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3798011960 tx=0x7f3798011e20 comp rx=0 tx=0).stop 2024-05-09T06:28:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 shutdown_connections 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f379006be30 0x7f379006e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37ac09f520 0x7f37ac14fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37ac09c950 0x7f37ac09efe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 --2- 172.21.15.1:0/163736689 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f379c0039b0 0x7f37ac14d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 >> 172.21.15.1:0/163736689 conn(0x7f37ac0d1d70 msgr2=0x7f37ac0cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 shutdown_connections 2024-05-09T06:28:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:49.569+0000 7f37b2af5640 1 -- 172.21.15.1:0/163736689 wait complete. 2024-05-09T06:28:49.562 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-05-09T06:28:49.086030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237464,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737256,"statfs":{"total":773094113280,"available":772850950144,"internally_reserved":0,"allocated":4759552,"data_stored":2705844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001889"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804839+0000","last_change":"2024-05-09T06:28:44.104786+0000","last_active":"2024-05-09T06:28:45.804839+0000","last_peered":"2024-05-09T06:28:45.804839+0000","last_clean":"2024-05-09T06:28:45.804839+0000","last_became_active":"2024-05-09T06:28:44.104499+0000","last_became_peered":"2024-05-09T06:28:44.104499+0000","last_unstale":"2024-05-09T06:28:45.804839+0000","last_undegraded":"2024-05-09T06:28:45.804839+0000","last_fullsized":"2024-05-09T06:28:45.804839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.380616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804104+0000","last_change":"2024-05-09T06:28:44.102227+0000","last_active":"2024-05-09T06:28:45.804104+0000","last_peered":"2024-05-09T06:28:45.804104+0000","last_clean":"2024-05-09T06:28:45.804104+0000","last_became_active":"2024-05-09T06:28:44.101728+0000","last_became_peered":"2024-05-09T06:28:44.101728+0000","last_unstale":"2024-05-09T06:28:45.804104+0000","last_undegraded":"2024-05-09T06:28:45.804104+0000","last_fullsized":"2024-05-09T06:28:45.804104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:27:36.391827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804289+0000","last_change":"2024-05-09T06:28:44.122863+0000","last_active":"2024-05-09T06:28:45.804289+0000","last_peered":"2024-05-09T06:28:45.804289+0000","last_clean":"2024-05-09T06:28:45.804289+0000","last_became_active":"2024-05-09T06:28:44.122578+0000","last_became_peered":"2024-05-09T06:28:44.122578+0000","last_unstale":"2024-05-09T06:28:45.804289+0000","last_undegraded":"2024-05-09T06:28:45.804289+0000","last_fullsized":"2024-05-09T06:28:45.804289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088431+0000","last_change":"2024-05-09T06:28:44.094799+0000","last_active":"2024-05-09T06:28:45.088431+0000","last_peered":"2024-05-09T06:28:45.088431+0000","last_clean":"2024-05-09T06:28:45.088431+0000","last_became_active":"2024-05-09T06:28:44.094382+0000","last_became_peered":"2024-05-09T06:28:44.094382+0000","last_unstale":"2024-05-09T06:28:45.088431+0000","last_undegraded":"2024-05-09T06:28:45.088431+0000","last_fullsized":"2024-05-09T06:28:45.088431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:45.166417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088180+0000","last_change":"2024-05-09T06:28:44.095845+0000","last_active":"2024-05-09T06:28:45.088180+0000","last_peered":"2024-05-09T06:28:45.088180+0000","last_clean":"2024-05-09T06:28:45.088180+0000","last_became_active":"2024-05-09T06:28:44.095531+0000","last_became_peered":"2024-05-09T06:28:44.095531+0000","last_unstale":"2024-05-09T06:28:45.088180+0000","last_undegraded":"2024-05-09T06:28:45.088180+0000","last_fullsized":"2024-05-09T06:28:45.088180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:13.911970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804818+0000","last_change":"2024-05-09T06:28:44.108110+0000","last_active":"2024-05-09T06:28:45.804818+0000","last_peered":"2024-05-09T06:28:45.804818+0000","last_clean":"2024-05-09T06:28:45.804818+0000","last_became_active":"2024-05-09T06:28:44.107839+0000","last_became_peered":"2024-05-09T06:28:44.107839+0000","last_unstale":"2024-05-09T06:28:45.804818+0000","last_undegraded":"2024-05-09T06:28:45.804818+0000","last_fullsized":"2024-05-09T06:28:45.804818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803942+0000","last_change":"2024-05-09T06:28:44.118588+0000","last_active":"2024-05-09T06:28:45.803942+0000","last_peered":"2024-05-09T06:28:45.803942+0000","last_clean":"2024-05-09T06:28:45.803942+0000","last_became_active":"2024-05-09T06:28:44.118265+0000","last_became_peered":"2024-05-09T06:28:44.118265+0000","last_unstale":"2024-05-09T06:28:45.803942+0000","last_undegraded":"2024-05-09T06:28:45.803942+0000","last_fullsized":"2024-05-09T06:28:45.803942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:31.663288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804603+0000","last_change":"2024-05-09T06:28:44.109578+0000","last_active":"2024-05-09T06:28:45.804603+0000","last_peered":"2024-05-09T06:28:45.804603+0000","last_clean":"2024-05-09T06:28:45.804603+0000","last_became_active":"2024-05-09T06:28:44.109257+0000","last_became_peered":"2024-05-09T06:28:44.109257+0000","last_unstale":"2024-05-09T06:28:45.804603+0000","last_undegraded":"2024-05-09T06:28:45.804603+0000","last_fullsized":"2024-05-09T06:28:45.804603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:46.300031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804344+0000","last_change":"2024-05-09T06:28:44.119996+0000","last_active":"2024-05-09T06:28:45.804344+0000","last_peered":"2024-05-09T06:28:45.804344+0000","last_clean":"2024-05-09T06:28:45.804344+0000","last_became_active":"2024-05-09T06:28:44.119614+0000","last_became_peered":"2024-05-09T06:28:44.119614+0000","last_unstale":"2024-05-09T06:28:45.804344+0000","last_undegraded":"2024-05-09T06:28:45.804344+0000","last_fullsized":"2024-05-09T06:28:45.804344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804283+0000","last_change":"2024-05-09T06:28:44.115094+0000","last_active":"2024-05-09T06:28:45.804283+0000","last_peered":"2024-05-09T06:28:45.804283+0000","last_clean":"2024-05-09T06:28:45.804283+0000","last_became_active":"2024-05-09T06:28:44.113946+0000","last_became_peered":"2024-05-09T06:28:44.113946+0000","last_unstale":"2024-05-09T06:28:45.804283+0000","last_undegraded":"2024-05-09T06:28:45.804283+0000","last_fullsized":"2024-05-09T06:28:45.804283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804492+0000","last_change":"2024-05-09T06:28:44.121433+0000","last_active":"2024-05-09T06:28:45.804492+0000","last_peered":"2024-05-09T06:28:45.804492+0000","last_clean":"2024-05-09T06:28:45.804492+0000","last_became_active":"2024-05-09T06:28:44.121156+0000","last_became_peered":"2024-05-09T06:28:44.121156+0000","last_unstale":"2024-05-09T06:28:45.804492+0000","last_undegraded":"2024-05-09T06:28:45.804492+0000","last_fullsized":"2024-05-09T06:28:45.804492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:44.500703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803969+0000","last_change":"2024-05-09T06:28:44.106606+0000","last_active":"2024-05-09T06:28:45.803969+0000","last_peered":"2024-05-09T06:28:45.803969+0000","last_clean":"2024-05-09T06:28:45.803969+0000","last_became_active":"2024-05-09T06:28:44.106308+0000","last_became_peered":"2024-05-09T06:28:44.106308+0000","last_unstale":"2024-05-09T06:28:45.803969+0000","last_undegraded":"2024-05-09T06:28:45.803969+0000","last_fullsized":"2024-05-09T06:28:45.803969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804480+0000","last_change":"2024-05-09T06:28:44.114434+0000","last_active":"2024-05-09T06:28:45.804480+0000","last_peered":"2024-05-09T06:28:45.804480+0000","last_clean":"2024-05-09T06:28:45.804480+0000","last_became_active":"2024-05-09T06:28:44.114160+0000","last_became_peered":"2024-05-09T06:28:44.114160+0000","last_unstale":"2024-05-09T06:28:45.804480+0000","last_undegraded":"2024-05-09T06:28:45.804480+0000","last_fullsized":"2024-05-09T06:28:45.804480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:27:36.391827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804911+0000","last_change":"2024-05-09T06:28:44.109304+0000","last_active":"2024-05-09T06:28:45.804911+0000","last_peered":"2024-05-09T06:28:45.804911+0000","last_clean":"2024-05-09T06:28:45.804911+0000","last_became_active":"2024-05-09T06:28:44.109040+0000","last_became_peered":"2024-05-09T06:28:44.109040+0000","last_unstale":"2024-05-09T06:28:45.804911+0000","last_undegraded":"2024-05-09T06:28:45.804911+0000","last_fullsized":"2024-05-09T06:28:45.804911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:33.781532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803886+0000","last_change":"2024-05-09T06:28:44.113660+0000","last_active":"2024-05-09T06:28:45.803886+0000","last_peered":"2024-05-09T06:28:45.803886+0000","last_clean":"2024-05-09T06:28:45.803886+0000","last_became_active":"2024-05-09T06:28:44.113363+0000","last_became_peered":"2024-05-09T06:28:44.113363+0000","last_unstale":"2024-05-09T06:28:45.803886+0000","last_undegraded":"2024-05-09T06:28:45.803886+0000","last_fullsized":"2024-05-09T06:28:45.803886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:21.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803980+0000","last_change":"2024-05-09T06:28:44.110093+0000","last_active":"2024-05-09T06:28:45.803980+0000","last_peered":"2024-05-09T06:28:45.803980+0000","last_clean":"2024-05-09T06:28:45.803980+0000","last_became_active":"2024-05-09T06:28:44.109808+0000","last_became_peered":"2024-05-09T06:28:44.109808+0000","last_unstale":"2024-05-09T06:28:45.803980+0000","last_undegraded":"2024-05-09T06:28:45.803980+0000","last_fullsized":"2024-05-09T06:28:45.803980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804640+0000","last_change":"2024-05-09T06:28:44.120866+0000","last_active":"2024-05-09T06:28:45.804640+0000","last_peered":"2024-05-09T06:28:45.804640+0000","last_clean":"2024-05-09T06:28:45.804640+0000","last_became_active":"2024-05-09T06:28:44.120306+0000","last_became_peered":"2024-05-09T06:28:44.120306+0000","last_unstale":"2024-05-09T06:28:45.804640+0000","last_undegraded":"2024-05-09T06:28:45.804640+0000","last_fullsized":"2024-05-09T06:28:45.804640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:26.240783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092876+0000","last_change":"2024-05-09T06:28:44.116340+0000","last_active":"2024-05-09T06:28:45.092876+0000","last_peered":"2024-05-09T06:28:45.092876+0000","last_clean":"2024-05-09T06:28:45.092876+0000","last_became_active":"2024-05-09T06:28:44.116064+0000","last_became_peered":"2024-05-09T06:28:44.116064+0000","last_unstale":"2024-05-09T06:28:45.092876+0000","last_undegraded":"2024-05-09T06:28:45.092876+0000","last_fullsized":"2024-05-09T06:28:45.092876+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088514+0000","last_change":"2024-05-09T06:28:44.102368+0000","last_active":"2024-05-09T06:28:45.088514+0000","last_peered":"2024-05-09T06:28:45.088514+0000","last_clean":"2024-05-09T06:28:45.088514+0000","last_became_active":"2024-05-09T06:28:44.100272+0000","last_became_peered":"2024-05-09T06:28:44.100272+0000","last_unstale":"2024-05-09T06:28:45.088514+0000","last_undegraded":"2024-05-09T06:28:45.088514+0000","last_fullsized":"2024-05-09T06:28:45.088514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:10.558347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092717+0000","last_change":"2024-05-09T06:28:44.114339+0000","last_active":"2024-05-09T06:28:45.092717+0000","last_peered":"2024-05-09T06:28:45.092717+0000","last_clean":"2024-05-09T06:28:45.092717+0000","last_became_active":"2024-05-09T06:28:44.114062+0000","last_became_peered":"2024-05-09T06:28:44.114062+0000","last_unstale":"2024-05-09T06:28:45.092717+0000","last_undegraded":"2024-05-09T06:28:45.092717+0000","last_fullsized":"2024-05-09T06:28:45.092717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:38.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804400+0000","last_change":"2024-05-09T06:28:44.108477+0000","last_active":"2024-05-09T06:28:45.804400+0000","last_peered":"2024-05-09T06:28:45.804400+0000","last_clean":"2024-05-09T06:28:45.804400+0000","last_became_active":"2024-05-09T06:28:44.108024+0000","last_became_peered":"2024-05-09T06:28:44.108024+0000","last_unstale":"2024-05-09T06:28:45.804400+0000","last_undegraded":"2024-05-09T06:28:45.804400+0000","last_fullsized":"2024-05-09T06:28:45.804400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:46.300031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805358+0000","last_change":"2024-05-09T06:28:44.102172+0000","last_active":"2024-05-09T06:28:45.805358+0000","last_peered":"2024-05-09T06:28:45.805358+0000","last_clean":"2024-05-09T06:28:45.805358+0000","last_became_active":"2024-05-09T06:28:44.101809+0000","last_became_peered":"2024-05-09T06:28:44.101809+0000","last_unstale":"2024-05-09T06:28:45.805358+0000","last_undegraded":"2024-05-09T06:28:45.805358+0000","last_fullsized":"2024-05-09T06:28:45.805358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:38.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804452+0000","last_change":"2024-05-09T06:28:44.113027+0000","last_active":"2024-05-09T06:28:45.804452+0000","last_peered":"2024-05-09T06:28:45.804452+0000","last_clean":"2024-05-09T06:28:45.804452+0000","last_became_active":"2024-05-09T06:28:44.112758+0000","last_became_peered":"2024-05-09T06:28:44.112758+0000","last_unstale":"2024-05-09T06:28:45.804452+0000","last_undegraded":"2024-05-09T06:28:45.804452+0000","last_fullsized":"2024-05-09T06:28:45.804452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:08.470818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088146+0000","last_change":"2024-05-09T06:28:44.112780+0000","last_active":"2024-05-09T06:28:45.088146+0000","last_peered":"2024-05-09T06:28:45.088146+0000","last_clean":"2024-05-09T06:28:45.088146+0000","last_became_active":"2024-05-09T06:28:44.112469+0000","last_became_peered":"2024-05-09T06:28:44.112469+0000","last_unstale":"2024-05-09T06:28:45.088146+0000","last_undegraded":"2024-05-09T06:28:45.088146+0000","last_fullsized":"2024-05-09T06:28:45.088146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:26.315583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092961+0000","last_change":"2024-05-09T06:28:44.103846+0000","last_active":"2024-05-09T06:28:45.092961+0000","last_peered":"2024-05-09T06:28:45.092961+0000","last_clean":"2024-05-09T06:28:45.092961+0000","last_became_active":"2024-05-09T06:28:44.103434+0000","last_became_peered":"2024-05-09T06:28:44.103434+0000","last_unstale":"2024-05-09T06:28:45.092961+0000","last_undegraded":"2024-05-09T06:28:45.092961+0000","last_fullsized":"2024-05-09T06:28:45.092961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804862+0000","last_change":"2024-05-09T06:28:44.116152+0000","last_active":"2024-05-09T06:28:45.804862+0000","last_peered":"2024-05-09T06:28:45.804862+0000","last_clean":"2024-05-09T06:28:45.804862+0000","last_became_active":"2024-05-09T06:28:44.115817+0000","last_became_peered":"2024-05-09T06:28:44.115817+0000","last_unstale":"2024-05-09T06:28:45.804862+0000","last_undegraded":"2024-05-09T06:28:45.804862+0000","last_fullsized":"2024-05-09T06:28:45.804862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092854+0000","last_change":"2024-05-09T06:28:44.104275+0000","last_active":"2024-05-09T06:28:45.092854+0000","last_peered":"2024-05-09T06:28:45.092854+0000","last_clean":"2024-05-09T06:28:45.092854+0000","last_became_active":"2024-05-09T06:28:44.103316+0000","last_became_peered":"2024-05-09T06:28:44.103316+0000","last_unstale":"2024-05-09T06:28:45.092854+0000","last_undegraded":"2024-05-09T06:28:45.092854+0000","last_fullsized":"2024-05-09T06:28:45.092854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:10.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804184+0000","last_change":"2024-05-09T06:28:44.113497+0000","last_active":"2024-05-09T06:28:45.804184+0000","last_peered":"2024-05-09T06:28:45.804184+0000","last_clean":"2024-05-09T06:28:45.804184+0000","last_became_active":"2024-05-09T06:28:44.113122+0000","last_became_peered":"2024-05-09T06:28:44.113122+0000","last_unstale":"2024-05-09T06:28:45.804184+0000","last_undegraded":"2024-05-09T06:28:45.804184+0000","last_fullsized":"2024-05-09T06:28:45.804184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:42.412580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092736+0000","last_change":"2024-05-09T06:28:44.104112+0000","last_active":"2024-05-09T06:28:45.092736+0000","last_peered":"2024-05-09T06:28:45.092736+0000","last_clean":"2024-05-09T06:28:45.092736+0000","last_became_active":"2024-05-09T06:28:44.103833+0000","last_became_peered":"2024-05-09T06:28:44.103833+0000","last_unstale":"2024-05-09T06:28:45.092736+0000","last_undegraded":"2024-05-09T06:28:45.092736+0000","last_fullsized":"2024-05-09T06:28:45.092736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:15.692861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092645+0000","last_change":"2024-05-09T06:28:44.102495+0000","last_active":"2024-05-09T06:28:45.092645+0000","last_peered":"2024-05-09T06:28:45.092645+0000","last_clean":"2024-05-09T06:28:45.092645+0000","last_became_active":"2024-05-09T06:28:44.102156+0000","last_became_peered":"2024-05-09T06:28:44.102156+0000","last_unstale":"2024-05-09T06:28:45.092645+0000","last_undegraded":"2024-05-09T06:28:45.092645+0000","last_fullsized":"2024-05-09T06:28:45.092645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:16.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805285+0000","last_change":"2024-05-09T06:28:44.113731+0000","last_active":"2024-05-09T06:28:45.805285+0000","last_peered":"2024-05-09T06:28:45.805285+0000","last_clean":"2024-05-09T06:28:45.805285+0000","last_became_active":"2024-05-09T06:28:44.113431+0000","last_became_peered":"2024-05-09T06:28:44.113431+0000","last_unstale":"2024-05-09T06:28:45.805285+0000","last_undegraded":"2024-05-09T06:28:45.805285+0000","last_fullsized":"2024-05-09T06:28:45.805285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:31.663288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804238+0000","last_change":"2024-05-09T06:28:44.101739+0000","last_active":"2024-05-09T06:28:45.804238+0000","last_peered":"2024-05-09T06:28:45.804238+0000","last_clean":"2024-05-09T06:28:45.804238+0000","last_became_active":"2024-05-09T06:28:44.101444+0000","last_became_peered":"2024-05-09T06:28:44.101444+0000","last_unstale":"2024-05-09T06:28:45.804238+0000","last_undegraded":"2024-05-09T06:28:45.804238+0000","last_fullsized":"2024-05-09T06:28:45.804238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804014+0000","last_change":"2024-05-09T06:28:44.115581+0000","last_active":"2024-05-09T06:28:45.804014+0000","last_peered":"2024-05-09T06:28:45.804014+0000","last_clean":"2024-05-09T06:28:45.804014+0000","last_became_active":"2024-05-09T06:28:44.115283+0000","last_became_peered":"2024-05-09T06:28:44.115283+0000","last_unstale":"2024-05-09T06:28:45.804014+0000","last_undegraded":"2024-05-09T06:28:45.804014+0000","last_fullsized":"2024-05-09T06:28:45.804014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092873+0000","last_change":"2024-05-09T06:28:43.097628+0000","last_active":"2024-05-09T06:28:45.092873+0000","last_peered":"2024-05-09T06:28:45.092873+0000","last_clean":"2024-05-09T06:28:45.092873+0000","last_became_active":"2024-05-09T06:28:43.097348+0000","last_became_peered":"2024-05-09T06:28:43.097348+0000","last_unstale":"2024-05-09T06:28:45.092873+0000","last_undegraded":"2024-05-09T06:28:45.092873+0000","last_fullsized":"2024-05-09T06:28:45.092873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805166+0000","last_change":"2024-05-09T06:28:44.121090+0000","last_active":"2024-05-09T06:28:45.805166+0000","last_peered":"2024-05-09T06:28:45.805166+0000","last_clean":"2024-05-09T06:28:45.805166+0000","last_became_active":"2024-05-09T06:28:44.120245+0000","last_became_peered":"2024-05-09T06:28:44.120245+0000","last_unstale":"2024-05-09T06:28:45.805166+0000","last_undegraded":"2024-05-09T06:28:45.805166+0000","last_fullsized":"2024-05-09T06:28:45.805166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:08.470818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803965+0000","last_change":"2024-05-09T06:28:43.104623+0000","last_active":"2024-05-09T06:28:45.803965+0000","last_peered":"2024-05-09T06:28:45.803965+0000","last_clean":"2024-05-09T06:28:45.803965+0000","last_became_active":"2024-05-09T06:28:43.104335+0000","last_became_peered":"2024-05-09T06:28:43.104335+0000","last_unstale":"2024-05-09T06:28:45.803965+0000","last_undegraded":"2024-05-09T06:28:45.803965+0000","last_fullsized":"2024-05-09T06:28:45.803965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805734+0000","last_change":"2024-05-09T06:28:44.112338+0000","last_active":"2024-05-09T06:28:45.805734+0000","last_peered":"2024-05-09T06:28:45.805734+0000","last_clean":"2024-05-09T06:28:45.805734+0000","last_became_active":"2024-05-09T06:28:44.112063+0000","last_became_peered":"2024-05-09T06:28:44.112063+0000","last_unstale":"2024-05-09T06:28:45.805734+0000","last_undegraded":"2024-05-09T06:28:45.805734+0000","last_fullsized":"2024-05-09T06:28:45.805734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805196+0000","last_change":"2024-05-09T06:28:43.105205+0000","last_active":"2024-05-09T06:28:45.805196+0000","last_peered":"2024-05-09T06:28:45.805196+0000","last_clean":"2024-05-09T06:28:45.805196+0000","last_became_active":"2024-05-09T06:28:43.104872+0000","last_became_peered":"2024-05-09T06:28:43.104872+0000","last_unstale":"2024-05-09T06:28:45.805196+0000","last_undegraded":"2024-05-09T06:28:45.805196+0000","last_fullsized":"2024-05-09T06:28:45.805196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:15.837324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804664+0000","last_change":"2024-05-09T06:28:44.105815+0000","last_active":"2024-05-09T06:28:45.804664+0000","last_peered":"2024-05-09T06:28:45.804664+0000","last_clean":"2024-05-09T06:28:45.804664+0000","last_became_active":"2024-05-09T06:28:44.105538+0000","last_became_peered":"2024-05-09T06:28:44.105538+0000","last_unstale":"2024-05-09T06:28:45.804664+0000","last_undegraded":"2024-05-09T06:28:45.804664+0000","last_fullsized":"2024-05-09T06:28:45.804664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:57.011376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805211+0000","last_change":"2024-05-09T06:28:43.103990+0000","last_active":"2024-05-09T06:28:45.805211+0000","last_peered":"2024-05-09T06:28:45.805211+0000","last_clean":"2024-05-09T06:28:45.805211+0000","last_became_active":"2024-05-09T06:28:43.103672+0000","last_became_peered":"2024-05-09T06:28:43.103672+0000","last_unstale":"2024-05-09T06:28:45.805211+0000","last_undegraded":"2024-05-09T06:28:45.805211+0000","last_fullsized":"2024-05-09T06:28:45.805211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:12.906616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803700+0000","last_change":"2024-05-09T06:28:44.105046+0000","last_active":"2024-05-09T06:28:45.803700+0000","last_peered":"2024-05-09T06:28:45.803700+0000","last_clean":"2024-05-09T06:28:45.803700+0000","last_became_active":"2024-05-09T06:28:44.104736+0000","last_became_peered":"2024-05-09T06:28:44.104736+0000","last_unstale":"2024-05-09T06:28:45.803700+0000","last_undegraded":"2024-05-09T06:28:45.803700+0000","last_fullsized":"2024-05-09T06:28:45.803700+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804591+0000","last_change":"2024-05-09T06:28:43.107734+0000","last_active":"2024-05-09T06:28:45.804591+0000","last_peered":"2024-05-09T06:28:45.804591+0000","last_clean":"2024-05-09T06:28:45.804591+0000","last_became_active":"2024-05-09T06:28:43.107428+0000","last_became_peered":"2024-05-09T06:28:43.107428+0000","last_unstale":"2024-05-09T06:28:45.804591+0000","last_undegraded":"2024-05-09T06:28:45.804591+0000","last_fullsized":"2024-05-09T06:28:45.804591+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:15.837324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804926+0000","last_change":"2024-05-09T06:28:44.114590+0000","last_active":"2024-05-09T06:28:45.804926+0000","last_peered":"2024-05-09T06:28:45.804926+0000","last_clean":"2024-05-09T06:28:45.804926+0000","last_became_active":"2024-05-09T06:28:44.111994+0000","last_became_peered":"2024-05-09T06:28:44.111994+0000","last_unstale":"2024-05-09T06:28:45.804926+0000","last_undegraded":"2024-05-09T06:28:45.804926+0000","last_fullsized":"2024-05-09T06:28:45.804926+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:03.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804135+0000","last_change":"2024-05-09T06:28:43.107479+0000","last_active":"2024-05-09T06:28:45.804135+0000","last_peered":"2024-05-09T06:28:45.804135+0000","last_clean":"2024-05-09T06:28:45.804135+0000","last_became_active":"2024-05-09T06:28:43.107175+0000","last_became_peered":"2024-05-09T06:28:43.107175+0000","last_unstale":"2024-05-09T06:28:45.804135+0000","last_undegraded":"2024-05-09T06:28:45.804135+0000","last_fullsized":"2024-05-09T06:28:45.804135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:31.021123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805652+0000","last_change":"2024-05-09T06:28:44.117973+0000","last_active":"2024-05-09T06:28:45.805652+0000","last_peered":"2024-05-09T06:28:45.805652+0000","last_clean":"2024-05-09T06:28:45.805652+0000","last_became_active":"2024-05-09T06:28:44.117650+0000","last_became_peered":"2024-05-09T06:28:44.117650+0000","last_unstale":"2024-05-09T06:28:45.805652+0000","last_undegraded":"2024-05-09T06:28:45.805652+0000","last_fullsized":"2024-05-09T06:28:45.805652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:10.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804607+0000","last_change":"2024-05-09T06:28:43.106257+0000","last_active":"2024-05-09T06:28:45.804607+0000","last_peered":"2024-05-09T06:28:45.804607+0000","last_clean":"2024-05-09T06:28:45.804607+0000","last_became_active":"2024-05-09T06:28:43.106027+0000","last_became_peered":"2024-05-09T06:28:43.106027+0000","last_unstale":"2024-05-09T06:28:45.804607+0000","last_undegraded":"2024-05-09T06:28:45.804607+0000","last_fullsized":"2024-05-09T06:28:45.804607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:01.004710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803981+0000","last_change":"2024-05-09T06:28:44.111157+0000","last_active":"2024-05-09T06:28:45.803981+0000","last_peered":"2024-05-09T06:28:45.803981+0000","last_clean":"2024-05-09T06:28:45.803981+0000","last_became_active":"2024-05-09T06:28:44.110878+0000","last_became_peered":"2024-05-09T06:28:44.110878+0000","last_unstale":"2024-05-09T06:28:45.803981+0000","last_undegraded":"2024-05-09T06:28:45.803981+0000","last_fullsized":"2024-05-09T06:28:45.803981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804714+0000","last_change":"2024-05-09T06:28:43.099299+0000","last_active":"2024-05-09T06:28:45.804714+0000","last_peered":"2024-05-09T06:28:45.804714+0000","last_clean":"2024-05-09T06:28:45.804714+0000","last_became_active":"2024-05-09T06:28:43.099001+0000","last_became_peered":"2024-05-09T06:28:43.099001+0000","last_unstale":"2024-05-09T06:28:45.804714+0000","last_undegraded":"2024-05-09T06:28:45.804714+0000","last_fullsized":"2024-05-09T06:28:45.804714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:21.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088489+0000","last_change":"2024-05-09T06:28:44.095549+0000","last_active":"2024-05-09T06:28:45.088489+0000","last_peered":"2024-05-09T06:28:45.088489+0000","last_clean":"2024-05-09T06:28:45.088489+0000","last_became_active":"2024-05-09T06:28:44.095259+0000","last_became_peered":"2024-05-09T06:28:44.095259+0000","last_unstale":"2024-05-09T06:28:45.088489+0000","last_undegraded":"2024-05-09T06:28:45.088489+0000","last_fullsized":"2024-05-09T06:28:45.088489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:06.903246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804887+0000","last_change":"2024-05-09T06:28:43.101397+0000","last_active":"2024-05-09T06:28:45.804887+0000","last_peered":"2024-05-09T06:28:45.804887+0000","last_clean":"2024-05-09T06:28:45.804887+0000","last_became_active":"2024-05-09T06:28:43.101067+0000","last_became_peered":"2024-05-09T06:28:43.101067+0000","last_unstale":"2024-05-09T06:28:45.804887+0000","last_undegraded":"2024-05-09T06:28:45.804887+0000","last_fullsized":"2024-05-09T06:28:45.804887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:56.133420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804428+0000","last_change":"2024-05-09T06:28:44.108877+0000","last_active":"2024-05-09T06:28:45.804428+0000","last_peered":"2024-05-09T06:28:45.804428+0000","last_clean":"2024-05-09T06:28:45.804428+0000","last_became_active":"2024-05-09T06:28:44.108580+0000","last_became_peered":"2024-05-09T06:28:44.108580+0000","last_unstale":"2024-05-09T06:28:45.804428+0000","last_undegraded":"2024-05-09T06:28:45.804428+0000","last_fullsized":"2024-05-09T06:28:45.804428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804402+0000","last_change":"2024-05-09T06:28:43.110342+0000","last_active":"2024-05-09T06:28:45.804402+0000","last_peered":"2024-05-09T06:28:45.804402+0000","last_clean":"2024-05-09T06:28:45.804402+0000","last_became_active":"2024-05-09T06:28:43.110095+0000","last_became_peered":"2024-05-09T06:28:43.110095+0000","last_unstale":"2024-05-09T06:28:45.804402+0000","last_undegraded":"2024-05-09T06:28:45.804402+0000","last_fullsized":"2024-05-09T06:28:45.804402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:32.776178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803975+0000","last_change":"2024-05-09T06:28:44.111068+0000","last_active":"2024-05-09T06:28:45.803975+0000","last_peered":"2024-05-09T06:28:45.803975+0000","last_clean":"2024-05-09T06:28:45.803975+0000","last_became_active":"2024-05-09T06:28:44.110780+0000","last_became_peered":"2024-05-09T06:28:44.110780+0000","last_unstale":"2024-05-09T06:28:45.803975+0000","last_undegraded":"2024-05-09T06:28:45.803975+0000","last_fullsized":"2024-05-09T06:28:45.803975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804836+0000","last_change":"2024-05-09T06:28:43.106566+0000","last_active":"2024-05-09T06:28:45.804836+0000","last_peered":"2024-05-09T06:28:45.804836+0000","last_clean":"2024-05-09T06:28:45.804836+0000","last_became_active":"2024-05-09T06:28:43.105323+0000","last_became_peered":"2024-05-09T06:28:43.105323+0000","last_unstale":"2024-05-09T06:28:45.804836+0000","last_undegraded":"2024-05-09T06:28:45.804836+0000","last_fullsized":"2024-05-09T06:28:45.804836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:09.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805517+0000","last_change":"2024-05-09T06:28:44.103166+0000","last_active":"2024-05-09T06:28:45.805517+0000","last_peered":"2024-05-09T06:28:45.805517+0000","last_clean":"2024-05-09T06:28:45.805517+0000","last_became_active":"2024-05-09T06:28:44.102891+0000","last_became_peered":"2024-05-09T06:28:44.102891+0000","last_unstale":"2024-05-09T06:28:45.805517+0000","last_undegraded":"2024-05-09T06:28:45.805517+0000","last_fullsized":"2024-05-09T06:28:45.805517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:15.692861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804356+0000","last_change":"2024-05-09T06:28:43.105704+0000","last_active":"2024-05-09T06:28:45.804356+0000","last_peered":"2024-05-09T06:28:45.804356+0000","last_clean":"2024-05-09T06:28:45.804356+0000","last_became_active":"2024-05-09T06:28:43.105384+0000","last_became_peered":"2024-05-09T06:28:43.105384+0000","last_unstale":"2024-05-09T06:28:45.804356+0000","last_undegraded":"2024-05-09T06:28:45.804356+0000","last_fullsized":"2024-05-09T06:28:45.804356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:51.065483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803607+0000","last_change":"2024-05-09T06:28:44.115185+0000","last_active":"2024-05-09T06:28:45.803607+0000","last_peered":"2024-05-09T06:28:45.803607+0000","last_clean":"2024-05-09T06:28:45.803607+0000","last_became_active":"2024-05-09T06:28:44.114856+0000","last_became_peered":"2024-05-09T06:28:44.114856+0000","last_unstale":"2024-05-09T06:28:45.803607+0000","last_undegraded":"2024-05-09T06:28:45.803607+0000","last_fullsized":"2024-05-09T06:28:45.803607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804527+0000","last_change":"2024-05-09T06:28:43.097679+0000","last_active":"2024-05-09T06:28:45.804527+0000","last_peered":"2024-05-09T06:28:45.804527+0000","last_clean":"2024-05-09T06:28:45.804527+0000","last_became_active":"2024-05-09T06:28:43.097373+0000","last_became_peered":"2024-05-09T06:28:43.097373+0000","last_unstale":"2024-05-09T06:28:45.804527+0000","last_undegraded":"2024-05-09T06:28:45.804527+0000","last_fullsized":"2024-05-09T06:28:45.804527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:40.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804000+0000","last_change":"2024-05-09T06:28:44.101203+0000","last_active":"2024-05-09T06:28:45.804000+0000","last_peered":"2024-05-09T06:28:45.804000+0000","last_clean":"2024-05-09T06:28:45.804000+0000","last_became_active":"2024-05-09T06:28:44.100650+0000","last_became_peered":"2024-05-09T06:28:44.100650+0000","last_unstale":"2024-05-09T06:28:45.804000+0000","last_undegraded":"2024-05-09T06:28:45.804000+0000","last_fullsized":"2024-05-09T06:28:45.804000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804933+0000","last_change":"2024-05-09T06:28:43.107557+0000","last_active":"2024-05-09T06:28:45.804933+0000","last_peered":"2024-05-09T06:28:45.804933+0000","last_clean":"2024-05-09T06:28:45.804933+0000","last_became_active":"2024-05-09T06:28:43.107217+0000","last_became_peered":"2024-05-09T06:28:43.107217+0000","last_unstale":"2024-05-09T06:28:45.804933+0000","last_undegraded":"2024-05-09T06:28:45.804933+0000","last_fullsized":"2024-05-09T06:28:45.804933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:32.776178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803980+0000","last_change":"2024-05-09T06:28:44.114434+0000","last_active":"2024-05-09T06:28:45.803980+0000","last_peered":"2024-05-09T06:28:45.803980+0000","last_clean":"2024-05-09T06:28:45.803980+0000","last_became_active":"2024-05-09T06:28:44.114161+0000","last_became_peered":"2024-05-09T06:28:44.114161+0000","last_unstale":"2024-05-09T06:28:45.803980+0000","last_undegraded":"2024-05-09T06:28:45.803980+0000","last_fullsized":"2024-05-09T06:28:45.803980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092496+0000","last_change":"2024-05-09T06:28:43.096518+0000","last_active":"2024-05-09T06:28:45.092496+0000","last_peered":"2024-05-09T06:28:45.092496+0000","last_clean":"2024-05-09T06:28:45.092496+0000","last_became_active":"2024-05-09T06:28:43.096235+0000","last_became_peered":"2024-05-09T06:28:43.096235+0000","last_unstale":"2024-05-09T06:28:45.092496+0000","last_undegraded":"2024-05-09T06:28:45.092496+0000","last_fullsized":"2024-05-09T06:28:45.092496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.161063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803851+0000","last_change":"2024-05-09T06:28:44.102832+0000","last_active":"2024-05-09T06:28:45.803851+0000","last_peered":"2024-05-09T06:28:45.803851+0000","last_clean":"2024-05-09T06:28:45.803851+0000","last_became_active":"2024-05-09T06:28:44.102511+0000","last_became_peered":"2024-05-09T06:28:44.102511+0000","last_unstale":"2024-05-09T06:28:45.803851+0000","last_undegraded":"2024-05-09T06:28:45.803851+0000","last_fullsized":"2024-05-09T06:28:45.803851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:42.412580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804109+0000","last_change":"2024-05-09T06:28:43.108183+0000","last_active":"2024-05-09T06:28:45.804109+0000","last_peered":"2024-05-09T06:28:45.804109+0000","last_clean":"2024-05-09T06:28:45.804109+0000","last_became_active":"2024-05-09T06:28:43.107072+0000","last_became_peered":"2024-05-09T06:28:43.107072+0000","last_unstale":"2024-05-09T06:28:45.804109+0000","last_undegraded":"2024-05-09T06:28:45.804109+0000","last_fullsized":"2024-05-09T06:28:45.804109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:01.004710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804243+0000","last_change":"2024-05-09T06:28:44.108644+0000","last_active":"2024-05-09T06:28:45.804243+0000","last_peered":"2024-05-09T06:28:45.804243+0000","last_clean":"2024-05-09T06:28:45.804243+0000","last_became_active":"2024-05-09T06:28:44.108353+0000","last_became_peered":"2024-05-09T06:28:44.108353+0000","last_unstale":"2024-05-09T06:28:45.804243+0000","last_undegraded":"2024-05-09T06:28:45.804243+0000","last_fullsized":"2024-05-09T06:28:45.804243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:44.500703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805059+0000","last_change":"2024-05-09T06:28:43.108305+0000","last_active":"2024-05-09T06:28:45.805059+0000","last_peered":"2024-05-09T06:28:45.805059+0000","last_clean":"2024-05-09T06:28:45.805059+0000","last_became_active":"2024-05-09T06:28:43.107930+0000","last_became_peered":"2024-05-09T06:28:43.107930+0000","last_unstale":"2024-05-09T06:28:45.805059+0000","last_undegraded":"2024-05-09T06:28:45.805059+0000","last_fullsized":"2024-05-09T06:28:45.805059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:14.687508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804748+0000","last_change":"2024-05-09T06:28:44.117972+0000","last_active":"2024-05-09T06:28:45.804748+0000","last_peered":"2024-05-09T06:28:45.804748+0000","last_clean":"2024-05-09T06:28:45.804748+0000","last_became_active":"2024-05-09T06:28:44.116446+0000","last_became_peered":"2024-05-09T06:28:44.116446+0000","last_unstale":"2024-05-09T06:28:45.804748+0000","last_undegraded":"2024-05-09T06:28:45.804748+0000","last_fullsized":"2024-05-09T06:28:45.804748+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805776+0000","last_change":"2024-05-09T06:28:43.106417+0000","last_active":"2024-05-09T06:28:45.805776+0000","last_peered":"2024-05-09T06:28:45.805776+0000","last_clean":"2024-05-09T06:28:45.805776+0000","last_became_active":"2024-05-09T06:28:43.106004+0000","last_became_peered":"2024-05-09T06:28:43.106004+0000","last_unstale":"2024-05-09T06:28:45.805776+0000","last_undegraded":"2024-05-09T06:28:45.805776+0000","last_fullsized":"2024-05-09T06:28:45.805776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805694+0000","last_change":"2024-05-09T06:28:44.104985+0000","last_active":"2024-05-09T06:28:45.805694+0000","last_peered":"2024-05-09T06:28:45.805694+0000","last_clean":"2024-05-09T06:28:45.805694+0000","last_became_active":"2024-05-09T06:28:44.104710+0000","last_became_peered":"2024-05-09T06:28:44.104710+0000","last_unstale":"2024-05-09T06:28:45.805694+0000","last_undegraded":"2024-05-09T06:28:45.805694+0000","last_fullsized":"2024-05-09T06:28:45.805694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088639+0000","last_change":"2024-05-09T06:28:43.105231+0000","last_active":"2024-05-09T06:28:45.088639+0000","last_peered":"2024-05-09T06:28:45.088639+0000","last_clean":"2024-05-09T06:28:45.088639+0000","last_became_active":"2024-05-09T06:28:43.104944+0000","last_became_peered":"2024-05-09T06:28:43.104944+0000","last_unstale":"2024-05-09T06:28:45.088639+0000","last_undegraded":"2024-05-09T06:28:45.088639+0000","last_fullsized":"2024-05-09T06:28:45.088639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:51.001350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088813+0000","last_change":"2024-05-09T06:28:44.103963+0000","last_active":"2024-05-09T06:28:45.088813+0000","last_peered":"2024-05-09T06:28:45.088813+0000","last_clean":"2024-05-09T06:28:45.088813+0000","last_became_active":"2024-05-09T06:28:44.103648+0000","last_became_peered":"2024-05-09T06:28:44.103648+0000","last_unstale":"2024-05-09T06:28:45.088813+0000","last_undegraded":"2024-05-09T06:28:45.088813+0000","last_fullsized":"2024-05-09T06:28:45.088813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:32.026477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804112+0000","last_change":"2024-05-09T06:28:43.104127+0000","last_active":"2024-05-09T06:28:45.804112+0000","last_peered":"2024-05-09T06:28:45.804112+0000","last_clean":"2024-05-09T06:28:45.804112+0000","last_became_active":"2024-05-09T06:28:43.103790+0000","last_became_peered":"2024-05-09T06:28:43.103790+0000","last_unstale":"2024-05-09T06:28:45.804112+0000","last_undegraded":"2024-05-09T06:28:45.804112+0000","last_fullsized":"2024-05-09T06:28:45.804112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:59.651383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804580+0000","last_change":"2024-05-09T06:28:44.115343+0000","last_active":"2024-05-09T06:28:45.804580+0000","last_peered":"2024-05-09T06:28:45.804580+0000","last_clean":"2024-05-09T06:28:45.804580+0000","last_became_active":"2024-05-09T06:28:44.115062+0000","last_became_peered":"2024-05-09T06:28:44.115062+0000","last_unstale":"2024-05-09T06:28:45.804580+0000","last_undegraded":"2024-05-09T06:28:45.804580+0000","last_fullsized":"2024-05-09T06:28:45.804580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:35.562590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088292+0000","last_change":"2024-05-09T06:28:43.101721+0000","last_active":"2024-05-09T06:28:45.088292+0000","last_peered":"2024-05-09T06:28:45.088292+0000","last_clean":"2024-05-09T06:28:45.088292+0000","last_became_active":"2024-05-09T06:28:43.101362+0000","last_became_peered":"2024-05-09T06:28:43.101362+0000","last_unstale":"2024-05-09T06:28:45.088292+0000","last_undegraded":"2024-05-09T06:28:45.088292+0000","last_fullsized":"2024-05-09T06:28:45.088292+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:21.371244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803756+0000","last_change":"2024-05-09T06:28:44.110189+0000","last_active":"2024-05-09T06:28:45.803756+0000","last_peered":"2024-05-09T06:28:45.803756+0000","last_clean":"2024-05-09T06:28:45.803756+0000","last_became_active":"2024-05-09T06:28:44.109910+0000","last_became_peered":"2024-05-09T06:28:44.109910+0000","last_unstale":"2024-05-09T06:28:45.803756+0000","last_undegraded":"2024-05-09T06:28:45.803756+0000","last_fullsized":"2024-05-09T06:28:45.803756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:52.070837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088225+0000","last_change":"2024-05-09T06:28:43.100163+0000","last_active":"2024-05-09T06:28:45.088225+0000","last_peered":"2024-05-09T06:28:45.088225+0000","last_clean":"2024-05-09T06:28:45.088225+0000","last_became_active":"2024-05-09T06:28:43.099826+0000","last_became_peered":"2024-05-09T06:28:43.099826+0000","last_unstale":"2024-05-09T06:28:45.088225+0000","last_undegraded":"2024-05-09T06:28:45.088225+0000","last_fullsized":"2024-05-09T06:28:45.088225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:31:46.432443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803957+0000","last_change":"2024-05-09T06:28:44.109742+0000","last_active":"2024-05-09T06:28:45.803957+0000","last_peered":"2024-05-09T06:28:45.803957+0000","last_clean":"2024-05-09T06:28:45.803957+0000","last_became_active":"2024-05-09T06:28:44.109442+0000","last_became_peered":"2024-05-09T06:28:44.109442+0000","last_unstale":"2024-05-09T06:28:45.803957+0000","last_undegraded":"2024-05-09T06:28:45.803957+0000","last_fullsized":"2024-05-09T06:28:45.803957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:05.558151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805352+0000","last_change":"2024-05-09T06:28:43.105547+0000","last_active":"2024-05-09T06:28:45.805352+0000","last_peered":"2024-05-09T06:28:45.805352+0000","last_clean":"2024-05-09T06:28:45.805352+0000","last_became_active":"2024-05-09T06:28:43.104806+0000","last_became_peered":"2024-05-09T06:28:43.104806+0000","last_unstale":"2024-05-09T06:28:45.805352+0000","last_undegraded":"2024-05-09T06:28:45.805352+0000","last_fullsized":"2024-05-09T06:28:45.805352+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:49.647442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804029+0000","last_change":"2024-05-09T06:28:44.110733+0000","last_active":"2024-05-09T06:28:45.804029+0000","last_peered":"2024-05-09T06:28:45.804029+0000","last_clean":"2024-05-09T06:28:45.804029+0000","last_became_active":"2024-05-09T06:28:44.110340+0000","last_became_peered":"2024-05-09T06:28:44.110340+0000","last_unstale":"2024-05-09T06:28:45.804029+0000","last_undegraded":"2024-05-09T06:28:45.804029+0000","last_fullsized":"2024-05-09T06:28:45.804029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:33.448866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088218+0000","last_change":"2024-05-09T06:28:43.099690+0000","last_active":"2024-05-09T06:28:45.088218+0000","last_peered":"2024-05-09T06:28:45.088218+0000","last_clean":"2024-05-09T06:28:45.088218+0000","last_became_active":"2024-05-09T06:28:43.099173+0000","last_became_peered":"2024-05-09T06:28:43.099173+0000","last_unstale":"2024-05-09T06:28:45.088218+0000","last_undegraded":"2024-05-09T06:28:45.088218+0000","last_fullsized":"2024-05-09T06:28:45.088218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:19.174636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803997+0000","last_change":"2024-05-09T06:28:44.107406+0000","last_active":"2024-05-09T06:28:45.803997+0000","last_peered":"2024-05-09T06:28:45.803997+0000","last_clean":"2024-05-09T06:28:45.803997+0000","last_became_active":"2024-05-09T06:28:44.107105+0000","last_became_peered":"2024-05-09T06:28:44.107105+0000","last_unstale":"2024-05-09T06:28:45.803997+0000","last_undegraded":"2024-05-09T06:28:45.803997+0000","last_fullsized":"2024-05-09T06:28:45.803997+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:41.031233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805183+0000","last_change":"2024-05-09T06:28:43.109099+0000","last_active":"2024-05-09T06:28:45.805183+0000","last_peered":"2024-05-09T06:28:45.805183+0000","last_clean":"2024-05-09T06:28:45.805183+0000","last_became_active":"2024-05-09T06:28:43.108456+0000","last_became_peered":"2024-05-09T06:28:43.108456+0000","last_unstale":"2024-05-09T06:28:45.805183+0000","last_undegraded":"2024-05-09T06:28:45.805183+0000","last_fullsized":"2024-05-09T06:28:45.805183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:37.004402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804609+0000","last_change":"2024-05-09T06:28:44.115453+0000","last_active":"2024-05-09T06:28:45.804609+0000","last_peered":"2024-05-09T06:28:45.804609+0000","last_clean":"2024-05-09T06:28:45.804609+0000","last_became_active":"2024-05-09T06:28:44.115172+0000","last_became_peered":"2024-05-09T06:28:44.115172+0000","last_unstale":"2024-05-09T06:28:45.804609+0000","last_undegraded":"2024-05-09T06:28:45.804609+0000","last_fullsized":"2024-05-09T06:28:45.804609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:20.788897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088728+0000","last_change":"2024-05-09T06:28:43.105213+0000","last_active":"2024-05-09T06:28:45.088728+0000","last_peered":"2024-05-09T06:28:45.088728+0000","last_clean":"2024-05-09T06:28:45.088728+0000","last_became_active":"2024-05-09T06:28:43.104918+0000","last_became_peered":"2024-05-09T06:28:43.104918+0000","last_unstale":"2024-05-09T06:28:45.088728+0000","last_undegraded":"2024-05-09T06:28:45.088728+0000","last_fullsized":"2024-05-09T06:28:45.088728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:27.675907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804311+0000","last_change":"2024-05-09T06:28:44.113910+0000","last_active":"2024-05-09T06:28:45.804311+0000","last_peered":"2024-05-09T06:28:45.804311+0000","last_clean":"2024-05-09T06:28:45.804311+0000","last_became_active":"2024-05-09T06:28:44.113608+0000","last_became_peered":"2024-05-09T06:28:44.113608+0000","last_unstale":"2024-05-09T06:28:45.804311+0000","last_undegraded":"2024-05-09T06:28:45.804311+0000","last_fullsized":"2024-05-09T06:28:45.804311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:03.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804405+0000","last_change":"2024-05-09T06:28:43.102864+0000","last_active":"2024-05-09T06:28:45.804405+0000","last_peered":"2024-05-09T06:28:45.804405+0000","last_clean":"2024-05-09T06:28:45.804405+0000","last_became_active":"2024-05-09T06:28:43.102193+0000","last_became_peered":"2024-05-09T06:28:43.102193+0000","last_unstale":"2024-05-09T06:28:45.804405+0000","last_undegraded":"2024-05-09T06:28:45.804405+0000","last_fullsized":"2024-05-09T06:28:45.804405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803784+0000","last_change":"2024-05-09T06:28:44.111576+0000","last_active":"2024-05-09T06:28:45.803784+0000","last_peered":"2024-05-09T06:28:45.803784+0000","last_clean":"2024-05-09T06:28:45.803784+0000","last_became_active":"2024-05-09T06:28:44.111251+0000","last_became_peered":"2024-05-09T06:28:44.111251+0000","last_unstale":"2024-05-09T06:28:45.803784+0000","last_undegraded":"2024-05-09T06:28:45.803784+0000","last_fullsized":"2024-05-09T06:28:45.803784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:41.031233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804673+0000","last_change":"2024-05-09T06:28:43.097963+0000","last_active":"2024-05-09T06:28:45.804673+0000","last_peered":"2024-05-09T06:28:45.804673+0000","last_clean":"2024-05-09T06:28:45.804673+0000","last_became_active":"2024-05-09T06:28:43.097670+0000","last_became_peered":"2024-05-09T06:28:43.097670+0000","last_unstale":"2024-05-09T06:28:45.804673+0000","last_undegraded":"2024-05-09T06:28:45.804673+0000","last_fullsized":"2024-05-09T06:28:45.804673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:31.021123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804586+0000","last_change":"2024-05-09T06:28:44.107029+0000","last_active":"2024-05-09T06:28:45.804586+0000","last_peered":"2024-05-09T06:28:45.804586+0000","last_clean":"2024-05-09T06:28:45.804586+0000","last_became_active":"2024-05-09T06:28:44.106699+0000","last_became_peered":"2024-05-09T06:28:44.106699+0000","last_unstale":"2024-05-09T06:28:45.804586+0000","last_undegraded":"2024-05-09T06:28:45.804586+0000","last_fullsized":"2024-05-09T06:28:45.804586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092564+0000","last_change":"2024-05-09T06:28:43.094133+0000","last_active":"2024-05-09T06:28:45.092564+0000","last_peered":"2024-05-09T06:28:45.092564+0000","last_clean":"2024-05-09T06:28:45.092564+0000","last_became_active":"2024-05-09T06:28:43.093802+0000","last_became_peered":"2024-05-09T06:28:43.093802+0000","last_unstale":"2024-05-09T06:28:45.092564+0000","last_undegraded":"2024-05-09T06:28:45.092564+0000","last_fullsized":"2024-05-09T06:28:45.092564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805087+0000","last_change":"2024-05-09T06:28:44.104777+0000","last_active":"2024-05-09T06:28:45.805087+0000","last_peered":"2024-05-09T06:28:45.805087+0000","last_clean":"2024-05-09T06:28:45.805087+0000","last_became_active":"2024-05-09T06:28:44.104500+0000","last_became_peered":"2024-05-09T06:28:44.104500+0000","last_unstale":"2024-05-09T06:28:45.805087+0000","last_undegraded":"2024-05-09T06:28:45.805087+0000","last_fullsized":"2024-05-09T06:28:45.805087+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:16.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804745+0000","last_change":"2024-05-09T06:28:43.096894+0000","last_active":"2024-05-09T06:28:45.804745+0000","last_peered":"2024-05-09T06:28:45.804745+0000","last_clean":"2024-05-09T06:28:45.804745+0000","last_became_active":"2024-05-09T06:28:43.096592+0000","last_became_peered":"2024-05-09T06:28:43.096592+0000","last_unstale":"2024-05-09T06:28:45.804745+0000","last_undegraded":"2024-05-09T06:28:45.804745+0000","last_fullsized":"2024-05-09T06:28:45.804745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:02.760035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805088+0000","last_change":"2024-05-09T06:28:44.108785+0000","last_active":"2024-05-09T06:28:45.805088+0000","last_peered":"2024-05-09T06:28:45.805088+0000","last_clean":"2024-05-09T06:28:45.805088+0000","last_became_active":"2024-05-09T06:28:44.108468+0000","last_became_peered":"2024-05-09T06:28:44.108468+0000","last_unstale":"2024-05-09T06:28:45.805088+0000","last_undegraded":"2024-05-09T06:28:45.805088+0000","last_fullsized":"2024-05-09T06:28:45.805088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:21.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804706+0000","last_change":"2024-05-09T06:28:43.101342+0000","last_active":"2024-05-09T06:28:45.804706+0000","last_peered":"2024-05-09T06:28:45.804706+0000","last_clean":"2024-05-09T06:28:45.804706+0000","last_became_active":"2024-05-09T06:28:43.101027+0000","last_became_peered":"2024-05-09T06:28:43.101027+0000","last_unstale":"2024-05-09T06:28:45.804706+0000","last_undegraded":"2024-05-09T06:28:45.804706+0000","last_fullsized":"2024-05-09T06:28:45.804706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:44.009601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804956+0000","last_change":"2024-05-09T06:28:44.116083+0000","last_active":"2024-05-09T06:28:45.804956+0000","last_peered":"2024-05-09T06:28:45.804956+0000","last_clean":"2024-05-09T06:28:45.804956+0000","last_became_active":"2024-05-09T06:28:44.115762+0000","last_became_peered":"2024-05-09T06:28:44.115762+0000","last_unstale":"2024-05-09T06:28:45.804956+0000","last_undegraded":"2024-05-09T06:28:45.804956+0000","last_fullsized":"2024-05-09T06:28:45.804956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:33.448866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804194+0000","last_change":"2024-05-09T06:28:43.107403+0000","last_active":"2024-05-09T06:28:45.804194+0000","last_peered":"2024-05-09T06:28:45.804194+0000","last_clean":"2024-05-09T06:28:45.804194+0000","last_became_active":"2024-05-09T06:28:43.107064+0000","last_became_peered":"2024-05-09T06:28:43.107064+0000","last_unstale":"2024-05-09T06:28:45.804194+0000","last_undegraded":"2024-05-09T06:28:45.804194+0000","last_fullsized":"2024-05-09T06:28:45.804194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:35.212436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804686+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:28:45.804686+0000","last_peered":"2024-05-09T06:28:45.804686+0000","last_clean":"2024-05-09T06:28:45.804686+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:28:45.804686+0000","last_undegraded":"2024-05-09T06:28:45.804686+0000","last_fullsized":"2024-05-09T06:28:45.804686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":782086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":191682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":190106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":780438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:28:52.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:28:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.701+0000 7f128d6ba640 1 Processor -- start 2024-05-09T06:28:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.701+0000 7f128d6ba640 1 -- start start 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f128d6ba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f12880d4940 0x7f12880b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f128d6ba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12880b33c0 con 0x7f1288161950 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f128d6ba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12880b3660 con 0x7f12880d2b10 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f128d6ba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f12880d4dd0 con 0x7f12880d4940 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f1286ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1288161950 0x7f12880d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58600/0 (socket says 172.21.15.1:58600) 2024-05-09T06:28:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.702+0000 7f1286ffd640 1 -- 172.21.15.1:0/2380538784 learned_addr learned my addr 172.21.15.1:0/2380538784 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:52.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f12867fc640 1 --2- 172.21.15.1:0/2380538784 >> v2:172.21.15.6:3300/0 conn(0x7f12880d4940 0x7f12880b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140445215 0 0) 0x7f12880b33c0 con 0x7f1288161950 2024-05-09T06:28:52.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1270003520 con 0x7f1288161950 2024-05-09T06:28:52.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1718134319 0 0) 0x7f1270003520 con 0x7f1288161950 2024-05-09T06:28:52.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 >> v2:172.21.15.6:3300/0 conn(0x7f12880d4940 msgr2=0x7f12880b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 --2- 172.21.15.1:0/2380538784 >> v2:172.21.15.6:3300/0 conn(0x7f12880d4940 0x7f12880b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 >> v1:172.21.15.6:6789/0 conn(0x7f12880d2b10 legacy=0x7f12880b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12880b3940 con 0x7f1288161950 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f12780019f0 con 0x7f1288161950 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 >> v1:172.21.15.1:6789/0 conn(0x7f1288161950 legacy=0x7f12880d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f1270003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12700043b0 0x7f1270004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1270005080 0x7f127000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12880d4dd0 con 0x7f12700039b0 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f127000bfe0 con 0x7f12700043b0 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f127000c1e0 con 0x7f1270005080 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f1286ffd640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f1270003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f12867fc640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12700043b0 0x7f1270004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.703+0000 7f12877fe640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1270005080 0x7f127000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 -- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1270005080 msgr2=0x7f127000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:52.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1270005080 0x7f127000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 -- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12700043b0 msgr2=0x7f1270004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12700043b0 0x7f1270004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 -- 172.21.15.1:0/2380538784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1270003520 con 0x7f12700039b0 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f12877fe640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1270005080 0x7f127000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f12877fe640 1 -- 172.21.15.1:0/2380538784 reap_dead start 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1286ffd640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f1270003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f127800a880 tx=0x7f1270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5606dd39e131ff9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127800a4a0 con 0x7f12700039b0 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127802f070 con 0x7f12700039b0 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.704+0000 7f1285ffb640 1 -- 172.21.15.1:0/2380538784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802bd40 con 0x7f12700039b0 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 -- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 msgr2=0x7f1270003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f1270003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f127800a880 tx=0x7f1270005830 comp rx=0 tx=0).stop 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 -- 172.21.15.1:0/2380538784 shutdown_connections 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 --2- 172.21.15.1:0/2380538784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f1270003d90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 -- 172.21.15.1:0/2380538784 >> 172.21.15.1:0/2380538784 conn(0x7f12880d3e20 msgr2=0x7f12880d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:52.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 -- 172.21.15.1:0/2380538784 shutdown_connections 2024-05-09T06:28:52.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.705+0000 7f128d6ba640 1 -- 172.21.15.1:0/2380538784 wait complete. 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 Processor -- start 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 -- start start 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12881541a0 0x7f12880a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12880b8340 0x7f12880cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12880d1100 con 0x7f12700039b0 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f12880d0f80 con 0x7f12880b8340 2024-05-09T06:28:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f128d6ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f12880d1c00 con 0x7f12881541a0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.707+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41930/0 (socket says 172.21.15.1:41930) 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 -- 172.21.15.1:0/4092895349 learned_addr learned my addr 172.21.15.1:0/4092895349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f12877fe640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12880b8340 0x7f12880cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f12867fc640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12881541a0 0x7f12880a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 -- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12881541a0 msgr2=0x7f12880a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12881541a0 0x7f12880a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 -- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12880b8340 msgr2=0x7f12880cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12880b8340 0x7f12880cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 -- 172.21.15.1:0/4092895349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12880b72a0 con 0x7f12700039b0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f1286ffd640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f127800a9a0 tx=0x7f127802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802b8a0 con 0x7f12700039b0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127802ba40 con 0x7f12700039b0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12880b7580 con 0x7f12700039b0 2024-05-09T06:28:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802be10 con 0x7f12700039b0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.708+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12880b7b40 con 0x7f12700039b0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.709+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12880752b0 con 0x7f12700039b0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.710+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1278034470 con 0x7f12700039b0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.710+0000 7f126f7fe640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 0x7f127406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.710+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f12780ab370 con 0x7f12700039b0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.710+0000 7f12867fc640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 0x7f127406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.711+0000 7f12867fc640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 0x7f127406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1288161330 tx=0x7f127c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.715+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1278079530 con 0x7f12700039b0 2024-05-09T06:28:52.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.898+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f128806d6e0 con 0x7f127406be60 2024-05-09T06:28:52.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.904+0000 7f126f7fe640 1 -- 172.21.15.1:0/4092895349 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f128806d6e0 con 0x7f127406be60 2024-05-09T06:28:52.889 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:52.890 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:28:52.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 msgr2=0x7f127406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 0x7f127406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1288161330 tx=0x7f127c009190 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 msgr2=0x7f12880b6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f127800a9a0 tx=0x7f127802d070 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 shutdown_connections 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f127406be60 0x7f127406e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12880b8340 0x7f12880cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12881541a0 0x7f12880a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 --2- 172.21.15.1:0/4092895349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12700039b0 0x7f12880b6ba0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 >> 172.21.15.1:0/4092895349 conn(0x7f12880d3e20 msgr2=0x7f12880b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 shutdown_connections 2024-05-09T06:28:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:52.907+0000 7f128d6ba640 1 -- 172.21.15.1:0/4092895349 wait complete. 2024-05-09T06:28:52.900 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-05-09T06:28:51.086522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237464,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737256,"statfs":{"total":773094113280,"available":772850950144,"internally_reserved":0,"allocated":4759552,"data_stored":2705844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001673"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804839+0000","last_change":"2024-05-09T06:28:44.104786+0000","last_active":"2024-05-09T06:28:45.804839+0000","last_peered":"2024-05-09T06:28:45.804839+0000","last_clean":"2024-05-09T06:28:45.804839+0000","last_became_active":"2024-05-09T06:28:44.104499+0000","last_became_peered":"2024-05-09T06:28:44.104499+0000","last_unstale":"2024-05-09T06:28:45.804839+0000","last_undegraded":"2024-05-09T06:28:45.804839+0000","last_fullsized":"2024-05-09T06:28:45.804839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.380616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804104+0000","last_change":"2024-05-09T06:28:44.102227+0000","last_active":"2024-05-09T06:28:45.804104+0000","last_peered":"2024-05-09T06:28:45.804104+0000","last_clean":"2024-05-09T06:28:45.804104+0000","last_became_active":"2024-05-09T06:28:44.101728+0000","last_became_peered":"2024-05-09T06:28:44.101728+0000","last_unstale":"2024-05-09T06:28:45.804104+0000","last_undegraded":"2024-05-09T06:28:45.804104+0000","last_fullsized":"2024-05-09T06:28:45.804104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:27:36.391827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804289+0000","last_change":"2024-05-09T06:28:44.122863+0000","last_active":"2024-05-09T06:28:45.804289+0000","last_peered":"2024-05-09T06:28:45.804289+0000","last_clean":"2024-05-09T06:28:45.804289+0000","last_became_active":"2024-05-09T06:28:44.122578+0000","last_became_peered":"2024-05-09T06:28:44.122578+0000","last_unstale":"2024-05-09T06:28:45.804289+0000","last_undegraded":"2024-05-09T06:28:45.804289+0000","last_fullsized":"2024-05-09T06:28:45.804289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088431+0000","last_change":"2024-05-09T06:28:44.094799+0000","last_active":"2024-05-09T06:28:45.088431+0000","last_peered":"2024-05-09T06:28:45.088431+0000","last_clean":"2024-05-09T06:28:45.088431+0000","last_became_active":"2024-05-09T06:28:44.094382+0000","last_became_peered":"2024-05-09T06:28:44.094382+0000","last_unstale":"2024-05-09T06:28:45.088431+0000","last_undegraded":"2024-05-09T06:28:45.088431+0000","last_fullsized":"2024-05-09T06:28:45.088431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:45.166417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088180+0000","last_change":"2024-05-09T06:28:44.095845+0000","last_active":"2024-05-09T06:28:45.088180+0000","last_peered":"2024-05-09T06:28:45.088180+0000","last_clean":"2024-05-09T06:28:45.088180+0000","last_became_active":"2024-05-09T06:28:44.095531+0000","last_became_peered":"2024-05-09T06:28:44.095531+0000","last_unstale":"2024-05-09T06:28:45.088180+0000","last_undegraded":"2024-05-09T06:28:45.088180+0000","last_fullsized":"2024-05-09T06:28:45.088180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:13.911970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804818+0000","last_change":"2024-05-09T06:28:44.108110+0000","last_active":"2024-05-09T06:28:45.804818+0000","last_peered":"2024-05-09T06:28:45.804818+0000","last_clean":"2024-05-09T06:28:45.804818+0000","last_became_active":"2024-05-09T06:28:44.107839+0000","last_became_peered":"2024-05-09T06:28:44.107839+0000","last_unstale":"2024-05-09T06:28:45.804818+0000","last_undegraded":"2024-05-09T06:28:45.804818+0000","last_fullsized":"2024-05-09T06:28:45.804818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803942+0000","last_change":"2024-05-09T06:28:44.118588+0000","last_active":"2024-05-09T06:28:45.803942+0000","last_peered":"2024-05-09T06:28:45.803942+0000","last_clean":"2024-05-09T06:28:45.803942+0000","last_became_active":"2024-05-09T06:28:44.118265+0000","last_became_peered":"2024-05-09T06:28:44.118265+0000","last_unstale":"2024-05-09T06:28:45.803942+0000","last_undegraded":"2024-05-09T06:28:45.803942+0000","last_fullsized":"2024-05-09T06:28:45.803942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:31.663288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804603+0000","last_change":"2024-05-09T06:28:44.109578+0000","last_active":"2024-05-09T06:28:45.804603+0000","last_peered":"2024-05-09T06:28:45.804603+0000","last_clean":"2024-05-09T06:28:45.804603+0000","last_became_active":"2024-05-09T06:28:44.109257+0000","last_became_peered":"2024-05-09T06:28:44.109257+0000","last_unstale":"2024-05-09T06:28:45.804603+0000","last_undegraded":"2024-05-09T06:28:45.804603+0000","last_fullsized":"2024-05-09T06:28:45.804603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:46.300031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804344+0000","last_change":"2024-05-09T06:28:44.119996+0000","last_active":"2024-05-09T06:28:45.804344+0000","last_peered":"2024-05-09T06:28:45.804344+0000","last_clean":"2024-05-09T06:28:45.804344+0000","last_became_active":"2024-05-09T06:28:44.119614+0000","last_became_peered":"2024-05-09T06:28:44.119614+0000","last_unstale":"2024-05-09T06:28:45.804344+0000","last_undegraded":"2024-05-09T06:28:45.804344+0000","last_fullsized":"2024-05-09T06:28:45.804344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804283+0000","last_change":"2024-05-09T06:28:44.115094+0000","last_active":"2024-05-09T06:28:45.804283+0000","last_peered":"2024-05-09T06:28:45.804283+0000","last_clean":"2024-05-09T06:28:45.804283+0000","last_became_active":"2024-05-09T06:28:44.113946+0000","last_became_peered":"2024-05-09T06:28:44.113946+0000","last_unstale":"2024-05-09T06:28:45.804283+0000","last_undegraded":"2024-05-09T06:28:45.804283+0000","last_fullsized":"2024-05-09T06:28:45.804283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804492+0000","last_change":"2024-05-09T06:28:44.121433+0000","last_active":"2024-05-09T06:28:45.804492+0000","last_peered":"2024-05-09T06:28:45.804492+0000","last_clean":"2024-05-09T06:28:45.804492+0000","last_became_active":"2024-05-09T06:28:44.121156+0000","last_became_peered":"2024-05-09T06:28:44.121156+0000","last_unstale":"2024-05-09T06:28:45.804492+0000","last_undegraded":"2024-05-09T06:28:45.804492+0000","last_fullsized":"2024-05-09T06:28:45.804492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:44.500703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803969+0000","last_change":"2024-05-09T06:28:44.106606+0000","last_active":"2024-05-09T06:28:45.803969+0000","last_peered":"2024-05-09T06:28:45.803969+0000","last_clean":"2024-05-09T06:28:45.803969+0000","last_became_active":"2024-05-09T06:28:44.106308+0000","last_became_peered":"2024-05-09T06:28:44.106308+0000","last_unstale":"2024-05-09T06:28:45.803969+0000","last_undegraded":"2024-05-09T06:28:45.803969+0000","last_fullsized":"2024-05-09T06:28:45.803969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804480+0000","last_change":"2024-05-09T06:28:44.114434+0000","last_active":"2024-05-09T06:28:45.804480+0000","last_peered":"2024-05-09T06:28:45.804480+0000","last_clean":"2024-05-09T06:28:45.804480+0000","last_became_active":"2024-05-09T06:28:44.114160+0000","last_became_peered":"2024-05-09T06:28:44.114160+0000","last_unstale":"2024-05-09T06:28:45.804480+0000","last_undegraded":"2024-05-09T06:28:45.804480+0000","last_fullsized":"2024-05-09T06:28:45.804480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:27:36.391827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804911+0000","last_change":"2024-05-09T06:28:44.109304+0000","last_active":"2024-05-09T06:28:45.804911+0000","last_peered":"2024-05-09T06:28:45.804911+0000","last_clean":"2024-05-09T06:28:45.804911+0000","last_became_active":"2024-05-09T06:28:44.109040+0000","last_became_peered":"2024-05-09T06:28:44.109040+0000","last_unstale":"2024-05-09T06:28:45.804911+0000","last_undegraded":"2024-05-09T06:28:45.804911+0000","last_fullsized":"2024-05-09T06:28:45.804911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:33.781532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803886+0000","last_change":"2024-05-09T06:28:44.113660+0000","last_active":"2024-05-09T06:28:45.803886+0000","last_peered":"2024-05-09T06:28:45.803886+0000","last_clean":"2024-05-09T06:28:45.803886+0000","last_became_active":"2024-05-09T06:28:44.113363+0000","last_became_peered":"2024-05-09T06:28:44.113363+0000","last_unstale":"2024-05-09T06:28:45.803886+0000","last_undegraded":"2024-05-09T06:28:45.803886+0000","last_fullsized":"2024-05-09T06:28:45.803886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:21.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803980+0000","last_change":"2024-05-09T06:28:44.110093+0000","last_active":"2024-05-09T06:28:45.803980+0000","last_peered":"2024-05-09T06:28:45.803980+0000","last_clean":"2024-05-09T06:28:45.803980+0000","last_became_active":"2024-05-09T06:28:44.109808+0000","last_became_peered":"2024-05-09T06:28:44.109808+0000","last_unstale":"2024-05-09T06:28:45.803980+0000","last_undegraded":"2024-05-09T06:28:45.803980+0000","last_fullsized":"2024-05-09T06:28:45.803980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804640+0000","last_change":"2024-05-09T06:28:44.120866+0000","last_active":"2024-05-09T06:28:45.804640+0000","last_peered":"2024-05-09T06:28:45.804640+0000","last_clean":"2024-05-09T06:28:45.804640+0000","last_became_active":"2024-05-09T06:28:44.120306+0000","last_became_peered":"2024-05-09T06:28:44.120306+0000","last_unstale":"2024-05-09T06:28:45.804640+0000","last_undegraded":"2024-05-09T06:28:45.804640+0000","last_fullsized":"2024-05-09T06:28:45.804640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:26.240783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092876+0000","last_change":"2024-05-09T06:28:44.116340+0000","last_active":"2024-05-09T06:28:45.092876+0000","last_peered":"2024-05-09T06:28:45.092876+0000","last_clean":"2024-05-09T06:28:45.092876+0000","last_became_active":"2024-05-09T06:28:44.116064+0000","last_became_peered":"2024-05-09T06:28:44.116064+0000","last_unstale":"2024-05-09T06:28:45.092876+0000","last_undegraded":"2024-05-09T06:28:45.092876+0000","last_fullsized":"2024-05-09T06:28:45.092876+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088514+0000","last_change":"2024-05-09T06:28:44.102368+0000","last_active":"2024-05-09T06:28:45.088514+0000","last_peered":"2024-05-09T06:28:45.088514+0000","last_clean":"2024-05-09T06:28:45.088514+0000","last_became_active":"2024-05-09T06:28:44.100272+0000","last_became_peered":"2024-05-09T06:28:44.100272+0000","last_unstale":"2024-05-09T06:28:45.088514+0000","last_undegraded":"2024-05-09T06:28:45.088514+0000","last_fullsized":"2024-05-09T06:28:45.088514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:10.558347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092717+0000","last_change":"2024-05-09T06:28:44.114339+0000","last_active":"2024-05-09T06:28:45.092717+0000","last_peered":"2024-05-09T06:28:45.092717+0000","last_clean":"2024-05-09T06:28:45.092717+0000","last_became_active":"2024-05-09T06:28:44.114062+0000","last_became_peered":"2024-05-09T06:28:44.114062+0000","last_unstale":"2024-05-09T06:28:45.092717+0000","last_undegraded":"2024-05-09T06:28:45.092717+0000","last_fullsized":"2024-05-09T06:28:45.092717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:38.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804400+0000","last_change":"2024-05-09T06:28:44.108477+0000","last_active":"2024-05-09T06:28:45.804400+0000","last_peered":"2024-05-09T06:28:45.804400+0000","last_clean":"2024-05-09T06:28:45.804400+0000","last_became_active":"2024-05-09T06:28:44.108024+0000","last_became_peered":"2024-05-09T06:28:44.108024+0000","last_unstale":"2024-05-09T06:28:45.804400+0000","last_undegraded":"2024-05-09T06:28:45.804400+0000","last_fullsized":"2024-05-09T06:28:45.804400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:46.300031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805358+0000","last_change":"2024-05-09T06:28:44.102172+0000","last_active":"2024-05-09T06:28:45.805358+0000","last_peered":"2024-05-09T06:28:45.805358+0000","last_clean":"2024-05-09T06:28:45.805358+0000","last_became_active":"2024-05-09T06:28:44.101809+0000","last_became_peered":"2024-05-09T06:28:44.101809+0000","last_unstale":"2024-05-09T06:28:45.805358+0000","last_undegraded":"2024-05-09T06:28:45.805358+0000","last_fullsized":"2024-05-09T06:28:45.805358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:38.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804452+0000","last_change":"2024-05-09T06:28:44.113027+0000","last_active":"2024-05-09T06:28:45.804452+0000","last_peered":"2024-05-09T06:28:45.804452+0000","last_clean":"2024-05-09T06:28:45.804452+0000","last_became_active":"2024-05-09T06:28:44.112758+0000","last_became_peered":"2024-05-09T06:28:44.112758+0000","last_unstale":"2024-05-09T06:28:45.804452+0000","last_undegraded":"2024-05-09T06:28:45.804452+0000","last_fullsized":"2024-05-09T06:28:45.804452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:08.470818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088146+0000","last_change":"2024-05-09T06:28:44.112780+0000","last_active":"2024-05-09T06:28:45.088146+0000","last_peered":"2024-05-09T06:28:45.088146+0000","last_clean":"2024-05-09T06:28:45.088146+0000","last_became_active":"2024-05-09T06:28:44.112469+0000","last_became_peered":"2024-05-09T06:28:44.112469+0000","last_unstale":"2024-05-09T06:28:45.088146+0000","last_undegraded":"2024-05-09T06:28:45.088146+0000","last_fullsized":"2024-05-09T06:28:45.088146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:26.315583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092961+0000","last_change":"2024-05-09T06:28:44.103846+0000","last_active":"2024-05-09T06:28:45.092961+0000","last_peered":"2024-05-09T06:28:45.092961+0000","last_clean":"2024-05-09T06:28:45.092961+0000","last_became_active":"2024-05-09T06:28:44.103434+0000","last_became_peered":"2024-05-09T06:28:44.103434+0000","last_unstale":"2024-05-09T06:28:45.092961+0000","last_undegraded":"2024-05-09T06:28:45.092961+0000","last_fullsized":"2024-05-09T06:28:45.092961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804862+0000","last_change":"2024-05-09T06:28:44.116152+0000","last_active":"2024-05-09T06:28:45.804862+0000","last_peered":"2024-05-09T06:28:45.804862+0000","last_clean":"2024-05-09T06:28:45.804862+0000","last_became_active":"2024-05-09T06:28:44.115817+0000","last_became_peered":"2024-05-09T06:28:44.115817+0000","last_unstale":"2024-05-09T06:28:45.804862+0000","last_undegraded":"2024-05-09T06:28:45.804862+0000","last_fullsized":"2024-05-09T06:28:45.804862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092854+0000","last_change":"2024-05-09T06:28:44.104275+0000","last_active":"2024-05-09T06:28:45.092854+0000","last_peered":"2024-05-09T06:28:45.092854+0000","last_clean":"2024-05-09T06:28:45.092854+0000","last_became_active":"2024-05-09T06:28:44.103316+0000","last_became_peered":"2024-05-09T06:28:44.103316+0000","last_unstale":"2024-05-09T06:28:45.092854+0000","last_undegraded":"2024-05-09T06:28:45.092854+0000","last_fullsized":"2024-05-09T06:28:45.092854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:10.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804184+0000","last_change":"2024-05-09T06:28:44.113497+0000","last_active":"2024-05-09T06:28:45.804184+0000","last_peered":"2024-05-09T06:28:45.804184+0000","last_clean":"2024-05-09T06:28:45.804184+0000","last_became_active":"2024-05-09T06:28:44.113122+0000","last_became_peered":"2024-05-09T06:28:44.113122+0000","last_unstale":"2024-05-09T06:28:45.804184+0000","last_undegraded":"2024-05-09T06:28:45.804184+0000","last_fullsized":"2024-05-09T06:28:45.804184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:42.412580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092736+0000","last_change":"2024-05-09T06:28:44.104112+0000","last_active":"2024-05-09T06:28:45.092736+0000","last_peered":"2024-05-09T06:28:45.092736+0000","last_clean":"2024-05-09T06:28:45.092736+0000","last_became_active":"2024-05-09T06:28:44.103833+0000","last_became_peered":"2024-05-09T06:28:44.103833+0000","last_unstale":"2024-05-09T06:28:45.092736+0000","last_undegraded":"2024-05-09T06:28:45.092736+0000","last_fullsized":"2024-05-09T06:28:45.092736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:15.692861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092645+0000","last_change":"2024-05-09T06:28:44.102495+0000","last_active":"2024-05-09T06:28:45.092645+0000","last_peered":"2024-05-09T06:28:45.092645+0000","last_clean":"2024-05-09T06:28:45.092645+0000","last_became_active":"2024-05-09T06:28:44.102156+0000","last_became_peered":"2024-05-09T06:28:44.102156+0000","last_unstale":"2024-05-09T06:28:45.092645+0000","last_undegraded":"2024-05-09T06:28:45.092645+0000","last_fullsized":"2024-05-09T06:28:45.092645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:16.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805285+0000","last_change":"2024-05-09T06:28:44.113731+0000","last_active":"2024-05-09T06:28:45.805285+0000","last_peered":"2024-05-09T06:28:45.805285+0000","last_clean":"2024-05-09T06:28:45.805285+0000","last_became_active":"2024-05-09T06:28:44.113431+0000","last_became_peered":"2024-05-09T06:28:44.113431+0000","last_unstale":"2024-05-09T06:28:45.805285+0000","last_undegraded":"2024-05-09T06:28:45.805285+0000","last_fullsized":"2024-05-09T06:28:45.805285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:29:31.663288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804238+0000","last_change":"2024-05-09T06:28:44.101739+0000","last_active":"2024-05-09T06:28:45.804238+0000","last_peered":"2024-05-09T06:28:45.804238+0000","last_clean":"2024-05-09T06:28:45.804238+0000","last_became_active":"2024-05-09T06:28:44.101444+0000","last_became_peered":"2024-05-09T06:28:44.101444+0000","last_unstale":"2024-05-09T06:28:45.804238+0000","last_undegraded":"2024-05-09T06:28:45.804238+0000","last_fullsized":"2024-05-09T06:28:45.804238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804014+0000","last_change":"2024-05-09T06:28:44.115581+0000","last_active":"2024-05-09T06:28:45.804014+0000","last_peered":"2024-05-09T06:28:45.804014+0000","last_clean":"2024-05-09T06:28:45.804014+0000","last_became_active":"2024-05-09T06:28:44.115283+0000","last_became_peered":"2024-05-09T06:28:44.115283+0000","last_unstale":"2024-05-09T06:28:45.804014+0000","last_undegraded":"2024-05-09T06:28:45.804014+0000","last_fullsized":"2024-05-09T06:28:45.804014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:26.634180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092873+0000","last_change":"2024-05-09T06:28:43.097628+0000","last_active":"2024-05-09T06:28:45.092873+0000","last_peered":"2024-05-09T06:28:45.092873+0000","last_clean":"2024-05-09T06:28:45.092873+0000","last_became_active":"2024-05-09T06:28:43.097348+0000","last_became_peered":"2024-05-09T06:28:43.097348+0000","last_unstale":"2024-05-09T06:28:45.092873+0000","last_undegraded":"2024-05-09T06:28:45.092873+0000","last_fullsized":"2024-05-09T06:28:45.092873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805166+0000","last_change":"2024-05-09T06:28:44.121090+0000","last_active":"2024-05-09T06:28:45.805166+0000","last_peered":"2024-05-09T06:28:45.805166+0000","last_clean":"2024-05-09T06:28:45.805166+0000","last_became_active":"2024-05-09T06:28:44.120245+0000","last_became_peered":"2024-05-09T06:28:44.120245+0000","last_unstale":"2024-05-09T06:28:45.805166+0000","last_undegraded":"2024-05-09T06:28:45.805166+0000","last_fullsized":"2024-05-09T06:28:45.805166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:08.470818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803965+0000","last_change":"2024-05-09T06:28:43.104623+0000","last_active":"2024-05-09T06:28:45.803965+0000","last_peered":"2024-05-09T06:28:45.803965+0000","last_clean":"2024-05-09T06:28:45.803965+0000","last_became_active":"2024-05-09T06:28:43.104335+0000","last_became_peered":"2024-05-09T06:28:43.104335+0000","last_unstale":"2024-05-09T06:28:45.803965+0000","last_undegraded":"2024-05-09T06:28:45.803965+0000","last_fullsized":"2024-05-09T06:28:45.803965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:09.552993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805734+0000","last_change":"2024-05-09T06:28:44.112338+0000","last_active":"2024-05-09T06:28:45.805734+0000","last_peered":"2024-05-09T06:28:45.805734+0000","last_clean":"2024-05-09T06:28:45.805734+0000","last_became_active":"2024-05-09T06:28:44.112063+0000","last_became_peered":"2024-05-09T06:28:44.112063+0000","last_unstale":"2024-05-09T06:28:45.805734+0000","last_undegraded":"2024-05-09T06:28:45.805734+0000","last_fullsized":"2024-05-09T06:28:45.805734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805196+0000","last_change":"2024-05-09T06:28:43.105205+0000","last_active":"2024-05-09T06:28:45.805196+0000","last_peered":"2024-05-09T06:28:45.805196+0000","last_clean":"2024-05-09T06:28:45.805196+0000","last_became_active":"2024-05-09T06:28:43.104872+0000","last_became_peered":"2024-05-09T06:28:43.104872+0000","last_unstale":"2024-05-09T06:28:45.805196+0000","last_undegraded":"2024-05-09T06:28:45.805196+0000","last_fullsized":"2024-05-09T06:28:45.805196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:15.837324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804664+0000","last_change":"2024-05-09T06:28:44.105815+0000","last_active":"2024-05-09T06:28:45.804664+0000","last_peered":"2024-05-09T06:28:45.804664+0000","last_clean":"2024-05-09T06:28:45.804664+0000","last_became_active":"2024-05-09T06:28:44.105538+0000","last_became_peered":"2024-05-09T06:28:44.105538+0000","last_unstale":"2024-05-09T06:28:45.804664+0000","last_undegraded":"2024-05-09T06:28:45.804664+0000","last_fullsized":"2024-05-09T06:28:45.804664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:57.011376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805211+0000","last_change":"2024-05-09T06:28:43.103990+0000","last_active":"2024-05-09T06:28:45.805211+0000","last_peered":"2024-05-09T06:28:45.805211+0000","last_clean":"2024-05-09T06:28:45.805211+0000","last_became_active":"2024-05-09T06:28:43.103672+0000","last_became_peered":"2024-05-09T06:28:43.103672+0000","last_unstale":"2024-05-09T06:28:45.805211+0000","last_undegraded":"2024-05-09T06:28:45.805211+0000","last_fullsized":"2024-05-09T06:28:45.805211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:12.906616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803700+0000","last_change":"2024-05-09T06:28:44.105046+0000","last_active":"2024-05-09T06:28:45.803700+0000","last_peered":"2024-05-09T06:28:45.803700+0000","last_clean":"2024-05-09T06:28:45.803700+0000","last_became_active":"2024-05-09T06:28:44.104736+0000","last_became_peered":"2024-05-09T06:28:44.104736+0000","last_unstale":"2024-05-09T06:28:45.803700+0000","last_undegraded":"2024-05-09T06:28:45.803700+0000","last_fullsized":"2024-05-09T06:28:45.803700+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:45.014955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804591+0000","last_change":"2024-05-09T06:28:43.107734+0000","last_active":"2024-05-09T06:28:45.804591+0000","last_peered":"2024-05-09T06:28:45.804591+0000","last_clean":"2024-05-09T06:28:45.804591+0000","last_became_active":"2024-05-09T06:28:43.107428+0000","last_became_peered":"2024-05-09T06:28:43.107428+0000","last_unstale":"2024-05-09T06:28:45.804591+0000","last_undegraded":"2024-05-09T06:28:45.804591+0000","last_fullsized":"2024-05-09T06:28:45.804591+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:15.837324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804926+0000","last_change":"2024-05-09T06:28:44.114590+0000","last_active":"2024-05-09T06:28:45.804926+0000","last_peered":"2024-05-09T06:28:45.804926+0000","last_clean":"2024-05-09T06:28:45.804926+0000","last_became_active":"2024-05-09T06:28:44.111994+0000","last_became_peered":"2024-05-09T06:28:44.111994+0000","last_unstale":"2024-05-09T06:28:45.804926+0000","last_undegraded":"2024-05-09T06:28:45.804926+0000","last_fullsized":"2024-05-09T06:28:45.804926+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:03.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804135+0000","last_change":"2024-05-09T06:28:43.107479+0000","last_active":"2024-05-09T06:28:45.804135+0000","last_peered":"2024-05-09T06:28:45.804135+0000","last_clean":"2024-05-09T06:28:45.804135+0000","last_became_active":"2024-05-09T06:28:43.107175+0000","last_became_peered":"2024-05-09T06:28:43.107175+0000","last_unstale":"2024-05-09T06:28:45.804135+0000","last_undegraded":"2024-05-09T06:28:45.804135+0000","last_fullsized":"2024-05-09T06:28:45.804135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:31.021123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805652+0000","last_change":"2024-05-09T06:28:44.117973+0000","last_active":"2024-05-09T06:28:45.805652+0000","last_peered":"2024-05-09T06:28:45.805652+0000","last_clean":"2024-05-09T06:28:45.805652+0000","last_became_active":"2024-05-09T06:28:44.117650+0000","last_became_peered":"2024-05-09T06:28:44.117650+0000","last_unstale":"2024-05-09T06:28:45.805652+0000","last_undegraded":"2024-05-09T06:28:45.805652+0000","last_fullsized":"2024-05-09T06:28:45.805652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:10.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804607+0000","last_change":"2024-05-09T06:28:43.106257+0000","last_active":"2024-05-09T06:28:45.804607+0000","last_peered":"2024-05-09T06:28:45.804607+0000","last_clean":"2024-05-09T06:28:45.804607+0000","last_became_active":"2024-05-09T06:28:43.106027+0000","last_became_peered":"2024-05-09T06:28:43.106027+0000","last_unstale":"2024-05-09T06:28:45.804607+0000","last_undegraded":"2024-05-09T06:28:45.804607+0000","last_fullsized":"2024-05-09T06:28:45.804607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:01.004710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803981+0000","last_change":"2024-05-09T06:28:44.111157+0000","last_active":"2024-05-09T06:28:45.803981+0000","last_peered":"2024-05-09T06:28:45.803981+0000","last_clean":"2024-05-09T06:28:45.803981+0000","last_became_active":"2024-05-09T06:28:44.110878+0000","last_became_peered":"2024-05-09T06:28:44.110878+0000","last_unstale":"2024-05-09T06:28:45.803981+0000","last_undegraded":"2024-05-09T06:28:45.803981+0000","last_fullsized":"2024-05-09T06:28:45.803981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804714+0000","last_change":"2024-05-09T06:28:43.099299+0000","last_active":"2024-05-09T06:28:45.804714+0000","last_peered":"2024-05-09T06:28:45.804714+0000","last_clean":"2024-05-09T06:28:45.804714+0000","last_became_active":"2024-05-09T06:28:43.099001+0000","last_became_peered":"2024-05-09T06:28:43.099001+0000","last_unstale":"2024-05-09T06:28:45.804714+0000","last_undegraded":"2024-05-09T06:28:45.804714+0000","last_fullsized":"2024-05-09T06:28:45.804714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:21.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088489+0000","last_change":"2024-05-09T06:28:44.095549+0000","last_active":"2024-05-09T06:28:45.088489+0000","last_peered":"2024-05-09T06:28:45.088489+0000","last_clean":"2024-05-09T06:28:45.088489+0000","last_became_active":"2024-05-09T06:28:44.095259+0000","last_became_peered":"2024-05-09T06:28:44.095259+0000","last_unstale":"2024-05-09T06:28:45.088489+0000","last_undegraded":"2024-05-09T06:28:45.088489+0000","last_fullsized":"2024-05-09T06:28:45.088489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:06.903246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804887+0000","last_change":"2024-05-09T06:28:43.101397+0000","last_active":"2024-05-09T06:28:45.804887+0000","last_peered":"2024-05-09T06:28:45.804887+0000","last_clean":"2024-05-09T06:28:45.804887+0000","last_became_active":"2024-05-09T06:28:43.101067+0000","last_became_peered":"2024-05-09T06:28:43.101067+0000","last_unstale":"2024-05-09T06:28:45.804887+0000","last_undegraded":"2024-05-09T06:28:45.804887+0000","last_fullsized":"2024-05-09T06:28:45.804887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:56.133420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804428+0000","last_change":"2024-05-09T06:28:44.108877+0000","last_active":"2024-05-09T06:28:45.804428+0000","last_peered":"2024-05-09T06:28:45.804428+0000","last_clean":"2024-05-09T06:28:45.804428+0000","last_became_active":"2024-05-09T06:28:44.108580+0000","last_became_peered":"2024-05-09T06:28:44.108580+0000","last_unstale":"2024-05-09T06:28:45.804428+0000","last_undegraded":"2024-05-09T06:28:45.804428+0000","last_fullsized":"2024-05-09T06:28:45.804428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:01.498257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804402+0000","last_change":"2024-05-09T06:28:43.110342+0000","last_active":"2024-05-09T06:28:45.804402+0000","last_peered":"2024-05-09T06:28:45.804402+0000","last_clean":"2024-05-09T06:28:45.804402+0000","last_became_active":"2024-05-09T06:28:43.110095+0000","last_became_peered":"2024-05-09T06:28:43.110095+0000","last_unstale":"2024-05-09T06:28:45.804402+0000","last_undegraded":"2024-05-09T06:28:45.804402+0000","last_fullsized":"2024-05-09T06:28:45.804402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:32.776178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803975+0000","last_change":"2024-05-09T06:28:44.111068+0000","last_active":"2024-05-09T06:28:45.803975+0000","last_peered":"2024-05-09T06:28:45.803975+0000","last_clean":"2024-05-09T06:28:45.803975+0000","last_became_active":"2024-05-09T06:28:44.110780+0000","last_became_peered":"2024-05-09T06:28:44.110780+0000","last_unstale":"2024-05-09T06:28:45.803975+0000","last_undegraded":"2024-05-09T06:28:45.803975+0000","last_fullsized":"2024-05-09T06:28:45.803975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:57.138774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804836+0000","last_change":"2024-05-09T06:28:43.106566+0000","last_active":"2024-05-09T06:28:45.804836+0000","last_peered":"2024-05-09T06:28:45.804836+0000","last_clean":"2024-05-09T06:28:45.804836+0000","last_became_active":"2024-05-09T06:28:43.105323+0000","last_became_peered":"2024-05-09T06:28:43.105323+0000","last_unstale":"2024-05-09T06:28:45.804836+0000","last_undegraded":"2024-05-09T06:28:45.804836+0000","last_fullsized":"2024-05-09T06:28:45.804836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:09.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805517+0000","last_change":"2024-05-09T06:28:44.103166+0000","last_active":"2024-05-09T06:28:45.805517+0000","last_peered":"2024-05-09T06:28:45.805517+0000","last_clean":"2024-05-09T06:28:45.805517+0000","last_became_active":"2024-05-09T06:28:44.102891+0000","last_became_peered":"2024-05-09T06:28:44.102891+0000","last_unstale":"2024-05-09T06:28:45.805517+0000","last_undegraded":"2024-05-09T06:28:45.805517+0000","last_fullsized":"2024-05-09T06:28:45.805517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:15.692861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804356+0000","last_change":"2024-05-09T06:28:43.105704+0000","last_active":"2024-05-09T06:28:45.804356+0000","last_peered":"2024-05-09T06:28:45.804356+0000","last_clean":"2024-05-09T06:28:45.804356+0000","last_became_active":"2024-05-09T06:28:43.105384+0000","last_became_peered":"2024-05-09T06:28:43.105384+0000","last_unstale":"2024-05-09T06:28:45.804356+0000","last_undegraded":"2024-05-09T06:28:45.804356+0000","last_fullsized":"2024-05-09T06:28:45.804356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:51.065483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803607+0000","last_change":"2024-05-09T06:28:44.115185+0000","last_active":"2024-05-09T06:28:45.803607+0000","last_peered":"2024-05-09T06:28:45.803607+0000","last_clean":"2024-05-09T06:28:45.803607+0000","last_became_active":"2024-05-09T06:28:44.114856+0000","last_became_peered":"2024-05-09T06:28:44.114856+0000","last_unstale":"2024-05-09T06:28:45.803607+0000","last_undegraded":"2024-05-09T06:28:45.803607+0000","last_fullsized":"2024-05-09T06:28:45.803607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804527+0000","last_change":"2024-05-09T06:28:43.097679+0000","last_active":"2024-05-09T06:28:45.804527+0000","last_peered":"2024-05-09T06:28:45.804527+0000","last_clean":"2024-05-09T06:28:45.804527+0000","last_became_active":"2024-05-09T06:28:43.097373+0000","last_became_peered":"2024-05-09T06:28:43.097373+0000","last_unstale":"2024-05-09T06:28:45.804527+0000","last_undegraded":"2024-05-09T06:28:45.804527+0000","last_fullsized":"2024-05-09T06:28:45.804527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:40.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804000+0000","last_change":"2024-05-09T06:28:44.101203+0000","last_active":"2024-05-09T06:28:45.804000+0000","last_peered":"2024-05-09T06:28:45.804000+0000","last_clean":"2024-05-09T06:28:45.804000+0000","last_became_active":"2024-05-09T06:28:44.100650+0000","last_became_peered":"2024-05-09T06:28:44.100650+0000","last_unstale":"2024-05-09T06:28:45.804000+0000","last_undegraded":"2024-05-09T06:28:45.804000+0000","last_fullsized":"2024-05-09T06:28:45.804000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804933+0000","last_change":"2024-05-09T06:28:43.107557+0000","last_active":"2024-05-09T06:28:45.804933+0000","last_peered":"2024-05-09T06:28:45.804933+0000","last_clean":"2024-05-09T06:28:45.804933+0000","last_became_active":"2024-05-09T06:28:43.107217+0000","last_became_peered":"2024-05-09T06:28:43.107217+0000","last_unstale":"2024-05-09T06:28:45.804933+0000","last_undegraded":"2024-05-09T06:28:45.804933+0000","last_fullsized":"2024-05-09T06:28:45.804933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:32.776178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803980+0000","last_change":"2024-05-09T06:28:44.114434+0000","last_active":"2024-05-09T06:28:45.803980+0000","last_peered":"2024-05-09T06:28:45.803980+0000","last_clean":"2024-05-09T06:28:45.803980+0000","last_became_active":"2024-05-09T06:28:44.114161+0000","last_became_peered":"2024-05-09T06:28:44.114161+0000","last_unstale":"2024-05-09T06:28:45.803980+0000","last_undegraded":"2024-05-09T06:28:45.803980+0000","last_fullsized":"2024-05-09T06:28:45.803980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092496+0000","last_change":"2024-05-09T06:28:43.096518+0000","last_active":"2024-05-09T06:28:45.092496+0000","last_peered":"2024-05-09T06:28:45.092496+0000","last_clean":"2024-05-09T06:28:45.092496+0000","last_became_active":"2024-05-09T06:28:43.096235+0000","last_became_peered":"2024-05-09T06:28:43.096235+0000","last_unstale":"2024-05-09T06:28:45.092496+0000","last_undegraded":"2024-05-09T06:28:45.092496+0000","last_fullsized":"2024-05-09T06:28:45.092496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:44.161063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803851+0000","last_change":"2024-05-09T06:28:44.102832+0000","last_active":"2024-05-09T06:28:45.803851+0000","last_peered":"2024-05-09T06:28:45.803851+0000","last_clean":"2024-05-09T06:28:45.803851+0000","last_became_active":"2024-05-09T06:28:44.102511+0000","last_became_peered":"2024-05-09T06:28:44.102511+0000","last_unstale":"2024-05-09T06:28:45.803851+0000","last_undegraded":"2024-05-09T06:28:45.803851+0000","last_fullsized":"2024-05-09T06:28:45.803851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:42.412580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804109+0000","last_change":"2024-05-09T06:28:43.108183+0000","last_active":"2024-05-09T06:28:45.804109+0000","last_peered":"2024-05-09T06:28:45.804109+0000","last_clean":"2024-05-09T06:28:45.804109+0000","last_became_active":"2024-05-09T06:28:43.107072+0000","last_became_peered":"2024-05-09T06:28:43.107072+0000","last_unstale":"2024-05-09T06:28:45.804109+0000","last_undegraded":"2024-05-09T06:28:45.804109+0000","last_fullsized":"2024-05-09T06:28:45.804109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:01.004710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804243+0000","last_change":"2024-05-09T06:28:44.108644+0000","last_active":"2024-05-09T06:28:45.804243+0000","last_peered":"2024-05-09T06:28:45.804243+0000","last_clean":"2024-05-09T06:28:45.804243+0000","last_became_active":"2024-05-09T06:28:44.108353+0000","last_became_peered":"2024-05-09T06:28:44.108353+0000","last_unstale":"2024-05-09T06:28:45.804243+0000","last_undegraded":"2024-05-09T06:28:45.804243+0000","last_fullsized":"2024-05-09T06:28:45.804243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:44.500703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805059+0000","last_change":"2024-05-09T06:28:43.108305+0000","last_active":"2024-05-09T06:28:45.805059+0000","last_peered":"2024-05-09T06:28:45.805059+0000","last_clean":"2024-05-09T06:28:45.805059+0000","last_became_active":"2024-05-09T06:28:43.107930+0000","last_became_peered":"2024-05-09T06:28:43.107930+0000","last_unstale":"2024-05-09T06:28:45.805059+0000","last_undegraded":"2024-05-09T06:28:45.805059+0000","last_fullsized":"2024-05-09T06:28:45.805059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:14.687508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804748+0000","last_change":"2024-05-09T06:28:44.117972+0000","last_active":"2024-05-09T06:28:45.804748+0000","last_peered":"2024-05-09T06:28:45.804748+0000","last_clean":"2024-05-09T06:28:45.804748+0000","last_became_active":"2024-05-09T06:28:44.116446+0000","last_became_peered":"2024-05-09T06:28:44.116446+0000","last_unstale":"2024-05-09T06:28:45.804748+0000","last_undegraded":"2024-05-09T06:28:45.804748+0000","last_fullsized":"2024-05-09T06:28:45.804748+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805776+0000","last_change":"2024-05-09T06:28:43.106417+0000","last_active":"2024-05-09T06:28:45.805776+0000","last_peered":"2024-05-09T06:28:45.805776+0000","last_clean":"2024-05-09T06:28:45.805776+0000","last_became_active":"2024-05-09T06:28:43.106004+0000","last_became_peered":"2024-05-09T06:28:43.106004+0000","last_unstale":"2024-05-09T06:28:45.805776+0000","last_undegraded":"2024-05-09T06:28:45.805776+0000","last_fullsized":"2024-05-09T06:28:45.805776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.310229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805694+0000","last_change":"2024-05-09T06:28:44.104985+0000","last_active":"2024-05-09T06:28:45.805694+0000","last_peered":"2024-05-09T06:28:45.805694+0000","last_clean":"2024-05-09T06:28:45.805694+0000","last_became_active":"2024-05-09T06:28:44.104710+0000","last_became_peered":"2024-05-09T06:28:44.104710+0000","last_unstale":"2024-05-09T06:28:45.805694+0000","last_undegraded":"2024-05-09T06:28:45.805694+0000","last_fullsized":"2024-05-09T06:28:45.805694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:31.966815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088639+0000","last_change":"2024-05-09T06:28:43.105231+0000","last_active":"2024-05-09T06:28:45.088639+0000","last_peered":"2024-05-09T06:28:45.088639+0000","last_clean":"2024-05-09T06:28:45.088639+0000","last_became_active":"2024-05-09T06:28:43.104944+0000","last_became_peered":"2024-05-09T06:28:43.104944+0000","last_unstale":"2024-05-09T06:28:45.088639+0000","last_undegraded":"2024-05-09T06:28:45.088639+0000","last_fullsized":"2024-05-09T06:28:45.088639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:51.001350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088813+0000","last_change":"2024-05-09T06:28:44.103963+0000","last_active":"2024-05-09T06:28:45.088813+0000","last_peered":"2024-05-09T06:28:45.088813+0000","last_clean":"2024-05-09T06:28:45.088813+0000","last_became_active":"2024-05-09T06:28:44.103648+0000","last_became_peered":"2024-05-09T06:28:44.103648+0000","last_unstale":"2024-05-09T06:28:45.088813+0000","last_undegraded":"2024-05-09T06:28:45.088813+0000","last_fullsized":"2024-05-09T06:28:45.088813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:32.026477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804112+0000","last_change":"2024-05-09T06:28:43.104127+0000","last_active":"2024-05-09T06:28:45.804112+0000","last_peered":"2024-05-09T06:28:45.804112+0000","last_clean":"2024-05-09T06:28:45.804112+0000","last_became_active":"2024-05-09T06:28:43.103790+0000","last_became_peered":"2024-05-09T06:28:43.103790+0000","last_unstale":"2024-05-09T06:28:45.804112+0000","last_undegraded":"2024-05-09T06:28:45.804112+0000","last_fullsized":"2024-05-09T06:28:45.804112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:59.651383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804580+0000","last_change":"2024-05-09T06:28:44.115343+0000","last_active":"2024-05-09T06:28:45.804580+0000","last_peered":"2024-05-09T06:28:45.804580+0000","last_clean":"2024-05-09T06:28:45.804580+0000","last_became_active":"2024-05-09T06:28:44.115062+0000","last_became_peered":"2024-05-09T06:28:44.115062+0000","last_unstale":"2024-05-09T06:28:45.804580+0000","last_undegraded":"2024-05-09T06:28:45.804580+0000","last_fullsized":"2024-05-09T06:28:45.804580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:35.562590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088292+0000","last_change":"2024-05-09T06:28:43.101721+0000","last_active":"2024-05-09T06:28:45.088292+0000","last_peered":"2024-05-09T06:28:45.088292+0000","last_clean":"2024-05-09T06:28:45.088292+0000","last_became_active":"2024-05-09T06:28:43.101362+0000","last_became_peered":"2024-05-09T06:28:43.101362+0000","last_unstale":"2024-05-09T06:28:45.088292+0000","last_undegraded":"2024-05-09T06:28:45.088292+0000","last_fullsized":"2024-05-09T06:28:45.088292+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:21.371244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803756+0000","last_change":"2024-05-09T06:28:44.110189+0000","last_active":"2024-05-09T06:28:45.803756+0000","last_peered":"2024-05-09T06:28:45.803756+0000","last_clean":"2024-05-09T06:28:45.803756+0000","last_became_active":"2024-05-09T06:28:44.109910+0000","last_became_peered":"2024-05-09T06:28:44.109910+0000","last_unstale":"2024-05-09T06:28:45.803756+0000","last_undegraded":"2024-05-09T06:28:45.803756+0000","last_fullsized":"2024-05-09T06:28:45.803756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:52.070837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088225+0000","last_change":"2024-05-09T06:28:43.100163+0000","last_active":"2024-05-09T06:28:45.088225+0000","last_peered":"2024-05-09T06:28:45.088225+0000","last_clean":"2024-05-09T06:28:45.088225+0000","last_became_active":"2024-05-09T06:28:43.099826+0000","last_became_peered":"2024-05-09T06:28:43.099826+0000","last_unstale":"2024-05-09T06:28:45.088225+0000","last_undegraded":"2024-05-09T06:28:45.088225+0000","last_fullsized":"2024-05-09T06:28:45.088225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:31:46.432443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803957+0000","last_change":"2024-05-09T06:28:44.109742+0000","last_active":"2024-05-09T06:28:45.803957+0000","last_peered":"2024-05-09T06:28:45.803957+0000","last_clean":"2024-05-09T06:28:45.803957+0000","last_became_active":"2024-05-09T06:28:44.109442+0000","last_became_peered":"2024-05-09T06:28:44.109442+0000","last_unstale":"2024-05-09T06:28:45.803957+0000","last_undegraded":"2024-05-09T06:28:45.803957+0000","last_fullsized":"2024-05-09T06:28:45.803957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:05.558151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805352+0000","last_change":"2024-05-09T06:28:43.105547+0000","last_active":"2024-05-09T06:28:45.805352+0000","last_peered":"2024-05-09T06:28:45.805352+0000","last_clean":"2024-05-09T06:28:45.805352+0000","last_became_active":"2024-05-09T06:28:43.104806+0000","last_became_peered":"2024-05-09T06:28:43.104806+0000","last_unstale":"2024-05-09T06:28:45.805352+0000","last_undegraded":"2024-05-09T06:28:45.805352+0000","last_fullsized":"2024-05-09T06:28:45.805352+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:49.647442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804029+0000","last_change":"2024-05-09T06:28:44.110733+0000","last_active":"2024-05-09T06:28:45.804029+0000","last_peered":"2024-05-09T06:28:45.804029+0000","last_clean":"2024-05-09T06:28:45.804029+0000","last_became_active":"2024-05-09T06:28:44.110340+0000","last_became_peered":"2024-05-09T06:28:44.110340+0000","last_unstale":"2024-05-09T06:28:45.804029+0000","last_undegraded":"2024-05-09T06:28:45.804029+0000","last_fullsized":"2024-05-09T06:28:45.804029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:33.448866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088218+0000","last_change":"2024-05-09T06:28:43.099690+0000","last_active":"2024-05-09T06:28:45.088218+0000","last_peered":"2024-05-09T06:28:45.088218+0000","last_clean":"2024-05-09T06:28:45.088218+0000","last_became_active":"2024-05-09T06:28:43.099173+0000","last_became_peered":"2024-05-09T06:28:43.099173+0000","last_unstale":"2024-05-09T06:28:45.088218+0000","last_undegraded":"2024-05-09T06:28:45.088218+0000","last_fullsized":"2024-05-09T06:28:45.088218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:19.174636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803997+0000","last_change":"2024-05-09T06:28:44.107406+0000","last_active":"2024-05-09T06:28:45.803997+0000","last_peered":"2024-05-09T06:28:45.803997+0000","last_clean":"2024-05-09T06:28:45.803997+0000","last_became_active":"2024-05-09T06:28:44.107105+0000","last_became_peered":"2024-05-09T06:28:44.107105+0000","last_unstale":"2024-05-09T06:28:45.803997+0000","last_undegraded":"2024-05-09T06:28:45.803997+0000","last_fullsized":"2024-05-09T06:28:45.803997+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:41.031233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805183+0000","last_change":"2024-05-09T06:28:43.109099+0000","last_active":"2024-05-09T06:28:45.805183+0000","last_peered":"2024-05-09T06:28:45.805183+0000","last_clean":"2024-05-09T06:28:45.805183+0000","last_became_active":"2024-05-09T06:28:43.108456+0000","last_became_peered":"2024-05-09T06:28:43.108456+0000","last_unstale":"2024-05-09T06:28:45.805183+0000","last_undegraded":"2024-05-09T06:28:45.805183+0000","last_fullsized":"2024-05-09T06:28:45.805183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:37.004402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804609+0000","last_change":"2024-05-09T06:28:44.115453+0000","last_active":"2024-05-09T06:28:45.804609+0000","last_peered":"2024-05-09T06:28:45.804609+0000","last_clean":"2024-05-09T06:28:45.804609+0000","last_became_active":"2024-05-09T06:28:44.115172+0000","last_became_peered":"2024-05-09T06:28:44.115172+0000","last_unstale":"2024-05-09T06:28:45.804609+0000","last_undegraded":"2024-05-09T06:28:45.804609+0000","last_fullsized":"2024-05-09T06:28:45.804609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:20.788897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.088728+0000","last_change":"2024-05-09T06:28:43.105213+0000","last_active":"2024-05-09T06:28:45.088728+0000","last_peered":"2024-05-09T06:28:45.088728+0000","last_clean":"2024-05-09T06:28:45.088728+0000","last_became_active":"2024-05-09T06:28:43.104918+0000","last_became_peered":"2024-05-09T06:28:43.104918+0000","last_unstale":"2024-05-09T06:28:45.088728+0000","last_undegraded":"2024-05-09T06:28:45.088728+0000","last_fullsized":"2024-05-09T06:28:45.088728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:27.675907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804311+0000","last_change":"2024-05-09T06:28:44.113910+0000","last_active":"2024-05-09T06:28:45.804311+0000","last_peered":"2024-05-09T06:28:45.804311+0000","last_clean":"2024-05-09T06:28:45.804311+0000","last_became_active":"2024-05-09T06:28:44.113608+0000","last_became_peered":"2024-05-09T06:28:44.113608+0000","last_unstale":"2024-05-09T06:28:45.804311+0000","last_undegraded":"2024-05-09T06:28:45.804311+0000","last_fullsized":"2024-05-09T06:28:45.804311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:03.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804405+0000","last_change":"2024-05-09T06:28:43.102864+0000","last_active":"2024-05-09T06:28:45.804405+0000","last_peered":"2024-05-09T06:28:45.804405+0000","last_clean":"2024-05-09T06:28:45.804405+0000","last_became_active":"2024-05-09T06:28:43.102193+0000","last_became_peered":"2024-05-09T06:28:43.102193+0000","last_unstale":"2024-05-09T06:28:45.804405+0000","last_undegraded":"2024-05-09T06:28:45.804405+0000","last_fullsized":"2024-05-09T06:28:45.804405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.803784+0000","last_change":"2024-05-09T06:28:44.111576+0000","last_active":"2024-05-09T06:28:45.803784+0000","last_peered":"2024-05-09T06:28:45.803784+0000","last_clean":"2024-05-09T06:28:45.803784+0000","last_became_active":"2024-05-09T06:28:44.111251+0000","last_became_peered":"2024-05-09T06:28:44.111251+0000","last_unstale":"2024-05-09T06:28:45.803784+0000","last_undegraded":"2024-05-09T06:28:45.803784+0000","last_fullsized":"2024-05-09T06:28:45.803784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:41.031233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804673+0000","last_change":"2024-05-09T06:28:43.097963+0000","last_active":"2024-05-09T06:28:45.804673+0000","last_peered":"2024-05-09T06:28:45.804673+0000","last_clean":"2024-05-09T06:28:45.804673+0000","last_became_active":"2024-05-09T06:28:43.097670+0000","last_became_peered":"2024-05-09T06:28:43.097670+0000","last_unstale":"2024-05-09T06:28:45.804673+0000","last_undegraded":"2024-05-09T06:28:45.804673+0000","last_fullsized":"2024-05-09T06:28:45.804673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:31.021123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804586+0000","last_change":"2024-05-09T06:28:44.107029+0000","last_active":"2024-05-09T06:28:45.804586+0000","last_peered":"2024-05-09T06:28:45.804586+0000","last_clean":"2024-05-09T06:28:45.804586+0000","last_became_active":"2024-05-09T06:28:44.106699+0000","last_became_peered":"2024-05-09T06:28:44.106699+0000","last_unstale":"2024-05-09T06:28:45.804586+0000","last_undegraded":"2024-05-09T06:28:45.804586+0000","last_fullsized":"2024-05-09T06:28:45.804586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:29.172513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.092564+0000","last_change":"2024-05-09T06:28:43.094133+0000","last_active":"2024-05-09T06:28:45.092564+0000","last_peered":"2024-05-09T06:28:45.092564+0000","last_clean":"2024-05-09T06:28:45.092564+0000","last_became_active":"2024-05-09T06:28:43.093802+0000","last_became_peered":"2024-05-09T06:28:43.093802+0000","last_unstale":"2024-05-09T06:28:45.092564+0000","last_undegraded":"2024-05-09T06:28:45.092564+0000","last_fullsized":"2024-05-09T06:28:45.092564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:44:05.897892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805087+0000","last_change":"2024-05-09T06:28:44.104777+0000","last_active":"2024-05-09T06:28:45.805087+0000","last_peered":"2024-05-09T06:28:45.805087+0000","last_clean":"2024-05-09T06:28:45.805087+0000","last_became_active":"2024-05-09T06:28:44.104500+0000","last_became_peered":"2024-05-09T06:28:44.104500+0000","last_unstale":"2024-05-09T06:28:45.805087+0000","last_undegraded":"2024-05-09T06:28:45.805087+0000","last_fullsized":"2024-05-09T06:28:45.805087+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:16.842678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804745+0000","last_change":"2024-05-09T06:28:43.096894+0000","last_active":"2024-05-09T06:28:45.804745+0000","last_peered":"2024-05-09T06:28:45.804745+0000","last_clean":"2024-05-09T06:28:45.804745+0000","last_became_active":"2024-05-09T06:28:43.096592+0000","last_became_peered":"2024-05-09T06:28:43.096592+0000","last_unstale":"2024-05-09T06:28:45.804745+0000","last_undegraded":"2024-05-09T06:28:45.804745+0000","last_fullsized":"2024-05-09T06:28:45.804745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:02.760035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.805088+0000","last_change":"2024-05-09T06:28:44.108785+0000","last_active":"2024-05-09T06:28:45.805088+0000","last_peered":"2024-05-09T06:28:45.805088+0000","last_clean":"2024-05-09T06:28:45.805088+0000","last_became_active":"2024-05-09T06:28:44.108468+0000","last_became_peered":"2024-05-09T06:28:44.108468+0000","last_unstale":"2024-05-09T06:28:45.805088+0000","last_undegraded":"2024-05-09T06:28:45.805088+0000","last_fullsized":"2024-05-09T06:28:45.805088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:21.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804706+0000","last_change":"2024-05-09T06:28:43.101342+0000","last_active":"2024-05-09T06:28:45.804706+0000","last_peered":"2024-05-09T06:28:45.804706+0000","last_clean":"2024-05-09T06:28:45.804706+0000","last_became_active":"2024-05-09T06:28:43.101027+0000","last_became_peered":"2024-05-09T06:28:43.101027+0000","last_unstale":"2024-05-09T06:28:45.804706+0000","last_undegraded":"2024-05-09T06:28:45.804706+0000","last_fullsized":"2024-05-09T06:28:45.804706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:44.009601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804956+0000","last_change":"2024-05-09T06:28:44.116083+0000","last_active":"2024-05-09T06:28:45.804956+0000","last_peered":"2024-05-09T06:28:45.804956+0000","last_clean":"2024-05-09T06:28:45.804956+0000","last_became_active":"2024-05-09T06:28:44.115762+0000","last_became_peered":"2024-05-09T06:28:44.115762+0000","last_unstale":"2024-05-09T06:28:45.804956+0000","last_undegraded":"2024-05-09T06:28:45.804956+0000","last_fullsized":"2024-05-09T06:28:45.804956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:43.078294+0000","last_clean_scrub_stamp":"2024-05-09T06:28:43.078294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:56:33.448866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804194+0000","last_change":"2024-05-09T06:28:43.107403+0000","last_active":"2024-05-09T06:28:45.804194+0000","last_peered":"2024-05-09T06:28:45.804194+0000","last_clean":"2024-05-09T06:28:45.804194+0000","last_became_active":"2024-05-09T06:28:43.107064+0000","last_became_peered":"2024-05-09T06:28:43.107064+0000","last_unstale":"2024-05-09T06:28:45.804194+0000","last_undegraded":"2024-05-09T06:28:45.804194+0000","last_fullsized":"2024-05-09T06:28:45.804194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:28:42.072940+0000","last_clean_scrub_stamp":"2024-05-09T06:28:42.072940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:35.212436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-09T06:28:45.804686+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:28:45.804686+0000","last_peered":"2024-05-09T06:28:45.804686+0000","last_clean":"2024-05-09T06:28:45.804686+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:28:45.804686+0000","last_undegraded":"2024-05-09T06:28:45.804686+0000","last_fullsized":"2024-05-09T06:28:45.804686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":782086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":191682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":190106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":780438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:28:52.904 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:28:52.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:28:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.070+0000 7f68580d4640 1 Processor -- start 2024-05-09T06:28:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.070+0000 7f68580d4640 1 -- start start 2024-05-09T06:28:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f68580d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6850158a50 0x7f68500718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f68580d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f685006ede0 0x7f6850071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f68580d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685015b420 con 0x7f685015c4a0 2024-05-09T06:28:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f68580d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6850158e30 con 0x7f685006ede0 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f68580d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6850072480 con 0x7f6850158a50 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f685006ede0 0x7f6850071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f685006ede0 0x7f6850071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41946/0 (socket says 172.21.15.1:41946) 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 -- 172.21.15.1:0/3021420974 learned_addr learned my addr 172.21.15.1:0/3021420974 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6855e70640 1 --2- 172.21.15.1:0/3021420974 >> v2:172.21.15.6:3300/0 conn(0x7f6850158a50 0x7f68500718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 -- 172.21.15.1:0/3021420974 >> v2:172.21.15.6:3300/0 conn(0x7f6850158a50 msgr2=0x7f68500718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 --2- 172.21.15.1:0/3021420974 >> v2:172.21.15.6:3300/0 conn(0x7f6850158a50 0x7f68500718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 -- 172.21.15.1:0/3021420974 >> v1:172.21.15.6:6789/0 conn(0x7f685015c4a0 legacy=0x7f685006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.071+0000 7f6856671640 1 -- 172.21.15.1:0/3021420974 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685015b6e0 con 0x7f685006ede0 2024-05-09T06:28:53.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.072+0000 7f6856671640 1 --2- 172.21.15.1:0/3021420974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f685006ede0 0x7f6850071e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6850073950 tx=0x7f684802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d71a16e4875ac4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.072+0000 7f6854e6e640 1 -- 172.21.15.1:0/3021420974 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848036070 con 0x7f685006ede0 2024-05-09T06:28:53.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.072+0000 7f6854e6e640 1 -- 172.21.15.1:0/3021420974 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6848031600 con 0x7f685006ede0 2024-05-09T06:28:53.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.072+0000 7f6854e6e640 1 -- 172.21.15.1:0/3021420974 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480319d0 con 0x7f685006ede0 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 -- 172.21.15.1:0/3021420974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f685006ede0 msgr2=0x7f6850071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 --2- 172.21.15.1:0/3021420974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f685006ede0 0x7f6850071e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6850073950 tx=0x7f684802a730 comp rx=0 tx=0).stop 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 -- 172.21.15.1:0/3021420974 shutdown_connections 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 --2- 172.21.15.1:0/3021420974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f685006ede0 0x7f6850071e00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 --2- 172.21.15.1:0/3021420974 >> v2:172.21.15.6:3300/0 conn(0x7f6850158a50 0x7f68500718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 -- 172.21.15.1:0/3021420974 >> 172.21.15.1:0/3021420974 conn(0x7f68500cc4d0 msgr2=0x7f68500cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 -- 172.21.15.1:0/3021420974 shutdown_connections 2024-05-09T06:28:53.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.073+0000 7f68580d4640 1 -- 172.21.15.1:0/3021420974 wait complete. 2024-05-09T06:28:53.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.074+0000 7f68580d4640 1 Processor -- start 2024-05-09T06:28:53.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.074+0000 7f68580d4640 1 -- start start 2024-05-09T06:28:53.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f685006ede0 0x7f68500b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 0x7f685015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f685015bf80 con 0x7f6850158a50 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f685015bca0 con 0x7f685015c4a0 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f68580d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6850070ea0 con 0x7f685006ede0 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41948/0 (socket says 172.21.15.1:41948) 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 -- 172.21.15.1:0/503461589 learned_addr learned my addr 172.21.15.1:0/503461589 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f6856671640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 0x7f685015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f6855e70640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f685006ede0 0x7f68500b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 -- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f685006ede0 msgr2=0x7f68500b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f685006ede0 0x7f68500b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 -- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 msgr2=0x7f685015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 0x7f685015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f685566f640 1 -- 172.21.15.1:0/503461589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501649d0 con 0x7f6850158a50 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.075+0000 7f6856671640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 0x7f685015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f685566f640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f684000c370 tx=0x7f684000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f6850158a50 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68400052d0 con 0x7f6850158a50 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6850164c90 con 0x7f6850158a50 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840003850 con 0x7f6850158a50 2024-05-09T06:28:53.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.076+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501675e0 con 0x7f6850158a50 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.077+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68500752b0 con 0x7f6850158a50 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.077+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f684001a070 con 0x7f6850158a50 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.078+0000 7f68467fc640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 0x7f683406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.078+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f684008a310 con 0x7f6850158a50 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.078+0000 7f6855e70640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 0x7f683406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.078+0000 7f6855e70640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 0x7f683406e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f684c009a10 tx=0x7f684c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.082+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6840058b30 con 0x7f6850158a50 2024-05-09T06:28:53.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.338+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6850165510 con 0x7f6850158a50 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.340+0000 7f68467fc640 1 -- 172.21.15.1:0/503461589 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f6840058430 con 0x7f6850158a50 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:28:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:28:53.323 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:28:53.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 msgr2=0x7f683406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 0x7f683406e320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f684c009a10 tx=0x7f684c008040 comp rx=0 tx=0).stop 2024-05-09T06:28:53.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 msgr2=0x7f6850159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f684000c370 tx=0x7f684000c830 comp rx=0 tx=0).stop 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 shutdown_connections 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f683406be60 0x7f683406e320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f685015c4a0 0x7f685015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6850158a50 0x7f6850159f10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 --2- 172.21.15.1:0/503461589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f685006ede0 0x7f68500b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 >> 172.21.15.1:0/503461589 conn(0x7f68500cc4d0 msgr2=0x7f68501590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 shutdown_connections 2024-05-09T06:28:53.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.342+0000 7f68580d4640 1 -- 172.21.15.1:0/503461589 wait complete. 2024-05-09T06:28:53.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:28:53.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.457+0000 7f3fa1f63640 1 Processor -- start 2024-05-09T06:28:53.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.457+0000 7f3fa1f63640 1 -- start start 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3fa1f63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f9c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3fa1f63640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9c0d04a0 con 0x7f3f9c15ece0 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3fa1f63640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9c0722c0 con 0x7f3f9c0b0080 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3fa1f63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0725a0 con 0x7f3f9c071ce0 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3f9bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3f9c15ece0 0x7f3f9c06b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58608/0 (socket says 172.21.15.1:58608) 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.458+0000 7f3f9bfff640 1 -- 172.21.15.1:0/3055770494 learned_addr learned my addr 172.21.15.1:0/3055770494 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9affd640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f9c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665726491 0 0) 0x7f3f9c0d04a0 con 0x7f3f9c15ece0 2024-05-09T06:28:53.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f84003520 con 0x7f3f9c15ece0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496557715 0 0) 0x7f3f84003520 con 0x7f3f9c15ece0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 msgr2=0x7f3f9c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f9c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 >> v1:172.21.15.6:6789/0 conn(0x7f3f9c0b0080 legacy=0x7f3f9c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c072880 con 0x7f3f9c15ece0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3f940019f0 con 0x7f3f9c15ece0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 >> v1:172.21.15.1:6789/0 conn(0x7f3f9c15ece0 legacy=0x7f3f9c06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f840039e0 0x7f3f84003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f84004d40 0x7f3f840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0725a0 con 0x7f3f84004d40 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8400ba70 con 0x7f3f840039e0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8400bc70 con 0x7f3f9c071ce0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f84004d40 0x7f3f840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9b7fe640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f840039e0 0x7f3f84003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9affd640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 -- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 msgr2=0x7f3f84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c071ce0 0x7f3f84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 -- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f840039e0 msgr2=0x7f3f84003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f840039e0 0x7f3f84003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.459+0000 7f3f9bfff640 1 -- 172.21.15.1:0/3055770494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f84003520 con 0x7f3f84004d40 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.460+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f84004d40 0x7f3f840051c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3f94002230 tx=0x7f3f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8823f783c2470e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.460+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400a4a0 con 0x7f3f84004d40 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.460+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9402f070 con 0x7f3f84004d40 2024-05-09T06:28:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.460+0000 7f3f9a7fc640 1 -- 172.21.15.1:0/3055770494 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9402bcb0 con 0x7f3f84004d40 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 -- 172.21.15.1:0/3055770494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f84004d40 msgr2=0x7f3f840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/3055770494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f84004d40 0x7f3f840051c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3f94002230 tx=0x7f3f84005830 comp rx=0 tx=0).stop 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3f9bfff640 1 -- 172.21.15.1:0/3055770494 reap_dead start 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 -- 172.21.15.1:0/3055770494 shutdown_connections 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 -- 172.21.15.1:0/3055770494 >> 172.21.15.1:0/3055770494 conn(0x7f3f9c0d0b50 msgr2=0x7f3f9c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 -- 172.21.15.1:0/3055770494 shutdown_connections 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.461+0000 7f3fa1f63640 1 -- 172.21.15.1:0/3055770494 wait complete. 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.462+0000 7f3fa1f63640 1 Processor -- start 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.462+0000 7f3fa1f63640 1 -- start start 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 0x7f3f9c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 0x7f3f9c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c15ff40 con 0x7f3f9c159400 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c15e7c0 con 0x7f3f9c0ca570 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3fa1f63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c06b8f0 con 0x7f3f9c0c8210 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41978/0 (socket says 172.21.15.1:41978) 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/2923342242 learned_addr learned my addr 172.21.15.1:0/2923342242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9affd640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 0x7f3f9c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 msgr2=0x7f3f9c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 0x7f3f9c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 0x7f3f9c0b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 msgr2=0x7f3f9c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 0x7f3f9c0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/2923342242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0c8a70 con 0x7f3f9c159400 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9affd640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 0x7f3f9c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.463+0000 7f3f9bfff640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 0x7f3f9c0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3f9b7fe640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00f850 tx=0x7f3f8c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00fe40 con 0x7f3f9c159400 2024-05-09T06:28:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c018070 con 0x7f3f9c159400 2024-05-09T06:28:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c0c8d50 con 0x7f3f9c159400 2024-05-09T06:28:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00d730 con 0x7f3f9c159400 2024-05-09T06:28:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.464+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c0c9310 con 0x7f3f9c159400 2024-05-09T06:28:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.465+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c0752b0 con 0x7f3f9c159400 2024-05-09T06:28:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.465+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f8c00d8d0 con 0x7f3f9c159400 2024-05-09T06:28:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.466+0000 7f3f83fff640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.466+0000 7f3f9affd640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.466+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3f8c08ae60 con 0x7f3f9c159400 2024-05-09T06:28:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.466+0000 7f3f9affd640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b6850 tx=0x7f3f90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.470+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f8c059710 con 0x7f3f9c159400 2024-05-09T06:28:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.693+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f9c0ca950 con 0x7f3f9c159400 2024-05-09T06:28:53.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.694+0000 7f3f83fff640 1 -- 172.21.15.1:0/2923342242 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7f3f8c059010 con 0x7f3f9c159400 2024-05-09T06:28:53.676 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:53.676 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:28:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.695+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 msgr2=0x7f3f8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.695+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b6850 tx=0x7f3f90008040 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 msgr2=0x7f3f9c0ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00f850 tx=0x7f3f8c00fd10 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 shutdown_connections 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0c8210 0x7f3f9c0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0ca570 0x7f3f9c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 --2- 172.21.15.1:0/2923342242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c159400 0x7f3f9c0ca030 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 >> 172.21.15.1:0/2923342242 conn(0x7f3f9c0d0b50 msgr2=0x7f3f9c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 shutdown_connections 2024-05-09T06:28:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:53.696+0000 7f3fa1f63640 1 -- 172.21.15.1:0/2923342242 wait complete. 2024-05-09T06:28:54.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:28:43.462346+0000', 'modified': '2024-05-09T06:28:46.479534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5550': {'gid': 5550, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6834/642815597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6832', 'nonce': 642815597}, {'type': 'v1', 'addr': '172.21.15.6:6834', 'nonce': 642815597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5550, 'qdb_cluster': [5550]} 2024-05-09T06:28:54.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:28:54.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T06:28:54.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.218+0000 7f74911ed640 1 Processor -- start 2024-05-09T06:28:54.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.218+0000 7f74911ed640 1 -- start start 2024-05-09T06:28:54.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f74911ed640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f748c06e6d0 0x7f748c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f74911ed640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0ce0e0 con 0x7f748c0b3300 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f74911ed640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0ce3c0 con 0x7f748c0cf180 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f74911ed640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f748c0ce6a0 con 0x7f748c06e6d0 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f748ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f748c06e6d0 0x7f748c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f748b59e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f748c0b3300 0x7f748c0b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58612/0 (socket says 172.21.15.1:58612) 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.219+0000 7f748b59e640 1 -- 172.21.15.1:0/3056932437 learned_addr learned my addr 172.21.15.1:0/3056932437 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f748ad9d640 1 -- 172.21.15.1:0/3056932437 >> v1:172.21.15.6:6789/0 conn(0x7f748c0cf180 legacy=0x7f748c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f748ad9d640 1 -- 172.21.15.1:0/3056932437 >> v1:172.21.15.1:6789/0 conn(0x7f748c0b3300 legacy=0x7f748c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f748ad9d640 1 -- 172.21.15.1:0/3056932437 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0ce980 con 0x7f748c06e6d0 2024-05-09T06:28:54.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f7489d9b640 1 -- 172.21.15.1:0/3056932437 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676783618 0 0) 0x7f748c0ce0e0 con 0x7f748c0b3300 2024-05-09T06:28:54.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f748ad9d640 1 --2- 172.21.15.1:0/3056932437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c06eab0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9fd82547d151a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f7489d9b640 1 -- 172.21.15.1:0/3056932437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c037070 con 0x7f748c06e6d0 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.220+0000 7f7489d9b640 1 -- 172.21.15.1:0/3056932437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c032440 con 0x7f748c06e6d0 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f7489d9b640 1 -- 172.21.15.1:0/3056932437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c032810 con 0x7f748c06e6d0 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 -- 172.21.15.1:0/3056932437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 msgr2=0x7f748c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 --2- 172.21.15.1:0/3056932437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c06eab0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02a230 comp rx=0 tx=0).stop 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 -- 172.21.15.1:0/3056932437 shutdown_connections 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 --2- 172.21.15.1:0/3056932437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c06eab0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 -- 172.21.15.1:0/3056932437 >> 172.21.15.1:0/3056932437 conn(0x7f748c0cc4b0 msgr2=0x7f748c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 -- 172.21.15.1:0/3056932437 shutdown_connections 2024-05-09T06:28:54.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.221+0000 7f74911ed640 1 -- 172.21.15.1:0/3056932437 wait complete. 2024-05-09T06:28:54.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 Processor -- start 2024-05-09T06:28:54.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 -- start start 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f748c0b3300 0x7f748c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c0cf180 0x7f748c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f748c070ed0 con 0x7f748c06e6d0 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f748c06eff0 con 0x7f748c0cf180 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.223+0000 7f74911ed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f748c1447c0 con 0x7f748c0b3300 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41990/0 (socket says 172.21.15.1:41990) 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 -- 172.21.15.1:0/1452073841 learned_addr learned my addr 172.21.15.1:0/1452073841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748b59e640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c0cf180 0x7f748c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748a59c640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f748c0b3300 0x7f748c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 -- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f748c0b3300 msgr2=0x7f748c0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f748c0b3300 0x7f748c0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 -- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c0cf180 msgr2=0x7f748c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c0cf180 0x7f748c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 -- 172.21.15.1:0/1452073841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0a9620 con 0x7f748c06e6d0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f74837fe640 1 -- 172.21.15.1:0/1452073841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c037040 con 0x7f748c06e6d0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f74837fe640 1 -- 172.21.15.1:0/1452073841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c042070 con 0x7f748c06e6d0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c0a9900 con 0x7f748c06e6d0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f74837fe640 1 -- 172.21.15.1:0/1452073841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c030a70 con 0x7f748c06e6d0 2024-05-09T06:28:54.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.224+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c15efd0 con 0x7f748c06e6d0 2024-05-09T06:28:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.226+0000 7f74837fe640 1 -- 172.21.15.1:0/1452073841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f747c030c10 con 0x7f748c06e6d0 2024-05-09T06:28:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.226+0000 7f74837fe640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 0x7f746c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.226+0000 7f74837fe640 1 -- 172.21.15.1:0/1452073841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f747c0ab940 con 0x7f748c06e6d0 2024-05-09T06:28:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.226+0000 7f748a59c640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 0x7f746c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.227+0000 7f748a59c640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 0x7f746c06e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f748c0b0640 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.227+0000 7f7493fcf740 1 Processor -- start 2024-05-09T06:28:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.227+0000 7f7493fcf740 1 -- start start 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715556670 0x557715556a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55771552c350 0x5577155725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55771548f4d0 con 0x557715573f10 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5577154373e0 con 0x55771552c350 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55771548b490 con 0x557715556670 2024-05-09T06:28:54.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41998/0 (socket says 172.21.15.1:41998) 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f748a59c640 1 -- 172.21.15.1:0/674537169 learned_addr learned my addr 172.21.15.1:0/674537169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f748b59e640 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55771552c350 0x5577155725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.228+0000 7f748ad9d640 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715556670 0x557715556a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 -- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715556670 msgr2=0x557715556a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715556670 0x557715556a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 -- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55771552c350 msgr2=0x5577155725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55771552c350 0x5577155725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 -- 172.21.15.1:0/674537169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577151a32d0 con 0x557715573f10 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f748a59c640 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7474001630 tx=0x7f7474012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8669b25af2af20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f7480ff9640 1 -- 172.21.15.1:0/674537169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474025070 con 0x557715573f10 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f7480ff9640 1 -- 172.21.15.1:0/674537169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747401b050 con 0x557715573f10 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f7480ff9640 1 -- 172.21.15.1:0/674537169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474014de0 con 0x557715573f10 2024-05-09T06:28:54.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f7493fcf740 1 -- 172.21.15.1:0/674537169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 msgr2=0x557715574460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.229+0000 7f7493fcf740 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7474001630 tx=0x7f7474012000 comp rx=0 tx=0).stop 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 -- 172.21.15.1:0/674537169 shutdown_connections 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55771552c350 0x5577155725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715573f10 0x557715574460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 --2- 172.21.15.1:0/674537169 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715556670 0x557715556a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 -- 172.21.15.1:0/674537169 >> 172.21.15.1:0/674537169 conn(0x557715528c80 msgr2=0x557715571480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 -- 172.21.15.1:0/674537169 shutdown_connections 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 -- 172.21.15.1:0/674537169 wait complete. 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 -- start start 2024-05-09T06:28:54.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.230+0000 7f7493fcf740 1 Processor -- start 2024-05-09T06:28:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715568a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55771552d0f0 con 0x557715556670 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55771544a8e0 con 0x557715573f10 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x557715526bb0 con 0x557715552fc0 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f7493fcf740 10 client.0 _ll_register_callbacks cb 0x5577154179c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f748ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f748ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42004/0 (socket says 172.21.15.1:42004) 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f748ad9d640 1 -- 172.21.15.1:0/4282986570 learned_addr learned my addr 172.21.15.1:0/4282986570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f748b59e640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715568a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.231+0000 7f748a59c640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 -- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 msgr2=0x557715568a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715568a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 -- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 msgr2=0x557715568530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 -- 172.21.15.1:0/4282986570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771526d6b0 con 0x557715556670 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748b59e640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748a59c640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f748ad9d640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f747c075f70 tx=0x7f747c032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fba62ce2c87c23 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c079f70 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f747c0821b0 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747401b050 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f747c082400 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 10 client.5745 fetch_fsmap learned FSMap version 18 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5577155589f0 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747401b050 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f747c03a030 con 0x557715556670 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 10 client.5745 fetch_fsmap finished waiting for FSMap version 18 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 10 client.5745 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 20 client.5745 populate_metadata read hostname 'smithi001' 2024-05-09T06:28:54.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715585370 0x557715557fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 4 client.5745 mds_command: new command op to 5550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.232+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 --> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771549cc00 con 0x557715585370 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.233+0000 7f748a59c640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715585370 0x557715557fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.233+0000 7f7469ffb640 10 client.5745 ms_handle_connect on v2:172.21.15.6:6832/642815597 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.233+0000 7f748a59c640 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715585370 0x557715557fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef294b8943b1078 server_cookie=48d870e550572283 in_seq=0 out_seq=0 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7469ffb640 1 -- 172.21.15.1:0/4282986570 <== mds.0 v2:172.21.15.6:6832/642815597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55771549cc00 con 0x557715585370 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7469ffb640 10 client.5745 handle_command_reply: tid=0 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 client.5745 shutdown 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 msgr2=0x55771555a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f747c075f70 tx=0x7f747c032440 comp rx=0 tx=0).stop 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 shutdown_connections 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715585370 0x557715557fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7469ffb640 0 client.5745 ms_handle_reset on v2:172.21.15.6:6832/642815597 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 --2- 172.21.15.1:0/4282986570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x55771555a140 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 >> 172.21.15.1:0/4282986570 conn(0x55771552c350 msgr2=0x557715575290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 shutdown_connections 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 1 -- 172.21.15.1:0/4282986570 wait complete. 2024-05-09T06:28:54.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.234+0000 7f7493fcf740 20 client.5745 trim_cache size 0 max 16384 2024-05-09T06:28:54.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.252+0000 7f7493fcf740 1 Processor -- start 2024-05-09T06:28:54.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 -- start start 2024-05-09T06:28:54.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715555a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1fb640 con 0x557715556670 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f748c14a720 con 0x557715573f10 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f748c14ad70 con 0x557715552fc0 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42020/0 (socket says 172.21.15.1:42020) 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.253+0000 7f748a59c640 1 -- 172.21.15.1:0/1916871774 learned_addr learned my addr 172.21.15.1:0/1916871774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 -- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 msgr2=0x557715555a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715555a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 -- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 msgr2=0x557715568a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 -- 172.21.15.1:0/1916871774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557715581430 con 0x557715556670 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f748a59c640 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f7474015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d4d87fa4528bac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f746bfff640 1 -- 172.21.15.1:0/1916871774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474036070 con 0x557715556670 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f746bfff640 1 -- 172.21.15.1:0/1916871774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747402c840 con 0x557715556670 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f746bfff640 1 -- 172.21.15.1:0/1916871774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474025040 con 0x557715556670 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 -- 172.21.15.1:0/1916871774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 msgr2=0x557715568510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7474001eb0 tx=0x7f7474015000 comp rx=0 tx=0).stop 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 -- 172.21.15.1:0/1916871774 shutdown_connections 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715568510 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1916871774 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x557715555a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 -- 172.21.15.1:0/1916871774 >> 172.21.15.1:0/1916871774 conn(0x55771552c350 msgr2=0x55771552c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.254+0000 7f7493fcf740 1 -- 172.21.15.1:0/1916871774 shutdown_connections 2024-05-09T06:28:54.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.255+0000 7f7493fcf740 1 -- 172.21.15.1:0/1916871774 wait complete. 2024-05-09T06:28:54.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.255+0000 7f7493fcf740 1 -- start start 2024-05-09T06:28:54.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.255+0000 7f7493fcf740 1 Processor -- start 2024-05-09T06:28:54.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x5577155ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5577155851f0 con 0x557715556670 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55771556af50 con 0x557715573f10 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x557715585840 con 0x557715552fc0 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f7493fcf740 10 client.0 _ll_register_callbacks cb 0x7f747c006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42030/0 (socket says 172.21.15.1:42030) 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 -- 172.21.15.1:0/1908306106 learned_addr learned my addr 172.21.15.1:0/1908306106 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x5577155ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748b59e640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 -- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 msgr2=0x5577155ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x5577155ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 -- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 msgr2=0x557715568510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.256+0000 7f748a59c640 1 -- 172.21.15.1:0/1908306106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771556a150 con 0x557715556670 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f748a59c640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7474034040 tx=0x7f7474001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7045a037151e304e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474036030 con 0x557715556670 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f747402c640 con 0x557715556670 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747402c840 con 0x557715556670 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7474025040 con 0x557715556670 2024-05-09T06:28:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 10 client.5751 fetch_fsmap learned FSMap version 18 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557715584420 con 0x557715556670 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747402c840 con 0x557715556670 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f747402c040 con 0x557715556670 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 10 client.5751 fetch_fsmap finished waiting for FSMap version 18 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 10 client.5751 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 20 client.5751 populate_metadata read hostname 'smithi001' 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715583830 0x557715583c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 4 client.5751 mds_command: new command op to 5550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 --> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] -- command(tid 0: {"prefix": "status"}) -- 0x557715496f50 con 0x557715583830 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.257+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715583830 0x557715583c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7469ffb640 10 client.5751 ms_handle_connect on v2:172.21.15.6:6832/642815597 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f748ad9d640 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715583830 0x557715583c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee43405f499f353 server_cookie=95a4bd8f0ea15d60 in_seq=0 out_seq=0 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7469ffb640 1 -- 172.21.15.1:0/1908306106 <== mds.0 v2:172.21.15.6:6832/642815597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x557715496f50 con 0x557715583830 2024-05-09T06:28:54.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7469ffb640 10 client.5751 handle_command_reply: tid=0 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 client.5751 shutdown 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 msgr2=0x557715569540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7474034040 tx=0x7f7474001120 comp rx=0 tx=0).stop 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 shutdown_connections 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:6832/642815597,v1:172.21.15.6:6834/642815597] conn(0x557715583830 0x557715583c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557715573f10 0x557715568510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7469ffb640 0 client.5751 ms_handle_reset on v2:172.21.15.6:6832/642815597 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557715556670 0x557715569540 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 --2- 172.21.15.1:0/1908306106 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557715552fc0 0x5577155ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 >> 172.21.15.1:0/1908306106 conn(0x55771552c350 msgr2=0x5577155731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 shutdown_connections 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 1 -- 172.21.15.1:0/1908306106 wait complete. 2024-05-09T06:28:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.258+0000 7f7493fcf740 20 client.5751 trim_cache size 0 max 16384 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 5550, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 10.173528551, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 18, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 38, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 38, 2024-05-09T06:28:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 17.209054401 2024-05-09T06:28:54.248 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:28:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 msgr2=0x7f746c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 0x7f746c06e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f748c0b0640 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-05-09T06:28:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 msgr2=0x7f748c0af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 shutdown_connections 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746c06be30 0x7f746c06e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c0cf180 0x7f748c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f748c0b3300 0x7f748c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 --2- 172.21.15.1:0/1452073841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f748c06e6d0 0x7f748c0af960 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 >> 172.21.15.1:0/1452073841 conn(0x7f748c0cc4b0 msgr2=0x7f748c0b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 shutdown_connections 2024-05-09T06:28:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.259+0000 7f74911ed640 1 -- 172.21.15.1:0/1452073841 wait complete. 2024-05-09T06:28:54.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:28:54.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.374+0000 7f6fe055a640 1 Processor -- start 2024-05-09T06:28:54.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.374+0000 7f6fe055a640 1 -- start start 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fe055a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fd8071ce0 0x7f6fd80cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fe055a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd815f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fe055a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd80d04a0 con 0x7f6fd80b0080 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fe055a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd815f670 con 0x7f6fd8071ce0 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fe055a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80722c0 con 0x7f6fd815ece0 2024-05-09T06:28:54.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fddaf5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fd8071ce0 0x7f6fd80cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fde2f6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6fd80b0080 0x7f6fd815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58622/0 (socket says 172.21.15.1:58622) 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fde2f6640 1 -- 172.21.15.1:0/3811368872 learned_addr learned my addr 172.21.15.1:0/3811368872 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fdeaf7640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fddaf5640 1 -- 172.21.15.1:0/3811368872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 msgr2=0x7f6fd815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fddaf5640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fddaf5640 1 -- 172.21.15.1:0/3811368872 >> v1:172.21.15.1:6789/0 conn(0x7f6fd80b0080 legacy=0x7f6fd815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fddaf5640 1 -- 172.21.15.1:0/3811368872 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd80725a0 con 0x7f6fd8071ce0 2024-05-09T06:28:54.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.375+0000 7f6fdd2f4640 1 -- 172.21.15.1:0/3811368872 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712263605 0 0) 0x7f6fd80d04a0 con 0x7f6fd80b0080 2024-05-09T06:28:54.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.376+0000 7f6fddaf5640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80cbb60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80097a0 tx=0x7f6fc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90bd44beb120327 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.376+0000 7f6fdd2f4640 1 -- 172.21.15.1:0/3811368872 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8037070 con 0x7f6fd8071ce0 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.376+0000 7f6fdd2f4640 1 -- 172.21.15.1:0/3811368872 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc8032440 con 0x7f6fd8071ce0 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.376+0000 7f6fdd2f4640 1 -- 172.21.15.1:0/3811368872 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8032810 con 0x7f6fd8071ce0 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 -- 172.21.15.1:0/3811368872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd8071ce0 msgr2=0x7f6fd80cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80cbb60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80097a0 tx=0x7f6fc802a180 comp rx=0 tx=0).stop 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 -- 172.21.15.1:0/3811368872 shutdown_connections 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 --2- 172.21.15.1:0/3811368872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80cbb60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 -- 172.21.15.1:0/3811368872 >> 172.21.15.1:0/3811368872 conn(0x7f6fd80d0b50 msgr2=0x7f6fd80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 -- 172.21.15.1:0/3811368872 shutdown_connections 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.377+0000 7f6fe055a640 1 -- 172.21.15.1:0/3811368872 wait complete. 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.378+0000 7f6fe055a640 1 Processor -- start 2024-05-09T06:28:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.378+0000 7f6fe055a640 1 -- start start 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd80c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd815e7c0 con 0x7f6fd80b0080 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8072b60 con 0x7f6fd8071ce0 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd815ff40 con 0x7f6fd815ece0 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42052/0 (socket says 172.21.15.1:42052) 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 -- 172.21.15.1:0/1045476172 learned_addr learned my addr 172.21.15.1:0/1045476172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 -- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 msgr2=0x7f6fd80c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd80c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 -- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd8071ce0 msgr2=0x7f6fd80c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 -- 172.21.15.1:0/1045476172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd80c4f80 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fddaf5640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8035040 tx=0x7f6fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8037040 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc8030770 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd80c5260 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8030b80 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.379+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd80c5820 con 0x7f6fd80b0080 2024-05-09T06:28:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.380+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd80752b0 con 0x7f6fd80b0080 2024-05-09T06:28:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.381+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fc8044020 con 0x7f6fd80b0080 2024-05-09T06:28:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.381+0000 7f6fceffd640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 0x7f6fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.381+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6fc80ab500 con 0x7f6fd80b0080 2024-05-09T06:28:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.382+0000 7f6fde2f6640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 0x7f6fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.382+0000 7f6fde2f6640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 0x7f6fbc06e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4007fd0 tx=0x7f6fd4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.386+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fc8032e20 con 0x7f6fd80b0080 2024-05-09T06:28:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.609+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6fd80672a0 con 0x7f6fd80b0080 2024-05-09T06:28:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.609+0000 7f6fceffd640 1 -- 172.21.15.1:0/1045476172 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f6fc8079d20 con 0x7f6fd80b0080 2024-05-09T06:28:54.591 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 msgr2=0x7f6fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 0x7f6fbc06e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4007fd0 tx=0x7f6fd4007920 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 msgr2=0x7f6fd81539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8035040 tx=0x7f6fc80046c0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 shutdown_connections 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6fbc06be30 0x7f6fbc06e2f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fd815ece0 0x7f6fd80c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fd80b0080 0x7f6fd81539a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 --2- 172.21.15.1:0/1045476172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd8071ce0 0x7f6fd80c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 >> 172.21.15.1:0/1045476172 conn(0x7f6fd80d0b50 msgr2=0x7f6fd80b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 shutdown_connections 2024-05-09T06:28:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54.611+0000 7f6fe055a640 1 -- 172.21.15.1:0/1045476172 wait complete. 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:28:54.604 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:28:54.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:54.605 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:28:54.621 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:28:54.622 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.623 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:28:54.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:54.624 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:28:54.686 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:28:54.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:54.688 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:28:54.744 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:28:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:28:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:28:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:28:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:28:54.745 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:28:54.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:28:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:54.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:54.875 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:28:54.891 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:28:54.891 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:28:54.891 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:28:54.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:54.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:28:54.958 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:28:54.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:54.959 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:55.015 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:28:55.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:28:55.015 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:55.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:28:55.138 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:28:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:28:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:28:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:28:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:28:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:28:55.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.198 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:28:55.251 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:28:55.252 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:28:55.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:28:55.319 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:28:55.376 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:28:55.377 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:28:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:28:55.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.379 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:28:55.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:28:55.397+0000 7f48cb361480 -1 init, newargv = 0x562b68e42320 newargc=15 2024-05-09T06:28:55.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[52250]: starting ceph client 2024-05-09T06:28:55.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[52250]: starting fuse 2024-05-09T06:28:55.399 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:28:55.400 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:28:55.400 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:28:55.401 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:28:55.402 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:55.402 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:28:55.402 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:55.520 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.52250.asok 2024-05-09T06:28:55.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:55.531 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52250.asok 2024-05-09T06:28:55.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52250.asok status 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "52250", 2024-05-09T06:28:55.670 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 5766, 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 5766 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 133415422 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:28:55.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 133415422 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.5766 192.168.0.1:0/133415422", 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/133415422", 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 18, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 38, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:28:55.672 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:28:55.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.679 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:28:55.679 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:55.695 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:28:55.695 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:55.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.695 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:28:55.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.763 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-09T06:28:55.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.825 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-09T06:28:55.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.885 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-09T06:28:55.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:55.943 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-09T06:28:56.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:56.005 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-09T06:28:56.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:28:56.063 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./ 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./rfile 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./parentdir 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./parentdir/pfile 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./parentdir/childdir 2024-05-09T06:28:56.129 INFO:teuthology.orchestra.run.smithi001.stdout:./parentdir/childdir/cfile 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-09T06:28:56.130 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.237 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:28:56.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-09T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.354 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627778 2024-05-09T06:28:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-09T06:28:56.363 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.471 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:28:56.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.480 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.480 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.481 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.481 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.481 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-09T06:28:56.481 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.588 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628279 2024-05-09T06:28:56.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-09T06:28:56.598 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.706 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:28:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-09T06:28:56.716 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:28:56.824 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628280 2024-05-09T06:28:56.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:28:56.833 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:56.833 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:56.890 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:57.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.053+0000 7f08dd11d640 1 Processor -- start 2024-05-09T06:28:57.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.053+0000 7f08dd11d640 1 -- start start 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.053+0000 7f08dd11d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08dd11d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08d80b2a60 0x7f08d80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08dd11d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d80b5230 con 0x7f08d80b4da0 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08dd11d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f08d8071c20 con 0x7f08d80b2a60 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08dd11d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f08d806eb40 con 0x7f08d81454b0 2024-05-09T06:28:57.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08d80b2a60 0x7f08d80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08d80b2a60 0x7f08d80b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42066/0 (socket says 172.21.15.1:42066) 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 -- 172.21.15.1:0/1792277455 learned_addr learned my addr 172.21.15.1:0/1792277455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d6d9d640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 -- 172.21.15.1:0/1792277455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 msgr2=0x7f08d80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 -- 172.21.15.1:0/1792277455 >> v1:172.21.15.6:6789/0 conn(0x7f08d80b4da0 legacy=0x7f08d815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.054+0000 7f08d659c640 1 -- 172.21.15.1:0/1792277455 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d806ee20 con 0x7f08d80b2a60 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08d659c640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80b4860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009870 tx=0x7f08c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3479241728b971d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08d5d9b640 1 -- 172.21.15.1:0/1792277455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0037070 con 0x7f08d80b2a60 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08d5d9b640 1 -- 172.21.15.1:0/1792277455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0032440 con 0x7f08d80b2a60 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08d5d9b640 1 -- 172.21.15.1:0/1792277455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0032810 con 0x7f08d80b2a60 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08dd11d640 1 -- 172.21.15.1:0/1792277455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 msgr2=0x7f08d80b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.055+0000 7f08dd11d640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80b4860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009870 tx=0x7f08c002a270 comp rx=0 tx=0).stop 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 -- 172.21.15.1:0/1792277455 shutdown_connections 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80b4860 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 --2- 172.21.15.1:0/1792277455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 -- 172.21.15.1:0/1792277455 >> 172.21.15.1:0/1792277455 conn(0x7f08d80cc4b0 msgr2=0x7f08d80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 -- 172.21.15.1:0/1792277455 shutdown_connections 2024-05-09T06:28:57.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.056+0000 7f08dd11d640 1 -- 172.21.15.1:0/1792277455 wait complete. 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.057+0000 7f08dd11d640 1 Processor -- start 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.057+0000 7f08dd11d640 1 -- start start 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08d80b4da0 0x7f08d815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08d8145890 con 0x7f08d80b2a60 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f08d806dbc0 con 0x7f08d80b4da0 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08dd11d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f08d806e0e0 con 0x7f08d81454b0 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42068/0 (socket says 172.21.15.1:42068) 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 -- 172.21.15.1:0/4118276880 learned_addr learned my addr 172.21.15.1:0/4118276880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d659c640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08d80b4da0 0x7f08d815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d759e640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 -- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 msgr2=0x7f08d80ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 -- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08d80b4da0 msgr2=0x7f08d815b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08d80b4da0 0x7f08d815b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.058+0000 7f08d6d9d640 1 -- 172.21.15.1:0/4118276880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d80aea60 con 0x7f08d80b2a60 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08d6d9d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08d000efc0 tx=0x7f08d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0018020 con 0x7f08d80b2a60 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0014480 con 0x7f08d80b2a60 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d80ad480 con 0x7f08d80b2a60 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0014850 con 0x7f08d80b2a60 2024-05-09T06:28:57.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.059+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d80ada40 con 0x7f08d80b2a60 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.060+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d80752b0 con 0x7f08d80b2a60 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.060+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08d0005d10 con 0x7f08d80b2a60 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.061+0000 7f08cf7fe640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 0x7f08b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.061+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f08d008b9a0 con 0x7f08d80b2a60 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.061+0000 7f08d659c640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 0x7f08b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.061+0000 7f08d659c640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 0x7f08b806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08c0034000 tx=0x7f08c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.065+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08d0022070 con 0x7f08d80b2a60 2024-05-09T06:28:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.250+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08d8072fa0 con 0x7f08d80b2a60 2024-05-09T06:28:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.251+0000 7f08cf7fe640 1 -- 172.21.15.1:0/4118276880 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16810 (secure 0 0 0) 0x7f08d0010040 con 0x7f08d80b2a60 2024-05-09T06:28:57.233 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:57.234 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:45.085064+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 msgr2=0x7f08b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 0x7f08b806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08c0034000 tx=0x7f08c002e000 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 msgr2=0x7f08d80ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08d000efc0 tx=0x7f08d000c490 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 shutdown_connections 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f08b806be30 0x7f08b806e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f08d81454b0 0x7f08d80ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08d80b4da0 0x7f08d815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 --2- 172.21.15.1:0/4118276880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d80b2a60 0x7f08d80ae520 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 >> 172.21.15.1:0/4118276880 conn(0x7f08d80cc4b0 msgr2=0x7f08d80af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 shutdown_connections 2024-05-09T06:28:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.253+0000 7f08dd11d640 1 -- 172.21.15.1:0/4118276880 wait complete. 2024-05-09T06:28:57.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:28:41.915369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:28:42.435634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:28:57.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:57.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.372+0000 7f74797a9640 1 Processor -- start 2024-05-09T06:28:57.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.372+0000 7f74797a9640 1 -- start start 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f74797a9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7474071fd0 0x7f747406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f74797a9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747415b420 con 0x7f7474158a50 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f74797a9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747415b6e0 con 0x7f747415c4a0 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f74797a9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7474158e30 con 0x7f7474071fd0 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f7472ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7474158a50 0x7f74740718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58636/0 (socket says 172.21.15.1:58636) 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f7472ffd640 1 -- 172.21.15.1:0/4241906644 learned_addr learned my addr 172.21.15.1:0/4241906644 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860129401 0 0) 0x7f747415b420 con 0x7f7474158a50 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7460003520 con 0x7f7474158a50 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.373+0000 7f74727fc640 1 --2- 172.21.15.1:0/4241906644 >> v2:172.21.15.6:3300/0 conn(0x7f7474071fd0 0x7f747406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 50041582 0 0) 0x7f7460003520 con 0x7f7474158a50 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 >> v2:172.21.15.6:3300/0 conn(0x7f7474071fd0 msgr2=0x7f747406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 --2- 172.21.15.1:0/4241906644 >> v2:172.21.15.6:3300/0 conn(0x7f7474071fd0 0x7f747406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 >> v1:172.21.15.6:6789/0 conn(0x7f747415c4a0 legacy=0x7f747415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747415b9c0 con 0x7f7474158a50 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f746c001670 con 0x7f7474158a50 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 >> v1:172.21.15.1:6789/0 conn(0x7f7474158a50 legacy=0x7f74740718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74600039b0 0x7f7460003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 0x7f7460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74600051c0 0x7f746000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7474158e30 con 0x7f7460004800 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f746000c060 con 0x7f74600039b0 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f746000c260 con 0x7f74600051c0 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 0x7f7460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7472ffd640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74600039b0 0x7f7460003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74737fe640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74600051c0 0x7f746000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 -- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74600051c0 msgr2=0x7f746000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74600051c0 0x7f746000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 -- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74600039b0 msgr2=0x7f7460003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74600039b0 0x7f7460003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74727fc640 1 -- 172.21.15.1:0/4241906644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7460003520 con 0x7f7460004800 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f7472ffd640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74600039b0 0x7f7460003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74737fe640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74600051c0 0x7f746000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.374+0000 7f74737fe640 1 -- 172.21.15.1:0/4241906644 reap_dead start 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f74727fc640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 0x7f7460004c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ac0 tx=0x7f7460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e27aced84f6ec54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c036070 con 0x7f7460004800 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c031400 con 0x7f7460004800 2024-05-09T06:28:57.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f7471ffb640 1 -- 172.21.15.1:0/4241906644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c009690 con 0x7f7460004800 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f74797a9640 1 -- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 msgr2=0x7f7460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.375+0000 7f74797a9640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 0x7f7460004c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ac0 tx=0x7f7460005830 comp rx=0 tx=0).stop 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.376+0000 7f74797a9640 1 -- 172.21.15.1:0/4241906644 shutdown_connections 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.376+0000 7f74797a9640 1 --2- 172.21.15.1:0/4241906644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7460004800 0x7f7460004c80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.376+0000 7f74797a9640 1 -- 172.21.15.1:0/4241906644 >> 172.21.15.1:0/4241906644 conn(0x7f74740cc4d0 msgr2=0x7f74740cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.376+0000 7f74797a9640 1 -- 172.21.15.1:0/4241906644 shutdown_connections 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.376+0000 7f74797a9640 1 -- 172.21.15.1:0/4241906644 wait complete. 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.377+0000 7f74797a9640 1 Processor -- start 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.377+0000 7f74797a9640 1 -- start start 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7460004800 0x7f747415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7474145110 0x7f747406e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f747415bf80 con 0x7f74741f3850 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7474072480 con 0x7f7460004800 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74797a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7474070ea0 con 0x7f7474145110 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42078/0 (socket says 172.21.15.1:42078) 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 -- 172.21.15.1:0/2944601566 learned_addr learned my addr 172.21.15.1:0/2944601566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74737fe640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7474145110 0x7f747406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f7472ffd640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7460004800 0x7f747415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 -- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7474145110 msgr2=0x7f747406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7474145110 0x7f747406e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 -- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7460004800 msgr2=0x7f747415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7460004800 0x7f747415e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 -- 172.21.15.1:0/2944601566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747415ef70 con 0x7f74741f3850 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.378+0000 7f74727fc640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f745c009be0 tx=0x7f745c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.379+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c03f020 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.379+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c037040 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.379+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747415f250 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.379+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c036070 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.379+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747415a7e0 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.380+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74740752b0 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.380+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f745c042020 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.381+0000 7f746b7fe640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 0x7f745406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.381+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f745c0ab080 con 0x7f74741f3850 2024-05-09T06:28:57.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.381+0000 7f7472ffd640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 0x7f745406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.381+0000 7f7472ffd640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 0x7f745406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f746c001eb0 tx=0x7f746c001af0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.385+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f745c049030 con 0x7f74741f3850 2024-05-09T06:28:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.567+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7474071960 con 0x7f74741f3850 2024-05-09T06:28:57.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.567+0000 7f746b7fe640 1 -- 172.21.15.1:0/2944601566 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f745c0799f0 con 0x7f74741f3850 2024-05-09T06:28:57.549 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 msgr2=0x7f745406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 0x7f745406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f746c001eb0 tx=0x7f746c001af0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 msgr2=0x7f747415a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f745c009be0 tx=0x7f745c004730 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 shutdown_connections 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f745406be30 0x7f745406e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7474145110 0x7f747406e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74741f3850 0x7f747415a2a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 --2- 172.21.15.1:0/2944601566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7460004800 0x7f747415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 >> 172.21.15.1:0/2944601566 conn(0x7f74740cc4d0 msgr2=0x7f74740ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 shutdown_connections 2024-05-09T06:28:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.569+0000 7f74797a9640 1 -- 172.21.15.1:0/2944601566 wait complete. 2024-05-09T06:28:57.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:57.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.687+0000 7f76d323f640 1 Processor -- start 2024-05-09T06:28:57.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.687+0000 7f76d323f640 1 -- start start 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d323f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 0x7f76cc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d323f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d323f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76cc15e900 con 0x7f76cc15e390 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d323f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f76cc0b1b90 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d323f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76cc15ebe0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42080/0 (socket says 172.21.15.1:42080) 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 -- 172.21.15.1:0/2100732336 learned_addr learned my addr 172.21.15.1:0/2100732336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76cbfff640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 0x7f76cc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 -- 172.21.15.1:0/2100732336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 msgr2=0x7f76cc0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 0x7f76cc0b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 -- 172.21.15.1:0/2100732336 >> v1:172.21.15.6:6789/0 conn(0x7f76cc15e390 legacy=0x7f76cc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76d17dc640 1 -- 172.21.15.1:0/2100732336 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc15eec0 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.688+0000 7f76cbfff640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 0x7f76cc0b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.689+0000 7f76d17dc640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009e50 tx=0x7f76c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9333be542efe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.689+0000 7f76cb7fe640 1 -- 172.21.15.1:0/2100732336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4036070 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.689+0000 7f76cb7fe640 1 -- 172.21.15.1:0/2100732336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4031600 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.689+0000 7f76cb7fe640 1 -- 172.21.15.1:0/2100732336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40319d0 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 -- 172.21.15.1:0/2100732336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0cc8e0 msgr2=0x7f76cc0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009e50 tx=0x7f76c402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 -- 172.21.15.1:0/2100732336 shutdown_connections 2024-05-09T06:28:57.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0ccd50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 --2- 172.21.15.1:0/2100732336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b11f0 0x7f76cc0b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 -- 172.21.15.1:0/2100732336 >> 172.21.15.1:0/2100732336 conn(0x7f76cc0b1da0 msgr2=0x7f76cc0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 -- 172.21.15.1:0/2100732336 shutdown_connections 2024-05-09T06:28:57.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.690+0000 7f76d323f640 1 -- 172.21.15.1:0/2100732336 wait complete. 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.691+0000 7f76d323f640 1 Processor -- start 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.691+0000 7f76d323f640 1 -- start start 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15e390 0x7f76cc0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc162ba0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc15f6c0 con 0x7f76cc0cc8e0 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d323f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76cc0cffb0 con 0x7f76cc15e390 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57770/0 (socket says 172.21.15.1:57770) 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 -- 172.21.15.1:0/806051699 learned_addr learned my addr 172.21.15.1:0/806051699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d17dc640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15e390 0x7f76cc0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76cbfff640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 -- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15e390 msgr2=0x7f76cc0d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15e390 0x7f76cc0d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 -- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 msgr2=0x7f76cc0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 -- 172.21.15.1:0/806051699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc0b5530 con 0x7f76cc0b11f0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76cbfff640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.692+0000 7f76d0fdb640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76c000b420 tx=0x7f76c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.693+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011830 con 0x7f76cc0b11f0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.693+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c0016070 con 0x7f76cc0b11f0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.693+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc0b2ff0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.693+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011c10 con 0x7f76cc0b11f0 2024-05-09T06:28:57.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.693+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc0b35b0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.694+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76cc0752b0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.694+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76c001b460 con 0x7f76cc0b11f0 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.695+0000 7f76c97fa640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 0x7f76b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.695+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f76c008aa80 con 0x7f76cc0b11f0 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.695+0000 7f76cbfff640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 0x7f76b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.695+0000 7f76cbfff640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 0x7f76b006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f76b8006fd0 tx=0x7f76b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.699+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76c0020020 con 0x7f76cc0b11f0 2024-05-09T06:28:57.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.880+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76cc0b2ae0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.881+0000 7f76c97fa640 1 -- 172.21.15.1:0/806051699 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f76c00592a0 con 0x7f76cc0b11f0 2024-05-09T06:28:57.863 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 msgr2=0x7f76b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 0x7f76b006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f76b8006fd0 tx=0x7f76b8008040 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 msgr2=0x7f76cc0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76c000b420 tx=0x7f76c000b8e0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 shutdown_connections 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76b006be30 0x7f76b006e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15e390 0x7f76cc0d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0cc8e0 0x7f76cc0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 --2- 172.21.15.1:0/806051699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc0b11f0 0x7f76cc0b4ff0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 >> 172.21.15.1:0/806051699 conn(0x7f76cc0b1da0 msgr2=0x7f76cc15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 shutdown_connections 2024-05-09T06:28:57.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:57.883+0000 7f76d323f640 1 -- 172.21.15.1:0/806051699 wait complete. 2024-05-09T06:28:57.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.000+0000 7f5d6ecfc640 1 Processor -- start 2024-05-09T06:28:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.000+0000 7f5d6ecfc640 1 -- start start 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6ecfc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d6815cfb0 0x7f5d68070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6ecfc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6806cf40 con 0x7f5d6806dd00 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6ecfc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6806d220 con 0x7f5d6815c4c0 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6ecfc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5d68156980 con 0x7f5d6815cfb0 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6d299640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d6815cfb0 0x7f5d68070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d5ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5d6806dd00 0x7f5d6815c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59326/0 (socket says 172.21.15.1:59326) 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d5ffff640 1 -- 172.21.15.1:0/3434364348 learned_addr learned my addr 172.21.15.1:0/3434364348 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6d299640 1 -- 172.21.15.1:0/3434364348 >> v1:172.21.15.6:6789/0 conn(0x7f5d6815c4c0 legacy=0x7f5d680b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6d299640 1 -- 172.21.15.1:0/3434364348 >> v1:172.21.15.1:6789/0 conn(0x7f5d6806dd00 legacy=0x7f5d6815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d6d299640 1 -- 172.21.15.1:0/3434364348 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6806d500 con 0x7f5d6815cfb0 2024-05-09T06:28:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.001+0000 7f5d5f7fe640 1 -- 172.21.15.1:0/3434364348 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225963827 0 0) 0x7f5d6806cf40 con 0x7f5d6806dd00 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.002+0000 7f5d6d299640 1 --2- 172.21.15.1:0/3434364348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6815cfb0 0x7f5d68070c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009d90 tx=0x7f5d6002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a8d31844d34c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.002+0000 7f5d5f7fe640 1 -- 172.21.15.1:0/3434364348 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60031600 con 0x7f5d6815cfb0 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.002+0000 7f5d5f7fe640 1 -- 172.21.15.1:0/3434364348 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d600317a0 con 0x7f5d6815cfb0 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.002+0000 7f5d5f7fe640 1 -- 172.21.15.1:0/3434364348 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60031b70 con 0x7f5d6815cfb0 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/3434364348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6815cfb0 msgr2=0x7f5d68070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/3434364348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6815cfb0 0x7f5d68070c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009d90 tx=0x7f5d6002a610 comp rx=0 tx=0).stop 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/3434364348 shutdown_connections 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/3434364348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6815cfb0 0x7f5d68070c10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/3434364348 >> 172.21.15.1:0/3434364348 conn(0x7f5d680cc4d0 msgr2=0x7f5d680cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/3434364348 shutdown_connections 2024-05-09T06:28:57.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.003+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/3434364348 wait complete. 2024-05-09T06:28:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.004+0000 7f5d6ecfc640 1 Processor -- start 2024-05-09T06:28:57.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.004+0000 7f5d6ecfc640 1 -- start start 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d6815c4c0 0x7f5d6815af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 0x7f5d6815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6806d870 con 0x7f5d6806dd00 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6806e180 con 0x7f5d6815cfb0 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ecfc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d680b2c40 con 0x7f5d6815c4c0 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57790/0 (socket says 172.21.15.1:57790) 2024-05-09T06:28:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 -- 172.21.15.1:0/273161448 learned_addr learned my addr 172.21.15.1:0/273161448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d5ffff640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d6815c4c0 0x7f5d6815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6d299640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 0x7f5d6815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 -- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d6815c4c0 msgr2=0x7f5d6815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d6815c4c0 0x7f5d6815af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 -- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 msgr2=0x7f5d6815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 0x7f5d6815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6ca98640 1 -- 172.21.15.1:0/273161448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68158880 con 0x7f5d6806dd00 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.005+0000 7f5d6d299640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 0x7f5d6815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d6ca98640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c430 tx=0x7f5d6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64011790 con 0x7f5d6806dd00 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d64017070 con 0x7f5d6806dd00 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64011bb0 con 0x7f5d6806dd00 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d68158b60 con 0x7f5d6806dd00 2024-05-09T06:28:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.006+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d68159120 con 0x7f5d6806dd00 2024-05-09T06:28:57.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.007+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d680752b0 con 0x7f5d6806dd00 2024-05-09T06:28:57.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.007+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d6401c460 con 0x7f5d6806dd00 2024-05-09T06:28:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.008+0000 7f5d5d7fa640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.008+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5d6408ae80 con 0x7f5d6806dd00 2024-05-09T06:28:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.008+0000 7f5d5ffff640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.008+0000 7f5d5ffff640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d681429c0 tx=0x7f5d58007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.012+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d64059730 con 0x7f5d6806dd00 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.196+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5d68142a10 con 0x7f5d6806dd00 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.196+0000 7f5d5d7fa640 1 -- 172.21.15.1:0/273161448 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f5d64059030 con 0x7f5d6806dd00 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:58.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 msgr2=0x7f5d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d681429c0 tx=0x7f5d58007b50 comp rx=0 tx=0).stop 2024-05-09T06:28:58.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 msgr2=0x7f5d6815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c430 tx=0x7f5d6400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 shutdown_connections 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d6815cfb0 0x7f5d6815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d6815c4c0 0x7f5d6815af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 --2- 172.21.15.1:0/273161448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d6806dd00 0x7f5d6815aa30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 >> 172.21.15.1:0/273161448 conn(0x7f5d680cc4d0 msgr2=0x7f5d680b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 shutdown_connections 2024-05-09T06:28:58.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.198+0000 7f5d6ecfc640 1 -- 172.21.15.1:0/273161448 wait complete. 2024-05-09T06:28:58.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:58.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.314+0000 7f598e941640 1 Processor -- start 2024-05-09T06:28:58.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.314+0000 7f598e941640 1 -- start start 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598e941640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5988163c70 0x7f5988166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598e941640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59880ce0e0 con 0x7f598806e6d0 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598e941640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59880ce3c0 con 0x7f59880cf180 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598e941640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f59880ce6a0 con 0x7f5988163c70 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598cede640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5988163c70 0x7f5988166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f5987fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f598806e6d0 0x7f598806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59340/0 (socket says 172.21.15.1:59340) 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f5987fff640 1 -- 172.21.15.1:0/1160574431 learned_addr learned my addr 172.21.15.1:0/1160574431 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598cede640 1 -- 172.21.15.1:0/1160574431 >> v1:172.21.15.6:6789/0 conn(0x7f59880cf180 legacy=0x7f59880cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598cede640 1 -- 172.21.15.1:0/1160574431 >> v1:172.21.15.1:6789/0 conn(0x7f598806e6d0 legacy=0x7f598806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f598cede640 1 -- 172.21.15.1:0/1160574431 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59880ce980 con 0x7f5988163c70 2024-05-09T06:28:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.315+0000 7f5986ffd640 1 -- 172.21.15.1:0/1160574431 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872642344 0 0) 0x7f59880ce0e0 con 0x7f598806e6d0 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.316+0000 7f598cede640 1 --2- 172.21.15.1:0/1160574431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5988163c70 0x7f5988166060 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5980009d90 tx=0x7f598002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faaf9ea2551ac9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.316+0000 7f5986ffd640 1 -- 172.21.15.1:0/1160574431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980007740 con 0x7f5988163c70 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.316+0000 7f5986ffd640 1 -- 172.21.15.1:0/1160574431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59800078e0 con 0x7f5988163c70 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.316+0000 7f5986ffd640 1 -- 172.21.15.1:0/1160574431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980007cb0 con 0x7f5988163c70 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 -- 172.21.15.1:0/1160574431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5988163c70 msgr2=0x7f5988166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 --2- 172.21.15.1:0/1160574431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5988163c70 0x7f5988166060 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5980009d90 tx=0x7f598002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 -- 172.21.15.1:0/1160574431 shutdown_connections 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 --2- 172.21.15.1:0/1160574431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5988163c70 0x7f5988166060 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 -- 172.21.15.1:0/1160574431 >> 172.21.15.1:0/1160574431 conn(0x7f59880cc4b0 msgr2=0x7f59880cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 -- 172.21.15.1:0/1160574431 shutdown_connections 2024-05-09T06:28:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.317+0000 7f598e941640 1 -- 172.21.15.1:0/1160574431 wait complete. 2024-05-09T06:28:58.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.318+0000 7f598e941640 1 Processor -- start 2024-05-09T06:28:58.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.318+0000 7f598e941640 1 -- start start 2024-05-09T06:28:58.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f598806e6d0 0x7f5988165a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5988163c70 0x7f5988156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59881447c0 con 0x7f59880cf180 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5988070ed0 con 0x7f598806e6d0 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598e941640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5988144940 con 0x7f5988163c70 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57794/0 (socket says 172.21.15.1:57794) 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 -- 172.21.15.1:0/2911690870 learned_addr learned my addr 172.21.15.1:0/2911690870 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f5987fff640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f598806e6d0 0x7f5988165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f598cede640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5988163c70 0x7f5988156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 -- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5988163c70 msgr2=0x7f5988156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5988163c70 0x7f5988156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 -- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f598806e6d0 msgr2=0x7f5988165a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f598806e6d0 0x7f5988165a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 -- 172.21.15.1:0/2911690870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5988165fb0 con 0x7f59880cf180 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f59877fe640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c430 tx=0x7f597c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.319+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c011790 con 0x7f59880cf180 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.320+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c017070 con 0x7f59880cf180 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.320+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5988138e30 con 0x7f59880cf180 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.320+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c011b70 con 0x7f59880cf180 2024-05-09T06:28:58.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.320+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5988157d50 con 0x7f59880cf180 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.321+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59880752b0 con 0x7f59880cf180 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.321+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597c011d10 con 0x7f59880cf180 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.321+0000 7f5984ff9640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 0x7f597406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.322+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f597c08ad50 con 0x7f59880cf180 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.322+0000 7f5987fff640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 0x7f597406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.322+0000 7f5987fff640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 0x7f597406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5978007fd0 tx=0x7f5978007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.326+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f597c08e030 con 0x7f59880cf180 2024-05-09T06:28:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.523+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f598806cf50 con 0x7f59880cf180 2024-05-09T06:28:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.523+0000 7f5984ff9640 1 -- 172.21.15.1:0/2911690870 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f597c058f20 con 0x7f59880cf180 2024-05-09T06:28:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:28:58.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 msgr2=0x7f597406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 0x7f597406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5978007fd0 tx=0x7f5978007de0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 msgr2=0x7f5988167770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c430 tx=0x7f597c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 shutdown_connections 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f597406be30 0x7f597406e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5988163c70 0x7f5988156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59880cf180 0x7f5988167770 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 --2- 172.21.15.1:0/2911690870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f598806e6d0 0x7f5988165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 >> 172.21.15.1:0/2911690870 conn(0x7f59880cc4b0 msgr2=0x7f598806d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 shutdown_connections 2024-05-09T06:28:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.525+0000 7f598e941640 1 -- 172.21.15.1:0/2911690870 wait complete. 2024-05-09T06:28:58.518 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:28:58.518 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:28:58.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:28:58.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.696+0000 7f2b5c4a5640 1 Processor -- start 2024-05-09T06:28:58.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 -- start start 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b54161950 0x7f2b540d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b540d4940 0x7f2b540d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2b540d0f80 0x7f2b540cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540d4dd0 con 0x7f2b540d4940 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540b33c0 con 0x7f2b540d0f80 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.697+0000 7f2b5c4a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2b540b3660 con 0x7f2b54161950 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b540d4940 0x7f2b540d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b540d4940 0x7f2b540d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57806/0 (socket says 172.21.15.1:57806) 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 -- 172.21.15.1:0/1368086468 learned_addr learned my addr 172.21.15.1:0/1368086468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b5aa42640 1 --2- 172.21.15.1:0/1368086468 >> v2:172.21.15.6:3300/0 conn(0x7f2b540d0f80 0x7f2b540cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b5a241640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b54161950 0x7f2b540d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 -- 172.21.15.1:0/1368086468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b54161950 msgr2=0x7f2b540d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b54161950 0x7f2b540d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 -- 172.21.15.1:0/1368086468 >> v2:172.21.15.6:3300/0 conn(0x7f2b540d0f80 msgr2=0x7f2b540cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 --2- 172.21.15.1:0/1368086468 >> v2:172.21.15.6:3300/0 conn(0x7f2b540d0f80 0x7f2b540cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.698+0000 7f2b59a40640 1 -- 172.21.15.1:0/1368086468 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b540b3940 con 0x7f2b540d4940 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b59a40640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d4940 0x7f2b540d2940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2b5408c710 tx=0x7f2b4402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2f30c9359f3de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b5923f640 1 -- 172.21.15.1:0/1368086468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44037070 con 0x7f2b540d4940 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b5923f640 1 -- 172.21.15.1:0/1368086468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b44032440 con 0x7f2b540d4940 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b5923f640 1 -- 172.21.15.1:0/1368086468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44032810 con 0x7f2b540d4940 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/1368086468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d4940 msgr2=0x7f2b540d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.699+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d4940 0x7f2b540d2940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2b5408c710 tx=0x7f2b4402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/1368086468 shutdown_connections 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/1368086468 >> v2:172.21.15.6:3300/0 conn(0x7f2b540d0f80 0x7f2b540cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d4940 0x7f2b540d2940 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/1368086468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b54161950 0x7f2b540d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/1368086468 >> 172.21.15.1:0/1368086468 conn(0x7f2b540d3e20 msgr2=0x7f2b540d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/1368086468 shutdown_connections 2024-05-09T06:28:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.700+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/1368086468 wait complete. 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.701+0000 7f2b5c4a5640 1 Processor -- start 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.701+0000 7f2b5c4a5640 1 -- start start 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b54161950 0x7f2b540cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b540cc010 0x7f2b540cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d1d80 con 0x7f2b540d0f80 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d1c00 con 0x7f2b54161950 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2b540d1f00 con 0x7f2b540cc010 2024-05-09T06:28:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57822/0 (socket says 172.21.15.1:57822) 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 -- 172.21.15.1:0/4053916112 learned_addr learned my addr 172.21.15.1:0/4053916112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5aa42640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b540cc010 0x7f2b540cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 -- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b540cc010 msgr2=0x7f2b540cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b540cc010 0x7f2b540cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 -- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b54161950 msgr2=0x7f2b540cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b54161950 0x7f2b540cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 -- 172.21.15.1:0/4053916112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b541517d0 con 0x7f2b540d0f80 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5a241640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000e8a0 tx=0x7f2b5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50017070 con 0x7f2b540d0f80 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5000c940 con 0x7f2b540d0f80 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b540cc4b0 con 0x7f2b540d0f80 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000cd30 con 0x7f2b540d0f80 2024-05-09T06:28:58.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.702+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b540cca70 con 0x7f2b540d0f80 2024-05-09T06:28:58.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.703+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b540752b0 con 0x7f2b540d0f80 2024-05-09T06:28:58.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.704+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b5001a070 con 0x7f2b540d0f80 2024-05-09T06:28:58.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.704+0000 7f2b4affd640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.704+0000 7f2b59a40640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.704+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f2b5008b2a0 con 0x7f2b540d0f80 2024-05-09T06:28:58.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.705+0000 7f2b59a40640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 0x7f2b3806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034000 tx=0x7f2b4402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.709+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b50059a20 con 0x7f2b540d0f80 2024-05-09T06:28:58.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.892+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b540d21e0 con 0x7f2b540d0f80 2024-05-09T06:28:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.893+0000 7f2b4affd640 1 -- 172.21.15.1:0/4053916112 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16810 (secure 0 0 0) 0x7f2b50059320 con 0x7f2b540d0f80 2024-05-09T06:28:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:28:58.875 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:28:45.085064+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:28:58.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 msgr2=0x7f2b3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 0x7f2b3806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034000 tx=0x7f2b4402c000 comp rx=0 tx=0).stop 2024-05-09T06:28:58.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 msgr2=0x7f2b54151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000e8a0 tx=0x7f2b5000ed60 comp rx=0 tx=0).stop 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 shutdown_connections 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b540cc010 0x7f2b540cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b54161950 0x7f2b540cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 --2- 172.21.15.1:0/4053916112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b540d0f80 0x7f2b54151290 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 >> 172.21.15.1:0/4053916112 conn(0x7f2b540d3e20 msgr2=0x7f2b540b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 shutdown_connections 2024-05-09T06:28:58.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:58.895+0000 7f2b5c4a5640 1 -- 172.21.15.1:0/4053916112 wait complete. 2024-05-09T06:28:58.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:28:41.915369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:28:42.435634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:28:58.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:28:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.013+0000 7f9833314640 1 Processor -- start 2024-05-09T06:28:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.013+0000 7f9833314640 1 -- start start 2024-05-09T06:28:58.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f9833314640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f982c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f9833314640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f982c06ede0 0x7f982c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f9833314640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c15b420 con 0x7f982c15c4a0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f9833314640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f982c158e30 con 0x7f982c06ede0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f9833314640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f982c072480 con 0x7f982c158a50 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f98308af640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f982c15c4a0 0x7f982c06e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59352/0 (socket says 172.21.15.1:59352) 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f98308af640 1 -- 172.21.15.1:0/3556367760 learned_addr learned my addr 172.21.15.1:0/3556367760 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f98318b1640 1 --2- 172.21.15.1:0/3556367760 >> v2:172.21.15.6:3300/0 conn(0x7f982c06ede0 0x7f982c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.014+0000 7f98310b0640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f982c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146825925 0 0) 0x7f982c15b420 con 0x7f982c15c4a0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9814003520 con 0x7f982c15c4a0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2660966935 0 0) 0x7f9814003520 con 0x7f982c15c4a0 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 msgr2=0x7f982c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f982c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 >> v2:172.21.15.6:3300/0 conn(0x7f982c06ede0 msgr2=0x7f982c071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 --2- 172.21.15.1:0/3556367760 >> v2:172.21.15.6:3300/0 conn(0x7f982c06ede0 0x7f982c071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c15b6e0 con 0x7f982c15c4a0 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f9820001670 con 0x7f982c15c4a0 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 >> v1:172.21.15.1:6789/0 conn(0x7f982c15c4a0 legacy=0x7f982c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f98140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814003f30 0x7f9814004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9814004d40 0x7f98140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f982c158e30 con 0x7f9814004d40 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f982c072480 con 0x7f9814003f30 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f981400b9c0 con 0x7f982c158a50 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9814004d40 0x7f98140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98310b0640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f98140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98308af640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814003f30 0x7f9814004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 -- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 msgr2=0x7f98140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f98140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 -- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814003f30 msgr2=0x7f9814004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9814003f30 0x7f9814004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98318b1640 1 -- 172.21.15.1:0/3556367760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9814003520 con 0x7f9814004d40 2024-05-09T06:28:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.015+0000 7f98310b0640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c158a50 0x7f98140039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.016+0000 7f98318b1640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9814004d40 0x7f98140051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f981c009fc0 tx=0x7f9814005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e083b203ca2cf7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.016+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c009c10 con 0x7f9814004d40 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.016+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c036070 con 0x7f9814004d40 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.016+0000 7f982bfff640 1 -- 172.21.15.1:0/3556367760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c02d070 con 0x7f9814004d40 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 -- 172.21.15.1:0/3556367760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9814004d40 msgr2=0x7f98140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 --2- 172.21.15.1:0/3556367760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9814004d40 0x7f98140051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f981c009fc0 tx=0x7f9814005830 comp rx=0 tx=0).stop 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f98318b1640 1 -- 172.21.15.1:0/3556367760 reap_dead start 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 -- 172.21.15.1:0/3556367760 shutdown_connections 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 -- 172.21.15.1:0/3556367760 >> 172.21.15.1:0/3556367760 conn(0x7f982c0cc4d0 msgr2=0x7f982c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 -- 172.21.15.1:0/3556367760 shutdown_connections 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.017+0000 7f9833314640 1 -- 172.21.15.1:0/3556367760 wait complete. 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.018+0000 7f9833314640 1 Processor -- start 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.018+0000 7f9833314640 1 -- start start 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c156900 0x7f982c144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f982c0b59f0 0x7f982c144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f982c15bf80 con 0x7f982c0b5610 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f982c15bca0 con 0x7f982c0b59f0 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f9833314640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f982c070ea0 con 0x7f982c156900 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57834/0 (socket says 172.21.15.1:57834) 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 -- 172.21.15.1:0/3875668482 learned_addr learned my addr 172.21.15.1:0/3875668482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98308af640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c156900 0x7f982c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98318b1640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f982c0b59f0 0x7f982c144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 -- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c156900 msgr2=0x7f982c144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c156900 0x7f982c144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 -- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f982c0b59f0 msgr2=0x7f982c144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f982c0b59f0 0x7f982c144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 -- 172.21.15.1:0/3875668482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c1517f0 con 0x7f982c0b5610 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.019+0000 7f98310b0640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9824011700 tx=0x7f9824011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.020+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982400f9c0 con 0x7f982c0b5610 2024-05-09T06:28:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.020+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982400fb60 con 0x7f982c0b5610 2024-05-09T06:28:59.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.020+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824015460 con 0x7f982c0b5610 2024-05-09T06:28:59.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.020+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c151ad0 con 0x7f982c0b5610 2024-05-09T06:28:59.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.020+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c152090 con 0x7f982c0b5610 2024-05-09T06:28:59.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.021+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0752b0 con 0x7f982c0b5610 2024-05-09T06:28:59.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.021+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9824015600 con 0x7f982c0b5610 2024-05-09T06:28:59.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.022+0000 7f9829ffb640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 0x7f981806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.022+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f982408ba60 con 0x7f982c0b5610 2024-05-09T06:28:59.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.022+0000 7f98308af640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 0x7f981806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.022+0000 7f98308af640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 0x7f981806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9820000fb0 tx=0x7f9820009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.026+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9824019030 con 0x7f982c0b5610 2024-05-09T06:28:59.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.212+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f982c0b2110 con 0x7f982c0b5610 2024-05-09T06:28:59.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.212+0000 7f9829ffb640 1 -- 172.21.15.1:0/3875668482 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f982405a260 con 0x7f982c0b5610 2024-05-09T06:28:59.194 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:28:59.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 msgr2=0x7f981806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 0x7f981806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9820000fb0 tx=0x7f9820009040 comp rx=0 tx=0).stop 2024-05-09T06:28:59.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 msgr2=0x7f982c1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9824011700 tx=0x7f9824011bc0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 shutdown_connections 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f981806be30 0x7f981806e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f982c0b59f0 0x7f982c144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f982c156900 0x7f982c144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 --2- 172.21.15.1:0/3875668482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0b5610 0x7f982c1563c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 >> 172.21.15.1:0/3875668482 conn(0x7f982c0cc4d0 msgr2=0x7f982c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 shutdown_connections 2024-05-09T06:28:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.214+0000 7f9833314640 1 -- 172.21.15.1:0/3875668482 wait complete. 2024-05-09T06:28:59.207 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:28:59.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.331+0000 7f374f2cd640 1 Processor -- start 2024-05-09T06:28:59.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.331+0000 7f374f2cd640 1 -- start start 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374f2cd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3748158a50 0x7f37480718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374f2cd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374815b420 con 0x7f374815c4a0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374f2cd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374815b6e0 con 0x7f374806ede0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374f2cd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3748158e30 con 0x7f3748158a50 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374c868640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f374815c4a0 0x7f374806e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59362/0 (socket says 172.21.15.1:59362) 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374c868640 1 -- 172.21.15.1:0/3151638506 learned_addr learned my addr 172.21.15.1:0/3151638506 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f374d069640 1 --2- 172.21.15.1:0/3151638506 >> v2:172.21.15.6:3300/0 conn(0x7f3748158a50 0x7f37480718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390263470 0 0) 0x7f374815b420 con 0x7f374815c4a0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.332+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3730003520 con 0x7f374815c4a0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2281838447 0 0) 0x7f3730003520 con 0x7f374815c4a0 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 >> v2:172.21.15.6:3300/0 conn(0x7f3748158a50 msgr2=0x7f37480718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 --2- 172.21.15.1:0/3151638506 >> v2:172.21.15.6:3300/0 conn(0x7f3748158a50 0x7f37480718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 >> v1:172.21.15.6:6789/0 conn(0x7f374806ede0 legacy=0x7f374815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374815b9c0 con 0x7f374815c4a0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f373c001690 con 0x7f374815c4a0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 >> v1:172.21.15.1:6789/0 conn(0x7f374815c4a0 legacy=0x7f374806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37300039b0 0x7f3730003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3730004800 0x7f3730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f373000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3748158e30 con 0x7f37300051c0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f373000c060 con 0x7f37300039b0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f373000c260 con 0x7f3730004800 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f373000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d069640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37300039b0 0x7f3730003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374c868640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3730004800 0x7f3730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 -- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3730004800 msgr2=0x7f3730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3730004800 0x7f3730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 -- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37300039b0 msgr2=0x7f3730003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37300039b0 0x7f3730003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374d86a640 1 -- 172.21.15.1:0/3151638506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3730003520 con 0x7f37300051c0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.333+0000 7f374c868640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3730004800 0x7f3730004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.334+0000 7f374d86a640 1 -- 172.21.15.1:0/3151638506 reap_dead start 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.334+0000 7f374d86a640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f373000b9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f374806c2c0 tx=0x7f3730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5bd6f31d8e2d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.334+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738001370 con 0x7f37300051c0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.334+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37380052d0 con 0x7f37300051c0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.334+0000 7f3747fff640 1 -- 172.21.15.1:0/3151638506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373802ab90 con 0x7f37300051c0 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 -- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 msgr2=0x7f373000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f373000b9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f374806c2c0 tx=0x7f3730005830 comp rx=0 tx=0).stop 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 -- 172.21.15.1:0/3151638506 shutdown_connections 2024-05-09T06:28:59.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 --2- 172.21.15.1:0/3151638506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f373000b9c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 -- 172.21.15.1:0/3151638506 >> 172.21.15.1:0/3151638506 conn(0x7f37480cc4d0 msgr2=0x7f37480cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 -- 172.21.15.1:0/3151638506 shutdown_connections 2024-05-09T06:28:59.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.335+0000 7f374f2cd640 1 -- 172.21.15.1:0/3151638506 wait complete. 2024-05-09T06:28:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.336+0000 7f374f2cd640 1 Processor -- start 2024-05-09T06:28:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.336+0000 7f374f2cd640 1 -- start start 2024-05-09T06:28:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37480b4270 0x7f37480b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37480b5680 0x7f374815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f374815bf80 con 0x7f37300051c0 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3748072480 con 0x7f37480b4270 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374f2cd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3748070ea0 con 0x7f37480b5680 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57848/0 (socket says 172.21.15.1:57848) 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 -- 172.21.15.1:0/1601983877 learned_addr learned my addr 172.21.15.1:0/1601983877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 -- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37480b5680 msgr2=0x7f374815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37480b5680 0x7f374815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 -- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37480b4270 msgr2=0x7f37480b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37480b4270 0x7f37480b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 -- 172.21.15.1:0/1601983877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37480b34b0 con 0x7f37300051c0 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f374d069640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3740011e30 tx=0x7f374000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740015630 con 0x7f37300051c0 2024-05-09T06:28:59.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.337+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37400157d0 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.338+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37480b3790 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.338+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740015bc0 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.338+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37480b3d50 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.339+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37480752b0 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.339+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f374001d400 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.340+0000 7f3745ffb640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 0x7f373406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.340+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f374008c0b0 con 0x7f37300051c0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.340+0000 7f374c868640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 0x7f373406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.340+0000 7f374c868640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 0x7f373406e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373c001c10 tx=0x7f373c001150 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.344+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3740019030 con 0x7f37300051c0 2024-05-09T06:28:59.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.525+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3748071d80 con 0x7f37300051c0 2024-05-09T06:28:59.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.526+0000 7f3745ffb640 1 -- 172.21.15.1:0/1601983877 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f3740010040 con 0x7f37300051c0 2024-05-09T06:28:59.508 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 msgr2=0x7f373406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 0x7f373406e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373c001c10 tx=0x7f373c001150 comp rx=0 tx=0).stop 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 msgr2=0x7f374814fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3740011e30 tx=0x7f374000f580 comp rx=0 tx=0).stop 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 shutdown_connections 2024-05-09T06:28:59.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f373406be60 0x7f373406e320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f37480b5680 0x7f374815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37480b4270 0x7f37480b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 --2- 172.21.15.1:0/1601983877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37300051c0 0x7f374814fc50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 >> 172.21.15.1:0/1601983877 conn(0x7f37480cc4d0 msgr2=0x7f3748158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 shutdown_connections 2024-05-09T06:28:59.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.528+0000 7f374f2cd640 1 -- 172.21.15.1:0/1601983877 wait complete. 2024-05-09T06:28:59.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:28:59.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.646+0000 7f96e44fc640 1 Processor -- start 2024-05-09T06:28:59.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.646+0000 7f96e44fc640 1 -- start start 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.647+0000 7f96e44fc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f96dc0d45f0 0x7f96dc15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.647+0000 7f96e44fc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc161fe0 con 0x7f96dc0d1c70 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.647+0000 7f96e44fc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc1622c0 con 0x7f96dc15e190 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.647+0000 7f96e44fc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f96dc15f580 con 0x7f96dc0d45f0 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1a97640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f96dc0d1c70 0x7f96dc0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59372/0 (socket says 172.21.15.1:59372) 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1a97640 1 -- 172.21.15.1:0/2576576587 learned_addr learned my addr 172.21.15.1:0/2576576587 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2576576587 >> v2:172.21.15.6:3300/0 conn(0x7f96dc0d45f0 0x7f96dc15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257278183 0 0) 0x7f96dc161fe0 con 0x7f96dc0d1c70 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c4003520 con 0x7f96dc0d1c70 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116245686 0 0) 0x7f96dc1622c0 con 0x7f96dc15e190 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96dc161fe0 con 0x7f96dc15e190 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3461110767 0 0) 0x7f96c4003520 con 0x7f96dc0d1c70 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 >> v2:172.21.15.6:3300/0 conn(0x7f96dc0d45f0 msgr2=0x7f96dc15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 --2- 172.21.15.1:0/2576576587 >> v2:172.21.15.6:3300/0 conn(0x7f96dc0d45f0 0x7f96dc15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 >> v1:172.21.15.6:6789/0 conn(0x7f96dc15e190 legacy=0x7f96dc0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1625a0 con 0x7f96dc0d1c70 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f96d0001670 con 0x7f96dc0d1c70 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 >> v1:172.21.15.1:6789/0 conn(0x7f96dc0d1c70 legacy=0x7f96dc0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c4003b40 0x7f96c4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96c4004800 0x7f96c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 0x7f96c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96c400c010 con 0x7f96c40051c0 2024-05-09T06:28:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96c400c1c0 con 0x7f96c4004800 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96c400c3c0 con 0x7f96c4003b40 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 0x7f96c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e1a97640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96c4004800 0x7f96c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.648+0000 7f96e2298640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c4003b40 0x7f96c4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 -- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c4003b40 msgr2=0x7f96c4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c4003b40 0x7f96c4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 -- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96c4004800 msgr2=0x7f96c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96c4004800 0x7f96c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 -- 172.21.15.1:0/2576576587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c4003520 con 0x7f96c40051c0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2298640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c4003b40 0x7f96c4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 -- 172.21.15.1:0/2576576587 reap_dead start 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.649+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 0x7f96c400b9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f90 tx=0x7f96c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3860171bf79abf38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc009c10 con 0x7f96c40051c0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc031440 con 0x7f96c40051c0 2024-05-09T06:28:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e1296640 1 -- 172.21.15.1:0/2576576587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc031810 con 0x7f96c40051c0 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 -- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 msgr2=0x7f96c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 0x7f96c400b9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f90 tx=0x7f96c4005830 comp rx=0 tx=0).stop 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 -- 172.21.15.1:0/2576576587 shutdown_connections 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2576576587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96c40051c0 0x7f96c400b9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 -- 172.21.15.1:0/2576576587 >> 172.21.15.1:0/2576576587 conn(0x7f96dc073340 msgr2=0x7f96dc09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 -- 172.21.15.1:0/2576576587 shutdown_connections 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.650+0000 7f96e44fc640 1 -- 172.21.15.1:0/2576576587 wait complete. 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.651+0000 7f96e44fc640 1 Processor -- start 2024-05-09T06:28:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.651+0000 7f96e44fc640 1 -- start start 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c40051c0 0x7f96dc14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96dc1f3c90 0x7f96dc1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b56d0 con 0x7f96dc1607d0 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc161c60 con 0x7f96dc1f3c90 2024-05-09T06:28:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e44fc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0b5850 con 0x7f96c40051c0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e1a97640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e1a97640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57864/0 (socket says 172.21.15.1:57864) 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.652+0000 7f96e1a97640 1 -- 172.21.15.1:0/2438406456 learned_addr learned my addr 172.21.15.1:0/2438406456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e2a99640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96dc1f3c90 0x7f96dc1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 -- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c40051c0 msgr2=0x7f96dc14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c40051c0 0x7f96dc14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 -- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96dc1f3c90 msgr2=0x7f96dc1f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96dc1f3c90 0x7f96dc1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 -- 172.21.15.1:0/2438406456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1f49c0 con 0x7f96dc1607d0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e1a97640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004fa0 tx=0x7f96d00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d001a070 con 0x7f96dc1607d0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d000f720 con 0x7f96dc1607d0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1f70f0 con 0x7f96dc1607d0 2024-05-09T06:28:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d000fad0 con 0x7f96dc1607d0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.653+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1f76b0 con 0x7f96dc1607d0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.654+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc161190 con 0x7f96dc1607d0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.655+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96d001d070 con 0x7f96dc1607d0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.655+0000 7f96daffd640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 0x7f96c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.655+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f96d008a8a0 con 0x7f96dc1607d0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.655+0000 7f96e2298640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 0x7f96c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.656+0000 7f96e2298640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 0x7f96c806e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000f60 tx=0x7f96d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.659+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96d00590c0 con 0x7f96dc1607d0 2024-05-09T06:28:59.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.843+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96dc0752b0 con 0x7f96dc1607d0 2024-05-09T06:28:59.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.843+0000 7f96daffd640 1 -- 172.21.15.1:0/2438406456 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f96d00589c0 con 0x7f96dc1607d0 2024-05-09T06:28:59.825 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:28:59.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 msgr2=0x7f96c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 0x7f96c806e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000f60 tx=0x7f96d4009040 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 msgr2=0x7f96dc160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004fa0 tx=0x7f96d00074f0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 shutdown_connections 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96c806be60 0x7f96c806e320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96dc1f3c90 0x7f96dc1f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96dc1607d0 0x7f96dc160c50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 --2- 172.21.15.1:0/2438406456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96c40051c0 0x7f96dc14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 >> 172.21.15.1:0/2438406456 conn(0x7f96dc073340 msgr2=0x7f96dc0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 shutdown_connections 2024-05-09T06:28:59.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.845+0000 7f96e44fc640 1 -- 172.21.15.1:0/2438406456 wait complete. 2024-05-09T06:28:59.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:28:59.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.961+0000 7fd457509640 1 Processor -- start 2024-05-09T06:28:59.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.961+0000 7fd457509640 1 -- start start 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd457509640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd45006e6d0 0x7fd45006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd457509640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4500cf180 0x7fd4500cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd457509640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4500ce0e0 con 0x7fd4500cdab0 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd457509640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd4500cdec0 con 0x7fd4500cf180 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd457509640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd4500ce3c0 con 0x7fd45006e6d0 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd454aa4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4500cf180 0x7fd4500cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd455aa6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd4500cdab0 0x7fd45015e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59382/0 (socket says 172.21.15.1:59382) 2024-05-09T06:28:59.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd455aa6640 1 -- 172.21.15.1:0/1934535818 learned_addr learned my addr 172.21.15.1:0/1934535818 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd4552a5640 1 --2- 172.21.15.1:0/1934535818 >> v2:172.21.15.6:3300/0 conn(0x7fd45006e6d0 0x7fd45006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd454aa4640 1 -- 172.21.15.1:0/1934535818 >> v2:172.21.15.6:3300/0 conn(0x7fd45006e6d0 msgr2=0x7fd45006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1934535818 >> v2:172.21.15.6:3300/0 conn(0x7fd45006e6d0 0x7fd45006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd454aa4640 1 -- 172.21.15.1:0/1934535818 >> v1:172.21.15.1:6789/0 conn(0x7fd4500cdab0 legacy=0x7fd45015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd454aa4640 1 -- 172.21.15.1:0/1934535818 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4500ce6a0 con 0x7fd4500cf180 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.962+0000 7fd447fff640 1 -- 172.21.15.1:0/1934535818 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312958721 0 0) 0x7fd4500ce0e0 con 0x7fd4500cdab0 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.963+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1934535818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4500cf180 0x7fd4500cd570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd450074650 tx=0x7fd44002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419077d26114628d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.963+0000 7fd447fff640 1 -- 172.21.15.1:0/1934535818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440037070 con 0x7fd4500cf180 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.963+0000 7fd447fff640 1 -- 172.21.15.1:0/1934535818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440032440 con 0x7fd4500cf180 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.963+0000 7fd447fff640 1 -- 172.21.15.1:0/1934535818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440032810 con 0x7fd4500cf180 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 -- 172.21.15.1:0/1934535818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4500cf180 msgr2=0x7fd4500cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 --2- 172.21.15.1:0/1934535818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4500cf180 0x7fd4500cd570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd450074650 tx=0x7fd44002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 -- 172.21.15.1:0/1934535818 shutdown_connections 2024-05-09T06:28:59.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 --2- 172.21.15.1:0/1934535818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4500cf180 0x7fd4500cd570 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 --2- 172.21.15.1:0/1934535818 >> v2:172.21.15.6:3300/0 conn(0x7fd45006e6d0 0x7fd45006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 -- 172.21.15.1:0/1934535818 >> 172.21.15.1:0/1934535818 conn(0x7fd4500cc4b0 msgr2=0x7fd4500cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:59.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 -- 172.21.15.1:0/1934535818 shutdown_connections 2024-05-09T06:28:59.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.964+0000 7fd457509640 1 -- 172.21.15.1:0/1934535818 wait complete. 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.965+0000 7fd457509640 1 Processor -- start 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.965+0000 7fd457509640 1 -- start start 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 0x7fd450159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 0x7fd450159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd45006eff0 con 0x7fd45006e6d0 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd4500cec60 con 0x7fd4500cdab0 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd457509640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd450070ed0 con 0x7fd4500cf180 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57878/0 (socket says 172.21.15.1:57878) 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 -- 172.21.15.1:0/1029176491 learned_addr learned my addr 172.21.15.1:0/1029176491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 0x7fd450159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 -- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 msgr2=0x7fd450159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd455aa6640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 0x7fd450159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 0x7fd450159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 -- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 msgr2=0x7fd450159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 0x7fd450159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd4552a5640 1 -- 172.21.15.1:0/1029176491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45015a5a0 con 0x7fd45006e6d0 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 0x7fd450159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.966+0000 7fd455aa6640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 0x7fd450159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd4552a5640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e870 tx=0x7fd44c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c017070 con 0x7fd45006e6d0 2024-05-09T06:28:59.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c00c940 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45015a880 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00cd30 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.967+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501f3010 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.968+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4500752b0 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.968+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd44c01a070 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.969+0000 7fd445ffb640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 0x7fd43406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.969+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd44c08b1d0 con 0x7fd45006e6d0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.969+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 0x7fd43406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:59.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.970+0000 7fd454aa4640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 0x7fd43406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:59.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:28:59.973+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd44c059a80 con 0x7fd45006e6d0 2024-05-09T06:29:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.172+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd45015f7e0 con 0x7fd45006e6d0 2024-05-09T06:29:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.173+0000 7fd445ffb640 1 -- 172.21.15.1:0/1029176491 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7fd44c059380 con 0x7fd45006e6d0 2024-05-09T06:29:00.155 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.174+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 msgr2=0x7fd43406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.174+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 0x7fd43406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 msgr2=0x7fd4501f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e870 tx=0x7fd44c00ed30 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 shutdown_connections 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd43406be30 0x7fd43406e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4500cf180 0x7fd450159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4500cdab0 0x7fd450159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 --2- 172.21.15.1:0/1029176491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd45006e6d0 0x7fd4501f2ad0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 >> 172.21.15.1:0/1029176491 conn(0x7fd4500cc4b0 msgr2=0x7fd4500b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 shutdown_connections 2024-05-09T06:29:00.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:00.175+0000 7fd457509640 1 -- 172.21.15.1:0/1029176491 wait complete. 2024-05-09T06:29:00.167 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:29:00.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:00.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:00.192 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[52250]: fuse finished with error 0 and tester_r 0 2024-05-09T06:29:00.193 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:29:06.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:29:06.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:06.295 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:29:06.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:06.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.491+0000 7fec26076640 1 Processor -- start 2024-05-09T06:29:06.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.491+0000 7fec26076640 1 -- start start 2024-05-09T06:29:06.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.491+0000 7fec26076640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 0x7fec200b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.491+0000 7fec26076640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 0x7fec200b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.491+0000 7fec26076640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200b33c0 con 0x7fec20161950 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec26076640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fec200d4dd0 con 0x7fec200d2b10 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec26076640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fec200b3660 con 0x7fec200d4940 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec16ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 0x7fec200b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1f7fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fec20161950 0x7fec200d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:49486/0 (socket says 172.21.15.1:49486) 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 0x7fec200b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1f7fe640 1 -- 172.21.15.1:0/2361279380 learned_addr learned my addr 172.21.15.1:0/2361279380 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1effd640 1 -- 172.21.15.1:0/2361279380 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090126762 0 0) 0x7fec200b33c0 con 0x7fec20161950 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1effd640 1 -- 172.21.15.1:0/2361279380 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec0c003520 con 0x7fec20161950 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec16ffd640 1 -- 172.21.15.1:0/2361279380 >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 msgr2=0x7fec200b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec16ffd640 1 --2- 172.21.15.1:0/2361279380 >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 0x7fec200b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec16ffd640 1 -- 172.21.15.1:0/2361279380 >> v1:172.21.15.6:6789/0 conn(0x7fec20161950 legacy=0x7fec200d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec16ffd640 1 -- 172.21.15.1:0/2361279380 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec200b3940 con 0x7fec200d4940 2024-05-09T06:29:06.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.492+0000 7fec1ffff640 1 --2- 172.21.15.1:0/2361279380 >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 0x7fec200b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:29:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.493+0000 7fec16ffd640 1 --2- 172.21.15.1:0/2361279380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 0x7fec200b4140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec080097a0 tx=0x7fec0802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe919540157f250 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.493+0000 7fec1effd640 1 -- 172.21.15.1:0/2361279380 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08037070 con 0x7fec200d4940 2024-05-09T06:29:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec1effd640 1 -- 172.21.15.1:0/2361279380 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec08032440 con 0x7fec200d4940 2024-05-09T06:29:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec1effd640 1 -- 172.21.15.1:0/2361279380 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08032810 con 0x7fec200d4940 2024-05-09T06:29:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 -- 172.21.15.1:0/2361279380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 msgr2=0x7fec200b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 --2- 172.21.15.1:0/2361279380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 0x7fec200b4140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec080097a0 tx=0x7fec0802a180 comp rx=0 tx=0).stop 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 -- 172.21.15.1:0/2361279380 shutdown_connections 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 --2- 172.21.15.1:0/2361279380 >> v2:172.21.15.6:3300/0 conn(0x7fec200d2b10 0x7fec200b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 --2- 172.21.15.1:0/2361279380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200d4940 0x7fec200b4140 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 -- 172.21.15.1:0/2361279380 >> 172.21.15.1:0/2361279380 conn(0x7fec200d3e20 msgr2=0x7fec200d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 -- 172.21.15.1:0/2361279380 shutdown_connections 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.494+0000 7fec26076640 1 -- 172.21.15.1:0/2361279380 wait complete. 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.495+0000 7fec26076640 1 Processor -- start 2024-05-09T06:29:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 -- start start 2024-05-09T06:29:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 0x7fec200b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 0x7fec200cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d1100 con 0x7fec20161950 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec200d0f80 con 0x7fec200d2b10 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.496+0000 7fec26076640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fec200d1c00 con 0x7fec200cbf40 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57882/0 (socket says 172.21.15.1:57882) 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 -- 172.21.15.1:0/2590246794 learned_addr learned my addr 172.21.15.1:0/2590246794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1ffff640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 0x7fec200cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1f7fe640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 0x7fec200b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 -- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 msgr2=0x7fec200cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 0x7fec200cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 -- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 msgr2=0x7fec200b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 0x7fec200b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 -- 172.21.15.1:0/2590246794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec201517b0 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1f7fe640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 0x7fec200b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1ffff640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 0x7fec200cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec16ffd640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fec08035040 tx=0x7fec080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08041030 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0803d4e0 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec200b72a0 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08037040 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.497+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec200b6ce0 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.498+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec200752b0 con 0x7fec20161950 2024-05-09T06:29:06.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.499+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec08044070 con 0x7fec20161950 2024-05-09T06:29:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.499+0000 7fec1cff9640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 0x7fec0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.499+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fec080ab6b0 con 0x7fec20161950 2024-05-09T06:29:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.499+0000 7fec1f7fe640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 0x7fec0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.500+0000 7fec1f7fe640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 0x7fec0006e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f60 tx=0x7fec10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.504+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec08079ec0 con 0x7fec20161950 2024-05-09T06:29:06.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.725+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec2008dec0 con 0x7fec20161950 2024-05-09T06:29:06.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.725+0000 7fec1cff9640 1 -- 172.21.15.1:0/2590246794 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7fec080797c0 con 0x7fec20161950 2024-05-09T06:29:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:06.708 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:06.708 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 msgr2=0x7fec0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 0x7fec0006e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f60 tx=0x7fec10009040 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 msgr2=0x7fec20151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fec08035040 tx=0x7fec080046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 shutdown_connections 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec0006be30 0x7fec0006e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec200cbf40 0x7fec200cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec20161950 0x7fec20151270 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.727+0000 7fec26076640 1 --2- 172.21.15.1:0/2590246794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec200d2b10 0x7fec200b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.728+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 >> 172.21.15.1:0/2590246794 conn(0x7fec200d3e20 msgr2=0x7fec200d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:06.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.728+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 shutdown_connections 2024-05-09T06:29:06.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.728+0000 7fec26076640 1 -- 172.21.15.1:0/2590246794 wait complete. 2024-05-09T06:29:06.721 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-09T06:29:06.862 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2024-05-09T06:29:06.862 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:29:06.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:06.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.984+0000 7fec7b053640 1 Processor -- start 2024-05-09T06:29:06.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.984+0000 7fec7b053640 1 -- start start 2024-05-09T06:29:06.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec7b053640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec74161950 0x7fec740d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec7b053640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fec740d4940 0x7fec740d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec7b053640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec740b33c0 con 0x7fec740d0f80 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec7b053640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fec740d4dd0 con 0x7fec740d4940 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec7b053640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fec740b3660 con 0x7fec74161950 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.985+0000 7fec73fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fec740d4940 0x7fec740d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fec740d4940 0x7fec740d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57890/0 (socket says 172.21.15.1:57890) 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 -- 172.21.15.1:0/1001505347 learned_addr learned my addr 172.21.15.1:0/1001505347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec78def640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec74161950 0x7fec740d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 -- 172.21.15.1:0/1001505347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec74161950 msgr2=0x7fec740d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec74161950 0x7fec740d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 -- 172.21.15.1:0/1001505347 >> v1:172.21.15.6:6789/0 conn(0x7fec740d0f80 legacy=0x7fec740cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.986+0000 7fec73fff640 1 -- 172.21.15.1:0/1001505347 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740b3940 con 0x7fec740d4940 2024-05-09T06:29:06.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec73fff640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec740d4940 0x7fec740d2940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec740b3590 tx=0x7fec6802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db3f1ffaaf4bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec737fe640 1 -- 172.21.15.1:0/1001505347 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68037070 con 0x7fec740d4940 2024-05-09T06:29:06.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec737fe640 1 -- 172.21.15.1:0/1001505347 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec68032440 con 0x7fec740d4940 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec737fe640 1 -- 172.21.15.1:0/1001505347 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68032810 con 0x7fec740d4940 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 -- 172.21.15.1:0/1001505347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec740d4940 msgr2=0x7fec740d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec740d4940 0x7fec740d2940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec740b3590 tx=0x7fec6802a250 comp rx=0 tx=0).stop 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 -- 172.21.15.1:0/1001505347 shutdown_connections 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec740d4940 0x7fec740d2940 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 --2- 172.21.15.1:0/1001505347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec74161950 0x7fec740d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.987+0000 7fec7b053640 1 -- 172.21.15.1:0/1001505347 >> 172.21.15.1:0/1001505347 conn(0x7fec740d3e20 msgr2=0x7fec740d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.988+0000 7fec7b053640 1 -- 172.21.15.1:0/1001505347 shutdown_connections 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.988+0000 7fec7b053640 1 -- 172.21.15.1:0/1001505347 wait complete. 2024-05-09T06:29:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.989+0000 7fec7b053640 1 Processor -- start 2024-05-09T06:29:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.989+0000 7fec7b053640 1 -- start start 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec740d0f80 0x7fec741f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec741f1110 0x7fec7415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec740b52d0 con 0x7fec74161950 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec740b5150 con 0x7fec741f1110 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fec740b5450 con 0x7fec740d0f80 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57906/0 (socket says 172.21.15.1:57906) 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 -- 172.21.15.1:0/3911660899 learned_addr learned my addr 172.21.15.1:0/3911660899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec78def640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec740d0f80 0x7fec741f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec795f0640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec741f1110 0x7fec7415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 -- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec740d0f80 msgr2=0x7fec741f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec740d0f80 0x7fec741f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 -- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec741f1110 msgr2=0x7fec7415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec741f1110 0x7fec7415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 -- 172.21.15.1:0/3911660899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740cbad0 con 0x7fec74161950 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec73fff640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec6802a290 tx=0x7fec680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68037040 con 0x7fec74161950 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec68042070 con 0x7fec74161950 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68030880 con 0x7fec74161950 2024-05-09T06:29:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.990+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec740cbdb0 con 0x7fec74161950 2024-05-09T06:29:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.991+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7415bed0 con 0x7fec74161950 2024-05-09T06:29:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.992+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec740cc2d0 con 0x7fec74161950 2024-05-09T06:29:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.992+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec68030a20 con 0x7fec74161950 2024-05-09T06:29:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.992+0000 7fec717fa640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 0x7fec6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.993+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fec680ab920 con 0x7fec74161950 2024-05-09T06:29:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.993+0000 7fec78def640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 0x7fec6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.993+0000 7fec78def640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 0x7fec6006e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec7415b800 tx=0x7fec64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:06.997+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec6807a120 con 0x7fec74161950 2024-05-09T06:29:07.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.219+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec740752b0 con 0x7fec74161950 2024-05-09T06:29:07.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.219+0000 7fec717fa640 1 -- 172.21.15.1:0/3911660899 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7fec68079a20 con 0x7fec74161950 2024-05-09T06:29:07.201 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:07.201 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:07.202 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:07.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 msgr2=0x7fec6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 0x7fec6006e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec7415b800 tx=0x7fec64008040 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 msgr2=0x7fec740cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec6802a290 tx=0x7fec680046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 shutdown_connections 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fec6006be30 0x7fec6006e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec741f1110 0x7fec7415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec74161950 0x7fec740cb360 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 --2- 172.21.15.1:0/3911660899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fec740d0f80 0x7fec741f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 >> 172.21.15.1:0/3911660899 conn(0x7fec740d3e20 msgr2=0x7fec740d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 shutdown_connections 2024-05-09T06:29:07.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:07.221+0000 7fec7b053640 1 -- 172.21.15.1:0/3911660899 wait complete. 2024-05-09T06:29:07.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2024-05-09T06:29:07.343 INFO:teuthology.orchestra.run.smithi006.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-09T06:29:07.344 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-09T06:29:17.345 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-09T06:29:17.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.416+0000 7f539d11ff40 1 Processor -- start 2024-05-09T06:29:17.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.416+0000 7f539d11ff40 1 -- start start 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 0x55e660942b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e660a03f40 0x55e660973a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e6609700b0 con 0x55e66098c840 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e6608eac80 con 0x55e660990290 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539d11ff40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e66098b920 con 0x55e660a03f40 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539a1a8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539a1a8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58452/0 (socket says 172.21.15.1:58452) 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.417+0000 7f539a1a8640 1 -- 172.21.15.1:0/22054720 learned_addr learned my addr 172.21.15.1:0/22054720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 -- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e660a03f40 msgr2=0x55e660973a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e660a03f40 0x55e660973a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 -- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 msgr2=0x55e660942b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f53999a7640 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 0x55e660942b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 0x55e660942b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 -- 172.21.15.1:0/22054720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66086fcb0 con 0x55e66098c840 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f53999a7640 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 0x55e660942b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f539a1a8640 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c4b0 tx=0x7f538c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faf8a86dca37ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f53989a5640 1 -- 172.21.15.1:0/22054720 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0116b0 con 0x55e66098c840 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f53989a5640 1 -- 172.21.15.1:0/22054720 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c017070 con 0x55e66098c840 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.418+0000 7f53989a5640 1 -- 172.21.15.1:0/22054720 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c011a90 con 0x55e66098c840 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- 172.21.15.1:0/22054720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 msgr2=0x55e66095a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c4b0 tx=0x7f538c00c970 comp rx=0 tx=0).stop 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- 172.21.15.1:0/22054720 shutdown_connections 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e660a03f40 0x55e660973a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660990290 0x55e660942b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 --2- 172.21.15.1:0/22054720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e66098c840 0x55e66095a460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- 172.21.15.1:0/22054720 >> 172.21.15.1:0/22054720 conn(0x55e660990e90 msgr2=0x55e660991270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- 172.21.15.1:0/22054720 shutdown_connections 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- 172.21.15.1:0/22054720 wait complete. 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 Processor -- start 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.419+0000 7f539d11ff40 1 -- start start 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e66098c840 0x55e66095b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 0x55e660973a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660a03f40 0x55e66095d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e660a19b40 con 0x55e660990290 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e660a1a9e0 con 0x55e660a03f40 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539d11ff40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e660a1a720 con 0x55e66098c840 2024-05-09T06:29:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f53999a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 0x55e660973a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f53999a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 0x55e660973a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58456/0 (socket says 172.21.15.1:58456) 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f53999a7640 1 -- 172.21.15.1:0/1121596337 learned_addr learned my addr 172.21.15.1:0/1121596337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539a1a8640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e66098c840 0x55e66095b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.420+0000 7f539a9a9640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660a03f40 0x55e66095d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 -- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e66098c840 msgr2=0x55e66095b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e66098c840 0x55e66095b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 -- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660a03f40 msgr2=0x55e66095d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e660a03f40 0x55e66095d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 -- 172.21.15.1:0/1121596337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e660943b40 con 0x55e660990290 2024-05-09T06:29:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f53999a7640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 0x55e660973a40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f539000cef0 tx=0x7f539000a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f538b7fe640 1 -- 172.21.15.1:0/1121596337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390008040 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f538b7fe640 1 -- 172.21.15.1:0/1121596337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5390014020 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e66095aff0 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f538b7fe640 1 -- 172.21.15.1:0/1121596337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539000f400 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.421+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66095aad0 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.423+0000 7f538b7fe640 1 -- 172.21.15.1:0/1121596337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f539000f5a0 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.423+0000 7f538b7fe640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f537c06a810 0x7f537c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.423+0000 7f538b7fe640 1 -- 172.21.15.1:0/1121596337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f53900899e0 con 0x55e660990290 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.423+0000 7f539a1a8640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f537c06a810 0x7f537c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.424+0000 7f539d11ff40 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e660973f80 0x55e66095d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.424+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55e660ab46a0 con 0x55e660973f80 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.424+0000 7f539a9a9640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e660973f80 0x55e66095d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.424+0000 7f539a1a8640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f537c06a810 0x7f537c06ccd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f538c014040 tx=0x7f538c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.424+0000 7f539a9a9640 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e660973f80 0x55e66095d780 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539a9a9640 1 -- 172.21.15.1:0/1121596337 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f539405c040 con 0x55e660973f80 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e660973f80 msgr2=0x55e66095d780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e660973f80 0x55e66095d780 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f537c06a810 msgr2=0x7f537c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f537c06a810 0x7f537c06ccd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f538c014040 tx=0x7f538c011000 comp rx=0 tx=0).stop 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 msgr2=0x55e660973a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 --2- 172.21.15.1:0/1121596337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e660990290 0x55e660973a40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f539000cef0 tx=0x7f539000a5d0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539a9a9640 1 -- 172.21.15.1:0/1121596337 reap_dead start 2024-05-09T06:29:17.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 shutdown_connections 2024-05-09T06:29:17.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 >> 172.21.15.1:0/1121596337 conn(0x55e660990e90 msgr2=0x55e660972b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 shutdown_connections 2024-05-09T06:29:17.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.425+0000 7f539d11ff40 1 -- 172.21.15.1:0/1121596337 wait complete. 2024-05-09T06:29:17.414 INFO:teuthology.orchestra.run.smithi001.stdout:mytag 2024-05-09T06:29:17.415 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-09T06:29:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.489+0000 7fb4a9661f40 1 Processor -- start 2024-05-09T06:29:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.489+0000 7fb4a9661f40 1 -- start start 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b8217a290 0x562b8217a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562b820f4c80 con 0x562b82218e30 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562b820f8cc0 con 0x562b8217a290 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562b82195670 con 0x7fb490003970 2024-05-09T06:29:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58476/0 (socket says 172.21.15.1:58476) 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 -- 172.21.15.1:0/901321466 learned_addr learned my addr 172.21.15.1:0/901321466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 -- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 msgr2=0x562b8214d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 -- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b8217a290 msgr2=0x562b8217a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b8217a290 0x562b8217a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 -- 172.21.15.1:0/901321466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8217b1a0 con 0x562b82218e30 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.490+0000 7fb4a6961640 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e3b0 tx=0x7fb49c00e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72019212c235afae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a595f640 1 -- 172.21.15.1:0/901321466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c001b80 con 0x562b82218e30 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a595f640 1 -- 172.21.15.1:0/901321466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c001d20 con 0x562b82218e30 2024-05-09T06:29:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a595f640 1 -- 172.21.15.1:0/901321466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c011690 con 0x562b82218e30 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 -- 172.21.15.1:0/901321466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 msgr2=0x562b8221e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e3b0 tx=0x7fb49c00e7b0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 -- 172.21.15.1:0/901321466 shutdown_connections 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b8217a290 0x562b8217a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b82218e30 0x562b8221e790 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/901321466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 -- 172.21.15.1:0/901321466 >> 172.21.15.1:0/901321466 conn(0x562b8214bb00 msgr2=0x562b821791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 -- 172.21.15.1:0/901321466 shutdown_connections 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.491+0000 7fb4a9661f40 1 -- 172.21.15.1:0/901321466 wait complete. 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.492+0000 7fb4a9661f40 1 Processor -- start 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.492+0000 7fb4a9661f40 1 -- start start 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 0x562b82216760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b82218e30 0x562b8214c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562b8221f4e0 con 0x562b8217a290 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562b8221f200 con 0x562b82218e30 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562b8214ded0 con 0x7fb490003970 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 0x562b82216760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 0x562b82216760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58490/0 (socket says 172.21.15.1:58490) 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 -- 172.21.15.1:0/921254236 learned_addr learned my addr 172.21.15.1:0/921254236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a6961640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b82218e30 0x562b8214c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7963640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 -- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 msgr2=0x562b8214cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb490003970 0x562b8214cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 -- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b82218e30 msgr2=0x562b8214c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562b82218e30 0x562b8214c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 -- 172.21.15.1:0/921254236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b82178c90 con 0x562b8217a290 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a7162640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 0x562b82216760 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb498001730 tx=0x7fb49800e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a495d640 1 -- 172.21.15.1:0/921254236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800f710 con 0x562b8217a290 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a495d640 1 -- 172.21.15.1:0/921254236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498013030 con 0x562b8217a290 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562b82225880 con 0x562b8217a290 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a495d640 1 -- 172.21.15.1:0/921254236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800faf0 con 0x562b8217a290 2024-05-09T06:29:17.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.493+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b82216ca0 con 0x562b8217a290 2024-05-09T06:29:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.495+0000 7fb4a495d640 1 -- 172.21.15.1:0/921254236 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb49800fc90 con 0x562b8217a290 2024-05-09T06:29:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.495+0000 7fb4a495d640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49406a840 0x7fb49406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.495+0000 7fb4a495d640 1 -- 172.21.15.1:0/921254236 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fb49808a4b0 con 0x562b8217a290 2024-05-09T06:29:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.495+0000 7fb4a6961640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49406a840 0x7fb49406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.496+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x562b822188d0 0x562b822bb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.496+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x562b8221a6e0 con 0x562b822188d0 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.496+0000 7fb4a7963640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x562b822188d0 0x562b822bb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.496+0000 7fb4a6961640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49406a840 0x7fb49406cd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e350 tx=0x7fb49c020040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.496+0000 7fb4a7963640 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x562b822188d0 0x562b822bb930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a7963640 1 -- 172.21.15.1:0/921254236 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fb4a004db30 con 0x562b822188d0 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x562b822188d0 msgr2=0x562b822bb930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x562b822188d0 0x562b822bb930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49406a840 msgr2=0x7fb49406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49406a840 0x7fb49406cd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e350 tx=0x7fb49c020040 comp rx=0 tx=0).stop 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 msgr2=0x562b82216760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 --2- 172.21.15.1:0/921254236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8217a290 0x562b82216760 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb498001730 tx=0x7fb49800e870 comp rx=0 tx=0).stop 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a7963640 1 -- 172.21.15.1:0/921254236 reap_dead start 2024-05-09T06:29:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 shutdown_connections 2024-05-09T06:29:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 >> 172.21.15.1:0/921254236 conn(0x562b8214bb00 msgr2=0x562b8214ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 shutdown_connections 2024-05-09T06:29:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.497+0000 7fb4a9661f40 1 -- 172.21.15.1:0/921254236 wait complete. 2024-05-09T06:29:17.485 INFO:teuthology.orchestra.run.smithi001.stdout:mytag 2024-05-09T06:29:17.485 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-09T06:29:17.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.549+0000 7f8fab8f6f40 1 Processor -- start 2024-05-09T06:29:17.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.549+0000 7f8fab8f6f40 1 -- start start 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 0x55f3733a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f37338bbb0 0x55f3732d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f373279c80 con 0x55f373389350 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f37338c6c0 con 0x55f3732d1de0 2024-05-09T06:29:17.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f37339f6f0 con 0x55f37338bbb0 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fa8bf8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fa8bf8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58500/0 (socket says 172.21.15.1:58500) 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.550+0000 7f8fa8bf8640 1 -- 172.21.15.1:0/2447484033 learned_addr learned my addr 172.21.15.1:0/2447484033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa9bfa640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f37338bbb0 0x55f3732d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa93f9640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 0x55f3733a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 -- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f37338bbb0 msgr2=0x55f3732d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f37338bbb0 0x55f3732d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 -- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 msgr2=0x55f3733a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 0x55f3733a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 -- 172.21.15.1:0/2447484033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3731fecb0 con 0x55f373389350 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa93f9640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 0x55f3733a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001d90 tx=0x7f8f9400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d629de3156557d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa37fe640 1 -- 172.21.15.1:0/2447484033 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94012020 con 0x55f373389350 2024-05-09T06:29:17.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa37fe640 1 -- 172.21.15.1:0/2447484033 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f9400e670 con 0x55f373389350 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fa37fe640 1 -- 172.21.15.1:0/2447484033 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400ea80 con 0x55f373389350 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/2447484033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 msgr2=0x55f3733004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001d90 tx=0x7f8f9400b6c0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/2447484033 shutdown_connections 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f37338bbb0 0x55f3732d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f373389350 0x55f3733004e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/2447484033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f3732d1de0 0x55f3733a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/2447484033 >> 172.21.15.1:0/2447484033 conn(0x55f3732d2d10 msgr2=0x55f3733195b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.551+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/2447484033 shutdown_connections 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.552+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/2447484033 wait complete. 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.552+0000 7f8fab8f6f40 1 Processor -- start 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.552+0000 7f8fab8f6f40 1 -- start start 2024-05-09T06:29:17.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f3732d1de0 0x55f373389950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f373389350 0x55f3732d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 0x55f373387340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f37339f920 con 0x55f37338bbb0 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f3732cfd30 con 0x55f373389350 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fab8f6f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f37331a320 con 0x55f3732d1de0 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 0x55f373387340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 0x55f373387340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58504/0 (socket says 172.21.15.1:58504) 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 -- 172.21.15.1:0/1597061747 learned_addr learned my addr 172.21.15.1:0/1597061747 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa93f9640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f3732d1de0 0x55f373389950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f373389350 0x55f3732d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 -- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f3732d1de0 msgr2=0x55f373389950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f3732d1de0 0x55f373389950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 -- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f373389350 msgr2=0x55f3732d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f373389350 0x55f3732d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 -- 172.21.15.1:0/1597061747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f373389e90 con 0x55f37338bbb0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.553+0000 7f8fa9bfa640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 0x55f373387340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4025bc0 tx=0x7f8fa4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.554+0000 7f8fa27fc640 1 -- 172.21.15.1:0/1597061747 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4064020 con 0x55f37338bbb0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.554+0000 7f8fa27fc640 1 -- 172.21.15.1:0/1597061747 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa405a040 con 0x55f37338bbb0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.554+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f373388670 con 0x55f37338bbb0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.554+0000 7f8fa27fc640 1 -- 172.21.15.1:0/1597061747 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4059070 con 0x55f37338bbb0 2024-05-09T06:29:17.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.554+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f373388910 con 0x55f37338bbb0 2024-05-09T06:29:17.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.555+0000 7f8fa27fc640 1 -- 172.21.15.1:0/1597061747 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fa4067020 con 0x55f37338bbb0 2024-05-09T06:29:17.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa27fc640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8f8c06a810 0x7f8f8c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa27fc640 1 -- 172.21.15.1:0/1597061747 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f8fa40cd570 con 0x55f37338bbb0 2024-05-09T06:29:17.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa93f9640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8f8c06a810 0x7f8f8c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55f3733a66f0 0x55f373440490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55f3733a7ba0 con 0x55f3733a66f0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa93f9640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8f8c06a810 0x7f8f8c06ccd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005af0 tx=0x7f8f9c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55f3733a66f0 0x55f373440490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.556+0000 7f8fa8bf8640 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55f3733a66f0 0x55f373440490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fa8bf8640 1 -- 172.21.15.1:0/1597061747 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f8f9400e820 con 0x55f3733a66f0 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55f3733a66f0 msgr2=0x55f373440490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55f3733a66f0 0x55f373440490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8f8c06a810 msgr2=0x7f8f8c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8f8c06a810 0x7f8f8c06ccd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005af0 tx=0x7f8f9c007040 comp rx=0 tx=0).stop 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 msgr2=0x55f373387340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 --2- 172.21.15.1:0/1597061747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f37338bbb0 0x55f373387340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4025bc0 tx=0x7f8fa4002fb0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fa9bfa640 1 -- 172.21.15.1:0/1597061747 reap_dead start 2024-05-09T06:29:17.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 shutdown_connections 2024-05-09T06:29:17.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 >> 172.21.15.1:0/1597061747 conn(0x55f3732d2d10 msgr2=0x55f373381be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 shutdown_connections 2024-05-09T06:29:17.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.557+0000 7f8fab8f6f40 1 -- 172.21.15.1:0/1597061747 wait complete. 2024-05-09T06:29:17.545 INFO:teuthology.orchestra.run.smithi001.stdout:mytag 2024-05-09T06:29:17.545 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-09T06:29:17.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 Processor -- start 2024-05-09T06:29:17.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 -- start start 2024-05-09T06:29:17.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 0x5567533732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55675338c440 0x556753385450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5567532ebc80 con 0x55675336e0f0 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5567532efcc0 con 0x556753346620 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55675338b170 con 0x55675338c440 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.608+0000 7fc86f4e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58518/0 (socket says 172.21.15.1:58518) 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 -- 172.21.15.1:0/2737972513 learned_addr learned my addr 172.21.15.1:0/2737972513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc8704e2640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55675338c440 0x556753385450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86fce1640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 0x5567533732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 -- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55675338c440 msgr2=0x556753385450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55675338c440 0x556753385450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 -- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 msgr2=0x5567533732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 0x5567533732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 -- 172.21.15.1:0/2737972513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556753417ff0 con 0x55675336e0f0 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86fce1640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 0x5567533732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc86400a3f0 tx=0x7fc86400a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f493b8e84a5c615 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86e4de640 1 -- 172.21.15.1:0/2737972513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864010070 con 0x55675336e0f0 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86e4de640 1 -- 172.21.15.1:0/2737972513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc86400c8f0 con 0x55675336e0f0 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.609+0000 7fc86e4de640 1 -- 172.21.15.1:0/2737972513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cce0 con 0x55675336e0f0 2024-05-09T06:29:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- 172.21.15.1:0/2737972513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 msgr2=0x556753386a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc86400a3f0 tx=0x7fc86400a8b0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- 172.21.15.1:0/2737972513 shutdown_connections 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55675338c440 0x556753385450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753386a30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/2737972513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556753346620 0x5567533732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- 172.21.15.1:0/2737972513 >> 172.21.15.1:0/2737972513 conn(0x5567533707c0 msgr2=0x556753345f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- 172.21.15.1:0/2737972513 shutdown_connections 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- 172.21.15.1:0/2737972513 wait complete. 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 Processor -- start 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.610+0000 7fc8721e0f40 1 -- start start 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556753346620 0x5567533884e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753413f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55675338c440 0x556753385930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55675340ef40 con 0x55675336e0f0 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556753386280 con 0x55675338c440 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8721e0f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556753386f70 con 0x556753346620 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753413f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753413f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58526/0 (socket says 172.21.15.1:58526) 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 -- 172.21.15.1:0/1244883036 learned_addr learned my addr 172.21.15.1:0/1244883036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86fce1640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556753346620 0x5567533884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8704e2640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55675338c440 0x556753385930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 -- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556753346620 msgr2=0x5567533884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556753346620 0x5567533884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 -- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55675338c440 msgr2=0x556753385930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55675338c440 0x556753385930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86f4e0640 1 -- 172.21.15.1:0/1244883036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567534a9d70 con 0x55675336e0f0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc8704e2640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55675338c440 0x556753385930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.611+0000 7fc86fce1640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556753346620 0x5567533884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc86f4e0640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753413f60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc86400f000 tx=0x7fc864002f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc86d4dc640 1 -- 172.21.15.1:0/1244883036 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864025070 con 0x55675336e0f0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc86d4dc640 1 -- 172.21.15.1:0/1244883036 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8640158e0 con 0x55675336e0f0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567533878f0 con 0x55675336e0f0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc86d4dc640 1 -- 172.21.15.1:0/1244883036 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864010050 con 0x55675336e0f0 2024-05-09T06:29:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.612+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675340a620 con 0x55675336e0f0 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.613+0000 7fc86d4dc640 1 -- 172.21.15.1:0/1244883036 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc864028070 con 0x55675336e0f0 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc86d4dc640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc85c06a810 0x7fc85c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc86d4dc640 1 -- 172.21.15.1:0/1244883036 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fc86408b120 con 0x55675336e0f0 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc86fce1640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc85c06a810 0x7fc85c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x556753388a20 0x5567534b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x556753385e70 con 0x556753388a20 2024-05-09T06:29:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc8704e2640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x556753388a20 0x5567534b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.614+0000 7fc86fce1640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc85c06a810 0x7fc85c06ccd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc860057000 tx=0x7fc860052000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8704e2640 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x556753388a20 0x5567534b6290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8704e2640 1 -- 172.21.15.1:0/1244883036 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fc868002700 con 0x556753388a20 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x556753388a20 msgr2=0x5567534b6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x556753388a20 0x5567534b6290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc85c06a810 msgr2=0x7fc85c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc85c06a810 0x7fc85c06ccd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc860057000 tx=0x7fc860052000 comp rx=0 tx=0).stop 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 msgr2=0x556753413f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 --2- 172.21.15.1:0/1244883036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55675336e0f0 0x556753413f60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc86400f000 tx=0x7fc864002f30 comp rx=0 tx=0).stop 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8704e2640 1 -- 172.21.15.1:0/1244883036 reap_dead start 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 shutdown_connections 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 >> 172.21.15.1:0/1244883036 conn(0x5567533707c0 msgr2=0x556753345f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 shutdown_connections 2024-05-09T06:29:17.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.615+0000 7fc8721e0f40 1 -- 172.21.15.1:0/1244883036 wait complete. 2024-05-09T06:29:17.604 INFO:teuthology.orchestra.run.smithi001.stdout:mytag 2024-05-09T06:29:17.605 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-09T06:29:17.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.669+0000 7f39319e0f40 1 Processor -- start 2024-05-09T06:29:17.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.669+0000 7f39319e0f40 1 -- start start 2024-05-09T06:29:17.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046e190 0x55a6d0491e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046f040 0x55a6d03b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6d035fc80 con 0x55a6d046eaf0 2024-05-09T06:29:17.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a6d0363cc0 con 0x55a6d046f040 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f39319e0f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a6d0475890 con 0x55a6d046e190 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58552/0 (socket says 172.21.15.1:58552) 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 -- 172.21.15.1:0/3623522301 learned_addr learned my addr 172.21.15.1:0/3623522301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392f4e1640 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046e190 0x55a6d0491e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392ece0640 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046f040 0x55a6d03b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 -- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046e190 msgr2=0x55a6d0491e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046e190 0x55a6d0491e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 -- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046f040 msgr2=0x55a6d03b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046f040 0x55a6d03b9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 -- 172.21.15.1:0/3623522301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d03e7e90 con 0x55a6d046eaf0 2024-05-09T06:29:17.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392fce2640 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3928025990 tx=0x7f3928008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613f0c75df9b6675 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392dcde640 1 -- 172.21.15.1:0/3623522301 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928055040 con 0x55a6d046eaf0 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392dcde640 1 -- 172.21.15.1:0/3623522301 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3928051440 con 0x55a6d046eaf0 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.670+0000 7f392dcde640 1 -- 172.21.15.1:0/3623522301 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928006270 con 0x55a6d046eaf0 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 -- 172.21.15.1:0/3623522301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 msgr2=0x55a6d03ba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3928025990 tx=0x7f3928008030 comp rx=0 tx=0).stop 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 -- 172.21.15.1:0/3623522301 shutdown_connections 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046eaf0 0x55a6d03ba450 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046f040 0x55a6d03b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 --2- 172.21.15.1:0/3623522301 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046e190 0x55a6d0491e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 -- 172.21.15.1:0/3623522301 >> 172.21.15.1:0/3623522301 conn(0x55a6d03e47c0 msgr2=0x55a6d046f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 -- 172.21.15.1:0/3623522301 shutdown_connections 2024-05-09T06:29:17.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 -- 172.21.15.1:0/3623522301 wait complete. 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.671+0000 7f39319e0f40 1 Processor -- start 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 -- start start 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046e190 0x55a6d04707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046eaf0 0x55a6d0488c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 0x55a6d04891c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6d048a090 con 0x55a6d046f040 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a6d0489cf0 con 0x55a6d046e190 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f39319e0f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a6d03e79d0 con 0x55a6d046eaf0 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f392fce2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 0x55a6d04891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f392fce2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 0x55a6d04891c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58566/0 (socket says 172.21.15.1:58566) 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f392fce2640 1 -- 172.21.15.1:0/894071815 learned_addr learned my addr 172.21.15.1:0/894071815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f392f4e1640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046e190 0x55a6d04707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.672+0000 7f392ece0640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046eaf0 0x55a6d0488c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 -- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046eaf0 msgr2=0x55a6d0488c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046eaf0 0x55a6d0488c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 -- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046e190 msgr2=0x55a6d04707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046e190 0x55a6d04707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 -- 172.21.15.1:0/894071815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d03fcef0 con 0x55a6d046f040 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392f4e1640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6d046e190 0x55a6d04707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392ece0640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6d046eaf0 0x55a6d0488c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392fce2640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 0x55a6d04891c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3928061040 tx=0x7f39280015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392ccdc640 1 -- 172.21.15.1:0/894071815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392806d070 con 0x55a6d046f040 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392ccdc640 1 -- 172.21.15.1:0/894071815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39280519f0 con 0x55a6d046f040 2024-05-09T06:29:17.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a6d03fd1d0 con 0x55a6d046f040 2024-05-09T06:29:17.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f392ccdc640 1 -- 172.21.15.1:0/894071815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928055070 con 0x55a6d046f040 2024-05-09T06:29:17.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.673+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d03b7490 con 0x55a6d046f040 2024-05-09T06:29:17.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.674+0000 7f392ccdc640 1 -- 172.21.15.1:0/894071815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3928070070 con 0x55a6d046f040 2024-05-09T06:29:17.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f392ccdc640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f391c06a840 0x7f391c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f392ccdc640 1 -- 172.21.15.1:0/894071815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f39280cdb10 con 0x55a6d046f040 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f392f4e1640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f391c06a840 0x7f391c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f39319e0f40 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a6d0470d10 0x55a6d048afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55a6d0489840 con 0x55a6d0470d10 2024-05-09T06:29:17.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f392ece0640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a6d0470d10 0x55a6d048afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.675+0000 7f392f4e1640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f391c06a840 0x7f391c06cd00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f392000c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f392ece0640 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a6d0470d10 0x55a6d048afc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f392ece0640 1 -- 172.21.15.1:0/894071815 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f39240096e0 con 0x55a6d0470d10 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a6d0470d10 msgr2=0x55a6d048afc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a6d0470d10 0x55a6d048afc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f391c06a840 msgr2=0x7f391c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f391c06a840 0x7f391c06cd00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f392000c040 comp rx=0 tx=0).stop 2024-05-09T06:29:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 msgr2=0x55a6d04891c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 --2- 172.21.15.1:0/894071815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6d046f040 0x55a6d04891c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3928061040 tx=0x7f39280015b0 comp rx=0 tx=0).stop 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f392fce2640 1 -- 172.21.15.1:0/894071815 reap_dead start 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 shutdown_connections 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 >> 172.21.15.1:0/894071815 conn(0x55a6d03e47c0 msgr2=0x55a6d0401360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 shutdown_connections 2024-05-09T06:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:17.676+0000 7f39319e0f40 1 -- 172.21.15.1:0/894071815 wait complete. 2024-05-09T06:29:17.670 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:29:17.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:29:17.723 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:29:17.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:29:17.742 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:29:17.779 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:29:17.779 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:29:17.780 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:29:17.780 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:17.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:29:17.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:29:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:17.803 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:29:17.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:17.861 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T06:29:17.861 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:29:17.861 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:29:17.861 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:17.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:29:17.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:29:17.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:17.882 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:29:17.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:17.942 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:29:17.942 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:29:17.943 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:29:17.943 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:17.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:29:17.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:29:17.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:17.960 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:29:18.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:18.018 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:29:18.018 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:29:18.018 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:29:18.018 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:18.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:29:18.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:29:18.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:18.038 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:29:18.098 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:29:18.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:18.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:18.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.223+0000 7f9e64509640 1 Processor -- start 2024-05-09T06:29:18.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.223+0000 7f9e64509640 1 -- start start 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e64509640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9e5c0b2a60 0x7f9e5c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e64509640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9e5c0b4f70 0x7f9e5c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e64509640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c06eb40 con 0x7f9e5c1454b0 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e64509640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c06ee20 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e64509640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c06d8e0 con 0x7f9e5c0b2a60 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9e5c0b4f70 0x7f9e5c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9e5c0b4f70 0x7f9e5c070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43300/0 (socket says 172.21.15.1:43300) 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 -- 172.21.15.1:0/2005307792 learned_addr learned my addr 172.21.15.1:0/2005307792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 -- 172.21.15.1:0/2005307792 >> v2:172.21.15.6:3300/0 conn(0x7f9e5c0b2a60 msgr2=0x7f9e5c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e61aa4640 1 --2- 172.21.15.1:0/2005307792 >> v2:172.21.15.6:3300/0 conn(0x7f9e5c0b2a60 0x7f9e5c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 --2- 172.21.15.1:0/2005307792 >> v2:172.21.15.6:3300/0 conn(0x7f9e5c0b2a60 0x7f9e5c0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 -- 172.21.15.1:0/2005307792 >> v1:172.21.15.6:6789/0 conn(0x7f9e5c1454b0 legacy=0x7f9e5c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.224+0000 7f9e62aa6640 1 -- 172.21.15.1:0/2005307792 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c06dbc0 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.225+0000 7f9e62aa6640 1 --2- 172.21.15.1:0/2005307792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c070c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c04f830 tx=0x7f9e5402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26874d65dfb7faa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.225+0000 7f9e612a3640 1 -- 172.21.15.1:0/2005307792 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54036070 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.225+0000 7f9e612a3640 1 -- 172.21.15.1:0/2005307792 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e54031600 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.225+0000 7f9e612a3640 1 -- 172.21.15.1:0/2005307792 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540319d0 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 -- 172.21.15.1:0/2005307792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 msgr2=0x7f9e5c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 --2- 172.21.15.1:0/2005307792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c070c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c04f830 tx=0x7f9e5402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 -- 172.21.15.1:0/2005307792 shutdown_connections 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 --2- 172.21.15.1:0/2005307792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c070c10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 --2- 172.21.15.1:0/2005307792 >> v2:172.21.15.6:3300/0 conn(0x7f9e5c0b2a60 0x7f9e5c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 -- 172.21.15.1:0/2005307792 >> 172.21.15.1:0/2005307792 conn(0x7f9e5c0cc4b0 msgr2=0x7f9e5c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 -- 172.21.15.1:0/2005307792 shutdown_connections 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.226+0000 7f9e64509640 1 -- 172.21.15.1:0/2005307792 wait complete. 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.227+0000 7f9e64509640 1 Processor -- start 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.227+0000 7f9e64509640 1 -- start start 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e5c0b2a60 0x7f9e5c13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e5c1454b0 0x7f9e5c1e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c06e260 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c06e0e0 con 0x7f9e5c1454b0 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e64509640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c071150 con 0x7f9e5c0b2a60 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43316/0 (socket says 172.21.15.1:43316) 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 -- 172.21.15.1:0/1542574984 learned_addr learned my addr 172.21.15.1:0/1542574984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e62aa6640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e5c1454b0 0x7f9e5c1e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e622a5640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e5c0b2a60 0x7f9e5c13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 -- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e5c0b2a60 msgr2=0x7f9e5c13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e5c0b2a60 0x7f9e5c13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 -- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e5c1454b0 msgr2=0x7f9e5c1e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e5c1454b0 0x7f9e5c1e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 -- 172.21.15.1:0/1542574984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c1e3f20 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.228+0000 7f9e61aa4640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c7f0 tx=0x7f9e4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.229+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c015070 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.229+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c004240 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.229+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c1e4200 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.229+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007620 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.229+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c140300 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.230+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c1e4720 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.230+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e4c0077c0 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.231+0000 7f9e52ffd640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 0x7f9e4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.231+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f9e4c08b230 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.231+0000 7f9e622a5640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 0x7f9e4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.231+0000 7f9e622a5640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 0x7f9e4006e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.235+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e5c1e4720 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.455+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e5c0752b0 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.455+0000 7f9e52ffd640 1 -- 172.21.15.1:0/1542574984 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7f9e4c059ac0 con 0x7f9e5c0b4f70 2024-05-09T06:29:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:18.437 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:18.438 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:18.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 msgr2=0x7f9e4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 0x7f9e4006e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58008040 comp rx=0 tx=0).stop 2024-05-09T06:29:18.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 msgr2=0x7f9e5c1e1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c7f0 tx=0x7f9e4c00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 shutdown_connections 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9e4006be30 0x7f9e4006e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e5c1454b0 0x7f9e5c1e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e5c0b4f70 0x7f9e5c1e1260 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 --2- 172.21.15.1:0/1542574984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9e5c0b2a60 0x7f9e5c13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 >> 172.21.15.1:0/1542574984 conn(0x7f9e5c0cc4b0 msgr2=0x7f9e5c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 shutdown_connections 2024-05-09T06:29:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.457+0000 7f9e64509640 1 -- 172.21.15.1:0/1542574984 wait complete. 2024-05-09T06:29:18.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:18.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.573+0000 7f1a65072640 1 Processor -- start 2024-05-09T06:29:18.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.573+0000 7f1a65072640 1 -- start start 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a65072640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a6015db10 0x7f1a600b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a65072640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a65072640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a60070f40 con 0x7f1a600715d0 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a65072640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1a600cf170 con 0x7f1a6015db10 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a65072640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1a600cf450 con 0x7f1a6015e6d0 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a6015db10 0x7f1a600b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a6015db10 0x7f1a600b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43330/0 (socket says 172.21.15.1:43330) 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/1887542823 learned_addr learned my addr 172.21.15.1:0/1887542823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5f59e640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/1887542823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 msgr2=0x7f1a6006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a6006e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/1887542823 >> v1:172.21.15.6:6789/0 conn(0x7f1a600715d0 legacy=0x7f1a6015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.574+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/1887542823 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a60070c10 con 0x7f1a6015db10 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.575+0000 7f1a5ed9d640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a6015db10 0x7f1a600b2a60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000af60 tx=0x7f1a5002b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c06560577da5e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.575+0000 7f1a5dd9b640 1 -- 172.21.15.1:0/1887542823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50038070 con 0x7f1a6015db10 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.575+0000 7f1a5dd9b640 1 -- 172.21.15.1:0/1887542823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5000a2d0 con 0x7f1a6015db10 2024-05-09T06:29:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.575+0000 7f1a5dd9b640 1 -- 172.21.15.1:0/1887542823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50008720 con 0x7f1a6015db10 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 -- 172.21.15.1:0/1887542823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a6015db10 msgr2=0x7f1a600b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a6015db10 0x7f1a600b2a60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000af60 tx=0x7f1a5002b830 comp rx=0 tx=0).stop 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 -- 172.21.15.1:0/1887542823 shutdown_connections 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a6006e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 --2- 172.21.15.1:0/1887542823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a6015db10 0x7f1a600b2a60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 -- 172.21.15.1:0/1887542823 >> 172.21.15.1:0/1887542823 conn(0x7f1a600cc4b0 msgr2=0x7f1a600cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:18.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 -- 172.21.15.1:0/1887542823 shutdown_connections 2024-05-09T06:29:18.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.576+0000 7f1a65072640 1 -- 172.21.15.1:0/1887542823 wait complete. 2024-05-09T06:29:18.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.577+0000 7f1a65072640 1 Processor -- start 2024-05-09T06:29:18.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.577+0000 7f1a65072640 1 -- start start 2024-05-09T06:29:18.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6015db10 0x7f1a60156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a601575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6015edc0 con 0x7f1a600715d0 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6015ec40 con 0x7f1a6015db10 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a65072640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6006cd30 con 0x7f1a6015e6d0 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43336/0 (socket says 172.21.15.1:43336) 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.578+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/850845144 learned_addr learned my addr 172.21.15.1:0/850845144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 msgr2=0x7f1a601575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a601575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6015db10 msgr2=0x7f1a60156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6015db10 0x7f1a60156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 -- 172.21.15.1:0/850845144 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a600afee0 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a5ed9d640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a50036040 tx=0x7f1a50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50038040 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50043070 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a600b01c0 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000a620 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.579+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a600b0780 con 0x7f1a600715d0 2024-05-09T06:29:18.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.580+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a600752b0 con 0x7f1a600715d0 2024-05-09T06:29:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.580+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a50048460 con 0x7f1a600715d0 2024-05-09T06:29:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.581+0000 7f1a577fe640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 0x7f1a4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.581+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f1a500ab870 con 0x7f1a600715d0 2024-05-09T06:29:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.581+0000 7f1a5e59c640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 0x7f1a4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.581+0000 7f1a5e59c640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 0x7f1a4006e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a48000f60 tx=0x7f1a480079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.585+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a5007a070 con 0x7f1a600715d0 2024-05-09T06:29:18.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.769+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a600b0ba0 con 0x7f1a600715d0 2024-05-09T06:29:18.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.770+0000 7f1a577fe640 1 -- 172.21.15.1:0/850845144 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16803 (secure 0 0 0) 0x7f1a50079970 con 0x7f1a600715d0 2024-05-09T06:29:18.752 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:18.752 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":40,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:04.152378+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 msgr2=0x7f1a4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 0x7f1a4006e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a48000f60 tx=0x7f1a480079d0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 msgr2=0x7f1a600af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a50036040 tx=0x7f1a50004a40 comp rx=0 tx=0).stop 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 shutdown_connections 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a4006be30 0x7f1a4006e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015e6d0 0x7f1a601575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6015db10 0x7f1a60156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 --2- 172.21.15.1:0/850845144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a600715d0 0x7f1a600af6b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 >> 172.21.15.1:0/850845144 conn(0x7f1a600cc4b0 msgr2=0x7f1a600b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:18.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 shutdown_connections 2024-05-09T06:29:18.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.772+0000 7f1a65072640 1 -- 172.21.15.1:0/850845144 wait complete. 2024-05-09T06:29:18.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:28:41.915369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:28:42.435634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:29:18.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:29:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.891+0000 7ffb9c914640 1 Processor -- start 2024-05-09T06:29:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.891+0000 7ffb9c914640 1 -- start start 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9c914640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9c914640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb980b4de0 0x7ffb9815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9c914640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9806cf40 con 0x7ffb980b5260 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9c914640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffb98156980 con 0x7ffb980b4de0 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9c914640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9806d870 con 0x7ffb9815c4c0 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb980b4de0 0x7ffb9815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb980b4de0 0x7ffb9815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43342/0 (socket says 172.21.15.1:43342) 2024-05-09T06:29:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 -- 172.21.15.1:0/961849632 learned_addr learned my addr 172.21.15.1:0/961849632 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb9659c640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 -- 172.21.15.1:0/961849632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 msgr2=0x7ffb9806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 -- 172.21.15.1:0/961849632 >> v1:172.21.15.6:6789/0 conn(0x7ffb980b5260 legacy=0x7ffb98070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.892+0000 7ffb95d9b640 1 -- 172.21.15.1:0/961849632 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9806d220 con 0x7ffb980b4de0 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb95d9b640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b4de0 0x7ffb9815c8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b92460232ca33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb9559a640 1 -- 172.21.15.1:0/961849632 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c037070 con 0x7ffb980b4de0 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb9559a640 1 -- 172.21.15.1:0/961849632 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c032440 con 0x7ffb980b4de0 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb9559a640 1 -- 172.21.15.1:0/961849632 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c032810 con 0x7ffb980b4de0 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb9c914640 1 -- 172.21.15.1:0/961849632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b4de0 msgr2=0x7ffb9815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.893+0000 7ffb9c914640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b4de0 0x7ffb9815c8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009870 tx=0x7ffb8c02a270 comp rx=0 tx=0).stop 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 -- 172.21.15.1:0/961849632 shutdown_connections 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b4de0 0x7ffb9815c8a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 --2- 172.21.15.1:0/961849632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 -- 172.21.15.1:0/961849632 >> 172.21.15.1:0/961849632 conn(0x7ffb980cc4d0 msgr2=0x7ffb980cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:18.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 -- 172.21.15.1:0/961849632 shutdown_connections 2024-05-09T06:29:18.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.894+0000 7ffb9c914640 1 -- 172.21.15.1:0/961849632 wait complete. 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.895+0000 7ffb9c914640 1 Processor -- start 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.895+0000 7ffb9c914640 1 -- start start 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb980b4de0 0x7ffb98156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb980b2dc0 con 0x7ffb980b5260 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffb980b2c40 con 0x7ffb980b4de0 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9c914640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9815cde0 con 0x7ffb9815c4c0 2024-05-09T06:29:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43350/0 (socket says 172.21.15.1:43350) 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 -- 172.21.15.1:0/1244087748 learned_addr learned my addr 172.21.15.1:0/1244087748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 -- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 msgr2=0x7ffb9806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb96d9d640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb9659c640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb980b4de0 0x7ffb98156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 -- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb980b4de0 msgr2=0x7ffb98156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb980b4de0 0x7ffb98156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 -- 172.21.15.1:0/1244087748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb98157340 con 0x7ffb980b5260 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.896+0000 7ffb95d9b640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000c00 tx=0x7ffb8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.897+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c037040 con 0x7ffb980b5260 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.897+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c042070 con 0x7ffb980b5260 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.897+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb98157620 con 0x7ffb980b5260 2024-05-09T06:29:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.897+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c030880 con 0x7ffb980b5260 2024-05-09T06:29:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.897+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb98157be0 con 0x7ffb980b5260 2024-05-09T06:29:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.898+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb980752b0 con 0x7ffb980b5260 2024-05-09T06:29:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.898+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb8c030a20 con 0x7ffb980b5260 2024-05-09T06:29:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.899+0000 7ffb86ffd640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 0x7ffb7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.899+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7ffb8c0ab8d0 con 0x7ffb980b5260 2024-05-09T06:29:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.899+0000 7ffb9659c640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 0x7ffb7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.899+0000 7ffb9659c640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 0x7ffb7806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009970 tx=0x7ffb88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:18.903+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb8c07a0d0 con 0x7ffb980b5260 2024-05-09T06:29:19.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.084+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb9815e710 con 0x7ffb980b5260 2024-05-09T06:29:19.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.084+0000 7ffb86ffd640 1 -- 172.21.15.1:0/1244087748 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7ffb8c0799d0 con 0x7ffb980b5260 2024-05-09T06:29:19.066 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 msgr2=0x7ffb7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 0x7ffb7806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009970 tx=0x7ffb88008040 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 msgr2=0x7ffb98154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000c00 tx=0x7ffb8c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 shutdown_connections 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb7806be30 0x7ffb7806e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb9815c4c0 0x7ffb9806dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb980b5260 0x7ffb98154fc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 --2- 172.21.15.1:0/1244087748 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb980b4de0 0x7ffb98156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 >> 172.21.15.1:0/1244087748 conn(0x7ffb980cc4d0 msgr2=0x7ffb9815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 shutdown_connections 2024-05-09T06:29:19.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.086+0000 7ffb9c914640 1 -- 172.21.15.1:0/1244087748 wait complete. 2024-05-09T06:29:19.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:29:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.206+0000 7f0d614a5640 1 Processor -- start 2024-05-09T06:29:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.206+0000 7f0d614a5640 1 -- start start 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d614a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d614a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d614a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d5c0d2e20 con 0x7f0d5c0cf690 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d614a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0d04a0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d614a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0cfc00 con 0x7f0d5c0d0eb0 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43354/0 (socket says 172.21.15.1:43354) 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5affd640 1 -- 172.21.15.1:0/2697899920 learned_addr learned my addr 172.21.15.1:0/2697899920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5b7fe640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5affd640 1 -- 172.21.15.1:0/2697899920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 msgr2=0x7f0d5c0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.207+0000 7f0d5affd640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c0b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d5affd640 1 -- 172.21.15.1:0/2697899920 >> v1:172.21.15.6:6789/0 conn(0x7f0d5c0cf690 legacy=0x7f0d5c0d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d5affd640 1 -- 172.21.15.1:0/2697899920 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0cfee0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d5b7fe640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d5affd640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009870 tx=0x7f0d4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195c179bf4b402c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d59ffb640 1 -- 172.21.15.1:0/2697899920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c037070 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d59ffb640 1 -- 172.21.15.1:0/2697899920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c032440 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.208+0000 7f0d59ffb640 1 -- 172.21.15.1:0/2697899920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c032810 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 -- 172.21.15.1:0/2697899920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 msgr2=0x7f0d5c0d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009870 tx=0x7f0d4c02a210 comp rx=0 tx=0).stop 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 -- 172.21.15.1:0/2697899920 shutdown_connections 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 --2- 172.21.15.1:0/2697899920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0d48b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 -- 172.21.15.1:0/2697899920 >> 172.21.15.1:0/2697899920 conn(0x7f0d5c0d3d90 msgr2=0x7f0d5c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 -- 172.21.15.1:0/2697899920 shutdown_connections 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.209+0000 7f0d614a5640 1 -- 172.21.15.1:0/2697899920 wait complete. 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.210+0000 7f0d614a5640 1 Processor -- start 2024-05-09T06:29:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 -- start start 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d5c0cf690 0x7f0d5c0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c06ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b3f00 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b34a0 con 0x7f0d5c0cf690 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d614a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0b4080 con 0x7f0d5c0d0eb0 2024-05-09T06:29:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43366/0 (socket says 172.21.15.1:43366) 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.211+0000 7f0d5b7fe640 1 -- 172.21.15.1:0/84235320 learned_addr learned my addr 172.21.15.1:0/84235320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5affd640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d5c0cf690 0x7f0d5c0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5a7fc640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c06ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 -- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 msgr2=0x7f0d5c06ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c06ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 -- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d5c0cf690 msgr2=0x7f0d5c0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d5c0cf690 0x7f0d5c0cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 -- 172.21.15.1:0/84235320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0cb840 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d5b7fe640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400cf20 tx=0x7f0d5400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5401a070 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5400d840 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c0cbb20 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400dc30 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.212+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c0cc0e0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.213+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c0752b0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.214+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d5401d070 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.214+0000 7f0d437fe640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 0x7f0d4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.214+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f0d5408b8c0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.214+0000 7f0d5affd640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 0x7f0d4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.215+0000 7f0d5affd640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 0x7f0d4806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c034000 tx=0x7f0d4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.218+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d540599f0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.403+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d5c0b3040 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.404+0000 7f0d437fe640 1 -- 172.21.15.1:0/84235320 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f0d5405dcf0 con 0x7f0d5c15e3d0 2024-05-09T06:29:19.386 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:29:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 msgr2=0x7f0d4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 0x7f0d4806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c034000 tx=0x7f0d4c02e000 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 msgr2=0x7f0d5c0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400cf20 tx=0x7f0d5400a7d0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 shutdown_connections 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d4806be30 0x7f0d4806e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d5c15e3d0 0x7f0d5c0cae60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d5c0d0eb0 0x7f0d5c06ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 --2- 172.21.15.1:0/84235320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d5c0cf690 0x7f0d5c0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 >> 172.21.15.1:0/84235320 conn(0x7f0d5c0d3d90 msgr2=0x7f0d5c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 shutdown_connections 2024-05-09T06:29:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.406+0000 7f0d614a5640 1 -- 172.21.15.1:0/84235320 wait complete. 2024-05-09T06:29:19.399 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:29:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.527+0000 7f1571ca7640 1 Processor -- start 2024-05-09T06:29:19.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.527+0000 7f1571ca7640 1 -- start start 2024-05-09T06:29:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f1571ca7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f156c15e190 0x7f156c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f1571ca7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0d1c70 0x7f156c0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f1571ca7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156c161fe0 con 0x7f156c0b56d0 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f1571ca7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f156c15f580 con 0x7f156c15e190 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f1571ca7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f156c1622c0 con 0x7f156c0d1c70 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.528+0000 7f156b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f156c15e190 0x7f156c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f156c15e190 0x7f156c15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43372/0 (socket says 172.21.15.1:43372) 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 -- 172.21.15.1:0/1903791456 learned_addr learned my addr 172.21.15.1:0/1903791456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156affd640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0d1c70 0x7f156c0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 -- 172.21.15.1:0/1903791456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0d1c70 msgr2=0x7f156c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0d1c70 0x7f156c0d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 -- 172.21.15.1:0/1903791456 >> v1:172.21.15.6:6789/0 conn(0x7f156c0b56d0 legacy=0x7f156c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.529+0000 7f156b7fe640 1 -- 172.21.15.1:0/1903791456 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c1625a0 con 0x7f156c15e190 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c15e190 0x7f156c15f780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1560009870 tx=0x7f156002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffd89957cb1a82e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f156a7fc640 1 -- 172.21.15.1:0/1903791456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560037070 con 0x7f156c15e190 2024-05-09T06:29:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f156a7fc640 1 -- 172.21.15.1:0/1903791456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1560032440 con 0x7f156c15e190 2024-05-09T06:29:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f156a7fc640 1 -- 172.21.15.1:0/1903791456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560032810 con 0x7f156c15e190 2024-05-09T06:29:19.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f1571ca7640 1 -- 172.21.15.1:0/1903791456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c15e190 msgr2=0x7f156c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.530+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c15e190 0x7f156c15f780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1560009870 tx=0x7f156002a270 comp rx=0 tx=0).stop 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 -- 172.21.15.1:0/1903791456 shutdown_connections 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0d1c70 0x7f156c0d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1903791456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c15e190 0x7f156c15f780 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 -- 172.21.15.1:0/1903791456 >> 172.21.15.1:0/1903791456 conn(0x7f156c073340 msgr2=0x7f156c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 -- 172.21.15.1:0/1903791456 shutdown_connections 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.531+0000 7f1571ca7640 1 -- 172.21.15.1:0/1903791456 wait complete. 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.532+0000 7f1571ca7640 1 Processor -- start 2024-05-09T06:29:19.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.532+0000 7f1571ca7640 1 -- start start 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 0x7f156c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 0x7f156c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f156c0d4420 con 0x7f156c0d1c70 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f156c161c60 con 0x7f156c15e190 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f1571ca7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f156c0d45a0 con 0x7f156c0b56d0 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43376/0 (socket says 172.21.15.1:43376) 2024-05-09T06:29:19.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 -- 172.21.15.1:0/1508786638 learned_addr learned my addr 172.21.15.1:0/1508786638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 0x7f156c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156bfff640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 0x7f156c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 -- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 msgr2=0x7f156c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 0x7f156c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 -- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 msgr2=0x7f156c0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 0x7f156c0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 -- 172.21.15.1:0/1508786638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c1f5320 con 0x7f156c0d1c70 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 0x7f156c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156bfff640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 0x7f156c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.533+0000 7f156affd640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f155c00e980 tx=0x7f155c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.534+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c017070 con 0x7f156c0d1c70 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.534+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155c00ca00 con 0x7f156c0d1c70 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.534+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c00cdd0 con 0x7f156c0d1c70 2024-05-09T06:29:19.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.534+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156c1f5600 con 0x7f156c0d1c70 2024-05-09T06:29:19.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.534+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156c1f5bc0 con 0x7f156c0d1c70 2024-05-09T06:29:19.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.535+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156c0752b0 con 0x7f156c0d1c70 2024-05-09T06:29:19.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.535+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f155c01a070 con 0x7f156c0d1c70 2024-05-09T06:29:19.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.536+0000 7f1553fff640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 0x7f155806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.536+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f155c08b420 con 0x7f156c0d1c70 2024-05-09T06:29:19.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.536+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 0x7f155806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.536+0000 7f156b7fe640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 0x7f155806e320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1560034000 tx=0x7f156002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.544+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f155c059b80 con 0x7f156c0d1c70 2024-05-09T06:29:19.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.728+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f156c08e1c0 con 0x7f156c0d1c70 2024-05-09T06:29:19.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.728+0000 7f1553fff640 1 -- 172.21.15.1:0/1508786638 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f155c059480 con 0x7f156c0d1c70 2024-05-09T06:29:19.713 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:29:19.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 msgr2=0x7f155806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 0x7f155806e320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1560034000 tx=0x7f156002e000 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 msgr2=0x7f156c0d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f155c00e980 tx=0x7f155c00ee40 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 shutdown_connections 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f155806be60 0x7f155806e320 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f156c15e190 0x7f156c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f156c0d1c70 0x7f156c0d0790 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.730+0000 7f1571ca7640 1 --2- 172.21.15.1:0/1508786638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f156c0b56d0 0x7f156c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.731+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 >> 172.21.15.1:0/1508786638 conn(0x7f156c073340 msgr2=0x7f156c0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.731+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 shutdown_connections 2024-05-09T06:29:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.731+0000 7f1571ca7640 1 -- 172.21.15.1:0/1508786638 wait complete. 2024-05-09T06:29:19.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.848+0000 7fbfb4807640 1 Processor -- start 2024-05-09T06:29:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.848+0000 7fbfb4807640 1 -- start start 2024-05-09T06:29:19.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfb4807640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbfb01454b0 0x7fbfb00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfb4807640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb00b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfb4807640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfb00b5230 con 0x7fbfb00b4da0 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfb4807640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb0071c20 con 0x7fbfb01454b0 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfb4807640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb006eb40 con 0x7fbfb00b2a60 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfae59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbfb01454b0 0x7fbfb00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfaed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbfb00b4da0 0x7fbfb015f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53924/0 (socket says 172.21.15.1:53924) 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfaed9d640 1 -- 172.21.15.1:0/951140435 learned_addr learned my addr 172.21.15.1:0/951140435 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.849+0000 7fbfadd9b640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb00b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.850+0000 7fbfae59c640 1 -- 172.21.15.1:0/951140435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 msgr2=0x7fbfb00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.850+0000 7fbfae59c640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb00b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.850+0000 7fbfae59c640 1 -- 172.21.15.1:0/951140435 >> v1:172.21.15.1:6789/0 conn(0x7fbfb00b4da0 legacy=0x7fbfb015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.850+0000 7fbfae59c640 1 -- 172.21.15.1:0/951140435 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb006ee20 con 0x7fbfb01454b0 2024-05-09T06:29:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.850+0000 7fbfad59a640 1 -- 172.21.15.1:0/951140435 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861675011 0 0) 0x7fbfb00b5230 con 0x7fbfb00b4da0 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfae59c640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb01454b0 0x7fbfb00b4320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00097a0 tx=0x7fbfa002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2d88a7a81ae9e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfad59a640 1 -- 172.21.15.1:0/951140435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0037070 con 0x7fbfb01454b0 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfad59a640 1 -- 172.21.15.1:0/951140435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa0032440 con 0x7fbfb01454b0 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfad59a640 1 -- 172.21.15.1:0/951140435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0032810 con 0x7fbfb01454b0 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfb4807640 1 -- 172.21.15.1:0/951140435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb01454b0 msgr2=0x7fbfb00b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.851+0000 7fbfb4807640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb01454b0 0x7fbfb00b4320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00097a0 tx=0x7fbfa002a180 comp rx=0 tx=0).stop 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 -- 172.21.15.1:0/951140435 shutdown_connections 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb00b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 --2- 172.21.15.1:0/951140435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb01454b0 0x7fbfb00b4320 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 -- 172.21.15.1:0/951140435 >> 172.21.15.1:0/951140435 conn(0x7fbfb00cc4b0 msgr2=0x7fbfb00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 -- 172.21.15.1:0/951140435 shutdown_connections 2024-05-09T06:29:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.852+0000 7fbfb4807640 1 -- 172.21.15.1:0/951140435 wait complete. 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.853+0000 7fbfb4807640 1 Processor -- start 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.853+0000 7fbfb4807640 1 -- start start 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb01f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 0x7fbfb00cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0145890 con 0x7fbfb00b4da0 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb006dbc0 con 0x7fbfb01454b0 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb006e0e0 con 0x7fbfb00b2a60 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43402/0 (socket says 172.21.15.1:43402) 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 -- 172.21.15.1:0/659851141 learned_addr learned my addr 172.21.15.1:0/659851141 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfae59c640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb01f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfaed9d640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 0x7fbfb00cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 -- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 msgr2=0x7fbfb01f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb01f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 -- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 msgr2=0x7fbfb00cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 0x7fbfb00cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 -- 172.21.15.1:0/659851141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb00ce0e0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfaed9d640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 0x7fbfb00cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfadd9b640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d5f0 tx=0x7fbf9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980125f0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98019070 con 0x7fbfb00b4da0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb00ce3c0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.854+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980129d0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.855+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb00ce980 con 0x7fbfb00b4da0 2024-05-09T06:29:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.855+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb00752b0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.856+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf98012b70 con 0x7fbfb00b4da0 2024-05-09T06:29:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.856+0000 7fbfa6ffd640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 0x7fbf9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.857+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fbf9808b3d0 con 0x7fbfb00b4da0 2024-05-09T06:29:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.857+0000 7fbfae59c640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 0x7fbf9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.857+0000 7fbfae59c640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 0x7fbf9006e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0034000 tx=0x7fbfa002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:19.865+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf98059bd0 con 0x7fbfb00b4da0 2024-05-09T06:29:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.086+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfb0072fa0 con 0x7fbfb00b4da0 2024-05-09T06:29:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.087+0000 7fbfa6ffd640 1 -- 172.21.15.1:0/659851141 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7fbf980594d0 con 0x7fbfb00b4da0 2024-05-09T06:29:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:20.069 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:20.069 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 msgr2=0x7fbf9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 0x7fbf9006e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0034000 tx=0x7fbfa002e000 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 msgr2=0x7fbfb00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d5f0 tx=0x7fbf9800dab0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 shutdown_connections 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbf9006be30 0x7fbf9006e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfb01454b0 0x7fbfb00cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfb00b4da0 0x7fbfb00cd570 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 --2- 172.21.15.1:0/659851141 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfb00b2a60 0x7fbfb01f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 >> 172.21.15.1:0/659851141 conn(0x7fbfb00cc4b0 msgr2=0x7fbfb01415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 shutdown_connections 2024-05-09T06:29:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.089+0000 7fbfb4807640 1 -- 172.21.15.1:0/659851141 wait complete. 2024-05-09T06:29:20.082 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.204+0000 7fc44bf9d640 1 Processor -- start 2024-05-09T06:29:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.204+0000 7fc44bf9d640 1 -- start start 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44bf9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc4440d4940 0x7fc4440b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44bf9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4440d2b10 0x7fc4440b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44bf9d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440b33c0 con 0x7fc444161950 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44bf9d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440d4dd0 con 0x7fc4440d4940 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44bf9d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440b3660 con 0x7fc4440d2b10 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449538640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc4440d4940 0x7fc4440b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449d39640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc444161950 0x7fc4440d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53930/0 (socket says 172.21.15.1:53930) 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449d39640 1 -- 172.21.15.1:0/2418434058 learned_addr learned my addr 172.21.15.1:0/2418434058 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc44a53a640 1 --2- 172.21.15.1:0/2418434058 >> v2:172.21.15.6:3300/0 conn(0x7fc4440d2b10 0x7fc4440b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449538640 1 -- 172.21.15.1:0/2418434058 >> v2:172.21.15.6:3300/0 conn(0x7fc4440d2b10 msgr2=0x7fc4440b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449538640 1 --2- 172.21.15.1:0/2418434058 >> v2:172.21.15.6:3300/0 conn(0x7fc4440d2b10 0x7fc4440b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449538640 1 -- 172.21.15.1:0/2418434058 >> v1:172.21.15.1:6789/0 conn(0x7fc444161950 legacy=0x7fc4440d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc448d37640 1 -- 172.21.15.1:0/2418434058 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049203951 0 0) 0x7fc4440b33c0 con 0x7fc444161950 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.205+0000 7fc449538640 1 -- 172.21.15.1:0/2418434058 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440b3940 con 0x7fc4440d4940 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.206+0000 7fc449538640 1 --2- 172.21.15.1:0/2418434058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4440d4940 0x7fc4440b4140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc44408e540 tx=0x7fc43402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db646633cb559890 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.206+0000 7fc448d37640 1 -- 172.21.15.1:0/2418434058 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434037070 con 0x7fc4440d4940 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.206+0000 7fc448d37640 1 -- 172.21.15.1:0/2418434058 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434032440 con 0x7fc4440d4940 2024-05-09T06:29:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.206+0000 7fc448d37640 1 -- 172.21.15.1:0/2418434058 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434032810 con 0x7fc4440d4940 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2418434058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4440d4940 msgr2=0x7fc4440b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2418434058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4440d4940 0x7fc4440b4140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc44408e540 tx=0x7fc43402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2418434058 shutdown_connections 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2418434058 >> v2:172.21.15.6:3300/0 conn(0x7fc4440d2b10 0x7fc4440b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2418434058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4440d4940 0x7fc4440b4140 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2418434058 >> 172.21.15.1:0/2418434058 conn(0x7fc4440d3e20 msgr2=0x7fc4440d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2418434058 shutdown_connections 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.207+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2418434058 wait complete. 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.208+0000 7fc44bf9d640 1 Processor -- start 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.208+0000 7fc44bf9d640 1 -- start start 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 0x7fc4440a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 0x7fc44406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440d1100 con 0x7fc444161950 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440d0f80 con 0x7fc4440d2b10 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc44bf9d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440d1c00 con 0x7fc4441f4d10 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc449538640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc449538640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43424/0 (socket says 172.21.15.1:43424) 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.209+0000 7fc449538640 1 -- 172.21.15.1:0/2247295375 learned_addr learned my addr 172.21.15.1:0/2247295375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449d39640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 0x7fc4440a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 -- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 msgr2=0x7fc44406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc44a53a640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 0x7fc44406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 0x7fc44406cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 -- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 msgr2=0x7fc4440a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 0x7fc4440a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 -- 172.21.15.1:0/2247295375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440b6a40 con 0x7fc444161950 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc44a53a640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 0x7fc44406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449d39640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 0x7fc4440a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc449538640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc434035040 tx=0x7fc4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434037050 con 0x7fc444161950 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434038040 con 0x7fc444161950 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4441f4ae0 con 0x7fc444161950 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434004450 con 0x7fc444161950 2024-05-09T06:29:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.210+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4440b6fa0 con 0x7fc444161950 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.211+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4440752b0 con 0x7fc444161950 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.212+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc43403c6d0 con 0x7fc444161950 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.212+0000 7fc43a7fc640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 0x7fc42806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.212+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fc4340ab990 con 0x7fc444161950 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.212+0000 7fc449d39640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 0x7fc42806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.213+0000 7fc449d39640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 0x7fc42806e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc440000f60 tx=0x7fc440009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.220+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4440b6a40 con 0x7fc444161950 2024-05-09T06:29:20.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.406+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4440d21e0 con 0x7fc444161950 2024-05-09T06:29:20.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.407+0000 7fc43a7fc640 1 -- 172.21.15.1:0/2247295375 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16803 (secure 0 0 0) 0x7fc43407a220 con 0x7fc444161950 2024-05-09T06:29:20.389 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:20.389 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":40,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:04.152378+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:20.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 msgr2=0x7fc42806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 0x7fc42806e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc440000f60 tx=0x7fc440009040 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 msgr2=0x7fc4441f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc434035040 tx=0x7fc4340046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 shutdown_connections 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc42806be30 0x7fc42806e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4441f4d10 0x7fc44406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444161950 0x7fc4441f45a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 --2- 172.21.15.1:0/2247295375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4440d2b10 0x7fc4440a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 >> 172.21.15.1:0/2247295375 conn(0x7fc4440d3e20 msgr2=0x7fc4440b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 shutdown_connections 2024-05-09T06:29:20.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.409+0000 7fc44bf9d640 1 -- 172.21.15.1:0/2247295375 wait complete. 2024-05-09T06:29:20.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:29:20.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:20.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.524+0000 7fe229b7e640 1 Processor -- start 2024-05-09T06:29:20.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.524+0000 7fe229b7e640 1 -- start start 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2240b0080 0x7fe224071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe2240cbb60 0x7fe22415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe22415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe2240cbfe0 con 0x7fe2240b0080 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe2240d04a0 con 0x7fe2240cbb60 2024-05-09T06:29:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe229b7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe2240722c0 con 0x7fe22415fd70 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe2237fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2240b0080 0x7fe224071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe2237fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2240b0080 0x7fe224071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43426/0 (socket says 172.21.15.1:43426) 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.525+0000 7fe2237fe640 1 -- 172.21.15.1:0/3105210943 learned_addr learned my addr 172.21.15.1:0/3105210943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 -- 172.21.15.1:0/3105210943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 msgr2=0x7fe22415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe223fff640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe22415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe22415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 -- 172.21.15.1:0/3105210943 >> v2:172.21.15.6:3300/0 conn(0x7fe2240cbb60 msgr2=0x7fe22415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe222ffd640 1 --2- 172.21.15.1:0/3105210943 >> v2:172.21.15.6:3300/0 conn(0x7fe2240cbb60 0x7fe22415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 --2- 172.21.15.1:0/3105210943 >> v2:172.21.15.6:3300/0 conn(0x7fe2240cbb60 0x7fe22415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 -- 172.21.15.1:0/3105210943 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2240725a0 con 0x7fe2240b0080 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.526+0000 7fe2237fe640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe224071ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe224074b60 tx=0x7fe21402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab6fa2abaeb7035 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe2227fc640 1 -- 172.21.15.1:0/3105210943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214037070 con 0x7fe2240b0080 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe2227fc640 1 -- 172.21.15.1:0/3105210943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe214032440 con 0x7fe2240b0080 2024-05-09T06:29:20.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe2227fc640 1 -- 172.21.15.1:0/3105210943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214032810 con 0x7fe2240b0080 2024-05-09T06:29:20.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 -- 172.21.15.1:0/3105210943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 msgr2=0x7fe224071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe224071ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe224074b60 tx=0x7fe21402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 -- 172.21.15.1:0/3105210943 shutdown_connections 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe22415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 --2- 172.21.15.1:0/3105210943 >> v2:172.21.15.6:3300/0 conn(0x7fe2240cbb60 0x7fe22415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 --2- 172.21.15.1:0/3105210943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe224071ce0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 -- 172.21.15.1:0/3105210943 >> 172.21.15.1:0/3105210943 conn(0x7fe2240d0b50 msgr2=0x7fe2240d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 -- 172.21.15.1:0/3105210943 shutdown_connections 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.527+0000 7fe229b7e640 1 -- 172.21.15.1:0/3105210943 wait complete. 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.528+0000 7fe229b7e640 1 Processor -- start 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 -- start start 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 0x7fe2240c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe224160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe22415e7c0 con 0x7fe2240b0080 2024-05-09T06:29:20.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe224072b60 con 0x7fe2240cbb60 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe229b7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe22415f220 con 0x7fe22415fd70 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43432/0 (socket says 172.21.15.1:43432) 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.529+0000 7fe2237fe640 1 -- 172.21.15.1:0/1824770110 learned_addr learned my addr 172.21.15.1:0/1824770110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe223fff640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe224160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe222ffd640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 0x7fe2240c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 -- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 msgr2=0x7fe224160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe224160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 -- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 msgr2=0x7fe2240c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 0x7fe2240c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 -- 172.21.15.1:0/1824770110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2240c5370 con 0x7fe2240b0080 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe223fff640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe224160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe222ffd640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 0x7fe2240c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:20.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe2237fe640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe214000c00 tx=0x7fe2140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214037040 con 0x7fe2240b0080 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe214042070 con 0x7fe2240b0080 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2240c5650 con 0x7fe2240b0080 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214030bb0 con 0x7fe2240b0080 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.530+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2240c5c10 con 0x7fe2240b0080 2024-05-09T06:29:20.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.531+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2240752b0 con 0x7fe2240b0080 2024-05-09T06:29:20.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.531+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe214047460 con 0x7fe2240b0080 2024-05-09T06:29:20.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.532+0000 7fe20bfff640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 0x7fe21006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.532+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fe2140abc10 con 0x7fe2240b0080 2024-05-09T06:29:20.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.532+0000 7fe222ffd640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 0x7fe21006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.532+0000 7fe222ffd640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 0x7fe21006e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe22406dc20 tx=0x7fe218008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.536+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe214045360 con 0x7fe2240b0080 2024-05-09T06:29:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.758+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2240c5e50 con 0x7fe2240b0080 2024-05-09T06:29:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.759+0000 7fe20bfff640 1 -- 172.21.15.1:0/1824770110 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7fe21407a400 con 0x7fe2240b0080 2024-05-09T06:29:20.740 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:20.741 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 msgr2=0x7fe21006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 0x7fe21006e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe22406dc20 tx=0x7fe218008040 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 msgr2=0x7fe2241e7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe214000c00 tx=0x7fe2140046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 shutdown_connections 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe21006be30 0x7fe21006e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe22415fd70 0x7fe224160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2240cbb60 0x7fe2240c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 --2- 172.21.15.1:0/1824770110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2240b0080 0x7fe2241e7360 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 >> 172.21.15.1:0/1824770110 conn(0x7fe2240d0b50 msgr2=0x7fe2240d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 shutdown_connections 2024-05-09T06:29:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.761+0000 7fe229b7e640 1 -- 172.21.15.1:0/1824770110 wait complete. 2024-05-09T06:29:20.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:20.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.875+0000 7f775359e640 1 Processor -- start 2024-05-09T06:29:20.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.876+0000 7f775359e640 1 -- start start 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.876+0000 7f775359e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f774c0c6860 0x7f774c099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.876+0000 7f775359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0b2aa0 0x7f774c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.876+0000 7f775359e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c099930 con 0x7f774c0c6c40 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775359e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f774c0b1e20 con 0x7f774c0c6860 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f774c15ee90 con 0x7f774c0b2aa0 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f774c0c6860 0x7f774c099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f774c0c6860 0x7f774c099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43444/0 (socket says 172.21.15.1:43444) 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 -- 172.21.15.1:0/1881802532 learned_addr learned my addr 172.21.15.1:0/1881802532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f7750b39640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0b2aa0 0x7f774c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 -- 172.21.15.1:0/1881802532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0b2aa0 msgr2=0x7f774c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0b2aa0 0x7f774c15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 -- 172.21.15.1:0/1881802532 >> v1:172.21.15.6:6789/0 conn(0x7f774c0c6c40 legacy=0x7f774c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.877+0000 7f775133a640 1 -- 172.21.15.1:0/1881802532 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c162930 con 0x7f774c0c6860 2024-05-09T06:29:20.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775133a640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0c6860 0x7f774c099b30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e037f558f2bced35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f7743fff640 1 -- 172.21.15.1:0/1881802532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748037070 con 0x7f774c0c6860 2024-05-09T06:29:20.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f7743fff640 1 -- 172.21.15.1:0/1881802532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748032440 con 0x7f774c0c6860 2024-05-09T06:29:20.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f7743fff640 1 -- 172.21.15.1:0/1881802532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748032810 con 0x7f774c0c6860 2024-05-09T06:29:20.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 -- 172.21.15.1:0/1881802532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0c6860 msgr2=0x7f774c099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0c6860 0x7f774c099b30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774802a180 comp rx=0 tx=0).stop 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 -- 172.21.15.1:0/1881802532 shutdown_connections 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0b2aa0 0x7f774c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 --2- 172.21.15.1:0/1881802532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0c6860 0x7f774c099b30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.878+0000 7f775359e640 1 -- 172.21.15.1:0/1881802532 >> 172.21.15.1:0/1881802532 conn(0x7f774c0b1fb0 msgr2=0x7f774c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.879+0000 7f775359e640 1 -- 172.21.15.1:0/1881802532 shutdown_connections 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.879+0000 7f775359e640 1 -- 172.21.15.1:0/1881802532 wait complete. 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 Processor -- start 2024-05-09T06:29:20.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 -- start start 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f774c0c6860 0x7f774c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0c6c40 0x7f774c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f774c15eb20 con 0x7f774c0b2aa0 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f774c15fa30 con 0x7f774c0c6860 2024-05-09T06:29:20.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.880+0000 7f775359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0c70d0 con 0x7f774c0c6c40 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43456/0 (socket says 172.21.15.1:43456) 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 -- 172.21.15.1:0/3100852910 learned_addr learned my addr 172.21.15.1:0/3100852910 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f7751b3b640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0c6c40 0x7f774c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f7750b39640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f774c0c6860 0x7f774c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 -- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0c6c40 msgr2=0x7f774c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0c6c40 0x7f774c0b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 -- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f774c0c6860 msgr2=0x7f774c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f774c0c6860 0x7f774c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 -- 172.21.15.1:0/3100852910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0b4d40 con 0x7f774c0b2aa0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775133a640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7748000c00 tx=0x7f77480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748037040 con 0x7f774c0b2aa0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748042070 con 0x7f774c0b2aa0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c0b5020 con 0x7f774c0b2aa0 2024-05-09T06:29:20.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.881+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748030bd0 con 0x7f774c0b2aa0 2024-05-09T06:29:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.882+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c0b55e0 con 0x7f774c0b2aa0 2024-05-09T06:29:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.883+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c0752b0 con 0x7f774c0b2aa0 2024-05-09T06:29:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.883+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7748047460 con 0x7f774c0b2aa0 2024-05-09T06:29:20.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.883+0000 7f7741ffb640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 0x7f773006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.884+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f77480abc60 con 0x7f774c0b2aa0 2024-05-09T06:29:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.884+0000 7f7750b39640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 0x7f773006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:20.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.884+0000 7f7750b39640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 0x7f773006e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f773c006fd0 tx=0x7f773c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:20.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:20.888+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f774c099930 con 0x7f774c0b2aa0 2024-05-09T06:29:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.112+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f774c06ce30 con 0x7f774c0b2aa0 2024-05-09T06:29:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.113+0000 7f7741ffb640 1 -- 172.21.15.1:0/3100852910 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5163 (secure 0 0 0) 0x7f7748045360 con 0x7f774c0b2aa0 2024-05-09T06:29:21.095 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:21.095 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:28:43.462346+0000","modified":"2024-05-09T06:28:46.479534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/642815597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":642815597},{"type":"v1","addr":"172.21.15.6:6834","nonce":642815597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":2}]} 2024-05-09T06:29:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 18 2024-05-09T06:29:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 msgr2=0x7f773006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 0x7f773006e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f773c006fd0 tx=0x7f773c008040 comp rx=0 tx=0).stop 2024-05-09T06:29:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 msgr2=0x7f774c1568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7748000c00 tx=0x7f77480046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 shutdown_connections 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f773006be30 0x7f773006e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f774c0c6c40 0x7f774c0b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f774c0c6860 0x7f774c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 --2- 172.21.15.1:0/3100852910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f774c0b2aa0 0x7f774c1568f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 >> 172.21.15.1:0/3100852910 conn(0x7f774c0b1fb0 msgr2=0x7f774c0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 shutdown_connections 2024-05-09T06:29:21.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.115+0000 7f775359e640 1 -- 172.21.15.1:0/3100852910 wait complete. 2024-05-09T06:29:21.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:21.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.230+0000 7f6997dc4640 1 Processor -- start 2024-05-09T06:29:21.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.230+0000 7f6997dc4640 1 -- start start 2024-05-09T06:29:21.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6997dc4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 0x7f699006f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6997dc4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f699015e5b0 0x7f699006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6997dc4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f699006c2f0 con 0x7f69900d2890 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6997dc4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69900cc3c0 con 0x7f699015e5b0 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6997dc4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f69900ccc60 con 0x7f699015e1d0 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f699015e5b0 0x7f699006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f699015e5b0 0x7f699006fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43460/0 (socket says 172.21.15.1:43460) 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 -- 172.21.15.1:0/1072417523 learned_addr learned my addr 172.21.15.1:0/1072417523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6995b60640 1 --2- 172.21.15.1:0/1072417523 >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 0x7f699006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 -- 172.21.15.1:0/1072417523 >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 msgr2=0x7f699006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 --2- 172.21.15.1:0/1072417523 >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 0x7f699006f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 -- 172.21.15.1:0/1072417523 >> v1:172.21.15.6:6789/0 conn(0x7f69900d2890 legacy=0x7f69900cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f699535f640 1 -- 172.21.15.1:0/1072417523 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699006f500 con 0x7f699015e5b0 2024-05-09T06:29:21.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.231+0000 7f6995b60640 1 --2- 172.21.15.1:0/1072417523 >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 0x7f699006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:21.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.232+0000 7f699535f640 1 --2- 172.21.15.1:0/1072417523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e5b0 0x7f699006fdd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f69900704a0 tx=0x7f698002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3f03d74933906 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.232+0000 7f6994b5e640 1 -- 172.21.15.1:0/1072417523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980037070 con 0x7f699015e5b0 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.232+0000 7f6994b5e640 1 -- 172.21.15.1:0/1072417523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6980032440 con 0x7f699015e5b0 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.232+0000 7f6994b5e640 1 -- 172.21.15.1:0/1072417523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980032810 con 0x7f699015e5b0 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 -- 172.21.15.1:0/1072417523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e5b0 msgr2=0x7f699006fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 --2- 172.21.15.1:0/1072417523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e5b0 0x7f699006fdd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f69900704a0 tx=0x7f698002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 -- 172.21.15.1:0/1072417523 shutdown_connections 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 --2- 172.21.15.1:0/1072417523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e5b0 0x7f699006fdd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 --2- 172.21.15.1:0/1072417523 >> v2:172.21.15.6:3300/0 conn(0x7f699015e1d0 0x7f699006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 -- 172.21.15.1:0/1072417523 >> 172.21.15.1:0/1072417523 conn(0x7f69900d1d70 msgr2=0x7f69900d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 -- 172.21.15.1:0/1072417523 shutdown_connections 2024-05-09T06:29:21.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.233+0000 7f6997dc4640 1 -- 172.21.15.1:0/1072417523 wait complete. 2024-05-09T06:29:21.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.234+0000 7f6997dc4640 1 Processor -- start 2024-05-09T06:29:21.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.234+0000 7f6997dc4640 1 -- start start 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69900d2890 0x7f69901f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 0x7f699009f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f699006f1b0 con 0x7f699015e1d0 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69900cc8f0 con 0x7f69900d2890 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6997dc4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69900d0fc0 con 0x7f699015e5b0 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43472/0 (socket says 172.21.15.1:43472) 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 -- 172.21.15.1:0/473889156 learned_addr learned my addr 172.21.15.1:0/473889156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6995b60640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69900d2890 0x7f69901f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6996361640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 0x7f699009f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 -- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 msgr2=0x7f699009f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 0x7f699009f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 -- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69900d2890 msgr2=0x7f69901f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69900d2890 0x7f69901f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f699535f640 1 -- 172.21.15.1:0/473889156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69900c61e0 con 0x7f699015e1d0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.235+0000 7f6996361640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 0x7f699009f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f699535f640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f698002a1e0 tx=0x7f69800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980037040 con 0x7f699015e1d0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6980042070 con 0x7f699015e1d0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980030bb0 con 0x7f699015e1d0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69900c64c0 con 0x7f699015e1d0 2024-05-09T06:29:21.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.236+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699015fa40 con 0x7f699015e1d0 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.237+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699015f550 con 0x7f699015e1d0 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.237+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6980047460 con 0x7f699015e1d0 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.238+0000 7f69867fc640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 0x7f697406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.238+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f69800abd10 con 0x7f699015e1d0 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.238+0000 7f6995b60640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 0x7f697406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.238+0000 7f6995b60640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 0x7f697406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f698c00a990 tx=0x7f698c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.242+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f69800b1020 con 0x7f699015e1d0 2024-05-09T06:29:21.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.426+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69900752b0 con 0x7f699015e1d0 2024-05-09T06:29:21.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.427+0000 7f69867fc640 1 -- 172.21.15.1:0/473889156 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16803 (secure 0 0 0) 0x7f699006c2f0 con 0x7f699015e1d0 2024-05-09T06:29:21.409 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:21.409 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":40,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:04.152378+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:28:41.915369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-09T06:28:42.435634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 msgr2=0x7f697406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 0x7f697406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f698c00a990 tx=0x7f698c008040 comp rx=0 tx=0).stop 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 msgr2=0x7f699015fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f698002a1e0 tx=0x7f69800046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 shutdown_connections 2024-05-09T06:29:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697406be30 0x7f697406e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f699015e5b0 0x7f699009f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f699015e1d0 0x7f699015fee0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 --2- 172.21.15.1:0/473889156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69900d2890 0x7f69901f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 >> 172.21.15.1:0/473889156 conn(0x7f69900d1d70 msgr2=0x7f69900d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 shutdown_connections 2024-05-09T06:29:21.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.429+0000 7f6997dc4640 1 -- 172.21.15.1:0/473889156 wait complete. 2024-05-09T06:29:21.422 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:29:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.546+0000 7f96d5072640 1 Processor -- start 2024-05-09T06:29:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.546+0000 7f96d5072640 1 -- start start 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96d5072640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96d015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96d5072640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d0099930 con 0x7f96d00c6860 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96d5072640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d0162930 con 0x7f96d0099b30 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96d5072640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96d00b1e20 con 0x7f96d00b2aa0 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96ced9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f96d00c6860 0x7f96d00c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53942/0 (socket says 172.21.15.1:53942) 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96ced9d640 1 -- 172.21.15.1:0/1377554620 learned_addr learned my addr 172.21.15.1:0/1377554620 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3549020523 0 0) 0x7f96c0001490 con 0x7f96d00c6860 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96bc003520 con 0x7f96d00c6860 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.547+0000 7f96cf59e640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96d015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4155401040 0 0) 0x7f96bc003520 con 0x7f96d00c6860 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 msgr2=0x7f96d015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96d015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 >> v1:172.21.15.6:6789/0 conn(0x7f96d0099b30 legacy=0x7f96d015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d0162c10 con 0x7f96d00c6860 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f96c0001bc0 con 0x7f96d00c6860 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 >> v1:172.21.15.1:6789/0 conn(0x7f96d00c6860 legacy=0x7f96d00c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc005520 con 0x7f96bc0039e0 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc00bf00 con 0x7f96bc004800 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc00c080 con 0x7f96d00b2aa0 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ce59c640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96cf59e640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 -- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 msgr2=0x7f96bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d00b2aa0 0x7f96bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 -- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96bc004800 msgr2=0x7f96bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ced9d640 1 -- 172.21.15.1:0/1377554620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc003520 con 0x7f96bc0039e0 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.548+0000 7f96ce59c640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96bc003e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f96c000aa50 tx=0x7f96bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921fb3bef62713a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000a670 con 0x7f96bc0039e0 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c002f070 con 0x7f96bc0039e0 2024-05-09T06:29:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96cdd9b640 1 -- 172.21.15.1:0/1377554620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0034470 con 0x7f96bc0039e0 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96d5072640 1 -- 172.21.15.1:0/1377554620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 msgr2=0x7f96bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96d5072640 1 --2- 172.21.15.1:0/1377554620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96bc003e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f96c000aa50 tx=0x7f96bc005830 comp rx=0 tx=0).stop 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96cf59e640 1 -- 172.21.15.1:0/1377554620 reap_dead start 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.549+0000 7f96d5072640 1 -- 172.21.15.1:0/1377554620 shutdown_connections 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.550+0000 7f96d5072640 1 -- 172.21.15.1:0/1377554620 >> 172.21.15.1:0/1377554620 conn(0x7f96d00b1fb0 msgr2=0x7f96d00b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.550+0000 7f96d5072640 1 -- 172.21.15.1:0/1377554620 shutdown_connections 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.550+0000 7f96d5072640 1 -- 172.21.15.1:0/1377554620 wait complete. 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.551+0000 7f96d5072640 1 Processor -- start 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.551+0000 7f96d5072640 1 -- start start 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d0146d70 0x7f96d00cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d01461c0 0x7f96d0146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96d015fa30 con 0x7f96bc0039e0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96d015ee90 con 0x7f96d01461c0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96d5072640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96d015e4b0 con 0x7f96d0146d70 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43500/0 (socket says 172.21.15.1:43500) 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 -- 172.21.15.1:0/1472977641 learned_addr learned my addr 172.21.15.1:0/1472977641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96cf59e640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d01461c0 0x7f96d0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ce59c640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d0146d70 0x7f96d00cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 -- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d0146d70 msgr2=0x7f96d00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d0146d70 0x7f96d00cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 -- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d01461c0 msgr2=0x7f96d0146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d01461c0 0x7f96d0146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 -- 172.21.15.1:0/1472977641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d0146b80 con 0x7f96bc0039e0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.552+0000 7f96ced9d640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f96c000aa50 tx=0x7f96c0001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.553+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000a670 con 0x7f96bc0039e0 2024-05-09T06:29:21.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.553+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c002ba70 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.553+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c002be20 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.553+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d00b3660 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.553+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d00b3380 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.554+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d00752b0 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.554+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96c0034470 con 0x7f96bc0039e0 2024-05-09T06:29:21.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.555+0000 7f96c77fe640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 0x7f96b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.555+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f96c00ab470 con 0x7f96bc0039e0 2024-05-09T06:29:21.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.555+0000 7f96ce59c640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 0x7f96b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.555+0000 7f96ce59c640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 0x7f96b006e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96d0157db0 tx=0x7f96b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.559+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96d0099930 con 0x7f96bc0039e0 2024-05-09T06:29:21.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.779+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f96d006ce30 con 0x7f96bc0039e0 2024-05-09T06:29:21.761 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:29:21.779+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:29:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.781+0000 7f96c77fe640 1 -- 172.21.15.1:0/1472977641 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f96c0079be0 con 0x7f96bc0039e0 2024-05-09T06:29:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 msgr2=0x7f96b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 0x7f96b006e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96d0157db0 tx=0x7f96b8009040 comp rx=0 tx=0).stop 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 msgr2=0x7f96d00b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f96c000aa50 tx=0x7f96c0001f90 comp rx=0 tx=0).stop 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 shutdown_connections 2024-05-09T06:29:21.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96b006be60 0x7f96b006e320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d01461c0 0x7f96d0146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d0146d70 0x7f96d00cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 --2- 172.21.15.1:0/1472977641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96bc0039e0 0x7f96d00b5a10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 >> 172.21.15.1:0/1472977641 conn(0x7f96d00b1fb0 msgr2=0x7f96d00cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 shutdown_connections 2024-05-09T06:29:21.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.784+0000 7f96d5072640 1 -- 172.21.15.1:0/1472977641 wait complete. 2024-05-09T06:29:21.770 INFO:tasks.ceph.mds.d.smithi006.stderr: -6063> 2024-05-09T06:28:48.899+0000 7f92c8edc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:29:21.777 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:29:21.803 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:29:21.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.901+0000 7fcd6b69a640 1 Processor -- start 2024-05-09T06:29:21.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.901+0000 7fcd6b69a640 1 -- start start 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd6b69a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcd640c41e0 0x7fcd640ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd6b69a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd64072050 con 0x7fcd6406b1e0 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd6b69a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd64072330 con 0x7fcd640d06f0 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd6b69a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcd64072610 con 0x7fcd640c41e0 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd69c37640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcd6406b1e0 0x7fcd640cee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53954/0 (socket says 172.21.15.1:53954) 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd69c37640 1 -- 172.21.15.1:0/1902792775 learned_addr learned my addr 172.21.15.1:0/1902792775 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd68c35640 1 --2- 172.21.15.1:0/1902792775 >> v2:172.21.15.6:3300/0 conn(0x7fcd640c41e0 0x7fcd640ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.902+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267601989 0 0) 0x7fcd64072050 con 0x7fcd6406b1e0 2024-05-09T06:29:21.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd50003520 con 0x7fcd6406b1e0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642470029 0 0) 0x7fcd50003520 con 0x7fcd6406b1e0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 >> v2:172.21.15.6:3300/0 conn(0x7fcd640c41e0 msgr2=0x7fcd640ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 --2- 172.21.15.1:0/1902792775 >> v2:172.21.15.6:3300/0 conn(0x7fcd640c41e0 0x7fcd640ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 >> v1:172.21.15.6:6789/0 conn(0x7fcd640d06f0 legacy=0x7fcd640cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd640728f0 con 0x7fcd6406b1e0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fcd5c0019f0 con 0x7fcd6406b1e0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 >> v1:172.21.15.1:6789/0 conn(0x7fcd6406b1e0 legacy=0x7fcd640cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd500039b0 0x7fcd50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd50004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd50005080 0x7fcd5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64072610 con 0x7fcd500043b0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5000bfe0 con 0x7fcd500039b0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5000c1e0 con 0x7fcd50005080 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd50004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd69436640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd500039b0 0x7fcd50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd69c37640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd50005080 0x7fcd5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 -- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd50005080 msgr2=0x7fcd5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd50005080 0x7fcd5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 -- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd500039b0 msgr2=0x7fcd50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd500039b0 0x7fcd50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd68c35640 1 -- 172.21.15.1:0/1902792775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd50003520 con 0x7fcd500043b0 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.903+0000 7fcd69c37640 1 -- 172.21.15.1:0/1902792775 reap_dead start 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.904+0000 7fcd68c35640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd50004880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ac0 tx=0x7fcd50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa52aadb898d110 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.904+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54036070 con 0x7fcd500043b0 2024-05-09T06:29:21.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.904+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd54031400 con 0x7fcd500043b0 2024-05-09T06:29:21.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.904+0000 7fcd5bfff640 1 -- 172.21.15.1:0/1902792775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54009690 con 0x7fcd500043b0 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 -- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 msgr2=0x7fcd50004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd50004880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ac0 tx=0x7fcd50005830 comp rx=0 tx=0).stop 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 -- 172.21.15.1:0/1902792775 shutdown_connections 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/1902792775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd50004880 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 -- 172.21.15.1:0/1902792775 >> 172.21.15.1:0/1902792775 conn(0x7fcd640d0b50 msgr2=0x7fcd640d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 -- 172.21.15.1:0/1902792775 shutdown_connections 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.905+0000 7fcd6b69a640 1 -- 172.21.15.1:0/1902792775 wait complete. 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.906+0000 7fcd6b69a640 1 Processor -- start 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.906+0000 7fcd6b69a640 1 -- start start 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 0x7fcd6415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 0x7fcd640c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcd640cc860 con 0x7fcd500043b0 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd640cc6e0 con 0x7fcd640b1220 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd6b69a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd640cc9e0 con 0x7fcd6415bc80 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43516/0 (socket says 172.21.15.1:43516) 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 -- 172.21.15.1:0/524298854 learned_addr learned my addr 172.21.15.1:0/524298854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 -- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 msgr2=0x7fcd640c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69c37640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 0x7fcd640c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd68c35640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 0x7fcd6415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 0x7fcd640c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 -- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 msgr2=0x7fcd6415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 0x7fcd6415b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 -- 172.21.15.1:0/524298854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd640c7690 con 0x7fcd500043b0 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69c37640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 0x7fcd640c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd68c35640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 0x7fcd6415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.907+0000 7fcd69436640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f7f0 tx=0x7fcd6000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.908+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000fea0 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.908+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd60018070 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.908+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd640b0780 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.908+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000d730 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.908+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd640c73b0 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.909+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd640752b0 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.909+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd6000d8d0 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.910+0000 7fcd59ffb640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 0x7fcd4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.910+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fcd6008af30 con 0x7fcd500043b0 2024-05-09T06:29:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.910+0000 7fcd68c35640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 0x7fcd4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.910+0000 7fcd68c35640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 0x7fcd4806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd54033000 tx=0x7fcd54008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:21.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:21.914+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd60059770 con 0x7fcd500043b0 2024-05-09T06:29:22.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.134+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcd640a2040 con 0x7fcd500043b0 2024-05-09T06:29:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.782+0000 7fcd59ffb640 1 -- 172.21.15.1:0/524298854 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fcd60059070 con 0x7fcd500043b0 2024-05-09T06:29:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 msgr2=0x7fcd4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 0x7fcd4806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd54033000 tx=0x7fcd54008030 comp rx=0 tx=0).stop 2024-05-09T06:29:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 msgr2=0x7fcd640ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f7f0 tx=0x7fcd6000fcb0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 shutdown_connections 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd4806be30 0x7fcd4806e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd6415bc80 0x7fcd640c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd640b1220 0x7fcd6415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 --2- 172.21.15.1:0/524298854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd500043b0 0x7fcd640ca810 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 >> 172.21.15.1:0/524298854 conn(0x7fcd640d0b50 msgr2=0x7fcd6415ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 shutdown_connections 2024-05-09T06:29:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.785+0000 7fcd6b69a640 1 -- 172.21.15.1:0/524298854 wait complete. 2024-05-09T06:29:22.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:29:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.914+0000 7f2df8f30640 1 Processor -- start 2024-05-09T06:29:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.914+0000 7f2df8f30640 1 -- start start 2024-05-09T06:29:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df8f30640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df8f30640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2df40b2ed0 0x7f2df415e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df8f30640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df40d1cc0 con 0x7f2df40d2a10 2024-05-09T06:29:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df8f30640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2df415e870 con 0x7f2df40b2ed0 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df8f30640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40d4d10 con 0x7f2df40d2630 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2df40b2ed0 0x7f2df415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2df40b2ed0 0x7f2df415e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43528/0 (socket says 172.21.15.1:43528) 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 -- 172.21.15.1:0/1703425618 learned_addr learned my addr 172.21.15.1:0/1703425618 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 -- 172.21.15.1:0/1703425618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 msgr2=0x7f2df415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df259c640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df415ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 -- 172.21.15.1:0/1703425618 >> v1:172.21.15.6:6789/0 conn(0x7f2df40d2a10 legacy=0x7f2df40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df1d9b640 1 -- 172.21.15.1:0/1703425618 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40d1f80 con 0x7f2df40b2ed0 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.915+0000 7f2df259c640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df415ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.916+0000 7f2df1d9b640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40b2ed0 0x7f2df415e330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009870 tx=0x7f2de802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bee66dcd19a247 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.916+0000 7f2df159a640 1 -- 172.21.15.1:0/1703425618 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037070 con 0x7f2df40b2ed0 2024-05-09T06:29:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.916+0000 7f2df159a640 1 -- 172.21.15.1:0/1703425618 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de8032440 con 0x7f2df40b2ed0 2024-05-09T06:29:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.916+0000 7f2df159a640 1 -- 172.21.15.1:0/1703425618 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032810 con 0x7f2df40b2ed0 2024-05-09T06:29:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 -- 172.21.15.1:0/1703425618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40b2ed0 msgr2=0x7f2df415e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40b2ed0 0x7f2df415e330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009870 tx=0x7f2de802a210 comp rx=0 tx=0).stop 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 -- 172.21.15.1:0/1703425618 shutdown_connections 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40b2ed0 0x7f2df415e330 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 --2- 172.21.15.1:0/1703425618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df415ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 -- 172.21.15.1:0/1703425618 >> 172.21.15.1:0/1703425618 conn(0x7f2df40d3ee0 msgr2=0x7f2df40d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 -- 172.21.15.1:0/1703425618 shutdown_connections 2024-05-09T06:29:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.917+0000 7f2df8f30640 1 -- 172.21.15.1:0/1703425618 wait complete. 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.918+0000 7f2df8f30640 1 Processor -- start 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.918+0000 7f2df8f30640 1 -- start start 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2df40b2ed0 0x7f2df40b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40cfbb0 con 0x7f2df40d2a10 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40d4a00 con 0x7f2df40b2ed0 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df8f30640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40cfd30 con 0x7f2df40d2630 2024-05-09T06:29:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43544/0 (socket says 172.21.15.1:43544) 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 -- 172.21.15.1:0/742361654 learned_addr learned my addr 172.21.15.1:0/742361654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df1d9b640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df259c640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2df40b2ed0 0x7f2df40b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 -- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 msgr2=0x7f2df4156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df4156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 -- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2df40b2ed0 msgr2=0x7f2df40b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2df40b2ed0 0x7f2df40b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.919+0000 7f2df2d9d640 1 -- 172.21.15.1:0/742361654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40b97b0 con 0x7f2df40d2a10 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2df2d9d640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00c7b0 tx=0x7f2dec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec004b60 con 0x7f2df40d2a10 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec004d00 con 0x7f2df40d2a10 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df40cccf0 con 0x7f2df40d2a10 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00aa30 con 0x7f2df40d2a10 2024-05-09T06:29:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.920+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df40cd2b0 con 0x7f2df40d2a10 2024-05-09T06:29:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.921+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df4075320 con 0x7f2df40d2a10 2024-05-09T06:29:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.921+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dec00abd0 con 0x7f2df40d2a10 2024-05-09T06:29:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.922+0000 7f2de2ffd640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 0x7f2dd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.922+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f2dec08b210 con 0x7f2df40d2a10 2024-05-09T06:29:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.922+0000 7f2df259c640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 0x7f2dd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.922+0000 7f2df259c640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 0x7f2dd406e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2df40a1db0 tx=0x7f2de4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:22.926+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2dec020070 con 0x7f2df40d2a10 2024-05-09T06:29:23.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.114+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2df40ca420 con 0x7f2df40d2a10 2024-05-09T06:29:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.787+0000 7f2de2ffd640 1 -- 172.21.15.1:0/742361654 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7f2dec059250 con 0x7f2df40d2a10 2024-05-09T06:29:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:29:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.791+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 msgr2=0x7f2dd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.791+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 0x7f2dd406e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2df40a1db0 tx=0x7f2de4008040 comp rx=0 tx=0).stop 2024-05-09T06:29:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.791+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 msgr2=0x7f2df414a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.791+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00c7b0 tx=0x7f2dec00cc70 comp rx=0 tx=0).stop 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 shutdown_connections 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dd406be30 0x7f2dd406e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2df40d2a10 0x7f2df414a400 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2df40d2630 0x7f2df4156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 --2- 172.21.15.1:0/742361654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2df40b2ed0 0x7f2df40b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 >> 172.21.15.1:0/742361654 conn(0x7f2df40d3ee0 msgr2=0x7f2df40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 shutdown_connections 2024-05-09T06:29:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.792+0000 7f2df8f30640 1 -- 172.21.15.1:0/742361654 wait complete. 2024-05-09T06:29:23.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:29:23.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.915+0000 7f15e0b77640 1 Processor -- start 2024-05-09T06:29:23.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.915+0000 7f15e0b77640 1 -- start start 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15e0b77640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 0x7f15dc1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15e0b77640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f15dc0cf5a0 0x7f15dc0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15e0b77640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15dc0cdb30 con 0x7f15dc15bfa0 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15e0b77640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f15dc15fe50 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15e0b77640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f15dc0cd850 con 0x7f15dc15c380 2024-05-09T06:29:23.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15da59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f15dc15bfa0 0x7f15dc0ccd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53966/0 (socket says 172.21.15.1:53966) 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15da59c640 1 -- 172.21.15.1:0/2484212610 learned_addr learned my addr 172.21.15.1:0/2484212610 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15dad9d640 1 --2- 172.21.15.1:0/2484212610 >> v2:172.21.15.1:3300/0 conn(0x7f15dc0cf5a0 0x7f15dc0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15d9d9b640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 0x7f15dc1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15dad9d640 1 -- 172.21.15.1:0/2484212610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 msgr2=0x7f15dc1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15dad9d640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 0x7f15dc1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15dad9d640 1 -- 172.21.15.1:0/2484212610 >> v1:172.21.15.1:6789/0 conn(0x7f15dc15bfa0 legacy=0x7f15dc0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15dad9d640 1 -- 172.21.15.1:0/2484212610 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc1600f0 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15d9d9b640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 0x7f15dc1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:23.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.916+0000 7f15d959a640 1 -- 172.21.15.1:0/2484212610 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470213507 0 0) 0x7f15dc0cdb30 con 0x7f15dc15bfa0 2024-05-09T06:29:23.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15dad9d640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0cfa30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009d90 tx=0x7f15d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32e049557a99f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:23.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15d959a640 1 -- 172.21.15.1:0/2484212610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4031600 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15d959a640 1 -- 172.21.15.1:0/2484212610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d40317a0 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15d959a640 1 -- 172.21.15.1:0/2484212610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4031b70 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15e0b77640 1 -- 172.21.15.1:0/2484212610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc0cf5a0 msgr2=0x7f15dc0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.917+0000 7f15e0b77640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0cfa30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009d90 tx=0x7f15d402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 -- 172.21.15.1:0/2484212610 shutdown_connections 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15c380 0x7f15dc1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 --2- 172.21.15.1:0/2484212610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0cfa30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 -- 172.21.15.1:0/2484212610 >> 172.21.15.1:0/2484212610 conn(0x7f15dc071a50 msgr2=0x7f15dc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 -- 172.21.15.1:0/2484212610 shutdown_connections 2024-05-09T06:29:23.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.918+0000 7f15e0b77640 1 -- 172.21.15.1:0/2484212610 wait complete. 2024-05-09T06:29:23.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.919+0000 7f15e0b77640 1 Processor -- start 2024-05-09T06:29:23.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.919+0000 7f15e0b77640 1 -- start start 2024-05-09T06:29:23.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 0x7f15dc15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc0cd4a0 con 0x7f15dc15c380 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc074680 con 0x7f15dc0cf5a0 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15e0b77640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f15dc1f1940 con 0x7f15dc15bfa0 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43554/0 (socket says 172.21.15.1:43554) 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 -- 172.21.15.1:0/624439217 learned_addr learned my addr 172.21.15.1:0/624439217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15da59c640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15d9d9b640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 0x7f15dc15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 -- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 msgr2=0x7f15dc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 0x7f15dc15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 -- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15dc0cf5a0 msgr2=0x7f15dc0c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 -- 172.21.15.1:0/624439217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc0c4db0 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15d9d9b640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 0x7f15dc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.920+0000 7f15dad9d640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009eb0 tx=0x7f15d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.921+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4040070 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.921+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d4030b60 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.921+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc0c5330 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.921+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4009450 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.921+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc0c58f0 con 0x7f15dc15c380 2024-05-09T06:29:23.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.922+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15dc075320 con 0x7f15dc15c380 2024-05-09T06:29:23.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.922+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15d4039040 con 0x7f15dc15c380 2024-05-09T06:29:23.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.923+0000 7f15caffd640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 0x7f15bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.923+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5778+0+0 (secure 0 0 0) 0x7f15d40ac570 con 0x7f15dc15c380 2024-05-09T06:29:23.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.923+0000 7f15da59c640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 0x7f15bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.923+0000 7f15da59c640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 0x7f15bc06e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15cc007fd0 tx=0x7f15cc007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:23.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:23.927+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15d407aed0 con 0x7f15dc15c380 2024-05-09T06:29:24.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.112+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f15dc0b6300 con 0x7f15dc15c380 2024-05-09T06:29:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.788+0000 7f15caffd640 1 -- 172.21.15.1:0/624439217 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7f15d407a7d0 con 0x7f15dc15c380 2024-05-09T06:29:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.792+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 msgr2=0x7f15bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.792+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 0x7f15bc06e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15cc007fd0 tx=0x7f15cc007de0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.792+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 msgr2=0x7f15dc147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.792+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009eb0 tx=0x7f15d4004a40 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 shutdown_connections 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f15bc06be60 0x7f15bc06e320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15dc15c380 0x7f15dc147a40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f15dc15bfa0 0x7f15dc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 --2- 172.21.15.1:0/624439217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15dc0cf5a0 0x7f15dc0c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 >> 172.21.15.1:0/624439217 conn(0x7f15dc071a50 msgr2=0x7f15dc0c7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 shutdown_connections 2024-05-09T06:29:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.793+0000 7f15e0b77640 1 -- 172.21.15.1:0/624439217 wait complete. 2024-05-09T06:29:24.789 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:29:24.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 Processor -- start 2024-05-09T06:29:24.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 -- start start 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbe2015e390 0x7fbe200cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbe200b1900 0x7fbe200cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2015e900 con 0x7fbe200cf990 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2015ebe0 con 0x7fbe2015e390 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.920+0000 7fbe253c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2015eec0 con 0x7fbe200b1900 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbe2015e390 0x7fbe200cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1e7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbe200cf990 0x7fbe200b11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53982/0 (socket says 172.21.15.1:53982) 2024-05-09T06:29:24.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbe2015e390 0x7fbe200cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43558/0 (socket says 172.21.15.1:43558) 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1e7fc640 1 -- 172.21.15.1:0/3429501875 learned_addr learned my addr 172.21.15.1:0/3429501875 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1f7fe640 1 --2- 172.21.15.1:0/3429501875 >> v2:172.21.15.6:3300/0 conn(0x7fbe200b1900 0x7fbe200cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 -- 172.21.15.1:0/3429501875 >> v2:172.21.15.6:3300/0 conn(0x7fbe200b1900 msgr2=0x7fbe200cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 --2- 172.21.15.1:0/3429501875 >> v2:172.21.15.6:3300/0 conn(0x7fbe200b1900 0x7fbe200cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 -- 172.21.15.1:0/3429501875 >> v1:172.21.15.1:6789/0 conn(0x7fbe200cf990 legacy=0x7fbe200b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1dffb640 1 -- 172.21.15.1:0/3429501875 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428966391 0 0) 0x7fbe2015e900 con 0x7fbe200cf990 2024-05-09T06:29:24.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.921+0000 7fbe1effd640 1 -- 172.21.15.1:0/3429501875 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2015f1a0 con 0x7fbe2015e390 2024-05-09T06:29:24.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe1effd640 1 --2- 172.21.15.1:0/3429501875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe2015e390 0x7fbe200cf450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbe200721e0 tx=0x7fbe1002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c6ff8aaa4bc143 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:24.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe1dffb640 1 -- 172.21.15.1:0/3429501875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10007740 con 0x7fbe2015e390 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe1dffb640 1 -- 172.21.15.1:0/3429501875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100078e0 con 0x7fbe2015e390 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe1dffb640 1 -- 172.21.15.1:0/3429501875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10007cb0 con 0x7fbe2015e390 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe253c8640 1 -- 172.21.15.1:0/3429501875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe2015e390 msgr2=0x7fbe200cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.922+0000 7fbe253c8640 1 --2- 172.21.15.1:0/3429501875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe2015e390 0x7fbe200cf450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbe200721e0 tx=0x7fbe1002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 -- 172.21.15.1:0/3429501875 shutdown_connections 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 --2- 172.21.15.1:0/3429501875 >> v2:172.21.15.6:3300/0 conn(0x7fbe200b1900 0x7fbe200cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 --2- 172.21.15.1:0/3429501875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe2015e390 0x7fbe200cf450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 -- 172.21.15.1:0/3429501875 >> 172.21.15.1:0/3429501875 conn(0x7fbe200b1da0 msgr2=0x7fbe200b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 -- 172.21.15.1:0/3429501875 shutdown_connections 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.923+0000 7fbe253c8640 1 -- 172.21.15.1:0/3429501875 wait complete. 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.924+0000 7fbe253c8640 1 Processor -- start 2024-05-09T06:29:24.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.924+0000 7fbe253c8640 1 -- start start 2024-05-09T06:29:24.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe200b1900 0x7fbe200983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 0x7fbe20156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200cffb0 con 0x7fbe200cf990 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20162ba0 con 0x7fbe2015e390 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe253c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200cce20 con 0x7fbe200b1900 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43564/0 (socket says 172.21.15.1:43564) 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 -- 172.21.15.1:0/1400365345 learned_addr learned my addr 172.21.15.1:0/1400365345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1effd640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe200b1900 0x7fbe200983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1f7fe640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 0x7fbe20156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 -- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe200b1900 msgr2=0x7fbe200983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe200b1900 0x7fbe200983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 -- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 msgr2=0x7fbe20156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:24.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 0x7fbe20156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 -- 172.21.15.1:0/1400365345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe201e8230 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1f7fe640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 0x7fbe20156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe1e7fc640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c4a0 tx=0x7fbe1400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.925+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14018070 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.926+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe140074e0 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.926+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe20098b40 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.926+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004a20 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.926+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe20099100 con 0x7fbe200cf990 2024-05-09T06:29:24.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.927+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe200752b0 con 0x7fbe200cf990 2024-05-09T06:29:24.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.927+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe1401b070 con 0x7fbe200cf990 2024-05-09T06:29:24.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.928+0000 7fbe077fe640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 0x7fbe0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:24.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.928+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fbe1408a000 con 0x7fbe200cf990 2024-05-09T06:29:24.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.928+0000 7fbe1effd640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 0x7fbe0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:24.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.928+0000 7fbe1effd640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 0x7fbe0c06e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe10034000 tx=0x7fbe10033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:24.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:24.932+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe14058bb0 con 0x7fbe200cf990 2024-05-09T06:29:25.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.121+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbe200b2ae0 con 0x7fbe200cf990 2024-05-09T06:29:25.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.124+0000 7fbe077fe640 1 -- 172.21.15.1:0/1400365345 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fbe140584b0 con 0x7fbe200cf990 2024-05-09T06:29:25.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 msgr2=0x7fbe0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 0x7fbe0c06e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe10034000 tx=0x7fbe10033040 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 msgr2=0x7fbe201e7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c4a0 tx=0x7fbe1400c960 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 shutdown_connections 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe0c06be30 0x7fbe0c06e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe2015e390 0x7fbe20156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe200cf990 0x7fbe201e7cf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 --2- 172.21.15.1:0/1400365345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe200b1900 0x7fbe200983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.126+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 >> 172.21.15.1:0/1400365345 conn(0x7fbe200b1da0 msgr2=0x7fbe2006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.127+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 shutdown_connections 2024-05-09T06:29:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.127+0000 7fbe253c8640 1 -- 172.21.15.1:0/1400365345 wait complete. 2024-05-09T06:29:25.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-09T06:29:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.241+0000 7f8bff3c8640 1 Processor -- start 2024-05-09T06:29:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 -- start start 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 0x7f8bf80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf80b33c0 con 0x7f8bf80d4940 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf80d4dd0 con 0x7f8bf80d0f80 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.242+0000 7f8bff3c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf80b3660 con 0x7f8bf8161950 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd965640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 0x7f8bf80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:39848/0 (socket says 172.21.15.1:39848) 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfc963640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8bf80d4940 0x7f8bf80cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:56366/0 (socket says 172.21.15.1:56366) 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 -- 172.21.15.1:0/4047816045 learned_addr learned my addr 172.21.15.1:0/4047816045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 -- 172.21.15.1:0/4047816045 >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 msgr2=0x7f8bf80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 --2- 172.21.15.1:0/4047816045 >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 0x7f8bf80d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 -- 172.21.15.1:0/4047816045 >> v1:172.21.15.6:6789/0 conn(0x7f8bf80d4940 legacy=0x7f8bf80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bf7fff640 1 -- 172.21.15.1:0/4047816045 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293166759 0 0) 0x7f8bf80b33c0 con 0x7f8bf80d4940 2024-05-09T06:29:25.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd164640 1 -- 172.21.15.1:0/4047816045 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf80b3940 con 0x7f8bf8161950 2024-05-09T06:29:25.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.243+0000 7f8bfd965640 1 --2- 172.21.15.1:0/4047816045 >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 0x7f8bf80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.244+0000 7f8bfd164640 1 --2- 172.21.15.1:0/4047816045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80b3590 tx=0x7f8bf002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b104d9fea701022 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.244+0000 7f8bf7fff640 1 -- 172.21.15.1:0/4047816045 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0037070 con 0x7f8bf8161950 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.244+0000 7f8bf7fff640 1 -- 172.21.15.1:0/4047816045 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf0032440 con 0x7f8bf8161950 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.244+0000 7f8bf7fff640 1 -- 172.21.15.1:0/4047816045 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0032810 con 0x7f8bf8161950 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 -- 172.21.15.1:0/4047816045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 msgr2=0x7f8bf80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/4047816045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80b3590 tx=0x7f8bf002a250 comp rx=0 tx=0).stop 2024-05-09T06:29:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 -- 172.21.15.1:0/4047816045 shutdown_connections 2024-05-09T06:29:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/4047816045 >> v2:172.21.15.6:3300/0 conn(0x7f8bf80d0f80 0x7f8bf80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/4047816045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf8161950 0x7f8bf80d2400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 -- 172.21.15.1:0/4047816045 >> 172.21.15.1:0/4047816045 conn(0x7f8bf80d3e20 msgr2=0x7f8bf80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 -- 172.21.15.1:0/4047816045 shutdown_connections 2024-05-09T06:29:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.245+0000 7f8bff3c8640 1 -- 172.21.15.1:0/4047816045 wait complete. 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.246+0000 7f8bff3c8640 1 Processor -- start 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.246+0000 7f8bff3c8640 1 -- start start 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf80d0f80 0x7f8bf809f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bf80a1490 0x7f8bf80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b52d0 con 0x7f8bf8161950 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b5150 con 0x7f8bf80a1490 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bff3c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf80b5450 con 0x7f8bf80d0f80 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43570/0 (socket says 172.21.15.1:43570) 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 -- 172.21.15.1:0/253257450 learned_addr learned my addr 172.21.15.1:0/253257450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfd164640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf80d0f80 0x7f8bf809f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfd965640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bf80a1490 0x7f8bf80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 -- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf80d0f80 msgr2=0x7f8bf809f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf80d0f80 0x7f8bf809f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 -- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bf80a1490 msgr2=0x7f8bf80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bf80a1490 0x7f8bf80cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.247+0000 7f8bfc963640 1 -- 172.21.15.1:0/253257450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf80a03f0 con 0x7f8bf8161950 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bfc963640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c4d0 tx=0x7f8bec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec018070 con 0x7f8bf8161950 2024-05-09T06:29:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec007500 con 0x7f8bf8161950 2024-05-09T06:29:25.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf80a06d0 con 0x7f8bf8161950 2024-05-09T06:29:25.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004a20 con 0x7f8bf8161950 2024-05-09T06:29:25.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.248+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf80a0c90 con 0x7f8bf8161950 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.249+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf80752b0 con 0x7f8bf8161950 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.249+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bec004bc0 con 0x7f8bf8161950 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.250+0000 7f8bf5ffb640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 0x7f8be406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.250+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8bec08a040 con 0x7f8bf8161950 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.250+0000 7f8bfd164640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 0x7f8be406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.250+0000 7f8bfd164640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 0x7f8be406e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0034000 tx=0x7f8bf002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.254+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bec058ae0 con 0x7f8bf8161950 2024-05-09T06:29:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.509+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f8bf8074650 con 0x7f8bf8161950 2024-05-09T06:29:25.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.792+0000 7f8bf5ffb640 1 -- 172.21.15.1:0/253257450 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v134) ==== 142+0+0 (secure 0 0 0) 0x7f8bec0583e0 con 0x7f8bf8161950 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 msgr2=0x7f8be406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 0x7f8be406e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0034000 tx=0x7f8bf002e000 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 msgr2=0x7f8bf809fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c4d0 tx=0x7f8bec00c990 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 shutdown_connections 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8be406be30 0x7f8be406e2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bf80a1490 0x7f8bf80cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf8161950 0x7f8bf809fd70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 --2- 172.21.15.1:0/253257450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bf80d0f80 0x7f8bf809f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 >> 172.21.15.1:0/253257450 conn(0x7f8bf80d3e20 msgr2=0x7f8bf815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:25.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 shutdown_connections 2024-05-09T06:29:25.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.795+0000 7f8bff3c8640 1 -- 172.21.15.1:0/253257450 wait complete. 2024-05-09T06:29:25.788 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-09T06:29:25.789 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-09T06:29:25.789 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:29:25.789 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:25.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.912+0000 7f0505adf640 1 Processor -- start 2024-05-09T06:29:25.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 -- start start 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f050015db10 0x7f05000b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f050015e6d0 0x7f050006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0500070f40 con 0x7f05000715d0 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f05000cf170 con 0x7f050015e6d0 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.913+0000 7f0505adf640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05000cf450 con 0x7f050015db10 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04feffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f05000715d0 0x7f050015dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:53998/0 (socket says 172.21.15.1:53998) 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04feffd640 1 -- 172.21.15.1:0/2956832415 learned_addr learned my addr 172.21.15.1:0/2956832415 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fffff640 1 --2- 172.21.15.1:0/2956832415 >> v2:172.21.15.1:3300/0 conn(0x7f050015e6d0 0x7f050006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04ff7fe640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f050015db10 0x7f05000b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fffff640 1 -- 172.21.15.1:0/2956832415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f050015db10 msgr2=0x7f05000b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:25.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fffff640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f050015db10 0x7f05000b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fffff640 1 -- 172.21.15.1:0/2956832415 >> v1:172.21.15.1:6789/0 conn(0x7f05000715d0 legacy=0x7f050015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fe7fc640 1 -- 172.21.15.1:0/2956832415 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025732787 0 0) 0x7f0500070f40 con 0x7f05000715d0 2024-05-09T06:29:25.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.914+0000 7f04fffff640 1 -- 172.21.15.1:0/2956832415 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0500070c10 con 0x7f050015e6d0 2024-05-09T06:29:25.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f04fffff640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015e6d0 0x7f050006e340 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009d90 tx=0x7f04f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33466457d7edbc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f04fe7fc640 1 -- 172.21.15.1:0/2956832415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8007740 con 0x7f050015e6d0 2024-05-09T06:29:25.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f04fe7fc640 1 -- 172.21.15.1:0/2956832415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f80078e0 con 0x7f050015e6d0 2024-05-09T06:29:25.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f04fe7fc640 1 -- 172.21.15.1:0/2956832415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8007cb0 con 0x7f050015e6d0 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 -- 172.21.15.1:0/2956832415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015e6d0 msgr2=0x7f050006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015e6d0 0x7f050006e340 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009d90 tx=0x7f04f802a660 comp rx=0 tx=0).stop 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 -- 172.21.15.1:0/2956832415 shutdown_connections 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015e6d0 0x7f050006e340 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 --2- 172.21.15.1:0/2956832415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f050015db10 0x7f05000b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 -- 172.21.15.1:0/2956832415 >> 172.21.15.1:0/2956832415 conn(0x7f05000cc4b0 msgr2=0x7f05000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 -- 172.21.15.1:0/2956832415 shutdown_connections 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.915+0000 7f0505adf640 1 -- 172.21.15.1:0/2956832415 wait complete. 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.916+0000 7f0505adf640 1 Processor -- start 2024-05-09T06:29:25.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 -- start start 2024-05-09T06:29:25.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05000715d0 0x7f050014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 0x7f0500159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f050015edc0 con 0x7f050015db10 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f050015ec40 con 0x7f050015e6d0 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.917+0000 7f0505adf640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f050006cd30 con 0x7f05000715d0 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43590/0 (socket says 172.21.15.1:43590) 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 -- 172.21.15.1:0/2842314757 learned_addr learned my addr 172.21.15.1:0/2842314757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04fffff640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 0x7f0500159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04ff7fe640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05000715d0 0x7f050014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 -- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05000715d0 msgr2=0x7f050014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05000715d0 0x7f050014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 -- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 msgr2=0x7f0500159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 0x7f0500159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 -- 172.21.15.1:0/2842314757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0500151f60 con 0x7f050015db10 2024-05-09T06:29:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04fffff640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 0x7f0500159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04feffd640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c500 tx=0x7f04f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4018070 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4007580 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0500152240 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4004a20 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.918+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0500158910 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.919+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05000752b0 con 0x7f050015db10 2024-05-09T06:29:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.920+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04f4004bc0 con 0x7f050015db10 2024-05-09T06:29:25.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.920+0000 7f04e7fff640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:25.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.920+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f04f4089fb0 con 0x7f050015db10 2024-05-09T06:29:25.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.920+0000 7f04ff7fe640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:25.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.921+0000 7f04ff7fe640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 0x7f04ec06e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006fd0 tx=0x7f04f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:25.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:25.925+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f04f4058ae0 con 0x7f050015db10 2024-05-09T06:29:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.109+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05000b5d20 con 0x7f050015db10 2024-05-09T06:29:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.110+0000 7f04e7fff640 1 -- 172.21.15.1:0/2842314757 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13157 (secure 0 0 0) 0x7f04f40583e0 con 0x7f050015db10 2024-05-09T06:29:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:26.092 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":43,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:24.788113+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/642815597":"2024-05-10T06:29:21.780770+0000","172.21.15.6:6832/642815597":"2024-05-10T06:29:21.780770+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 msgr2=0x7f04ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 0x7f04ec06e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006fd0 tx=0x7f04f0008040 comp rx=0 tx=0).stop 2024-05-09T06:29:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 msgr2=0x7f05001583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c500 tx=0x7f04f400c9c0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 shutdown_connections 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04ec06be30 0x7f04ec06e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f050015e6d0 0x7f0500159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f050015db10 0x7f05001583d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 --2- 172.21.15.1:0/2842314757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05000715d0 0x7f050014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 >> 172.21.15.1:0/2842314757 conn(0x7f05000cc4b0 msgr2=0x7f05000cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 shutdown_connections 2024-05-09T06:29:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.112+0000 7f0505adf640 1 -- 172.21.15.1:0/2842314757 wait complete. 2024-05-09T06:29:26.105 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:29:26.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:29:26.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.230+0000 7f05270df640 1 Processor -- start 2024-05-09T06:29:26.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.230+0000 7f05270df640 1 -- start start 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f05270df640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f052015a4d0 0x7f05200cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f05270df640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05200cdb90 0x7f0520070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f05270df640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05200cf1b0 con 0x7f0520071440 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f05270df640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f052015a8b0 con 0x7f052015a4d0 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f05270df640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f052006dc10 con 0x7f05200cdb90 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f0524e7b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f052015a4d0 0x7f05200cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f052567c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0520071440 0x7f052015dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54000/0 (socket says 172.21.15.1:54000) 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f0524e7b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f052015a4d0 0x7f05200cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43606/0 (socket says 172.21.15.1:43606) 2024-05-09T06:29:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f052567c640 1 -- 172.21.15.1:0/4131528035 learned_addr learned my addr 172.21.15.1:0/4131528035 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.231+0000 7f051ffff640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05200cdb90 0x7f0520070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f0524e7b640 1 -- 172.21.15.1:0/4131528035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05200cdb90 msgr2=0x7f0520070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f0524e7b640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05200cdb90 0x7f0520070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f0524e7b640 1 -- 172.21.15.1:0/4131528035 >> v1:172.21.15.1:6789/0 conn(0x7f0520071440 legacy=0x7f052015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f0524e7b640 1 -- 172.21.15.1:0/4131528035 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05200cf490 con 0x7f052015a4d0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f051f7fe640 1 -- 172.21.15.1:0/4131528035 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603180329 0 0) 0x7f05200cf1b0 con 0x7f0520071440 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f0524e7b640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f052015a4d0 0x7f05200cea80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f051800bfa0 tx=0x7f051802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4726590530d659e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.232+0000 7f051f7fe640 1 -- 172.21.15.1:0/4131528035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051803b070 con 0x7f052015a4d0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f051f7fe640 1 -- 172.21.15.1:0/4131528035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05180336d0 con 0x7f052015a4d0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f051f7fe640 1 -- 172.21.15.1:0/4131528035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518033aa0 con 0x7f052015a4d0 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 -- 172.21.15.1:0/4131528035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f052015a4d0 msgr2=0x7f05200cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f052015a4d0 0x7f05200cea80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f051800bfa0 tx=0x7f051802c860 comp rx=0 tx=0).stop 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 -- 172.21.15.1:0/4131528035 shutdown_connections 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05200cdb90 0x7f0520070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 --2- 172.21.15.1:0/4131528035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f052015a4d0 0x7f05200cea80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 -- 172.21.15.1:0/4131528035 >> 172.21.15.1:0/4131528035 conn(0x7f05200cc4d0 msgr2=0x7f05200cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 -- 172.21.15.1:0/4131528035 shutdown_connections 2024-05-09T06:29:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.233+0000 7f05270df640 1 -- 172.21.15.1:0/4131528035 wait complete. 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.234+0000 7f05270df640 1 Processor -- start 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.234+0000 7f05270df640 1 -- start start 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0520071440 0x7f05201e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 0x7f05201f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05200b2be0 con 0x7f05200cdb90 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f05200b2a60 con 0x7f052015a4d0 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f05270df640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f052015e6e0 con 0x7f0520071440 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f051ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f051ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43622/0 (socket says 172.21.15.1:43622) 2024-05-09T06:29:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.235+0000 7f051ffff640 1 -- 172.21.15.1:0/2097498079 learned_addr learned my addr 172.21.15.1:0/2097498079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f0524e7b640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0520071440 0x7f05201e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f052567c640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 0x7f05201f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 -- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0520071440 msgr2=0x7f05201e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0520071440 0x7f05201e4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 -- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 msgr2=0x7f05201f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 0x7f05201f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 -- 172.21.15.1:0/2097498079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05200ae3d0 con 0x7f05200cdb90 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f052567c640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 0x7f05201f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051ffff640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f050800e9b0 tx=0x7f050800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508017070 con 0x7f05200cdb90 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050800c9a0 con 0x7f05200cdb90 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05200ae6b0 con 0x7f05200cdb90 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800cd90 con 0x7f05200cdb90 2024-05-09T06:29:26.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.236+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05200aec70 con 0x7f05200cdb90 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.237+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05200752b0 con 0x7f05200cdb90 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.238+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f050801a070 con 0x7f05200cdb90 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.238+0000 7f051d7fa640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.238+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f050808aff0 con 0x7f05200cdb90 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.238+0000 7f0524e7b640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.239+0000 7f0524e7b640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 0x7f050006e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0518038000 tx=0x7f0518031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.243+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0508059ba0 con 0x7f05200cdb90 2024-05-09T06:29:26.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.424+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05200cd570 con 0x7f05200cdb90 2024-05-09T06:29:26.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.425+0000 7f051d7fa640 1 -- 172.21.15.1:0/2097498079 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f05080594a0 con 0x7f05200cdb90 2024-05-09T06:29:26.407 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 msgr2=0x7f050006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 0x7f050006e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0518038000 tx=0x7f0518031000 comp rx=0 tx=0).stop 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 msgr2=0x7f05201f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f050800e9b0 tx=0x7f050800ee70 comp rx=0 tx=0).stop 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 shutdown_connections 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f050006be30 0x7f050006e2f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f052015a4d0 0x7f05201f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05200cdb90 0x7f05201f6d40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 --2- 172.21.15.1:0/2097498079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0520071440 0x7f05201e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 >> 172.21.15.1:0/2097498079 conn(0x7f05200cc4d0 msgr2=0x7f052006e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:26.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 shutdown_connections 2024-05-09T06:29:26.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.427+0000 7f05270df640 1 -- 172.21.15.1:0/2097498079 wait complete. 2024-05-09T06:29:26.420 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-09T06:29:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.543+0000 7fc5e9f10640 1 Processor -- start 2024-05-09T06:29:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.543+0000 7fc5e9f10640 1 -- start start 2024-05-09T06:29:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e9f10640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5e40b2a60 0x7fc5e415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e9f10640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5e415e500 0x7fc5e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e9f10640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e4070f40 con 0x7fc5e415db10 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e9f10640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e415e950 con 0x7fc5e40b2a60 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e9f10640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e40cf170 con 0x7fc5e415e500 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e37fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc5e415db10 0x7fc5e40715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54014/0 (socket says 172.21.15.1:54014) 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e37fe640 1 -- 172.21.15.1:0/1609384010 learned_addr learned my addr 172.21.15.1:0/1609384010 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/1609384010 >> v2:172.21.15.1:3300/0 conn(0x7fc5e40b2a60 0x7fc5e415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e3fff640 1 --2- 172.21.15.1:0/1609384010 >> v2:172.21.15.6:3300/0 conn(0x7fc5e415e500 0x7fc5e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5dbfff640 1 -- 172.21.15.1:0/1609384010 >> v2:172.21.15.6:3300/0 conn(0x7fc5e415e500 msgr2=0x7fc5e406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/1609384010 >> v2:172.21.15.6:3300/0 conn(0x7fc5e415e500 0x7fc5e406e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5dbfff640 1 -- 172.21.15.1:0/1609384010 >> v1:172.21.15.1:6789/0 conn(0x7fc5e415db10 legacy=0x7fc5e40715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5e2ffd640 1 -- 172.21.15.1:0/1609384010 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689103692 0 0) 0x7fc5e4070f40 con 0x7fc5e415db10 2024-05-09T06:29:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.544+0000 7fc5dbfff640 1 -- 172.21.15.1:0/1609384010 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e40cf450 con 0x7fc5e40b2a60 2024-05-09T06:29:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.545+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/1609384010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e40b2a60 0x7fc5e415dfc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40b2770 tx=0x7fc5cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3500430476bace server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.545+0000 7fc5e2ffd640 1 -- 172.21.15.1:0/1609384010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc037070 con 0x7fc5e40b2a60 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.545+0000 7fc5e2ffd640 1 -- 172.21.15.1:0/1609384010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc032440 con 0x7fc5e40b2a60 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.545+0000 7fc5e2ffd640 1 -- 172.21.15.1:0/1609384010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc02ae60 con 0x7fc5e40b2a60 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 -- 172.21.15.1:0/1609384010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e40b2a60 msgr2=0x7fc5e415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/1609384010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e40b2a60 0x7fc5e415dfc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40b2770 tx=0x7fc5cc02a110 comp rx=0 tx=0).stop 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 -- 172.21.15.1:0/1609384010 shutdown_connections 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/1609384010 >> v2:172.21.15.6:3300/0 conn(0x7fc5e415e500 0x7fc5e406e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/1609384010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e40b2a60 0x7fc5e415dfc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 -- 172.21.15.1:0/1609384010 >> 172.21.15.1:0/1609384010 conn(0x7fc5e40cc4b0 msgr2=0x7fc5e40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 -- 172.21.15.1:0/1609384010 shutdown_connections 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.546+0000 7fc5e9f10640 1 -- 172.21.15.1:0/1609384010 wait complete. 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.547+0000 7fc5e9f10640 1 Processor -- start 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.547+0000 7fc5e9f10640 1 -- start start 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 0x7fc5e41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 0x7fc5e414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e415edc0 con 0x7fc5e415db10 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e415ec40 con 0x7fc5e415e500 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e9f10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e406cd30 con 0x7fc5e40b2a60 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43638/0 (socket says 172.21.15.1:43638) 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 -- 172.21.15.1:0/3381620866 learned_addr learned my addr 172.21.15.1:0/3381620866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e37fe640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 0x7fc5e41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e3fff640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 0x7fc5e414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 -- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 msgr2=0x7fc5e41520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 0x7fc5e41520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 -- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 msgr2=0x7fc5e414ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 0x7fc5e414ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5dbfff640 1 -- 172.21.15.1:0/3381620866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e4151050 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e3fff640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 0x7fc5e414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.548+0000 7fc5e37fe640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 0x7fc5e41520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5dbfff640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000c00 tx=0x7fc5cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc037050 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc038040 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e4151330 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc004450 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.549+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41518f0 con 0x7fc5e415db10 2024-05-09T06:29:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.550+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e40752b0 con 0x7fc5e415db10 2024-05-09T06:29:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.550+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5cc030980 con 0x7fc5e415db10 2024-05-09T06:29:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.551+0000 7fc5e0ff9640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 0x7fc5c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.551+0000 7fc5e37fe640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 0x7fc5c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.551+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc5cc0ab8f0 con 0x7fc5e415db10 2024-05-09T06:29:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.551+0000 7fc5e37fe640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 0x7fc5c406e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000f60 tx=0x7fc5d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.556+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5cc07a540 con 0x7fc5e415db10 2024-05-09T06:29:26.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:26.816+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fc5e406d160 con 0x7fc5e415db10 2024-05-09T06:29:27.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.798+0000 7fc5e0ff9640 1 -- 172.21.15.1:0/3381620866 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v136) ==== 152+0+0 (secure 0 0 0) 0x7fc5cc079e40 con 0x7fc5e415db10 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 msgr2=0x7fc5c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 0x7fc5c406e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000f60 tx=0x7fc5d4009040 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 msgr2=0x7fc5e4150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000c00 tx=0x7fc5cc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 shutdown_connections 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5c406be30 0x7fc5c406e2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5e415e500 0x7fc5e414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5e415db10 0x7fc5e4150950 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 --2- 172.21.15.1:0/3381620866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5e40b2a60 0x7fc5e41520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 >> 172.21.15.1:0/3381620866 conn(0x7fc5e40cc4b0 msgr2=0x7fc5e41533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:27.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 shutdown_connections 2024-05-09T06:29:27.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.801+0000 7fc5e9f10640 1 -- 172.21.15.1:0/3381620866 wait complete. 2024-05-09T06:29:27.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:29:27.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.928+0000 7f76e9953640 1 Processor -- start 2024-05-09T06:29:27.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.928+0000 7f76e9953640 1 -- start start 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e9953640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f76e40d1c70 0x7f76e415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e9953640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76e40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e9953640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e4161fe0 con 0x7f76e415e190 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e9953640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f76e415f580 con 0x7f76e40d1c70 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e9953640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76e41622c0 con 0x7f76e40d45f0 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e2ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f76e415e190 0x7f76e40d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36446/0 (socket says 172.21.15.1:36446) 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.929+0000 7f76e2ffd640 1 -- 172.21.15.1:0/1704708023 learned_addr learned my addr 172.21.15.1:0/1704708023 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e37fe640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76e40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467436096 0 0) 0x7f76e4161fe0 con 0x7f76e415e190 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 --2- 172.21.15.1:0/1704708023 >> v2:172.21.15.6:3300/0 conn(0x7f76e40d1c70 0x7f76e415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76cc003520 con 0x7f76e415e190 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1897896526 0 0) 0x7f76cc003520 con 0x7f76e415e190 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 msgr2=0x7f76e40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76e40b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 >> v2:172.21.15.6:3300/0 conn(0x7f76e40d1c70 msgr2=0x7f76e415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 --2- 172.21.15.1:0/1704708023 >> v2:172.21.15.6:3300/0 conn(0x7f76e40d1c70 0x7f76e415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e41625a0 con 0x7f76e415e190 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f76d40019f0 con 0x7f76e415e190 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 >> v1:172.21.15.1:6789/0 conn(0x7f76e415e190 legacy=0x7f76e40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0039b0 0x7f76cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc004800 0x7f76cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41622c0 con 0x7f76cc004800 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc00b9c0 con 0x7f76cc0039b0 2024-05-09T06:29:27.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76cc00bb40 con 0x7f76e40d45f0 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc004800 0x7f76cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e37fe640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e2ffd640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0039b0 0x7f76cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 -- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 msgr2=0x7f76cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76e40d45f0 0x7f76cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 -- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0039b0 msgr2=0x7f76cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc0039b0 0x7f76cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.930+0000 7f76e27fc640 1 -- 172.21.15.1:0/1704708023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc003520 con 0x7f76cc004800 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.931+0000 7f76e27fc640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc004800 0x7f76cc004c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009a90 tx=0x7f76cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f24490c9207693 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.931+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8036070 con 0x7f76cc004800 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.931+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d8031400 con 0x7f76cc004800 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.931+0000 7f76e1ffb640 1 -- 172.21.15.1:0/1704708023 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8009690 con 0x7f76cc004800 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 -- 172.21.15.1:0/1704708023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc004800 msgr2=0x7f76cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 --2- 172.21.15.1:0/1704708023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76cc004800 0x7f76cc004c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009a90 tx=0x7f76cc005830 comp rx=0 tx=0).stop 2024-05-09T06:29:27.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e37fe640 1 -- 172.21.15.1:0/1704708023 reap_dead start 2024-05-09T06:29:27.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 -- 172.21.15.1:0/1704708023 shutdown_connections 2024-05-09T06:29:27.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 -- 172.21.15.1:0/1704708023 >> 172.21.15.1:0/1704708023 conn(0x7f76e4073340 msgr2=0x7f76e409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:27.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 -- 172.21.15.1:0/1704708023 shutdown_connections 2024-05-09T06:29:27.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.932+0000 7f76e9953640 1 -- 172.21.15.1:0/1704708023 wait complete. 2024-05-09T06:29:27.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.933+0000 7f76e9953640 1 Processor -- start 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.933+0000 7f76e9953640 1 -- start start 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc004800 0x7f76e414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76e40b8200 0x7f76e40b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76e40b56d0 con 0x7f76e415e570 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4161c60 con 0x7f76e40b8200 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e9953640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76e40b5850 con 0x7f76cc004800 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33634/0 (socket says 172.21.15.1:33634) 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 -- 172.21.15.1:0/3127085095 learned_addr learned my addr 172.21.15.1:0/3127085095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 -- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc004800 msgr2=0x7f76e414e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc004800 0x7f76e414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 -- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76e40b8200 msgr2=0x7f76e40b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:27.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76e40b8200 0x7f76e40b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.934+0000 7f76e27fc640 1 -- 172.21.15.1:0/3127085095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e40b8bc0 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76e27fc640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000c00 tx=0x7f76d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8040020 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d803b660 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e415bdb0 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8036040 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.935+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e40b9130 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.936+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e415ef10 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.936+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76d8043020 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.937+0000 7f76cbfff640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 0x7f76d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.937+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f76d80ab4b0 con 0x7f76e415e570 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.937+0000 7f76e2ffd640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 0x7f76d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:27.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.937+0000 7f76e2ffd640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 0x7f76d006e320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76d4002230 tx=0x7f76d40017b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:27.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:27.941+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76d804a030 con 0x7f76e415e570 2024-05-09T06:29:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.124+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f76e40752b0 con 0x7f76e415e570 2024-05-09T06:29:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.127+0000 7f76cbfff640 1 -- 172.21.15.1:0/3127085095 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f76d8079f40 con 0x7f76e415e570 2024-05-09T06:29:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 msgr2=0x7f76d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 0x7f76d006e320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76d4002230 tx=0x7f76d40017b0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 msgr2=0x7f76e415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000c00 tx=0x7f76d8004730 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 shutdown_connections 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f76d006be60 0x7f76d006e320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76e40b8200 0x7f76e40b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e415e570 0x7f76e415e9d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 --2- 172.21.15.1:0/3127085095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc004800 0x7f76e414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.129+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 >> 172.21.15.1:0/3127085095 conn(0x7f76e4073340 msgr2=0x7f76e40b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.130+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 shutdown_connections 2024-05-09T06:29:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.130+0000 7f76e9953640 1 -- 172.21.15.1:0/3127085095 wait complete. 2024-05-09T06:29:28.122 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:29:28.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:29:28.167 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:29:28.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:29:28.221 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:29:28.221 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:28.237 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:29:28.237 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:29:28.238 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:28.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:29:28.256 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:29:28.256 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:28.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:29:28.294 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:29:28.294 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:29:28.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:29:28.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:29:28.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.455+0000 7ff953612640 1 Processor -- start 2024-05-09T06:29:28.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 -- start start 2024-05-09T06:29:28.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 0x7ff94c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff94c0d4940 0x7ff94c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 0x7ff94c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c0d4dd0 con 0x7ff94c0d4940 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c0b33c0 con 0x7ff94c0d0f80 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.456+0000 7ff953612640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c0b3660 con 0x7ff94c161950 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff94c0d4940 0x7ff94c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff94c0d4940 0x7ff94c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33646/0 (socket says 172.21.15.1:33646) 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 -- 172.21.15.1:0/3493828059 learned_addr learned my addr 172.21.15.1:0/3493828059 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 -- 172.21.15.1:0/3493828059 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 msgr2=0x7ff94c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff9513ae640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 0x7ff94c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff951baf640 1 --2- 172.21.15.1:0/3493828059 >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 0x7ff94c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 0x7ff94c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 -- 172.21.15.1:0/3493828059 >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 msgr2=0x7ff94c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 --2- 172.21.15.1:0/3493828059 >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 0x7ff94c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff950bad640 1 -- 172.21.15.1:0/3493828059 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c0b3940 con 0x7ff94c0d4940 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff9513ae640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 0x7ff94c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.457+0000 7ff951baf640 1 --2- 172.21.15.1:0/3493828059 >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 0x7ff94c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff950bad640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c0d4940 0x7ff94c0d2940 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff94c08c710 tx=0x7ff93c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4f4f017a5af899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff943fff640 1 -- 172.21.15.1:0/3493828059 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c037070 con 0x7ff94c0d4940 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff943fff640 1 -- 172.21.15.1:0/3493828059 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c032440 con 0x7ff94c0d4940 2024-05-09T06:29:28.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff943fff640 1 -- 172.21.15.1:0/3493828059 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c032810 con 0x7ff94c0d4940 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 -- 172.21.15.1:0/3493828059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c0d4940 msgr2=0x7ff94c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c0d4940 0x7ff94c0d2940 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff94c08c710 tx=0x7ff93c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 -- 172.21.15.1:0/3493828059 shutdown_connections 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 --2- 172.21.15.1:0/3493828059 >> v2:172.21.15.6:3300/0 conn(0x7ff94c0d0f80 0x7ff94c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c0d4940 0x7ff94c0d2940 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 --2- 172.21.15.1:0/3493828059 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c161950 0x7ff94c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.458+0000 7ff953612640 1 -- 172.21.15.1:0/3493828059 >> 172.21.15.1:0/3493828059 conn(0x7ff94c0d3e20 msgr2=0x7ff94c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.459+0000 7ff953612640 1 -- 172.21.15.1:0/3493828059 shutdown_connections 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.459+0000 7ff953612640 1 -- 172.21.15.1:0/3493828059 wait complete. 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 Processor -- start 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 -- start start 2024-05-09T06:29:28.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c0d0f80 0x7ff94c09e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 0x7ff94c0b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c0d1d80 con 0x7ff94c161950 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c0d1c00 con 0x7ff94c0b84c0 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.460+0000 7ff953612640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c0d1f00 con 0x7ff94c0d0f80 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33654/0 (socket says 172.21.15.1:33654) 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 -- 172.21.15.1:0/4183140508 learned_addr learned my addr 172.21.15.1:0/4183140508 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff9513ae640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c0d0f80 0x7ff94c09e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff951baf640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 0x7ff94c0b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 -- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c0d0f80 msgr2=0x7ff94c09e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c0d0f80 0x7ff94c09e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 -- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 msgr2=0x7ff94c0b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 0x7ff94c0b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 -- 172.21.15.1:0/4183140508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c1537d0 con 0x7ff94c161950 2024-05-09T06:29:28.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff951baf640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 0x7ff94c0b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff950bad640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02a220 tx=0x7ff93c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c037040 con 0x7ff94c161950 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c042070 con 0x7ff94c161950 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c0b7890 con 0x7ff94c161950 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c030bb0 con 0x7ff94c161950 2024-05-09T06:29:28.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.461+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c0cb420 con 0x7ff94c161950 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.463+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94c0752b0 con 0x7ff94c161950 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.463+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff93c047460 con 0x7ff94c161950 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.463+0000 7ff941ffb640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 0x7ff93006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.464+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7ff93c0ab7d0 con 0x7ff94c161950 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.464+0000 7ff9513ae640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 0x7ff93006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.464+0000 7ff9513ae640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 0x7ff93006e320 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff94c06c7c0 tx=0x7ff948008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.468+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff93c045360 con 0x7ff94c161950 2024-05-09T06:29:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.687+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff94c0d21e0 con 0x7ff94c161950 2024-05-09T06:29:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.688+0000 7ff941ffb640 1 -- 172.21.15.1:0/4183140508 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3854 (secure 0 0 0) 0x7ff93c07a380 con 0x7ff94c161950 2024-05-09T06:29:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:28.670 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/372553402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":372553402},{"type":"v1","addr":"172.21.15.1:6837","nonce":372553402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2127274624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2127274624},{"type":"v1","addr":"172.21.15.6:6835","nonce":2127274624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3437200540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3437200540},{"type":"v1","addr":"172.21.15.6:6834","nonce":3437200540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1037815378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1037815378},{"type":"v1","addr":"172.21.15.1:6836","nonce":1037815378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-09T06:29:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 20 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 msgr2=0x7ff93006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 0x7ff93006e320 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff94c06c7c0 tx=0x7ff948008040 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 msgr2=0x7ff94c153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02a220 tx=0x7ff93c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 shutdown_connections 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff93006be60 0x7ff93006e320 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff94c0b84c0 0x7ff94c0b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff94c161950 0x7ff94c153290 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 --2- 172.21.15.1:0/4183140508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff94c0d0f80 0x7ff94c09e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 >> 172.21.15.1:0/4183140508 conn(0x7ff94c0d3e20 msgr2=0x7ff94c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 shutdown_connections 2024-05-09T06:29:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.690+0000 7ff953612640 1 -- 172.21.15.1:0/4183140508 wait complete. 2024-05-09T06:29:28.682 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:29:28.683 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:29:28.683 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:29:28.683 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:29:28.683 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:29:28.683 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:29:28.683 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:29:28.683 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:29:28.684 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:29:28.684 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:29:28.684 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:29:28.684 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:29:28.684 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:29:28.684 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:29:28.684 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:29:28.684 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:29:28.693 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:29:28.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:29:28.694 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:29:28.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:29:28.697 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:29:28.697 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:29:28.698 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:29:28.698 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:29:28.736 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:29:28.736 DEBUG:teuthology.parallel:result is None 2024-05-09T06:29:28.737 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:29:28.737 DEBUG:teuthology.parallel:result is None 2024-05-09T06:29:28.740 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:29:28.740 DEBUG:teuthology.parallel:result is None 2024-05-09T06:29:28.740 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:29:28.741 DEBUG:teuthology.parallel:result is None 2024-05-09T06:29:28.741 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:29:28.816 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:29:28.817 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:29:28.826 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:29:28.827 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:29:28.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.869+0000 7f2c91d70640 1 Processor -- start 2024-05-09T06:29:28.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 -- start start 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 0x7f2c8c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2c8c0d2b10 0x7f2c8c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c0b33c0 con 0x7f2c8c161950 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c0d4dd0 con 0x7f2c8c0d2b10 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c91d70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c0b3660 con 0x7f2c8c0d4940 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.870+0000 7f2c8b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2c8c161950 0x7f2c8c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36462/0 (socket says 172.21.15.1:36462) 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/1341713723 learned_addr learned my addr 172.21.15.1:0/1341713723 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8affd640 1 --2- 172.21.15.1:0/1341713723 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 0x7f2c8c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/1341713723 >> v2:172.21.15.6:3300/0 conn(0x7f2c8c0d2b10 0x7f2c8c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8a7fc640 1 -- 172.21.15.1:0/1341713723 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521950215 0 0) 0x7f2c8c0b33c0 con 0x7f2c8c161950 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8affd640 1 -- 172.21.15.1:0/1341713723 >> v2:172.21.15.6:3300/0 conn(0x7f2c8c0d2b10 msgr2=0x7f2c8c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8affd640 1 --2- 172.21.15.1:0/1341713723 >> v2:172.21.15.6:3300/0 conn(0x7f2c8c0d2b10 0x7f2c8c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8affd640 1 -- 172.21.15.1:0/1341713723 >> v1:172.21.15.1:6789/0 conn(0x7f2c8c161950 legacy=0x7f2c8c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.871+0000 7f2c8affd640 1 -- 172.21.15.1:0/1341713723 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c0b3940 con 0x7f2c8c0d4940 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c8affd640 1 --2- 172.21.15.1:0/1341713723 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 0x7f2c8c0b4140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c800097a0 tx=0x7f2c8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693ac963800dd572 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c8a7fc640 1 -- 172.21.15.1:0/1341713723 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037070 con 0x7f2c8c0d4940 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c8a7fc640 1 -- 172.21.15.1:0/1341713723 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80032440 con 0x7f2c8c0d4940 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c8a7fc640 1 -- 172.21.15.1:0/1341713723 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80032810 con 0x7f2c8c0d4940 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 -- 172.21.15.1:0/1341713723 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 msgr2=0x7f2c8c0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 --2- 172.21.15.1:0/1341713723 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 0x7f2c8c0b4140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c800097a0 tx=0x7f2c8002a180 comp rx=0 tx=0).stop 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 -- 172.21.15.1:0/1341713723 shutdown_connections 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 --2- 172.21.15.1:0/1341713723 >> v2:172.21.15.6:3300/0 conn(0x7f2c8c0d2b10 0x7f2c8c0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 --2- 172.21.15.1:0/1341713723 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d4940 0x7f2c8c0b4140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 -- 172.21.15.1:0/1341713723 >> 172.21.15.1:0/1341713723 conn(0x7f2c8c0d3e20 msgr2=0x7f2c8c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 -- 172.21.15.1:0/1341713723 shutdown_connections 2024-05-09T06:29:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.872+0000 7f2c91d70640 1 -- 172.21.15.1:0/1341713723 wait complete. 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.874+0000 7f2c91d70640 1 Processor -- start 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.874+0000 7f2c91d70640 1 -- start start 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.874+0000 7f2c91d70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d2b10 0x7f2c8c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.874+0000 7f2c91d70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.874+0000 7f2c91d70640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c8c151600 0x7f2c8c1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c91d70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0d1100 con 0x7f2c8c161950 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c91d70640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0d0f80 con 0x7f2c8c151600 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c91d70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c0d1c00 con 0x7f2c8c0d2b10 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33672/0 (socket says 172.21.15.1:33672) 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 -- 172.21.15.1:0/3113211179 learned_addr learned my addr 172.21.15.1:0/3113211179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:28.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8b7fe640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d2b10 0x7f2c8c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c8c151600 0x7f2c8c1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 -- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d2b10 msgr2=0x7f2c8c14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d2b10 0x7f2c8c14e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 -- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c8c151600 msgr2=0x7f2c8c1508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c8c151600 0x7f2c8c1508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 -- 172.21.15.1:0/3113211179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c09de10 con 0x7f2c8c161950 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c8affd640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c80000c00 tx=0x7f2c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037040 con 0x7f2c8c161950 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80042070 con 0x7f2c8c161950 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c14e880 con 0x7f2c8c161950 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.875+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80030940 con 0x7f2c8c161950 2024-05-09T06:29:28.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.876+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c14ee40 con 0x7f2c8c161950 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.876+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c0752b0 con 0x7f2c8c161950 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.877+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c80030ae0 con 0x7f2c8c161950 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.878+0000 7f2c73fff640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.878+0000 7f2c8b7fe640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.878+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f2c800ab610 con 0x7f2c8c161950 2024-05-09T06:29:28.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.878+0000 7f2c8b7fe640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 0x7f2c7806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c000f60 tx=0x7f2c7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:28.882+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c8007a120 con 0x7f2c8c161950 2024-05-09T06:29:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.078+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2c8c0d21e0 con 0x7f2c8c161950 2024-05-09T06:29:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.808+0000 7f2c73fff640 1 -- 172.21.15.1:0/3113211179 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) ==== 97+0+0 (secure 0 0 0) 0x7f2c80079a20 con 0x7f2c8c161950 2024-05-09T06:29:29.790 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 msgr2=0x7f2c7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 0x7f2c7806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c000f60 tx=0x7f2c7c009040 comp rx=0 tx=0).stop 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 msgr2=0x7f2c8c0b94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c80000c00 tx=0x7f2c800046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 shutdown_connections 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c7806be30 0x7f2c7806e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c8c151600 0x7f2c8c1508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c8c161950 0x7f2c8c0b94c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 --2- 172.21.15.1:0/3113211179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c8c0d2b10 0x7f2c8c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 >> 172.21.15.1:0/3113211179 conn(0x7f2c8c0d3e20 msgr2=0x7f2c8c0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 shutdown_connections 2024-05-09T06:29:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.811+0000 7f2c91d70640 1 -- 172.21.15.1:0/3113211179 wait complete. 2024-05-09T06:29:29.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:29:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.933+0000 7f500f9a9640 1 Processor -- start 2024-05-09T06:29:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.933+0000 7f500f9a9640 1 -- start start 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500f9a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080c6860 0x7f5008099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500f9a9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f50080b2aa0 0x7f500815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500f9a9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5008099930 con 0x7f50080c6c40 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500f9a9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f50080b1e20 con 0x7f50080b2aa0 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500f9a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f500815ee90 con 0x7f50080c6860 2024-05-09T06:29:29.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f50080b2aa0 0x7f500815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f50080b2aa0 0x7f500815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33674/0 (socket says 172.21.15.1:33674) 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 -- 172.21.15.1:0/3890527413 learned_addr learned my addr 172.21.15.1:0/3890527413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 -- 172.21.15.1:0/3890527413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080c6860 msgr2=0x7f5008099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 --2- 172.21.15.1:0/3890527413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080c6860 0x7f5008099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 -- 172.21.15.1:0/3890527413 >> v1:172.21.15.6:6789/0 conn(0x7f50080c6c40 legacy=0x7f500815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:29.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.934+0000 7f500cf44640 1 -- 172.21.15.1:0/3890527413 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5008162930 con 0x7f50080b2aa0 2024-05-09T06:29:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.935+0000 7f500cf44640 1 --2- 172.21.15.1:0/3890527413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080b2aa0 0x7f500815e4b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009870 tx=0x7f4ff802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279eb264a929aa7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.935+0000 7f4ffffff640 1 -- 172.21.15.1:0/3890527413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8037070 con 0x7f50080b2aa0 2024-05-09T06:29:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.935+0000 7f4ffffff640 1 -- 172.21.15.1:0/3890527413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff8032440 con 0x7f50080b2aa0 2024-05-09T06:29:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.935+0000 7f4ffffff640 1 -- 172.21.15.1:0/3890527413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8032810 con 0x7f50080b2aa0 2024-05-09T06:29:29.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 -- 172.21.15.1:0/3890527413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080b2aa0 msgr2=0x7f500815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 --2- 172.21.15.1:0/3890527413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080b2aa0 0x7f500815e4b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009870 tx=0x7f4ff802a180 comp rx=0 tx=0).stop 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 -- 172.21.15.1:0/3890527413 shutdown_connections 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 --2- 172.21.15.1:0/3890527413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080b2aa0 0x7f500815e4b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 --2- 172.21.15.1:0/3890527413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080c6860 0x7f5008099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 -- 172.21.15.1:0/3890527413 >> 172.21.15.1:0/3890527413 conn(0x7f50080b1fb0 msgr2=0x7f50080b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 -- 172.21.15.1:0/3890527413 shutdown_connections 2024-05-09T06:29:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.936+0000 7f500f9a9640 1 -- 172.21.15.1:0/3890527413 wait complete. 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.937+0000 7f500f9a9640 1 Processor -- start 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.937+0000 7f500f9a9640 1 -- start start 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 0x7f5008149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50080c6860 0x7f50080b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f500815eb20 con 0x7f50080c6c40 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f500815fa30 con 0x7f50080c6860 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500f9a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f50080c70d0 con 0x7f50080b2aa0 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33684/0 (socket says 172.21.15.1:33684) 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 -- 172.21.15.1:0/2796070569 learned_addr learned my addr 172.21.15.1:0/2796070569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500cf44640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50080c6860 0x7f50080b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500d745640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 0x7f5008149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 -- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 msgr2=0x7f5008149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 0x7f5008149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 -- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50080c6860 msgr2=0x7f50080b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50080c6860 0x7f50080b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 -- 172.21.15.1:0/2796070569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50080b3a10 con 0x7f50080c6c40 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500d745640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 0x7f5008149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.938+0000 7f500df46640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f500000c780 tx=0x7f500000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:29.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.939+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000017070 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.939+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5000003da0 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.939+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50080b6070 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.939+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000052b0 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.939+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50080cbe60 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.940+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50080752b0 con 0x7f50080c6c40 2024-05-09T06:29:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.940+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f500001a070 con 0x7f50080c6c40 2024-05-09T06:29:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.941+0000 7f4ffdffb640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 0x7f4fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.941+0000 7f500d745640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 0x7f4fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.941+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7f500008a160 con 0x7f50080c6c40 2024-05-09T06:29:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.942+0000 7f500d745640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 0x7f4fec06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5004009890 tx=0x7f5004008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:29.946+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5000058710 con 0x7f50080c6c40 2024-05-09T06:29:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.198+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f500806ce30 con 0x7f50080c6c40 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.198+0000 7f4ffdffb640 1 -- 172.21.15.1:0/2796070569 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f500005ca10 con 0x7f50080c6c40 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.180 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:29:30.183 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.184 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.185 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:29:30.186 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.189 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:29:30.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:29:30.191 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:29:30.191 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:29:30.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.193 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:29:30.193 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:29:30.193 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.193 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:29:30.195 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 msgr2=0x7f4fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 0x7f4fec06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5004009890 tx=0x7f5004008040 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 msgr2=0x7f50080b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f500000c780 tx=0x7f500000cc40 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 shutdown_connections 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fec06be30 0x7f4fec06e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50080c6c40 0x7f50080b34d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50080c6860 0x7f50080b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 --2- 172.21.15.1:0/2796070569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50080b2aa0 0x7f5008149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 >> 172.21.15.1:0/2796070569 conn(0x7f50080b1fb0 msgr2=0x7f500806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 shutdown_connections 2024-05-09T06:29:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.200+0000 7f500f9a9640 1 -- 172.21.15.1:0/2796070569 wait complete. 2024-05-09T06:29:30.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:30.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.320+0000 7f361739e640 1 Processor -- start 2024-05-09T06:29:30.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 -- start start 2024-05-09T06:29:30.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 0x7f36100d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36100b33c0 con 0x7f36100d0f80 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f36100d4dd0 con 0x7f3610161950 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.321+0000 7f361739e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f36100b3660 con 0x7f36100d4940 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f361513a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 0x7f36100d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:36386/0 (socket says 172.21.15.1:36386) 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 -- 172.21.15.1:0/3616119316 learned_addr learned my addr 172.21.15.1:0/3616119316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f361513a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 0x7f36100d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:47448/0 (socket says 172.21.15.1:47448) 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 -- 172.21.15.1:0/3616119316 >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 msgr2=0x7f36100d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 --2- 172.21.15.1:0/3616119316 >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 0x7f36100d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 -- 172.21.15.1:0/3616119316 >> v1:172.21.15.6:6789/0 conn(0x7f36100d0f80 legacy=0x7f36100cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.322+0000 7f3614939640 1 -- 172.21.15.1:0/3616119316 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36100b3940 con 0x7f36100d4940 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f3614939640 1 --2- 172.21.15.1:0/3616119316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36100b3590 tx=0x7f360402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84471d1e207717e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f3603fff640 1 -- 172.21.15.1:0/3616119316 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604037070 con 0x7f36100d4940 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f3603fff640 1 -- 172.21.15.1:0/3616119316 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604032440 con 0x7f36100d4940 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f3603fff640 1 -- 172.21.15.1:0/3616119316 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604032810 con 0x7f36100d4940 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 -- 172.21.15.1:0/3616119316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 msgr2=0x7f36100d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 --2- 172.21.15.1:0/3616119316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36100b3590 tx=0x7f360402a250 comp rx=0 tx=0).stop 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 -- 172.21.15.1:0/3616119316 shutdown_connections 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 --2- 172.21.15.1:0/3616119316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36100d4940 0x7f36100d2940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 --2- 172.21.15.1:0/3616119316 >> v2:172.21.15.6:3300/0 conn(0x7f3610161950 0x7f36100d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.323+0000 7f361739e640 1 -- 172.21.15.1:0/3616119316 >> 172.21.15.1:0/3616119316 conn(0x7f36100d3e20 msgr2=0x7f36100d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.324+0000 7f361739e640 1 -- 172.21.15.1:0/3616119316 shutdown_connections 2024-05-09T06:29:30.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.324+0000 7f361739e640 1 -- 172.21.15.1:0/3616119316 wait complete. 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 Processor -- start 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 -- start start 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36100d0f80 0x7f361015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f361015c290 0x7f361014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36100b52d0 con 0x7f3610161950 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f36100b5150 con 0x7f36100d0f80 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.325+0000 7f361739e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f36100b5450 con 0x7f361015c290 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f361513a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36100d0f80 0x7f361015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33692/0 (socket says 172.21.15.1:33692) 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 -- 172.21.15.1:0/3805723319 learned_addr learned my addr 172.21.15.1:0/3805723319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f361593b640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f361015c290 0x7f361014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 -- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f361015c290 msgr2=0x7f361014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f361015c290 0x7f361014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 -- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36100d0f80 msgr2=0x7f361015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36100d0f80 0x7f361015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 -- 172.21.15.1:0/3805723319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361015c780 con 0x7f3610161950 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3614939640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f36040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604041030 con 0x7f3610161950 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604030440 con 0x7f3610161950 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604037040 con 0x7f3610161950 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.326+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361015ca60 con 0x7f3610161950 2024-05-09T06:29:30.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.327+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361015d020 con 0x7f3610161950 2024-05-09T06:29:30.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.328+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36100752b0 con 0x7f3610161950 2024-05-09T06:29:30.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.328+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3604044070 con 0x7f3610161950 2024-05-09T06:29:30.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.329+0000 7f3601ffb640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 0x7f35f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.329+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7f36040ab180 con 0x7f3610161950 2024-05-09T06:29:30.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.329+0000 7f361513a640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 0x7f35f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.329+0000 7f361513a640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 0x7f35f406e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f360c009aa0 tx=0x7f360c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.333+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f36040797c0 con 0x7f3610161950 2024-05-09T06:29:30.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.519+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36100d21e0 con 0x7f3610161950 2024-05-09T06:29:30.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.520+0000 7f3601ffb640 1 -- 172.21.15.1:0/3805723319 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+13032 (secure 0 0 0) 0x7f360407dac0 con 0x7f3610161950 2024-05-09T06:29:30.502 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:30.502 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":47,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:29.815539+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 msgr2=0x7f35f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 0x7f35f406e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f360c009aa0 tx=0x7f360c009190 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 msgr2=0x7f361014cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f36040046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 shutdown_connections 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35f406be30 0x7f35f406e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f361015c290 0x7f361014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3610161950 0x7f361014cc90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 --2- 172.21.15.1:0/3805723319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36100d0f80 0x7f361015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 >> 172.21.15.1:0/3805723319 conn(0x7f36100d3e20 msgr2=0x7f36101603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 shutdown_connections 2024-05-09T06:29:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.522+0000 7f361739e640 1 -- 172.21.15.1:0/3805723319 wait complete. 2024-05-09T06:29:30.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:29:30.516 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:29:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.640+0000 7fa3fca9a640 1 Processor -- start 2024-05-09T06:29:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.640+0000 7fa3fca9a640 1 -- start start 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.641+0000 7fa3fca9a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 0x7fa3f806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.641+0000 7fa3fca9a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa3f815cfb0 0x7fa3f8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.641+0000 7fa3fca9a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f806cf40 con 0x7fa3f80b4de0 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.641+0000 7fa3fca9a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f8156980 con 0x7fa3f815cfb0 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.641+0000 7fa3fca9a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f806d870 con 0x7fa3f815c4c0 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa3f815cfb0 0x7fa3f8070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa3f815cfb0 0x7fa3f8070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33702/0 (socket says 172.21.15.1:33702) 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 -- 172.21.15.1:0/4240195478 learned_addr learned my addr 172.21.15.1:0/4240195478 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f659c640 1 --2- 172.21.15.1:0/4240195478 >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 0x7fa3f806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 -- 172.21.15.1:0/4240195478 >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 msgr2=0x7fa3f806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:30.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 --2- 172.21.15.1:0/4240195478 >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 0x7fa3f806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 -- 172.21.15.1:0/4240195478 >> v1:172.21.15.6:6789/0 conn(0x7fa3f80b4de0 legacy=0x7fa3f815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f6d9d640 1 -- 172.21.15.1:0/4240195478 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f806d220 con 0x7fa3f815cfb0 2024-05-09T06:29:30.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.642+0000 7fa3f659c640 1 --2- 172.21.15.1:0/4240195478 >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 0x7fa3f806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3f6d9d640 1 --2- 172.21.15.1:0/4240195478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f815cfb0 0x7fa3f8070c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa3f804fb80 tx=0x7fa3f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23b4b66bcbcf952 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3f559a640 1 -- 172.21.15.1:0/4240195478 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0036070 con 0x7fa3f815cfb0 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3f559a640 1 -- 172.21.15.1:0/4240195478 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0031600 con 0x7fa3f815cfb0 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3f559a640 1 -- 172.21.15.1:0/4240195478 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00319d0 con 0x7fa3f815cfb0 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3fca9a640 1 -- 172.21.15.1:0/4240195478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f815cfb0 msgr2=0x7fa3f8070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/4240195478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f815cfb0 0x7fa3f8070c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa3f804fb80 tx=0x7fa3f002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3fca9a640 1 -- 172.21.15.1:0/4240195478 shutdown_connections 2024-05-09T06:29:30.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/4240195478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f815cfb0 0x7fa3f8070c10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.643+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/4240195478 >> v2:172.21.15.6:3300/0 conn(0x7fa3f815c4c0 0x7fa3f806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.644+0000 7fa3fca9a640 1 -- 172.21.15.1:0/4240195478 >> 172.21.15.1:0/4240195478 conn(0x7fa3f80cc4d0 msgr2=0x7fa3f80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.644+0000 7fa3fca9a640 1 -- 172.21.15.1:0/4240195478 shutdown_connections 2024-05-09T06:29:30.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.644+0000 7fa3fca9a640 1 -- 172.21.15.1:0/4240195478 wait complete. 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.645+0000 7fa3fca9a640 1 Processor -- start 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.645+0000 7fa3fca9a640 1 -- start start 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3f815c4c0 0x7fa3f815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3f815cfb0 0x7fa3f815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80b2dc0 con 0x7fa3f80b4de0 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80b2c40 con 0x7fa3f815c4c0 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f80b5260 con 0x7fa3f815cfb0 2024-05-09T06:29:30.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33708/0 (socket says 172.21.15.1:33708) 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 -- 172.21.15.1:0/647877866 learned_addr learned my addr 172.21.15.1:0/647877866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 -- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3f815cfb0 msgr2=0x7fa3f815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3f815cfb0 0x7fa3f815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 -- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3f815c4c0 msgr2=0x7fa3f815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3f815c4c0 0x7fa3f815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 -- 172.21.15.1:0/647877866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f80a9a60 con 0x7fa3f80b4de0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3f659c640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e7e0 tx=0x7fa3e800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800f040 con 0x7fa3f80b4de0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e8004030 con 0x7fa3f80b4de0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f80a9d40 con 0x7fa3f80b4de0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004400 con 0x7fa3f80b4de0 2024-05-09T06:29:30.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.646+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f80aa300 con 0x7fa3f80b4de0 2024-05-09T06:29:30.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.648+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f80752b0 con 0x7fa3f80b4de0 2024-05-09T06:29:30.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.648+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3e800c6d0 con 0x7fa3f80b4de0 2024-05-09T06:29:30.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.649+0000 7fa3e6ffd640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 0x7fa3d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.649+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7fa3e808a9b0 con 0x7fa3f80b4de0 2024-05-09T06:29:30.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.649+0000 7fa3f5d9b640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 0x7fa3d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.649+0000 7fa3f5d9b640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 0x7fa3d806e320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec000f60 tx=0x7fa3ec0079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.653+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3e8017020 con 0x7fa3f80b4de0 2024-05-09T06:29:30.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.839+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3f80cbc50 con 0x7fa3f80b4de0 2024-05-09T06:29:30.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.839+0000 7fa3e6ffd640 1 -- 172.21.15.1:0/647877866 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fa3e80595b0 con 0x7fa3f80b4de0 2024-05-09T06:29:30.821 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 msgr2=0x7fa3d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 0x7fa3d806e320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec000f60 tx=0x7fa3ec0079d0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 msgr2=0x7fa3f80ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e7e0 tx=0x7fa3e800eca0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 shutdown_connections 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa3d806be60 0x7fa3d806e320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3f815cfb0 0x7fa3f815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3f815c4c0 0x7fa3f815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 --2- 172.21.15.1:0/647877866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f80b4de0 0x7fa3f80ac650 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.841+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 >> 172.21.15.1:0/647877866 conn(0x7fa3f80cc4d0 msgr2=0x7fa3f815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.842+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 shutdown_connections 2024-05-09T06:29:30.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.842+0000 7fa3fca9a640 1 -- 172.21.15.1:0/647877866 wait complete. 2024-05-09T06:29:30.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:29:30.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.956+0000 7fc7b4342640 1 Processor -- start 2024-05-09T06:29:30.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.956+0000 7fc7b4342640 1 -- start start 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.957+0000 7fc7b4342640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac161950 0x7fc7ac0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.957+0000 7fc7b4342640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc7ac0d4940 0x7fc7ac0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.957+0000 7fc7b4342640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ac0b33c0 con 0x7fc7ac0d0f80 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.957+0000 7fc7b4342640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ac0d4dd0 con 0x7fc7ac0d4940 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.957+0000 7fc7b4342640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac0b3660 con 0x7fc7ac161950 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc7ac0d4940 0x7fc7ac0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc7ac0d4940 0x7fc7ac0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33720/0 (socket says 172.21.15.1:33720) 2024-05-09T06:29:30.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3121814803 learned_addr learned my addr 172.21.15.1:0/3121814803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b20de640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac161950 0x7fc7ac0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3121814803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac161950 msgr2=0x7fc7ac0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac161950 0x7fc7ac0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3121814803 >> v1:172.21.15.6:6789/0 conn(0x7fc7ac0d0f80 legacy=0x7fc7ac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.958+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3121814803 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac0b3940 con 0x7fc7ac0d4940 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b18dd640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac0d4940 0x7fc7ac0d2940 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0b3590 tx=0x7fc79c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9771fc6d33931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b10dc640 1 -- 172.21.15.1:0/3121814803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c037070 con 0x7fc7ac0d4940 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b10dc640 1 -- 172.21.15.1:0/3121814803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c032440 con 0x7fc7ac0d4940 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b10dc640 1 -- 172.21.15.1:0/3121814803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c032810 con 0x7fc7ac0d4940 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b4342640 1 -- 172.21.15.1:0/3121814803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac0d4940 msgr2=0x7fc7ac0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.959+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac0d4940 0x7fc7ac0d2940 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0b3590 tx=0x7fc79c02a250 comp rx=0 tx=0).stop 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 -- 172.21.15.1:0/3121814803 shutdown_connections 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac0d4940 0x7fc7ac0d2940 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3121814803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac161950 0x7fc7ac0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 -- 172.21.15.1:0/3121814803 >> 172.21.15.1:0/3121814803 conn(0x7fc7ac0d3e20 msgr2=0x7fc7ac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 -- 172.21.15.1:0/3121814803 shutdown_connections 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.960+0000 7fc7b4342640 1 -- 172.21.15.1:0/3121814803 wait complete. 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 Processor -- start 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 -- start start 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac0d0f80 0x7fc7ac1f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 0x7fc7ac15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0b52d0 con 0x7fc7ac161950 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0b5150 con 0x7fc7ac0d0f80 2024-05-09T06:29:30.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.961+0000 7fc7b4342640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac0b5450 con 0x7fc7ac1f1110 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33734/0 (socket says 172.21.15.1:33734) 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3265049718 learned_addr learned my addr 172.21.15.1:0/3265049718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b20de640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac0d0f80 0x7fc7ac1f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b28df640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 0x7fc7ac15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 msgr2=0x7fc7ac15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 0x7fc7ac15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac0d0f80 msgr2=0x7fc7ac1f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac0d0f80 0x7fc7ac1f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 -- 172.21.15.1:0/3265049718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac0cbad0 con 0x7fc7ac161950 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b28df640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 0x7fc7ac15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b18dd640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02a290 tx=0x7fc79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c037040 con 0x7fc7ac161950 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c042070 con 0x7fc7ac161950 2024-05-09T06:29:30.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c030880 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac0cbdb0 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.962+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac15bed0 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.963+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac0cc2d0 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.964+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc79c030a20 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.964+0000 7fc7a2ffd640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 0x7fc79006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.965+0000 7fc7b20de640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 0x7fc79006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.965+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7fc79c0ab5b0 con 0x7fc7ac161950 2024-05-09T06:29:30.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.965+0000 7fc7b20de640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 0x7fc79006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac15b800 tx=0x7fc7a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:30.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:30.969+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc79c07a120 con 0x7fc7ac161950 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.155+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc7ac0752b0 con 0x7fc7ac161950 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.155+0000 7fc7a2ffd640 1 -- 172.21.15.1:0/3265049718 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) ==== 84+0+65 (secure 0 0 0) 0x7fc79c079a20 con 0x7fc7ac161950 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:31.137 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:29:31.138 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:29:31.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:31.138 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 msgr2=0x7fc79006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 0x7fc79006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac15b800 tx=0x7fc7a8008040 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 msgr2=0x7fc7ac0cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc79c02a290 tx=0x7fc79c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 shutdown_connections 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc79006be30 0x7fc79006e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7ac1f1110 0x7fc7ac15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7ac161950 0x7fc7ac0cb360 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 --2- 172.21.15.1:0/3265049718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac0d0f80 0x7fc7ac1f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 >> 172.21.15.1:0/3265049718 conn(0x7fc7ac0d3e20 msgr2=0x7fc7ac0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:31.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 shutdown_connections 2024-05-09T06:29:31.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.158+0000 7fc7b4342640 1 -- 172.21.15.1:0/3265049718 wait complete. 2024-05-09T06:29:31.150 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:29:31.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:29:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.276+0000 7f9b4e5ed640 1 Processor -- start 2024-05-09T06:29:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 -- start start 2024-05-09T06:29:31.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9b480d2630 0x7f9b4815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b480d1cc0 con 0x7f9b480b2ed0 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9b480d4d10 con 0x7f9b480d2630 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.277+0000 7f9b4e5ed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9b480d1f80 con 0x7f9b480d2a10 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b47fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9b480d2630 0x7f9b4815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:36430/0 (socket says 172.21.15.1:36430) 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 -- 172.21.15.1:0/1190926122 learned_addr learned my addr 172.21.15.1:0/1190926122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 -- 172.21.15.1:0/1190926122 >> v2:172.21.15.6:3300/0 conn(0x7f9b480d2630 msgr2=0x7f9b4815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 --2- 172.21.15.1:0/1190926122 >> v2:172.21.15.6:3300/0 conn(0x7f9b480d2630 0x7f9b4815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 -- 172.21.15.1:0/1190926122 >> v1:172.21.15.6:6789/0 conn(0x7f9b480b2ed0 legacy=0x7f9b480b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.278+0000 7f9b4cb8a640 1 -- 172.21.15.1:0/1190926122 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b480b0c00 con 0x7f9b480d2a10 2024-05-09T06:29:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b4cb8a640 1 --2- 172.21.15.1:0/1190926122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009e50 tx=0x7f9b4002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d21fbd40fe8f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b46ffd640 1 -- 172.21.15.1:0/1190926122 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40036070 con 0x7f9b480d2a10 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b46ffd640 1 -- 172.21.15.1:0/1190926122 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b40031600 con 0x7f9b480d2a10 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b46ffd640 1 -- 172.21.15.1:0/1190926122 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400319d0 con 0x7f9b480d2a10 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/1190926122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 msgr2=0x7f9b480cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.279+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/1190926122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009e50 tx=0x7f9b4002a710 comp rx=0 tx=0).stop 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/1190926122 shutdown_connections 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/1190926122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b480cfbb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/1190926122 >> v2:172.21.15.6:3300/0 conn(0x7f9b480d2630 0x7f9b4815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/1190926122 >> 172.21.15.1:0/1190926122 conn(0x7f9b480d3ee0 msgr2=0x7f9b480d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/1190926122 shutdown_connections 2024-05-09T06:29:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.280+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/1190926122 wait complete. 2024-05-09T06:29:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.281+0000 7f9b4e5ed640 1 Processor -- start 2024-05-09T06:29:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.281+0000 7f9b4e5ed640 1 -- start start 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 0x7f9b48149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b4814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4815e4b0 con 0x7f9b480b2ed0 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4815e330 con 0x7f9b480d2630 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4815e630 con 0x7f9b480d2a10 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33738/0 (socket says 172.21.15.1:33738) 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 -- 172.21.15.1:0/3940748887 learned_addr learned my addr 172.21.15.1:0/3940748887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b477fe640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 0x7f9b48149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4cb8a640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b4814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 -- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 msgr2=0x7f9b4814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b4814eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 -- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 msgr2=0x7f9b48149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 0x7f9b48149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 -- 172.21.15.1:0/3940748887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48149f90 con 0x7f9b480b2ed0 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b477fe640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 0x7f9b48149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4cb8a640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b4814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b47fff640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800e870 tx=0x7f9b3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38017070 con 0x7f9b480b2ed0 2024-05-09T06:29:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3800c940 con 0x7f9b480b2ed0 2024-05-09T06:29:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4814a270 con 0x7f9b480b2ed0 2024-05-09T06:29:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.282+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800cd30 con 0x7f9b480b2ed0 2024-05-09T06:29:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.283+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4814a830 con 0x7f9b480b2ed0 2024-05-09T06:29:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.284+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48075320 con 0x7f9b480b2ed0 2024-05-09T06:29:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.284+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b3801a070 con 0x7f9b480b2ed0 2024-05-09T06:29:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.284+0000 7f9b44ff9640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 0x7f9b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.285+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7f9b3808ab50 con 0x7f9b480b2ed0 2024-05-09T06:29:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.285+0000 7f9b477fe640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 0x7f9b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.285+0000 7f9b477fe640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 0x7f9b3406e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b481614d0 tx=0x7f9b3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.289+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b380596c0 con 0x7f9b480b2ed0 2024-05-09T06:29:31.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.471+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9b480ce320 con 0x7f9b480b2ed0 2024-05-09T06:29:31.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.819+0000 7f9b44ff9640 1 -- 172.21.15.1:0/3940748887 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) ==== 138+0+0 (secure 0 0 0) 0x7f9b38058fc0 con 0x7f9b480b2ed0 2024-05-09T06:29:31.801 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:29:31.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.822+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 msgr2=0x7f9b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.822+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 0x7f9b3406e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b481614d0 tx=0x7f9b3c009040 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.822+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 msgr2=0x7f9b48160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.822+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800e870 tx=0x7f9b3800ed30 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 shutdown_connections 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b3406be30 0x7f9b3406e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9b480d2a10 0x7f9b4814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b480d2630 0x7f9b48149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 --2- 172.21.15.1:0/3940748887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b480b2ed0 0x7f9b48160560 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 >> 172.21.15.1:0/3940748887 conn(0x7f9b480d3ee0 msgr2=0x7f9b4815ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 shutdown_connections 2024-05-09T06:29:31.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.823+0000 7f9b4e5ed640 1 -- 172.21.15.1:0/3940748887 wait complete. 2024-05-09T06:29:31.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:29:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.950+0000 7fad27b70640 1 Processor -- start 2024-05-09T06:29:31.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.950+0000 7fad27b70640 1 -- start start 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad27b70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 0x7fad200cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad27b70640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fad200b18f0 0x7fad200ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad27b70640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad20161c90 con 0x7fad200cfa20 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad27b70640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fad2015e550 con 0x7fad200b18f0 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad27b70640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad20161f00 con 0x7fad2015e170 2024-05-09T06:29:31.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fad200b18f0 0x7fad200ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fad200b18f0 0x7fad200ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33754/0 (socket says 172.21.15.1:33754) 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 -- 172.21.15.1:0/4188594431 learned_addr learned my addr 172.21.15.1:0/4188594431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2590c640 1 --2- 172.21.15.1:0/4188594431 >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 0x7fad200cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 -- 172.21.15.1:0/4188594431 >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 msgr2=0x7fad200cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 --2- 172.21.15.1:0/4188594431 >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 0x7fad200cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 -- 172.21.15.1:0/4188594431 >> v1:172.21.15.6:6789/0 conn(0x7fad200cfa20 legacy=0x7fad200b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2610d640 1 -- 172.21.15.1:0/4188594431 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad201621e0 con 0x7fad200b18f0 2024-05-09T06:29:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.951+0000 7fad2590c640 1 --2- 172.21.15.1:0/4188594431 >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 0x7fad200cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:31.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.952+0000 7fad2610d640 1 --2- 172.21.15.1:0/4188594431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200b18f0 0x7fad200ccb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fad20072790 tx=0x7fad1802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff530f8bcefafa81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.952+0000 7fad2490a640 1 -- 172.21.15.1:0/4188594431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18036070 con 0x7fad200b18f0 2024-05-09T06:29:31.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.952+0000 7fad2490a640 1 -- 172.21.15.1:0/4188594431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad18031600 con 0x7fad200b18f0 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.952+0000 7fad2490a640 1 -- 172.21.15.1:0/4188594431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180319d0 con 0x7fad200b18f0 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 -- 172.21.15.1:0/4188594431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200b18f0 msgr2=0x7fad200ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 --2- 172.21.15.1:0/4188594431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200b18f0 0x7fad200ccb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fad20072790 tx=0x7fad1802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 -- 172.21.15.1:0/4188594431 shutdown_connections 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 --2- 172.21.15.1:0/4188594431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200b18f0 0x7fad200ccb70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 --2- 172.21.15.1:0/4188594431 >> v2:172.21.15.6:3300/0 conn(0x7fad2015e170 0x7fad200cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 -- 172.21.15.1:0/4188594431 >> 172.21.15.1:0/4188594431 conn(0x7fad200b1da0 msgr2=0x7fad200b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 -- 172.21.15.1:0/4188594431 shutdown_connections 2024-05-09T06:29:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.953+0000 7fad27b70640 1 -- 172.21.15.1:0/4188594431 wait complete. 2024-05-09T06:29:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.954+0000 7fad27b70640 1 Processor -- start 2024-05-09T06:29:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.954+0000 7fad27b70640 1 -- start start 2024-05-09T06:29:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 0x7fad2015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 0x7fad201567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad201629e0 con 0x7fad200cfa20 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad200cffc0 con 0x7fad2015e170 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad27b70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad201530d0 con 0x7fad200b18f0 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33760/0 (socket says 172.21.15.1:33760) 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 -- 172.21.15.1:0/3296890544 learned_addr learned my addr 172.21.15.1:0/3296890544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2610d640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 0x7fad201567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2590c640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 0x7fad2015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 -- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 msgr2=0x7fad2015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 0x7fad2015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 -- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 msgr2=0x7fad201567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 0x7fad201567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2510b640 1 -- 172.21.15.1:0/3296890544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad200b4af0 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2590c640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 0x7fad2015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.955+0000 7fad2610d640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 0x7fad201567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad2510b640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c440 tx=0x7fad1000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10017070 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad100035f0 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad20152b10 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad100039e0 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.956+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad200b4d90 con 0x7fad200cfa20 2024-05-09T06:29:31.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.957+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad200752b0 con 0x7fad200cfa20 2024-05-09T06:29:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.958+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad1001a070 con 0x7fad200cfa20 2024-05-09T06:29:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.958+0000 7fad167fc640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 0x7fad0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.958+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5619+0+0 (secure 0 0 0) 0x7fad1008a460 con 0x7fad200cfa20 2024-05-09T06:29:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.958+0000 7fad2590c640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 0x7fad0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.959+0000 7fad2590c640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 0x7fad0406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad2006e220 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:31.962+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad10021020 con 0x7fad200cfa20 2024-05-09T06:29:32.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.146+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fad200b2b00 con 0x7fad200cfa20 2024-05-09T06:29:32.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.822+0000 7fad167fc640 1 -- 172.21.15.1:0/3296890544 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) ==== 170+0+0 (secure 0 0 0) 0x7fad10058fd0 con 0x7fad200cfa20 2024-05-09T06:29:32.804 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:29:32.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.827+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 msgr2=0x7fad0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.827+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 0x7fad0406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad2006e220 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-05-09T06:29:32.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.827+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 msgr2=0x7fad200999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.827+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c440 tx=0x7fad1000c900 comp rx=0 tx=0).stop 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 shutdown_connections 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad0406be30 0x7fad0406e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad2015e170 0x7fad201567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad200cfa20 0x7fad200999e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 --2- 172.21.15.1:0/3296890544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad200b18f0 0x7fad2015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.828+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 >> 172.21.15.1:0/3296890544 conn(0x7fad200b1da0 msgr2=0x7fad200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.829+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 shutdown_connections 2024-05-09T06:29:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.829+0000 7fad27b70640 1 -- 172.21.15.1:0/3296890544 wait complete. 2024-05-09T06:29:32.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:29:32.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.966+0000 7f0f43777640 1 Processor -- start 2024-05-09T06:29:32.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.967+0000 7f0f43777640 1 -- start start 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.967+0000 7f0f43777640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0f3c15e190 0x7f0f3c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.967+0000 7f0f43777640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c161fe0 con 0x7f0f3c0d45f0 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.967+0000 7f0f43777640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c1622c0 con 0x7f0f3c0d1c70 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.967+0000 7f0f43777640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0f3c15f580 con 0x7f0f3c15e190 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41d14640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0f3c0d45f0 0x7f0f3c0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36466/0 (socket says 172.21.15.1:36466) 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41d14640 1 -- 172.21.15.1:0/3867023310 learned_addr learned my addr 172.21.15.1:0/3867023310 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41513640 1 --2- 172.21.15.1:0/3867023310 >> v2:172.21.15.6:3300/0 conn(0x7f0f3c15e190 0x7f0f3c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f3bfff640 1 -- 172.21.15.1:0/3867023310 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979743771 0 0) 0x7f0f3c161fe0 con 0x7f0f3c0d45f0 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f3bfff640 1 -- 172.21.15.1:0/3867023310 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f24003520 con 0x7f0f3c0d45f0 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41513640 1 -- 172.21.15.1:0/3867023310 >> v1:172.21.15.6:6789/0 conn(0x7f0f3c0d1c70 legacy=0x7f0f3c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41513640 1 -- 172.21.15.1:0/3867023310 >> v1:172.21.15.1:6789/0 conn(0x7f0f3c0d45f0 legacy=0x7f0f3c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.968+0000 7f0f41513640 1 -- 172.21.15.1:0/3867023310 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c1625a0 con 0x7f0f3c15e190 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f41513640 1 --2- 172.21.15.1:0/3867023310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c15e190 0x7f0f3c15f780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23fe5cf8911ab18d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f3bfff640 1 -- 172.21.15.1:0/3867023310 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c007740 con 0x7f0f3c15e190 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f3bfff640 1 -- 172.21.15.1:0/3867023310 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0078e0 con 0x7f0f3c15e190 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f3bfff640 1 -- 172.21.15.1:0/3867023310 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c007cb0 con 0x7f0f3c15e190 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f43777640 1 -- 172.21.15.1:0/3867023310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c15e190 msgr2=0x7f0f3c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.969+0000 7f0f43777640 1 --2- 172.21.15.1:0/3867023310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c15e190 0x7f0f3c15f780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c02a6f0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.970+0000 7f0f43777640 1 -- 172.21.15.1:0/3867023310 shutdown_connections 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.970+0000 7f0f43777640 1 --2- 172.21.15.1:0/3867023310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c15e190 0x7f0f3c15f780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.970+0000 7f0f43777640 1 -- 172.21.15.1:0/3867023310 >> 172.21.15.1:0/3867023310 conn(0x7f0f3c073340 msgr2=0x7f0f3c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.970+0000 7f0f43777640 1 -- 172.21.15.1:0/3867023310 shutdown_connections 2024-05-09T06:29:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.970+0000 7f0f43777640 1 -- 172.21.15.1:0/3867023310 wait complete. 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.971+0000 7f0f43777640 1 Processor -- start 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.971+0000 7f0f43777640 1 -- start start 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 0x7f0f3c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f3c0d45f0 0x7f0f3c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0b56d0 con 0x7f0f3c15e190 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c161c60 con 0x7f0f3c0d1c70 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f43777640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0b5850 con 0x7f0f3c0d45f0 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33770/0 (socket says 172.21.15.1:33770) 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 -- 172.21.15.1:0/3815209390 learned_addr learned my addr 172.21.15.1:0/3815209390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f40d12640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f3c0d45f0 0x7f0f3c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41513640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 0x7f0f3c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 -- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f3c0d45f0 msgr2=0x7f0f3c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f3c0d45f0 0x7f0f3c0cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 -- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 msgr2=0x7f0f3c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 0x7f0f3c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 -- 172.21.15.1:0/3815209390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0cc940 con 0x7f0f3c15e190 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41513640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 0x7f0f3c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.972+0000 7f0f41d14640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c400 tx=0x7f0f3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.973+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34017070 con 0x7f0f3c15e190 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.973+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f34005ce0 con 0x7f0f3c15e190 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.973+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c0ccbe0 con 0x7f0f3c15e190 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.973+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f340072d0 con 0x7f0f3c15e190 2024-05-09T06:29:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.973+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c0cd1a0 con 0x7f0f3c15e190 2024-05-09T06:29:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.974+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c0752b0 con 0x7f0f3c15e190 2024-05-09T06:29:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.974+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f3401a070 con 0x7f0f3c15e190 2024-05-09T06:29:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.975+0000 7f0f39ffb640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 0x7f0f2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.975+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6001+0+0 (secure 0 0 0) 0x7f0f3408a190 con 0x7f0f3c15e190 2024-05-09T06:29:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.975+0000 7f0f41513640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 0x7f0f2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:32.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.976+0000 7f0f41513640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 0x7f0f2806e320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c034000 tx=0x7f0f2c033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:32.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:32.980+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f34058450 con 0x7f0f3c15e190 2024-05-09T06:29:33.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.203+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0f3c08e1c0 con 0x7f0f3c15e190 2024-05-09T06:29:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.838+0000 7f0f39ffb640 1 -- 172.21.15.1:0/3815209390 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7f0f3405c750 con 0x7f0f3c15e190 2024-05-09T06:29:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 7 and data pool 8 2024-05-09T06:29:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.849+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 msgr2=0x7f0f2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.849+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 0x7f0f2806e320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c034000 tx=0x7f0f2c033040 comp rx=0 tx=0).stop 2024-05-09T06:29:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.849+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 msgr2=0x7f0f3c159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.849+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c400 tx=0x7f0f3400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 shutdown_connections 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0f2806be60 0x7f0f2806e320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f3c15e190 0x7f0f3c159500 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f3c0d45f0 0x7f0f3c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 --2- 172.21.15.1:0/3815209390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f3c0d1c70 0x7f0f3c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.850+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 >> 172.21.15.1:0/3815209390 conn(0x7f0f3c073340 msgr2=0x7f0f3c15c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:33.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.854+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 shutdown_connections 2024-05-09T06:29:33.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.854+0000 7f0f43777640 1 -- 172.21.15.1:0/3815209390 wait complete. 2024-05-09T06:29:33.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.987+0000 7f03e102a640 1 Processor -- start 2024-05-09T06:29:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.987+0000 7f03e102a640 1 -- start start 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03e102a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc06ffa0 0x7f03dc0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03e102a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03dc15e5b0 0x7f03dc0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03e102a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03dc06c2f0 con 0x7f03dc15e1d0 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03e102a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc0cc3c0 con 0x7f03dc15e5b0 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03e102a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0ccc60 con 0x7f03dc06ffa0 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03dc15e5b0 0x7f03dc0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03dad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f03dc15e1d0 0x7f03dc06f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36470/0 (socket says 172.21.15.1:36470) 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03dc15e5b0 0x7f03dc0d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33772/0 (socket says 172.21.15.1:33772) 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03dad9d640 1 -- 172.21.15.1:0/2298241396 learned_addr learned my addr 172.21.15.1:0/2298241396 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03d3fff640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc06ffa0 0x7f03dc0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 -- 172.21.15.1:0/2298241396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc06ffa0 msgr2=0x7f03dc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc06ffa0 0x7f03dc0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 -- 172.21.15.1:0/2298241396 >> v1:172.21.15.1:6789/0 conn(0x7f03dc15e1d0 legacy=0x7f03dc06f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03da59c640 1 -- 172.21.15.1:0/2298241396 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209834584 0 0) 0x7f03cc001110 con 0x7f03dc15e1d0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.988+0000 7f03db59e640 1 -- 172.21.15.1:0/2298241396 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc06f500 con 0x7f03dc15e5b0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.989+0000 7f03db59e640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc15e5b0 0x7f03dc0d04f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009d90 tx=0x7f03d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a73d3094d3a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.989+0000 7f03da59c640 1 -- 172.21.15.1:0/2298241396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031600 con 0x7f03dc15e5b0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.989+0000 7f03da59c640 1 -- 172.21.15.1:0/2298241396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d40317a0 con 0x7f03dc15e5b0 2024-05-09T06:29:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.989+0000 7f03da59c640 1 -- 172.21.15.1:0/2298241396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031b70 con 0x7f03dc15e5b0 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 -- 172.21.15.1:0/2298241396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc15e5b0 msgr2=0x7f03dc0d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc15e5b0 0x7f03dc0d04f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009d90 tx=0x7f03d402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 -- 172.21.15.1:0/2298241396 shutdown_connections 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc06ffa0 0x7f03dc0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 --2- 172.21.15.1:0/2298241396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc15e5b0 0x7f03dc0d04f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 -- 172.21.15.1:0/2298241396 >> 172.21.15.1:0/2298241396 conn(0x7f03dc0d1d70 msgr2=0x7f03dc0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 -- 172.21.15.1:0/2298241396 shutdown_connections 2024-05-09T06:29:33.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.990+0000 7f03e102a640 1 -- 172.21.15.1:0/2298241396 wait complete. 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.991+0000 7f03e102a640 1 Processor -- start 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.991+0000 7f03e102a640 1 -- start start 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03dc15e1d0 0x7f03dc1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 0x7f03dc0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc06f1b0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0cc8f0 con 0x7f03dc15e1d0 2024-05-09T06:29:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03e102a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0ce100 con 0x7f03dc15e5b0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33780/0 (socket says 172.21.15.1:33780) 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 -- 172.21.15.1:0/2041161155 learned_addr learned my addr 172.21.15.1:0/2041161155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03db59e640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 0x7f03dc0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03d3fff640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03dc15e1d0 0x7f03dc1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 -- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 msgr2=0x7f03dc0afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 0x7f03dc0afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 -- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03dc15e1d0 msgr2=0x7f03dc1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03dc15e1d0 0x7f03dc1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 -- 172.21.15.1:0/2041161155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1f4770 con 0x7f03dc06ffa0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03db59e640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 0x7f03dc0afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03dad9d640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c7b0 tx=0x7f03cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.992+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc018070 con 0x7f03dc06ffa0 2024-05-09T06:29:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.993+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc0072d0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.993+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc1f4a50 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.993+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0047e0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.993+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1577a0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.994+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc0752b0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.994+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03cc01b070 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.995+0000 7f03d37fe640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 0x7f03bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.995+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f03cc08a2e0 con 0x7f03dc06ffa0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.995+0000 7f03d3fff640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 0x7f03bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.995+0000 7f03d3fff640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 0x7f03bc06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b1550 tx=0x7f03c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:33.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:33.999+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03dc06c2f0 con 0x7f03dc06ffa0 2024-05-09T06:29:34.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.184+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03dc0d0de0 con 0x7f03dc06ffa0 2024-05-09T06:29:34.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.185+0000 7f03d37fe640 1 -- 172.21.15.1:0/2041161155 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16823 (secure 0 0 0) 0x7f03cc058b90 con 0x7f03dc06ffa0 2024-05-09T06:29:34.167 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:34.167 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":50,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:33.204635+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 msgr2=0x7f03bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 0x7f03bc06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b1550 tx=0x7f03c4008040 comp rx=0 tx=0).stop 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 msgr2=0x7f03dc157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c7b0 tx=0x7f03cc00cc70 comp rx=0 tx=0).stop 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 shutdown_connections 2024-05-09T06:29:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f03bc06be30 0x7f03bc06e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03dc15e5b0 0x7f03dc0afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03dc15e1d0 0x7f03dc1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 --2- 172.21.15.1:0/2041161155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03dc06ffa0 0x7f03dc157260 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 >> 172.21.15.1:0/2041161155 conn(0x7f03dc0d1d70 msgr2=0x7f03dc06f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 shutdown_connections 2024-05-09T06:29:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.187+0000 7f03e102a640 1 -- 172.21.15.1:0/2041161155 wait complete. 2024-05-09T06:29:34.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:34.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.303+0000 7ff230b09640 1 Processor -- start 2024-05-09T06:29:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.303+0000 7ff230b09640 1 -- start start 2024-05-09T06:29:34.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff230b09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff22c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff230b09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff22c0715d0 0x7ff22c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff230b09640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22c070f40 con 0x7ff22c15e500 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff230b09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff22c0cf170 con 0x7ff22c0715d0 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff230b09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c0cf450 con 0x7ff22c15db10 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22ad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff22c15e500 0x7ff22c06e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36482/0 (socket says 172.21.15.1:36482) 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22ad9d640 1 -- 172.21.15.1:0/4182780371 learned_addr learned my addr 172.21.15.1:0/4182780371 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22a59c640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff22c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff229d9b640 1 --2- 172.21.15.1:0/4182780371 >> v2:172.21.15.6:3300/0 conn(0x7ff22c0715d0 0x7ff22c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553617920 0 0) 0x7ff22c070f40 con 0x7ff22c15e500 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff214003520 con 0x7ff22c15e500 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1039656435 0 0) 0x7ff214003520 con 0x7ff22c15e500 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 msgr2=0x7ff22c0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff22c0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 >> v2:172.21.15.6:3300/0 conn(0x7ff22c0715d0 msgr2=0x7ff22c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 --2- 172.21.15.1:0/4182780371 >> v2:172.21.15.6:3300/0 conn(0x7ff22c0715d0 0x7ff22c15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c070c10 con 0x7ff22c15e500 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7ff2240019f0 con 0x7ff22c15e500 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.304+0000 7ff229d9b640 1 --2- 172.21.15.1:0/4182780371 >> v2:172.21.15.6:3300/0 conn(0x7ff22c0715d0 0x7ff22c15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 >> v1:172.21.15.1:6789/0 conn(0x7ff22c15e500 legacy=0x7ff22c06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff2140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff214003f30 0x7ff214004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c0cf170 con 0x7ff214004d40 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c0cf450 con 0x7ff214003f30 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff21400b9c0 con 0x7ff22c15db10 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22a59c640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff2140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff229d9b640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff214003f30 0x7ff214004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 -- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 msgr2=0x7ff2140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff22c15db10 0x7ff2140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 -- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff214003f30 msgr2=0x7ff214004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff214003f30 0x7ff214004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.305+0000 7ff22ad9d640 1 -- 172.21.15.1:0/4182780371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff214003520 con 0x7ff214004d40 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff2240013d0 tx=0x7ff214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874af7f0788e1357 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22400a4e0 con 0x7ff214004d40 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22402f070 con 0x7ff214004d40 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff22959a640 1 -- 172.21.15.1:0/4182780371 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22402bcd0 con 0x7ff214004d40 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff230b09640 1 -- 172.21.15.1:0/4182780371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff214004d40 msgr2=0x7ff2140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff230b09640 1 --2- 172.21.15.1:0/4182780371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff2240013d0 tx=0x7ff214005830 comp rx=0 tx=0).stop 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.306+0000 7ff22ad9d640 1 -- 172.21.15.1:0/4182780371 reap_dead start 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.307+0000 7ff230b09640 1 -- 172.21.15.1:0/4182780371 shutdown_connections 2024-05-09T06:29:34.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.307+0000 7ff230b09640 1 -- 172.21.15.1:0/4182780371 >> 172.21.15.1:0/4182780371 conn(0x7ff22c0cc4b0 msgr2=0x7ff22c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:34.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.307+0000 7ff230b09640 1 -- 172.21.15.1:0/4182780371 shutdown_connections 2024-05-09T06:29:34.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.307+0000 7ff230b09640 1 -- 172.21.15.1:0/4182780371 wait complete. 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.308+0000 7ff230b09640 1 Processor -- start 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.308+0000 7ff230b09640 1 -- start start 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff214004d40 0x7ff22c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff22c151f80 0x7ff22c14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15edc0 con 0x7ff22c15a240 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15ec40 con 0x7ff22c151f80 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff230b09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c0b2a60 con 0x7ff214004d40 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33810/0 (socket says 172.21.15.1:33810) 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 -- 172.21.15.1:0/1885693210 learned_addr learned my addr 172.21.15.1:0/1885693210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22a59c640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff214004d40 0x7ff22c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff229d9b640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff22c151f80 0x7ff22c14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 -- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff214004d40 msgr2=0x7ff22c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff214004d40 0x7ff22c150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 -- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff22c151f80 msgr2=0x7ff22c14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff22c151f80 0x7ff22c14aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 -- 172.21.15.1:0/1885693210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c14b590 con 0x7ff22c15a240 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff22ad9d640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a0b0 tx=0x7ff224001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224002040 con 0x7ff22c15a240 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22400a4c0 con 0x7ff22c15a240 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.309+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224004d80 con 0x7ff22c15a240 2024-05-09T06:29:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.310+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c150ee0 con 0x7ff22c15a240 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.310+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1514a0 con 0x7ff22c15a240 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.310+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c0752b0 con 0x7ff22c15a240 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.311+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff224034470 con 0x7ff22c15a240 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.312+0000 7ff21affd640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 0x7ff20c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.312+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7ff2240aa900 con 0x7ff22c15a240 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.312+0000 7ff22a59c640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 0x7ff20c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.312+0000 7ff22a59c640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 0x7ff20c06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff22c14be70 tx=0x7ff21c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.316+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff22c14b590 con 0x7ff22c15a240 2024-05-09T06:29:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.503+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff22c0b3fb0 con 0x7ff22c15a240 2024-05-09T06:29:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.504+0000 7ff21affd640 1 -- 172.21.15.1:0/1885693210 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16823 (secure 0 0 0) 0x7ff224078ad0 con 0x7ff22c15a240 2024-05-09T06:29:34.486 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:34.487 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":50,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:33.204635+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 msgr2=0x7ff20c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 0x7ff20c06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff22c14be70 tx=0x7ff21c009190 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 msgr2=0x7ff22c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a0b0 tx=0x7ff224001dc0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 shutdown_connections 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff20c06be30 0x7ff20c06e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff22c15a240 0x7ff22c15ba90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff22c151f80 0x7ff22c14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 --2- 172.21.15.1:0/1885693210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff214004d40 0x7ff22c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.506+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 >> 172.21.15.1:0/1885693210 conn(0x7ff22c0cc4b0 msgr2=0x7ff22c161430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.507+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 shutdown_connections 2024-05-09T06:29:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.507+0000 7ff230b09640 1 -- 172.21.15.1:0/1885693210 wait complete. 2024-05-09T06:29:34.500 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:29:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.625+0000 7f6a13031640 1 Processor -- start 2024-05-09T06:29:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.625+0000 7f6a13031640 1 -- start start 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a13031640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a13031640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a0c0cf8e0 0x7f6a0c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a13031640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c072050 con 0x7f6a0c0c41e0 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a13031640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c0cd160 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a13031640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c072330 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33818/0 (socket says 172.21.15.1:33818) 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 -- 172.21.15.1:0/3407266593 learned_addr learned my addr 172.21.15.1:0/3407266593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a115ce640 1 --2- 172.21.15.1:0/3407266593 >> v2:172.21.15.6:3300/0 conn(0x7f6a0c0cf8e0 0x7f6a0c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 -- 172.21.15.1:0/3407266593 >> v2:172.21.15.6:3300/0 conn(0x7f6a0c0cf8e0 msgr2=0x7f6a0c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 --2- 172.21.15.1:0/3407266593 >> v2:172.21.15.6:3300/0 conn(0x7f6a0c0cf8e0 0x7f6a0c0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 -- 172.21.15.1:0/3407266593 >> v1:172.21.15.6:6789/0 conn(0x7f6a0c0c41e0 legacy=0x7f6a0c161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.626+0000 7f6a10dcd640 1 -- 172.21.15.1:0/3407266593 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c072610 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.627+0000 7f6a10dcd640 1 --2- 172.21.15.1:0/3407266593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0b3780 tx=0x7f6a0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e44f048e6e426fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.627+0000 7f6a037fe640 1 -- 172.21.15.1:0/3407266593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08037070 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.627+0000 7f6a037fe640 1 -- 172.21.15.1:0/3407266593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08032440 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.627+0000 7f6a037fe640 1 -- 172.21.15.1:0/3407266593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08032810 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 -- 172.21.15.1:0/3407266593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0d06f0 msgr2=0x7f6a0c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 --2- 172.21.15.1:0/3407266593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0b3780 tx=0x7f6a0802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 -- 172.21.15.1:0/3407266593 shutdown_connections 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 --2- 172.21.15.1:0/3407266593 >> v2:172.21.15.6:3300/0 conn(0x7f6a0c0cf8e0 0x7f6a0c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 --2- 172.21.15.1:0/3407266593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0d06f0 0x7f6a0c0cc6e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 -- 172.21.15.1:0/3407266593 >> 172.21.15.1:0/3407266593 conn(0x7f6a0c0d0b50 msgr2=0x7f6a0c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 -- 172.21.15.1:0/3407266593 shutdown_connections 2024-05-09T06:29:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.628+0000 7f6a13031640 1 -- 172.21.15.1:0/3407266593 wait complete. 2024-05-09T06:29:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.629+0000 7f6a13031640 1 Processor -- start 2024-05-09T06:29:34.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.629+0000 7f6a13031640 1 -- start start 2024-05-09T06:29:34.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a0c0c41e0 0x7f6a0c1f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a0c0d06f0 0x7f6a0c1e3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c06b1e0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c071ce0 con 0x7f6a0c0c41e0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a13031640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c06b360 con 0x7f6a0c0d06f0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33832/0 (socket says 172.21.15.1:33832) 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 -- 172.21.15.1:0/3754477899 learned_addr learned my addr 172.21.15.1:0/3754477899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a10dcd640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a0c0c41e0 0x7f6a0c1f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a115ce640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a0c0d06f0 0x7f6a0c1e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 -- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a0c0d06f0 msgr2=0x7f6a0c1e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a0c0d06f0 0x7f6a0c1e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 -- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a0c0c41e0 msgr2=0x7f6a0c1f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a0c0c41e0 0x7f6a0c1f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:34.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.630+0000 7f6a03fff640 1 -- 172.21.15.1:0/3754477899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0a0ee0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a03fff640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c460 tx=0x7f69fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc016020 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc012870 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc012c60 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c0a11c0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.631+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c0a1780 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.632+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c0752b0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.633+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69fc019020 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.633+0000 7f6a017fa640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 0x7f69f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:34.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.633+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f69fc08abd0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.633+0000 7f6a10dcd640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 0x7f69f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:34.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.634+0000 7f6a10dcd640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 0x7f69f806e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a08034000 tx=0x7f6a0802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:34.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.637+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f69fc020030 con 0x7f6a0c0cf8e0 2024-05-09T06:29:34.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:34.863+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6a0c072f30 con 0x7f6a0c0cf8e0 2024-05-09T06:29:35.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.853+0000 7f6a017fa640 1 -- 172.21.15.1:0/3754477899 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f69fc058da0 con 0x7f6a0c0cf8e0 2024-05-09T06:29:35.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.855+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 msgr2=0x7f69f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.855+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 0x7f69f806e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a08034000 tx=0x7f6a0802e000 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.855+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 msgr2=0x7f6a0c0a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.855+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c460 tx=0x7f69fc00c920 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 shutdown_connections 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69f806be30 0x7f69f806e2f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a0c0d06f0 0x7f6a0c1e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c0cf8e0 0x7f6a0c0a0420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 --2- 172.21.15.1:0/3754477899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a0c0c41e0 0x7f6a0c1f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 >> 172.21.15.1:0/3754477899 conn(0x7f6a0c0d0b50 msgr2=0x7f6a0c06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 shutdown_connections 2024-05-09T06:29:35.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.856+0000 7f6a13031640 1 -- 172.21.15.1:0/3754477899 wait complete. 2024-05-09T06:29:35.851 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:29:35.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:29:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.983+0000 7f3e66672640 1 Processor -- start 2024-05-09T06:29:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.983+0000 7f3e66672640 1 -- start start 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e66672640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e600d45f0 0x7f3e6015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e66672640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e60161fe0 con 0x7f3e6015e190 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e66672640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e601622c0 con 0x7f3e600d1c70 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e66672640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6015f580 con 0x7f3e600d45f0 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e600d45f0 0x7f3e6015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e600d45f0 0x7f3e6015f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33836/0 (socket says 172.21.15.1:33836) 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e5ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3e6015e190 0x7f3e600d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36494/0 (socket says 172.21.15.1:36494) 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 -- 172.21.15.1:0/683166597 learned_addr learned my addr 172.21.15.1:0/683166597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 -- 172.21.15.1:0/683166597 >> v1:172.21.15.6:6789/0 conn(0x7f3e600d1c70 legacy=0x7f3e600b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 -- 172.21.15.1:0/683166597 >> v1:172.21.15.1:6789/0 conn(0x7f3e6015e190 legacy=0x7f3e600d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.984+0000 7f3e64c0f640 1 -- 172.21.15.1:0/683166597 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e601625a0 con 0x7f3e600d45f0 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.985+0000 7f3e64c0f640 1 --2- 172.21.15.1:0/683166597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e600d45f0 0x7f3e6015f780 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e5802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754806b542429183 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.985+0000 7f3e5f7fe640 1 -- 172.21.15.1:0/683166597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58036070 con 0x7f3e600d45f0 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.985+0000 7f3e5f7fe640 1 -- 172.21.15.1:0/683166597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58031600 con 0x7f3e600d45f0 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.985+0000 7f3e5f7fe640 1 -- 172.21.15.1:0/683166597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580319d0 con 0x7f3e600d45f0 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 -- 172.21.15.1:0/683166597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e600d45f0 msgr2=0x7f3e6015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 --2- 172.21.15.1:0/683166597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e600d45f0 0x7f3e6015f780 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e5802a760 comp rx=0 tx=0).stop 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 -- 172.21.15.1:0/683166597 shutdown_connections 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 --2- 172.21.15.1:0/683166597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e600d45f0 0x7f3e6015f780 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 -- 172.21.15.1:0/683166597 >> 172.21.15.1:0/683166597 conn(0x7f3e60073340 msgr2=0x7f3e6009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 -- 172.21.15.1:0/683166597 shutdown_connections 2024-05-09T06:29:35.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.986+0000 7f3e66672640 1 -- 172.21.15.1:0/683166597 wait complete. 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.987+0000 7f3e66672640 1 Processor -- start 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.987+0000 7f3e66672640 1 -- start start 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e600d1c70 0x7f3e601f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e600d45f0 0x7f3e601f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600b56d0 con 0x7f3e6015e190 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60161c60 con 0x7f3e600d45f0 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e66672640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600b5850 con 0x7f3e600d1c70 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33844/0 (socket says 172.21.15.1:33844) 2024-05-09T06:29:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 -- 172.21.15.1:0/3948409659 learned_addr learned my addr 172.21.15.1:0/3948409659 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e5ffff640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e600d1c70 0x7f3e601f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 -- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e600d1c70 msgr2=0x7f3e601f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e57fff640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e600d45f0 0x7f3e601f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e600d1c70 0x7f3e601f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 -- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e600d45f0 msgr2=0x7f3e601f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e600d45f0 0x7f3e601f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 -- 172.21.15.1:0/3948409659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e601f7dd0 con 0x7f3e6015e190 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.988+0000 7f3e64c0f640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.989+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58041020 con 0x7f3e6015e190 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.989+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58007780 con 0x7f3e6015e190 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.989+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58036040 con 0x7f3e6015e190 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.989+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e601f80b0 con 0x7f3e6015e190 2024-05-09T06:29:35.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.989+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e600b6e60 con 0x7f3e6015e190 2024-05-09T06:29:35.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.990+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e600752b0 con 0x7f3e6015e190 2024-05-09T06:29:35.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.990+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e58044020 con 0x7f3e6015e190 2024-05-09T06:29:35.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.991+0000 7f3e5d7fa640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 0x7f3e4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:35.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.991+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f3e580aba00 con 0x7f3e6015e190 2024-05-09T06:29:35.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.991+0000 7f3e5ffff640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 0x7f3e4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:35.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.991+0000 7f3e5ffff640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 0x7f3e4006e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000f60 tx=0x7f3e50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:35.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:35.995+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e5804b070 con 0x7f3e6015e190 2024-05-09T06:29:36.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.216+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3e6008e1c0 con 0x7f3e6015e190 2024-05-09T06:29:36.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.857+0000 7f3e5d7fa640 1 -- 172.21.15.1:0/3948409659 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f3e5807a2b0 con 0x7f3e6015e190 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 msgr2=0x7f3e4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 0x7f3e4006e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000f60 tx=0x7f3e50009040 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 msgr2=0x7f3e60158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e58004a40 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 shutdown_connections 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e4006be60 0x7f3e4006e320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6015e190 0x7f3e60158ca0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e600d45f0 0x7f3e601f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 --2- 172.21.15.1:0/3948409659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e600d1c70 0x7f3e601f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 >> 172.21.15.1:0/3948409659 conn(0x7f3e60073340 msgr2=0x7f3e6006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:36.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 shutdown_connections 2024-05-09T06:29:36.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.860+0000 7f3e66672640 1 -- 172.21.15.1:0/3948409659 wait complete. 2024-05-09T06:29:36.855 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:29:36.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.983+0000 7f4b3b441640 1 Processor -- start 2024-05-09T06:29:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.984+0000 7f4b3b441640 1 -- start start 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.984+0000 7f4b3b441640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4b340d2b10 0x7f4b340b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b3b441640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b340b33c0 con 0x7f4b340d4940 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b3b441640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b340b3660 con 0x7f4b34161950 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b3b441640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4b340d4dd0 con 0x7f4b340d2b10 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4b340d2b10 0x7f4b340b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4b340d2b10 0x7f4b340b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33858/0 (socket says 172.21.15.1:33858) 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 -- 172.21.15.1:0/2468848205 learned_addr learned my addr 172.21.15.1:0/2468848205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 -- 172.21.15.1:0/2468848205 >> v1:172.21.15.6:6789/0 conn(0x7f4b34161950 legacy=0x7f4b340d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 -- 172.21.15.1:0/2468848205 >> v1:172.21.15.1:6789/0 conn(0x7f4b340d4940 legacy=0x7f4b340b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.985+0000 7f4b399de640 1 -- 172.21.15.1:0/2468848205 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b340b3940 con 0x7f4b340d2b10 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b399de640 1 --2- 172.21.15.1:0/2468848205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b340d2b10 0x7f4b340b4850 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009e50 tx=0x7f4b2c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10092cd9549da95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b2bfff640 1 -- 172.21.15.1:0/2468848205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c036070 con 0x7f4b340d2b10 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b2bfff640 1 -- 172.21.15.1:0/2468848205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c031600 con 0x7f4b340d2b10 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b2bfff640 1 -- 172.21.15.1:0/2468848205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0319f0 con 0x7f4b340d2b10 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b3b441640 1 -- 172.21.15.1:0/2468848205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b340d2b10 msgr2=0x7f4b340b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2468848205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b340d2b10 0x7f4b340b4850 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009e50 tx=0x7f4b2c02a7c0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b3b441640 1 -- 172.21.15.1:0/2468848205 shutdown_connections 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2468848205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b340d2b10 0x7f4b340b4850 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.986+0000 7f4b3b441640 1 -- 172.21.15.1:0/2468848205 >> 172.21.15.1:0/2468848205 conn(0x7f4b340d3e20 msgr2=0x7f4b340d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.987+0000 7f4b3b441640 1 -- 172.21.15.1:0/2468848205 shutdown_connections 2024-05-09T06:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.987+0000 7f4b3b441640 1 -- 172.21.15.1:0/2468848205 wait complete. 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.988+0000 7f4b3b441640 1 Processor -- start 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.988+0000 7f4b3b441640 1 -- start start 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.988+0000 7f4b3b441640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340d2b10 0x7f4b340cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 0x7f4b3406c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b340d1100 con 0x7f4b34161950 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b340d0f80 con 0x7f4b340d2b10 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4b340d1c00 con 0x7f4b340b6a10 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33864/0 (socket says 172.21.15.1:33864) 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 -- 172.21.15.1:0/2480159199 learned_addr learned my addr 172.21.15.1:0/2480159199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b391dd640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340d2b10 0x7f4b340cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b399de640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 0x7f4b3406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 -- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 msgr2=0x7f4b3406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 0x7f4b3406c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 -- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340d2b10 msgr2=0x7f4b340cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340d2b10 0x7f4b340cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 -- 172.21.15.1:0/2480159199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b340b6ea0 con 0x7f4b34161950 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b399de640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 0x7f4b3406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b389dc640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c450 tx=0x7f4b2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240052d0 con 0x7f4b34161950 2024-05-09T06:29:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b24005470 con 0x7f4b34161950 2024-05-09T06:29:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24012870 con 0x7f4b34161950 2024-05-09T06:29:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.989+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b340b7180 con 0x7f4b34161950 2024-05-09T06:29:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.990+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b340b7740 con 0x7f4b34161950 2024-05-09T06:29:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.991+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b340752b0 con 0x7f4b34161950 2024-05-09T06:29:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.991+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b24012a10 con 0x7f4b34161950 2024-05-09T06:29:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.991+0000 7f4b29ffb640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.992+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4b2408acc0 con 0x7f4b34161950 2024-05-09T06:29:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.992+0000 7f4b391dd640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.992+0000 7f4b391dd640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 0x7f4b1806e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000f60 tx=0x7f4b30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:36.996+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b24091080 con 0x7f4b34161950 2024-05-09T06:29:37.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.222+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b34074650 con 0x7f4b34161950 2024-05-09T06:29:37.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.223+0000 7f4b29ffb640 1 -- 172.21.15.1:0/2480159199 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7f4b24059440 con 0x7f4b34161950 2024-05-09T06:29:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:37.205 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:29:37.205 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 msgr2=0x7f4b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 0x7f4b1806e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000f60 tx=0x7f4b30009040 comp rx=0 tx=0).stop 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 msgr2=0x7f4b341570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c450 tx=0x7f4b2400c910 comp rx=0 tx=0).stop 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 shutdown_connections 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b340b6a10 0x7f4b3406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b34161950 0x7f4b341570e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 --2- 172.21.15.1:0/2480159199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b340d2b10 0x7f4b340cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 >> 172.21.15.1:0/2480159199 conn(0x7f4b340d3e20 msgr2=0x7f4b340d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:37.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 shutdown_connections 2024-05-09T06:29:37.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.225+0000 7f4b3b441640 1 -- 172.21.15.1:0/2480159199 wait complete. 2024-05-09T06:29:37.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:29:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.341+0000 7f895af28640 1 Processor -- start 2024-05-09T06:29:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.341+0000 7f895af28640 1 -- start start 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f895af28640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f89540d07b0 0x7f89540b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f895af28640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f89540b0d00 0x7f895406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f895af28640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89540d2c80 con 0x7f895415c320 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f895af28640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f89540d29a0 con 0x7f89540b0d00 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f895af28640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f89540cd430 con 0x7f89540d07b0 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f89540b0d00 0x7f895406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f89540b0d00 0x7f895406f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33876/0 (socket says 172.21.15.1:33876) 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 -- 172.21.15.1:0/3189985167 learned_addr learned my addr 172.21.15.1:0/3189985167 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f894bfff640 1 --2- 172.21.15.1:0/3189985167 >> v2:172.21.15.6:3300/0 conn(0x7f89540d07b0 0x7f89540b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 -- 172.21.15.1:0/3189985167 >> v2:172.21.15.6:3300/0 conn(0x7f89540d07b0 msgr2=0x7f89540b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 --2- 172.21.15.1:0/3189985167 >> v2:172.21.15.6:3300/0 conn(0x7f89540d07b0 0x7f89540b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 -- 172.21.15.1:0/3189985167 >> v1:172.21.15.6:6789/0 conn(0x7f895415c320 legacy=0x7f89540d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.342+0000 7f89594c5640 1 -- 172.21.15.1:0/3189985167 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89540cd710 con 0x7f89540b0d00 2024-05-09T06:29:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f89594c5640 1 --2- 172.21.15.1:0/3189985167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540b0d00 0x7f895406f010 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f895406c1c0 tx=0x7f894c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be0f6256cfddd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f894b7fe640 1 -- 172.21.15.1:0/3189985167 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c036070 con 0x7f89540b0d00 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f894b7fe640 1 -- 172.21.15.1:0/3189985167 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c031600 con 0x7f89540b0d00 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f894b7fe640 1 -- 172.21.15.1:0/3189985167 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0319d0 con 0x7f89540b0d00 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f895af28640 1 -- 172.21.15.1:0/3189985167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540b0d00 msgr2=0x7f895406f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.343+0000 7f895af28640 1 --2- 172.21.15.1:0/3189985167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540b0d00 0x7f895406f010 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f895406c1c0 tx=0x7f894c02a780 comp rx=0 tx=0).stop 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 -- 172.21.15.1:0/3189985167 shutdown_connections 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 --2- 172.21.15.1:0/3189985167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540b0d00 0x7f895406f010 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 --2- 172.21.15.1:0/3189985167 >> v2:172.21.15.6:3300/0 conn(0x7f89540d07b0 0x7f89540b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 -- 172.21.15.1:0/3189985167 >> 172.21.15.1:0/3189985167 conn(0x7f89540d1d70 msgr2=0x7f89540d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 -- 172.21.15.1:0/3189985167 shutdown_connections 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.344+0000 7f895af28640 1 -- 172.21.15.1:0/3189985167 wait complete. 2024-05-09T06:29:37.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.345+0000 7f895af28640 1 Processor -- start 2024-05-09T06:29:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.345+0000 7f895af28640 1 -- start start 2024-05-09T06:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 0x7f895409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f895415c320 0x7f89541f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f895406f550 con 0x7f89540d07b0 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f89540cfde0 con 0x7f895415c320 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f895af28640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f895406f6d0 con 0x7f89540b0d00 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33890/0 (socket says 172.21.15.1:33890) 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 -- 172.21.15.1:0/1854500960 learned_addr learned my addr 172.21.15.1:0/1854500960 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f89594c5640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f895415c320 0x7f89541f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f8958cc4640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 0x7f895409bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 -- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 msgr2=0x7f895409bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 0x7f895409bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 -- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f895415c320 msgr2=0x7f89541f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f895415c320 0x7f89541f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f894bfff640 1 -- 172.21.15.1:0/1854500960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89540b7540 con 0x7f89540d07b0 2024-05-09T06:29:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.346+0000 7f8958cc4640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 0x7f895409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f894bfff640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f894400c7f0 tx=0x7f894400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944015070 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8944004240 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895409c720 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944003610 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.347+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895409cce0 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.348+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89540752b0 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.348+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89440037b0 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.349+0000 7f89497fa640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 0x7f894006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.349+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f894408ae60 con 0x7f89540d07b0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.349+0000 7f8958cc4640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 0x7f894006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.349+0000 7f8958cc4640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 0x7f894006e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f895000a810 tx=0x7f8950008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.353+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8944059660 con 0x7f89540d07b0 2024-05-09T06:29:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.541+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89540d17e0 con 0x7f89540d07b0 2024-05-09T06:29:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.543+0000 7f89497fa640 1 -- 172.21.15.1:0/1854500960 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16810 (secure 0 0 0) 0x7f8944058f60 con 0x7f89540d07b0 2024-05-09T06:29:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:37.525 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":51,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:29:34.838700+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 msgr2=0x7f894006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 0x7f894006e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f895000a810 tx=0x7f8950008040 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 msgr2=0x7f895409c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f894400c7f0 tx=0x7f894400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 shutdown_connections 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f894006be30 0x7f894006e2f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f895415c320 0x7f89541f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89540d07b0 0x7f895409c0f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 --2- 172.21.15.1:0/1854500960 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89540b0d00 0x7f895409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 >> 172.21.15.1:0/1854500960 conn(0x7f89540d1d70 msgr2=0x7f89540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 shutdown_connections 2024-05-09T06:29:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.545+0000 7f895af28640 1 -- 172.21.15.1:0/1854500960 wait complete. 2024-05-09T06:29:37.539 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:29:37.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:29:37.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.661+0000 7ff97a17e640 1 Processor -- start 2024-05-09T06:29:37.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.661+0000 7ff97a17e640 1 -- start start 2024-05-09T06:29:37.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff97a17e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff97406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff97a17e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff974070f40 con 0x7ff97415db10 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff97a17e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9740cf170 con 0x7ff9740b2a60 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff97a17e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff9740cf450 con 0x7ff97415e6d0 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff9737fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff97415db10 0x7ff9740715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36516/0 (socket says 172.21.15.1:36516) 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff9737fe640 1 -- 172.21.15.1:0/2521854292 learned_addr learned my addr 172.21.15.1:0/2521854292 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337840808 0 0) 0x7ff974070f40 con 0x7ff97415db10 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff973fff640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff97406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.662+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95c003440 con 0x7ff97415db10 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3271940881 0 0) 0x7ff95c003440 con 0x7ff97415db10 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 msgr2=0x7ff97406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff97406e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 >> v1:172.21.15.6:6789/0 conn(0x7ff9740b2a60 legacy=0x7ff97415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff974070c10 con 0x7ff97415db10 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7ff964001670 con 0x7ff97415db10 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 >> v1:172.21.15.1:6789/0 conn(0x7ff97415db10 legacy=0x7ff9740715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff95c003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff95c004050 0x7ff95c00a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff95c00aff0 0x7ff95c00d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740cf450 con 0x7ff95c004050 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c00dd60 con 0x7ff95c00aff0 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff95c00df60 con 0x7ff97415e6d0 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff95c004050 0x7ff95c00a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff973fff640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff95c00aff0 0x7ff95c00d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9737fe640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff95c003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 -- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 msgr2=0x7ff95c003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff95c003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 -- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff95c00aff0 msgr2=0x7ff95c00d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff95c00aff0 0x7ff95c00d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff972ffd640 1 -- 172.21.15.1:0/2521854292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c003440 con 0x7ff95c004050 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.663+0000 7ff9737fe640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff97415e6d0 0x7ff95c003ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.664+0000 7ff972ffd640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff95c004050 0x7ff95c00a860 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff968001750 tx=0x7ff95c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f70589473cef082 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.664+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968001370 con 0x7ff95c004050 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.664+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9680052d0 con 0x7ff95c004050 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.664+0000 7ff9727fc640 1 -- 172.21.15.1:0/2521854292 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96802abe0 con 0x7ff95c004050 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.664+0000 7ff97a17e640 1 -- 172.21.15.1:0/2521854292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff95c004050 msgr2=0x7ff95c00a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff97a17e640 1 --2- 172.21.15.1:0/2521854292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff95c004050 0x7ff95c00a860 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff968001750 tx=0x7ff95c003730 comp rx=0 tx=0).stop 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff973fff640 1 -- 172.21.15.1:0/2521854292 reap_dead start 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff97a17e640 1 -- 172.21.15.1:0/2521854292 shutdown_connections 2024-05-09T06:29:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff97a17e640 1 -- 172.21.15.1:0/2521854292 >> 172.21.15.1:0/2521854292 conn(0x7ff9740cc4b0 msgr2=0x7ff9740cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff97a17e640 1 -- 172.21.15.1:0/2521854292 shutdown_connections 2024-05-09T06:29:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.665+0000 7ff97a17e640 1 -- 172.21.15.1:0/2521854292 wait complete. 2024-05-09T06:29:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.666+0000 7ff97a17e640 1 Processor -- start 2024-05-09T06:29:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.666+0000 7ff97a17e640 1 -- start start 2024-05-09T06:29:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff95c004050 0x7ff9740c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9740aeb50 0x7ff9740c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff97406cd30 con 0x7ff9740c68a0 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff97415ee50 con 0x7ff9740aeb50 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff97a17e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff97406ceb0 con 0x7ff95c004050 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33902/0 (socket says 172.21.15.1:33902) 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 -- 172.21.15.1:0/954389649 learned_addr learned my addr 172.21.15.1:0/954389649 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff9737fe640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff95c004050 0x7ff9740c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 -- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff95c004050 msgr2=0x7ff9740c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff95c004050 0x7ff9740c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 -- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9740aeb50 msgr2=0x7ff9740c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9740aeb50 0x7ff9740c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 -- 172.21.15.1:0/954389649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740c50e0 con 0x7ff9740c68a0 2024-05-09T06:29:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff973fff640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff9740b2790 tx=0x7ff96c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.667+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00fde0 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.668+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c015430 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.668+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9740c53c0 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.668+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c015820 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.668+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9740aefb0 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.669+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9740752b0 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.669+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff96c0159c0 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.669+0000 7ff95bfff640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.670+0000 7ff9737fe640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.670+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7ff96c08be50 con 0x7ff9740c68a0 2024-05-09T06:29:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.670+0000 7ff9737fe640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 0x7ff96006e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff964001eb0 tx=0x7ff964000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:37.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.674+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff96c019030 con 0x7ff9740c68a0 2024-05-09T06:29:37.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.857+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9740af270 con 0x7ff96006be30 2024-05-09T06:29:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.863+0000 7ff95bfff640 1 -- 172.21.15.1:0/954389649 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7ff9740af270 con 0x7ff96006be30 2024-05-09T06:29:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:37.849 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 msgr2=0x7ff96006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 0x7ff96006e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff964001eb0 tx=0x7ff964000ed0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 msgr2=0x7ff9740cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff9740b2790 tx=0x7ff96c00f580 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 shutdown_connections 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff96006be30 0x7ff96006e2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9740c68a0 0x7ff9740cb450 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9740aeb50 0x7ff9740c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 --2- 172.21.15.1:0/954389649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff95c004050 0x7ff9740c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 >> 172.21.15.1:0/954389649 conn(0x7ff9740cc4b0 msgr2=0x7ff97406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 shutdown_connections 2024-05-09T06:29:37.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:37.866+0000 7ff97a17e640 1 -- 172.21.15.1:0/954389649 wait complete. 2024-05-09T06:29:37.859 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-05-09T06:29:37.096005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238428,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736292,"statfs":{"total":773094113280,"available":772849963008,"internally_reserved":0,"allocated":5742592,"data_stored":3179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.274802"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827195+0000","last_change":"2024-05-09T06:29:32.827195+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827195+0000","last_undegraded":"2024-05-09T06:29:32.827195+0000","last_fullsized":"2024-05-09T06:29:32.827195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830494+0000","last_change":"2024-05-09T06:29:32.830494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830494+0000","last_undegraded":"2024-05-09T06:29:32.830494+0000","last_fullsized":"2024-05-09T06:29:32.830494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840400+0000","last_change":"2024-05-09T06:29:32.840400+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840400+0000","last_undegraded":"2024-05-09T06:29:32.840400+0000","last_fullsized":"2024-05-09T06:29:32.840400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828893+0000","last_change":"2024-05-09T06:29:32.828893+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828893+0000","last_undegraded":"2024-05-09T06:29:32.828893+0000","last_fullsized":"2024-05-09T06:29:32.828893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832054+0000","last_change":"2024-05-09T06:29:32.832054+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832054+0000","last_undegraded":"2024-05-09T06:29:32.832054+0000","last_fullsized":"2024-05-09T06:29:32.832054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826494+0000","last_change":"2024-05-09T06:29:32.826494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826494+0000","last_undegraded":"2024-05-09T06:29:32.826494+0000","last_fullsized":"2024-05-09T06:29:32.826494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840466+0000","last_change":"2024-05-09T06:29:32.840466+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840466+0000","last_undegraded":"2024-05-09T06:29:32.840466+0000","last_fullsized":"2024-05-09T06:29:32.840466+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834488+0000","last_change":"2024-05-09T06:29:32.834488+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834488+0000","last_undegraded":"2024-05-09T06:29:32.834488+0000","last_fullsized":"2024-05-09T06:29:32.834488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835155+0000","last_change":"2024-05-09T06:29:32.835155+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835155+0000","last_undegraded":"2024-05-09T06:29:32.835155+0000","last_fullsized":"2024-05-09T06:29:32.835155+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827341+0000","last_change":"2024-05-09T06:29:32.827341+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827341+0000","last_undegraded":"2024-05-09T06:29:32.827341+0000","last_fullsized":"2024-05-09T06:29:32.827341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828942+0000","last_change":"2024-05-09T06:29:32.828942+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828942+0000","last_undegraded":"2024-05-09T06:29:32.828942+0000","last_fullsized":"2024-05-09T06:29:32.828942+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832892+0000","last_change":"2024-05-09T06:29:32.832892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832892+0000","last_undegraded":"2024-05-09T06:29:32.832892+0000","last_fullsized":"2024-05-09T06:29:32.832892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827743+0000","last_change":"2024-05-09T06:29:32.827743+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827743+0000","last_undegraded":"2024-05-09T06:29:32.827743+0000","last_fullsized":"2024-05-09T06:29:32.827743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836723+0000","last_change":"2024-05-09T06:29:32.836723+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836723+0000","last_undegraded":"2024-05-09T06:29:32.836723+0000","last_fullsized":"2024-05-09T06:29:32.836723+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828834+0000","last_change":"2024-05-09T06:29:32.828834+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828834+0000","last_undegraded":"2024-05-09T06:29:32.828834+0000","last_fullsized":"2024-05-09T06:29:32.828834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826180+0000","last_change":"2024-05-09T06:29:32.826180+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826180+0000","last_undegraded":"2024-05-09T06:29:32.826180+0000","last_fullsized":"2024-05-09T06:29:32.826180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836265+0000","last_change":"2024-05-09T06:29:32.836265+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836265+0000","last_undegraded":"2024-05-09T06:29:32.836265+0000","last_fullsized":"2024-05-09T06:29:32.836265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830004+0000","last_change":"2024-05-09T06:29:32.830004+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830004+0000","last_undegraded":"2024-05-09T06:29:32.830004+0000","last_fullsized":"2024-05-09T06:29:32.830004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835867+0000","last_change":"2024-05-09T06:29:32.835867+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835867+0000","last_undegraded":"2024-05-09T06:29:32.835867+0000","last_fullsized":"2024-05-09T06:29:32.835867+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.837023+0000","last_change":"2024-05-09T06:29:32.837023+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.837023+0000","last_undegraded":"2024-05-09T06:29:32.837023+0000","last_fullsized":"2024-05-09T06:29:32.837023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836127+0000","last_change":"2024-05-09T06:29:32.836127+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836127+0000","last_undegraded":"2024-05-09T06:29:32.836127+0000","last_fullsized":"2024-05-09T06:29:32.836127+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832922+0000","last_change":"2024-05-09T06:29:32.832922+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832922+0000","last_undegraded":"2024-05-09T06:29:32.832922+0000","last_fullsized":"2024-05-09T06:29:32.832922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828814+0000","last_change":"2024-05-09T06:29:32.828814+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828814+0000","last_undegraded":"2024-05-09T06:29:32.828814+0000","last_fullsized":"2024-05-09T06:29:32.828814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835263+0000","last_change":"2024-05-09T06:29:32.835263+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835263+0000","last_undegraded":"2024-05-09T06:29:32.835263+0000","last_fullsized":"2024-05-09T06:29:32.835263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834730+0000","last_change":"2024-05-09T06:29:32.834730+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834730+0000","last_undegraded":"2024-05-09T06:29:32.834730+0000","last_fullsized":"2024-05-09T06:29:32.834730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834843+0000","last_change":"2024-05-09T06:29:32.834843+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834843+0000","last_undegraded":"2024-05-09T06:29:32.834843+0000","last_fullsized":"2024-05-09T06:29:32.834843+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826703+0000","last_change":"2024-05-09T06:29:32.826703+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826703+0000","last_undegraded":"2024-05-09T06:29:32.826703+0000","last_fullsized":"2024-05-09T06:29:32.826703+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836464+0000","last_change":"2024-05-09T06:29:32.836464+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836464+0000","last_undegraded":"2024-05-09T06:29:32.836464+0000","last_fullsized":"2024-05-09T06:29:32.836464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834001+0000","last_change":"2024-05-09T06:29:32.834001+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834001+0000","last_undegraded":"2024-05-09T06:29:32.834001+0000","last_fullsized":"2024-05-09T06:29:32.834001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829477+0000","last_change":"2024-05-09T06:29:32.829477+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829477+0000","last_undegraded":"2024-05-09T06:29:32.829477+0000","last_fullsized":"2024-05-09T06:29:32.829477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833492+0000","last_change":"2024-05-09T06:29:32.833492+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833492+0000","last_undegraded":"2024-05-09T06:29:32.833492+0000","last_fullsized":"2024-05-09T06:29:32.833492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833542+0000","last_change":"2024-05-09T06:29:32.833542+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833542+0000","last_undegraded":"2024-05-09T06:29:32.833542+0000","last_fullsized":"2024-05-09T06:29:32.833542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828830+0000","last_change":"2024-05-09T06:29:32.828830+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828830+0000","last_undegraded":"2024-05-09T06:29:32.828830+0000","last_fullsized":"2024-05-09T06:29:32.828830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849652+0000","last_change":"2024-05-09T06:29:32.843136+0000","last_active":"2024-05-09T06:29:32.849652+0000","last_peered":"2024-05-09T06:29:32.849652+0000","last_clean":"2024-05-09T06:29:32.849652+0000","last_became_active":"2024-05-09T06:29:32.842842+0000","last_became_peered":"2024-05-09T06:29:32.842842+0000","last_unstale":"2024-05-09T06:29:32.849652+0000","last_undegraded":"2024-05-09T06:29:32.849652+0000","last_fullsized":"2024-05-09T06:29:32.849652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835845+0000","last_change":"2024-05-09T06:29:32.835845+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835845+0000","last_undegraded":"2024-05-09T06:29:32.835845+0000","last_fullsized":"2024-05-09T06:29:32.835845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843214+0000","last_change":"2024-05-09T06:29:32.842035+0000","last_active":"2024-05-09T06:29:32.843214+0000","last_peered":"2024-05-09T06:29:32.843214+0000","last_clean":"2024-05-09T06:29:32.843214+0000","last_became_active":"2024-05-09T06:29:32.841756+0000","last_became_peered":"2024-05-09T06:29:32.841756+0000","last_unstale":"2024-05-09T06:29:32.843214+0000","last_undegraded":"2024-05-09T06:29:32.843214+0000","last_fullsized":"2024-05-09T06:29:32.843214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836568+0000","last_change":"2024-05-09T06:29:32.836568+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836568+0000","last_undegraded":"2024-05-09T06:29:32.836568+0000","last_fullsized":"2024-05-09T06:29:32.836568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844302+0000","last_change":"2024-05-09T06:29:32.842788+0000","last_active":"2024-05-09T06:29:32.844302+0000","last_peered":"2024-05-09T06:29:32.844302+0000","last_clean":"2024-05-09T06:29:32.844302+0000","last_became_active":"2024-05-09T06:29:32.842461+0000","last_became_peered":"2024-05-09T06:29:32.842461+0000","last_unstale":"2024-05-09T06:29:32.844302+0000","last_undegraded":"2024-05-09T06:29:32.844302+0000","last_fullsized":"2024-05-09T06:29:32.844302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836686+0000","last_change":"2024-05-09T06:29:32.836686+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836686+0000","last_undegraded":"2024-05-09T06:29:32.836686+0000","last_fullsized":"2024-05-09T06:29:32.836686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852441+0000","last_change":"2024-05-09T06:29:32.845755+0000","last_active":"2024-05-09T06:29:32.852441+0000","last_peered":"2024-05-09T06:29:32.852441+0000","last_clean":"2024-05-09T06:29:32.852441+0000","last_became_active":"2024-05-09T06:29:32.845443+0000","last_became_peered":"2024-05-09T06:29:32.845443+0000","last_unstale":"2024-05-09T06:29:32.852441+0000","last_undegraded":"2024-05-09T06:29:32.852441+0000","last_fullsized":"2024-05-09T06:29:32.852441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:37.820825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833767+0000","last_change":"2024-05-09T06:29:32.833767+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833767+0000","last_undegraded":"2024-05-09T06:29:32.833767+0000","last_fullsized":"2024-05-09T06:29:32.833767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844351+0000","last_change":"2024-05-09T06:29:32.831050+0000","last_active":"2024-05-09T06:29:32.844351+0000","last_peered":"2024-05-09T06:29:32.844351+0000","last_clean":"2024-05-09T06:29:32.844351+0000","last_became_active":"2024-05-09T06:29:32.830760+0000","last_became_peered":"2024-05-09T06:29:32.830760+0000","last_unstale":"2024-05-09T06:29:32.844351+0000","last_undegraded":"2024-05-09T06:29:32.844351+0000","last_fullsized":"2024-05-09T06:29:32.844351+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828961+0000","last_change":"2024-05-09T06:29:32.828961+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828961+0000","last_undegraded":"2024-05-09T06:29:32.828961+0000","last_fullsized":"2024-05-09T06:29:32.828961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849673+0000","last_change":"2024-05-09T06:29:32.842910+0000","last_active":"2024-05-09T06:29:32.849673+0000","last_peered":"2024-05-09T06:29:32.849673+0000","last_clean":"2024-05-09T06:29:32.849673+0000","last_became_active":"2024-05-09T06:29:32.842534+0000","last_became_peered":"2024-05-09T06:29:32.842534+0000","last_unstale":"2024-05-09T06:29:32.849673+0000","last_undegraded":"2024-05-09T06:29:32.849673+0000","last_fullsized":"2024-05-09T06:29:32.849673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:33.233890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831887+0000","last_change":"2024-05-09T06:29:32.831887+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831887+0000","last_undegraded":"2024-05-09T06:29:32.831887+0000","last_fullsized":"2024-05-09T06:29:32.831887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844998+0000","last_change":"2024-05-09T06:29:32.840442+0000","last_active":"2024-05-09T06:29:32.844998+0000","last_peered":"2024-05-09T06:29:32.844998+0000","last_clean":"2024-05-09T06:29:32.844998+0000","last_became_active":"2024-05-09T06:29:32.840016+0000","last_became_peered":"2024-05-09T06:29:32.840016+0000","last_unstale":"2024-05-09T06:29:32.844998+0000","last_undegraded":"2024-05-09T06:29:32.844998+0000","last_fullsized":"2024-05-09T06:29:32.844998+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829388+0000","last_change":"2024-05-09T06:29:32.829388+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829388+0000","last_undegraded":"2024-05-09T06:29:32.829388+0000","last_fullsized":"2024-05-09T06:29:32.829388+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846231+0000","last_change":"2024-05-09T06:29:32.844658+0000","last_active":"2024-05-09T06:29:32.846231+0000","last_peered":"2024-05-09T06:29:32.846231+0000","last_clean":"2024-05-09T06:29:32.846231+0000","last_became_active":"2024-05-09T06:29:32.844008+0000","last_became_peered":"2024-05-09T06:29:32.844008+0000","last_unstale":"2024-05-09T06:29:32.846231+0000","last_undegraded":"2024-05-09T06:29:32.846231+0000","last_fullsized":"2024-05-09T06:29:32.846231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848806+0000","last_change":"2024-05-09T06:29:32.845642+0000","last_active":"2024-05-09T06:29:32.848806+0000","last_peered":"2024-05-09T06:29:32.848806+0000","last_clean":"2024-05-09T06:29:32.848806+0000","last_became_active":"2024-05-09T06:29:32.845324+0000","last_became_peered":"2024-05-09T06:29:32.845324+0000","last_unstale":"2024-05-09T06:29:32.848806+0000","last_undegraded":"2024-05-09T06:29:32.848806+0000","last_fullsized":"2024-05-09T06:29:32.848806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836258+0000","last_change":"2024-05-09T06:29:32.836258+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836258+0000","last_undegraded":"2024-05-09T06:29:32.836258+0000","last_fullsized":"2024-05-09T06:29:32.836258+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852598+0000","last_change":"2024-05-09T06:29:32.847975+0000","last_active":"2024-05-09T06:29:32.852598+0000","last_peered":"2024-05-09T06:29:32.852598+0000","last_clean":"2024-05-09T06:29:32.852598+0000","last_became_active":"2024-05-09T06:29:32.847772+0000","last_became_peered":"2024-05-09T06:29:32.847772+0000","last_unstale":"2024-05-09T06:29:32.852598+0000","last_undegraded":"2024-05-09T06:29:32.852598+0000","last_fullsized":"2024-05-09T06:29:32.852598+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831892+0000","last_change":"2024-05-09T06:29:32.831892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831892+0000","last_undegraded":"2024-05-09T06:29:32.831892+0000","last_fullsized":"2024-05-09T06:29:32.831892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845661+0000","last_change":"2024-05-09T06:29:32.843594+0000","last_active":"2024-05-09T06:29:32.845661+0000","last_peered":"2024-05-09T06:29:32.845661+0000","last_clean":"2024-05-09T06:29:32.845661+0000","last_became_active":"2024-05-09T06:29:32.843233+0000","last_became_peered":"2024-05-09T06:29:32.843233+0000","last_unstale":"2024-05-09T06:29:32.845661+0000","last_undegraded":"2024-05-09T06:29:32.845661+0000","last_fullsized":"2024-05-09T06:29:32.845661+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844934+0000","last_change":"2024-05-09T06:29:32.841265+0000","last_active":"2024-05-09T06:29:32.844934+0000","last_peered":"2024-05-09T06:29:32.844934+0000","last_clean":"2024-05-09T06:29:32.844934+0000","last_became_active":"2024-05-09T06:29:32.840989+0000","last_became_peered":"2024-05-09T06:29:32.840989+0000","last_unstale":"2024-05-09T06:29:32.844934+0000","last_undegraded":"2024-05-09T06:29:32.844934+0000","last_fullsized":"2024-05-09T06:29:32.844934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:10.424765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834060+0000","last_change":"2024-05-09T06:29:32.834060+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834060+0000","last_undegraded":"2024-05-09T06:29:32.834060+0000","last_fullsized":"2024-05-09T06:29:32.834060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846007+0000","last_change":"2024-05-09T06:29:32.842270+0000","last_active":"2024-05-09T06:29:32.846007+0000","last_peered":"2024-05-09T06:29:32.846007+0000","last_clean":"2024-05-09T06:29:32.846007+0000","last_became_active":"2024-05-09T06:29:32.841999+0000","last_became_peered":"2024-05-09T06:29:32.841999+0000","last_unstale":"2024-05-09T06:29:32.846007+0000","last_undegraded":"2024-05-09T06:29:32.846007+0000","last_fullsized":"2024-05-09T06:29:32.846007+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832434+0000","last_change":"2024-05-09T06:29:32.832434+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832434+0000","last_undegraded":"2024-05-09T06:29:32.832434+0000","last_fullsized":"2024-05-09T06:29:32.832434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852458+0000","last_change":"2024-05-09T06:29:32.846452+0000","last_active":"2024-05-09T06:29:32.852458+0000","last_peered":"2024-05-09T06:29:32.852458+0000","last_clean":"2024-05-09T06:29:32.852458+0000","last_became_active":"2024-05-09T06:29:32.846152+0000","last_became_peered":"2024-05-09T06:29:32.846152+0000","last_unstale":"2024-05-09T06:29:32.852458+0000","last_undegraded":"2024-05-09T06:29:32.852458+0000","last_fullsized":"2024-05-09T06:29:32.852458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:35.033218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834435+0000","last_change":"2024-05-09T06:29:32.834435+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834435+0000","last_undegraded":"2024-05-09T06:29:32.834435+0000","last_fullsized":"2024-05-09T06:29:32.834435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845010+0000","last_change":"2024-05-09T06:29:32.840887+0000","last_active":"2024-05-09T06:29:32.845010+0000","last_peered":"2024-05-09T06:29:32.845010+0000","last_clean":"2024-05-09T06:29:32.845010+0000","last_became_active":"2024-05-09T06:29:32.840544+0000","last_became_peered":"2024-05-09T06:29:32.840544+0000","last_unstale":"2024-05-09T06:29:32.845010+0000","last_undegraded":"2024-05-09T06:29:32.845010+0000","last_fullsized":"2024-05-09T06:29:32.845010+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832901+0000","last_change":"2024-05-09T06:29:32.832901+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832901+0000","last_undegraded":"2024-05-09T06:29:32.832901+0000","last_fullsized":"2024-05-09T06:29:32.832901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843689+0000","last_change":"2024-05-09T06:29:32.839558+0000","last_active":"2024-05-09T06:29:32.843689+0000","last_peered":"2024-05-09T06:29:32.843689+0000","last_clean":"2024-05-09T06:29:32.843689+0000","last_became_active":"2024-05-09T06:29:32.839220+0000","last_became_peered":"2024-05-09T06:29:32.839220+0000","last_unstale":"2024-05-09T06:29:32.843689+0000","last_undegraded":"2024-05-09T06:29:32.843689+0000","last_fullsized":"2024-05-09T06:29:32.843689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:03.213349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831995+0000","last_change":"2024-05-09T06:29:32.831995+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831995+0000","last_undegraded":"2024-05-09T06:29:32.831995+0000","last_fullsized":"2024-05-09T06:29:32.831995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848583+0000","last_change":"2024-05-09T06:29:32.847677+0000","last_active":"2024-05-09T06:29:32.848583+0000","last_peered":"2024-05-09T06:29:32.848583+0000","last_clean":"2024-05-09T06:29:32.848583+0000","last_became_active":"2024-05-09T06:29:32.847372+0000","last_became_peered":"2024-05-09T06:29:32.847372+0000","last_unstale":"2024-05-09T06:29:32.848583+0000","last_undegraded":"2024-05-09T06:29:32.848583+0000","last_fullsized":"2024-05-09T06:29:32.848583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825768+0000","last_change":"2024-05-09T06:29:32.825768+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825768+0000","last_undegraded":"2024-05-09T06:29:32.825768+0000","last_fullsized":"2024-05-09T06:29:32.825768+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843231+0000","last_change":"2024-05-09T06:29:32.842351+0000","last_active":"2024-05-09T06:29:32.843231+0000","last_peered":"2024-05-09T06:29:32.843231+0000","last_clean":"2024-05-09T06:29:32.843231+0000","last_became_active":"2024-05-09T06:29:32.842065+0000","last_became_peered":"2024-05-09T06:29:32.842065+0000","last_unstale":"2024-05-09T06:29:32.843231+0000","last_undegraded":"2024-05-09T06:29:32.843231+0000","last_fullsized":"2024-05-09T06:29:32.843231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:57.204005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831451+0000","last_change":"2024-05-09T06:29:32.831451+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831451+0000","last_undegraded":"2024-05-09T06:29:32.831451+0000","last_fullsized":"2024-05-09T06:29:32.831451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849727+0000","last_change":"2024-05-09T06:29:32.844205+0000","last_active":"2024-05-09T06:29:32.849727+0000","last_peered":"2024-05-09T06:29:32.849727+0000","last_clean":"2024-05-09T06:29:32.849727+0000","last_became_active":"2024-05-09T06:29:32.843719+0000","last_became_peered":"2024-05-09T06:29:32.843719+0000","last_unstale":"2024-05-09T06:29:32.849727+0000","last_undegraded":"2024-05-09T06:29:32.849727+0000","last_fullsized":"2024-05-09T06:29:32.849727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:22.182054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827283+0000","last_change":"2024-05-09T06:29:32.827283+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827283+0000","last_undegraded":"2024-05-09T06:29:32.827283+0000","last_fullsized":"2024-05-09T06:29:32.827283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852495+0000","last_change":"2024-05-09T06:29:32.847048+0000","last_active":"2024-05-09T06:29:32.852495+0000","last_peered":"2024-05-09T06:29:32.852495+0000","last_clean":"2024-05-09T06:29:32.852495+0000","last_became_active":"2024-05-09T06:29:32.846701+0000","last_became_peered":"2024-05-09T06:29:32.846701+0000","last_unstale":"2024-05-09T06:29:32.852495+0000","last_undegraded":"2024-05-09T06:29:32.852495+0000","last_fullsized":"2024-05-09T06:29:32.852495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833611+0000","last_change":"2024-05-09T06:29:32.833611+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833611+0000","last_undegraded":"2024-05-09T06:29:32.833611+0000","last_fullsized":"2024-05-09T06:29:32.833611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.842410+0000","last_change":"2024-05-09T06:29:32.840423+0000","last_active":"2024-05-09T06:29:32.842410+0000","last_peered":"2024-05-09T06:29:32.842410+0000","last_clean":"2024-05-09T06:29:32.842410+0000","last_became_active":"2024-05-09T06:29:32.840136+0000","last_became_peered":"2024-05-09T06:29:32.840136+0000","last_unstale":"2024-05-09T06:29:32.842410+0000","last_undegraded":"2024-05-09T06:29:32.842410+0000","last_fullsized":"2024-05-09T06:29:32.842410+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836148+0000","last_change":"2024-05-09T06:29:32.836148+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836148+0000","last_undegraded":"2024-05-09T06:29:32.836148+0000","last_fullsized":"2024-05-09T06:29:32.836148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843738+0000","last_change":"2024-05-09T06:29:32.839491+0000","last_active":"2024-05-09T06:29:32.843738+0000","last_peered":"2024-05-09T06:29:32.843738+0000","last_clean":"2024-05-09T06:29:32.843738+0000","last_became_active":"2024-05-09T06:29:32.839083+0000","last_became_peered":"2024-05-09T06:29:32.839083+0000","last_unstale":"2024-05-09T06:29:32.843738+0000","last_undegraded":"2024-05-09T06:29:32.843738+0000","last_fullsized":"2024-05-09T06:29:32.843738+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:43.479453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829862+0000","last_change":"2024-05-09T06:29:32.829862+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829862+0000","last_undegraded":"2024-05-09T06:29:32.829862+0000","last_fullsized":"2024-05-09T06:29:32.829862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845866+0000","last_change":"2024-05-09T06:29:32.840100+0000","last_active":"2024-05-09T06:29:32.845866+0000","last_peered":"2024-05-09T06:29:32.845866+0000","last_clean":"2024-05-09T06:29:32.845866+0000","last_became_active":"2024-05-09T06:29:32.839832+0000","last_became_peered":"2024-05-09T06:29:32.839832+0000","last_unstale":"2024-05-09T06:29:32.845866+0000","last_undegraded":"2024-05-09T06:29:32.845866+0000","last_fullsized":"2024-05-09T06:29:32.845866+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826694+0000","last_change":"2024-05-09T06:29:32.826694+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826694+0000","last_undegraded":"2024-05-09T06:29:32.826694+0000","last_fullsized":"2024-05-09T06:29:32.826694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.835648+0000","last_change":"2024-05-09T06:29:32.832884+0000","last_active":"2024-05-09T06:29:32.835648+0000","last_peered":"2024-05-09T06:29:32.835648+0000","last_clean":"2024-05-09T06:29:32.835648+0000","last_became_active":"2024-05-09T06:29:32.832612+0000","last_became_peered":"2024-05-09T06:29:32.832612+0000","last_unstale":"2024-05-09T06:29:32.835648+0000","last_undegraded":"2024-05-09T06:29:32.835648+0000","last_fullsized":"2024-05-09T06:29:32.835648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:50.231444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833045+0000","last_change":"2024-05-09T06:29:32.833045+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833045+0000","last_undegraded":"2024-05-09T06:29:32.833045+0000","last_fullsized":"2024-05-09T06:29:32.833045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852509+0000","last_change":"2024-05-09T06:29:32.846605+0000","last_active":"2024-05-09T06:29:32.852509+0000","last_peered":"2024-05-09T06:29:32.852509+0000","last_clean":"2024-05-09T06:29:32.852509+0000","last_became_active":"2024-05-09T06:29:32.846230+0000","last_became_peered":"2024-05-09T06:29:32.846230+0000","last_unstale":"2024-05-09T06:29:32.852509+0000","last_undegraded":"2024-05-09T06:29:32.852509+0000","last_fullsized":"2024-05-09T06:29:32.852509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825813+0000","last_change":"2024-05-09T06:29:32.825813+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825813+0000","last_undegraded":"2024-05-09T06:29:32.825813+0000","last_fullsized":"2024-05-09T06:29:32.825813+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845682+0000","last_change":"2024-05-09T06:29:32.844437+0000","last_active":"2024-05-09T06:29:32.845682+0000","last_peered":"2024-05-09T06:29:32.845682+0000","last_clean":"2024-05-09T06:29:32.845682+0000","last_became_active":"2024-05-09T06:29:32.843623+0000","last_became_peered":"2024-05-09T06:29:32.843623+0000","last_unstale":"2024-05-09T06:29:32.845682+0000","last_undegraded":"2024-05-09T06:29:32.845682+0000","last_fullsized":"2024-05-09T06:29:32.845682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836215+0000","last_change":"2024-05-09T06:29:32.836215+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836215+0000","last_undegraded":"2024-05-09T06:29:32.836215+0000","last_fullsized":"2024-05-09T06:29:32.836215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.847445+0000","last_change":"2024-05-09T06:29:32.845924+0000","last_active":"2024-05-09T06:29:32.847445+0000","last_peered":"2024-05-09T06:29:32.847445+0000","last_clean":"2024-05-09T06:29:32.847445+0000","last_became_active":"2024-05-09T06:29:32.845637+0000","last_became_peered":"2024-05-09T06:29:32.845637+0000","last_unstale":"2024-05-09T06:29:32.847445+0000","last_undegraded":"2024-05-09T06:29:32.847445+0000","last_fullsized":"2024-05-09T06:29:32.847445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830680+0000","last_change":"2024-05-09T06:29:32.830680+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830680+0000","last_undegraded":"2024-05-09T06:29:32.830680+0000","last_fullsized":"2024-05-09T06:29:32.830680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844884+0000","last_change":"2024-05-09T06:29:32.841755+0000","last_active":"2024-05-09T06:29:32.844884+0000","last_peered":"2024-05-09T06:29:32.844884+0000","last_clean":"2024-05-09T06:29:32.844884+0000","last_became_active":"2024-05-09T06:29:32.840696+0000","last_became_peered":"2024-05-09T06:29:32.840696+0000","last_unstale":"2024-05-09T06:29:32.844884+0000","last_undegraded":"2024-05-09T06:29:32.844884+0000","last_fullsized":"2024-05-09T06:29:32.844884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:46.375838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832731+0000","last_change":"2024-05-09T06:29:32.832731+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832731+0000","last_undegraded":"2024-05-09T06:29:32.832731+0000","last_fullsized":"2024-05-09T06:29:32.832731+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845304+0000","last_change":"2024-05-09T06:29:32.841567+0000","last_active":"2024-05-09T06:29:32.845304+0000","last_peered":"2024-05-09T06:29:32.845304+0000","last_clean":"2024-05-09T06:29:32.845304+0000","last_became_active":"2024-05-09T06:29:32.841246+0000","last_became_peered":"2024-05-09T06:29:32.841246+0000","last_unstale":"2024-05-09T06:29:32.845304+0000","last_undegraded":"2024-05-09T06:29:32.845304+0000","last_fullsized":"2024-05-09T06:29:32.845304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:21.190037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831387+0000","last_change":"2024-05-09T06:29:32.831387+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831387+0000","last_undegraded":"2024-05-09T06:29:32.831387+0000","last_fullsized":"2024-05-09T06:29:32.831387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849643+0000","last_change":"2024-05-09T06:29:32.843603+0000","last_active":"2024-05-09T06:29:32.849643+0000","last_peered":"2024-05-09T06:29:32.849643+0000","last_clean":"2024-05-09T06:29:32.849643+0000","last_became_active":"2024-05-09T06:29:32.843314+0000","last_became_peered":"2024-05-09T06:29:32.843314+0000","last_unstale":"2024-05-09T06:29:32.849643+0000","last_undegraded":"2024-05-09T06:29:32.849643+0000","last_fullsized":"2024-05-09T06:29:32.849643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833783+0000","last_change":"2024-05-09T06:29:32.833783+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833783+0000","last_undegraded":"2024-05-09T06:29:32.833783+0000","last_fullsized":"2024-05-09T06:29:32.833783+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849604+0000","last_change":"2024-05-09T06:29:32.843469+0000","last_active":"2024-05-09T06:29:32.849604+0000","last_peered":"2024-05-09T06:29:32.849604+0000","last_clean":"2024-05-09T06:29:32.849604+0000","last_became_active":"2024-05-09T06:29:32.842907+0000","last_became_peered":"2024-05-09T06:29:32.842907+0000","last_unstale":"2024-05-09T06:29:32.849604+0000","last_undegraded":"2024-05-09T06:29:32.849604+0000","last_fullsized":"2024-05-09T06:29:32.849604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835859+0000","last_change":"2024-05-09T06:29:32.835859+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835859+0000","last_undegraded":"2024-05-09T06:29:32.835859+0000","last_fullsized":"2024-05-09T06:29:32.835859+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845941+0000","last_change":"2024-05-09T06:29:32.843492+0000","last_active":"2024-05-09T06:29:32.845941+0000","last_peered":"2024-05-09T06:29:32.845941+0000","last_clean":"2024-05-09T06:29:32.845941+0000","last_became_active":"2024-05-09T06:29:32.842665+0000","last_became_peered":"2024-05-09T06:29:32.842665+0000","last_unstale":"2024-05-09T06:29:32.845941+0000","last_undegraded":"2024-05-09T06:29:32.845941+0000","last_fullsized":"2024-05-09T06:29:32.845941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.830855+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:29:32.830855+0000","last_peered":"2024-05-09T06:29:32.830855+0000","last_clean":"2024-05-09T06:29:32.830855+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:29:32.830855+0000","last_undegraded":"2024-05-09T06:29:32.830855+0000","last_fullsized":"2024-05-09T06:29:32.830855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:29:37.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.022+0000 7f7ab3e31640 1 Processor -- start 2024-05-09T06:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.022+0000 7f7ab3e31640 1 -- start start 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab3e31640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7aac163c70 0x7f7aac166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab3e31640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aac0ce0e0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab3e31640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aac0ce3c0 con 0x7f7aac0cf180 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab3e31640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7aac0ce6a0 con 0x7f7aac163c70 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7aac163c70 0x7f7aac166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7aac163c70 0x7f7aac166060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40170/0 (socket says 172.21.15.1:40170) 2024-05-09T06:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 -- 172.21.15.1:0/4135240541 learned_addr learned my addr 172.21.15.1:0/4135240541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab1bcd640 1 --1- 172.21.15.1:0/4135240541 >> v1:172.21.15.1:6789/0 conn(0x7f7aac06e6d0 0x7f7aac06eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38902/0 (socket says 172.21.15.1:38902) 2024-05-09T06:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 -- 172.21.15.1:0/4135240541 >> v1:172.21.15.6:6789/0 conn(0x7f7aac0cf180 legacy=0x7f7aac0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 -- 172.21.15.1:0/4135240541 >> v1:172.21.15.1:6789/0 conn(0x7f7aac06e6d0 legacy=0x7f7aac06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.023+0000 7f7ab23ce640 1 -- 172.21.15.1:0/4135240541 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0ce980 con 0x7f7aac163c70 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.024+0000 7f7ab23ce640 1 --2- 172.21.15.1:0/4135240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac163c70 0x7f7aac166060 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009e50 tx=0x7f7aa402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c12337aed9d3e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.024+0000 7f7ab0bcb640 1 -- 172.21.15.1:0/4135240541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4036070 con 0x7f7aac163c70 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.024+0000 7f7ab0bcb640 1 -- 172.21.15.1:0/4135240541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4031600 con 0x7f7aac163c70 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.024+0000 7f7ab0bcb640 1 -- 172.21.15.1:0/4135240541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40319f0 con 0x7f7aac163c70 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 -- 172.21.15.1:0/4135240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac163c70 msgr2=0x7f7aac166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/4135240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac163c70 0x7f7aac166060 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009e50 tx=0x7f7aa402a7c0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 -- 172.21.15.1:0/4135240541 shutdown_connections 2024-05-09T06:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/4135240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac163c70 0x7f7aac166060 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 -- 172.21.15.1:0/4135240541 >> 172.21.15.1:0/4135240541 conn(0x7f7aac0cc4b0 msgr2=0x7f7aac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 -- 172.21.15.1:0/4135240541 shutdown_connections 2024-05-09T06:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.025+0000 7f7ab3e31640 1 -- 172.21.15.1:0/4135240541 wait complete. 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.026+0000 7f7ab3e31640 1 Processor -- start 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.026+0000 7f7ab3e31640 1 -- start start 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac0cf180 0x7f7aac0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac163c70 0x7f7aac13fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1447c0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac070ed0 con 0x7f7aac163c70 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac144940 con 0x7f7aac0cf180 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40178/0 (socket says 172.21.15.1:40178) 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 -- 172.21.15.1:0/2152183810 learned_addr learned my addr 172.21.15.1:0/2152183810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab23ce640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac163c70 0x7f7aac13fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab13cc640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac0cf180 0x7f7aac0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 -- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac0cf180 msgr2=0x7f7aac0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac0cf180 0x7f7aac0b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 -- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac163c70 msgr2=0x7f7aac13fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac163c70 0x7f7aac13fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 -- 172.21.15.1:0/2152183810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0b1740 con 0x7f7aac06e6d0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab1bcd640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c3e0 tx=0x7f7aa800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80074e0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa8007680 con 0x7f7aac06e6d0 2024-05-09T06:29:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac0a9520 con 0x7f7aac06e6d0 2024-05-09T06:29:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80049e0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.027+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac0a9ae0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.028+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac0752b0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.029+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7aa801a020 con 0x7f7aac06e6d0 2024-05-09T06:29:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.030+0000 7f7aa27fc640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 0x7f7a9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.030+0000 7f7ab13cc640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 0x7f7a9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.030+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7aa808a7e0 con 0x7f7aac06e6d0 2024-05-09T06:29:38.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.030+0000 7f7ab13cc640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 0x7f7a9006e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7aac141ce0 tx=0x7f7a9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.034+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7aa8059000 con 0x7f7aac06e6d0 2024-05-09T06:29:38.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.216+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7aac06cf50 con 0x7f7a9006be30 2024-05-09T06:29:38.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.221+0000 7f7aa27fc640 1 -- 172.21.15.1:0/2152183810 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7f7aac06cf50 con 0x7f7a9006be30 2024-05-09T06:29:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 msgr2=0x7f7a9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 0x7f7a9006e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7aac141ce0 tx=0x7f7a9c009040 comp rx=0 tx=0).stop 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 msgr2=0x7f7aac159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c3e0 tx=0x7f7aa800c8a0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 shutdown_connections 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a9006be30 0x7f7a9006e2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac163c70 0x7f7aac13fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac0cf180 0x7f7aac0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 --2- 172.21.15.1:0/2152183810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7aac06e6d0 0x7f7aac159bd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 >> 172.21.15.1:0/2152183810 conn(0x7f7aac0cc4b0 msgr2=0x7f7aac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 shutdown_connections 2024-05-09T06:29:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.224+0000 7f7ab3e31640 1 -- 172.21.15.1:0/2152183810 wait complete. 2024-05-09T06:29:38.221 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-05-09T06:29:37.096005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238428,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736292,"statfs":{"total":773094113280,"available":772849963008,"internally_reserved":0,"allocated":5742592,"data_stored":3179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.274802"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827195+0000","last_change":"2024-05-09T06:29:32.827195+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827195+0000","last_undegraded":"2024-05-09T06:29:32.827195+0000","last_fullsized":"2024-05-09T06:29:32.827195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830494+0000","last_change":"2024-05-09T06:29:32.830494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830494+0000","last_undegraded":"2024-05-09T06:29:32.830494+0000","last_fullsized":"2024-05-09T06:29:32.830494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840400+0000","last_change":"2024-05-09T06:29:32.840400+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840400+0000","last_undegraded":"2024-05-09T06:29:32.840400+0000","last_fullsized":"2024-05-09T06:29:32.840400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828893+0000","last_change":"2024-05-09T06:29:32.828893+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828893+0000","last_undegraded":"2024-05-09T06:29:32.828893+0000","last_fullsized":"2024-05-09T06:29:32.828893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832054+0000","last_change":"2024-05-09T06:29:32.832054+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832054+0000","last_undegraded":"2024-05-09T06:29:32.832054+0000","last_fullsized":"2024-05-09T06:29:32.832054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826494+0000","last_change":"2024-05-09T06:29:32.826494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826494+0000","last_undegraded":"2024-05-09T06:29:32.826494+0000","last_fullsized":"2024-05-09T06:29:32.826494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840466+0000","last_change":"2024-05-09T06:29:32.840466+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840466+0000","last_undegraded":"2024-05-09T06:29:32.840466+0000","last_fullsized":"2024-05-09T06:29:32.840466+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834488+0000","last_change":"2024-05-09T06:29:32.834488+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834488+0000","last_undegraded":"2024-05-09T06:29:32.834488+0000","last_fullsized":"2024-05-09T06:29:32.834488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835155+0000","last_change":"2024-05-09T06:29:32.835155+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835155+0000","last_undegraded":"2024-05-09T06:29:32.835155+0000","last_fullsized":"2024-05-09T06:29:32.835155+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827341+0000","last_change":"2024-05-09T06:29:32.827341+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827341+0000","last_undegraded":"2024-05-09T06:29:32.827341+0000","last_fullsized":"2024-05-09T06:29:32.827341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828942+0000","last_change":"2024-05-09T06:29:32.828942+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828942+0000","last_undegraded":"2024-05-09T06:29:32.828942+0000","last_fullsized":"2024-05-09T06:29:32.828942+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832892+0000","last_change":"2024-05-09T06:29:32.832892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832892+0000","last_undegraded":"2024-05-09T06:29:32.832892+0000","last_fullsized":"2024-05-09T06:29:32.832892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827743+0000","last_change":"2024-05-09T06:29:32.827743+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827743+0000","last_undegraded":"2024-05-09T06:29:32.827743+0000","last_fullsized":"2024-05-09T06:29:32.827743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836723+0000","last_change":"2024-05-09T06:29:32.836723+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836723+0000","last_undegraded":"2024-05-09T06:29:32.836723+0000","last_fullsized":"2024-05-09T06:29:32.836723+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828834+0000","last_change":"2024-05-09T06:29:32.828834+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828834+0000","last_undegraded":"2024-05-09T06:29:32.828834+0000","last_fullsized":"2024-05-09T06:29:32.828834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826180+0000","last_change":"2024-05-09T06:29:32.826180+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826180+0000","last_undegraded":"2024-05-09T06:29:32.826180+0000","last_fullsized":"2024-05-09T06:29:32.826180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836265+0000","last_change":"2024-05-09T06:29:32.836265+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836265+0000","last_undegraded":"2024-05-09T06:29:32.836265+0000","last_fullsized":"2024-05-09T06:29:32.836265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830004+0000","last_change":"2024-05-09T06:29:32.830004+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830004+0000","last_undegraded":"2024-05-09T06:29:32.830004+0000","last_fullsized":"2024-05-09T06:29:32.830004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835867+0000","last_change":"2024-05-09T06:29:32.835867+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835867+0000","last_undegraded":"2024-05-09T06:29:32.835867+0000","last_fullsized":"2024-05-09T06:29:32.835867+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.837023+0000","last_change":"2024-05-09T06:29:32.837023+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.837023+0000","last_undegraded":"2024-05-09T06:29:32.837023+0000","last_fullsized":"2024-05-09T06:29:32.837023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836127+0000","last_change":"2024-05-09T06:29:32.836127+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836127+0000","last_undegraded":"2024-05-09T06:29:32.836127+0000","last_fullsized":"2024-05-09T06:29:32.836127+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832922+0000","last_change":"2024-05-09T06:29:32.832922+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832922+0000","last_undegraded":"2024-05-09T06:29:32.832922+0000","last_fullsized":"2024-05-09T06:29:32.832922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828814+0000","last_change":"2024-05-09T06:29:32.828814+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828814+0000","last_undegraded":"2024-05-09T06:29:32.828814+0000","last_fullsized":"2024-05-09T06:29:32.828814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835263+0000","last_change":"2024-05-09T06:29:32.835263+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835263+0000","last_undegraded":"2024-05-09T06:29:32.835263+0000","last_fullsized":"2024-05-09T06:29:32.835263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834730+0000","last_change":"2024-05-09T06:29:32.834730+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834730+0000","last_undegraded":"2024-05-09T06:29:32.834730+0000","last_fullsized":"2024-05-09T06:29:32.834730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834843+0000","last_change":"2024-05-09T06:29:32.834843+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834843+0000","last_undegraded":"2024-05-09T06:29:32.834843+0000","last_fullsized":"2024-05-09T06:29:32.834843+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826703+0000","last_change":"2024-05-09T06:29:32.826703+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826703+0000","last_undegraded":"2024-05-09T06:29:32.826703+0000","last_fullsized":"2024-05-09T06:29:32.826703+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836464+0000","last_change":"2024-05-09T06:29:32.836464+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836464+0000","last_undegraded":"2024-05-09T06:29:32.836464+0000","last_fullsized":"2024-05-09T06:29:32.836464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834001+0000","last_change":"2024-05-09T06:29:32.834001+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834001+0000","last_undegraded":"2024-05-09T06:29:32.834001+0000","last_fullsized":"2024-05-09T06:29:32.834001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829477+0000","last_change":"2024-05-09T06:29:32.829477+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829477+0000","last_undegraded":"2024-05-09T06:29:32.829477+0000","last_fullsized":"2024-05-09T06:29:32.829477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833492+0000","last_change":"2024-05-09T06:29:32.833492+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833492+0000","last_undegraded":"2024-05-09T06:29:32.833492+0000","last_fullsized":"2024-05-09T06:29:32.833492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833542+0000","last_change":"2024-05-09T06:29:32.833542+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833542+0000","last_undegraded":"2024-05-09T06:29:32.833542+0000","last_fullsized":"2024-05-09T06:29:32.833542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828830+0000","last_change":"2024-05-09T06:29:32.828830+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828830+0000","last_undegraded":"2024-05-09T06:29:32.828830+0000","last_fullsized":"2024-05-09T06:29:32.828830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849652+0000","last_change":"2024-05-09T06:29:32.843136+0000","last_active":"2024-05-09T06:29:32.849652+0000","last_peered":"2024-05-09T06:29:32.849652+0000","last_clean":"2024-05-09T06:29:32.849652+0000","last_became_active":"2024-05-09T06:29:32.842842+0000","last_became_peered":"2024-05-09T06:29:32.842842+0000","last_unstale":"2024-05-09T06:29:32.849652+0000","last_undegraded":"2024-05-09T06:29:32.849652+0000","last_fullsized":"2024-05-09T06:29:32.849652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835845+0000","last_change":"2024-05-09T06:29:32.835845+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835845+0000","last_undegraded":"2024-05-09T06:29:32.835845+0000","last_fullsized":"2024-05-09T06:29:32.835845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843214+0000","last_change":"2024-05-09T06:29:32.842035+0000","last_active":"2024-05-09T06:29:32.843214+0000","last_peered":"2024-05-09T06:29:32.843214+0000","last_clean":"2024-05-09T06:29:32.843214+0000","last_became_active":"2024-05-09T06:29:32.841756+0000","last_became_peered":"2024-05-09T06:29:32.841756+0000","last_unstale":"2024-05-09T06:29:32.843214+0000","last_undegraded":"2024-05-09T06:29:32.843214+0000","last_fullsized":"2024-05-09T06:29:32.843214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836568+0000","last_change":"2024-05-09T06:29:32.836568+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836568+0000","last_undegraded":"2024-05-09T06:29:32.836568+0000","last_fullsized":"2024-05-09T06:29:32.836568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844302+0000","last_change":"2024-05-09T06:29:32.842788+0000","last_active":"2024-05-09T06:29:32.844302+0000","last_peered":"2024-05-09T06:29:32.844302+0000","last_clean":"2024-05-09T06:29:32.844302+0000","last_became_active":"2024-05-09T06:29:32.842461+0000","last_became_peered":"2024-05-09T06:29:32.842461+0000","last_unstale":"2024-05-09T06:29:32.844302+0000","last_undegraded":"2024-05-09T06:29:32.844302+0000","last_fullsized":"2024-05-09T06:29:32.844302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836686+0000","last_change":"2024-05-09T06:29:32.836686+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836686+0000","last_undegraded":"2024-05-09T06:29:32.836686+0000","last_fullsized":"2024-05-09T06:29:32.836686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852441+0000","last_change":"2024-05-09T06:29:32.845755+0000","last_active":"2024-05-09T06:29:32.852441+0000","last_peered":"2024-05-09T06:29:32.852441+0000","last_clean":"2024-05-09T06:29:32.852441+0000","last_became_active":"2024-05-09T06:29:32.845443+0000","last_became_peered":"2024-05-09T06:29:32.845443+0000","last_unstale":"2024-05-09T06:29:32.852441+0000","last_undegraded":"2024-05-09T06:29:32.852441+0000","last_fullsized":"2024-05-09T06:29:32.852441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:37.820825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833767+0000","last_change":"2024-05-09T06:29:32.833767+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833767+0000","last_undegraded":"2024-05-09T06:29:32.833767+0000","last_fullsized":"2024-05-09T06:29:32.833767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844351+0000","last_change":"2024-05-09T06:29:32.831050+0000","last_active":"2024-05-09T06:29:32.844351+0000","last_peered":"2024-05-09T06:29:32.844351+0000","last_clean":"2024-05-09T06:29:32.844351+0000","last_became_active":"2024-05-09T06:29:32.830760+0000","last_became_peered":"2024-05-09T06:29:32.830760+0000","last_unstale":"2024-05-09T06:29:32.844351+0000","last_undegraded":"2024-05-09T06:29:32.844351+0000","last_fullsized":"2024-05-09T06:29:32.844351+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828961+0000","last_change":"2024-05-09T06:29:32.828961+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828961+0000","last_undegraded":"2024-05-09T06:29:32.828961+0000","last_fullsized":"2024-05-09T06:29:32.828961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849673+0000","last_change":"2024-05-09T06:29:32.842910+0000","last_active":"2024-05-09T06:29:32.849673+0000","last_peered":"2024-05-09T06:29:32.849673+0000","last_clean":"2024-05-09T06:29:32.849673+0000","last_became_active":"2024-05-09T06:29:32.842534+0000","last_became_peered":"2024-05-09T06:29:32.842534+0000","last_unstale":"2024-05-09T06:29:32.849673+0000","last_undegraded":"2024-05-09T06:29:32.849673+0000","last_fullsized":"2024-05-09T06:29:32.849673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:33.233890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831887+0000","last_change":"2024-05-09T06:29:32.831887+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831887+0000","last_undegraded":"2024-05-09T06:29:32.831887+0000","last_fullsized":"2024-05-09T06:29:32.831887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844998+0000","last_change":"2024-05-09T06:29:32.840442+0000","last_active":"2024-05-09T06:29:32.844998+0000","last_peered":"2024-05-09T06:29:32.844998+0000","last_clean":"2024-05-09T06:29:32.844998+0000","last_became_active":"2024-05-09T06:29:32.840016+0000","last_became_peered":"2024-05-09T06:29:32.840016+0000","last_unstale":"2024-05-09T06:29:32.844998+0000","last_undegraded":"2024-05-09T06:29:32.844998+0000","last_fullsized":"2024-05-09T06:29:32.844998+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829388+0000","last_change":"2024-05-09T06:29:32.829388+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829388+0000","last_undegraded":"2024-05-09T06:29:32.829388+0000","last_fullsized":"2024-05-09T06:29:32.829388+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846231+0000","last_change":"2024-05-09T06:29:32.844658+0000","last_active":"2024-05-09T06:29:32.846231+0000","last_peered":"2024-05-09T06:29:32.846231+0000","last_clean":"2024-05-09T06:29:32.846231+0000","last_became_active":"2024-05-09T06:29:32.844008+0000","last_became_peered":"2024-05-09T06:29:32.844008+0000","last_unstale":"2024-05-09T06:29:32.846231+0000","last_undegraded":"2024-05-09T06:29:32.846231+0000","last_fullsized":"2024-05-09T06:29:32.846231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848806+0000","last_change":"2024-05-09T06:29:32.845642+0000","last_active":"2024-05-09T06:29:32.848806+0000","last_peered":"2024-05-09T06:29:32.848806+0000","last_clean":"2024-05-09T06:29:32.848806+0000","last_became_active":"2024-05-09T06:29:32.845324+0000","last_became_peered":"2024-05-09T06:29:32.845324+0000","last_unstale":"2024-05-09T06:29:32.848806+0000","last_undegraded":"2024-05-09T06:29:32.848806+0000","last_fullsized":"2024-05-09T06:29:32.848806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836258+0000","last_change":"2024-05-09T06:29:32.836258+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836258+0000","last_undegraded":"2024-05-09T06:29:32.836258+0000","last_fullsized":"2024-05-09T06:29:32.836258+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852598+0000","last_change":"2024-05-09T06:29:32.847975+0000","last_active":"2024-05-09T06:29:32.852598+0000","last_peered":"2024-05-09T06:29:32.852598+0000","last_clean":"2024-05-09T06:29:32.852598+0000","last_became_active":"2024-05-09T06:29:32.847772+0000","last_became_peered":"2024-05-09T06:29:32.847772+0000","last_unstale":"2024-05-09T06:29:32.852598+0000","last_undegraded":"2024-05-09T06:29:32.852598+0000","last_fullsized":"2024-05-09T06:29:32.852598+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831892+0000","last_change":"2024-05-09T06:29:32.831892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831892+0000","last_undegraded":"2024-05-09T06:29:32.831892+0000","last_fullsized":"2024-05-09T06:29:32.831892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845661+0000","last_change":"2024-05-09T06:29:32.843594+0000","last_active":"2024-05-09T06:29:32.845661+0000","last_peered":"2024-05-09T06:29:32.845661+0000","last_clean":"2024-05-09T06:29:32.845661+0000","last_became_active":"2024-05-09T06:29:32.843233+0000","last_became_peered":"2024-05-09T06:29:32.843233+0000","last_unstale":"2024-05-09T06:29:32.845661+0000","last_undegraded":"2024-05-09T06:29:32.845661+0000","last_fullsized":"2024-05-09T06:29:32.845661+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844934+0000","last_change":"2024-05-09T06:29:32.841265+0000","last_active":"2024-05-09T06:29:32.844934+0000","last_peered":"2024-05-09T06:29:32.844934+0000","last_clean":"2024-05-09T06:29:32.844934+0000","last_became_active":"2024-05-09T06:29:32.840989+0000","last_became_peered":"2024-05-09T06:29:32.840989+0000","last_unstale":"2024-05-09T06:29:32.844934+0000","last_undegraded":"2024-05-09T06:29:32.844934+0000","last_fullsized":"2024-05-09T06:29:32.844934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:10.424765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834060+0000","last_change":"2024-05-09T06:29:32.834060+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834060+0000","last_undegraded":"2024-05-09T06:29:32.834060+0000","last_fullsized":"2024-05-09T06:29:32.834060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846007+0000","last_change":"2024-05-09T06:29:32.842270+0000","last_active":"2024-05-09T06:29:32.846007+0000","last_peered":"2024-05-09T06:29:32.846007+0000","last_clean":"2024-05-09T06:29:32.846007+0000","last_became_active":"2024-05-09T06:29:32.841999+0000","last_became_peered":"2024-05-09T06:29:32.841999+0000","last_unstale":"2024-05-09T06:29:32.846007+0000","last_undegraded":"2024-05-09T06:29:32.846007+0000","last_fullsized":"2024-05-09T06:29:32.846007+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832434+0000","last_change":"2024-05-09T06:29:32.832434+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832434+0000","last_undegraded":"2024-05-09T06:29:32.832434+0000","last_fullsized":"2024-05-09T06:29:32.832434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852458+0000","last_change":"2024-05-09T06:29:32.846452+0000","last_active":"2024-05-09T06:29:32.852458+0000","last_peered":"2024-05-09T06:29:32.852458+0000","last_clean":"2024-05-09T06:29:32.852458+0000","last_became_active":"2024-05-09T06:29:32.846152+0000","last_became_peered":"2024-05-09T06:29:32.846152+0000","last_unstale":"2024-05-09T06:29:32.852458+0000","last_undegraded":"2024-05-09T06:29:32.852458+0000","last_fullsized":"2024-05-09T06:29:32.852458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:35.033218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834435+0000","last_change":"2024-05-09T06:29:32.834435+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834435+0000","last_undegraded":"2024-05-09T06:29:32.834435+0000","last_fullsized":"2024-05-09T06:29:32.834435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845010+0000","last_change":"2024-05-09T06:29:32.840887+0000","last_active":"2024-05-09T06:29:32.845010+0000","last_peered":"2024-05-09T06:29:32.845010+0000","last_clean":"2024-05-09T06:29:32.845010+0000","last_became_active":"2024-05-09T06:29:32.840544+0000","last_became_peered":"2024-05-09T06:29:32.840544+0000","last_unstale":"2024-05-09T06:29:32.845010+0000","last_undegraded":"2024-05-09T06:29:32.845010+0000","last_fullsized":"2024-05-09T06:29:32.845010+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832901+0000","last_change":"2024-05-09T06:29:32.832901+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832901+0000","last_undegraded":"2024-05-09T06:29:32.832901+0000","last_fullsized":"2024-05-09T06:29:32.832901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843689+0000","last_change":"2024-05-09T06:29:32.839558+0000","last_active":"2024-05-09T06:29:32.843689+0000","last_peered":"2024-05-09T06:29:32.843689+0000","last_clean":"2024-05-09T06:29:32.843689+0000","last_became_active":"2024-05-09T06:29:32.839220+0000","last_became_peered":"2024-05-09T06:29:32.839220+0000","last_unstale":"2024-05-09T06:29:32.843689+0000","last_undegraded":"2024-05-09T06:29:32.843689+0000","last_fullsized":"2024-05-09T06:29:32.843689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:03.213349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831995+0000","last_change":"2024-05-09T06:29:32.831995+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831995+0000","last_undegraded":"2024-05-09T06:29:32.831995+0000","last_fullsized":"2024-05-09T06:29:32.831995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848583+0000","last_change":"2024-05-09T06:29:32.847677+0000","last_active":"2024-05-09T06:29:32.848583+0000","last_peered":"2024-05-09T06:29:32.848583+0000","last_clean":"2024-05-09T06:29:32.848583+0000","last_became_active":"2024-05-09T06:29:32.847372+0000","last_became_peered":"2024-05-09T06:29:32.847372+0000","last_unstale":"2024-05-09T06:29:32.848583+0000","last_undegraded":"2024-05-09T06:29:32.848583+0000","last_fullsized":"2024-05-09T06:29:32.848583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825768+0000","last_change":"2024-05-09T06:29:32.825768+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825768+0000","last_undegraded":"2024-05-09T06:29:32.825768+0000","last_fullsized":"2024-05-09T06:29:32.825768+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843231+0000","last_change":"2024-05-09T06:29:32.842351+0000","last_active":"2024-05-09T06:29:32.843231+0000","last_peered":"2024-05-09T06:29:32.843231+0000","last_clean":"2024-05-09T06:29:32.843231+0000","last_became_active":"2024-05-09T06:29:32.842065+0000","last_became_peered":"2024-05-09T06:29:32.842065+0000","last_unstale":"2024-05-09T06:29:32.843231+0000","last_undegraded":"2024-05-09T06:29:32.843231+0000","last_fullsized":"2024-05-09T06:29:32.843231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:57.204005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831451+0000","last_change":"2024-05-09T06:29:32.831451+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831451+0000","last_undegraded":"2024-05-09T06:29:32.831451+0000","last_fullsized":"2024-05-09T06:29:32.831451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849727+0000","last_change":"2024-05-09T06:29:32.844205+0000","last_active":"2024-05-09T06:29:32.849727+0000","last_peered":"2024-05-09T06:29:32.849727+0000","last_clean":"2024-05-09T06:29:32.849727+0000","last_became_active":"2024-05-09T06:29:32.843719+0000","last_became_peered":"2024-05-09T06:29:32.843719+0000","last_unstale":"2024-05-09T06:29:32.849727+0000","last_undegraded":"2024-05-09T06:29:32.849727+0000","last_fullsized":"2024-05-09T06:29:32.849727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:22.182054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827283+0000","last_change":"2024-05-09T06:29:32.827283+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827283+0000","last_undegraded":"2024-05-09T06:29:32.827283+0000","last_fullsized":"2024-05-09T06:29:32.827283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852495+0000","last_change":"2024-05-09T06:29:32.847048+0000","last_active":"2024-05-09T06:29:32.852495+0000","last_peered":"2024-05-09T06:29:32.852495+0000","last_clean":"2024-05-09T06:29:32.852495+0000","last_became_active":"2024-05-09T06:29:32.846701+0000","last_became_peered":"2024-05-09T06:29:32.846701+0000","last_unstale":"2024-05-09T06:29:32.852495+0000","last_undegraded":"2024-05-09T06:29:32.852495+0000","last_fullsized":"2024-05-09T06:29:32.852495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833611+0000","last_change":"2024-05-09T06:29:32.833611+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833611+0000","last_undegraded":"2024-05-09T06:29:32.833611+0000","last_fullsized":"2024-05-09T06:29:32.833611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.842410+0000","last_change":"2024-05-09T06:29:32.840423+0000","last_active":"2024-05-09T06:29:32.842410+0000","last_peered":"2024-05-09T06:29:32.842410+0000","last_clean":"2024-05-09T06:29:32.842410+0000","last_became_active":"2024-05-09T06:29:32.840136+0000","last_became_peered":"2024-05-09T06:29:32.840136+0000","last_unstale":"2024-05-09T06:29:32.842410+0000","last_undegraded":"2024-05-09T06:29:32.842410+0000","last_fullsized":"2024-05-09T06:29:32.842410+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836148+0000","last_change":"2024-05-09T06:29:32.836148+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836148+0000","last_undegraded":"2024-05-09T06:29:32.836148+0000","last_fullsized":"2024-05-09T06:29:32.836148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843738+0000","last_change":"2024-05-09T06:29:32.839491+0000","last_active":"2024-05-09T06:29:32.843738+0000","last_peered":"2024-05-09T06:29:32.843738+0000","last_clean":"2024-05-09T06:29:32.843738+0000","last_became_active":"2024-05-09T06:29:32.839083+0000","last_became_peered":"2024-05-09T06:29:32.839083+0000","last_unstale":"2024-05-09T06:29:32.843738+0000","last_undegraded":"2024-05-09T06:29:32.843738+0000","last_fullsized":"2024-05-09T06:29:32.843738+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:43.479453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829862+0000","last_change":"2024-05-09T06:29:32.829862+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829862+0000","last_undegraded":"2024-05-09T06:29:32.829862+0000","last_fullsized":"2024-05-09T06:29:32.829862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845866+0000","last_change":"2024-05-09T06:29:32.840100+0000","last_active":"2024-05-09T06:29:32.845866+0000","last_peered":"2024-05-09T06:29:32.845866+0000","last_clean":"2024-05-09T06:29:32.845866+0000","last_became_active":"2024-05-09T06:29:32.839832+0000","last_became_peered":"2024-05-09T06:29:32.839832+0000","last_unstale":"2024-05-09T06:29:32.845866+0000","last_undegraded":"2024-05-09T06:29:32.845866+0000","last_fullsized":"2024-05-09T06:29:32.845866+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826694+0000","last_change":"2024-05-09T06:29:32.826694+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826694+0000","last_undegraded":"2024-05-09T06:29:32.826694+0000","last_fullsized":"2024-05-09T06:29:32.826694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.835648+0000","last_change":"2024-05-09T06:29:32.832884+0000","last_active":"2024-05-09T06:29:32.835648+0000","last_peered":"2024-05-09T06:29:32.835648+0000","last_clean":"2024-05-09T06:29:32.835648+0000","last_became_active":"2024-05-09T06:29:32.832612+0000","last_became_peered":"2024-05-09T06:29:32.832612+0000","last_unstale":"2024-05-09T06:29:32.835648+0000","last_undegraded":"2024-05-09T06:29:32.835648+0000","last_fullsized":"2024-05-09T06:29:32.835648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:50.231444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833045+0000","last_change":"2024-05-09T06:29:32.833045+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833045+0000","last_undegraded":"2024-05-09T06:29:32.833045+0000","last_fullsized":"2024-05-09T06:29:32.833045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852509+0000","last_change":"2024-05-09T06:29:32.846605+0000","last_active":"2024-05-09T06:29:32.852509+0000","last_peered":"2024-05-09T06:29:32.852509+0000","last_clean":"2024-05-09T06:29:32.852509+0000","last_became_active":"2024-05-09T06:29:32.846230+0000","last_became_peered":"2024-05-09T06:29:32.846230+0000","last_unstale":"2024-05-09T06:29:32.852509+0000","last_undegraded":"2024-05-09T06:29:32.852509+0000","last_fullsized":"2024-05-09T06:29:32.852509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825813+0000","last_change":"2024-05-09T06:29:32.825813+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825813+0000","last_undegraded":"2024-05-09T06:29:32.825813+0000","last_fullsized":"2024-05-09T06:29:32.825813+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845682+0000","last_change":"2024-05-09T06:29:32.844437+0000","last_active":"2024-05-09T06:29:32.845682+0000","last_peered":"2024-05-09T06:29:32.845682+0000","last_clean":"2024-05-09T06:29:32.845682+0000","last_became_active":"2024-05-09T06:29:32.843623+0000","last_became_peered":"2024-05-09T06:29:32.843623+0000","last_unstale":"2024-05-09T06:29:32.845682+0000","last_undegraded":"2024-05-09T06:29:32.845682+0000","last_fullsized":"2024-05-09T06:29:32.845682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836215+0000","last_change":"2024-05-09T06:29:32.836215+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836215+0000","last_undegraded":"2024-05-09T06:29:32.836215+0000","last_fullsized":"2024-05-09T06:29:32.836215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.847445+0000","last_change":"2024-05-09T06:29:32.845924+0000","last_active":"2024-05-09T06:29:32.847445+0000","last_peered":"2024-05-09T06:29:32.847445+0000","last_clean":"2024-05-09T06:29:32.847445+0000","last_became_active":"2024-05-09T06:29:32.845637+0000","last_became_peered":"2024-05-09T06:29:32.845637+0000","last_unstale":"2024-05-09T06:29:32.847445+0000","last_undegraded":"2024-05-09T06:29:32.847445+0000","last_fullsized":"2024-05-09T06:29:32.847445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830680+0000","last_change":"2024-05-09T06:29:32.830680+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830680+0000","last_undegraded":"2024-05-09T06:29:32.830680+0000","last_fullsized":"2024-05-09T06:29:32.830680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844884+0000","last_change":"2024-05-09T06:29:32.841755+0000","last_active":"2024-05-09T06:29:32.844884+0000","last_peered":"2024-05-09T06:29:32.844884+0000","last_clean":"2024-05-09T06:29:32.844884+0000","last_became_active":"2024-05-09T06:29:32.840696+0000","last_became_peered":"2024-05-09T06:29:32.840696+0000","last_unstale":"2024-05-09T06:29:32.844884+0000","last_undegraded":"2024-05-09T06:29:32.844884+0000","last_fullsized":"2024-05-09T06:29:32.844884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:46.375838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832731+0000","last_change":"2024-05-09T06:29:32.832731+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832731+0000","last_undegraded":"2024-05-09T06:29:32.832731+0000","last_fullsized":"2024-05-09T06:29:32.832731+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845304+0000","last_change":"2024-05-09T06:29:32.841567+0000","last_active":"2024-05-09T06:29:32.845304+0000","last_peered":"2024-05-09T06:29:32.845304+0000","last_clean":"2024-05-09T06:29:32.845304+0000","last_became_active":"2024-05-09T06:29:32.841246+0000","last_became_peered":"2024-05-09T06:29:32.841246+0000","last_unstale":"2024-05-09T06:29:32.845304+0000","last_undegraded":"2024-05-09T06:29:32.845304+0000","last_fullsized":"2024-05-09T06:29:32.845304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:21.190037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831387+0000","last_change":"2024-05-09T06:29:32.831387+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831387+0000","last_undegraded":"2024-05-09T06:29:32.831387+0000","last_fullsized":"2024-05-09T06:29:32.831387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849643+0000","last_change":"2024-05-09T06:29:32.843603+0000","last_active":"2024-05-09T06:29:32.849643+0000","last_peered":"2024-05-09T06:29:32.849643+0000","last_clean":"2024-05-09T06:29:32.849643+0000","last_became_active":"2024-05-09T06:29:32.843314+0000","last_became_peered":"2024-05-09T06:29:32.843314+0000","last_unstale":"2024-05-09T06:29:32.849643+0000","last_undegraded":"2024-05-09T06:29:32.849643+0000","last_fullsized":"2024-05-09T06:29:32.849643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833783+0000","last_change":"2024-05-09T06:29:32.833783+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833783+0000","last_undegraded":"2024-05-09T06:29:32.833783+0000","last_fullsized":"2024-05-09T06:29:32.833783+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849604+0000","last_change":"2024-05-09T06:29:32.843469+0000","last_active":"2024-05-09T06:29:32.849604+0000","last_peered":"2024-05-09T06:29:32.849604+0000","last_clean":"2024-05-09T06:29:32.849604+0000","last_became_active":"2024-05-09T06:29:32.842907+0000","last_became_peered":"2024-05-09T06:29:32.842907+0000","last_unstale":"2024-05-09T06:29:32.849604+0000","last_undegraded":"2024-05-09T06:29:32.849604+0000","last_fullsized":"2024-05-09T06:29:32.849604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835859+0000","last_change":"2024-05-09T06:29:32.835859+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835859+0000","last_undegraded":"2024-05-09T06:29:32.835859+0000","last_fullsized":"2024-05-09T06:29:32.835859+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845941+0000","last_change":"2024-05-09T06:29:32.843492+0000","last_active":"2024-05-09T06:29:32.845941+0000","last_peered":"2024-05-09T06:29:32.845941+0000","last_clean":"2024-05-09T06:29:32.845941+0000","last_became_active":"2024-05-09T06:29:32.842665+0000","last_became_peered":"2024-05-09T06:29:32.842665+0000","last_unstale":"2024-05-09T06:29:32.845941+0000","last_undegraded":"2024-05-09T06:29:32.845941+0000","last_fullsized":"2024-05-09T06:29:32.845941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.830855+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:29:32.830855+0000","last_peered":"2024-05-09T06:29:32.830855+0000","last_clean":"2024-05-09T06:29:32.830855+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:29:32.830855+0000","last_undegraded":"2024-05-09T06:29:32.830855+0000","last_fullsized":"2024-05-09T06:29:32.830855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:29:38.226 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:29:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.384+0000 7fc759fce640 1 Processor -- start 2024-05-09T06:29:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.384+0000 7fc759fce640 1 -- start start 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc759fce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 0x7fc754099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc759fce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc75415e4b0 0x7fc75415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc759fce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc754099930 con 0x7fc7540b2aa0 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc759fce640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc7540b1e20 con 0x7fc75415e4b0 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc759fce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc75415ee90 con 0x7fc7540c6860 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc75415e4b0 0x7fc75415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc752ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc7540b2aa0 0x7fc7540c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38906/0 (socket says 172.21.15.1:38906) 2024-05-09T06:29:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc752ffd640 1 -- 172.21.15.1:0/1521703596 learned_addr learned my addr 172.21.15.1:0/1521703596 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc75415e4b0 0x7fc75415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40182/0 (socket says 172.21.15.1:40182) 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc7537fe640 1 --2- 172.21.15.1:0/1521703596 >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 0x7fc754099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 -- 172.21.15.1:0/1521703596 >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 msgr2=0x7fc754099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 --2- 172.21.15.1:0/1521703596 >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 0x7fc754099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 -- 172.21.15.1:0/1521703596 >> v1:172.21.15.1:6789/0 conn(0x7fc7540b2aa0 legacy=0x7fc7540c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc753fff640 1 -- 172.21.15.1:0/1521703596 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754162930 con 0x7fc75415e4b0 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc7527fc640 1 -- 172.21.15.1:0/1521703596 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218935869 0 0) 0x7fc748001110 con 0x7fc7540b2aa0 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.385+0000 7fc7537fe640 1 --2- 172.21.15.1:0/1521703596 >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 0x7fc754099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.386+0000 7fc753fff640 1 --2- 172.21.15.1:0/1521703596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc75415e4b0 0x7fc75415f320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc7540722c0 tx=0x7fc74c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7b4ea2a0e33578 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.386+0000 7fc7527fc640 1 -- 172.21.15.1:0/1521703596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c007740 con 0x7fc75415e4b0 2024-05-09T06:29:38.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.386+0000 7fc7527fc640 1 -- 172.21.15.1:0/1521703596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c0078e0 con 0x7fc75415e4b0 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.386+0000 7fc7527fc640 1 -- 172.21.15.1:0/1521703596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c007cb0 con 0x7fc75415e4b0 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 -- 172.21.15.1:0/1521703596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc75415e4b0 msgr2=0x7fc75415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 --2- 172.21.15.1:0/1521703596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc75415e4b0 0x7fc75415f320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc7540722c0 tx=0x7fc74c02a680 comp rx=0 tx=0).stop 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 -- 172.21.15.1:0/1521703596 shutdown_connections 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 --2- 172.21.15.1:0/1521703596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc75415e4b0 0x7fc75415f320 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 --2- 172.21.15.1:0/1521703596 >> v2:172.21.15.6:3300/0 conn(0x7fc7540c6860 0x7fc754099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 -- 172.21.15.1:0/1521703596 >> 172.21.15.1:0/1521703596 conn(0x7fc7540b1fb0 msgr2=0x7fc7540b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 -- 172.21.15.1:0/1521703596 shutdown_connections 2024-05-09T06:29:38.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.387+0000 7fc759fce640 1 -- 172.21.15.1:0/1521703596 wait complete. 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.388+0000 7fc759fce640 1 Processor -- start 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.388+0000 7fc759fce640 1 -- start start 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 0x7fc7540b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 0x7fc7540b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc75415f860 con 0x7fc7540b2aa0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc75415eb20 con 0x7fc7540c6860 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc759fce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc75415f9e0 con 0x7fc75415e4b0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40188/0 (socket says 172.21.15.1:40188) 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 -- 172.21.15.1:0/1303180787 learned_addr learned my addr 172.21.15.1:0/1303180787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc752ffd640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 0x7fc7540b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc753fff640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 0x7fc7540b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 -- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 msgr2=0x7fc7540b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 0x7fc7540b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 -- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 msgr2=0x7fc7540b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 0x7fc7540b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc7537fe640 1 -- 172.21.15.1:0/1303180787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754159ac0 con 0x7fc7540b2aa0 2024-05-09T06:29:38.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc752ffd640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 0x7fc7540b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.389+0000 7fc753fff640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 0x7fc7540b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc7537fe640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e7e0 tx=0x7fc74400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74400f040 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc744004030 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7540b3c10 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744004400 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.390+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7540b41d0 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.391+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7540752b0 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.391+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc74400c6d0 con 0x7fc7540b2aa0 2024-05-09T06:29:38.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.392+0000 7fc73bfff640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 0x7fc74006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.392+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fc74408add0 con 0x7fc7540b2aa0 2024-05-09T06:29:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.392+0000 7fc752ffd640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 0x7fc74006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.392+0000 7fc752ffd640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 0x7fc74006e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7480013b0 tx=0x7fc748009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.396+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc744017020 con 0x7fc7540b2aa0 2024-05-09T06:29:38.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.578+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc75406ce30 con 0x7fc74006be30 2024-05-09T06:29:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.584+0000 7fc73bfff640 1 -- 172.21.15.1:0/1303180787 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7fc75406ce30 con 0x7fc74006be30 2024-05-09T06:29:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:38.569 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 msgr2=0x7fc74006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 0x7fc74006e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7480013b0 tx=0x7fc748009040 comp rx=0 tx=0).stop 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 msgr2=0x7fc754159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e7e0 tx=0x7fc74400eca0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 shutdown_connections 2024-05-09T06:29:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc74006be30 0x7fc74006e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc75415e4b0 0x7fc7540b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7540c6860 0x7fc7540b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 --2- 172.21.15.1:0/1303180787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7540b2aa0 0x7fc754159580 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 >> 172.21.15.1:0/1303180787 conn(0x7fc7540b1fb0 msgr2=0x7fc7540cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 shutdown_connections 2024-05-09T06:29:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.587+0000 7fc759fce640 1 -- 172.21.15.1:0/1303180787 wait complete. 2024-05-09T06:29:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-05-09T06:29:37.096005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238428,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736292,"statfs":{"total":773094113280,"available":772849963008,"internally_reserved":0,"allocated":5742592,"data_stored":3179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.274802"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827195+0000","last_change":"2024-05-09T06:29:32.827195+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827195+0000","last_undegraded":"2024-05-09T06:29:32.827195+0000","last_fullsized":"2024-05-09T06:29:32.827195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830494+0000","last_change":"2024-05-09T06:29:32.830494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830494+0000","last_undegraded":"2024-05-09T06:29:32.830494+0000","last_fullsized":"2024-05-09T06:29:32.830494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840400+0000","last_change":"2024-05-09T06:29:32.840400+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840400+0000","last_undegraded":"2024-05-09T06:29:32.840400+0000","last_fullsized":"2024-05-09T06:29:32.840400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828893+0000","last_change":"2024-05-09T06:29:32.828893+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828893+0000","last_undegraded":"2024-05-09T06:29:32.828893+0000","last_fullsized":"2024-05-09T06:29:32.828893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832054+0000","last_change":"2024-05-09T06:29:32.832054+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832054+0000","last_undegraded":"2024-05-09T06:29:32.832054+0000","last_fullsized":"2024-05-09T06:29:32.832054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826494+0000","last_change":"2024-05-09T06:29:32.826494+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826494+0000","last_undegraded":"2024-05-09T06:29:32.826494+0000","last_fullsized":"2024-05-09T06:29:32.826494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.840466+0000","last_change":"2024-05-09T06:29:32.840466+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.840466+0000","last_undegraded":"2024-05-09T06:29:32.840466+0000","last_fullsized":"2024-05-09T06:29:32.840466+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834488+0000","last_change":"2024-05-09T06:29:32.834488+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834488+0000","last_undegraded":"2024-05-09T06:29:32.834488+0000","last_fullsized":"2024-05-09T06:29:32.834488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835155+0000","last_change":"2024-05-09T06:29:32.835155+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835155+0000","last_undegraded":"2024-05-09T06:29:32.835155+0000","last_fullsized":"2024-05-09T06:29:32.835155+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827341+0000","last_change":"2024-05-09T06:29:32.827341+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827341+0000","last_undegraded":"2024-05-09T06:29:32.827341+0000","last_fullsized":"2024-05-09T06:29:32.827341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828942+0000","last_change":"2024-05-09T06:29:32.828942+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828942+0000","last_undegraded":"2024-05-09T06:29:32.828942+0000","last_fullsized":"2024-05-09T06:29:32.828942+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832892+0000","last_change":"2024-05-09T06:29:32.832892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832892+0000","last_undegraded":"2024-05-09T06:29:32.832892+0000","last_fullsized":"2024-05-09T06:29:32.832892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827743+0000","last_change":"2024-05-09T06:29:32.827743+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827743+0000","last_undegraded":"2024-05-09T06:29:32.827743+0000","last_fullsized":"2024-05-09T06:29:32.827743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836723+0000","last_change":"2024-05-09T06:29:32.836723+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836723+0000","last_undegraded":"2024-05-09T06:29:32.836723+0000","last_fullsized":"2024-05-09T06:29:32.836723+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828834+0000","last_change":"2024-05-09T06:29:32.828834+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828834+0000","last_undegraded":"2024-05-09T06:29:32.828834+0000","last_fullsized":"2024-05-09T06:29:32.828834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826180+0000","last_change":"2024-05-09T06:29:32.826180+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826180+0000","last_undegraded":"2024-05-09T06:29:32.826180+0000","last_fullsized":"2024-05-09T06:29:32.826180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836265+0000","last_change":"2024-05-09T06:29:32.836265+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836265+0000","last_undegraded":"2024-05-09T06:29:32.836265+0000","last_fullsized":"2024-05-09T06:29:32.836265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830004+0000","last_change":"2024-05-09T06:29:32.830004+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830004+0000","last_undegraded":"2024-05-09T06:29:32.830004+0000","last_fullsized":"2024-05-09T06:29:32.830004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835867+0000","last_change":"2024-05-09T06:29:32.835867+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835867+0000","last_undegraded":"2024-05-09T06:29:32.835867+0000","last_fullsized":"2024-05-09T06:29:32.835867+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.837023+0000","last_change":"2024-05-09T06:29:32.837023+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.837023+0000","last_undegraded":"2024-05-09T06:29:32.837023+0000","last_fullsized":"2024-05-09T06:29:32.837023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836127+0000","last_change":"2024-05-09T06:29:32.836127+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836127+0000","last_undegraded":"2024-05-09T06:29:32.836127+0000","last_fullsized":"2024-05-09T06:29:32.836127+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832922+0000","last_change":"2024-05-09T06:29:32.832922+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832922+0000","last_undegraded":"2024-05-09T06:29:32.832922+0000","last_fullsized":"2024-05-09T06:29:32.832922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828814+0000","last_change":"2024-05-09T06:29:32.828814+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828814+0000","last_undegraded":"2024-05-09T06:29:32.828814+0000","last_fullsized":"2024-05-09T06:29:32.828814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835263+0000","last_change":"2024-05-09T06:29:32.835263+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835263+0000","last_undegraded":"2024-05-09T06:29:32.835263+0000","last_fullsized":"2024-05-09T06:29:32.835263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834730+0000","last_change":"2024-05-09T06:29:32.834730+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834730+0000","last_undegraded":"2024-05-09T06:29:32.834730+0000","last_fullsized":"2024-05-09T06:29:32.834730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834843+0000","last_change":"2024-05-09T06:29:32.834843+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834843+0000","last_undegraded":"2024-05-09T06:29:32.834843+0000","last_fullsized":"2024-05-09T06:29:32.834843+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826703+0000","last_change":"2024-05-09T06:29:32.826703+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826703+0000","last_undegraded":"2024-05-09T06:29:32.826703+0000","last_fullsized":"2024-05-09T06:29:32.826703+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836464+0000","last_change":"2024-05-09T06:29:32.836464+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836464+0000","last_undegraded":"2024-05-09T06:29:32.836464+0000","last_fullsized":"2024-05-09T06:29:32.836464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834001+0000","last_change":"2024-05-09T06:29:32.834001+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834001+0000","last_undegraded":"2024-05-09T06:29:32.834001+0000","last_fullsized":"2024-05-09T06:29:32.834001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829477+0000","last_change":"2024-05-09T06:29:32.829477+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829477+0000","last_undegraded":"2024-05-09T06:29:32.829477+0000","last_fullsized":"2024-05-09T06:29:32.829477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833492+0000","last_change":"2024-05-09T06:29:32.833492+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833492+0000","last_undegraded":"2024-05-09T06:29:32.833492+0000","last_fullsized":"2024-05-09T06:29:32.833492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833542+0000","last_change":"2024-05-09T06:29:32.833542+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833542+0000","last_undegraded":"2024-05-09T06:29:32.833542+0000","last_fullsized":"2024-05-09T06:29:32.833542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828830+0000","last_change":"2024-05-09T06:29:32.828830+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828830+0000","last_undegraded":"2024-05-09T06:29:32.828830+0000","last_fullsized":"2024-05-09T06:29:32.828830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849652+0000","last_change":"2024-05-09T06:29:32.843136+0000","last_active":"2024-05-09T06:29:32.849652+0000","last_peered":"2024-05-09T06:29:32.849652+0000","last_clean":"2024-05-09T06:29:32.849652+0000","last_became_active":"2024-05-09T06:29:32.842842+0000","last_became_peered":"2024-05-09T06:29:32.842842+0000","last_unstale":"2024-05-09T06:29:32.849652+0000","last_undegraded":"2024-05-09T06:29:32.849652+0000","last_fullsized":"2024-05-09T06:29:32.849652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835845+0000","last_change":"2024-05-09T06:29:32.835845+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835845+0000","last_undegraded":"2024-05-09T06:29:32.835845+0000","last_fullsized":"2024-05-09T06:29:32.835845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843214+0000","last_change":"2024-05-09T06:29:32.842035+0000","last_active":"2024-05-09T06:29:32.843214+0000","last_peered":"2024-05-09T06:29:32.843214+0000","last_clean":"2024-05-09T06:29:32.843214+0000","last_became_active":"2024-05-09T06:29:32.841756+0000","last_became_peered":"2024-05-09T06:29:32.841756+0000","last_unstale":"2024-05-09T06:29:32.843214+0000","last_undegraded":"2024-05-09T06:29:32.843214+0000","last_fullsized":"2024-05-09T06:29:32.843214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836568+0000","last_change":"2024-05-09T06:29:32.836568+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836568+0000","last_undegraded":"2024-05-09T06:29:32.836568+0000","last_fullsized":"2024-05-09T06:29:32.836568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844302+0000","last_change":"2024-05-09T06:29:32.842788+0000","last_active":"2024-05-09T06:29:32.844302+0000","last_peered":"2024-05-09T06:29:32.844302+0000","last_clean":"2024-05-09T06:29:32.844302+0000","last_became_active":"2024-05-09T06:29:32.842461+0000","last_became_peered":"2024-05-09T06:29:32.842461+0000","last_unstale":"2024-05-09T06:29:32.844302+0000","last_undegraded":"2024-05-09T06:29:32.844302+0000","last_fullsized":"2024-05-09T06:29:32.844302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836686+0000","last_change":"2024-05-09T06:29:32.836686+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836686+0000","last_undegraded":"2024-05-09T06:29:32.836686+0000","last_fullsized":"2024-05-09T06:29:32.836686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852441+0000","last_change":"2024-05-09T06:29:32.845755+0000","last_active":"2024-05-09T06:29:32.852441+0000","last_peered":"2024-05-09T06:29:32.852441+0000","last_clean":"2024-05-09T06:29:32.852441+0000","last_became_active":"2024-05-09T06:29:32.845443+0000","last_became_peered":"2024-05-09T06:29:32.845443+0000","last_unstale":"2024-05-09T06:29:32.852441+0000","last_undegraded":"2024-05-09T06:29:32.852441+0000","last_fullsized":"2024-05-09T06:29:32.852441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:37.820825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833767+0000","last_change":"2024-05-09T06:29:32.833767+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833767+0000","last_undegraded":"2024-05-09T06:29:32.833767+0000","last_fullsized":"2024-05-09T06:29:32.833767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844351+0000","last_change":"2024-05-09T06:29:32.831050+0000","last_active":"2024-05-09T06:29:32.844351+0000","last_peered":"2024-05-09T06:29:32.844351+0000","last_clean":"2024-05-09T06:29:32.844351+0000","last_became_active":"2024-05-09T06:29:32.830760+0000","last_became_peered":"2024-05-09T06:29:32.830760+0000","last_unstale":"2024-05-09T06:29:32.844351+0000","last_undegraded":"2024-05-09T06:29:32.844351+0000","last_fullsized":"2024-05-09T06:29:32.844351+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.828961+0000","last_change":"2024-05-09T06:29:32.828961+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.828961+0000","last_undegraded":"2024-05-09T06:29:32.828961+0000","last_fullsized":"2024-05-09T06:29:32.828961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849673+0000","last_change":"2024-05-09T06:29:32.842910+0000","last_active":"2024-05-09T06:29:32.849673+0000","last_peered":"2024-05-09T06:29:32.849673+0000","last_clean":"2024-05-09T06:29:32.849673+0000","last_became_active":"2024-05-09T06:29:32.842534+0000","last_became_peered":"2024-05-09T06:29:32.842534+0000","last_unstale":"2024-05-09T06:29:32.849673+0000","last_undegraded":"2024-05-09T06:29:32.849673+0000","last_fullsized":"2024-05-09T06:29:32.849673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:33.233890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831887+0000","last_change":"2024-05-09T06:29:32.831887+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831887+0000","last_undegraded":"2024-05-09T06:29:32.831887+0000","last_fullsized":"2024-05-09T06:29:32.831887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844998+0000","last_change":"2024-05-09T06:29:32.840442+0000","last_active":"2024-05-09T06:29:32.844998+0000","last_peered":"2024-05-09T06:29:32.844998+0000","last_clean":"2024-05-09T06:29:32.844998+0000","last_became_active":"2024-05-09T06:29:32.840016+0000","last_became_peered":"2024-05-09T06:29:32.840016+0000","last_unstale":"2024-05-09T06:29:32.844998+0000","last_undegraded":"2024-05-09T06:29:32.844998+0000","last_fullsized":"2024-05-09T06:29:32.844998+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829388+0000","last_change":"2024-05-09T06:29:32.829388+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829388+0000","last_undegraded":"2024-05-09T06:29:32.829388+0000","last_fullsized":"2024-05-09T06:29:32.829388+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846231+0000","last_change":"2024-05-09T06:29:32.844658+0000","last_active":"2024-05-09T06:29:32.846231+0000","last_peered":"2024-05-09T06:29:32.846231+0000","last_clean":"2024-05-09T06:29:32.846231+0000","last_became_active":"2024-05-09T06:29:32.844008+0000","last_became_peered":"2024-05-09T06:29:32.844008+0000","last_unstale":"2024-05-09T06:29:32.846231+0000","last_undegraded":"2024-05-09T06:29:32.846231+0000","last_fullsized":"2024-05-09T06:29:32.846231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848806+0000","last_change":"2024-05-09T06:29:32.845642+0000","last_active":"2024-05-09T06:29:32.848806+0000","last_peered":"2024-05-09T06:29:32.848806+0000","last_clean":"2024-05-09T06:29:32.848806+0000","last_became_active":"2024-05-09T06:29:32.845324+0000","last_became_peered":"2024-05-09T06:29:32.845324+0000","last_unstale":"2024-05-09T06:29:32.848806+0000","last_undegraded":"2024-05-09T06:29:32.848806+0000","last_fullsized":"2024-05-09T06:29:32.848806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836258+0000","last_change":"2024-05-09T06:29:32.836258+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836258+0000","last_undegraded":"2024-05-09T06:29:32.836258+0000","last_fullsized":"2024-05-09T06:29:32.836258+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852598+0000","last_change":"2024-05-09T06:29:32.847975+0000","last_active":"2024-05-09T06:29:32.852598+0000","last_peered":"2024-05-09T06:29:32.852598+0000","last_clean":"2024-05-09T06:29:32.852598+0000","last_became_active":"2024-05-09T06:29:32.847772+0000","last_became_peered":"2024-05-09T06:29:32.847772+0000","last_unstale":"2024-05-09T06:29:32.852598+0000","last_undegraded":"2024-05-09T06:29:32.852598+0000","last_fullsized":"2024-05-09T06:29:32.852598+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:52.605087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831892+0000","last_change":"2024-05-09T06:29:32.831892+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831892+0000","last_undegraded":"2024-05-09T06:29:32.831892+0000","last_fullsized":"2024-05-09T06:29:32.831892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845661+0000","last_change":"2024-05-09T06:29:32.843594+0000","last_active":"2024-05-09T06:29:32.845661+0000","last_peered":"2024-05-09T06:29:32.845661+0000","last_clean":"2024-05-09T06:29:32.845661+0000","last_became_active":"2024-05-09T06:29:32.843233+0000","last_became_peered":"2024-05-09T06:29:32.843233+0000","last_unstale":"2024-05-09T06:29:32.845661+0000","last_undegraded":"2024-05-09T06:29:32.845661+0000","last_fullsized":"2024-05-09T06:29:32.845661+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831210+0000","last_change":"2024-05-09T06:29:32.831210+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831210+0000","last_undegraded":"2024-05-09T06:29:32.831210+0000","last_fullsized":"2024-05-09T06:29:32.831210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844934+0000","last_change":"2024-05-09T06:29:32.841265+0000","last_active":"2024-05-09T06:29:32.844934+0000","last_peered":"2024-05-09T06:29:32.844934+0000","last_clean":"2024-05-09T06:29:32.844934+0000","last_became_active":"2024-05-09T06:29:32.840989+0000","last_became_peered":"2024-05-09T06:29:32.840989+0000","last_unstale":"2024-05-09T06:29:32.844934+0000","last_undegraded":"2024-05-09T06:29:32.844934+0000","last_fullsized":"2024-05-09T06:29:32.844934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:10.424765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834060+0000","last_change":"2024-05-09T06:29:32.834060+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834060+0000","last_undegraded":"2024-05-09T06:29:32.834060+0000","last_fullsized":"2024-05-09T06:29:32.834060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.846007+0000","last_change":"2024-05-09T06:29:32.842270+0000","last_active":"2024-05-09T06:29:32.846007+0000","last_peered":"2024-05-09T06:29:32.846007+0000","last_clean":"2024-05-09T06:29:32.846007+0000","last_became_active":"2024-05-09T06:29:32.841999+0000","last_became_peered":"2024-05-09T06:29:32.841999+0000","last_unstale":"2024-05-09T06:29:32.846007+0000","last_undegraded":"2024-05-09T06:29:32.846007+0000","last_fullsized":"2024-05-09T06:29:32.846007+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:52.611641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832434+0000","last_change":"2024-05-09T06:29:32.832434+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832434+0000","last_undegraded":"2024-05-09T06:29:32.832434+0000","last_fullsized":"2024-05-09T06:29:32.832434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852458+0000","last_change":"2024-05-09T06:29:32.846452+0000","last_active":"2024-05-09T06:29:32.852458+0000","last_peered":"2024-05-09T06:29:32.852458+0000","last_clean":"2024-05-09T06:29:32.852458+0000","last_became_active":"2024-05-09T06:29:32.846152+0000","last_became_peered":"2024-05-09T06:29:32.846152+0000","last_unstale":"2024-05-09T06:29:32.852458+0000","last_undegraded":"2024-05-09T06:29:32.852458+0000","last_fullsized":"2024-05-09T06:29:32.852458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:35.033218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.834435+0000","last_change":"2024-05-09T06:29:32.834435+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.834435+0000","last_undegraded":"2024-05-09T06:29:32.834435+0000","last_fullsized":"2024-05-09T06:29:32.834435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845010+0000","last_change":"2024-05-09T06:29:32.840887+0000","last_active":"2024-05-09T06:29:32.845010+0000","last_peered":"2024-05-09T06:29:32.845010+0000","last_clean":"2024-05-09T06:29:32.845010+0000","last_became_active":"2024-05-09T06:29:32.840544+0000","last_became_peered":"2024-05-09T06:29:32.840544+0000","last_unstale":"2024-05-09T06:29:32.845010+0000","last_undegraded":"2024-05-09T06:29:32.845010+0000","last_fullsized":"2024-05-09T06:29:32.845010+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832901+0000","last_change":"2024-05-09T06:29:32.832901+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832901+0000","last_undegraded":"2024-05-09T06:29:32.832901+0000","last_fullsized":"2024-05-09T06:29:32.832901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843689+0000","last_change":"2024-05-09T06:29:32.839558+0000","last_active":"2024-05-09T06:29:32.843689+0000","last_peered":"2024-05-09T06:29:32.843689+0000","last_clean":"2024-05-09T06:29:32.843689+0000","last_became_active":"2024-05-09T06:29:32.839220+0000","last_became_peered":"2024-05-09T06:29:32.839220+0000","last_unstale":"2024-05-09T06:29:32.843689+0000","last_undegraded":"2024-05-09T06:29:32.843689+0000","last_fullsized":"2024-05-09T06:29:32.843689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:03.213349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831995+0000","last_change":"2024-05-09T06:29:32.831995+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831995+0000","last_undegraded":"2024-05-09T06:29:32.831995+0000","last_fullsized":"2024-05-09T06:29:32.831995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.848583+0000","last_change":"2024-05-09T06:29:32.847677+0000","last_active":"2024-05-09T06:29:32.848583+0000","last_peered":"2024-05-09T06:29:32.848583+0000","last_clean":"2024-05-09T06:29:32.848583+0000","last_became_active":"2024-05-09T06:29:32.847372+0000","last_became_peered":"2024-05-09T06:29:32.847372+0000","last_unstale":"2024-05-09T06:29:32.848583+0000","last_undegraded":"2024-05-09T06:29:32.848583+0000","last_fullsized":"2024-05-09T06:29:32.848583+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825768+0000","last_change":"2024-05-09T06:29:32.825768+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825768+0000","last_undegraded":"2024-05-09T06:29:32.825768+0000","last_fullsized":"2024-05-09T06:29:32.825768+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843231+0000","last_change":"2024-05-09T06:29:32.842351+0000","last_active":"2024-05-09T06:29:32.843231+0000","last_peered":"2024-05-09T06:29:32.843231+0000","last_clean":"2024-05-09T06:29:32.843231+0000","last_became_active":"2024-05-09T06:29:32.842065+0000","last_became_peered":"2024-05-09T06:29:32.842065+0000","last_unstale":"2024-05-09T06:29:32.843231+0000","last_undegraded":"2024-05-09T06:29:32.843231+0000","last_fullsized":"2024-05-09T06:29:32.843231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:57.204005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831451+0000","last_change":"2024-05-09T06:29:32.831451+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831451+0000","last_undegraded":"2024-05-09T06:29:32.831451+0000","last_fullsized":"2024-05-09T06:29:32.831451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849727+0000","last_change":"2024-05-09T06:29:32.844205+0000","last_active":"2024-05-09T06:29:32.849727+0000","last_peered":"2024-05-09T06:29:32.849727+0000","last_clean":"2024-05-09T06:29:32.849727+0000","last_became_active":"2024-05-09T06:29:32.843719+0000","last_became_peered":"2024-05-09T06:29:32.843719+0000","last_unstale":"2024-05-09T06:29:32.849727+0000","last_undegraded":"2024-05-09T06:29:32.849727+0000","last_fullsized":"2024-05-09T06:29:32.849727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:22.182054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.827283+0000","last_change":"2024-05-09T06:29:32.827283+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.827283+0000","last_undegraded":"2024-05-09T06:29:32.827283+0000","last_fullsized":"2024-05-09T06:29:32.827283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852495+0000","last_change":"2024-05-09T06:29:32.847048+0000","last_active":"2024-05-09T06:29:32.852495+0000","last_peered":"2024-05-09T06:29:32.852495+0000","last_clean":"2024-05-09T06:29:32.852495+0000","last_became_active":"2024-05-09T06:29:32.846701+0000","last_became_peered":"2024-05-09T06:29:32.846701+0000","last_unstale":"2024-05-09T06:29:32.852495+0000","last_undegraded":"2024-05-09T06:29:32.852495+0000","last_fullsized":"2024-05-09T06:29:32.852495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833611+0000","last_change":"2024-05-09T06:29:32.833611+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833611+0000","last_undegraded":"2024-05-09T06:29:32.833611+0000","last_fullsized":"2024-05-09T06:29:32.833611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.842410+0000","last_change":"2024-05-09T06:29:32.840423+0000","last_active":"2024-05-09T06:29:32.842410+0000","last_peered":"2024-05-09T06:29:32.842410+0000","last_clean":"2024-05-09T06:29:32.842410+0000","last_became_active":"2024-05-09T06:29:32.840136+0000","last_became_peered":"2024-05-09T06:29:32.840136+0000","last_unstale":"2024-05-09T06:29:32.842410+0000","last_undegraded":"2024-05-09T06:29:32.842410+0000","last_fullsized":"2024-05-09T06:29:32.842410+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836148+0000","last_change":"2024-05-09T06:29:32.836148+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836148+0000","last_undegraded":"2024-05-09T06:29:32.836148+0000","last_fullsized":"2024-05-09T06:29:32.836148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.843738+0000","last_change":"2024-05-09T06:29:32.839491+0000","last_active":"2024-05-09T06:29:32.843738+0000","last_peered":"2024-05-09T06:29:32.843738+0000","last_clean":"2024-05-09T06:29:32.843738+0000","last_became_active":"2024-05-09T06:29:32.839083+0000","last_became_peered":"2024-05-09T06:29:32.839083+0000","last_unstale":"2024-05-09T06:29:32.843738+0000","last_undegraded":"2024-05-09T06:29:32.843738+0000","last_fullsized":"2024-05-09T06:29:32.843738+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:43.479453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.829862+0000","last_change":"2024-05-09T06:29:32.829862+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.829862+0000","last_undegraded":"2024-05-09T06:29:32.829862+0000","last_fullsized":"2024-05-09T06:29:32.829862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845866+0000","last_change":"2024-05-09T06:29:32.840100+0000","last_active":"2024-05-09T06:29:32.845866+0000","last_peered":"2024-05-09T06:29:32.845866+0000","last_clean":"2024-05-09T06:29:32.845866+0000","last_became_active":"2024-05-09T06:29:32.839832+0000","last_became_peered":"2024-05-09T06:29:32.839832+0000","last_unstale":"2024-05-09T06:29:32.845866+0000","last_undegraded":"2024-05-09T06:29:32.845866+0000","last_fullsized":"2024-05-09T06:29:32.845866+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.826694+0000","last_change":"2024-05-09T06:29:32.826694+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.826694+0000","last_undegraded":"2024-05-09T06:29:32.826694+0000","last_fullsized":"2024-05-09T06:29:32.826694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.835648+0000","last_change":"2024-05-09T06:29:32.832884+0000","last_active":"2024-05-09T06:29:32.835648+0000","last_peered":"2024-05-09T06:29:32.835648+0000","last_clean":"2024-05-09T06:29:32.835648+0000","last_became_active":"2024-05-09T06:29:32.832612+0000","last_became_peered":"2024-05-09T06:29:32.832612+0000","last_unstale":"2024-05-09T06:29:32.835648+0000","last_undegraded":"2024-05-09T06:29:32.835648+0000","last_fullsized":"2024-05-09T06:29:32.835648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:50.231444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833045+0000","last_change":"2024-05-09T06:29:32.833045+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833045+0000","last_undegraded":"2024-05-09T06:29:32.833045+0000","last_fullsized":"2024-05-09T06:29:32.833045+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.852509+0000","last_change":"2024-05-09T06:29:32.846605+0000","last_active":"2024-05-09T06:29:32.852509+0000","last_peered":"2024-05-09T06:29:32.852509+0000","last_clean":"2024-05-09T06:29:32.852509+0000","last_became_active":"2024-05-09T06:29:32.846230+0000","last_became_peered":"2024-05-09T06:29:32.846230+0000","last_unstale":"2024-05-09T06:29:32.852509+0000","last_undegraded":"2024-05-09T06:29:32.852509+0000","last_fullsized":"2024-05-09T06:29:32.852509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.825813+0000","last_change":"2024-05-09T06:29:32.825813+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.825813+0000","last_undegraded":"2024-05-09T06:29:32.825813+0000","last_fullsized":"2024-05-09T06:29:32.825813+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845682+0000","last_change":"2024-05-09T06:29:32.844437+0000","last_active":"2024-05-09T06:29:32.845682+0000","last_peered":"2024-05-09T06:29:32.845682+0000","last_clean":"2024-05-09T06:29:32.845682+0000","last_became_active":"2024-05-09T06:29:32.843623+0000","last_became_peered":"2024-05-09T06:29:32.843623+0000","last_unstale":"2024-05-09T06:29:32.845682+0000","last_undegraded":"2024-05-09T06:29:32.845682+0000","last_fullsized":"2024-05-09T06:29:32.845682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.836215+0000","last_change":"2024-05-09T06:29:32.836215+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.836215+0000","last_undegraded":"2024-05-09T06:29:32.836215+0000","last_fullsized":"2024-05-09T06:29:32.836215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.847445+0000","last_change":"2024-05-09T06:29:32.845924+0000","last_active":"2024-05-09T06:29:32.847445+0000","last_peered":"2024-05-09T06:29:32.847445+0000","last_clean":"2024-05-09T06:29:32.847445+0000","last_became_active":"2024-05-09T06:29:32.845637+0000","last_became_peered":"2024-05-09T06:29:32.845637+0000","last_unstale":"2024-05-09T06:29:32.847445+0000","last_undegraded":"2024-05-09T06:29:32.847445+0000","last_fullsized":"2024-05-09T06:29:32.847445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.830680+0000","last_change":"2024-05-09T06:29:32.830680+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.830680+0000","last_undegraded":"2024-05-09T06:29:32.830680+0000","last_fullsized":"2024-05-09T06:29:32.830680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.844884+0000","last_change":"2024-05-09T06:29:32.841755+0000","last_active":"2024-05-09T06:29:32.844884+0000","last_peered":"2024-05-09T06:29:32.844884+0000","last_clean":"2024-05-09T06:29:32.844884+0000","last_became_active":"2024-05-09T06:29:32.840696+0000","last_became_peered":"2024-05-09T06:29:32.840696+0000","last_unstale":"2024-05-09T06:29:32.844884+0000","last_undegraded":"2024-05-09T06:29:32.844884+0000","last_fullsized":"2024-05-09T06:29:32.844884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:46.375838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.832731+0000","last_change":"2024-05-09T06:29:32.832731+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.832731+0000","last_undegraded":"2024-05-09T06:29:32.832731+0000","last_fullsized":"2024-05-09T06:29:32.832731+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845304+0000","last_change":"2024-05-09T06:29:32.841567+0000","last_active":"2024-05-09T06:29:32.845304+0000","last_peered":"2024-05-09T06:29:32.845304+0000","last_clean":"2024-05-09T06:29:32.845304+0000","last_became_active":"2024-05-09T06:29:32.841246+0000","last_became_peered":"2024-05-09T06:29:32.841246+0000","last_unstale":"2024-05-09T06:29:32.845304+0000","last_undegraded":"2024-05-09T06:29:32.845304+0000","last_fullsized":"2024-05-09T06:29:32.845304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:21.190037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.831387+0000","last_change":"2024-05-09T06:29:32.831387+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.831387+0000","last_undegraded":"2024-05-09T06:29:32.831387+0000","last_fullsized":"2024-05-09T06:29:32.831387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849643+0000","last_change":"2024-05-09T06:29:32.843603+0000","last_active":"2024-05-09T06:29:32.849643+0000","last_peered":"2024-05-09T06:29:32.849643+0000","last_clean":"2024-05-09T06:29:32.849643+0000","last_became_active":"2024-05-09T06:29:32.843314+0000","last_became_peered":"2024-05-09T06:29:32.843314+0000","last_unstale":"2024-05-09T06:29:32.849643+0000","last_undegraded":"2024-05-09T06:29:32.849643+0000","last_fullsized":"2024-05-09T06:29:32.849643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:27.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.833783+0000","last_change":"2024-05-09T06:29:32.833783+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.833783+0000","last_undegraded":"2024-05-09T06:29:32.833783+0000","last_fullsized":"2024-05-09T06:29:32.833783+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.849604+0000","last_change":"2024-05-09T06:29:32.843469+0000","last_active":"2024-05-09T06:29:32.849604+0000","last_peered":"2024-05-09T06:29:32.849604+0000","last_clean":"2024-05-09T06:29:32.849604+0000","last_became_active":"2024-05-09T06:29:32.842907+0000","last_became_peered":"2024-05-09T06:29:32.842907+0000","last_unstale":"2024-05-09T06:29:32.849604+0000","last_undegraded":"2024-05-09T06:29:32.849604+0000","last_fullsized":"2024-05-09T06:29:32.849604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-09T06:29:32.835859+0000","last_change":"2024-05-09T06:29:32.835859+0000","last_active":"2024-05-09T06:29:32.820766+0000","last_peered":"2024-05-09T06:29:32.820766+0000","last_clean":"2024-05-09T06:29:32.820766+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:29:32.835859+0000","last_undegraded":"2024-05-09T06:29:32.835859+0000","last_fullsized":"2024-05-09T06:29:32.835859+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.845941+0000","last_change":"2024-05-09T06:29:32.843492+0000","last_active":"2024-05-09T06:29:32.845941+0000","last_peered":"2024-05-09T06:29:32.845941+0000","last_clean":"2024-05-09T06:29:32.845941+0000","last_became_active":"2024-05-09T06:29:32.842665+0000","last_became_peered":"2024-05-09T06:29:32.842665+0000","last_unstale":"2024-05-09T06:29:32.845941+0000","last_undegraded":"2024-05-09T06:29:32.845941+0000","last_fullsized":"2024-05-09T06:29:32.845941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T06:29:32.830855+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:29:32.830855+0000","last_peered":"2024-05-09T06:29:32.830855+0000","last_clean":"2024-05-09T06:29:32.830855+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:29:32.830855+0000","last_undegraded":"2024-05-09T06:29:32.830855+0000","last_fullsized":"2024-05-09T06:29:32.830855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":249794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":840162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.827195+0000', 'last_change': '2024-05-09T06:29:32.827195+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.827195+0000', 'last_undegraded': '2024-05-09T06:29:32.827195+0000', 'last_fullsized': '2024-05-09T06:29:32.827195+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.830494+0000', 'last_change': '2024-05-09T06:29:32.830494+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.830494+0000', 'last_undegraded': '2024-05-09T06:29:32.830494+0000', 'last_fullsized': '2024-05-09T06:29:32.830494+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.840400+0000', 'last_change': '2024-05-09T06:29:32.840400+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.840400+0000', 'last_undegraded': '2024-05-09T06:29:32.840400+0000', 'last_fullsized': '2024-05-09T06:29:32.840400+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828893+0000', 'last_change': '2024-05-09T06:29:32.828893+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828893+0000', 'last_undegraded': '2024-05-09T06:29:32.828893+0000', 'last_fullsized': '2024-05-09T06:29:32.828893+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832054+0000', 'last_change': '2024-05-09T06:29:32.832054+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832054+0000', 'last_undegraded': '2024-05-09T06:29:32.832054+0000', 'last_fullsized': '2024-05-09T06:29:32.832054+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.826494+0000', 'last_change': '2024-05-09T06:29:32.826494+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.826494+0000', 'last_undegraded': '2024-05-09T06:29:32.826494+0000', 'last_fullsized': '2024-05-09T06:29:32.826494+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.840466+0000', 'last_change': '2024-05-09T06:29:32.840466+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.840466+0000', 'last_undegraded': '2024-05-09T06:29:32.840466+0000', 'last_fullsized': '2024-05-09T06:29:32.840466+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834488+0000', 'last_change': '2024-05-09T06:29:32.834488+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834488+0000', 'last_undegraded': '2024-05-09T06:29:32.834488+0000', 'last_fullsized': '2024-05-09T06:29:32.834488+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.835155+0000', 'last_change': '2024-05-09T06:29:32.835155+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.835155+0000', 'last_undegraded': '2024-05-09T06:29:32.835155+0000', 'last_fullsized': '2024-05-09T06:29:32.835155+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-05-09T06:29:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.827341+0000', 'last_change': '2024-05-09T06:29:32.827341+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.827341+0000', 'last_undegraded': '2024-05-09T06:29:32.827341+0000', 'last_fullsized': '2024-05-09T06:29:32.827341+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828942+0000', 'last_change': '2024-05-09T06:29:32.828942+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828942+0000', 'last_undegraded': '2024-05-09T06:29:32.828942+0000', 'last_fullsized': '2024-05-09T06:29:32.828942+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832892+0000', 'last_change': '2024-05-09T06:29:32.832892+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832892+0000', 'last_undegraded': '2024-05-09T06:29:32.832892+0000', 'last_fullsized': '2024-05-09T06:29:32.832892+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.827743+0000', 'last_change': '2024-05-09T06:29:32.827743+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.827743+0000', 'last_undegraded': '2024-05-09T06:29:32.827743+0000', 'last_fullsized': '2024-05-09T06:29:32.827743+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836723+0000', 'last_change': '2024-05-09T06:29:32.836723+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836723+0000', 'last_undegraded': '2024-05-09T06:29:32.836723+0000', 'last_fullsized': '2024-05-09T06:29:32.836723+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-05-09T06:29:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828834+0000', 'last_change': '2024-05-09T06:29:32.828834+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828834+0000', 'last_undegraded': '2024-05-09T06:29:32.828834+0000', 'last_fullsized': '2024-05-09T06:29:32.828834+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.826180+0000', 'last_change': '2024-05-09T06:29:32.826180+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.826180+0000', 'last_undegraded': '2024-05-09T06:29:32.826180+0000', 'last_fullsized': '2024-05-09T06:29:32.826180+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836265+0000', 'last_change': '2024-05-09T06:29:32.836265+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836265+0000', 'last_undegraded': '2024-05-09T06:29:32.836265+0000', 'last_fullsized': '2024-05-09T06:29:32.836265+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.830004+0000', 'last_change': '2024-05-09T06:29:32.830004+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.830004+0000', 'last_undegraded': '2024-05-09T06:29:32.830004+0000', 'last_fullsized': '2024-05-09T06:29:32.830004+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.835867+0000', 'last_change': '2024-05-09T06:29:32.835867+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.835867+0000', 'last_undegraded': '2024-05-09T06:29:32.835867+0000', 'last_fullsized': '2024-05-09T06:29:32.835867+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.837023+0000', 'last_change': '2024-05-09T06:29:32.837023+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.837023+0000', 'last_undegraded': '2024-05-09T06:29:32.837023+0000', 'last_fullsized': '2024-05-09T06:29:32.837023+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-05-09T06:29:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836127+0000', 'last_change': '2024-05-09T06:29:32.836127+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836127+0000', 'last_undegraded': '2024-05-09T06:29:32.836127+0000', 'last_fullsized': '2024-05-09T06:29:32.836127+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832922+0000', 'last_change': '2024-05-09T06:29:32.832922+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832922+0000', 'last_undegraded': '2024-05-09T06:29:32.832922+0000', 'last_fullsized': '2024-05-09T06:29:32.832922+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828814+0000', 'last_change': '2024-05-09T06:29:32.828814+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828814+0000', 'last_undegraded': '2024-05-09T06:29:32.828814+0000', 'last_fullsized': '2024-05-09T06:29:32.828814+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.835263+0000', 'last_change': '2024-05-09T06:29:32.835263+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.835263+0000', 'last_undegraded': '2024-05-09T06:29:32.835263+0000', 'last_fullsized': '2024-05-09T06:29:32.835263+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834730+0000', 'last_change': '2024-05-09T06:29:32.834730+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834730+0000', 'last_undegraded': '2024-05-09T06:29:32.834730+0000', 'last_fullsized': '2024-05-09T06:29:32.834730+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834843+0000', 'last_change': '2024-05-09T06:29:32.834843+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834843+0000', 'last_undegraded': '2024-05-09T06:29:32.834843+0000', 'last_fullsized': '2024-05-09T06:29:32.834843+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-05-09T06:29:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.826703+0000', 'last_change': '2024-05-09T06:29:32.826703+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.826703+0000', 'last_undegraded': '2024-05-09T06:29:32.826703+0000', 'last_fullsized': '2024-05-09T06:29:32.826703+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836464+0000', 'last_change': '2024-05-09T06:29:32.836464+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836464+0000', 'last_undegraded': '2024-05-09T06:29:32.836464+0000', 'last_fullsized': '2024-05-09T06:29:32.836464+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834001+0000', 'last_change': '2024-05-09T06:29:32.834001+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834001+0000', 'last_undegraded': '2024-05-09T06:29:32.834001+0000', 'last_fullsized': '2024-05-09T06:29:32.834001+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.829477+0000', 'last_change': '2024-05-09T06:29:32.829477+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.829477+0000', 'last_undegraded': '2024-05-09T06:29:32.829477+0000', 'last_fullsized': '2024-05-09T06:29:32.829477+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833492+0000', 'last_change': '2024-05-09T06:29:32.833492+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833492+0000', 'last_undegraded': '2024-05-09T06:29:32.833492+0000', 'last_fullsized': '2024-05-09T06:29:32.833492+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833542+0000', 'last_change': '2024-05-09T06:29:32.833542+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833542+0000', 'last_undegraded': '2024-05-09T06:29:32.833542+0000', 'last_fullsized': '2024-05-09T06:29:32.833542+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828830+0000', 'last_change': '2024-05-09T06:29:32.828830+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828830+0000', 'last_undegraded': '2024-05-09T06:29:32.828830+0000', 'last_fullsized': '2024-05-09T06:29:32.828830+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.835845+0000', 'last_change': '2024-05-09T06:29:32.835845+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.835845+0000', 'last_undegraded': '2024-05-09T06:29:32.835845+0000', 'last_fullsized': '2024-05-09T06:29:32.835845+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836568+0000', 'last_change': '2024-05-09T06:29:32.836568+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836568+0000', 'last_undegraded': '2024-05-09T06:29:32.836568+0000', 'last_fullsized': '2024-05-09T06:29:32.836568+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836686+0000', 'last_change': '2024-05-09T06:29:32.836686+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836686+0000', 'last_undegraded': '2024-05-09T06:29:32.836686+0000', 'last_fullsized': '2024-05-09T06:29:32.836686+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-05-09T06:29:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833767+0000', 'last_change': '2024-05-09T06:29:32.833767+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833767+0000', 'last_undegraded': '2024-05-09T06:29:32.833767+0000', 'last_fullsized': '2024-05-09T06:29:32.833767+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.828961+0000', 'last_change': '2024-05-09T06:29:32.828961+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.828961+0000', 'last_undegraded': '2024-05-09T06:29:32.828961+0000', 'last_fullsized': '2024-05-09T06:29:32.828961+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831887+0000', 'last_change': '2024-05-09T06:29:32.831887+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831887+0000', 'last_undegraded': '2024-05-09T06:29:32.831887+0000', 'last_fullsized': '2024-05-09T06:29:32.831887+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.829388+0000', 'last_change': '2024-05-09T06:29:32.829388+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.829388+0000', 'last_undegraded': '2024-05-09T06:29:32.829388+0000', 'last_fullsized': '2024-05-09T06:29:32.829388+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-05-09T06:29:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831210+0000', 'last_change': '2024-05-09T06:29:32.831210+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831210+0000', 'last_undegraded': '2024-05-09T06:29:32.831210+0000', 'last_fullsized': '2024-05-09T06:29:32.831210+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836258+0000', 'last_change': '2024-05-09T06:29:32.836258+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836258+0000', 'last_undegraded': '2024-05-09T06:29:32.836258+0000', 'last_fullsized': '2024-05-09T06:29:32.836258+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831892+0000', 'last_change': '2024-05-09T06:29:32.831892+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831892+0000', 'last_undegraded': '2024-05-09T06:29:32.831892+0000', 'last_fullsized': '2024-05-09T06:29:32.831892+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831210+0000', 'last_change': '2024-05-09T06:29:32.831210+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831210+0000', 'last_undegraded': '2024-05-09T06:29:32.831210+0000', 'last_fullsized': '2024-05-09T06:29:32.831210+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834060+0000', 'last_change': '2024-05-09T06:29:32.834060+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834060+0000', 'last_undegraded': '2024-05-09T06:29:32.834060+0000', 'last_fullsized': '2024-05-09T06:29:32.834060+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832434+0000', 'last_change': '2024-05-09T06:29:32.832434+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832434+0000', 'last_undegraded': '2024-05-09T06:29:32.832434+0000', 'last_fullsized': '2024-05-09T06:29:32.832434+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.834435+0000', 'last_change': '2024-05-09T06:29:32.834435+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.834435+0000', 'last_undegraded': '2024-05-09T06:29:32.834435+0000', 'last_fullsized': '2024-05-09T06:29:32.834435+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832901+0000', 'last_change': '2024-05-09T06:29:32.832901+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832901+0000', 'last_undegraded': '2024-05-09T06:29:32.832901+0000', 'last_fullsized': '2024-05-09T06:29:32.832901+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831995+0000', 'last_change': '2024-05-09T06:29:32.831995+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831995+0000', 'last_undegraded': '2024-05-09T06:29:32.831995+0000', 'last_fullsized': '2024-05-09T06:29:32.831995+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.825768+0000', 'last_change': '2024-05-09T06:29:32.825768+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.825768+0000', 'last_undegraded': '2024-05-09T06:29:32.825768+0000', 'last_fullsized': '2024-05-09T06:29:32.825768+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831451+0000', 'last_change': '2024-05-09T06:29:32.831451+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831451+0000', 'last_undegraded': '2024-05-09T06:29:32.831451+0000', 'last_fullsized': '2024-05-09T06:29:32.831451+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-05-09T06:29:38.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.827283+0000', 'last_change': '2024-05-09T06:29:32.827283+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.827283+0000', 'last_undegraded': '2024-05-09T06:29:32.827283+0000', 'last_fullsized': '2024-05-09T06:29:32.827283+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833611+0000', 'last_change': '2024-05-09T06:29:32.833611+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833611+0000', 'last_undegraded': '2024-05-09T06:29:32.833611+0000', 'last_fullsized': '2024-05-09T06:29:32.833611+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836148+0000', 'last_change': '2024-05-09T06:29:32.836148+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836148+0000', 'last_undegraded': '2024-05-09T06:29:32.836148+0000', 'last_fullsized': '2024-05-09T06:29:32.836148+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.829862+0000', 'last_change': '2024-05-09T06:29:32.829862+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.829862+0000', 'last_undegraded': '2024-05-09T06:29:32.829862+0000', 'last_fullsized': '2024-05-09T06:29:32.829862+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.826694+0000', 'last_change': '2024-05-09T06:29:32.826694+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.826694+0000', 'last_undegraded': '2024-05-09T06:29:32.826694+0000', 'last_fullsized': '2024-05-09T06:29:32.826694+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833045+0000', 'last_change': '2024-05-09T06:29:32.833045+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833045+0000', 'last_undegraded': '2024-05-09T06:29:32.833045+0000', 'last_fullsized': '2024-05-09T06:29:32.833045+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.825813+0000', 'last_change': '2024-05-09T06:29:32.825813+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.825813+0000', 'last_undegraded': '2024-05-09T06:29:32.825813+0000', 'last_fullsized': '2024-05-09T06:29:32.825813+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.836215+0000', 'last_change': '2024-05-09T06:29:32.836215+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.836215+0000', 'last_undegraded': '2024-05-09T06:29:32.836215+0000', 'last_fullsized': '2024-05-09T06:29:32.836215+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.830680+0000', 'last_change': '2024-05-09T06:29:32.830680+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.830680+0000', 'last_undegraded': '2024-05-09T06:29:32.830680+0000', 'last_fullsized': '2024-05-09T06:29:32.830680+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.832731+0000', 'last_change': '2024-05-09T06:29:32.832731+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.832731+0000', 'last_undegraded': '2024-05-09T06:29:32.832731+0000', 'last_fullsized': '2024-05-09T06:29:32.832731+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.831387+0000', 'last_change': '2024-05-09T06:29:32.831387+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.831387+0000', 'last_undegraded': '2024-05-09T06:29:32.831387+0000', 'last_fullsized': '2024-05-09T06:29:32.831387+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-05-09T06:29:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.833783+0000', 'last_change': '2024-05-09T06:29:32.833783+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.833783+0000', 'last_undegraded': '2024-05-09T06:29:32.833783+0000', 'last_fullsized': '2024-05-09T06:29:32.833783+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:29:38.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-05-09T06:29:38.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:29:32.835859+0000', 'last_change': '2024-05-09T06:29:32.835859+0000', 'last_active': '2024-05-09T06:29:32.820766+0000', 'last_peered': '2024-05-09T06:29:32.820766+0000', 'last_clean': '2024-05-09T06:29:32.820766+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:29:32.835859+0000', 'last_undegraded': '2024-05-09T06:29:32.835859+0000', 'last_fullsized': '2024-05-09T06:29:32.835859+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'last_clean_scrub_stamp': '2024-05-09T06:29:32.820766+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:29:38.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-09T06:29:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.742+0000 7f65ad709640 1 Processor -- start 2024-05-09T06:29:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.742+0000 7f65ad709640 1 -- start start 2024-05-09T06:29:38.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65ad709640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f65a815e1d0 0x7f65a806f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65ad709640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65a815e5b0 0x7f65a806fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65ad709640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a806c2f0 con 0x7f65a80d2890 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65ad709640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80cc3c0 con 0x7f65a815e5b0 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65ad709640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80ccc60 con 0x7f65a815e1d0 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f659e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65a815e5b0 0x7f65a806fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65a77fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f65a80d2890 0x7f65a80cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38910/0 (socket says 172.21.15.1:38910) 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65a77fe640 1 -- 172.21.15.1:0/1442280270 learned_addr learned my addr 172.21.15.1:0/1442280270 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65a6ffd640 1 --2- 172.21.15.1:0/1442280270 >> v2:172.21.15.6:3300/0 conn(0x7f65a815e1d0 0x7f65a806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65a67fc640 1 -- 172.21.15.1:0/1442280270 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181373014 0 0) 0x7f65a0001490 con 0x7f65a80d2890 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f65a67fc640 1 -- 172.21.15.1:0/1442280270 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6594003520 con 0x7f65a80d2890 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f659e7fc640 1 -- 172.21.15.1:0/1442280270 >> v2:172.21.15.6:3300/0 conn(0x7f65a815e1d0 msgr2=0x7f65a806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f659e7fc640 1 --2- 172.21.15.1:0/1442280270 >> v2:172.21.15.6:3300/0 conn(0x7f65a815e1d0 0x7f65a806f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f659e7fc640 1 -- 172.21.15.1:0/1442280270 >> v1:172.21.15.1:6789/0 conn(0x7f65a80d2890 legacy=0x7f65a80cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.743+0000 7f659e7fc640 1 -- 172.21.15.1:0/1442280270 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a806f500 con 0x7f65a815e5b0 2024-05-09T06:29:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.744+0000 7f659e7fc640 1 --2- 172.21.15.1:0/1442280270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a806fdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f65a80704a0 tx=0x7f659002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd374a353d1cf32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.744+0000 7f65a67fc640 1 -- 172.21.15.1:0/1442280270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590037070 con 0x7f65a815e5b0 2024-05-09T06:29:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.744+0000 7f65a67fc640 1 -- 172.21.15.1:0/1442280270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590032440 con 0x7f65a815e5b0 2024-05-09T06:29:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.744+0000 7f65a67fc640 1 -- 172.21.15.1:0/1442280270 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659002ae60 con 0x7f65a815e5b0 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 -- 172.21.15.1:0/1442280270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 msgr2=0x7f65a806fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 --2- 172.21.15.1:0/1442280270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a806fdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f65a80704a0 tx=0x7f659002a110 comp rx=0 tx=0).stop 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 -- 172.21.15.1:0/1442280270 shutdown_connections 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 --2- 172.21.15.1:0/1442280270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a806fdd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 --2- 172.21.15.1:0/1442280270 >> v2:172.21.15.6:3300/0 conn(0x7f65a815e1d0 0x7f65a806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 -- 172.21.15.1:0/1442280270 >> 172.21.15.1:0/1442280270 conn(0x7f65a80d1d70 msgr2=0x7f65a80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 -- 172.21.15.1:0/1442280270 shutdown_connections 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.745+0000 7f65ad709640 1 -- 172.21.15.1:0/1442280270 wait complete. 2024-05-09T06:29:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.746+0000 7f65ad709640 1 Processor -- start 2024-05-09T06:29:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.746+0000 7f65ad709640 1 -- start start 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65a80d2890 0x7f65a81f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65a815e1d0 0x7f65a815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65a806f1b0 con 0x7f65a815e5b0 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80cc8f0 con 0x7f65a815e1d0 2024-05-09T06:29:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65ad709640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f65a80d0fc0 con 0x7f65a80d2890 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40214/0 (socket says 172.21.15.1:40214) 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 -- 172.21.15.1:0/3907673476 learned_addr learned my addr 172.21.15.1:0/3907673476 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a6ffd640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65a80d2890 0x7f65a81f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f659e7fc640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65a815e1d0 0x7f65a815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 -- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65a80d2890 msgr2=0x7f65a81f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65a80d2890 0x7f65a81f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 -- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65a815e1d0 msgr2=0x7f65a815f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65a815e1d0 0x7f65a815f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 -- 172.21.15.1:0/3907673476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a81522d0 con 0x7f65a815e5b0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f65a77fe640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f65a000cb40 tx=0x7f65a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.747+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0017070 con 0x7f65a815e5b0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.748+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a000ac40 con 0x7f65a815e5b0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.748+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a80c8fe0 con 0x7f65a815e5b0 2024-05-09T06:29:38.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.748+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0012400 con 0x7f65a815e5b0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.748+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a80c95a0 con 0x7f65a815e5b0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.749+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a80752b0 con 0x7f65a815e5b0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.749+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65a001a070 con 0x7f65a815e5b0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.750+0000 7f659ffff640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.750+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f65a008abe0 con 0x7f65a815e5b0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.750+0000 7f65a6ffd640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.750+0000 7f65a6ffd640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 0x7f658806e320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6598009ad0 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:38.754+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65a806c2f0 con 0x7f65a815e5b0 2024-05-09T06:29:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.006+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f65a80cdfa0 con 0x7f65a815e5b0 2024-05-09T06:29:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.007+0000 7f659ffff640 1 -- 172.21.15.1:0/3907673476 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1582 (secure 0 0 0) 0x7f65a0058db0 con 0x7f65a815e5b0 2024-05-09T06:29:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:38.989 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1715236027,"num_in_osds":8,"osd_in_since":1715236019,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":244150272,"bytes_avail":772849963008,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6045}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-05-09T06:29:15.092088+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 msgr2=0x7f658806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 0x7f658806e320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6598009ad0 tx=0x7f6598008040 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 msgr2=0x7f65a80ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f65a000cb40 tx=0x7f65a000a060 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 shutdown_connections 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65a815e5b0 0x7f65a80ca080 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65a815e1d0 0x7f65a815f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 --2- 172.21.15.1:0/3907673476 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65a80d2890 0x7f65a81f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 >> 172.21.15.1:0/3907673476 conn(0x7f65a80d1d70 msgr2=0x7f65a80d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 shutdown_connections 2024-05-09T06:29:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.009+0000 7f65ad709640 1 -- 172.21.15.1:0/3907673476 wait complete. 2024-05-09T06:29:39.002 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-09T06:29:39.002 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:29:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.125+0000 7f5c1937e640 1 Processor -- start 2024-05-09T06:29:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.125+0000 7f5c1937e640 1 -- start start 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c1937e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5c1415e1d0 0x7f5c1406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c1937e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140d2890 0x7f5c140d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c1937e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1406c2f0 con 0x7f5c1415e5b0 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c1937e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5c140cc3c0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c1937e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c140ccc60 con 0x7f5c140d2890 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5c1415e1d0 0x7f5c1406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5c1415e1d0 0x7f5c1406f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40222/0 (socket says 172.21.15.1:40222) 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 -- 172.21.15.1:0/1406163243 learned_addr learned my addr 172.21.15.1:0/1406163243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c137fe640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140d2890 0x7f5c140d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 -- 172.21.15.1:0/1406163243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140d2890 msgr2=0x7f5c140d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140d2890 0x7f5c140d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 -- 172.21.15.1:0/1406163243 >> v1:172.21.15.6:6789/0 conn(0x7f5c1415e5b0 legacy=0x7f5c140cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.126+0000 7f5c12ffd640 1 -- 172.21.15.1:0/1406163243 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1406f500 con 0x7f5c1415e1d0 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c12ffd640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1406f890 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009870 tx=0x7f5c0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854686799d355fdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c127fc640 1 -- 172.21.15.1:0/1406163243 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04037070 con 0x7f5c1415e1d0 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c127fc640 1 -- 172.21.15.1:0/1406163243 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c04032440 con 0x7f5c1415e1d0 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c127fc640 1 -- 172.21.15.1:0/1406163243 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04032810 con 0x7f5c1415e1d0 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c1937e640 1 -- 172.21.15.1:0/1406163243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 msgr2=0x7f5c1406f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.127+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1406f890 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009870 tx=0x7f5c0402a180 comp rx=0 tx=0).stop 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 -- 172.21.15.1:0/1406163243 shutdown_connections 2024-05-09T06:29:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140d2890 0x7f5c140d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1406163243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1406f890 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 -- 172.21.15.1:0/1406163243 >> 172.21.15.1:0/1406163243 conn(0x7f5c140d1d70 msgr2=0x7f5c140d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 -- 172.21.15.1:0/1406163243 shutdown_connections 2024-05-09T06:29:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.128+0000 7f5c1937e640 1 -- 172.21.15.1:0/1406163243 wait complete. 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.129+0000 7f5c1937e640 1 Processor -- start 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.129+0000 7f5c1937e640 1 -- start start 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.129+0000 7f5c1937e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140d2890 0x7f5c1414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c1415e5b0 0x7f5c1409eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406f1b0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c140cc8f0 con 0x7f5c140d2890 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1406fdd0 con 0x7f5c1415e5b0 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40238/0 (socket says 172.21.15.1:40238) 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 -- 172.21.15.1:0/1276198289 learned_addr learned my addr 172.21.15.1:0/1276198289 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c12ffd640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140d2890 0x7f5c1414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 -- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c1415e5b0 msgr2=0x7f5c1409eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c137fe640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c1415e5b0 0x7f5c1409eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c1415e5b0 0x7f5c1409eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 -- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140d2890 msgr2=0x7f5c1414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140d2890 0x7f5c1414fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 -- 172.21.15.1:0/1276198289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c141502e0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0bfff640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b3c0 tx=0x7f5bfc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc011830 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc016070 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c141506f0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc011c10 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.130+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c14150cb0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.131+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c140b13a0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.132+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bfc01b460 con 0x7f5c1415e1d0 2024-05-09T06:29:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.132+0000 7f5c0b7fe640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 0x7f5bf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.132+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f5bfc08ab20 con 0x7f5c1415e1d0 2024-05-09T06:29:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.132+0000 7f5c12ffd640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 0x7f5bf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.133+0000 7f5c12ffd640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 0x7f5bf406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c04034000 tx=0x7f5c0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.137+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c140b13a0 con 0x7f5c1415e1d0 2024-05-09T06:29:39.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.315+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c140cbdc0 con 0x7f5bf406be30 2024-05-09T06:29:39.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.321+0000 7f5c0b7fe640 1 -- 172.21.15.1:0/1276198289 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f5c0403c460 con 0x7f5bf406be30 2024-05-09T06:29:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:39.310 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:29:39.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 msgr2=0x7f5bf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 0x7f5bf406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c04034000 tx=0x7f5c0402e000 comp rx=0 tx=0).stop 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 msgr2=0x7f5c1409e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b3c0 tx=0x7f5bfc00b880 comp rx=0 tx=0).stop 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 shutdown_connections 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf406be30 0x7f5bf406e2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c1415e5b0 0x7f5c1409eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c1415e1d0 0x7f5c1409e5d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 --2- 172.21.15.1:0/1276198289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140d2890 0x7f5c1414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 >> 172.21.15.1:0/1276198289 conn(0x7f5c140d1d70 msgr2=0x7f5c1406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:39.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 shutdown_connections 2024-05-09T06:29:39.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:39.324+0000 7f5c1937e640 1 -- 172.21.15.1:0/1276198289 wait complete. 2024-05-09T06:29:39.317 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-05-09T06:29:39.096301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238780,"kb_used_data":5960,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735940,"statfs":{"total":773094113280,"available":772849602560,"internally_reserved":0,"allocated":6103040,"data_stored":3348116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.272588"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154204+0000","last_change":"2024-05-09T06:29:33.859341+0000","last_active":"2024-05-09T06:29:37.154204+0000","last_peered":"2024-05-09T06:29:37.154204+0000","last_clean":"2024-05-09T06:29:37.154204+0000","last_became_active":"2024-05-09T06:29:33.859037+0000","last_became_peered":"2024-05-09T06:29:33.859037+0000","last_unstale":"2024-05-09T06:29:37.154204+0000","last_undegraded":"2024-05-09T06:29:37.154204+0000","last_fullsized":"2024-05-09T06:29:37.154204+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:49.415506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847402+0000","last_change":"2024-05-09T06:29:33.855717+0000","last_active":"2024-05-09T06:29:35.847402+0000","last_peered":"2024-05-09T06:29:35.847402+0000","last_clean":"2024-05-09T06:29:35.847402+0000","last_became_active":"2024-05-09T06:29:33.855439+0000","last_became_peered":"2024-05-09T06:29:33.855439+0000","last_unstale":"2024-05-09T06:29:35.847402+0000","last_undegraded":"2024-05-09T06:29:35.847402+0000","last_fullsized":"2024-05-09T06:29:35.847402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476298+0000","last_change":"2024-05-09T06:29:33.856794+0000","last_active":"2024-05-09T06:29:35.476298+0000","last_peered":"2024-05-09T06:29:35.476298+0000","last_clean":"2024-05-09T06:29:35.476298+0000","last_became_active":"2024-05-09T06:29:33.854541+0000","last_became_peered":"2024-05-09T06:29:33.854541+0000","last_unstale":"2024-05-09T06:29:35.476298+0000","last_undegraded":"2024-05-09T06:29:35.476298+0000","last_fullsized":"2024-05-09T06:29:35.476298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:31:33.577516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847161+0000","last_change":"2024-05-09T06:29:33.852260+0000","last_active":"2024-05-09T06:29:35.847161+0000","last_peered":"2024-05-09T06:29:35.847161+0000","last_clean":"2024-05-09T06:29:35.847161+0000","last_became_active":"2024-05-09T06:29:33.851504+0000","last_became_peered":"2024-05-09T06:29:33.851504+0000","last_unstale":"2024-05-09T06:29:35.847161+0000","last_undegraded":"2024-05-09T06:29:35.847161+0000","last_fullsized":"2024-05-09T06:29:35.847161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843697+0000","last_change":"2024-05-09T06:29:33.848187+0000","last_active":"2024-05-09T06:29:34.843697+0000","last_peered":"2024-05-09T06:29:34.843697+0000","last_clean":"2024-05-09T06:29:34.843697+0000","last_became_active":"2024-05-09T06:29:33.847863+0000","last_became_peered":"2024-05-09T06:29:33.847863+0000","last_unstale":"2024-05-09T06:29:34.843697+0000","last_undegraded":"2024-05-09T06:29:34.843697+0000","last_fullsized":"2024-05-09T06:29:34.843697+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:12.008315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153922+0000","last_change":"2024-05-09T06:29:33.858538+0000","last_active":"2024-05-09T06:29:37.153922+0000","last_peered":"2024-05-09T06:29:37.153922+0000","last_clean":"2024-05-09T06:29:37.153922+0000","last_became_active":"2024-05-09T06:29:33.858226+0000","last_became_peered":"2024-05-09T06:29:33.858226+0000","last_unstale":"2024-05-09T06:29:37.153922+0000","last_undegraded":"2024-05-09T06:29:37.153922+0000","last_fullsized":"2024-05-09T06:29:37.153922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:50.081220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475888+0000","last_change":"2024-05-09T06:29:33.868884+0000","last_active":"2024-05-09T06:29:35.475888+0000","last_peered":"2024-05-09T06:29:35.475888+0000","last_clean":"2024-05-09T06:29:35.475888+0000","last_became_active":"2024-05-09T06:29:33.868607+0000","last_became_peered":"2024-05-09T06:29:33.868607+0000","last_unstale":"2024-05-09T06:29:35.475888+0000","last_undegraded":"2024-05-09T06:29:35.475888+0000","last_fullsized":"2024-05-09T06:29:35.475888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:37.905161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154183+0000","last_change":"2024-05-09T06:29:33.861341+0000","last_active":"2024-05-09T06:29:37.154183+0000","last_peered":"2024-05-09T06:29:37.154183+0000","last_clean":"2024-05-09T06:29:37.154183+0000","last_became_active":"2024-05-09T06:29:33.861035+0000","last_became_peered":"2024-05-09T06:29:33.861035+0000","last_unstale":"2024-05-09T06:29:37.154183+0000","last_undegraded":"2024-05-09T06:29:37.154183+0000","last_fullsized":"2024-05-09T06:29:37.154183+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:44.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154916+0000","last_change":"2024-05-09T06:29:33.863838+0000","last_active":"2024-05-09T06:29:37.154916+0000","last_peered":"2024-05-09T06:29:37.154916+0000","last_clean":"2024-05-09T06:29:37.154916+0000","last_became_active":"2024-05-09T06:29:33.863322+0000","last_became_peered":"2024-05-09T06:29:33.863322+0000","last_unstale":"2024-05-09T06:29:37.154916+0000","last_undegraded":"2024-05-09T06:29:37.154916+0000","last_fullsized":"2024-05-09T06:29:37.154916+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:10.424586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154240+0000","last_change":"2024-05-09T06:29:33.854285+0000","last_active":"2024-05-09T06:29:37.154240+0000","last_peered":"2024-05-09T06:29:37.154240+0000","last_clean":"2024-05-09T06:29:37.154240+0000","last_became_active":"2024-05-09T06:29:33.853949+0000","last_became_peered":"2024-05-09T06:29:33.853949+0000","last_unstale":"2024-05-09T06:29:37.154240+0000","last_undegraded":"2024-05-09T06:29:37.154240+0000","last_fullsized":"2024-05-09T06:29:37.154240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.842011+0000","last_change":"2024-05-09T06:29:33.859806+0000","last_active":"2024-05-09T06:29:34.842011+0000","last_peered":"2024-05-09T06:29:34.842011+0000","last_clean":"2024-05-09T06:29:34.842011+0000","last_became_active":"2024-05-09T06:29:33.859531+0000","last_became_peered":"2024-05-09T06:29:33.859531+0000","last_unstale":"2024-05-09T06:29:34.842011+0000","last_undegraded":"2024-05-09T06:29:34.842011+0000","last_fullsized":"2024-05-09T06:29:34.842011+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:32.155052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154777+0000","last_change":"2024-05-09T06:29:33.850683+0000","last_active":"2024-05-09T06:29:37.154777+0000","last_peered":"2024-05-09T06:29:37.154777+0000","last_clean":"2024-05-09T06:29:37.154777+0000","last_became_active":"2024-05-09T06:29:33.850373+0000","last_became_peered":"2024-05-09T06:29:33.850373+0000","last_unstale":"2024-05-09T06:29:37.154777+0000","last_undegraded":"2024-05-09T06:29:37.154777+0000","last_fullsized":"2024-05-09T06:29:37.154777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:04.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153789+0000","last_change":"2024-05-09T06:29:33.855174+0000","last_active":"2024-05-09T06:29:37.153789+0000","last_peered":"2024-05-09T06:29:37.153789+0000","last_clean":"2024-05-09T06:29:37.153789+0000","last_became_active":"2024-05-09T06:29:33.854849+0000","last_became_peered":"2024-05-09T06:29:33.854849+0000","last_unstale":"2024-05-09T06:29:37.153789+0000","last_undegraded":"2024-05-09T06:29:37.153789+0000","last_fullsized":"2024-05-09T06:29:37.153789+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:22.199322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476329+0000","last_change":"2024-05-09T06:29:33.853018+0000","last_active":"2024-05-09T06:29:35.476329+0000","last_peered":"2024-05-09T06:29:35.476329+0000","last_clean":"2024-05-09T06:29:35.476329+0000","last_became_active":"2024-05-09T06:29:33.852724+0000","last_became_peered":"2024-05-09T06:29:33.852724+0000","last_unstale":"2024-05-09T06:29:35.476329+0000","last_undegraded":"2024-05-09T06:29:35.476329+0000","last_fullsized":"2024-05-09T06:29:35.476329+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:19.792349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846985+0000","last_change":"2024-05-09T06:29:33.851034+0000","last_active":"2024-05-09T06:29:35.846985+0000","last_peered":"2024-05-09T06:29:35.846985+0000","last_clean":"2024-05-09T06:29:35.846985+0000","last_became_active":"2024-05-09T06:29:33.850701+0000","last_became_peered":"2024-05-09T06:29:33.850701+0000","last_unstale":"2024-05-09T06:29:35.846985+0000","last_undegraded":"2024-05-09T06:29:35.846985+0000","last_fullsized":"2024-05-09T06:29:35.846985+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:09.864980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153718+0000","last_change":"2024-05-09T06:29:33.855894+0000","last_active":"2024-05-09T06:29:37.153718+0000","last_peered":"2024-05-09T06:29:37.153718+0000","last_clean":"2024-05-09T06:29:37.153718+0000","last_became_active":"2024-05-09T06:29:33.855618+0000","last_became_peered":"2024-05-09T06:29:33.855618+0000","last_unstale":"2024-05-09T06:29:37.153718+0000","last_undegraded":"2024-05-09T06:29:37.153718+0000","last_fullsized":"2024-05-09T06:29:37.153718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:11.434050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476531+0000","last_change":"2024-05-09T06:29:33.862175+0000","last_active":"2024-05-09T06:29:35.476531+0000","last_peered":"2024-05-09T06:29:35.476531+0000","last_clean":"2024-05-09T06:29:35.476531+0000","last_became_active":"2024-05-09T06:29:33.861447+0000","last_became_peered":"2024-05-09T06:29:33.861447+0000","last_unstale":"2024-05-09T06:29:35.476531+0000","last_undegraded":"2024-05-09T06:29:35.476531+0000","last_fullsized":"2024-05-09T06:29:35.476531+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:25.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847496+0000","last_change":"2024-05-09T06:29:33.857676+0000","last_active":"2024-05-09T06:29:35.847496+0000","last_peered":"2024-05-09T06:29:35.847496+0000","last_clean":"2024-05-09T06:29:35.847496+0000","last_became_active":"2024-05-09T06:29:33.857359+0000","last_became_peered":"2024-05-09T06:29:33.857359+0000","last_unstale":"2024-05-09T06:29:35.847496+0000","last_undegraded":"2024-05-09T06:29:35.847496+0000","last_fullsized":"2024-05-09T06:29:35.847496+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:23.972099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476303+0000","last_change":"2024-05-09T06:29:33.863734+0000","last_active":"2024-05-09T06:29:35.476303+0000","last_peered":"2024-05-09T06:29:35.476303+0000","last_clean":"2024-05-09T06:29:35.476303+0000","last_became_active":"2024-05-09T06:29:33.863237+0000","last_became_peered":"2024-05-09T06:29:33.863237+0000","last_unstale":"2024-05-09T06:29:35.476303+0000","last_undegraded":"2024-05-09T06:29:35.476303+0000","last_fullsized":"2024-05-09T06:29:35.476303+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:18.007333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624319+0000","last_change":"2024-05-09T06:29:33.851658+0000","last_active":"2024-05-09T06:29:35.624319+0000","last_peered":"2024-05-09T06:29:35.624319+0000","last_clean":"2024-05-09T06:29:35.624319+0000","last_became_active":"2024-05-09T06:29:33.851386+0000","last_became_peered":"2024-05-09T06:29:33.851386+0000","last_unstale":"2024-05-09T06:29:35.624319+0000","last_undegraded":"2024-05-09T06:29:35.624319+0000","last_fullsized":"2024-05-09T06:29:35.624319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:47.385123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624177+0000","last_change":"2024-05-09T06:29:33.850971+0000","last_active":"2024-05-09T06:29:35.624177+0000","last_peered":"2024-05-09T06:29:35.624177+0000","last_clean":"2024-05-09T06:29:35.624177+0000","last_became_active":"2024-05-09T06:29:33.850661+0000","last_became_peered":"2024-05-09T06:29:33.850661+0000","last_unstale":"2024-05-09T06:29:35.624177+0000","last_undegraded":"2024-05-09T06:29:35.624177+0000","last_fullsized":"2024-05-09T06:29:35.624177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:11.434050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154412+0000","last_change":"2024-05-09T06:29:33.855891+0000","last_active":"2024-05-09T06:29:37.154412+0000","last_peered":"2024-05-09T06:29:37.154412+0000","last_clean":"2024-05-09T06:29:37.154412+0000","last_became_active":"2024-05-09T06:29:33.855496+0000","last_became_peered":"2024-05-09T06:29:33.855496+0000","last_unstale":"2024-05-09T06:29:37.154412+0000","last_undegraded":"2024-05-09T06:29:37.154412+0000","last_fullsized":"2024-05-09T06:29:37.154412+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.979878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841734+0000","last_change":"2024-05-09T06:29:33.859907+0000","last_active":"2024-05-09T06:29:34.841734+0000","last_peered":"2024-05-09T06:29:34.841734+0000","last_clean":"2024-05-09T06:29:34.841734+0000","last_became_active":"2024-05-09T06:29:33.859630+0000","last_became_peered":"2024-05-09T06:29:33.859630+0000","last_unstale":"2024-05-09T06:29:34.841734+0000","last_undegraded":"2024-05-09T06:29:34.841734+0000","last_fullsized":"2024-05-09T06:29:34.841734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:55.300623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475810+0000","last_change":"2024-05-09T06:29:33.868187+0000","last_active":"2024-05-09T06:29:35.475810+0000","last_peered":"2024-05-09T06:29:35.475810+0000","last_clean":"2024-05-09T06:29:35.475810+0000","last_became_active":"2024-05-09T06:29:33.867033+0000","last_became_peered":"2024-05-09T06:29:33.867033+0000","last_unstale":"2024-05-09T06:29:35.475810+0000","last_undegraded":"2024-05-09T06:29:35.475810+0000","last_fullsized":"2024-05-09T06:29:35.475810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:41.475551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476676+0000","last_change":"2024-05-09T06:29:33.858635+0000","last_active":"2024-05-09T06:29:35.476676+0000","last_peered":"2024-05-09T06:29:35.476676+0000","last_clean":"2024-05-09T06:29:35.476676+0000","last_became_active":"2024-05-09T06:29:33.858357+0000","last_became_peered":"2024-05-09T06:29:33.858357+0000","last_unstale":"2024-05-09T06:29:35.476676+0000","last_undegraded":"2024-05-09T06:29:35.476676+0000","last_fullsized":"2024-05-09T06:29:35.476676+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:10.424586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154807+0000","last_change":"2024-05-09T06:29:33.856819+0000","last_active":"2024-05-09T06:29:37.154807+0000","last_peered":"2024-05-09T06:29:37.154807+0000","last_clean":"2024-05-09T06:29:37.154807+0000","last_became_active":"2024-05-09T06:29:33.856519+0000","last_became_peered":"2024-05-09T06:29:33.856519+0000","last_unstale":"2024-05-09T06:29:37.154807+0000","last_undegraded":"2024-05-09T06:29:37.154807+0000","last_fullsized":"2024-05-09T06:29:37.154807+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:25.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154743+0000","last_change":"2024-05-09T06:29:33.857606+0000","last_active":"2024-05-09T06:29:37.154743+0000","last_peered":"2024-05-09T06:29:37.154743+0000","last_clean":"2024-05-09T06:29:37.154743+0000","last_became_active":"2024-05-09T06:29:33.857308+0000","last_became_peered":"2024-05-09T06:29:33.857308+0000","last_unstale":"2024-05-09T06:29:37.154743+0000","last_undegraded":"2024-05-09T06:29:37.154743+0000","last_fullsized":"2024-05-09T06:29:37.154743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624445+0000","last_change":"2024-05-09T06:29:33.866435+0000","last_active":"2024-05-09T06:29:35.624445+0000","last_peered":"2024-05-09T06:29:35.624445+0000","last_clean":"2024-05-09T06:29:35.624445+0000","last_became_active":"2024-05-09T06:29:33.866180+0000","last_became_peered":"2024-05-09T06:29:33.866180+0000","last_unstale":"2024-05-09T06:29:35.624445+0000","last_undegraded":"2024-05-09T06:29:35.624445+0000","last_fullsized":"2024-05-09T06:29:35.624445+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476173+0000","last_change":"2024-05-09T06:29:33.854059+0000","last_active":"2024-05-09T06:29:35.476173+0000","last_peered":"2024-05-09T06:29:35.476173+0000","last_clean":"2024-05-09T06:29:35.476173+0000","last_became_active":"2024-05-09T06:29:33.853274+0000","last_became_peered":"2024-05-09T06:29:33.853274+0000","last_unstale":"2024-05-09T06:29:35.476173+0000","last_undegraded":"2024-05-09T06:29:35.476173+0000","last_fullsized":"2024-05-09T06:29:35.476173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:02.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847083+0000","last_change":"2024-05-09T06:29:33.859781+0000","last_active":"2024-05-09T06:29:35.847083+0000","last_peered":"2024-05-09T06:29:35.847083+0000","last_clean":"2024-05-09T06:29:35.847083+0000","last_became_active":"2024-05-09T06:29:33.859387+0000","last_became_peered":"2024-05-09T06:29:33.859387+0000","last_unstale":"2024-05-09T06:29:35.847083+0000","last_undegraded":"2024-05-09T06:29:35.847083+0000","last_fullsized":"2024-05-09T06:29:35.847083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:49.415506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475911+0000","last_change":"2024-05-09T06:29:33.854017+0000","last_active":"2024-05-09T06:29:35.475911+0000","last_peered":"2024-05-09T06:29:35.475911+0000","last_clean":"2024-05-09T06:29:35.475911+0000","last_became_active":"2024-05-09T06:29:33.853732+0000","last_became_peered":"2024-05-09T06:29:33.853732+0000","last_unstale":"2024-05-09T06:29:35.475911+0000","last_undegraded":"2024-05-09T06:29:35.475911+0000","last_fullsized":"2024-05-09T06:29:35.475911+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:50.510790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154365+0000","last_change":"2024-05-09T06:29:33.850569+0000","last_active":"2024-05-09T06:29:37.154365+0000","last_peered":"2024-05-09T06:29:37.154365+0000","last_clean":"2024-05-09T06:29:37.154365+0000","last_became_active":"2024-05-09T06:29:33.849787+0000","last_became_peered":"2024-05-09T06:29:33.849787+0000","last_unstale":"2024-05-09T06:29:37.154365+0000","last_undegraded":"2024-05-09T06:29:37.154365+0000","last_fullsized":"2024-05-09T06:29:37.154365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846810+0000","last_change":"2024-05-09T06:29:33.851022+0000","last_active":"2024-05-09T06:29:35.846810+0000","last_peered":"2024-05-09T06:29:35.846810+0000","last_clean":"2024-05-09T06:29:35.846810+0000","last_became_active":"2024-05-09T06:29:33.850725+0000","last_became_peered":"2024-05-09T06:29:33.850725+0000","last_unstale":"2024-05-09T06:29:35.846810+0000","last_undegraded":"2024-05-09T06:29:35.846810+0000","last_fullsized":"2024-05-09T06:29:35.846810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624345+0000","last_change":"2024-05-09T06:29:32.843136+0000","last_active":"2024-05-09T06:29:35.624345+0000","last_peered":"2024-05-09T06:29:35.624345+0000","last_clean":"2024-05-09T06:29:35.624345+0000","last_became_active":"2024-05-09T06:29:32.842842+0000","last_became_peered":"2024-05-09T06:29:32.842842+0000","last_unstale":"2024-05-09T06:29:35.624345+0000","last_undegraded":"2024-05-09T06:29:35.624345+0000","last_fullsized":"2024-05-09T06:29:35.624345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154274+0000","last_change":"2024-05-09T06:29:33.855731+0000","last_active":"2024-05-09T06:29:37.154274+0000","last_peered":"2024-05-09T06:29:37.154274+0000","last_clean":"2024-05-09T06:29:37.154274+0000","last_became_active":"2024-05-09T06:29:33.854105+0000","last_became_peered":"2024-05-09T06:29:33.854105+0000","last_unstale":"2024-05-09T06:29:37.154274+0000","last_undegraded":"2024-05-09T06:29:37.154274+0000","last_fullsized":"2024-05-09T06:29:37.154274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154214+0000","last_change":"2024-05-09T06:29:32.842035+0000","last_active":"2024-05-09T06:29:37.154214+0000","last_peered":"2024-05-09T06:29:37.154214+0000","last_clean":"2024-05-09T06:29:37.154214+0000","last_became_active":"2024-05-09T06:29:32.841756+0000","last_became_peered":"2024-05-09T06:29:32.841756+0000","last_unstale":"2024-05-09T06:29:37.154214+0000","last_undegraded":"2024-05-09T06:29:37.154214+0000","last_fullsized":"2024-05-09T06:29:37.154214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624279+0000","last_change":"2024-05-09T06:29:33.851242+0000","last_active":"2024-05-09T06:29:35.624279+0000","last_peered":"2024-05-09T06:29:35.624279+0000","last_clean":"2024-05-09T06:29:35.624279+0000","last_became_active":"2024-05-09T06:29:33.850868+0000","last_became_peered":"2024-05-09T06:29:33.850868+0000","last_unstale":"2024-05-09T06:29:35.624279+0000","last_undegraded":"2024-05-09T06:29:35.624279+0000","last_fullsized":"2024-05-09T06:29:35.624279+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:50.081220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154698+0000","last_change":"2024-05-09T06:29:32.842788+0000","last_active":"2024-05-09T06:29:37.154698+0000","last_peered":"2024-05-09T06:29:37.154698+0000","last_clean":"2024-05-09T06:29:37.154698+0000","last_became_active":"2024-05-09T06:29:32.842461+0000","last_became_peered":"2024-05-09T06:29:32.842461+0000","last_unstale":"2024-05-09T06:29:37.154698+0000","last_undegraded":"2024-05-09T06:29:37.154698+0000","last_fullsized":"2024-05-09T06:29:37.154698+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153999+0000","last_change":"2024-05-09T06:29:33.856526+0000","last_active":"2024-05-09T06:29:37.153999+0000","last_peered":"2024-05-09T06:29:37.153999+0000","last_clean":"2024-05-09T06:29:37.153999+0000","last_became_active":"2024-05-09T06:29:33.855880+0000","last_became_peered":"2024-05-09T06:29:33.855880+0000","last_unstale":"2024-05-09T06:29:37.153999+0000","last_undegraded":"2024-05-09T06:29:37.153999+0000","last_fullsized":"2024-05-09T06:29:37.153999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:18.007333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154570+0000","last_change":"2024-05-09T06:29:32.845755+0000","last_active":"2024-05-09T06:29:37.154570+0000","last_peered":"2024-05-09T06:29:37.154570+0000","last_clean":"2024-05-09T06:29:37.154570+0000","last_became_active":"2024-05-09T06:29:32.845443+0000","last_became_peered":"2024-05-09T06:29:32.845443+0000","last_unstale":"2024-05-09T06:29:37.154570+0000","last_undegraded":"2024-05-09T06:29:37.154570+0000","last_fullsized":"2024-05-09T06:29:37.154570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:43.479453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843799+0000","last_change":"2024-05-09T06:29:33.856302+0000","last_active":"2024-05-09T06:29:34.843799+0000","last_peered":"2024-05-09T06:29:34.843799+0000","last_clean":"2024-05-09T06:29:34.843799+0000","last_became_active":"2024-05-09T06:29:33.856003+0000","last_became_peered":"2024-05-09T06:29:33.856003+0000","last_unstale":"2024-05-09T06:29:34.843799+0000","last_undegraded":"2024-05-09T06:29:34.843799+0000","last_fullsized":"2024-05-09T06:29:34.843799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:35.393848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476052+0000","last_change":"2024-05-09T06:29:32.831050+0000","last_active":"2024-05-09T06:29:35.476052+0000","last_peered":"2024-05-09T06:29:35.476052+0000","last_clean":"2024-05-09T06:29:35.476052+0000","last_became_active":"2024-05-09T06:29:32.830760+0000","last_became_peered":"2024-05-09T06:29:32.830760+0000","last_unstale":"2024-05-09T06:29:35.476052+0000","last_undegraded":"2024-05-09T06:29:35.476052+0000","last_fullsized":"2024-05-09T06:29:35.476052+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:51.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847294+0000","last_change":"2024-05-09T06:29:33.859705+0000","last_active":"2024-05-09T06:29:35.847294+0000","last_peered":"2024-05-09T06:29:35.847294+0000","last_clean":"2024-05-09T06:29:35.847294+0000","last_became_active":"2024-05-09T06:29:33.859407+0000","last_became_peered":"2024-05-09T06:29:33.859407+0000","last_unstale":"2024-05-09T06:29:35.847294+0000","last_undegraded":"2024-05-09T06:29:35.847294+0000","last_fullsized":"2024-05-09T06:29:35.847294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843564+0000","last_change":"2024-05-09T06:29:32.842910+0000","last_active":"2024-05-09T06:29:34.843564+0000","last_peered":"2024-05-09T06:29:34.843564+0000","last_clean":"2024-05-09T06:29:34.843564+0000","last_became_active":"2024-05-09T06:29:32.842534+0000","last_became_peered":"2024-05-09T06:29:32.842534+0000","last_unstale":"2024-05-09T06:29:34.843564+0000","last_undegraded":"2024-05-09T06:29:34.843564+0000","last_fullsized":"2024-05-09T06:29:34.843564+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154966+0000","last_change":"2024-05-09T06:29:33.869072+0000","last_active":"2024-05-09T06:29:37.154966+0000","last_peered":"2024-05-09T06:29:37.154966+0000","last_clean":"2024-05-09T06:29:37.154966+0000","last_became_active":"2024-05-09T06:29:33.868589+0000","last_became_peered":"2024-05-09T06:29:33.868589+0000","last_unstale":"2024-05-09T06:29:37.154966+0000","last_undegraded":"2024-05-09T06:29:37.154966+0000","last_fullsized":"2024-05-09T06:29:37.154966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:11.329525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624139+0000","last_change":"2024-05-09T06:29:32.840442+0000","last_active":"2024-05-09T06:29:35.624139+0000","last_peered":"2024-05-09T06:29:35.624139+0000","last_clean":"2024-05-09T06:29:35.624139+0000","last_became_active":"2024-05-09T06:29:32.840016+0000","last_became_peered":"2024-05-09T06:29:32.840016+0000","last_unstale":"2024-05-09T06:29:35.624139+0000","last_undegraded":"2024-05-09T06:29:35.624139+0000","last_fullsized":"2024-05-09T06:29:35.624139+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847453+0000","last_change":"2024-05-09T06:29:33.858289+0000","last_active":"2024-05-09T06:29:35.847453+0000","last_peered":"2024-05-09T06:29:35.847453+0000","last_clean":"2024-05-09T06:29:35.847453+0000","last_became_active":"2024-05-09T06:29:33.857996+0000","last_became_peered":"2024-05-09T06:29:33.857996+0000","last_unstale":"2024-05-09T06:29:35.847453+0000","last_undegraded":"2024-05-09T06:29:35.847453+0000","last_fullsized":"2024-05-09T06:29:35.847453+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:38.253814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475849+0000","last_change":"2024-05-09T06:29:32.844658+0000","last_active":"2024-05-09T06:29:35.475849+0000","last_peered":"2024-05-09T06:29:35.475849+0000","last_clean":"2024-05-09T06:29:35.475849+0000","last_became_active":"2024-05-09T06:29:32.844008+0000","last_became_peered":"2024-05-09T06:29:32.844008+0000","last_unstale":"2024-05-09T06:29:35.475849+0000","last_undegraded":"2024-05-09T06:29:35.475849+0000","last_fullsized":"2024-05-09T06:29:35.475849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843296+0000","last_change":"2024-05-09T06:29:33.839883+0000","last_active":"2024-05-09T06:29:34.843296+0000","last_peered":"2024-05-09T06:29:34.843296+0000","last_clean":"2024-05-09T06:29:34.843296+0000","last_became_active":"2024-05-09T06:29:33.839523+0000","last_became_peered":"2024-05-09T06:29:33.839523+0000","last_unstale":"2024-05-09T06:29:34.843296+0000","last_undegraded":"2024-05-09T06:29:34.843296+0000","last_fullsized":"2024-05-09T06:29:34.843296+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:43.743388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847388+0000","last_change":"2024-05-09T06:29:32.845642+0000","last_active":"2024-05-09T06:29:35.847388+0000","last_peered":"2024-05-09T06:29:35.847388+0000","last_clean":"2024-05-09T06:29:35.847388+0000","last_became_active":"2024-05-09T06:29:32.845324+0000","last_became_peered":"2024-05-09T06:29:32.845324+0000","last_unstale":"2024-05-09T06:29:35.847388+0000","last_undegraded":"2024-05-09T06:29:35.847388+0000","last_fullsized":"2024-05-09T06:29:35.847388+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:34.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154470+0000","last_change":"2024-05-09T06:29:33.851428+0000","last_active":"2024-05-09T06:29:37.154470+0000","last_peered":"2024-05-09T06:29:37.154470+0000","last_clean":"2024-05-09T06:29:37.154470+0000","last_became_active":"2024-05-09T06:29:33.851141+0000","last_became_peered":"2024-05-09T06:29:33.851141+0000","last_unstale":"2024-05-09T06:29:37.154470+0000","last_undegraded":"2024-05-09T06:29:37.154470+0000","last_fullsized":"2024-05-09T06:29:37.154470+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:23.972099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843499+0000","last_change":"2024-05-09T06:29:32.847975+0000","last_active":"2024-05-09T06:29:34.843499+0000","last_peered":"2024-05-09T06:29:34.843499+0000","last_clean":"2024-05-09T06:29:34.843499+0000","last_became_active":"2024-05-09T06:29:32.847772+0000","last_became_peered":"2024-05-09T06:29:32.847772+0000","last_unstale":"2024-05-09T06:29:34.843499+0000","last_undegraded":"2024-05-09T06:29:34.843499+0000","last_fullsized":"2024-05-09T06:29:34.843499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:35.033218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476166+0000","last_change":"2024-05-09T06:29:33.857945+0000","last_active":"2024-05-09T06:29:35.476166+0000","last_peered":"2024-05-09T06:29:35.476166+0000","last_clean":"2024-05-09T06:29:35.476166+0000","last_became_active":"2024-05-09T06:29:33.857660+0000","last_became_peered":"2024-05-09T06:29:33.857660+0000","last_unstale":"2024-05-09T06:29:35.476166+0000","last_undegraded":"2024-05-09T06:29:35.476166+0000","last_fullsized":"2024-05-09T06:29:35.476166+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:42.541481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847372+0000","last_change":"2024-05-09T06:29:32.843594+0000","last_active":"2024-05-09T06:29:35.847372+0000","last_peered":"2024-05-09T06:29:35.847372+0000","last_clean":"2024-05-09T06:29:35.847372+0000","last_became_active":"2024-05-09T06:29:32.843233+0000","last_became_peered":"2024-05-09T06:29:32.843233+0000","last_unstale":"2024-05-09T06:29:35.847372+0000","last_undegraded":"2024-05-09T06:29:35.847372+0000","last_fullsized":"2024-05-09T06:29:35.847372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:10.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843727+0000","last_change":"2024-05-09T06:29:33.855808+0000","last_active":"2024-05-09T06:29:34.843727+0000","last_peered":"2024-05-09T06:29:34.843727+0000","last_clean":"2024-05-09T06:29:34.843727+0000","last_became_active":"2024-05-09T06:29:33.855489+0000","last_became_peered":"2024-05-09T06:29:33.855489+0000","last_unstale":"2024-05-09T06:29:34.843727+0000","last_undegraded":"2024-05-09T06:29:34.843727+0000","last_fullsized":"2024-05-09T06:29:34.843727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:52.735911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154732+0000","last_change":"2024-05-09T06:29:32.841265+0000","last_active":"2024-05-09T06:29:37.154732+0000","last_peered":"2024-05-09T06:29:37.154732+0000","last_clean":"2024-05-09T06:29:37.154732+0000","last_became_active":"2024-05-09T06:29:32.840989+0000","last_became_peered":"2024-05-09T06:29:32.840989+0000","last_unstale":"2024-05-09T06:29:37.154732+0000","last_undegraded":"2024-05-09T06:29:37.154732+0000","last_fullsized":"2024-05-09T06:29:37.154732+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:41.597314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154272+0000","last_change":"2024-05-09T06:29:33.851436+0000","last_active":"2024-05-09T06:29:37.154272+0000","last_peered":"2024-05-09T06:29:37.154272+0000","last_clean":"2024-05-09T06:29:37.154272+0000","last_became_active":"2024-05-09T06:29:33.851042+0000","last_became_peered":"2024-05-09T06:29:33.851042+0000","last_unstale":"2024-05-09T06:29:37.154272+0000","last_undegraded":"2024-05-09T06:29:37.154272+0000","last_fullsized":"2024-05-09T06:29:37.154272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:41.475551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476311+0000","last_change":"2024-05-09T06:29:32.842270+0000","last_active":"2024-05-09T06:29:35.476311+0000","last_peered":"2024-05-09T06:29:35.476311+0000","last_clean":"2024-05-09T06:29:35.476311+0000","last_became_active":"2024-05-09T06:29:32.841999+0000","last_became_peered":"2024-05-09T06:29:32.841999+0000","last_unstale":"2024-05-09T06:29:35.476311+0000","last_undegraded":"2024-05-09T06:29:35.476311+0000","last_fullsized":"2024-05-09T06:29:35.476311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:34.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476537+0000","last_change":"2024-05-09T06:29:33.863499+0000","last_active":"2024-05-09T06:29:35.476537+0000","last_peered":"2024-05-09T06:29:35.476537+0000","last_clean":"2024-05-09T06:29:35.476537+0000","last_became_active":"2024-05-09T06:29:33.862695+0000","last_became_peered":"2024-05-09T06:29:33.862695+0000","last_unstale":"2024-05-09T06:29:35.476537+0000","last_undegraded":"2024-05-09T06:29:35.476537+0000","last_fullsized":"2024-05-09T06:29:35.476537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.979878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624249+0000","last_change":"2024-05-09T06:29:32.846452+0000","last_active":"2024-05-09T06:29:35.624249+0000","last_peered":"2024-05-09T06:29:35.624249+0000","last_clean":"2024-05-09T06:29:35.624249+0000","last_became_active":"2024-05-09T06:29:32.846152+0000","last_became_peered":"2024-05-09T06:29:32.846152+0000","last_unstale":"2024-05-09T06:29:35.624249+0000","last_undegraded":"2024-05-09T06:29:35.624249+0000","last_fullsized":"2024-05-09T06:29:35.624249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154884+0000","last_change":"2024-05-09T06:29:33.848338+0000","last_active":"2024-05-09T06:29:37.154884+0000","last_peered":"2024-05-09T06:29:37.154884+0000","last_clean":"2024-05-09T06:29:37.154884+0000","last_became_active":"2024-05-09T06:29:33.848055+0000","last_became_peered":"2024-05-09T06:29:33.848055+0000","last_unstale":"2024-05-09T06:29:37.154884+0000","last_undegraded":"2024-05-09T06:29:37.154884+0000","last_fullsized":"2024-05-09T06:29:37.154884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:52.227907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846992+0000","last_change":"2024-05-09T06:29:32.840887+0000","last_active":"2024-05-09T06:29:35.846992+0000","last_peered":"2024-05-09T06:29:35.846992+0000","last_clean":"2024-05-09T06:29:35.846992+0000","last_became_active":"2024-05-09T06:29:32.840544+0000","last_became_peered":"2024-05-09T06:29:32.840544+0000","last_unstale":"2024-05-09T06:29:35.846992+0000","last_undegraded":"2024-05-09T06:29:35.846992+0000","last_fullsized":"2024-05-09T06:29:35.846992+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476605+0000","last_change":"2024-05-09T06:29:33.861924+0000","last_active":"2024-05-09T06:29:35.476605+0000","last_peered":"2024-05-09T06:29:35.476605+0000","last_clean":"2024-05-09T06:29:35.476605+0000","last_became_active":"2024-05-09T06:29:33.860239+0000","last_became_peered":"2024-05-09T06:29:33.860239+0000","last_unstale":"2024-05-09T06:29:35.476605+0000","last_undegraded":"2024-05-09T06:29:35.476605+0000","last_fullsized":"2024-05-09T06:29:35.476605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:52.227907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154408+0000","last_change":"2024-05-09T06:29:32.839558+0000","last_active":"2024-05-09T06:29:37.154408+0000","last_peered":"2024-05-09T06:29:37.154408+0000","last_clean":"2024-05-09T06:29:37.154408+0000","last_became_active":"2024-05-09T06:29:32.839220+0000","last_became_peered":"2024-05-09T06:29:32.839220+0000","last_unstale":"2024-05-09T06:29:37.154408+0000","last_undegraded":"2024-05-09T06:29:37.154408+0000","last_fullsized":"2024-05-09T06:29:37.154408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:48.406221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154324+0000","last_change":"2024-05-09T06:29:33.856259+0000","last_active":"2024-05-09T06:29:37.154324+0000","last_peered":"2024-05-09T06:29:37.154324+0000","last_clean":"2024-05-09T06:29:37.154324+0000","last_became_active":"2024-05-09T06:29:33.855452+0000","last_became_peered":"2024-05-09T06:29:33.855452+0000","last_unstale":"2024-05-09T06:29:37.154324+0000","last_undegraded":"2024-05-09T06:29:37.154324+0000","last_fullsized":"2024-05-09T06:29:37.154324+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:22.627476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476673+0000","last_change":"2024-05-09T06:29:32.847677+0000","last_active":"2024-05-09T06:29:35.476673+0000","last_peered":"2024-05-09T06:29:35.476673+0000","last_clean":"2024-05-09T06:29:35.476673+0000","last_became_active":"2024-05-09T06:29:32.847372+0000","last_became_peered":"2024-05-09T06:29:32.847372+0000","last_unstale":"2024-05-09T06:29:35.476673+0000","last_undegraded":"2024-05-09T06:29:35.476673+0000","last_fullsized":"2024-05-09T06:29:35.476673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:37.820825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153793+0000","last_change":"2024-05-09T06:29:33.856671+0000","last_active":"2024-05-09T06:29:37.153793+0000","last_peered":"2024-05-09T06:29:37.153793+0000","last_clean":"2024-05-09T06:29:37.153793+0000","last_became_active":"2024-05-09T06:29:33.856343+0000","last_became_peered":"2024-05-09T06:29:33.856343+0000","last_unstale":"2024-05-09T06:29:37.153793+0000","last_undegraded":"2024-05-09T06:29:37.153793+0000","last_fullsized":"2024-05-09T06:29:37.153793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:47.385123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841946+0000","last_change":"2024-05-09T06:29:32.842351+0000","last_active":"2024-05-09T06:29:34.841946+0000","last_peered":"2024-05-09T06:29:34.841946+0000","last_clean":"2024-05-09T06:29:34.841946+0000","last_became_active":"2024-05-09T06:29:32.842065+0000","last_became_peered":"2024-05-09T06:29:32.842065+0000","last_unstale":"2024-05-09T06:29:34.841946+0000","last_undegraded":"2024-05-09T06:29:34.841946+0000","last_fullsized":"2024-05-09T06:29:34.841946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:14.973970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843411+0000","last_change":"2024-05-09T06:29:33.856898+0000","last_active":"2024-05-09T06:29:34.843411+0000","last_peered":"2024-05-09T06:29:34.843411+0000","last_clean":"2024-05-09T06:29:34.843411+0000","last_became_active":"2024-05-09T06:29:33.856499+0000","last_became_peered":"2024-05-09T06:29:33.856499+0000","last_unstale":"2024-05-09T06:29:34.843411+0000","last_undegraded":"2024-05-09T06:29:34.843411+0000","last_fullsized":"2024-05-09T06:29:34.843411+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:37.440909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624340+0000","last_change":"2024-05-09T06:29:32.844205+0000","last_active":"2024-05-09T06:29:35.624340+0000","last_peered":"2024-05-09T06:29:35.624340+0000","last_clean":"2024-05-09T06:29:35.624340+0000","last_became_active":"2024-05-09T06:29:32.843719+0000","last_became_peered":"2024-05-09T06:29:32.843719+0000","last_unstale":"2024-05-09T06:29:35.624340+0000","last_undegraded":"2024-05-09T06:29:35.624340+0000","last_fullsized":"2024-05-09T06:29:35.624340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153939+0000","last_change":"2024-05-09T06:29:33.860109+0000","last_active":"2024-05-09T06:29:37.153939+0000","last_peered":"2024-05-09T06:29:37.153939+0000","last_clean":"2024-05-09T06:29:37.153939+0000","last_became_active":"2024-05-09T06:29:33.859797+0000","last_became_peered":"2024-05-09T06:29:33.859797+0000","last_unstale":"2024-05-09T06:29:37.153939+0000","last_undegraded":"2024-05-09T06:29:37.153939+0000","last_fullsized":"2024-05-09T06:29:37.153939+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624480+0000","last_change":"2024-05-09T06:29:32.847048+0000","last_active":"2024-05-09T06:29:35.624480+0000","last_peered":"2024-05-09T06:29:35.624480+0000","last_clean":"2024-05-09T06:29:35.624480+0000","last_became_active":"2024-05-09T06:29:32.846701+0000","last_became_peered":"2024-05-09T06:29:32.846701+0000","last_unstale":"2024-05-09T06:29:35.624480+0000","last_undegraded":"2024-05-09T06:29:35.624480+0000","last_fullsized":"2024-05-09T06:29:35.624480+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154413+0000","last_change":"2024-05-09T06:29:33.852145+0000","last_active":"2024-05-09T06:29:37.154413+0000","last_peered":"2024-05-09T06:29:37.154413+0000","last_clean":"2024-05-09T06:29:37.154413+0000","last_became_active":"2024-05-09T06:29:33.851840+0000","last_became_peered":"2024-05-09T06:29:33.851840+0000","last_unstale":"2024-05-09T06:29:37.154413+0000","last_undegraded":"2024-05-09T06:29:37.154413+0000","last_fullsized":"2024-05-09T06:29:37.154413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:38.253814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154062+0000","last_change":"2024-05-09T06:29:32.840423+0000","last_active":"2024-05-09T06:29:37.154062+0000","last_peered":"2024-05-09T06:29:37.154062+0000","last_clean":"2024-05-09T06:29:37.154062+0000","last_became_active":"2024-05-09T06:29:32.840136+0000","last_became_peered":"2024-05-09T06:29:32.840136+0000","last_unstale":"2024-05-09T06:29:37.154062+0000","last_undegraded":"2024-05-09T06:29:37.154062+0000","last_fullsized":"2024-05-09T06:29:37.154062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624242+0000","last_change":"2024-05-09T06:29:33.850945+0000","last_active":"2024-05-09T06:29:35.624242+0000","last_peered":"2024-05-09T06:29:35.624242+0000","last_clean":"2024-05-09T06:29:35.624242+0000","last_became_active":"2024-05-09T06:29:33.850590+0000","last_became_peered":"2024-05-09T06:29:33.850590+0000","last_unstale":"2024-05-09T06:29:35.624242+0000","last_undegraded":"2024-05-09T06:29:35.624242+0000","last_fullsized":"2024-05-09T06:29:35.624242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:22.199322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154504+0000","last_change":"2024-05-09T06:29:32.839491+0000","last_active":"2024-05-09T06:29:37.154504+0000","last_peered":"2024-05-09T06:29:37.154504+0000","last_clean":"2024-05-09T06:29:37.154504+0000","last_became_active":"2024-05-09T06:29:32.839083+0000","last_became_peered":"2024-05-09T06:29:32.839083+0000","last_unstale":"2024-05-09T06:29:37.154504+0000","last_undegraded":"2024-05-09T06:29:37.154504+0000","last_fullsized":"2024-05-09T06:29:37.154504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.071935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476433+0000","last_change":"2024-05-09T06:29:33.853821+0000","last_active":"2024-05-09T06:29:35.476433+0000","last_peered":"2024-05-09T06:29:35.476433+0000","last_clean":"2024-05-09T06:29:35.476433+0000","last_became_active":"2024-05-09T06:29:33.853524+0000","last_became_peered":"2024-05-09T06:29:33.853524+0000","last_unstale":"2024-05-09T06:29:35.476433+0000","last_undegraded":"2024-05-09T06:29:35.476433+0000","last_fullsized":"2024-05-09T06:29:35.476433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:22.627476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476065+0000","last_change":"2024-05-09T06:29:32.840100+0000","last_active":"2024-05-09T06:29:35.476065+0000","last_peered":"2024-05-09T06:29:35.476065+0000","last_clean":"2024-05-09T06:29:35.476065+0000","last_became_active":"2024-05-09T06:29:32.839832+0000","last_became_peered":"2024-05-09T06:29:32.839832+0000","last_unstale":"2024-05-09T06:29:35.476065+0000","last_undegraded":"2024-05-09T06:29:35.476065+0000","last_fullsized":"2024-05-09T06:29:35.476065+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154350+0000","last_change":"2024-05-09T06:29:33.852908+0000","last_active":"2024-05-09T06:29:37.154350+0000","last_peered":"2024-05-09T06:29:37.154350+0000","last_clean":"2024-05-09T06:29:37.154350+0000","last_became_active":"2024-05-09T06:29:33.852553+0000","last_became_peered":"2024-05-09T06:29:33.852553+0000","last_unstale":"2024-05-09T06:29:37.154350+0000","last_undegraded":"2024-05-09T06:29:37.154350+0000","last_fullsized":"2024-05-09T06:29:37.154350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841653+0000","last_change":"2024-05-09T06:29:32.832884+0000","last_active":"2024-05-09T06:29:34.841653+0000","last_peered":"2024-05-09T06:29:34.841653+0000","last_clean":"2024-05-09T06:29:34.841653+0000","last_became_active":"2024-05-09T06:29:32.832612+0000","last_became_peered":"2024-05-09T06:29:32.832612+0000","last_unstale":"2024-05-09T06:29:34.841653+0000","last_undegraded":"2024-05-09T06:29:34.841653+0000","last_fullsized":"2024-05-09T06:29:34.841653+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:09.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154528+0000","last_change":"2024-05-09T06:29:33.849812+0000","last_active":"2024-05-09T06:29:37.154528+0000","last_peered":"2024-05-09T06:29:37.154528+0000","last_clean":"2024-05-09T06:29:37.154528+0000","last_became_active":"2024-05-09T06:29:33.849351+0000","last_became_peered":"2024-05-09T06:29:33.849351+0000","last_unstale":"2024-05-09T06:29:37.154528+0000","last_undegraded":"2024-05-09T06:29:37.154528+0000","last_fullsized":"2024-05-09T06:29:37.154528+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:41.208467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624396+0000","last_change":"2024-05-09T06:29:32.846605+0000","last_active":"2024-05-09T06:29:35.624396+0000","last_peered":"2024-05-09T06:29:35.624396+0000","last_clean":"2024-05-09T06:29:35.624396+0000","last_became_active":"2024-05-09T06:29:32.846230+0000","last_became_peered":"2024-05-09T06:29:32.846230+0000","last_unstale":"2024-05-09T06:29:35.624396+0000","last_undegraded":"2024-05-09T06:29:35.624396+0000","last_fullsized":"2024-05-09T06:29:35.624396+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154070+0000","last_change":"2024-05-09T06:29:33.866676+0000","last_active":"2024-05-09T06:29:37.154070+0000","last_peered":"2024-05-09T06:29:37.154070+0000","last_clean":"2024-05-09T06:29:37.154070+0000","last_became_active":"2024-05-09T06:29:33.866409+0000","last_became_peered":"2024-05-09T06:29:33.866409+0000","last_unstale":"2024-05-09T06:29:37.154070+0000","last_undegraded":"2024-05-09T06:29:37.154070+0000","last_fullsized":"2024-05-09T06:29:37.154070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:09.864980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847488+0000","last_change":"2024-05-09T06:29:32.844437+0000","last_active":"2024-05-09T06:29:35.847488+0000","last_peered":"2024-05-09T06:29:35.847488+0000","last_clean":"2024-05-09T06:29:35.847488+0000","last_became_active":"2024-05-09T06:29:32.843623+0000","last_became_peered":"2024-05-09T06:29:32.843623+0000","last_unstale":"2024-05-09T06:29:35.847488+0000","last_undegraded":"2024-05-09T06:29:35.847488+0000","last_fullsized":"2024-05-09T06:29:35.847488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:51.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154151+0000","last_change":"2024-05-09T06:29:33.840080+0000","last_active":"2024-05-09T06:29:37.154151+0000","last_peered":"2024-05-09T06:29:37.154151+0000","last_clean":"2024-05-09T06:29:37.154151+0000","last_became_active":"2024-05-09T06:29:33.839786+0000","last_became_peered":"2024-05-09T06:29:33.839786+0000","last_unstale":"2024-05-09T06:29:37.154151+0000","last_undegraded":"2024-05-09T06:29:37.154151+0000","last_fullsized":"2024-05-09T06:29:37.154151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:19.792349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476343+0000","last_change":"2024-05-09T06:29:32.845924+0000","last_active":"2024-05-09T06:29:35.476343+0000","last_peered":"2024-05-09T06:29:35.476343+0000","last_clean":"2024-05-09T06:29:35.476343+0000","last_became_active":"2024-05-09T06:29:32.845637+0000","last_became_peered":"2024-05-09T06:29:32.845637+0000","last_unstale":"2024-05-09T06:29:35.476343+0000","last_undegraded":"2024-05-09T06:29:35.476343+0000","last_fullsized":"2024-05-09T06:29:35.476343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:10.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476547+0000","last_change":"2024-05-09T06:29:33.852983+0000","last_active":"2024-05-09T06:29:35.476547+0000","last_peered":"2024-05-09T06:29:35.476547+0000","last_clean":"2024-05-09T06:29:35.476547+0000","last_became_active":"2024-05-09T06:29:33.852630+0000","last_became_peered":"2024-05-09T06:29:33.852630+0000","last_unstale":"2024-05-09T06:29:35.476547+0000","last_undegraded":"2024-05-09T06:29:35.476547+0000","last_fullsized":"2024-05-09T06:29:35.476547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:04.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154242+0000","last_change":"2024-05-09T06:29:32.841755+0000","last_active":"2024-05-09T06:29:37.154242+0000","last_peered":"2024-05-09T06:29:37.154242+0000","last_clean":"2024-05-09T06:29:37.154242+0000","last_became_active":"2024-05-09T06:29:32.840696+0000","last_became_peered":"2024-05-09T06:29:32.840696+0000","last_unstale":"2024-05-09T06:29:37.154242+0000","last_undegraded":"2024-05-09T06:29:37.154242+0000","last_fullsized":"2024-05-09T06:29:37.154242+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:08.855695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154325+0000","last_change":"2024-05-09T06:29:33.854703+0000","last_active":"2024-05-09T06:29:37.154325+0000","last_peered":"2024-05-09T06:29:37.154325+0000","last_clean":"2024-05-09T06:29:37.154325+0000","last_became_active":"2024-05-09T06:29:33.854388+0000","last_became_peered":"2024-05-09T06:29:33.854388+0000","last_unstale":"2024-05-09T06:29:37.154325+0000","last_undegraded":"2024-05-09T06:29:37.154325+0000","last_fullsized":"2024-05-09T06:29:37.154325+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154523+0000","last_change":"2024-05-09T06:29:32.841567+0000","last_active":"2024-05-09T06:29:37.154523+0000","last_peered":"2024-05-09T06:29:37.154523+0000","last_clean":"2024-05-09T06:29:37.154523+0000","last_became_active":"2024-05-09T06:29:32.841246+0000","last_became_peered":"2024-05-09T06:29:32.841246+0000","last_unstale":"2024-05-09T06:29:37.154523+0000","last_undegraded":"2024-05-09T06:29:37.154523+0000","last_fullsized":"2024-05-09T06:29:37.154523+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:09.775031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476381+0000","last_change":"2024-05-09T06:29:33.867290+0000","last_active":"2024-05-09T06:29:35.476381+0000","last_peered":"2024-05-09T06:29:35.476381+0000","last_clean":"2024-05-09T06:29:35.476381+0000","last_became_active":"2024-05-09T06:29:33.867012+0000","last_became_peered":"2024-05-09T06:29:33.867012+0000","last_unstale":"2024-05-09T06:29:35.476381+0000","last_undegraded":"2024-05-09T06:29:35.476381+0000","last_fullsized":"2024-05-09T06:29:35.476381+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:44.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843231+0000","last_change":"2024-05-09T06:29:32.843603+0000","last_active":"2024-05-09T06:29:34.843231+0000","last_peered":"2024-05-09T06:29:34.843231+0000","last_clean":"2024-05-09T06:29:34.843231+0000","last_became_active":"2024-05-09T06:29:32.843314+0000","last_became_peered":"2024-05-09T06:29:32.843314+0000","last_unstale":"2024-05-09T06:29:34.843231+0000","last_undegraded":"2024-05-09T06:29:34.843231+0000","last_fullsized":"2024-05-09T06:29:34.843231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154996+0000","last_change":"2024-05-09T06:29:33.856782+0000","last_active":"2024-05-09T06:29:37.154996+0000","last_peered":"2024-05-09T06:29:37.154996+0000","last_clean":"2024-05-09T06:29:37.154996+0000","last_became_active":"2024-05-09T06:29:33.856081+0000","last_became_peered":"2024-05-09T06:29:33.856081+0000","last_unstale":"2024-05-09T06:29:37.154996+0000","last_undegraded":"2024-05-09T06:29:37.154996+0000","last_fullsized":"2024-05-09T06:29:37.154996+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:03.895859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154526+0000","last_change":"2024-05-09T06:29:32.843469+0000","last_active":"2024-05-09T06:29:37.154526+0000","last_peered":"2024-05-09T06:29:37.154526+0000","last_clean":"2024-05-09T06:29:37.154526+0000","last_became_active":"2024-05-09T06:29:32.842907+0000","last_became_peered":"2024-05-09T06:29:32.842907+0000","last_unstale":"2024-05-09T06:29:37.154526+0000","last_undegraded":"2024-05-09T06:29:37.154526+0000","last_fullsized":"2024-05-09T06:29:37.154526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:03.213349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624032+0000","last_change":"2024-05-09T06:29:33.852402+0000","last_active":"2024-05-09T06:29:35.624032+0000","last_peered":"2024-05-09T06:29:35.624032+0000","last_clean":"2024-05-09T06:29:35.624032+0000","last_became_active":"2024-05-09T06:29:33.852053+0000","last_became_peered":"2024-05-09T06:29:33.852053+0000","last_unstale":"2024-05-09T06:29:35.624032+0000","last_undegraded":"2024-05-09T06:29:35.624032+0000","last_fullsized":"2024-05-09T06:29:35.624032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:04.222634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476546+0000","last_change":"2024-05-09T06:29:32.843492+0000","last_active":"2024-05-09T06:29:35.476546+0000","last_peered":"2024-05-09T06:29:35.476546+0000","last_clean":"2024-05-09T06:29:35.476546+0000","last_became_active":"2024-05-09T06:29:32.842665+0000","last_became_peered":"2024-05-09T06:29:32.842665+0000","last_unstale":"2024-05-09T06:29:35.476546+0000","last_undegraded":"2024-05-09T06:29:35.476546+0000","last_fullsized":"2024-05-09T06:29:35.476546+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846757+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:29:35.846757+0000","last_peered":"2024-05-09T06:29:35.846757+0000","last_clean":"2024-05-09T06:29:35.846757+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:29:35.846757+0000","last_undegraded":"2024-05-09T06:29:35.846757+0000","last_fullsized":"2024-05-09T06:29:35.846757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1208320,"data_stored":860812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":270972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":270902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":271954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":271932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":614400,"data_stored":270480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":860744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:29:39.793 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:29:39.811+0000 7f1f54836640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:29:42.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:29:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.461+0000 7ff0f13b7640 1 Processor -- start 2024-05-09T06:29:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.461+0000 7ff0f13b7640 1 -- start start 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec0d0d90 0x7ff0ec0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 0x7ff0ec161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0d1a40 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0cf730 con 0x7ff0ec1611e0 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0f13b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0cfa10 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40242/0 (socket says 172.21.15.1:40242) 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 -- 172.21.15.1:0/1689984362 learned_addr learned my addr 172.21.15.1:0/1689984362 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eb7fe640 1 --2- 172.21.15.1:0/1689984362 >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 0x7ff0ec161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0e27fc640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec0d0d90 0x7ff0ec0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 -- 172.21.15.1:0/1689984362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec0d0d90 msgr2=0x7ff0ec0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec0d0d90 0x7ff0ec0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 -- 172.21.15.1:0/1689984362 >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 msgr2=0x7ff0ec161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/1689984362 >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 0x7ff0ec161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eaffd640 1 -- 172.21.15.1:0/1689984362 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec0cfcf0 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.462+0000 7ff0eb7fe640 1 --2- 172.21.15.1:0/1689984362 >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 0x7ff0ec161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.463+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009870 tx=0x7ff0dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce269d0ed98b7749 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.463+0000 7ff0ea7fc640 1 -- 172.21.15.1:0/1689984362 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc037070 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.463+0000 7ff0ea7fc640 1 -- 172.21.15.1:0/1689984362 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc032440 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.463+0000 7ff0ea7fc640 1 -- 172.21.15.1:0/1689984362 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc032810 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 -- 172.21.15.1:0/1689984362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d09b0 msgr2=0x7ff0ec0b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009870 tx=0x7ff0dc02a180 comp rx=0 tx=0).stop 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 -- 172.21.15.1:0/1689984362 shutdown_connections 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/1689984362 >> v2:172.21.15.6:3300/0 conn(0x7ff0ec1611e0 0x7ff0ec161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec0d0d90 0x7ff0ec0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/1689984362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec0b6cc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 -- 172.21.15.1:0/1689984362 >> 172.21.15.1:0/1689984362 conn(0x7ff0ec0d3e70 msgr2=0x7ff0ec0d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 -- 172.21.15.1:0/1689984362 shutdown_connections 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.464+0000 7ff0f13b7640 1 -- 172.21.15.1:0/1689984362 wait complete. 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.465+0000 7ff0f13b7640 1 Processor -- start 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.465+0000 7ff0f13b7640 1 -- start start 2024-05-09T06:29:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec1f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec1611e0 0x7ff0ec09ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec161bb0 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0d4bc0 con 0x7ff0ec0d09b0 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0f13b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b52e0 con 0x7ff0ec1611e0 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40244/0 (socket says 172.21.15.1:40244) 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 -- 172.21.15.1:0/2607486721 learned_addr learned my addr 172.21.15.1:0/2607486721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0eb7fe640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec1611e0 0x7ff0ec09ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec1f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 -- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec1611e0 msgr2=0x7ff0ec09ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec1611e0 0x7ff0ec09ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 -- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0ec0d09b0 msgr2=0x7ff0ec1f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec1f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.466+0000 7ff0e27fc640 1 -- 172.21.15.1:0/2607486721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec09e130 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0e27fc640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400d800 tx=0x7ff0d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4018020 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d40096a0 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4009a70 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec09e410 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.467+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec09e9d0 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.468+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec0752b0 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.468+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0d401b020 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.469+0000 7ff0e3fff640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 0x7ff0cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.469+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7ff0d408b560 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.469+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 0x7ff0cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.469+0000 7ff0eaffd640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 0x7ff0cc06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.474+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0d4022020 con 0x7ff0ec0d0d90 2024-05-09T06:29:42.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.653+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0ec06b390 con 0x7ff0cc06be30 2024-05-09T06:29:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.659+0000 7ff0e3fff640 1 -- 172.21.15.1:0/2607486721 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7ff0ec06b390 con 0x7ff0cc06be30 2024-05-09T06:29:42.643 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 msgr2=0x7ff0cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 0x7ff0cc06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc02e000 comp rx=0 tx=0).stop 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 msgr2=0x7ff0ec1562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400d800 tx=0x7ff0d400dcc0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 shutdown_connections 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0cc06be30 0x7ff0cc06e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0ec1611e0 0x7ff0ec09ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0ec0d0d90 0x7ff0ec1562c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 --2- 172.21.15.1:0/2607486721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0ec0d09b0 0x7ff0ec1f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 >> 172.21.15.1:0/2607486721 conn(0x7ff0ec0d3e70 msgr2=0x7ff0ec06d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 shutdown_connections 2024-05-09T06:29:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.662+0000 7ff0f13b7640 1 -- 172.21.15.1:0/2607486721 wait complete. 2024-05-09T06:29:42.655 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-05-09T06:29:41.096789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238780,"kb_used_data":5960,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735940,"statfs":{"total":773094113280,"available":772849602560,"internally_reserved":0,"allocated":6103040,"data_stored":3348116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001472"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154204+0000","last_change":"2024-05-09T06:29:33.859341+0000","last_active":"2024-05-09T06:29:37.154204+0000","last_peered":"2024-05-09T06:29:37.154204+0000","last_clean":"2024-05-09T06:29:37.154204+0000","last_became_active":"2024-05-09T06:29:33.859037+0000","last_became_peered":"2024-05-09T06:29:33.859037+0000","last_unstale":"2024-05-09T06:29:37.154204+0000","last_undegraded":"2024-05-09T06:29:37.154204+0000","last_fullsized":"2024-05-09T06:29:37.154204+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:49.415506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847402+0000","last_change":"2024-05-09T06:29:33.855717+0000","last_active":"2024-05-09T06:29:35.847402+0000","last_peered":"2024-05-09T06:29:35.847402+0000","last_clean":"2024-05-09T06:29:35.847402+0000","last_became_active":"2024-05-09T06:29:33.855439+0000","last_became_peered":"2024-05-09T06:29:33.855439+0000","last_unstale":"2024-05-09T06:29:35.847402+0000","last_undegraded":"2024-05-09T06:29:35.847402+0000","last_fullsized":"2024-05-09T06:29:35.847402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476298+0000","last_change":"2024-05-09T06:29:33.856794+0000","last_active":"2024-05-09T06:29:35.476298+0000","last_peered":"2024-05-09T06:29:35.476298+0000","last_clean":"2024-05-09T06:29:35.476298+0000","last_became_active":"2024-05-09T06:29:33.854541+0000","last_became_peered":"2024-05-09T06:29:33.854541+0000","last_unstale":"2024-05-09T06:29:35.476298+0000","last_undegraded":"2024-05-09T06:29:35.476298+0000","last_fullsized":"2024-05-09T06:29:35.476298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:31:33.577516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847161+0000","last_change":"2024-05-09T06:29:33.852260+0000","last_active":"2024-05-09T06:29:35.847161+0000","last_peered":"2024-05-09T06:29:35.847161+0000","last_clean":"2024-05-09T06:29:35.847161+0000","last_became_active":"2024-05-09T06:29:33.851504+0000","last_became_peered":"2024-05-09T06:29:33.851504+0000","last_unstale":"2024-05-09T06:29:35.847161+0000","last_undegraded":"2024-05-09T06:29:35.847161+0000","last_fullsized":"2024-05-09T06:29:35.847161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843697+0000","last_change":"2024-05-09T06:29:33.848187+0000","last_active":"2024-05-09T06:29:34.843697+0000","last_peered":"2024-05-09T06:29:34.843697+0000","last_clean":"2024-05-09T06:29:34.843697+0000","last_became_active":"2024-05-09T06:29:33.847863+0000","last_became_peered":"2024-05-09T06:29:33.847863+0000","last_unstale":"2024-05-09T06:29:34.843697+0000","last_undegraded":"2024-05-09T06:29:34.843697+0000","last_fullsized":"2024-05-09T06:29:34.843697+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:12.008315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153922+0000","last_change":"2024-05-09T06:29:33.858538+0000","last_active":"2024-05-09T06:29:37.153922+0000","last_peered":"2024-05-09T06:29:37.153922+0000","last_clean":"2024-05-09T06:29:37.153922+0000","last_became_active":"2024-05-09T06:29:33.858226+0000","last_became_peered":"2024-05-09T06:29:33.858226+0000","last_unstale":"2024-05-09T06:29:37.153922+0000","last_undegraded":"2024-05-09T06:29:37.153922+0000","last_fullsized":"2024-05-09T06:29:37.153922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:50.081220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475888+0000","last_change":"2024-05-09T06:29:33.868884+0000","last_active":"2024-05-09T06:29:35.475888+0000","last_peered":"2024-05-09T06:29:35.475888+0000","last_clean":"2024-05-09T06:29:35.475888+0000","last_became_active":"2024-05-09T06:29:33.868607+0000","last_became_peered":"2024-05-09T06:29:33.868607+0000","last_unstale":"2024-05-09T06:29:35.475888+0000","last_undegraded":"2024-05-09T06:29:35.475888+0000","last_fullsized":"2024-05-09T06:29:35.475888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:37.905161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154183+0000","last_change":"2024-05-09T06:29:33.861341+0000","last_active":"2024-05-09T06:29:37.154183+0000","last_peered":"2024-05-09T06:29:37.154183+0000","last_clean":"2024-05-09T06:29:37.154183+0000","last_became_active":"2024-05-09T06:29:33.861035+0000","last_became_peered":"2024-05-09T06:29:33.861035+0000","last_unstale":"2024-05-09T06:29:37.154183+0000","last_undegraded":"2024-05-09T06:29:37.154183+0000","last_fullsized":"2024-05-09T06:29:37.154183+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:44.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154916+0000","last_change":"2024-05-09T06:29:33.863838+0000","last_active":"2024-05-09T06:29:37.154916+0000","last_peered":"2024-05-09T06:29:37.154916+0000","last_clean":"2024-05-09T06:29:37.154916+0000","last_became_active":"2024-05-09T06:29:33.863322+0000","last_became_peered":"2024-05-09T06:29:33.863322+0000","last_unstale":"2024-05-09T06:29:37.154916+0000","last_undegraded":"2024-05-09T06:29:37.154916+0000","last_fullsized":"2024-05-09T06:29:37.154916+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:10.424586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154240+0000","last_change":"2024-05-09T06:29:33.854285+0000","last_active":"2024-05-09T06:29:37.154240+0000","last_peered":"2024-05-09T06:29:37.154240+0000","last_clean":"2024-05-09T06:29:37.154240+0000","last_became_active":"2024-05-09T06:29:33.853949+0000","last_became_peered":"2024-05-09T06:29:33.853949+0000","last_unstale":"2024-05-09T06:29:37.154240+0000","last_undegraded":"2024-05-09T06:29:37.154240+0000","last_fullsized":"2024-05-09T06:29:37.154240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.842011+0000","last_change":"2024-05-09T06:29:33.859806+0000","last_active":"2024-05-09T06:29:34.842011+0000","last_peered":"2024-05-09T06:29:34.842011+0000","last_clean":"2024-05-09T06:29:34.842011+0000","last_became_active":"2024-05-09T06:29:33.859531+0000","last_became_peered":"2024-05-09T06:29:33.859531+0000","last_unstale":"2024-05-09T06:29:34.842011+0000","last_undegraded":"2024-05-09T06:29:34.842011+0000","last_fullsized":"2024-05-09T06:29:34.842011+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:32.155052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154777+0000","last_change":"2024-05-09T06:29:33.850683+0000","last_active":"2024-05-09T06:29:37.154777+0000","last_peered":"2024-05-09T06:29:37.154777+0000","last_clean":"2024-05-09T06:29:37.154777+0000","last_became_active":"2024-05-09T06:29:33.850373+0000","last_became_peered":"2024-05-09T06:29:33.850373+0000","last_unstale":"2024-05-09T06:29:37.154777+0000","last_undegraded":"2024-05-09T06:29:37.154777+0000","last_fullsized":"2024-05-09T06:29:37.154777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:04.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153789+0000","last_change":"2024-05-09T06:29:33.855174+0000","last_active":"2024-05-09T06:29:37.153789+0000","last_peered":"2024-05-09T06:29:37.153789+0000","last_clean":"2024-05-09T06:29:37.153789+0000","last_became_active":"2024-05-09T06:29:33.854849+0000","last_became_peered":"2024-05-09T06:29:33.854849+0000","last_unstale":"2024-05-09T06:29:37.153789+0000","last_undegraded":"2024-05-09T06:29:37.153789+0000","last_fullsized":"2024-05-09T06:29:37.153789+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:22.199322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476329+0000","last_change":"2024-05-09T06:29:33.853018+0000","last_active":"2024-05-09T06:29:35.476329+0000","last_peered":"2024-05-09T06:29:35.476329+0000","last_clean":"2024-05-09T06:29:35.476329+0000","last_became_active":"2024-05-09T06:29:33.852724+0000","last_became_peered":"2024-05-09T06:29:33.852724+0000","last_unstale":"2024-05-09T06:29:35.476329+0000","last_undegraded":"2024-05-09T06:29:35.476329+0000","last_fullsized":"2024-05-09T06:29:35.476329+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:19.792349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846985+0000","last_change":"2024-05-09T06:29:33.851034+0000","last_active":"2024-05-09T06:29:35.846985+0000","last_peered":"2024-05-09T06:29:35.846985+0000","last_clean":"2024-05-09T06:29:35.846985+0000","last_became_active":"2024-05-09T06:29:33.850701+0000","last_became_peered":"2024-05-09T06:29:33.850701+0000","last_unstale":"2024-05-09T06:29:35.846985+0000","last_undegraded":"2024-05-09T06:29:35.846985+0000","last_fullsized":"2024-05-09T06:29:35.846985+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:09.864980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153718+0000","last_change":"2024-05-09T06:29:33.855894+0000","last_active":"2024-05-09T06:29:37.153718+0000","last_peered":"2024-05-09T06:29:37.153718+0000","last_clean":"2024-05-09T06:29:37.153718+0000","last_became_active":"2024-05-09T06:29:33.855618+0000","last_became_peered":"2024-05-09T06:29:33.855618+0000","last_unstale":"2024-05-09T06:29:37.153718+0000","last_undegraded":"2024-05-09T06:29:37.153718+0000","last_fullsized":"2024-05-09T06:29:37.153718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:11.434050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476531+0000","last_change":"2024-05-09T06:29:33.862175+0000","last_active":"2024-05-09T06:29:35.476531+0000","last_peered":"2024-05-09T06:29:35.476531+0000","last_clean":"2024-05-09T06:29:35.476531+0000","last_became_active":"2024-05-09T06:29:33.861447+0000","last_became_peered":"2024-05-09T06:29:33.861447+0000","last_unstale":"2024-05-09T06:29:35.476531+0000","last_undegraded":"2024-05-09T06:29:35.476531+0000","last_fullsized":"2024-05-09T06:29:35.476531+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:25.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847496+0000","last_change":"2024-05-09T06:29:33.857676+0000","last_active":"2024-05-09T06:29:35.847496+0000","last_peered":"2024-05-09T06:29:35.847496+0000","last_clean":"2024-05-09T06:29:35.847496+0000","last_became_active":"2024-05-09T06:29:33.857359+0000","last_became_peered":"2024-05-09T06:29:33.857359+0000","last_unstale":"2024-05-09T06:29:35.847496+0000","last_undegraded":"2024-05-09T06:29:35.847496+0000","last_fullsized":"2024-05-09T06:29:35.847496+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:23.972099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476303+0000","last_change":"2024-05-09T06:29:33.863734+0000","last_active":"2024-05-09T06:29:35.476303+0000","last_peered":"2024-05-09T06:29:35.476303+0000","last_clean":"2024-05-09T06:29:35.476303+0000","last_became_active":"2024-05-09T06:29:33.863237+0000","last_became_peered":"2024-05-09T06:29:33.863237+0000","last_unstale":"2024-05-09T06:29:35.476303+0000","last_undegraded":"2024-05-09T06:29:35.476303+0000","last_fullsized":"2024-05-09T06:29:35.476303+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:18.007333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624319+0000","last_change":"2024-05-09T06:29:33.851658+0000","last_active":"2024-05-09T06:29:35.624319+0000","last_peered":"2024-05-09T06:29:35.624319+0000","last_clean":"2024-05-09T06:29:35.624319+0000","last_became_active":"2024-05-09T06:29:33.851386+0000","last_became_peered":"2024-05-09T06:29:33.851386+0000","last_unstale":"2024-05-09T06:29:35.624319+0000","last_undegraded":"2024-05-09T06:29:35.624319+0000","last_fullsized":"2024-05-09T06:29:35.624319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:47.385123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624177+0000","last_change":"2024-05-09T06:29:33.850971+0000","last_active":"2024-05-09T06:29:35.624177+0000","last_peered":"2024-05-09T06:29:35.624177+0000","last_clean":"2024-05-09T06:29:35.624177+0000","last_became_active":"2024-05-09T06:29:33.850661+0000","last_became_peered":"2024-05-09T06:29:33.850661+0000","last_unstale":"2024-05-09T06:29:35.624177+0000","last_undegraded":"2024-05-09T06:29:35.624177+0000","last_fullsized":"2024-05-09T06:29:35.624177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:11.434050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154412+0000","last_change":"2024-05-09T06:29:33.855891+0000","last_active":"2024-05-09T06:29:37.154412+0000","last_peered":"2024-05-09T06:29:37.154412+0000","last_clean":"2024-05-09T06:29:37.154412+0000","last_became_active":"2024-05-09T06:29:33.855496+0000","last_became_peered":"2024-05-09T06:29:33.855496+0000","last_unstale":"2024-05-09T06:29:37.154412+0000","last_undegraded":"2024-05-09T06:29:37.154412+0000","last_fullsized":"2024-05-09T06:29:37.154412+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.979878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841734+0000","last_change":"2024-05-09T06:29:33.859907+0000","last_active":"2024-05-09T06:29:34.841734+0000","last_peered":"2024-05-09T06:29:34.841734+0000","last_clean":"2024-05-09T06:29:34.841734+0000","last_became_active":"2024-05-09T06:29:33.859630+0000","last_became_peered":"2024-05-09T06:29:33.859630+0000","last_unstale":"2024-05-09T06:29:34.841734+0000","last_undegraded":"2024-05-09T06:29:34.841734+0000","last_fullsized":"2024-05-09T06:29:34.841734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:55.300623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475810+0000","last_change":"2024-05-09T06:29:33.868187+0000","last_active":"2024-05-09T06:29:35.475810+0000","last_peered":"2024-05-09T06:29:35.475810+0000","last_clean":"2024-05-09T06:29:35.475810+0000","last_became_active":"2024-05-09T06:29:33.867033+0000","last_became_peered":"2024-05-09T06:29:33.867033+0000","last_unstale":"2024-05-09T06:29:35.475810+0000","last_undegraded":"2024-05-09T06:29:35.475810+0000","last_fullsized":"2024-05-09T06:29:35.475810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:41.475551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476676+0000","last_change":"2024-05-09T06:29:33.858635+0000","last_active":"2024-05-09T06:29:35.476676+0000","last_peered":"2024-05-09T06:29:35.476676+0000","last_clean":"2024-05-09T06:29:35.476676+0000","last_became_active":"2024-05-09T06:29:33.858357+0000","last_became_peered":"2024-05-09T06:29:33.858357+0000","last_unstale":"2024-05-09T06:29:35.476676+0000","last_undegraded":"2024-05-09T06:29:35.476676+0000","last_fullsized":"2024-05-09T06:29:35.476676+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:10.424586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154807+0000","last_change":"2024-05-09T06:29:33.856819+0000","last_active":"2024-05-09T06:29:37.154807+0000","last_peered":"2024-05-09T06:29:37.154807+0000","last_clean":"2024-05-09T06:29:37.154807+0000","last_became_active":"2024-05-09T06:29:33.856519+0000","last_became_peered":"2024-05-09T06:29:33.856519+0000","last_unstale":"2024-05-09T06:29:37.154807+0000","last_undegraded":"2024-05-09T06:29:37.154807+0000","last_fullsized":"2024-05-09T06:29:37.154807+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:25.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154743+0000","last_change":"2024-05-09T06:29:33.857606+0000","last_active":"2024-05-09T06:29:37.154743+0000","last_peered":"2024-05-09T06:29:37.154743+0000","last_clean":"2024-05-09T06:29:37.154743+0000","last_became_active":"2024-05-09T06:29:33.857308+0000","last_became_peered":"2024-05-09T06:29:33.857308+0000","last_unstale":"2024-05-09T06:29:37.154743+0000","last_undegraded":"2024-05-09T06:29:37.154743+0000","last_fullsized":"2024-05-09T06:29:37.154743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624445+0000","last_change":"2024-05-09T06:29:33.866435+0000","last_active":"2024-05-09T06:29:35.624445+0000","last_peered":"2024-05-09T06:29:35.624445+0000","last_clean":"2024-05-09T06:29:35.624445+0000","last_became_active":"2024-05-09T06:29:33.866180+0000","last_became_peered":"2024-05-09T06:29:33.866180+0000","last_unstale":"2024-05-09T06:29:35.624445+0000","last_undegraded":"2024-05-09T06:29:35.624445+0000","last_fullsized":"2024-05-09T06:29:35.624445+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476173+0000","last_change":"2024-05-09T06:29:33.854059+0000","last_active":"2024-05-09T06:29:35.476173+0000","last_peered":"2024-05-09T06:29:35.476173+0000","last_clean":"2024-05-09T06:29:35.476173+0000","last_became_active":"2024-05-09T06:29:33.853274+0000","last_became_peered":"2024-05-09T06:29:33.853274+0000","last_unstale":"2024-05-09T06:29:35.476173+0000","last_undegraded":"2024-05-09T06:29:35.476173+0000","last_fullsized":"2024-05-09T06:29:35.476173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:02.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847083+0000","last_change":"2024-05-09T06:29:33.859781+0000","last_active":"2024-05-09T06:29:35.847083+0000","last_peered":"2024-05-09T06:29:35.847083+0000","last_clean":"2024-05-09T06:29:35.847083+0000","last_became_active":"2024-05-09T06:29:33.859387+0000","last_became_peered":"2024-05-09T06:29:33.859387+0000","last_unstale":"2024-05-09T06:29:35.847083+0000","last_undegraded":"2024-05-09T06:29:35.847083+0000","last_fullsized":"2024-05-09T06:29:35.847083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:49.415506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475911+0000","last_change":"2024-05-09T06:29:33.854017+0000","last_active":"2024-05-09T06:29:35.475911+0000","last_peered":"2024-05-09T06:29:35.475911+0000","last_clean":"2024-05-09T06:29:35.475911+0000","last_became_active":"2024-05-09T06:29:33.853732+0000","last_became_peered":"2024-05-09T06:29:33.853732+0000","last_unstale":"2024-05-09T06:29:35.475911+0000","last_undegraded":"2024-05-09T06:29:35.475911+0000","last_fullsized":"2024-05-09T06:29:35.475911+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:50.510790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154365+0000","last_change":"2024-05-09T06:29:33.850569+0000","last_active":"2024-05-09T06:29:37.154365+0000","last_peered":"2024-05-09T06:29:37.154365+0000","last_clean":"2024-05-09T06:29:37.154365+0000","last_became_active":"2024-05-09T06:29:33.849787+0000","last_became_peered":"2024-05-09T06:29:33.849787+0000","last_unstale":"2024-05-09T06:29:37.154365+0000","last_undegraded":"2024-05-09T06:29:37.154365+0000","last_fullsized":"2024-05-09T06:29:37.154365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:50.837895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846810+0000","last_change":"2024-05-09T06:29:33.851022+0000","last_active":"2024-05-09T06:29:35.846810+0000","last_peered":"2024-05-09T06:29:35.846810+0000","last_clean":"2024-05-09T06:29:35.846810+0000","last_became_active":"2024-05-09T06:29:33.850725+0000","last_became_peered":"2024-05-09T06:29:33.850725+0000","last_unstale":"2024-05-09T06:29:35.846810+0000","last_undegraded":"2024-05-09T06:29:35.846810+0000","last_fullsized":"2024-05-09T06:29:35.846810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624345+0000","last_change":"2024-05-09T06:29:32.843136+0000","last_active":"2024-05-09T06:29:35.624345+0000","last_peered":"2024-05-09T06:29:35.624345+0000","last_clean":"2024-05-09T06:29:35.624345+0000","last_became_active":"2024-05-09T06:29:32.842842+0000","last_became_peered":"2024-05-09T06:29:32.842842+0000","last_unstale":"2024-05-09T06:29:35.624345+0000","last_undegraded":"2024-05-09T06:29:35.624345+0000","last_fullsized":"2024-05-09T06:29:35.624345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154274+0000","last_change":"2024-05-09T06:29:33.855731+0000","last_active":"2024-05-09T06:29:37.154274+0000","last_peered":"2024-05-09T06:29:37.154274+0000","last_clean":"2024-05-09T06:29:37.154274+0000","last_became_active":"2024-05-09T06:29:33.854105+0000","last_became_peered":"2024-05-09T06:29:33.854105+0000","last_unstale":"2024-05-09T06:29:37.154274+0000","last_undegraded":"2024-05-09T06:29:37.154274+0000","last_fullsized":"2024-05-09T06:29:37.154274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:42.606599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154214+0000","last_change":"2024-05-09T06:29:32.842035+0000","last_active":"2024-05-09T06:29:37.154214+0000","last_peered":"2024-05-09T06:29:37.154214+0000","last_clean":"2024-05-09T06:29:37.154214+0000","last_became_active":"2024-05-09T06:29:32.841756+0000","last_became_peered":"2024-05-09T06:29:32.841756+0000","last_unstale":"2024-05-09T06:29:37.154214+0000","last_undegraded":"2024-05-09T06:29:37.154214+0000","last_fullsized":"2024-05-09T06:29:37.154214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624279+0000","last_change":"2024-05-09T06:29:33.851242+0000","last_active":"2024-05-09T06:29:35.624279+0000","last_peered":"2024-05-09T06:29:35.624279+0000","last_clean":"2024-05-09T06:29:35.624279+0000","last_became_active":"2024-05-09T06:29:33.850868+0000","last_became_peered":"2024-05-09T06:29:33.850868+0000","last_unstale":"2024-05-09T06:29:35.624279+0000","last_undegraded":"2024-05-09T06:29:35.624279+0000","last_fullsized":"2024-05-09T06:29:35.624279+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:50.081220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154698+0000","last_change":"2024-05-09T06:29:32.842788+0000","last_active":"2024-05-09T06:29:37.154698+0000","last_peered":"2024-05-09T06:29:37.154698+0000","last_clean":"2024-05-09T06:29:37.154698+0000","last_became_active":"2024-05-09T06:29:32.842461+0000","last_became_peered":"2024-05-09T06:29:32.842461+0000","last_unstale":"2024-05-09T06:29:37.154698+0000","last_undegraded":"2024-05-09T06:29:37.154698+0000","last_fullsized":"2024-05-09T06:29:37.154698+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:09.062369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153999+0000","last_change":"2024-05-09T06:29:33.856526+0000","last_active":"2024-05-09T06:29:37.153999+0000","last_peered":"2024-05-09T06:29:37.153999+0000","last_clean":"2024-05-09T06:29:37.153999+0000","last_became_active":"2024-05-09T06:29:33.855880+0000","last_became_peered":"2024-05-09T06:29:33.855880+0000","last_unstale":"2024-05-09T06:29:37.153999+0000","last_undegraded":"2024-05-09T06:29:37.153999+0000","last_fullsized":"2024-05-09T06:29:37.153999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:18.007333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154570+0000","last_change":"2024-05-09T06:29:32.845755+0000","last_active":"2024-05-09T06:29:37.154570+0000","last_peered":"2024-05-09T06:29:37.154570+0000","last_clean":"2024-05-09T06:29:37.154570+0000","last_became_active":"2024-05-09T06:29:32.845443+0000","last_became_peered":"2024-05-09T06:29:32.845443+0000","last_unstale":"2024-05-09T06:29:37.154570+0000","last_undegraded":"2024-05-09T06:29:37.154570+0000","last_fullsized":"2024-05-09T06:29:37.154570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:43.479453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843799+0000","last_change":"2024-05-09T06:29:33.856302+0000","last_active":"2024-05-09T06:29:34.843799+0000","last_peered":"2024-05-09T06:29:34.843799+0000","last_clean":"2024-05-09T06:29:34.843799+0000","last_became_active":"2024-05-09T06:29:33.856003+0000","last_became_peered":"2024-05-09T06:29:33.856003+0000","last_unstale":"2024-05-09T06:29:34.843799+0000","last_undegraded":"2024-05-09T06:29:34.843799+0000","last_fullsized":"2024-05-09T06:29:34.843799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:35.393848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476052+0000","last_change":"2024-05-09T06:29:32.831050+0000","last_active":"2024-05-09T06:29:35.476052+0000","last_peered":"2024-05-09T06:29:35.476052+0000","last_clean":"2024-05-09T06:29:35.476052+0000","last_became_active":"2024-05-09T06:29:32.830760+0000","last_became_peered":"2024-05-09T06:29:32.830760+0000","last_unstale":"2024-05-09T06:29:35.476052+0000","last_undegraded":"2024-05-09T06:29:35.476052+0000","last_fullsized":"2024-05-09T06:29:35.476052+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:51.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847294+0000","last_change":"2024-05-09T06:29:33.859705+0000","last_active":"2024-05-09T06:29:35.847294+0000","last_peered":"2024-05-09T06:29:35.847294+0000","last_clean":"2024-05-09T06:29:35.847294+0000","last_became_active":"2024-05-09T06:29:33.859407+0000","last_became_peered":"2024-05-09T06:29:33.859407+0000","last_unstale":"2024-05-09T06:29:35.847294+0000","last_undegraded":"2024-05-09T06:29:35.847294+0000","last_fullsized":"2024-05-09T06:29:35.847294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843564+0000","last_change":"2024-05-09T06:29:32.842910+0000","last_active":"2024-05-09T06:29:34.843564+0000","last_peered":"2024-05-09T06:29:34.843564+0000","last_clean":"2024-05-09T06:29:34.843564+0000","last_became_active":"2024-05-09T06:29:32.842534+0000","last_became_peered":"2024-05-09T06:29:32.842534+0000","last_unstale":"2024-05-09T06:29:34.843564+0000","last_undegraded":"2024-05-09T06:29:34.843564+0000","last_fullsized":"2024-05-09T06:29:34.843564+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:41.113803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154966+0000","last_change":"2024-05-09T06:29:33.869072+0000","last_active":"2024-05-09T06:29:37.154966+0000","last_peered":"2024-05-09T06:29:37.154966+0000","last_clean":"2024-05-09T06:29:37.154966+0000","last_became_active":"2024-05-09T06:29:33.868589+0000","last_became_peered":"2024-05-09T06:29:33.868589+0000","last_unstale":"2024-05-09T06:29:37.154966+0000","last_undegraded":"2024-05-09T06:29:37.154966+0000","last_fullsized":"2024-05-09T06:29:37.154966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:11.329525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624139+0000","last_change":"2024-05-09T06:29:32.840442+0000","last_active":"2024-05-09T06:29:35.624139+0000","last_peered":"2024-05-09T06:29:35.624139+0000","last_clean":"2024-05-09T06:29:35.624139+0000","last_became_active":"2024-05-09T06:29:32.840016+0000","last_became_peered":"2024-05-09T06:29:32.840016+0000","last_unstale":"2024-05-09T06:29:35.624139+0000","last_undegraded":"2024-05-09T06:29:35.624139+0000","last_fullsized":"2024-05-09T06:29:35.624139+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:36.447781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847453+0000","last_change":"2024-05-09T06:29:33.858289+0000","last_active":"2024-05-09T06:29:35.847453+0000","last_peered":"2024-05-09T06:29:35.847453+0000","last_clean":"2024-05-09T06:29:35.847453+0000","last_became_active":"2024-05-09T06:29:33.857996+0000","last_became_peered":"2024-05-09T06:29:33.857996+0000","last_unstale":"2024-05-09T06:29:35.847453+0000","last_undegraded":"2024-05-09T06:29:35.847453+0000","last_fullsized":"2024-05-09T06:29:35.847453+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:38.253814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.475849+0000","last_change":"2024-05-09T06:29:32.844658+0000","last_active":"2024-05-09T06:29:35.475849+0000","last_peered":"2024-05-09T06:29:35.475849+0000","last_clean":"2024-05-09T06:29:35.475849+0000","last_became_active":"2024-05-09T06:29:32.844008+0000","last_became_peered":"2024-05-09T06:29:32.844008+0000","last_unstale":"2024-05-09T06:29:35.475849+0000","last_undegraded":"2024-05-09T06:29:35.475849+0000","last_fullsized":"2024-05-09T06:29:35.475849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843296+0000","last_change":"2024-05-09T06:29:33.839883+0000","last_active":"2024-05-09T06:29:34.843296+0000","last_peered":"2024-05-09T06:29:34.843296+0000","last_clean":"2024-05-09T06:29:34.843296+0000","last_became_active":"2024-05-09T06:29:33.839523+0000","last_became_peered":"2024-05-09T06:29:33.839523+0000","last_unstale":"2024-05-09T06:29:34.843296+0000","last_undegraded":"2024-05-09T06:29:34.843296+0000","last_fullsized":"2024-05-09T06:29:34.843296+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:43.743388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847388+0000","last_change":"2024-05-09T06:29:32.845642+0000","last_active":"2024-05-09T06:29:35.847388+0000","last_peered":"2024-05-09T06:29:35.847388+0000","last_clean":"2024-05-09T06:29:35.847388+0000","last_became_active":"2024-05-09T06:29:32.845324+0000","last_became_peered":"2024-05-09T06:29:32.845324+0000","last_unstale":"2024-05-09T06:29:35.847388+0000","last_undegraded":"2024-05-09T06:29:35.847388+0000","last_fullsized":"2024-05-09T06:29:35.847388+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:34.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154470+0000","last_change":"2024-05-09T06:29:33.851428+0000","last_active":"2024-05-09T06:29:37.154470+0000","last_peered":"2024-05-09T06:29:37.154470+0000","last_clean":"2024-05-09T06:29:37.154470+0000","last_became_active":"2024-05-09T06:29:33.851141+0000","last_became_peered":"2024-05-09T06:29:33.851141+0000","last_unstale":"2024-05-09T06:29:37.154470+0000","last_undegraded":"2024-05-09T06:29:37.154470+0000","last_fullsized":"2024-05-09T06:29:37.154470+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:23.972099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843499+0000","last_change":"2024-05-09T06:29:32.847975+0000","last_active":"2024-05-09T06:29:34.843499+0000","last_peered":"2024-05-09T06:29:34.843499+0000","last_clean":"2024-05-09T06:29:34.843499+0000","last_became_active":"2024-05-09T06:29:32.847772+0000","last_became_peered":"2024-05-09T06:29:32.847772+0000","last_unstale":"2024-05-09T06:29:34.843499+0000","last_undegraded":"2024-05-09T06:29:34.843499+0000","last_fullsized":"2024-05-09T06:29:34.843499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:35.033218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476166+0000","last_change":"2024-05-09T06:29:33.857945+0000","last_active":"2024-05-09T06:29:35.476166+0000","last_peered":"2024-05-09T06:29:35.476166+0000","last_clean":"2024-05-09T06:29:35.476166+0000","last_became_active":"2024-05-09T06:29:33.857660+0000","last_became_peered":"2024-05-09T06:29:33.857660+0000","last_unstale":"2024-05-09T06:29:35.476166+0000","last_undegraded":"2024-05-09T06:29:35.476166+0000","last_fullsized":"2024-05-09T06:29:35.476166+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:42.541481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847372+0000","last_change":"2024-05-09T06:29:32.843594+0000","last_active":"2024-05-09T06:29:35.847372+0000","last_peered":"2024-05-09T06:29:35.847372+0000","last_clean":"2024-05-09T06:29:35.847372+0000","last_became_active":"2024-05-09T06:29:32.843233+0000","last_became_peered":"2024-05-09T06:29:32.843233+0000","last_unstale":"2024-05-09T06:29:35.847372+0000","last_undegraded":"2024-05-09T06:29:35.847372+0000","last_fullsized":"2024-05-09T06:29:35.847372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:10.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843727+0000","last_change":"2024-05-09T06:29:33.855808+0000","last_active":"2024-05-09T06:29:34.843727+0000","last_peered":"2024-05-09T06:29:34.843727+0000","last_clean":"2024-05-09T06:29:34.843727+0000","last_became_active":"2024-05-09T06:29:33.855489+0000","last_became_peered":"2024-05-09T06:29:33.855489+0000","last_unstale":"2024-05-09T06:29:34.843727+0000","last_undegraded":"2024-05-09T06:29:34.843727+0000","last_fullsized":"2024-05-09T06:29:34.843727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:52.735911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154732+0000","last_change":"2024-05-09T06:29:32.841265+0000","last_active":"2024-05-09T06:29:37.154732+0000","last_peered":"2024-05-09T06:29:37.154732+0000","last_clean":"2024-05-09T06:29:37.154732+0000","last_became_active":"2024-05-09T06:29:32.840989+0000","last_became_peered":"2024-05-09T06:29:32.840989+0000","last_unstale":"2024-05-09T06:29:37.154732+0000","last_undegraded":"2024-05-09T06:29:37.154732+0000","last_fullsized":"2024-05-09T06:29:37.154732+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:41.597314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154272+0000","last_change":"2024-05-09T06:29:33.851436+0000","last_active":"2024-05-09T06:29:37.154272+0000","last_peered":"2024-05-09T06:29:37.154272+0000","last_clean":"2024-05-09T06:29:37.154272+0000","last_became_active":"2024-05-09T06:29:33.851042+0000","last_became_peered":"2024-05-09T06:29:33.851042+0000","last_unstale":"2024-05-09T06:29:37.154272+0000","last_undegraded":"2024-05-09T06:29:37.154272+0000","last_fullsized":"2024-05-09T06:29:37.154272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:41.475551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476311+0000","last_change":"2024-05-09T06:29:32.842270+0000","last_active":"2024-05-09T06:29:35.476311+0000","last_peered":"2024-05-09T06:29:35.476311+0000","last_clean":"2024-05-09T06:29:35.476311+0000","last_became_active":"2024-05-09T06:29:32.841999+0000","last_became_peered":"2024-05-09T06:29:32.841999+0000","last_unstale":"2024-05-09T06:29:35.476311+0000","last_undegraded":"2024-05-09T06:29:35.476311+0000","last_fullsized":"2024-05-09T06:29:35.476311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:34.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476537+0000","last_change":"2024-05-09T06:29:33.863499+0000","last_active":"2024-05-09T06:29:35.476537+0000","last_peered":"2024-05-09T06:29:35.476537+0000","last_clean":"2024-05-09T06:29:35.476537+0000","last_became_active":"2024-05-09T06:29:33.862695+0000","last_became_peered":"2024-05-09T06:29:33.862695+0000","last_unstale":"2024-05-09T06:29:35.476537+0000","last_undegraded":"2024-05-09T06:29:35.476537+0000","last_fullsized":"2024-05-09T06:29:35.476537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:58.979878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624249+0000","last_change":"2024-05-09T06:29:32.846452+0000","last_active":"2024-05-09T06:29:35.624249+0000","last_peered":"2024-05-09T06:29:35.624249+0000","last_clean":"2024-05-09T06:29:35.624249+0000","last_became_active":"2024-05-09T06:29:32.846152+0000","last_became_peered":"2024-05-09T06:29:32.846152+0000","last_unstale":"2024-05-09T06:29:35.624249+0000","last_undegraded":"2024-05-09T06:29:35.624249+0000","last_fullsized":"2024-05-09T06:29:35.624249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154884+0000","last_change":"2024-05-09T06:29:33.848338+0000","last_active":"2024-05-09T06:29:37.154884+0000","last_peered":"2024-05-09T06:29:37.154884+0000","last_clean":"2024-05-09T06:29:37.154884+0000","last_became_active":"2024-05-09T06:29:33.848055+0000","last_became_peered":"2024-05-09T06:29:33.848055+0000","last_unstale":"2024-05-09T06:29:37.154884+0000","last_undegraded":"2024-05-09T06:29:37.154884+0000","last_fullsized":"2024-05-09T06:29:37.154884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:52.227907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846992+0000","last_change":"2024-05-09T06:29:32.840887+0000","last_active":"2024-05-09T06:29:35.846992+0000","last_peered":"2024-05-09T06:29:35.846992+0000","last_clean":"2024-05-09T06:29:35.846992+0000","last_became_active":"2024-05-09T06:29:32.840544+0000","last_became_peered":"2024-05-09T06:29:32.840544+0000","last_unstale":"2024-05-09T06:29:35.846992+0000","last_undegraded":"2024-05-09T06:29:35.846992+0000","last_fullsized":"2024-05-09T06:29:35.846992+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476605+0000","last_change":"2024-05-09T06:29:33.861924+0000","last_active":"2024-05-09T06:29:35.476605+0000","last_peered":"2024-05-09T06:29:35.476605+0000","last_clean":"2024-05-09T06:29:35.476605+0000","last_became_active":"2024-05-09T06:29:33.860239+0000","last_became_peered":"2024-05-09T06:29:33.860239+0000","last_unstale":"2024-05-09T06:29:35.476605+0000","last_undegraded":"2024-05-09T06:29:35.476605+0000","last_fullsized":"2024-05-09T06:29:35.476605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:52.227907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154408+0000","last_change":"2024-05-09T06:29:32.839558+0000","last_active":"2024-05-09T06:29:37.154408+0000","last_peered":"2024-05-09T06:29:37.154408+0000","last_clean":"2024-05-09T06:29:37.154408+0000","last_became_active":"2024-05-09T06:29:32.839220+0000","last_became_peered":"2024-05-09T06:29:32.839220+0000","last_unstale":"2024-05-09T06:29:37.154408+0000","last_undegraded":"2024-05-09T06:29:37.154408+0000","last_fullsized":"2024-05-09T06:29:37.154408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:48.406221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154324+0000","last_change":"2024-05-09T06:29:33.856259+0000","last_active":"2024-05-09T06:29:37.154324+0000","last_peered":"2024-05-09T06:29:37.154324+0000","last_clean":"2024-05-09T06:29:37.154324+0000","last_became_active":"2024-05-09T06:29:33.855452+0000","last_became_peered":"2024-05-09T06:29:33.855452+0000","last_unstale":"2024-05-09T06:29:37.154324+0000","last_undegraded":"2024-05-09T06:29:37.154324+0000","last_fullsized":"2024-05-09T06:29:37.154324+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:22.627476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476673+0000","last_change":"2024-05-09T06:29:32.847677+0000","last_active":"2024-05-09T06:29:35.476673+0000","last_peered":"2024-05-09T06:29:35.476673+0000","last_clean":"2024-05-09T06:29:35.476673+0000","last_became_active":"2024-05-09T06:29:32.847372+0000","last_became_peered":"2024-05-09T06:29:32.847372+0000","last_unstale":"2024-05-09T06:29:35.476673+0000","last_undegraded":"2024-05-09T06:29:35.476673+0000","last_fullsized":"2024-05-09T06:29:35.476673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:37.820825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153793+0000","last_change":"2024-05-09T06:29:33.856671+0000","last_active":"2024-05-09T06:29:37.153793+0000","last_peered":"2024-05-09T06:29:37.153793+0000","last_clean":"2024-05-09T06:29:37.153793+0000","last_became_active":"2024-05-09T06:29:33.856343+0000","last_became_peered":"2024-05-09T06:29:33.856343+0000","last_unstale":"2024-05-09T06:29:37.153793+0000","last_undegraded":"2024-05-09T06:29:37.153793+0000","last_fullsized":"2024-05-09T06:29:37.153793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:47.385123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841946+0000","last_change":"2024-05-09T06:29:32.842351+0000","last_active":"2024-05-09T06:29:34.841946+0000","last_peered":"2024-05-09T06:29:34.841946+0000","last_clean":"2024-05-09T06:29:34.841946+0000","last_became_active":"2024-05-09T06:29:32.842065+0000","last_became_peered":"2024-05-09T06:29:32.842065+0000","last_unstale":"2024-05-09T06:29:34.841946+0000","last_undegraded":"2024-05-09T06:29:34.841946+0000","last_fullsized":"2024-05-09T06:29:34.841946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:14.973970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843411+0000","last_change":"2024-05-09T06:29:33.856898+0000","last_active":"2024-05-09T06:29:34.843411+0000","last_peered":"2024-05-09T06:29:34.843411+0000","last_clean":"2024-05-09T06:29:34.843411+0000","last_became_active":"2024-05-09T06:29:33.856499+0000","last_became_peered":"2024-05-09T06:29:33.856499+0000","last_unstale":"2024-05-09T06:29:34.843411+0000","last_undegraded":"2024-05-09T06:29:34.843411+0000","last_fullsized":"2024-05-09T06:29:34.843411+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:37.440909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624340+0000","last_change":"2024-05-09T06:29:32.844205+0000","last_active":"2024-05-09T06:29:35.624340+0000","last_peered":"2024-05-09T06:29:35.624340+0000","last_clean":"2024-05-09T06:29:35.624340+0000","last_became_active":"2024-05-09T06:29:32.843719+0000","last_became_peered":"2024-05-09T06:29:32.843719+0000","last_unstale":"2024-05-09T06:29:35.624340+0000","last_undegraded":"2024-05-09T06:29:35.624340+0000","last_fullsized":"2024-05-09T06:29:35.624340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.153939+0000","last_change":"2024-05-09T06:29:33.860109+0000","last_active":"2024-05-09T06:29:37.153939+0000","last_peered":"2024-05-09T06:29:37.153939+0000","last_clean":"2024-05-09T06:29:37.153939+0000","last_became_active":"2024-05-09T06:29:33.859797+0000","last_became_peered":"2024-05-09T06:29:33.859797+0000","last_unstale":"2024-05-09T06:29:37.153939+0000","last_undegraded":"2024-05-09T06:29:37.153939+0000","last_fullsized":"2024-05-09T06:29:37.153939+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:10.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624480+0000","last_change":"2024-05-09T06:29:32.847048+0000","last_active":"2024-05-09T06:29:35.624480+0000","last_peered":"2024-05-09T06:29:35.624480+0000","last_clean":"2024-05-09T06:29:35.624480+0000","last_became_active":"2024-05-09T06:29:32.846701+0000","last_became_peered":"2024-05-09T06:29:32.846701+0000","last_unstale":"2024-05-09T06:29:35.624480+0000","last_undegraded":"2024-05-09T06:29:35.624480+0000","last_fullsized":"2024-05-09T06:29:35.624480+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:47.543123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154413+0000","last_change":"2024-05-09T06:29:33.852145+0000","last_active":"2024-05-09T06:29:37.154413+0000","last_peered":"2024-05-09T06:29:37.154413+0000","last_clean":"2024-05-09T06:29:37.154413+0000","last_became_active":"2024-05-09T06:29:33.851840+0000","last_became_peered":"2024-05-09T06:29:33.851840+0000","last_unstale":"2024-05-09T06:29:37.154413+0000","last_undegraded":"2024-05-09T06:29:37.154413+0000","last_fullsized":"2024-05-09T06:29:37.154413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:38.253814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154062+0000","last_change":"2024-05-09T06:29:32.840423+0000","last_active":"2024-05-09T06:29:37.154062+0000","last_peered":"2024-05-09T06:29:37.154062+0000","last_clean":"2024-05-09T06:29:37.154062+0000","last_became_active":"2024-05-09T06:29:32.840136+0000","last_became_peered":"2024-05-09T06:29:32.840136+0000","last_unstale":"2024-05-09T06:29:37.154062+0000","last_undegraded":"2024-05-09T06:29:37.154062+0000","last_fullsized":"2024-05-09T06:29:37.154062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:14.878187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624242+0000","last_change":"2024-05-09T06:29:33.850945+0000","last_active":"2024-05-09T06:29:35.624242+0000","last_peered":"2024-05-09T06:29:35.624242+0000","last_clean":"2024-05-09T06:29:35.624242+0000","last_became_active":"2024-05-09T06:29:33.850590+0000","last_became_peered":"2024-05-09T06:29:33.850590+0000","last_unstale":"2024-05-09T06:29:35.624242+0000","last_undegraded":"2024-05-09T06:29:35.624242+0000","last_fullsized":"2024-05-09T06:29:35.624242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:22.199322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154504+0000","last_change":"2024-05-09T06:29:32.839491+0000","last_active":"2024-05-09T06:29:37.154504+0000","last_peered":"2024-05-09T06:29:37.154504+0000","last_clean":"2024-05-09T06:29:37.154504+0000","last_became_active":"2024-05-09T06:29:32.839083+0000","last_became_peered":"2024-05-09T06:29:32.839083+0000","last_unstale":"2024-05-09T06:29:37.154504+0000","last_undegraded":"2024-05-09T06:29:37.154504+0000","last_fullsized":"2024-05-09T06:29:37.154504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.071935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476433+0000","last_change":"2024-05-09T06:29:33.853821+0000","last_active":"2024-05-09T06:29:35.476433+0000","last_peered":"2024-05-09T06:29:35.476433+0000","last_clean":"2024-05-09T06:29:35.476433+0000","last_became_active":"2024-05-09T06:29:33.853524+0000","last_became_peered":"2024-05-09T06:29:33.853524+0000","last_unstale":"2024-05-09T06:29:35.476433+0000","last_undegraded":"2024-05-09T06:29:35.476433+0000","last_fullsized":"2024-05-09T06:29:35.476433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:22.627476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476065+0000","last_change":"2024-05-09T06:29:32.840100+0000","last_active":"2024-05-09T06:29:35.476065+0000","last_peered":"2024-05-09T06:29:35.476065+0000","last_clean":"2024-05-09T06:29:35.476065+0000","last_became_active":"2024-05-09T06:29:32.839832+0000","last_became_peered":"2024-05-09T06:29:32.839832+0000","last_unstale":"2024-05-09T06:29:35.476065+0000","last_undegraded":"2024-05-09T06:29:35.476065+0000","last_fullsized":"2024-05-09T06:29:35.476065+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:36.431624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154350+0000","last_change":"2024-05-09T06:29:33.852908+0000","last_active":"2024-05-09T06:29:37.154350+0000","last_peered":"2024-05-09T06:29:37.154350+0000","last_clean":"2024-05-09T06:29:37.154350+0000","last_became_active":"2024-05-09T06:29:33.852553+0000","last_became_peered":"2024-05-09T06:29:33.852553+0000","last_unstale":"2024-05-09T06:29:37.154350+0000","last_undegraded":"2024-05-09T06:29:37.154350+0000","last_fullsized":"2024-05-09T06:29:37.154350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.841653+0000","last_change":"2024-05-09T06:29:32.832884+0000","last_active":"2024-05-09T06:29:34.841653+0000","last_peered":"2024-05-09T06:29:34.841653+0000","last_clean":"2024-05-09T06:29:34.841653+0000","last_became_active":"2024-05-09T06:29:32.832612+0000","last_became_peered":"2024-05-09T06:29:32.832612+0000","last_unstale":"2024-05-09T06:29:34.841653+0000","last_undegraded":"2024-05-09T06:29:34.841653+0000","last_fullsized":"2024-05-09T06:29:34.841653+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:09.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154528+0000","last_change":"2024-05-09T06:29:33.849812+0000","last_active":"2024-05-09T06:29:37.154528+0000","last_peered":"2024-05-09T06:29:37.154528+0000","last_clean":"2024-05-09T06:29:37.154528+0000","last_became_active":"2024-05-09T06:29:33.849351+0000","last_became_peered":"2024-05-09T06:29:33.849351+0000","last_unstale":"2024-05-09T06:29:37.154528+0000","last_undegraded":"2024-05-09T06:29:37.154528+0000","last_fullsized":"2024-05-09T06:29:37.154528+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:41.208467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624396+0000","last_change":"2024-05-09T06:29:32.846605+0000","last_active":"2024-05-09T06:29:35.624396+0000","last_peered":"2024-05-09T06:29:35.624396+0000","last_clean":"2024-05-09T06:29:35.624396+0000","last_became_active":"2024-05-09T06:29:32.846230+0000","last_became_peered":"2024-05-09T06:29:32.846230+0000","last_unstale":"2024-05-09T06:29:35.624396+0000","last_undegraded":"2024-05-09T06:29:35.624396+0000","last_fullsized":"2024-05-09T06:29:35.624396+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:42.734103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154070+0000","last_change":"2024-05-09T06:29:33.866676+0000","last_active":"2024-05-09T06:29:37.154070+0000","last_peered":"2024-05-09T06:29:37.154070+0000","last_clean":"2024-05-09T06:29:37.154070+0000","last_became_active":"2024-05-09T06:29:33.866409+0000","last_became_peered":"2024-05-09T06:29:33.866409+0000","last_unstale":"2024-05-09T06:29:37.154070+0000","last_undegraded":"2024-05-09T06:29:37.154070+0000","last_fullsized":"2024-05-09T06:29:37.154070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:09.864980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.847488+0000","last_change":"2024-05-09T06:29:32.844437+0000","last_active":"2024-05-09T06:29:35.847488+0000","last_peered":"2024-05-09T06:29:35.847488+0000","last_clean":"2024-05-09T06:29:35.847488+0000","last_became_active":"2024-05-09T06:29:32.843623+0000","last_became_peered":"2024-05-09T06:29:32.843623+0000","last_unstale":"2024-05-09T06:29:35.847488+0000","last_undegraded":"2024-05-09T06:29:35.847488+0000","last_fullsized":"2024-05-09T06:29:35.847488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:51.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154151+0000","last_change":"2024-05-09T06:29:33.840080+0000","last_active":"2024-05-09T06:29:37.154151+0000","last_peered":"2024-05-09T06:29:37.154151+0000","last_clean":"2024-05-09T06:29:37.154151+0000","last_became_active":"2024-05-09T06:29:33.839786+0000","last_became_peered":"2024-05-09T06:29:33.839786+0000","last_unstale":"2024-05-09T06:29:37.154151+0000","last_undegraded":"2024-05-09T06:29:37.154151+0000","last_fullsized":"2024-05-09T06:29:37.154151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:19.792349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476343+0000","last_change":"2024-05-09T06:29:32.845924+0000","last_active":"2024-05-09T06:29:35.476343+0000","last_peered":"2024-05-09T06:29:35.476343+0000","last_clean":"2024-05-09T06:29:35.476343+0000","last_became_active":"2024-05-09T06:29:32.845637+0000","last_became_peered":"2024-05-09T06:29:32.845637+0000","last_unstale":"2024-05-09T06:29:35.476343+0000","last_undegraded":"2024-05-09T06:29:35.476343+0000","last_fullsized":"2024-05-09T06:29:35.476343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:10.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476547+0000","last_change":"2024-05-09T06:29:33.852983+0000","last_active":"2024-05-09T06:29:35.476547+0000","last_peered":"2024-05-09T06:29:35.476547+0000","last_clean":"2024-05-09T06:29:35.476547+0000","last_became_active":"2024-05-09T06:29:33.852630+0000","last_became_peered":"2024-05-09T06:29:33.852630+0000","last_unstale":"2024-05-09T06:29:35.476547+0000","last_undegraded":"2024-05-09T06:29:35.476547+0000","last_fullsized":"2024-05-09T06:29:35.476547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:04.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154242+0000","last_change":"2024-05-09T06:29:32.841755+0000","last_active":"2024-05-09T06:29:37.154242+0000","last_peered":"2024-05-09T06:29:37.154242+0000","last_clean":"2024-05-09T06:29:37.154242+0000","last_became_active":"2024-05-09T06:29:32.840696+0000","last_became_peered":"2024-05-09T06:29:32.840696+0000","last_unstale":"2024-05-09T06:29:37.154242+0000","last_undegraded":"2024-05-09T06:29:37.154242+0000","last_fullsized":"2024-05-09T06:29:37.154242+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:08.855695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154325+0000","last_change":"2024-05-09T06:29:33.854703+0000","last_active":"2024-05-09T06:29:37.154325+0000","last_peered":"2024-05-09T06:29:37.154325+0000","last_clean":"2024-05-09T06:29:37.154325+0000","last_became_active":"2024-05-09T06:29:33.854388+0000","last_became_peered":"2024-05-09T06:29:33.854388+0000","last_unstale":"2024-05-09T06:29:37.154325+0000","last_undegraded":"2024-05-09T06:29:37.154325+0000","last_fullsized":"2024-05-09T06:29:37.154325+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:54:03.178493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154523+0000","last_change":"2024-05-09T06:29:32.841567+0000","last_active":"2024-05-09T06:29:37.154523+0000","last_peered":"2024-05-09T06:29:37.154523+0000","last_clean":"2024-05-09T06:29:37.154523+0000","last_became_active":"2024-05-09T06:29:32.841246+0000","last_became_peered":"2024-05-09T06:29:32.841246+0000","last_unstale":"2024-05-09T06:29:37.154523+0000","last_undegraded":"2024-05-09T06:29:37.154523+0000","last_fullsized":"2024-05-09T06:29:37.154523+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:09.775031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476381+0000","last_change":"2024-05-09T06:29:33.867290+0000","last_active":"2024-05-09T06:29:35.476381+0000","last_peered":"2024-05-09T06:29:35.476381+0000","last_clean":"2024-05-09T06:29:35.476381+0000","last_became_active":"2024-05-09T06:29:33.867012+0000","last_became_peered":"2024-05-09T06:29:33.867012+0000","last_unstale":"2024-05-09T06:29:35.476381+0000","last_undegraded":"2024-05-09T06:29:35.476381+0000","last_fullsized":"2024-05-09T06:29:35.476381+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:44.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:34.843231+0000","last_change":"2024-05-09T06:29:32.843603+0000","last_active":"2024-05-09T06:29:34.843231+0000","last_peered":"2024-05-09T06:29:34.843231+0000","last_clean":"2024-05-09T06:29:34.843231+0000","last_became_active":"2024-05-09T06:29:32.843314+0000","last_became_peered":"2024-05-09T06:29:32.843314+0000","last_unstale":"2024-05-09T06:29:34.843231+0000","last_undegraded":"2024-05-09T06:29:34.843231+0000","last_fullsized":"2024-05-09T06:29:34.843231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:45.744563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154996+0000","last_change":"2024-05-09T06:29:33.856782+0000","last_active":"2024-05-09T06:29:37.154996+0000","last_peered":"2024-05-09T06:29:37.154996+0000","last_clean":"2024-05-09T06:29:37.154996+0000","last_became_active":"2024-05-09T06:29:33.856081+0000","last_became_peered":"2024-05-09T06:29:33.856081+0000","last_unstale":"2024-05-09T06:29:37.154996+0000","last_undegraded":"2024-05-09T06:29:37.154996+0000","last_fullsized":"2024-05-09T06:29:37.154996+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:03.895859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:37.154526+0000","last_change":"2024-05-09T06:29:32.843469+0000","last_active":"2024-05-09T06:29:37.154526+0000","last_peered":"2024-05-09T06:29:37.154526+0000","last_clean":"2024-05-09T06:29:37.154526+0000","last_became_active":"2024-05-09T06:29:32.842907+0000","last_became_peered":"2024-05-09T06:29:32.842907+0000","last_unstale":"2024-05-09T06:29:37.154526+0000","last_undegraded":"2024-05-09T06:29:37.154526+0000","last_fullsized":"2024-05-09T06:29:37.154526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:03.213349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.624032+0000","last_change":"2024-05-09T06:29:33.852402+0000","last_active":"2024-05-09T06:29:35.624032+0000","last_peered":"2024-05-09T06:29:35.624032+0000","last_clean":"2024-05-09T06:29:35.624032+0000","last_became_active":"2024-05-09T06:29:33.852053+0000","last_became_peered":"2024-05-09T06:29:33.852053+0000","last_unstale":"2024-05-09T06:29:35.624032+0000","last_undegraded":"2024-05-09T06:29:35.624032+0000","last_fullsized":"2024-05-09T06:29:35.624032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:32.820766+0000","last_clean_scrub_stamp":"2024-05-09T06:29:32.820766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:04.222634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.476546+0000","last_change":"2024-05-09T06:29:32.843492+0000","last_active":"2024-05-09T06:29:35.476546+0000","last_peered":"2024-05-09T06:29:35.476546+0000","last_clean":"2024-05-09T06:29:35.476546+0000","last_became_active":"2024-05-09T06:29:32.842665+0000","last_became_peered":"2024-05-09T06:29:32.842665+0000","last_unstale":"2024-05-09T06:29:35.476546+0000","last_undegraded":"2024-05-09T06:29:35.476546+0000","last_fullsized":"2024-05-09T06:29:35.476546+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:29:31.811481+0000","last_clean_scrub_stamp":"2024-05-09T06:29:31.811481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:25.059490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-09T06:29:35.846757+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:29:35.846757+0000","last_peered":"2024-05-09T06:29:35.846757+0000","last_clean":"2024-05-09T06:29:35.846757+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:29:35.846757+0000","last_undegraded":"2024-05-09T06:29:35.846757+0000","last_fullsized":"2024-05-09T06:29:35.846757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1208320,"data_stored":860812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":270972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":270902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":271954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":271932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":614400,"data_stored":270480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":860744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:29:42.659 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:29:42.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:29:42.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.825+0000 7fabb4e38640 1 Processor -- start 2024-05-09T06:29:42.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.825+0000 7fabb4e38640 1 -- start start 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fabb01585d0 0x7fabb00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fabb00715d0 0x7fabb0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb0146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fabb01624a0 con 0x7fabb00715d0 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fabb0071aa0 con 0x7fabb01585d0 2024-05-09T06:29:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabb4e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fabb0071d80 con 0x7fabb0162090 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fabb00715d0 0x7fabb0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fabb00715d0 0x7fabb0161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40260/0 (socket says 172.21.15.1:40260) 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 -- 172.21.15.1:0/1729626136 learned_addr learned my addr 172.21.15.1:0/1729626136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 -- 172.21.15.1:0/1729626136 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 msgr2=0x7fabb0146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabae59c640 1 --2- 172.21.15.1:0/1729626136 >> v2:172.21.15.6:3300/0 conn(0x7fabb01585d0 0x7fabb00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7fabaed9d640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb0146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb0146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 -- 172.21.15.1:0/1729626136 >> v2:172.21.15.6:3300/0 conn(0x7fabb01585d0 msgr2=0x7fabb00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 --2- 172.21.15.1:0/1729626136 >> v2:172.21.15.6:3300/0 conn(0x7fabb01585d0 0x7fabb00b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.826+0000 7faba5d9b640 1 -- 172.21.15.1:0/1729626136 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb0072060 con 0x7fabb00715d0 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7faba5d9b640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb00715d0 0x7fabb0161b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b2a60 tx=0x7fab9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7826c38c91eee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7fabadd9b640 1 -- 172.21.15.1:0/1729626136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98037070 con 0x7fabb00715d0 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7fabadd9b640 1 -- 172.21.15.1:0/1729626136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab98032440 con 0x7fabb00715d0 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7fabadd9b640 1 -- 172.21.15.1:0/1729626136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98032810 con 0x7fabb00715d0 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7fabb4e38640 1 -- 172.21.15.1:0/1729626136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb00715d0 msgr2=0x7fabb0161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.827+0000 7fabb4e38640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb00715d0 0x7fabb0161b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b2a60 tx=0x7fab9802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 -- 172.21.15.1:0/1729626136 shutdown_connections 2024-05-09T06:29:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb0146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 --2- 172.21.15.1:0/1729626136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb00715d0 0x7fabb0161b50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 --2- 172.21.15.1:0/1729626136 >> v2:172.21.15.6:3300/0 conn(0x7fabb01585d0 0x7fabb00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 -- 172.21.15.1:0/1729626136 >> 172.21.15.1:0/1729626136 conn(0x7fabb00b1ba0 msgr2=0x7fabb00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 -- 172.21.15.1:0/1729626136 shutdown_connections 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.828+0000 7fabb4e38640 1 -- 172.21.15.1:0/1729626136 wait complete. 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.829+0000 7fabb4e38640 1 Processor -- start 2024-05-09T06:29:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.829+0000 7fabb4e38640 1 -- start start 2024-05-09T06:29:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabb00715d0 0x7fabb0153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb00cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01589b0 con 0x7fabb01585d0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0072620 con 0x7fabb00715d0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabb4e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fabb00b4190 con 0x7fabb0162090 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40274/0 (socket says 172.21.15.1:40274) 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 -- 172.21.15.1:0/505943781 learned_addr learned my addr 172.21.15.1:0/505943781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabaed9d640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb00cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7fabae59c640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabb00715d0 0x7fabb0153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 -- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 msgr2=0x7fabb00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb00cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 -- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabb00715d0 msgr2=0x7fabb0153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabb00715d0 0x7fabb0153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 -- 172.21.15.1:0/505943781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb006da50 con 0x7fabb01585d0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.830+0000 7faba5d9b640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.831+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98037040 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.831+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab98042070 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.831+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98030bb0 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.831+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb006dd10 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.831+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb006e2d0 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.832+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb0074610 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.832+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab98047460 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.833+0000 7faba77fe640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 0x7fab9006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.833+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fab980abc00 con 0x7fabb01585d0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.833+0000 7fabae59c640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 0x7fab9006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.833+0000 7fabae59c640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 0x7fab9006e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faba000b7c0 tx=0x7faba0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:42.837+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab980b2080 con 0x7fabb01585d0 2024-05-09T06:29:43.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.092+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fabb00752b0 con 0x7fabb01585d0 2024-05-09T06:29:43.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.094+0000 7faba77fe640 1 -- 172.21.15.1:0/505943781 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fab9807a420 con 0x7fabb01585d0 2024-05-09T06:29:43.076 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:29:43.076 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:29:43.076 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:29:43.077 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:29:43.077 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:29:43.077 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 msgr2=0x7fab9006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 0x7fab9006e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faba000b7c0 tx=0x7faba0008040 comp rx=0 tx=0).stop 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 msgr2=0x7fabb01494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab980046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 shutdown_connections 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fab9006be60 0x7fab9006e320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fabb0162090 0x7fabb00cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabb01585d0 0x7fabb01494a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 --2- 172.21.15.1:0/505943781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fabb00715d0 0x7fabb0153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 >> 172.21.15.1:0/505943781 conn(0x7fabb00b1ba0 msgr2=0x7fabb00b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 shutdown_connections 2024-05-09T06:29:43.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.097+0000 7fabb4e38640 1 -- 172.21.15.1:0/505943781 wait complete. 2024-05-09T06:29:43.090 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 Processor -- start 2024-05-09T06:29:43.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 -- start start 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2ec8161950 0x7f2ec80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec80b33c0 con 0x7f2ec80d4940 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec80d4dd0 con 0x7f2ec8161950 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.213+0000 7f2ece909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80b3660 con 0x7f2ec80d0f80 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ebffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2ec80d4940 0x7f2ec80cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38922/0 (socket says 172.21.15.1:38922) 2024-05-09T06:29:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ebffff640 1 -- 172.21.15.1:0/2838746172 learned_addr learned my addr 172.21.15.1:0/2838746172 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2eccea6640 1 --2- 172.21.15.1:0/2838746172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ec7fff640 1 --2- 172.21.15.1:0/2838746172 >> v2:172.21.15.6:3300/0 conn(0x7f2ec8161950 0x7f2ec80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860459070 0 0) 0x7f2ec80b33c0 con 0x7f2ec80d4940 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2eb4003520 con 0x7f2ec80d4940 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2256640759 0 0) 0x7f2eb4003520 con 0x7f2ec80d4940 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2eccea6640 1 -- 172.21.15.1:0/2838746172 >> v2:172.21.15.6:3300/0 conn(0x7f2ec8161950 msgr2=0x7f2ec80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2eccea6640 1 --2- 172.21.15.1:0/2838746172 >> v2:172.21.15.6:3300/0 conn(0x7f2ec8161950 0x7f2ec80d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2eccea6640 1 -- 172.21.15.1:0/2838746172 >> v1:172.21.15.1:6789/0 conn(0x7f2ec80d4940 legacy=0x7f2ec80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.214+0000 7f2eccea6640 1 -- 172.21.15.1:0/2838746172 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80b3940 con 0x7f2ec80d0f80 2024-05-09T06:29:43.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.215+0000 7f2eccea6640 1 --2- 172.21.15.1:0/2838746172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec80d2940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009d90 tx=0x7f2ec002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af38c2aab224bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.215+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0001670 con 0x7f2ec80d0f80 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.215+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0007540 con 0x7f2ec80d0f80 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.215+0000 7f2ec77fe640 1 -- 172.21.15.1:0/2838746172 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00078f0 con 0x7f2ec80d0f80 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 -- 172.21.15.1:0/2838746172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 msgr2=0x7f2ec80d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 --2- 172.21.15.1:0/2838746172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec80d2940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009d90 tx=0x7f2ec002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 -- 172.21.15.1:0/2838746172 shutdown_connections 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 --2- 172.21.15.1:0/2838746172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec80d2940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 --2- 172.21.15.1:0/2838746172 >> v2:172.21.15.6:3300/0 conn(0x7f2ec8161950 0x7f2ec80d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 -- 172.21.15.1:0/2838746172 >> 172.21.15.1:0/2838746172 conn(0x7f2ec80d3e20 msgr2=0x7f2ec80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 -- 172.21.15.1:0/2838746172 shutdown_connections 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.216+0000 7f2ece909640 1 -- 172.21.15.1:0/2838746172 wait complete. 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.217+0000 7f2ece909640 1 Processor -- start 2024-05-09T06:29:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.217+0000 7f2ece909640 1 -- start start 2024-05-09T06:29:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec81f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ec8161950 0x7f2ec8150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80b52d0 con 0x7f2ec81514c0 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80b5150 con 0x7f2ec8161950 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ece909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80b5450 con 0x7f2ec80d0f80 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40288/0 (socket says 172.21.15.1:40288) 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 -- 172.21.15.1:0/1266794976 learned_addr learned my addr 172.21.15.1:0/1266794976 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ec7fff640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec81f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ebffff640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ec8161950 0x7f2ec8150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 -- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 msgr2=0x7f2ec81f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec81f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 -- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ec8161950 msgr2=0x7f2ec8150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ec8161950 0x7f2ec8150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 -- 172.21.15.1:0/1266794976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81f74e0 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ec7fff640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec81f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2eccea6640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009eb0 tx=0x7f2ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.218+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0040070 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.219+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0031c40 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.219+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec80cc150 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.219+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0009450 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.219+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec80cc710 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.220+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec80752b0 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.220+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ec0039040 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.220+0000 7f2ec57fa640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 0x7f2ea806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.221+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f2ec00ac360 con 0x7f2ec81514c0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.221+0000 7f2ec7fff640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 0x7f2ea806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.221+0000 7f2ec7fff640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 0x7f2ea806e320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009af0 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.225+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ec007ac10 con 0x7f2ec81514c0 2024-05-09T06:29:43.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.445+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec8068580 con 0x7f2ec81514c0 2024-05-09T06:29:43.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.446+0000 7f2ec57fa640 1 -- 172.21.15.1:0/1266794976 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7f2ec007a510 con 0x7f2ec81514c0 2024-05-09T06:29:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:43.428 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:29:43.428 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 msgr2=0x7f2ea806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 0x7f2ea806e320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009af0 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 msgr2=0x7f2ec80b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009eb0 tx=0x7f2ec0004a40 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 shutdown_connections 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ea806be60 0x7f2ea806e320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ec81514c0 0x7f2ec80b93d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ec8161950 0x7f2ec8150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 --2- 172.21.15.1:0/1266794976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ec80d0f80 0x7f2ec81f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 >> 172.21.15.1:0/1266794976 conn(0x7f2ec80d3e20 msgr2=0x7f2ec81606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 shutdown_connections 2024-05-09T06:29:43.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.448+0000 7f2ece909640 1 -- 172.21.15.1:0/1266794976 wait complete. 2024-05-09T06:29:43.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:29:33.204942+0000', 'modified': '2024-05-09T06:29:36.218023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6045}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6045': {'gid': 6045, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.1:6836/3096552929', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 3096552929}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 3096552929}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6045, 'qdb_cluster': [6045]} 2024-05-09T06:29:43.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:29:43.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T06:29:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.969+0000 7f7db20ba640 1 Processor -- start 2024-05-09T06:29:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.969+0000 7f7db20ba640 1 -- start start 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.969+0000 7f7db20ba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7dac161950 0x7f7dac0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.969+0000 7f7db20ba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac0b33c0 con 0x7f7dac0d4940 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7db20ba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac0b3660 con 0x7f7dac0d0f80 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7db20ba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac0d4dd0 con 0x7f7dac161950 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7dac161950 0x7f7dac0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7dac161950 0x7f7dac0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40298/0 (socket says 172.21.15.1:40298) 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1819116496 learned_addr learned my addr 172.21.15.1:0/1819116496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7daaffd640 1 --1- 172.21.15.1:0/1819116496 >> v1:172.21.15.1:6789/0 conn(0x7f7dac0d4940 0x7f7dac0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38936/0 (socket says 172.21.15.1:38936) 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1819116496 >> v1:172.21.15.6:6789/0 conn(0x7f7dac0d0f80 legacy=0x7f7dac0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1819116496 >> v1:172.21.15.1:6789/0 conn(0x7f7dac0d4940 legacy=0x7f7dac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1819116496 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac0b3940 con 0x7f7dac161950 2024-05-09T06:29:43.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.970+0000 7f7daa7fc640 1 -- 172.21.15.1:0/1819116496 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955017541 0 0) 0x7f7dac0b33c0 con 0x7f7dac0d4940 2024-05-09T06:29:43.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/1819116496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac161950 0x7f7dac0d2400 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009870 tx=0x7f7d9c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c19f9586a30461 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7daa7fc640 1 -- 172.21.15.1:0/1819116496 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c037070 con 0x7f7dac161950 2024-05-09T06:29:43.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7daa7fc640 1 -- 172.21.15.1:0/1819116496 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c032440 con 0x7f7dac161950 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7daa7fc640 1 -- 172.21.15.1:0/1819116496 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c032810 con 0x7f7dac161950 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 -- 172.21.15.1:0/1819116496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac161950 msgr2=0x7f7dac0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 --2- 172.21.15.1:0/1819116496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac161950 0x7f7dac0d2400 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009870 tx=0x7f7d9c02a2a0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 -- 172.21.15.1:0/1819116496 shutdown_connections 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 --2- 172.21.15.1:0/1819116496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac161950 0x7f7dac0d2400 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 -- 172.21.15.1:0/1819116496 >> 172.21.15.1:0/1819116496 conn(0x7f7dac0d3e20 msgr2=0x7f7dac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.971+0000 7f7db20ba640 1 -- 172.21.15.1:0/1819116496 shutdown_connections 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.972+0000 7f7db20ba640 1 -- 172.21.15.1:0/1819116496 wait complete. 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.973+0000 7f7db20ba640 1 Processor -- start 2024-05-09T06:29:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.973+0000 7f7db20ba640 1 -- start start 2024-05-09T06:29:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.973+0000 7f7db20ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7dac161950 0x7f7dac0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dac1f3a40 0x7f7dac1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac0b5150 con 0x7f7dac0d0f80 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac0cf850 con 0x7f7dac1f3a40 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac0b52d0 con 0x7f7dac161950 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40308/0 (socket says 172.21.15.1:40308) 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 -- 172.21.15.1:0/3776825308 learned_addr learned my addr 172.21.15.1:0/3776825308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7daaffd640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7dac161950 0x7f7dac0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 -- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7dac161950 msgr2=0x7f7dac0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7dac161950 0x7f7dac0cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 -- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dac1f3a40 msgr2=0x7f7dac1f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dac1f3a40 0x7f7dac1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 -- 172.21.15.1:0/3776825308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac0cd530 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009870 tx=0x7f7d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7d93fff640 1 -- 172.21.15.1:0/3776825308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c041070 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7d93fff640 1 -- 172.21.15.1:0/3776825308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c03c870 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7d93fff640 1 -- 172.21.15.1:0/3776825308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c037040 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac0cd810 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.974+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1f5040 con 0x7f7dac0d0f80 2024-05-09T06:29:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.976+0000 7f7d93fff640 1 -- 172.21.15.1:0/3776825308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d9c044070 con 0x7f7dac0d0f80 2024-05-09T06:29:43.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.976+0000 7f7d93fff640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 0x7f7d9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.976+0000 7f7d93fff640 1 -- 172.21.15.1:0/3776825308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7d9c0abbe0 con 0x7f7dac0d0f80 2024-05-09T06:29:43.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.977+0000 7f7daaffd640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 0x7f7d9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.977+0000 7f7daaffd640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 0x7f7d9806e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0cee60 tx=0x7f7da0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.977+0000 7f7db4dc8740 1 Processor -- start 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.977+0000 7f7db4dc8740 1 -- start start 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc83e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc7f6350 0x5615fc83c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7594d0 con 0x5615fc820670 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7013e0 con 0x5615fc7f6350 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5615fc755490 con 0x5615fc83df10 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40322/0 (socket says 172.21.15.1:40322) 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.978+0000 7f7dab7fe640 1 -- 172.21.15.1:0/2936607042 learned_addr learned my addr 172.21.15.1:0/2936607042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7daaffd640 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc83e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dabfff640 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc7f6350 0x5615fc83c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 -- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 msgr2=0x5615fc83e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc83e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 -- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc7f6350 msgr2=0x5615fc83c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc7f6350 0x5615fc83c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 -- 172.21.15.1:0/2936607042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fc46d2d0 con 0x5615fc820670 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c035040 tx=0x7f7d9c079660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa9ad5388be31a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7d917fa640 1 -- 172.21.15.1:0/2936607042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c04b040 con 0x5615fc820670 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7d917fa640 1 -- 172.21.15.1:0/2936607042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c07e0b0 con 0x5615fc820670 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.979+0000 7f7d917fa640 1 -- 172.21.15.1:0/2936607042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0828e0 con 0x5615fc820670 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2936607042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 msgr2=0x5615fc820a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c035040 tx=0x7f7d9c079660 comp rx=0 tx=0).stop 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2936607042 shutdown_connections 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc7f6350 0x5615fc83c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc83e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2936607042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc820a50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2936607042 >> 172.21.15.1:0/2936607042 conn(0x5615fc7f2c80 msgr2=0x5615fc83b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2936607042 shutdown_connections 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2936607042 wait complete. 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.980+0000 7f7db4dc8740 1 -- start start 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 Processor -- start 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc821330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7f70f0 con 0x5615fc820670 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7148e0 con 0x5615fc839d00 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5615fc7f0bb0 con 0x5615fc83df10 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7db4dc8740 10 client.0 _ll_register_callbacks cb 0x5615fc6e19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40332/0 (socket says 172.21.15.1:40332) 2024-05-09T06:29:43.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1144952983 learned_addr learned my addr 172.21.15.1:0/1144952983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.981+0000 7f7daaffd640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc821330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dabfff640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 msgr2=0x5615fc821330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc821330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 msgr2=0x5615fc83a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 -- 172.21.15.1:0/1144952983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fc5376b0 con 0x5615fc820670 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dabfff640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c035040 tx=0x7f7d9c032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f77d0c9289f289 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c096060 con 0x5615fc820670 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7d9c082450 con 0x5615fc820670 2024-05-09T06:29:43.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d9c07e0b0 con 0x5615fc820670 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c04b040 con 0x5615fc820670 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7db4dc8740 10 client.6222 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x5615fc8220b0 con 0x5615fc820670 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c07e0b0 con 0x5615fc820670 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f7d9c079150 con 0x5615fc820670 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7db4dc8740 10 client.6222 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.982+0000 7f7db4dc8740 10 client.6222 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7db4dc8740 20 client.6222 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc823550 0x5615fc823930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7db4dc8740 4 client.6222 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615fc766c00 con 0x5615fc823550 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7daaffd640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc823550 0x5615fc823930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7d8a7fc640 10 client.6222 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:43.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.983+0000 7f7daaffd640 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc823550 0x5615fc823930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c63146be819d8e server_cookie=e4ca9b355000b923 in_seq=0 out_seq=0 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/1144952983 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615fc766c00 con 0x5615fc823550 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7d8a7fc640 10 client.6222 handle_command_reply: tid=0 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 client.6222 shutdown 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 msgr2=0x5615fc83cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c035040 tx=0x7f7d9c032f90 comp rx=0 tx=0).stop 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 shutdown_connections 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc823550 0x5615fc823930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7d8a7fc640 0 client.6222 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc821330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/1144952983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc83cae0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 >> 172.21.15.1:0/1144952983 conn(0x5615fc7f6350 msgr2=0x5615fc83f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 shutdown_connections 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 1 -- 172.21.15.1:0/1144952983 wait complete. 2024-05-09T06:29:43.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:43.984+0000 7f7db4dc8740 20 client.6222 trim_cache size 0 max 16384 2024-05-09T06:29:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.002+0000 7f7db4dc8740 1 Processor -- start 2024-05-09T06:29:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.002+0000 7f7db4dc8740 1 -- start start 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc820670 0x5615fc8551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac1ffb10 con 0x5615fc83df10 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac1feea0 con 0x5615fc839d00 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac1ff4c0 con 0x5615fc820670 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7dabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7dabfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40334/0 (socket says 172.21.15.1:40334) 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.003+0000 7f7dabfff640 1 -- 172.21.15.1:0/2328403721 learned_addr learned my addr 172.21.15.1:0/2328403721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc820670 0x5615fc8551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 -- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc820670 msgr2=0x5615fc8551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7daaffd640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc820670 0x5615fc8551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 -- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 msgr2=0x5615fc83aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 -- 172.21.15.1:0/2328403721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fc7f14c0 con 0x5615fc83df10 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7daaffd640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7dabfff640 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7da4005080 tx=0x7f7da4005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a5f2e0ab44d72b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7d90ff9640 1 -- 172.21.15.1:0/2328403721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4005790 con 0x5615fc83df10 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7d90ff9640 1 -- 172.21.15.1:0/2328403721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da4019070 con 0x5615fc83df10 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7d90ff9640 1 -- 172.21.15.1:0/2328403721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4001d30 con 0x5615fc83df10 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2328403721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 msgr2=0x5615fc84dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.004+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7da4005080 tx=0x7f7da4005540 comp rx=0 tx=0).stop 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2328403721 shutdown_connections 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc83df10 0x5615fc84dab0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc83aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/2328403721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc820670 0x5615fc8551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2328403721 >> 172.21.15.1:0/2328403721 conn(0x5615fc7f6350 msgr2=0x5615fc7f6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2328403721 shutdown_connections 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 -- 172.21.15.1:0/2328403721 wait complete. 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 -- start start 2024-05-09T06:29:43.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.005+0000 7f7db4dc8740 1 Processor -- start 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc80a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc80a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7f70f0 con 0x5615fc820670 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5615fc7f0bb0 con 0x5615fc839d00 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5615fc7148e0 con 0x5615fc83df10 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7db4dc8740 10 client.0 _ll_register_callbacks cb 0x7f7d9c079bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40342/0 (socket says 172.21.15.1:40342) 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 -- 172.21.15.1:0/489292869 learned_addr learned my addr 172.21.15.1:0/489292869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dabfff640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc80a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7daaffd640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc80a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 -- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 msgr2=0x5615fc80a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc80a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 -- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 msgr2=0x5615fc80a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc80a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7dab7fe640 1 -- 172.21.15.1:0/489292869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fc83b050 con 0x5615fc820670 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.006+0000 7f7daaffd640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc80a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7dab7fe640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0823b0 tx=0x7f7d9c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497f9b320b4ec471 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c07e0b0 con 0x5615fc820670 2024-05-09T06:29:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7d9c0802b0 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da4019070 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c0804b0 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 10 client.6228 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x5615fc83b050 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da4019070 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f7d9c082b30 con 0x5615fc820670 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 10 client.6228 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 10 client.6228 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 20 client.6228 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc83bc10 0x5615fc83bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 4 client.6228 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "status"}) -- 0x5615fc760f50 con 0x5615fc83bc10 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.007+0000 7f7daaffd640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc83bc10 0x5615fc83bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7daaffd640 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc83bc10 0x5615fc83bff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73aadd1a5e8a380d server_cookie=63baa59ed6b5a9db in_seq=0 out_seq=0 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7d8a7fc640 10 client.6228 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7d8a7fc640 1 -- 172.21.15.1:0/489292869 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5615fc760f50 con 0x5615fc83bc10 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7d8a7fc640 10 client.6228 handle_command_reply: tid=0 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 client.6228 shutdown 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 msgr2=0x5615fc8b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0823b0 tx=0x7f7d9c009240 comp rx=0 tx=0).stop 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 shutdown_connections 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x5615fc83bc10 0x5615fc83bff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5615fc83df10 0x5615fc80a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7d8a7fc640 0 client.6228 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5615fc839d00 0x5615fc80a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 --2- 172.21.15.1:0/489292869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc820670 0x5615fc8b4cd0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 >> 172.21.15.1:0/489292869 conn(0x5615fc7f6350 msgr2=0x5615fc834b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 shutdown_connections 2024-05-09T06:29:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 1 -- 172.21.15.1:0/489292869 wait complete. 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.008+0000 7f7db4dc8740 20 client.6228 trim_cache size 0 max 16384 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 6045, 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 10.16958977, 2024-05-09T06:29:43.997 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-05-09T06:29:43.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 51, 2024-05-09T06:29:43.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 51, 2024-05-09T06:29:43.998 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 15.171795709 2024-05-09T06:29:43.998 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 msgr2=0x7f7d9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 0x7f7d9806e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0cee60 tx=0x7f7da0009040 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 msgr2=0x7f7dac0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009870 tx=0x7f7d9c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 shutdown_connections 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7d9806be30 0x7f7d9806e2f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dac1f3a40 0x7f7dac1f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7dac161950 0x7f7dac0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 --2- 172.21.15.1:0/3776825308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dac0d0f80 0x7f7dac0cc950 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 >> 172.21.15.1:0/3776825308 conn(0x7f7dac0d3e20 msgr2=0x7f7dac0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 shutdown_connections 2024-05-09T06:29:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.009+0000 7f7db20ba640 1 -- 172.21.15.1:0/3776825308 wait complete. 2024-05-09T06:29:44.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:29:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.124+0000 7fedec909640 1 Processor -- start 2024-05-09T06:29:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.124+0000 7fedec909640 1 -- start start 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fedec909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fede806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fedec909640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede815eb40 con 0x7fede80ae2a0 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fedec909640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede80aef90 con 0x7fede815e050 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fedec909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fede80b3b80 con 0x7fede806e4f0 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede6d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fede80ae2a0 0x7fede80ae710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38950/0 (socket says 172.21.15.1:38950) 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede6d9d640 1 -- 172.21.15.1:0/2746789682 learned_addr learned my addr 172.21.15.1:0/2746789682 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fedddd9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fede806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874311611 0 0) 0x7fede815eb40 con 0x7fede80ae2a0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd4003520 con 0x7fede80ae2a0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1468203703 0 0) 0x7fedd4003520 con 0x7fede80ae2a0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 msgr2=0x7fede806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fede806e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 >> v1:172.21.15.6:6789/0 conn(0x7fede815e050 legacy=0x7fede815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80b3530 con 0x7fede80ae2a0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.125+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fede00019f0 con 0x7fede80ae2a0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 >> v1:172.21.15.1:6789/0 conn(0x7fede80ae2a0 legacy=0x7fede80ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd40039e0 0x7fedd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedd4004800 0x7fedd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fedd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede80b3b80 con 0x7fedd40039e0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fedd400baa0 con 0x7fedd4004800 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fedd400bca0 con 0x7fede806e4f0 2024-05-09T06:29:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd40039e0 0x7fedd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede6d9d640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fedd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fedddd9b640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedd4004800 0x7fedd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 -- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 msgr2=0x7fedd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede806e4f0 0x7fedd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 -- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedd4004800 msgr2=0x7fedd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedd4004800 0x7fedd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.126+0000 7fede659c640 1 -- 172.21.15.1:0/2746789682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd4003520 con 0x7fedd40039e0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fede659c640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd40039e0 0x7fedd4003e00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fedd8001720 tx=0x7fedd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327f6414c90a642b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8001370 con 0x7fedd40039e0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd80052d0 con 0x7fedd40039e0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fede5d9b640 1 -- 172.21.15.1:0/2746789682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd802ab90 con 0x7fedd40039e0 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 -- 172.21.15.1:0/2746789682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd40039e0 msgr2=0x7fedd4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 --2- 172.21.15.1:0/2746789682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd40039e0 0x7fedd4003e00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fedd8001720 tx=0x7fedd4005830 comp rx=0 tx=0).stop 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fede6d9d640 1 -- 172.21.15.1:0/2746789682 reap_dead start 2024-05-09T06:29:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 -- 172.21.15.1:0/2746789682 shutdown_connections 2024-05-09T06:29:44.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 -- 172.21.15.1:0/2746789682 >> 172.21.15.1:0/2746789682 conn(0x7fede806d270 msgr2=0x7fede806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:44.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 -- 172.21.15.1:0/2746789682 shutdown_connections 2024-05-09T06:29:44.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.127+0000 7fedec909640 1 -- 172.21.15.1:0/2746789682 wait complete. 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.129+0000 7fedec909640 1 Processor -- start 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.129+0000 7fedec909640 1 -- start start 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 0x7fede8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 0x7fede815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede806eb00 con 0x7fede81565d0 2024-05-09T06:29:44.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fede806e980 con 0x7fede8159d30 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fede806ec80 con 0x7fedd40039e0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40354/0 (socket says 172.21.15.1:40354) 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 -- 172.21.15.1:0/1245363802 learned_addr learned my addr 172.21.15.1:0/1245363802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fede6d9d640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 0x7fede815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fede659c640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 0x7fede8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 -- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 msgr2=0x7fede8155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 0x7fede8155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 -- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 msgr2=0x7fede815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 0x7fede815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 -- 172.21.15.1:0/1245363802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81e7240 con 0x7fede81565d0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fede6d9d640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 0x7fede815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fede659c640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 0x7fede8155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedddd9b640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fedd00119d0 tx=0x7fedd0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:44.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000fb50 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd000fcf0 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.130+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede80c5100 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.131+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000fea0 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.131+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81f42d0 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.132+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede81f40f0 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.132+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fedd0012040 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.132+0000 7feddf7fe640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 0x7fedc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.133+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fedd008b570 con 0x7fede81565d0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.133+0000 7fede659c640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 0x7fedc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:44.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.133+0000 7fede659c640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 0x7fedc806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedd8032000 tx=0x7fedd802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:44.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.137+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fede81f40f0 con 0x7fede81565d0 2024-05-09T06:29:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.358+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fede80752b0 con 0x7fede81565d0 2024-05-09T06:29:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.358+0000 7feddf7fe640 1 -- 172.21.15.1:0/1245363802 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fedd0019030 con 0x7fede81565d0 2024-05-09T06:29:44.340 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:29:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 msgr2=0x7fedc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 0x7fedc806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedd8032000 tx=0x7fedd802e000 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 msgr2=0x7fede81e6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fedd00119d0 tx=0x7fedd0011e90 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 shutdown_connections 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fedc806be60 0x7fedc806e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede8159d30 0x7fede815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede81565d0 0x7fede81e6d00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 --2- 172.21.15.1:0/1245363802 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fedd40039e0 0x7fede8155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 >> 172.21.15.1:0/1245363802 conn(0x7fede806d270 msgr2=0x7fede80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 shutdown_connections 2024-05-09T06:29:44.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44.360+0000 7fedec909640 1 -- 172.21.15.1:0/1245363802 wait complete. 2024-05-09T06:29:44.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:29:44.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:29:44.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:29:44.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:29:44.354 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:44.354 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:29:44.354 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:29:44.354 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:29:44.354 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:29:44.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.354 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:29:44.371 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42558sec preferred_lft 42558sec 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:29:44.372 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:29:44.373 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:29:44.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.374 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:29:44.436 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:29:44.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.438 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:29:44.495 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:29:44.495 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:29:44.495 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:29:44.495 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:29:44.496 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:29:44.496 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:29:44.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:29:44.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:29:44.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.625 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:29:44.641 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:29:44.641 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:29:44.641 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:29:44.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:29:44.708 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:29:44.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:29:44.708 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:44.765 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:29:44.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:29:44.765 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:44.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:29:44.888 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:29:44.945 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:29:44.945 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:29:44.945 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:29:44.945 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:29:44.945 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:29:44.946 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:29:44.947 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:29:44.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:44.948 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:29:45.001 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:29:45.002 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:29:45.044 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:29:45.070 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:29:45.128 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:29:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:29:45.130 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:29:45.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.131 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:29:45.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:29:45.148+0000 7f7de8c24480 -1 init, newargv = 0x557f73add220 newargc=15 2024-05-09T06:29:45.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[55766]: starting ceph client 2024-05-09T06:29:45.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[55766]: starting fuse 2024-05-09T06:29:45.149 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:29:45.149 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:29:45.150 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:29:45.151 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:29:45.152 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:29:45.270 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.55766.asok 2024-05-09T06:29:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:29:45.281 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55766.asok 2024-05-09T06:29:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55766.asok status 2024-05-09T06:29:45.419 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "55766", 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:29:45.420 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 6240, 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6240 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 785343863 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:29:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 785343863 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.6240 192.168.0.1:0/785343863", 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/785343863", 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 27, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 51, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:29:45.422 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:45.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.428 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:29:45.428 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:45.445 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:29:45.445 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:45.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.445 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:29:45.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.513 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-09T06:29:45.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:45.574 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-09T06:29:45.633 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:29:45.633 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:29:45.633 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:29:45.633 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:29:45.633 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-09T06:29:45.634 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:29:45.741 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:29:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:29:45.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:29:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.913+0000 7f2107fd5640 1 Processor -- start 2024-05-09T06:29:45.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.913+0000 7f2107fd5640 1 -- start start 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2107fd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f21000cf180 0x7f21000cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2107fd5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21000ce0e0 con 0x7f21000cdab0 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2107fd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21000ce3c0 con 0x7f210006e6d0 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2107fd5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f21000cdec0 con 0x7f21000cf180 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2106572640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f21000cdab0 0x7f21000b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38952/0 (socket says 172.21.15.1:38952) 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2106572640 1 -- 172.21.15.1:0/1036534076 learned_addr learned my addr 172.21.15.1:0/1036534076 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2105570640 1 --2- 172.21.15.1:0/1036534076 >> v2:172.21.15.6:3300/0 conn(0x7f21000cf180 0x7f21000cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416621708 0 0) 0x7f21000ce0e0 con 0x7f21000cdab0 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.914+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20ec003520 con 0x7f21000cdab0 2024-05-09T06:29:45.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338821813 0 0) 0x7f20ec003520 con 0x7f21000cdab0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 >> v2:172.21.15.6:3300/0 conn(0x7f21000cf180 msgr2=0x7f21000cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 --2- 172.21.15.1:0/1036534076 >> v2:172.21.15.6:3300/0 conn(0x7f21000cf180 0x7f21000cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 >> v1:172.21.15.6:6789/0 conn(0x7f210006e6d0 legacy=0x7f210006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21000ce6a0 con 0x7f21000cdab0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f20f80019f0 con 0x7f21000cdab0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 >> v1:172.21.15.1:6789/0 conn(0x7f21000cdab0 legacy=0x7f21000b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0039b0 0x7f20ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f20ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f20ec0051c0 0x7f20ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21000cdec0 con 0x7f20ec004800 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec00c060 con 0x7f20ec0039b0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec00c260 con 0x7f20ec0051c0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f20ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105d71640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0039b0 0x7f20ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2106572640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f20ec0051c0 0x7f20ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 -- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f20ec0051c0 msgr2=0x7f20ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f20ec0051c0 0x7f20ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 -- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0039b0 msgr2=0x7f20ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0039b0 0x7f20ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2105570640 1 -- 172.21.15.1:0/1036534076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec003520 con 0x7f20ec004800 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.915+0000 7f2106572640 1 -- 172.21.15.1:0/1036534076 reap_dead start 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.916+0000 7f2105570640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f20ec004c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009a90 tx=0x7f20ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6e039e0da8f039 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.916+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0036070 con 0x7f20ec004800 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.916+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f0031400 con 0x7f20ec004800 2024-05-09T06:29:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.916+0000 7f2104d6f640 1 -- 172.21.15.1:0/1036534076 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0009690 con 0x7f20ec004800 2024-05-09T06:29:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 -- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 msgr2=0x7f20ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f20ec004c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009a90 tx=0x7f20ec005830 comp rx=0 tx=0).stop 2024-05-09T06:29:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 -- 172.21.15.1:0/1036534076 shutdown_connections 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 --2- 172.21.15.1:0/1036534076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f20ec004c80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 -- 172.21.15.1:0/1036534076 >> 172.21.15.1:0/1036534076 conn(0x7f21000cc4b0 msgr2=0x7f21000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 -- 172.21.15.1:0/1036534076 shutdown_connections 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.917+0000 7f2107fd5640 1 -- 172.21.15.1:0/1036534076 wait complete. 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.918+0000 7f2107fd5640 1 Processor -- start 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.918+0000 7f2107fd5640 1 -- start start 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21001f4f30 0x7f210006d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f210013f040 0x7f210013f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2100070ed0 con 0x7f20ec004800 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f21000cec60 con 0x7f210013f040 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2107fd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f21001447c0 con 0x7f21001f4f30 2024-05-09T06:29:45.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40384/0 (socket says 172.21.15.1:40384) 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 -- 172.21.15.1:0/4093228044 learned_addr learned my addr 172.21.15.1:0/4093228044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 -- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21001f4f30 msgr2=0x7f210006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105570640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21001f4f30 0x7f210006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2106572640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f210013f040 0x7f210013f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21001f4f30 0x7f210006d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 -- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f210013f040 msgr2=0x7f210013f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f210013f040 0x7f210013f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 -- 172.21.15.1:0/4093228044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21000c4db0 con 0x7f20ec004800 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f2105d71640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00f5c0 tx=0x7f20fc00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0052e0 con 0x7f20ec004800 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.919+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc018070 con 0x7f20ec004800 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.920+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21000ada10 con 0x7f20ec004800 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.920+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc003830 con 0x7f20ec004800 2024-05-09T06:29:45.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.920+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21000adfd0 con 0x7f20ec004800 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.920+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21000752b0 con 0x7f20ec004800 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.921+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20fc0039d0 con 0x7f20ec004800 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.922+0000 7f20f67fc640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 0x7f20e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.922+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f20fc08ae40 con 0x7f20ec004800 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.922+0000 7f2105570640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 0x7f20e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.922+0000 7f2105570640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 0x7f20e406e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f20f0033000 tx=0x7f20f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:45.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:45.926+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20fc0595c0 con 0x7f20ec004800 2024-05-09T06:29:46.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.145+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210006cf50 con 0x7f20ec004800 2024-05-09T06:29:46.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.145+0000 7f20f67fc640 1 -- 172.21.15.1:0/4093228044 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7f20fc058ec0 con 0x7f20ec004800 2024-05-09T06:29:46.127 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:46.127 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:29:46.128 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:29:46.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 msgr2=0x7f20e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 0x7f20e406e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f20f0033000 tx=0x7f20f0008030 comp rx=0 tx=0).stop 2024-05-09T06:29:46.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 msgr2=0x7f21000ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00f5c0 tx=0x7f20fc00fa80 comp rx=0 tx=0).stop 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 shutdown_connections 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f20e406be30 0x7f20e406e2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f210013f040 0x7f210013f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f21001f4f30 0x7f210006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 --2- 172.21.15.1:0/4093228044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20ec004800 0x7f21000ae4f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 >> 172.21.15.1:0/4093228044 conn(0x7f21000cc4b0 msgr2=0x7f21000b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 shutdown_connections 2024-05-09T06:29:46.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.147+0000 7f2107fd5640 1 -- 172.21.15.1:0/4093228044 wait complete. 2024-05-09T06:29:46.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T06:29:46.267 INFO:teuthology.orchestra.run.smithi001.stdout:{"message":"","return_code":0} 2024-05-09T06:29:46.267 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:29:46.267 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-09T06:29:46.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.331+0000 7f59b37def40 1 Processor -- start 2024-05-09T06:29:46.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.331+0000 7f59b37def40 1 -- start start 2024-05-09T06:29:46.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f8aafda60 0x562f8aaf67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8aa597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562f8a9d3c80 con 0x7f59a00022a0 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562f8a9d7cc0 con 0x562f8aafda60 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b37def40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562f8aa74670 con 0x562f8ab04d70 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40404/0 (socket says 172.21.15.1:40404) 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 -- 172.21.15.1:0/11464608 learned_addr learned my addr 172.21.15.1:0/11464608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59abfff640 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f8aafda60 0x562f8aaf67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b1ae0640 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8aa597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 -- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 msgr2=0x562f8aa597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8aa597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 -- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f8aafda60 msgr2=0x562f8aaf67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f8aafda60 0x562f8aaf67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 -- 172.21.15.1:0/11464608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f8aafcf00 con 0x7f59a00022a0 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59b12df640 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59a40462a0 tx=0x7f59a404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbb26f9d10e97d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59ab7fe640 1 -- 172.21.15.1:0/11464608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a400a180 con 0x7f59a00022a0 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59ab7fe640 1 -- 172.21.15.1:0/11464608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a400a320 con 0x7f59a00022a0 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.332+0000 7f59ab7fe640 1 -- 172.21.15.1:0/11464608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40579d0 con 0x7f59a00022a0 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- 172.21.15.1:0/11464608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 msgr2=0x562f8aa2aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59a40462a0 tx=0x7f59a404f000 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- 172.21.15.1:0/11464608 shutdown_connections 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8aa597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f8aafda60 0x562f8aaf67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 --2- 172.21.15.1:0/11464608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59a00022a0 0x562f8aa2aee0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- 172.21.15.1:0/11464608 >> 172.21.15.1:0/11464608 conn(0x562f8aa2ab00 msgr2=0x562f8ab007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- 172.21.15.1:0/11464608 shutdown_connections 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- 172.21.15.1:0/11464608 wait complete. 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 Processor -- start 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.333+0000 7f59b37def40 1 -- start start 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 0x562f8aaf9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8ab00570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a00022a0 0x562f8aaf3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562f8aa2b420 con 0x562f8aafda60 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562f8aa5a360 con 0x7f59a00022a0 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b37def40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562f8aa737f0 con 0x562f8ab04d70 2024-05-09T06:29:46.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b12df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 0x562f8aaf9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b12df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 0x562f8aaf9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40406/0 (socket says 172.21.15.1:40406) 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b12df640 1 -- 172.21.15.1:0/570428650 learned_addr learned my addr 172.21.15.1:0/570428650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59b1ae0640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a00022a0 0x562f8aaf3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.334+0000 7f59abfff640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8ab00570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 -- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 msgr2=0x562f8ab00570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562f8ab04d70 0x562f8ab00570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 -- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a00022a0 msgr2=0x562f8aaf3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a00022a0 0x562f8aaf3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 -- 172.21.15.1:0/570428650 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f8ab00ab0 con 0x562f8aafda60 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b12df640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 0x562f8aaf9870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005630 tx=0x7f59a4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59aa7fc640 1 -- 172.21.15.1:0/570428650 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4071030 con 0x562f8aafda60 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59aa7fc640 1 -- 172.21.15.1:0/570428650 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4005330 con 0x562f8aafda60 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f8ab00d50 con 0x562f8aafda60 2024-05-09T06:29:46.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59aa7fc640 1 -- 172.21.15.1:0/570428650 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40577b0 con 0x562f8aafda60 2024-05-09T06:29:46.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.335+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f8aaf8d90 con 0x562f8aafda60 2024-05-09T06:29:46.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.336+0000 7f59aa7fc640 1 -- 172.21.15.1:0/570428650 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59a4074070 con 0x562f8aafda60 2024-05-09T06:29:46.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59aa7fc640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f599406a810 0x7f599406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59aa7fc640 1 -- 172.21.15.1:0/570428650 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f59a40cd9d0 con 0x562f8aafda60 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59abfff640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f599406a810 0x7f599406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59b37def40 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562f8aafc820 0x562f8ab01920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) -- 0x562f8ab01e60 con 0x562f8aafc820 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59b1ae0640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562f8aafc820 0x562f8ab01920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59abfff640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f599406a810 0x7f599406ccd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x562f8aaff9a0 tx=0x7f599c001580 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.337+0000 7f59b1ae0640 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562f8aafc820 0x562f8ab01920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b1ae0640 1 -- 172.21.15.1:0/570428650 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f59ac006bc0 con 0x562f8aafc820 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562f8aafc820 msgr2=0x562f8ab01920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562f8aafc820 0x562f8ab01920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f599406a810 msgr2=0x7f599406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f599406a810 0x7f599406ccd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x562f8aaff9a0 tx=0x7f599c001580 comp rx=0 tx=0).stop 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 msgr2=0x562f8aaf9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 --2- 172.21.15.1:0/570428650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f8aafda60 0x562f8aaf9870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005630 tx=0x7f59a4003020 comp rx=0 tx=0).stop 2024-05-09T06:29:46.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b1ae0640 1 -- 172.21.15.1:0/570428650 reap_dead start 2024-05-09T06:29:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 shutdown_connections 2024-05-09T06:29:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 >> 172.21.15.1:0/570428650 conn(0x562f8aa2ab00 msgr2=0x562f8aa597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 shutdown_connections 2024-05-09T06:29:46.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.338+0000 7f59b37def40 1 -- 172.21.15.1:0/570428650 wait complete. 2024-05-09T06:29:46.327 INFO:teuthology.orchestra.run.smithi001.stdout:Yalphaparent_a 2024-05-09T06:29:46.327 DEBUG:teuthology.orchestra.run.smithi001:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T06:29:46.425 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:46.425 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "ancestors": [ 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1099511627776, 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "alpha", 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1, 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "parent_a", 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T06:29:46.426 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 8, 2024-05-09T06:29:46.427 INFO:teuthology.orchestra.run.smithi001.stdout: "old_pools": [] 2024-05-09T06:29:46.427 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:46.427 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:46.443 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-09T06:29:46.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.507+0000 7fce2a7b3f40 1 Processor -- start 2024-05-09T06:29:46.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.507+0000 7fce2a7b3f40 1 -- start start 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.507+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.507+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59db30f0 0x562c59d87940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.507+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59dd1440 0x562c59dca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562c59d30c80 con 0x562c59d8b450 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562c59dd1820 con 0x562c59dd1440 2024-05-09T06:29:46.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562c59e54250 con 0x562c59db30f0 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40430/0 (socket says 172.21.15.1:40430) 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 -- 172.21.15.1:0/4106358587 learned_addr learned my addr 172.21.15.1:0/4106358587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce28ab5640 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59dd1440 0x562c59dca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce27ab3640 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59db30f0 0x562c59d87940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 -- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59db30f0 msgr2=0x562c59d87940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59db30f0 0x562c59d87940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 -- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59dd1440 msgr2=0x562c59dca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59dd1440 0x562c59dca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 -- 172.21.15.1:0/4106358587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c59cb5cb0 con 0x562c59d8b450 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce282b4640 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fce180095d0 tx=0x7fce18009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2131684ae0a6ea61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce26ab1640 1 -- 172.21.15.1:0/4106358587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18006680 con 0x562c59d8b450 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce26ab1640 1 -- 172.21.15.1:0/4106358587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18006820 con 0x562c59d8b450 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.508+0000 7fce26ab1640 1 -- 172.21.15.1:0/4106358587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18006c10 con 0x562c59d8b450 2024-05-09T06:29:46.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/4106358587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 msgr2=0x562c59dcf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fce180095d0 tx=0x7fce18009a90 comp rx=0 tx=0).stop 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/4106358587 shutdown_connections 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59dd1440 0x562c59dca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59db30f0 0x562c59d87940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/4106358587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dcf5a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/4106358587 >> 172.21.15.1:0/4106358587 conn(0x562c59db57c0 msgr2=0x562c59db8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/4106358587 shutdown_connections 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/4106358587 wait complete. 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.509+0000 7fce2a7b3f40 1 Processor -- start 2024-05-09T06:29:46.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 -- start start 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dc9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59db30f0 0x562c59dc9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59dd1440 0x562c59dca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562c59e53f70 con 0x562c59d8b450 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562c59db4f20 con 0x562c59db30f0 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce2a7b3f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562c59dcfef0 con 0x562c59dd1440 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce282b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dc9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce282b4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dc9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40440/0 (socket says 172.21.15.1:40440) 2024-05-09T06:29:46.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce282b4640 1 -- 172.21.15.1:0/1861477132 learned_addr learned my addr 172.21.15.1:0/1861477132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce27ab3640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59db30f0 0x562c59dc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.510+0000 7fce28ab5640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59dd1440 0x562c59dca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 -- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59dd1440 msgr2=0x562c59dca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562c59dd1440 0x562c59dca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 -- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59db30f0 msgr2=0x562c59dc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562c59db30f0 0x562c59dc9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 -- 172.21.15.1:0/1861477132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c59ef6ff0 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce282b4640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dc9110 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fce18009ad0 tx=0x7fce18003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce25aaf640 1 -- 172.21.15.1:0/1861477132 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18020070 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce25aaf640 1 -- 172.21.15.1:0/1861477132 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1801c640 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c59ef72d0 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce25aaf640 1 -- 172.21.15.1:0/1861477132 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18006680 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.511+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c59ef7890 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce25aaf640 1 -- 172.21.15.1:0/1861477132 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce18023070 con 0x562c59d8b450 2024-05-09T06:29:46.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce25aaf640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce1406a810 0x7fce1406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce25aaf640 1 -- 172.21.15.1:0/1861477132 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fce1808b2c0 con 0x562c59d8b450 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce27ab3640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce1406a810 0x7fce1406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562c59ef9e90 0x562c59e51af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) -- 0x562c59e52030 con 0x562c59ef9e90 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.513+0000 7fce28ab5640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562c59ef9e90 0x562c59e51af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.514+0000 7fce27ab3640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce1406a810 0x7fce1406ccd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce1c057000 tx=0x7fce1c052000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.514+0000 7fce28ab5640 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562c59ef9e90 0x562c59e51af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce28ab5640 1 -- 172.21.15.1:0/1861477132 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fce2000d8b0 con 0x562c59ef9e90 2024-05-09T06:29:46.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562c59ef9e90 msgr2=0x562c59e51af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x562c59ef9e90 0x562c59e51af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce1406a810 msgr2=0x7fce1406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce1406a810 0x7fce1406ccd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce1c057000 tx=0x7fce1c052000 comp rx=0 tx=0).stop 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 msgr2=0x562c59dc9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 --2- 172.21.15.1:0/1861477132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c59d8b450 0x562c59dc9110 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fce18009ad0 tx=0x7fce18003020 comp rx=0 tx=0).stop 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce28ab5640 1 -- 172.21.15.1:0/1861477132 reap_dead start 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 shutdown_connections 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 >> 172.21.15.1:0/1861477132 conn(0x562c59db57c0 msgr2=0x562c59db8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 shutdown_connections 2024-05-09T06:29:46.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.515+0000 7fce2a7b3f40 1 -- 172.21.15.1:0/1861477132 wait complete. 2024-05-09T06:29:46.500 DEBUG:teuthology.orchestra.run.smithi001:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-09T06:29:46.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:46.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-09T06:29:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.628+0000 7f7b71010640 1 Processor -- start 2024-05-09T06:29:46.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 -- start start 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b6c0b2ed0 0x7f7b6c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c15e870 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c0d1cc0 con 0x7f7b6c0b2ed0 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.629+0000 7f7b71010640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0d4d10 con 0x7f7b6c0d2630 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40456/0 (socket says 172.21.15.1:40456) 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 -- 172.21.15.1:0/3063391874 learned_addr learned my addr 172.21.15.1:0/3063391874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/3063391874 >> v2:172.21.15.6:3300/0 conn(0x7f7b6c0b2ed0 0x7f7b6c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 -- 172.21.15.1:0/3063391874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 msgr2=0x7f7b6c15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 -- 172.21.15.1:0/3063391874 >> v2:172.21.15.6:3300/0 conn(0x7f7b6c0b2ed0 msgr2=0x7f7b6c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/3063391874 >> v2:172.21.15.6:3300/0 conn(0x7f7b6c0b2ed0 0x7f7b6c15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.630+0000 7f7b6b59e640 1 -- 172.21.15.1:0/3063391874 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c0d1f80 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c08c520 tx=0x7f7b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f97d36e7352244 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b69d9b640 1 -- 172.21.15.1:0/3063391874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037070 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b69d9b640 1 -- 172.21.15.1:0/3063391874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64032440 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b69d9b640 1 -- 172.21.15.1:0/3063391874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64032810 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 -- 172.21.15.1:0/3063391874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 msgr2=0x7f7b6c0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c08c520 tx=0x7f7b6402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 -- 172.21.15.1:0/3063391874 shutdown_connections 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c0b1100 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 --2- 172.21.15.1:0/3063391874 >> v2:172.21.15.6:3300/0 conn(0x7f7b6c0b2ed0 0x7f7b6c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 --2- 172.21.15.1:0/3063391874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c15ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 -- 172.21.15.1:0/3063391874 >> 172.21.15.1:0/3063391874 conn(0x7f7b6c0d3ee0 msgr2=0x7f7b6c0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 -- 172.21.15.1:0/3063391874 shutdown_connections 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.631+0000 7f7b71010640 1 -- 172.21.15.1:0/3063391874 wait complete. 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.633+0000 7f7b71010640 1 Processor -- start 2024-05-09T06:29:46.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.633+0000 7f7b71010640 1 -- start start 2024-05-09T06:29:46.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 0x7f7b6c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c0b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0b1640 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0d4a00 con 0x7f7b6c0b2ed0 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b71010640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0b17c0 con 0x7f7b6c0d2630 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40464/0 (socket says 172.21.15.1:40464) 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 -- 172.21.15.1:0/4071957067 learned_addr learned my addr 172.21.15.1:0/4071957067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c0b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 0x7f7b6c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 -- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 msgr2=0x7f7b6c0b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c0b92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 -- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 msgr2=0x7f7b6c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 0x7f7b6c158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 -- 172.21.15.1:0/4071957067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c158cb0 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 0x7f7b6c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c0b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.634+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402a220 tx=0x7f7b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.635+0000 7f7b537fe640 1 -- 172.21.15.1:0/4071957067 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037040 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.635+0000 7f7b537fe640 1 -- 172.21.15.1:0/4071957067 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64042070 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.635+0000 7f7b537fe640 1 -- 172.21.15.1:0/4071957067 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64030bb0 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.635+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c0b4e60 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.635+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c0b5420 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.637+0000 7f7b537fe640 1 -- 172.21.15.1:0/4071957067 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b64047460 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b537fe640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 0x7f7b5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 Processor -- start 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b537fe640 1 -- 172.21.15.1:0/4071957067 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7b640abc80 con 0x7f7b6c0d2a10 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 0x7f7b5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 -- start start 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 0x7f7b5806e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007f50 tx=0x7f7b5c0074a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 0x560a16024d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a1604f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a15f88990 con 0x560a160249b0 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560a15f305b0 con 0x560a16061080 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.638+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560a15f84950 con 0x560a1604f470 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40468/0 (socket says 172.21.15.1:40468) 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 -- 172.21.15.1:0/2450771117 learned_addr learned my addr 172.21.15.1:0/2450771117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a1604f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 0x560a16024d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 -- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 msgr2=0x560a1604f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a1604f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 -- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 msgr2=0x560a16024d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 0x560a16024d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 -- 172.21.15.1:0/2450771117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a15d647e0 con 0x560a160249b0 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 0x560a16024d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e730 tx=0x7f7b5c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352eabdac16f7828 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b517fa640 1 -- 172.21.15.1:0/2450771117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00baf0 con 0x560a160249b0 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b517fa640 1 -- 172.21.15.1:0/2450771117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c00bc90 con 0x560a160249b0 2024-05-09T06:29:46.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.639+0000 7f7b517fa640 1 -- 172.21.15.1:0/2450771117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c01b660 con 0x560a160249b0 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 -- 172.21.15.1:0/2450771117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 msgr2=0x560a16026b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e730 tx=0x7f7b5c010000 comp rx=0 tx=0).stop 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 -- 172.21.15.1:0/2450771117 shutdown_connections 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a1604f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a16061080 0x560a16024d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2450771117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a160249b0 0x560a16026b10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 -- 172.21.15.1:0/2450771117 >> 172.21.15.1:0/2450771117 conn(0x560a15fec0a0 msgr2=0x560a1606a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 -- 172.21.15.1:0/2450771117 shutdown_connections 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.640+0000 7f7b73df4740 1 -- 172.21.15.1:0/2450771117 wait complete. 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.641+0000 7f7b73df4740 1 -- start start 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.641+0000 7f7b73df4740 1 Processor -- start 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a16065f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a16066950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a15f43b40 con 0x560a16061080 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560a1604cfc0 con 0x560a160249b0 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560a1603c3d0 con 0x560a1604f470 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 10 client.0 _ll_register_callbacks cb 0x560a15f380f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40482/0 (socket says 172.21.15.1:40482) 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 -- 172.21.15.1:0/376685663 learned_addr learned my addr 172.21.15.1:0/376685663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a16066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a16065f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 -- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 msgr2=0x560a16066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a16066950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 -- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 msgr2=0x560a16065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a16065f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 -- 172.21.15.1:0/376685663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a15dc4f70 con 0x560a16061080 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a16066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a16065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b640aa020 tx=0x7f7b64079eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b33bdce7741d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640a8060 con 0x560a16061080 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7b64076410 con 0x560a16061080 2024-05-09T06:29:46.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5c00bc90 con 0x560a16061080 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b64096050 con 0x560a16061080 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 10 client.6276 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.642+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x560a15c5c170 con 0x560a16061080 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c00bc90 con 0x560a16061080 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f7b6403a030 con 0x560a16061080 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 10 client.6276 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 10 client.6276 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 20 client.6276 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a1604fc60 0x560a1603b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 4 client.6276 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a15f960c0 con 0x560a1604fc60 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a1604fc60 0x560a1603b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a1604fc60 0x560a1603b480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=727acee86f628661 server_cookie=3fa878259c2f74f in_seq=0 out_seq=0 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.643+0000 7f7b49ffb640 10 client.6276 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b49ffb640 1 -- 172.21.15.1:0/376685663 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a15f960c0 con 0x560a1604fc60 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b49ffb640 10 client.6276 handle_command_reply: tid=0 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 client.6276 shutdown 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 msgr2=0x560a1603a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b640aa020 tx=0x7f7b64079eb0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 shutdown_connections 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a1604fc60 0x560a1603b480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a7c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a1604f470 0x560a16066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 --2- 172.21.15.1:0/376685663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a16065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b49ffb640 0 client.6276 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 >> 172.21.15.1:0/376685663 conn(0x560a15fec0a0 msgr2=0x560a1604c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 shutdown_connections 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 1 -- 172.21.15.1:0/376685663 wait complete. 2024-05-09T06:29:46.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.644+0000 7f7b73df4740 20 client.6276 trim_cache size 0 max 16384 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.649+0000 7f366f638640 1 Processor -- start 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.650+0000 7f366f638640 1 -- start start 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.650+0000 7f366f638640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f366806dce0 0x7f36680cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366f638640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36680d1630 con 0x7f366806e7d0 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366f638640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36680d1910 con 0x7f366815c160 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366f638640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f36680cd3e0 con 0x7f366806dce0 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366d3d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f366806dce0 0x7f36680cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366dbd5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f366806e7d0 0x7f366806b4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38958/0 (socket says 172.21.15.1:38958) 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366d3d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f366806dce0 0x7f36680cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40484/0 (socket says 172.21.15.1:40484) 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366dbd5640 1 -- 172.21.15.1:0/4076192883 learned_addr learned my addr 172.21.15.1:0/4076192883 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366d3d4640 1 -- 172.21.15.1:0/4076192883 >> v1:172.21.15.6:6789/0 conn(0x7f366815c160 legacy=0x7f366806e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366d3d4640 1 -- 172.21.15.1:0/4076192883 >> v1:172.21.15.1:6789/0 conn(0x7f366806e7d0 legacy=0x7f366806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.651+0000 7f366d3d4640 1 -- 172.21.15.1:0/4076192883 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36680ccd90 con 0x7f366806dce0 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366d3d4640 1 --2- 172.21.15.1:0/4076192883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366806dce0 0x7f36680cb160 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f365c0097a0 tx=0x7f365c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ee4c9ee01a46aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f365bfff640 1 -- 172.21.15.1:0/4076192883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c037070 con 0x7f366806dce0 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f365bfff640 1 -- 172.21.15.1:0/4076192883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c032440 con 0x7f366806dce0 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f365bfff640 1 -- 172.21.15.1:0/4076192883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c02ae60 con 0x7f366806dce0 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366f638640 1 -- 172.21.15.1:0/4076192883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366806dce0 msgr2=0x7f36680cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366f638640 1 --2- 172.21.15.1:0/4076192883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366806dce0 0x7f36680cb160 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f365c0097a0 tx=0x7f365c02a110 comp rx=0 tx=0).stop 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366f638640 1 -- 172.21.15.1:0/4076192883 shutdown_connections 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366f638640 1 --2- 172.21.15.1:0/4076192883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366806dce0 0x7f36680cb160 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.652+0000 7f366f638640 1 -- 172.21.15.1:0/4076192883 >> 172.21.15.1:0/4076192883 conn(0x7f36680d0b10 msgr2=0x7f36680d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.653+0000 7f366f638640 1 -- 172.21.15.1:0/4076192883 shutdown_connections 2024-05-09T06:29:46.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.653+0000 7f366f638640 1 -- 172.21.15.1:0/4076192883 wait complete. 2024-05-09T06:29:46.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.654+0000 7f366f638640 1 Processor -- start 2024-05-09T06:29:46.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.654+0000 7f366f638640 1 -- start start 2024-05-09T06:29:46.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366806dce0 0x7f36680c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f366806e7d0 0x7f3668148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366815c160 0x7f3668148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36680cc9b0 con 0x7f366815c160 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f36680cd070 con 0x7f366806dce0 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f36680ccb30 con 0x7f366806e7d0 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366815c160 0x7f3668148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366815c160 0x7f3668148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40492/0 (socket says 172.21.15.1:40492) 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366d3d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366806dce0 0x7f36680c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 -- 172.21.15.1:0/1647380541 learned_addr learned my addr 172.21.15.1:0/1647380541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366cbd3640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f366806e7d0 0x7f3668148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 -- 172.21.15.1:0/1647380541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f366806e7d0 msgr2=0x7f3668148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f366806e7d0 0x7f3668148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 -- 172.21.15.1:0/1647380541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366806dce0 msgr2=0x7f36680c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f366806dce0 0x7f36680c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 -- 172.21.15.1:0/1647380541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36680c74f0 con 0x7f366815c160 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366dbd5640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f366815c160 0x7f3668148f40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f366000cbf0 tx=0x7f366000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660017070 con 0x7f366815c160 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366000ad40 con 0x7f366815c160 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f366f638640 1 -- 172.21.15.1:0/1647380541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3668155e90 con 0x7f366815c160 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.655+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660012600 con 0x7f366815c160 2024-05-09T06:29:46.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.656+0000 7f366f638640 1 -- 172.21.15.1:0/1647380541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36680c8df0 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.656+0000 7f366f638640 1 -- 172.21.15.1:0/1647380541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f36680752b0 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.657+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f366001a070 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f3659ffb640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f364c06be30 0x7f364c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f366008ac80 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f366d3d4640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f364c06be30 0x7f364c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f36680c74f0 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f366f638640 1 -- 172.21.15.1:0/1647380541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f366815dd70 con 0x7f366815c160 2024-05-09T06:29:46.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.658+0000 7f366d3d4640 1 --2- 172.21.15.1:0/1647380541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f364c06be30 0x7f364c06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f365c034000 tx=0x7f365c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.663+0000 7f7b73df4740 1 Processor -- start 2024-05-09T06:29:46.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.663+0000 7f7b73df4740 1 -- start start 2024-05-09T06:29:46.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a160249b0 0x560a1601ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a1604f470 0x560a16052560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c200b10 con 0x560a16061080 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1ffc70 con 0x560a1604f470 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c1f8470 con 0x560a160249b0 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b6b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40494/0 (socket says 172.21.15.1:40494) 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b6b59e640 1 -- 172.21.15.1:0/2676095472 learned_addr learned my addr 172.21.15.1:0/2676095472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b6b59e640 1 -- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a160249b0 msgr2=0x560a1601ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.664+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a160249b0 0x560a1601ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b6b59e640 1 -- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a1604f470 msgr2=0x560a16052560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a1604f470 0x560a16052560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b6b59e640 1 -- 172.21.15.1:0/2676095472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a160522f0 con 0x560a16061080 2024-05-09T06:29:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7b64079d30 tx=0x7f7b6407c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60082623ee1947 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b50ff9640 1 -- 172.21.15.1:0/2676095472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6407d070 con 0x560a16061080 2024-05-09T06:29:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b50ff9640 1 -- 172.21.15.1:0/2676095472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64032e50 con 0x560a16061080 2024-05-09T06:29:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b50ff9640 1 -- 172.21.15.1:0/2676095472 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64045400 con 0x560a16061080 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b73df4740 1 -- 172.21.15.1:0/2676095472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 msgr2=0x560a1603a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.665+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7b64079d30 tx=0x7f7b6407c000 comp rx=0 tx=0).stop 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 -- 172.21.15.1:0/2676095472 shutdown_connections 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a16061080 0x560a1603a590 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a1604f470 0x560a16052560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 --2- 172.21.15.1:0/2676095472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a160249b0 0x560a1601ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 -- 172.21.15.1:0/2676095472 >> 172.21.15.1:0/2676095472 conn(0x560a15fec0a0 msgr2=0x560a16051be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 -- 172.21.15.1:0/2676095472 shutdown_connections 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 -- 172.21.15.1:0/2676095472 wait complete. 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 -- start start 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.666+0000 7f7b73df4740 1 Processor -- start 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a1603ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a16061080 0x560a160397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560a16020310 con 0x560a1604f470 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0b8730 con 0x560a160249b0 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560a16064ac0 con 0x560a16061080 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b73df4740 10 client.0 _ll_register_callbacks cb 0x7f7b64079b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40504/0 (socket says 172.21.15.1:40504) 2024-05-09T06:29:46.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 -- 172.21.15.1:0/1791041911 learned_addr learned my addr 172.21.15.1:0/1791041911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a1603ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6b59e640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a16061080 0x560a160397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 -- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a16061080 msgr2=0x560a160397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a16061080 0x560a160397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 -- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 msgr2=0x560a1603ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a1603ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.667+0000 7f7b6a59c640 1 -- 172.21.15.1:0/1791041911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1606d550 con 0x560a1604f470 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b6a59c640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c15a240 tx=0x7f7b60011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c351fda86a83a366 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60018030 con 0x560a1604f470 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f7b60013670 con 0x560a1604f470 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b64032e50 con 0x560a1604f470 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b60024050 con 0x560a1604f470 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 10 client.6291 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:46.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x560a1607d230 con 0x560a1604f470 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64032e50 con 0x560a1604f470 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f7b60027020 con 0x560a1604f470 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 10 client.6291 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 10 client.6291 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 20 client.6291 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a16039dd0 0x560a160e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 4 client.6291 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x560a15f90410 con 0x560a16039dd0 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.668+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a16039dd0 0x560a160e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b49ffb640 10 client.6291 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b6ad9d640 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a16039dd0 0x560a160e35e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab8c441e5da43fb server_cookie=c3b3a6c7004963c1 in_seq=0 out_seq=0 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b49ffb640 1 -- 172.21.15.1:0/1791041911 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a15f90410 con 0x560a16039dd0 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b49ffb640 10 client.6291 handle_command_reply: tid=0 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b73df4740 1 client.6291 shutdown 2024-05-09T06:29:46.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 msgr2=0x560a1603bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c15a240 tx=0x7f7b60011a50 comp rx=0 tx=0).stop 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.669+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 shutdown_connections 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x560a16039dd0 0x560a160e35e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560a16061080 0x560a160397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a1604f470 0x560a1603bf60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b49ffb640 0 client.6291 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 --2- 172.21.15.1:0/1791041911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560a160249b0 0x560a1603ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 >> 172.21.15.1:0/1791041911 conn(0x560a15fec0a0 msgr2=0x560a16052560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 shutdown_connections 2024-05-09T06:29:46.656 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f1f5082e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6240=0-4194304@1} caps={6240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6240 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ed483c2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 1 -- 172.21.15.1:0/1791041911 wait complete. 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b73df4740 20 client.6291 trim_cache size 0 max 16384 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "59d07d5a-723e-4fe0-951f-9602c328a4c9", 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:29:46.657 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:46.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 msgr2=0x7f7b5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 0x7f7b5806e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007f50 tx=0x7f7b5c0074a0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 msgr2=0x7f7b6c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402a220 tx=0x7f7b640046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 shutdown_connections 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b5806be30 0x7f7b5806e2f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b6c0d2a10 0x7f7b6c15a450 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b6c0d2630 0x7f7b6c0b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 --2- 172.21.15.1:0/4071957067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b6c0b2ed0 0x7f7b6c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 >> 172.21.15.1:0/4071957067 conn(0x7f7b6c0d3ee0 msgr2=0x7f7b6c0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 shutdown_connections 2024-05-09T06:29:46.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.670+0000 7f7b71010640 1 -- 172.21.15.1:0/4071957067 wait complete. 2024-05-09T06:29:46.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:29:46.663 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-09T06:29:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 Processor -- start 2024-05-09T06:29:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 -- start start 2024-05-09T06:29:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 0x7f498c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f498c0cf990 0x7f498c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498c15e900 con 0x7f498c0b1730 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f498c0b1ba0 con 0x7f498c0cf990 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.788+0000 7f4991038640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f498c15ebe0 con 0x7f498c15e390 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f498c0cf990 0x7f498c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f498c0cf990 0x7f498c0b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40516/0 (socket says 172.21.15.1:40516) 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 -- 172.21.15.1:0/3657240282 learned_addr learned my addr 172.21.15.1:0/3657240282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498ad9d640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 0x7f498c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 -- 172.21.15.1:0/3657240282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 msgr2=0x7f498c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 0x7f498c0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 -- 172.21.15.1:0/3657240282 >> v1:172.21.15.1:6789/0 conn(0x7f498c0b1730 legacy=0x7f498c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498a59c640 1 -- 172.21.15.1:0/3657240282 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c15eec0 con 0x7f498c0cf990 2024-05-09T06:29:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.789+0000 7f498ad9d640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 0x7f498c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f498a59c640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c0cf990 0x7f498c0b11f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f497c009870 tx=0x7f497c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e0d58e9848fe0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4989d9b640 1 -- 172.21.15.1:0/3657240282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c037070 con 0x7f498c0cf990 2024-05-09T06:29:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4989d9b640 1 -- 172.21.15.1:0/3657240282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c032440 con 0x7f498c0cf990 2024-05-09T06:29:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4989d9b640 1 -- 172.21.15.1:0/3657240282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c032810 con 0x7f498c0cf990 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 -- 172.21.15.1:0/3657240282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c0cf990 msgr2=0x7f498c0b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c0cf990 0x7f498c0b11f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f497c009870 tx=0x7f497c02a270 comp rx=0 tx=0).stop 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 -- 172.21.15.1:0/3657240282 shutdown_connections 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c0cf990 0x7f498c0b11f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 --2- 172.21.15.1:0/3657240282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c15e390 0x7f498c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 -- 172.21.15.1:0/3657240282 >> 172.21.15.1:0/3657240282 conn(0x7f498c0b1da0 msgr2=0x7f498c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 -- 172.21.15.1:0/3657240282 shutdown_connections 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.790+0000 7f4991038640 1 -- 172.21.15.1:0/3657240282 wait complete. 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.792+0000 7f4991038640 1 Processor -- start 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.792+0000 7f4991038640 1 -- start start 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 0x7f498c1e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 0x7f498c1e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0cd8f0 con 0x7f498c15e390 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f498c15f6c0 con 0x7f498c0cf990 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f498c162ba0 con 0x7f498c0b1730 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40518/0 (socket says 172.21.15.1:40518) 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 -- 172.21.15.1:0/2869095980 learned_addr learned my addr 172.21.15.1:0/2869095980 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498a59c640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 0x7f498c1e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498ad9d640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 0x7f498c1e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 -- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 msgr2=0x7f498c1e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 0x7f498c1e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 -- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 msgr2=0x7f498c1e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 0x7f498c1e92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 -- 172.21.15.1:0/2869095980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c1ea100 con 0x7f498c15e390 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498ad9d640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 0x7f498c1e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498a59c640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 0x7f498c1e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f498b59e640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f498400c7a0 tx=0x7f498400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f49837fe640 1 -- 172.21.15.1:0/2869095980 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984003b70 con 0x7f498c15e390 2024-05-09T06:29:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f49837fe640 1 -- 172.21.15.1:0/2869095980 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984003d30 con 0x7f498c15e390 2024-05-09T06:29:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1ea3e0 con 0x7f498c15e390 2024-05-09T06:29:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.793+0000 7f49837fe640 1 -- 172.21.15.1:0/2869095980 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400aa90 con 0x7f498c15e390 2024-05-09T06:29:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.794+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1533a0 con 0x7f498c15e390 2024-05-09T06:29:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.795+0000 7f49837fe640 1 -- 172.21.15.1:0/2869095980 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f498400ac30 con 0x7f498c15e390 2024-05-09T06:29:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.795+0000 7f49837fe640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 0x7f496c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.796+0000 7f49837fe640 1 -- 172.21.15.1:0/2869095980 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f498408aeb0 con 0x7f498c15e390 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.796+0000 7f498ad9d640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 0x7f496c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.796+0000 7f498ad9d640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 0x7f496c06e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4974006fd0 tx=0x7f4974008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.796+0000 7f4993e28740 1 Processor -- start 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 -- start start 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464a64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a95770 0x55a464a94af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a4649ced00 con 0x55a464a95030 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a4649764c0 con 0x55a464a95770 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.797+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a4649cacc0 con 0x55a464ab2650 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40530/0 (socket says 172.21.15.1:40530) 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 -- 172.21.15.1:0/909696290 learned_addr learned my addr 172.21.15.1:0/909696290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498a59c640 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a95770 0x55a464a94af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498ad9d640 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464a64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 -- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 msgr2=0x55a464a64fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464a64fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 -- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a95770 msgr2=0x55a464a94af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a95770 0x55a464a94af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 -- 172.21.15.1:0/909696290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4647ac7d0 con 0x55a464a95030 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f498b59e640 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4984061710 tx=0x7f49840589c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5f730cb57903ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f4980ff9640 1 -- 172.21.15.1:0/909696290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498405d410 con 0x55a464a95030 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f4980ff9640 1 -- 172.21.15.1:0/909696290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984061c40 con 0x55a464a95030 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.798+0000 7f4980ff9640 1 -- 172.21.15.1:0/909696290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984010dc0 con 0x55a464a95030 2024-05-09T06:29:46.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 -- 172.21.15.1:0/909696290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 msgr2=0x55a464a69e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4984061710 tx=0x7f49840589c0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 -- 172.21.15.1:0/909696290 shutdown_connections 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95030 0x55a464a69e40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a95770 0x55a464a94af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 --2- 172.21.15.1:0/909696290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464a64fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 -- 172.21.15.1:0/909696290 >> 172.21.15.1:0/909696290 conn(0x55a464a67250 msgr2=0x55a464aafaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 -- 172.21.15.1:0/909696290 shutdown_connections 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.799+0000 7f4993e28740 1 -- 172.21.15.1:0/909696290 wait complete. 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 -- start start 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 Processor -- start 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 0x55a464a968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a85a60 0x55a464ab1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a464a69400 con 0x55a464a95770 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a464989b90 con 0x55a464ab2650 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a464a687f0 con 0x55a464a85a60 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.800+0000 7f4993e28740 10 client.0 _ll_register_callbacks cb 0x55a4649c5da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40546/0 (socket says 172.21.15.1:40546) 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 -- 172.21.15.1:0/1560365645 learned_addr learned my addr 172.21.15.1:0/1560365645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498a59c640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 0x55a464a968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498b59e640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a85a60 0x55a464ab1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 -- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a85a60 msgr2=0x55a464ab1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a85a60 0x55a464ab1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 -- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 msgr2=0x55a464a968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 0x55a464a968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 -- 172.21.15.1:0/1560365645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4646e23f0 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498a59c640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 0x55a464a968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f497400ef80 tx=0x7f49740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58271c42fec5d563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974025020 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4974015d50 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4984061c40 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f497402f050 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4993e28740 10 client.6306 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.801+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55a4646a20f0 con 0x55a464a95770 2024-05-09T06:29:46.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984061c40 con 0x55a464a95770 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f4974032020 con 0x55a464a95770 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 10 client.6306 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 10 client.6306 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 20 client.6306 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464a97cb0 0x55a464a980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 4 client.6306 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4649dc430 con 0x55a464a97cb0 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f498a59c640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464a97cb0 0x55a464a980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f498a59c640 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464a97cb0 0x55a464a980f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb6f06dafe9cd9c1 server_cookie=12c5fb8bbcf3af61 in_seq=0 out_seq=0 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.802+0000 7f4969ffb640 10 client.6306 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4969ffb640 1 -- 172.21.15.1:0/1560365645 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4649dc430 con 0x55a464a97cb0 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4969ffb640 10 client.6306 handle_command_reply: tid=0 2024-05-09T06:29:46.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 client.6306 shutdown 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 msgr2=0x55a464aa71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f497400ef80 tx=0x7f49740046c0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 shutdown_connections 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464a97cb0 0x55a464a980f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a85a60 0x55a464ab1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464ab2650 0x55a464a968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4969ffb640 0 client.6306 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 --2- 172.21.15.1:0/1560365645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464aa71c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 >> 172.21.15.1:0/1560365645 conn(0x55a464a95030 msgr2=0x55a464a65e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 shutdown_connections 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 1 -- 172.21.15.1:0/1560365645 wait complete. 2024-05-09T06:29:46.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.803+0000 7f4993e28740 20 client.6306 trim_cache size 0 max 16384 2024-05-09T06:29:46.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.821+0000 7f4993e28740 1 Processor -- start 2024-05-09T06:29:46.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.821+0000 7f4993e28740 1 -- start start 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464aaf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f498c200e10 con 0x55a464a95770 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1ffe50 con 0x55a464a85a60 2024-05-09T06:29:46.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f498c0af490 con 0x55a464ab2650 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40560/0 (socket says 172.21.15.1:40560) 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 -- 172.21.15.1:0/856102746 learned_addr learned my addr 172.21.15.1:0/856102746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498ad9d640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498b59e640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464aaf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 -- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 msgr2=0x55a464aaf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464aaf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 -- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 msgr2=0x55a464a8fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 -- 172.21.15.1:0/856102746 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a464ab2010 con 0x55a464a95770 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498ad9d640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498b59e640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464aaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f498a59c640 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1fa41f86244f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f496bfff640 1 -- 172.21.15.1:0/856102746 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c037040 con 0x55a464a95770 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f496bfff640 1 -- 172.21.15.1:0/856102746 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c046dd0 con 0x55a464a95770 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.822+0000 7f496bfff640 1 -- 172.21.15.1:0/856102746 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c0465f0 con 0x55a464a95770 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- 172.21.15.1:0/856102746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 msgr2=0x55a464a90410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c02c000 comp rx=0 tx=0).stop 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- 172.21.15.1:0/856102746 shutdown_connections 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464ab2650 0x55a464aaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464a95770 0x55a464a90410 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 --2- 172.21.15.1:0/856102746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- 172.21.15.1:0/856102746 >> 172.21.15.1:0/856102746 conn(0x55a464a95030 msgr2=0x55a464a60670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- 172.21.15.1:0/856102746 shutdown_connections 2024-05-09T06:29:46.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- 172.21.15.1:0/856102746 wait complete. 2024-05-09T06:29:46.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 -- start start 2024-05-09T06:29:46.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.823+0000 7f4993e28740 1 Processor -- start 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a95770 0x55a464ab05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a464ab3f40 con 0x55a464ab2650 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0afae0 con 0x55a464a85a60 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a464a64de0 con 0x55a464a95770 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f4993e28740 10 client.0 _ll_register_callbacks cb 0x7f4974009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:29:46.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f498b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40564/0 (socket says 172.21.15.1:40564) 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.824+0000 7f498b59e640 1 -- 172.21.15.1:0/1481228492 learned_addr learned my addr 172.21.15.1:0/1481228492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 -- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a95770 msgr2=0x55a464ab05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498a59c640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a95770 0x55a464ab05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a95770 0x55a464ab05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 -- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 msgr2=0x55a464a8fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 -- 172.21.15.1:0/1481228492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a464a90990 con 0x55a464ab2650 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498b59e640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f498400a430 tx=0x7f498401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69550774f598395 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:29:46.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498405d410 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4984057100 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497c046dd0 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4984061c40 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 10 client.6312 fetch_fsmap learned FSMap version 27 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55a464a90470 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c046dd0 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f49840556d0 con 0x55a464ab2650 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 10 client.6312 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 20 client.6312 populate_metadata read hostname 'smithi001' 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464b23180 0x55a464a63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:46.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 4 client.6312 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4649d6780 con 0x55a464b23180 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.825+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464b23180 0x55a464a63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4969ffb640 10 client.6312 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f498ad9d640 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464b23180 0x55a464a63f50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581426a2c599f510 server_cookie=7e738d25aea308f0 in_seq=0 out_seq=0 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4969ffb640 1 -- 172.21.15.1:0/1481228492 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55a4649d6780 con 0x55a464b23180 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4969ffb640 10 client.6312 handle_command_reply: tid=0 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 client.6312 shutdown 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 msgr2=0x55a464ab0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f498400a430 tx=0x7f498401d040 comp rx=0 tx=0).stop 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 shutdown_connections 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x55a464b23180 0x55a464a63f50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a464ab2650 0x55a464ab0b10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a464a95770 0x55a464ab05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4969ffb640 0 client.6312 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 --2- 172.21.15.1:0/1481228492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a464a85a60 0x55a464a8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 >> 172.21.15.1:0/1481228492 conn(0x55a464a95030 msgr2=0x55a464ab3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 shutdown_connections 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 1 -- 172.21.15.1:0/1481228492 wait complete. 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.826+0000 7f4993e28740 20 client.6312 trim_cache size 0 max 16384 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:29:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "59d07d5a-723e-4fe0-951f-9602c328a4c9": { 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "59d07d5a-723e-4fe0-951f-9602c328a4c9", 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:29:46.817 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 msgr2=0x7f496c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 0x7f496c06e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4974006fd0 tx=0x7f4974008040 comp rx=0 tx=0).stop 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 msgr2=0x7f498c1e9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f498400c7a0 tx=0x7f498400cc60 comp rx=0 tx=0).stop 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 shutdown_connections 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f496c06be30 0x7f496c06e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f498c15e390 0x7f498c1e9830 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f498c0cf990 0x7f498c1e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 --2- 172.21.15.1:0/2869095980 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f498c0b1730 0x7f498c1e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:46.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 >> 172.21.15.1:0/2869095980 conn(0x7f498c0b1da0 msgr2=0x7f498c0cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:46.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 shutdown_connections 2024-05-09T06:29:46.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:46.827+0000 7f4991038640 1 -- 172.21.15.1:0/2869095980 wait complete. 2024-05-09T06:29:46.820 INFO:tasks.cephfs.filesystem:scrub status for tag:59d07d5a-723e-4fe0-951f-9602c328a4c9 - {'path': '/', 'tag': '59d07d5a-723e-4fe0-951f-9602c328a4c9', 'options': 'recursive,repair'} 2024-05-09T06:29:47.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:47.162+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-09T06:29:46.670433+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f3660056ee0 con 0x7f366815c160 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: cluster: 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: id: 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: health: HEALTH_OK 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: services: 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: mgr: y(active, since 2m), standbys: x 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: mds: 1/1 daemons up, 3 standby 2024-05-09T06:29:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: data: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: volumes: 1/1 healthy 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: pools: 3 pools, 97 pgs 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: objects: 24 objects, 579 KiB 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: pgs: 97 active+clean 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: io: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: client: 1.4 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.670433+0000 mds.c [INF] scrub queued for path: / 2024-05-09T06:29:47.146 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.670436+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:29:47.147 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.670441+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-09T06:29:47.147 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.671295+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-09T06:29:47.147 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.671307+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-09T06:29:47.147 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:46.671579+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:29:47.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:29:47.824 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:29:50.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:29:50.167+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 8 ==== log(1 entries from seq 7 at 2024-05-09T06:29:48.844216+0000) ==== 189+0+0 (secure 0 0 0) 0x7f3660057140 con 0x7f366815c160 2024-05-09T06:29:50.149 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:29:48.844216+0000 mds.c [INF] scrub summary: idle 2024-05-09T06:30:00.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:00.182+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 9 ==== log(1 entries from seq 619 at 2024-05-09T06:30:00.000192+0000) ==== 187+0+0 (secure 0 0 0) 0x7f3660089090 con 0x7f366815c160 2024-05-09T06:30:00.163 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:30:00.000192+0000 mon.a [INF] overall HEALTH_OK 2024-05-09T06:30:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:02.192+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 10 ==== log(2 entries from seq 620 at 2024-05-09T06:30:02.159451+0000) ==== 602+0+0 (secure 0 0 0) 0x7f36600590a0 con 0x7f366815c160 2024-05-09T06:30:03.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:03.200+0000 7f3659ffb640 1 -- 172.21.15.1:0/1647380541 <== mon.0 v2:172.21.15.1:3300/0 11 ==== log(2 entries from seq 622 at 2024-05-09T06:30:02.185271+0000) ==== 606+0+0 (secure 0 0 0) 0x7f3660004b60 con 0x7f366815c160 2024-05-09T06:30:16.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:16.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-09T06:30:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.958+0000 7fb8602ae640 1 Processor -- start 2024-05-09T06:30:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.958+0000 7fb8602ae640 1 -- start start 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.959+0000 7fb8602ae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 0x7fb85815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.959+0000 7fb8602ae640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb858161fe0 con 0x7fb8580d1c70 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.959+0000 7fb8602ae640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8581622c0 con 0x7fb8580d45f0 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.959+0000 7fb8602ae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb85815f580 con 0x7fb85815e190 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d849640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb8580d1c70 0x7fb8580d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46058/0 (socket says 172.21.15.1:46058) 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d849640 1 -- 172.21.15.1:0/2502608813 learned_addr learned my addr 172.21.15.1:0/2502608813 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2502608813 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 0x7fb85815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717647881 0 0) 0x7fb858161fe0 con 0x7fb8580d1c70 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb844003520 con 0x7fb8580d1c70 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501395000 0 0) 0x7fb8581622c0 con 0x7fb8580d45f0 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb858161fe0 con 0x7fb8580d45f0 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85e04a640 1 -- 172.21.15.1:0/2502608813 >> v1:172.21.15.6:6789/0 conn(0x7fb8580d45f0 legacy=0x7fb8580b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114874549 0 0) 0x7fb844003520 con 0x7fb8580d1c70 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85e04a640 1 -- 172.21.15.1:0/2502608813 >> v1:172.21.15.1:6789/0 conn(0x7fb8580d1c70 legacy=0x7fb8580d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.960+0000 7fb85e04a640 1 -- 172.21.15.1:0/2502608813 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8581625a0 con 0x7fb85815e190 2024-05-09T06:30:16.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.961+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2502608813 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 0x7fb85815f780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097d0 tx=0x7fb85402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d799f4641d65e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.961+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854037070 con 0x7fb85815e190 2024-05-09T06:30:16.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.961+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85402ac30 con 0x7fb85815e190 2024-05-09T06:30:16.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.961+0000 7fb85d048640 1 -- 172.21.15.1:0/2502608813 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848001670 con 0x7fb85815e190 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 -- 172.21.15.1:0/2502608813 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 msgr2=0x7fb85815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 --2- 172.21.15.1:0/2502608813 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 0x7fb85815f780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097d0 tx=0x7fb85402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 -- 172.21.15.1:0/2502608813 shutdown_connections 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 --2- 172.21.15.1:0/2502608813 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85815e190 0x7fb85815f780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 -- 172.21.15.1:0/2502608813 >> 172.21.15.1:0/2502608813 conn(0x7fb858073340 msgr2=0x7fb85809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 -- 172.21.15.1:0/2502608813 shutdown_connections 2024-05-09T06:30:16.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.962+0000 7fb8602ae640 1 -- 172.21.15.1:0/2502608813 wait complete. 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.963+0000 7fb8602ae640 1 Processor -- start 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.963+0000 7fb8602ae640 1 -- start start 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 0x7fb8580b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8580d45f0 0x7fb858158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580b56d0 con 0x7fb85815e190 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb858161c60 con 0x7fb8580d1c70 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb8602ae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580b5850 con 0x7fb8580d45f0 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38122/0 (socket says 172.21.15.1:38122) 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 -- 172.21.15.1:0/3349695526 learned_addr learned my addr 172.21.15.1:0/3349695526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85d849640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8580d45f0 0x7fb858158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 0x7fb8580b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 -- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8580d45f0 msgr2=0x7fb858158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8580d45f0 0x7fb858158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 -- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 msgr2=0x7fb8580b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 0x7fb8580b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e84b640 1 -- 172.21.15.1:0/3349695526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8580cdfd0 con 0x7fb85815e190 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.964+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 0x7fb8580b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c7a0 tx=0x7fb85000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb84effd640 1 -- 172.21.15.1:0/3349695526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850003b70 con 0x7fb85815e190 2024-05-09T06:30:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb84effd640 1 -- 172.21.15.1:0/3349695526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb850003d10 con 0x7fb85815e190 2024-05-09T06:30:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb858159640 con 0x7fb85815e190 2024-05-09T06:30:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb84effd640 1 -- 172.21.15.1:0/3349695526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000aa90 con 0x7fb85815e190 2024-05-09T06:30:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.965+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8580b7040 con 0x7fb85815e190 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.966+0000 7fb84effd640 1 -- 172.21.15.1:0/3349695526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb85000ac30 con 0x7fb85815e190 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.967+0000 7fb84effd640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 0x7fb83c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.967+0000 7fb84effd640 1 -- 172.21.15.1:0/3349695526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fb85008af10 con 0x7fb85815e190 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.967+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 0x7fb83c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.968+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 0x7fb83c06e320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb854034000 tx=0x7fb85402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.968+0000 7fb862fb6740 1 Processor -- start 2024-05-09T06:30:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.968+0000 7fb862fb6740 1 -- start start 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d45682af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683030 0x562d45657e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d455bcd00 con 0x562d456a0650 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562d455644c0 con 0x562d45683770 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562d455b8cc0 con 0x562d45683030 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38128/0 (socket says 172.21.15.1:38128) 2024-05-09T06:30:16.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 -- 172.21.15.1:0/2850152604 learned_addr learned my addr 172.21.15.1:0/2850152604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85d849640 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d45682af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e84b640 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683030 0x562d45657e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 -- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683030 msgr2=0x562d45657e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683030 0x562d45657e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 -- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 msgr2=0x562d45682af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d45682af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.969+0000 7fb85e04a640 1 -- 172.21.15.1:0/2850152604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4539a7d0 con 0x562d456a0650 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.970+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb85402a790 tx=0x7fb85402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b31cdba58ad4b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.970+0000 7fb83bfff640 1 -- 172.21.15.1:0/2850152604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854037040 con 0x562d456a0650 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.970+0000 7fb83bfff640 1 -- 172.21.15.1:0/2850152604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85404e070 con 0x562d456a0650 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.970+0000 7fb83bfff640 1 -- 172.21.15.1:0/2850152604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854032b40 con 0x562d456a0650 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- 172.21.15.1:0/2850152604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 msgr2=0x562d45652fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb85402a790 tx=0x7fb85402c000 comp rx=0 tx=0).stop 2024-05-09T06:30:16.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- 172.21.15.1:0/2850152604 shutdown_connections 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683030 0x562d45657e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d45682af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2850152604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d45652fe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- 172.21.15.1:0/2850152604 >> 172.21.15.1:0/2850152604 conn(0x562d45655250 msgr2=0x562d4569daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- 172.21.15.1:0/2850152604 shutdown_connections 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- 172.21.15.1:0/2850152604 wait complete. 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 -- start start 2024-05-09T06:30:16.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.971+0000 7fb862fb6740 1 Processor -- start 2024-05-09T06:30:16.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d4567e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d456a0650 0x562d4569eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d45657400 con 0x562d4569f420 2024-05-09T06:30:16.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562d45577b90 con 0x562d45683770 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.972+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562d456567f0 con 0x562d456a0650 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb862fb6740 10 client.0 _ll_register_callbacks cb 0x562d455b3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38130/0 (socket says 172.21.15.1:38130) 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e04a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d4567e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 -- 172.21.15.1:0/3554481424 learned_addr learned my addr 172.21.15.1:0/3554481424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85d849640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d456a0650 0x562d4569eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 -- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d456a0650 msgr2=0x562d4569eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d456a0650 0x562d4569eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 -- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 msgr2=0x562d4567e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d4567e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 -- 172.21.15.1:0/3554481424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d452d03f0 con 0x562d4569f420 2024-05-09T06:30:16.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb85e84b640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb850059020 tx=0x7fb850059790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b3da0dce959dbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb8397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85005d440 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fb85005d0f0 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85404e070 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb850061c70 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb862fb6740 10 client.6324 fetch_fsmap learned FSMap version 27 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.973+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x562d452900f0 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85404e070 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fb850072020 con 0x562d4569f420 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 10 client.6324 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 10 client.6324 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 20 client.6324 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d456af7d0 0x562d456afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 4 client.6324 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d455ca430 con 0x562d456af7d0 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d456af7d0 0x562d456afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb8397fa640 10 client.6324 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.974+0000 7fb85e04a640 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d456af7d0 0x562d456afbb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7e0776da41c4ee server_cookie=4a497b2693439f6d in_seq=0 out_seq=0 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb8397fa640 1 -- 172.21.15.1:0/3554481424 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d455ca430 con 0x562d456af7d0 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb8397fa640 10 client.6324 handle_command_reply: tid=0 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 client.6324 shutdown 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 msgr2=0x562d456aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb850059020 tx=0x7fb850059790 comp rx=0 tx=0).stop 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 shutdown_connections 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d456af7d0 0x562d456afbb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d456aedc0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d456a0650 0x562d4569eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb8397fa640 0 client.6324 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 --2- 172.21.15.1:0/3554481424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d45683770 0x562d4567e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 >> 172.21.15.1:0/3554481424 conn(0x562d45683030 msgr2=0x562d456a0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 shutdown_connections 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 1 -- 172.21.15.1:0/3554481424 wait complete. 2024-05-09T06:30:16.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.975+0000 7fb862fb6740 20 client.6324 trim_cache size 0 max 16384 2024-05-09T06:30:16.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.993+0000 7fb862fb6740 1 Processor -- start 2024-05-09T06:30:16.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.993+0000 7fb862fb6740 1 -- start start 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d456a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d456a0650 0x562d456589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb858200a40 con 0x562d4569f420 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb8581ffa20 con 0x562d456a0650 2024-05-09T06:30:16.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580b1790 con 0x562d45683770 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38132/0 (socket says 172.21.15.1:38132) 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 -- 172.21.15.1:0/2901174277 learned_addr learned my addr 172.21.15.1:0/2901174277 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85e84b640 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d456a0650 0x562d456589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85e04a640 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d456a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 -- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 msgr2=0x562d456a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d456a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 -- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d456a0650 msgr2=0x562d456589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d456a0650 0x562d456589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 -- 172.21.15.1:0/2901174277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d45696140 con 0x562d4569f420 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb85d849640 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8480059d0 tx=0x7fb848005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4610d0fedc00ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.994+0000 7fb83b7fe640 1 -- 172.21.15.1:0/2901174277 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84801a020 con 0x562d4569f420 2024-05-09T06:30:16.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb83b7fe640 1 -- 172.21.15.1:0/2901174277 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848013a30 con 0x562d4569f420 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb83b7fe640 1 -- 172.21.15.1:0/2901174277 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848013de0 con 0x562d4569f420 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 -- 172.21.15.1:0/2901174277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 msgr2=0x562d45658470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8480059d0 tx=0x7fb848005e90 comp rx=0 tx=0).stop 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 -- 172.21.15.1:0/2901174277 shutdown_connections 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d456a0650 0x562d456589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d4569f420 0x562d45658470 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 --2- 172.21.15.1:0/2901174277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d456a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 -- 172.21.15.1:0/2901174277 >> 172.21.15.1:0/2901174277 conn(0x562d45683030 msgr2=0x562d45696370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 -- 172.21.15.1:0/2901174277 shutdown_connections 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.995+0000 7fb862fb6740 1 -- 172.21.15.1:0/2901174277 wait complete. 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.996+0000 7fb862fb6740 1 -- start start 2024-05-09T06:30:16.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.996+0000 7fb862fb6740 1 Processor -- start 2024-05-09T06:30:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d45716c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d4569f420 0x562d4567c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d4569eda0 con 0x562d456a0650 2024-05-09T06:30:16.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580b1de0 con 0x562d4569f420 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562d45653440 con 0x562d45683770 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb862fb6740 10 client.0 _ll_register_callbacks cb 0x7fb850001830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38148/0 (socket says 172.21.15.1:38148) 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 -- 172.21.15.1:0/1633106253 learned_addr learned my addr 172.21.15.1:0/1633106253 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e04a640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d45716c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85d849640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d4569f420 0x562d4567c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 -- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 msgr2=0x562d45716c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d45716c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 -- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d4569f420 msgr2=0x562d4567c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d4569f420 0x562d4567c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 -- 172.21.15.1:0/1633106253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4567d5a0 con 0x562d456a0650 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e04a640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d45716c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:16.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb85e84b640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb85005d3b0 tx=0x7fb8500891e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b73563b34792693 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.997+0000 7fb8397fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85005d440 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fb850054ff0 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb848013a30 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb850058bd0 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 10 client.6330 fetch_fsmap learned FSMap version 27 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x562d4569ca70 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848013a30 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fb850072040 con 0x562d456a0650 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 10 client.6330 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 20 client.6330 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d4569d1b0 0x562d4569b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 4 client.6330 mds_command: new command op to 6045 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:30:16.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 --> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d455c4780 con 0x562d4569d1b0 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb85e04a640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d4569d1b0 0x562d4569b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb85e04a640 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d4569d1b0 0x562d4569b480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daeac4676a74692 server_cookie=891dd10d649aae98 in_seq=0 out_seq=0 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.998+0000 7fb8397fa640 10 client.6330 ms_handle_connect on v2:172.21.15.1:6834/3096552929 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb8397fa640 1 -- 172.21.15.1:0/1633106253 <== mds.0 v2:172.21.15.1:6834/3096552929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d455c4780 con 0x562d4569d1b0 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb8397fa640 10 client.6330 handle_command_reply: tid=0 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 client.6330 shutdown 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 msgr2=0x562d4567cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb85005d3b0 tx=0x7fb8500891e0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 shutdown_connections 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:6834/3096552929,v1:172.21.15.1:6836/3096552929] conn(0x562d4569d1b0 0x562d4569b480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d456a0650 0x562d4567cf10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d4569f420 0x562d4567c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 --2- 172.21.15.1:0/1633106253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562d45683770 0x562d45716c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb8397fa640 0 client.6330 ms_handle_reset on v2:172.21.15.1:6834/3096552929 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 >> 172.21.15.1:0/1633106253 conn(0x562d45683030 msgr2=0x562d456a1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 shutdown_connections 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 1 -- 172.21.15.1:0/1633106253 wait complete. 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb862fb6740 20 client.6330 trim_cache size 0 max 16384 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:30:16.987 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 msgr2=0x7fb83c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:16.999+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 0x7fb83c06e320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb854034000 tx=0x7fb85402e000 comp rx=0 tx=0).stop 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 msgr2=0x7fb8580b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c7a0 tx=0x7fb85000cc60 comp rx=0 tx=0).stop 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 shutdown_connections 2024-05-09T06:30:16.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb83c06be60 0x7fb83c06e320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb85815e190 0x7fb8580b75f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8580d45f0 0x7fb858158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 --2- 172.21.15.1:0/3349695526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8580d1c70 0x7fb8580b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 >> 172.21.15.1:0/3349695526 conn(0x7fb858073340 msgr2=0x7fb8580d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 shutdown_connections 2024-05-09T06:30:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.000+0000 7fb8602ae640 1 -- 172.21.15.1:0/3349695526 wait complete. 2024-05-09T06:30:16.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:30:17.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.218+0000 7f8dd48cd640 1 Processor -- start 2024-05-09T06:30:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.218+0000 7f8dd48cd640 1 -- start start 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dd48cd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dd006e6d0 0x7f8dd006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dd48cd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8dd00cf180 0x7f8dd00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dd48cd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd00ce0e0 con 0x7f8dd00cdab0 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dd48cd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00cdec0 con 0x7f8dd006e6d0 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dd48cd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00ce3c0 con 0x7f8dd00cf180 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dd006e6d0 0x7f8dd006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8dd006e6d0 0x7f8dd006eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38164/0 (socket says 172.21.15.1:38164) 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 -- 172.21.15.1:0/1861720424 learned_addr learned my addr 172.21.15.1:0/1861720424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dced9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8dd00cdab0 0x7f8dd015e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46064/0 (socket says 172.21.15.1:46064) 2024-05-09T06:30:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dcdd9b640 1 --2- 172.21.15.1:0/1861720424 >> v2:172.21.15.6:3300/0 conn(0x7f8dd00cf180 0x7f8dd00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 -- 172.21.15.1:0/1861720424 >> v2:172.21.15.6:3300/0 conn(0x7f8dd00cf180 msgr2=0x7f8dd00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 --2- 172.21.15.1:0/1861720424 >> v2:172.21.15.6:3300/0 conn(0x7f8dd00cf180 0x7f8dd00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 -- 172.21.15.1:0/1861720424 >> v1:172.21.15.1:6789/0 conn(0x7f8dd00cdab0 legacy=0x7f8dd015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.219+0000 7f8dce59c640 1 -- 172.21.15.1:0/1861720424 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd00ce6a0 con 0x7f8dd006e6d0 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.220+0000 7f8dce59c640 1 --2- 172.21.15.1:0/1861720424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd006e6d0 0x7f8dd006eab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00b27c0 tx=0x7f8dc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a26e37a15dfd26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.220+0000 7f8dcd59a640 1 -- 172.21.15.1:0/1861720424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0037070 con 0x7f8dd006e6d0 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.220+0000 7f8dcd59a640 1 -- 172.21.15.1:0/1861720424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc0032440 con 0x7f8dd006e6d0 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.220+0000 7f8dcd59a640 1 -- 172.21.15.1:0/1861720424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0032810 con 0x7f8dd006e6d0 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 -- 172.21.15.1:0/1861720424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd006e6d0 msgr2=0x7f8dd006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/1861720424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd006e6d0 0x7f8dd006eab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00b27c0 tx=0x7f8dc002a270 comp rx=0 tx=0).stop 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 -- 172.21.15.1:0/1861720424 shutdown_connections 2024-05-09T06:30:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/1861720424 >> v2:172.21.15.6:3300/0 conn(0x7f8dd00cf180 0x7f8dd00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/1861720424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd006e6d0 0x7f8dd006eab0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 -- 172.21.15.1:0/1861720424 >> 172.21.15.1:0/1861720424 conn(0x7f8dd00cc4b0 msgr2=0x7f8dd00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 -- 172.21.15.1:0/1861720424 shutdown_connections 2024-05-09T06:30:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.221+0000 7f8dd48cd640 1 -- 172.21.15.1:0/1861720424 wait complete. 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.222+0000 7f8dd48cd640 1 Processor -- start 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.222+0000 7f8dd48cd640 1 -- start start 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dd006e6d0 0x7f8dd015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dd00cdab0 0x7f8dd01671a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd006eff0 con 0x7f8dd00cf180 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00cec60 con 0x7f8dd006e6d0 2024-05-09T06:30:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dd48cd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd0070ed0 con 0x7f8dd00cdab0 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38168/0 (socket says 172.21.15.1:38168) 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 -- 172.21.15.1:0/3273530910 learned_addr learned my addr 172.21.15.1:0/3273530910 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dce59c640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dd006e6d0 0x7f8dd015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dcdd9b640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dd00cdab0 0x7f8dd01671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 -- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dd00cdab0 msgr2=0x7f8dd01671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dd00cdab0 0x7f8dd01671a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 -- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dd006e6d0 msgr2=0x7f8dd015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dd006e6d0 0x7f8dd015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.223+0000 7f8dced9d640 1 -- 172.21.15.1:0/3273530910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd0167aa0 con 0x7f8dd00cf180 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dced9d640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c780 tx=0x7f8dc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8017070 con 0x7f8dd00cf180 2024-05-09T06:30:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc800a640 con 0x7f8dd00cf180 2024-05-09T06:30:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd0167d80 con 0x7f8dd00cf180 2024-05-09T06:30:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800aa30 con 0x7f8dd00cf180 2024-05-09T06:30:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.224+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd0168340 con 0x7f8dd00cf180 2024-05-09T06:30:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.225+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd00752b0 con 0x7f8dd00cf180 2024-05-09T06:30:17.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.225+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8dc801a070 con 0x7f8dd00cf180 2024-05-09T06:30:17.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.226+0000 7f8dbeffd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.226+0000 7f8dce59c640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.226+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f8dc808a530 con 0x7f8dd00cf180 2024-05-09T06:30:17.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.227+0000 7f8dce59c640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 0x7f8db006e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0034000 tx=0x7f8dc002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.231+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8dc80586d0 con 0x7f8dd00cf180 2024-05-09T06:30:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.458+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dd006cf50 con 0x7f8dd00cf180 2024-05-09T06:30:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.458+0000 7f8dbeffd640 1 -- 172.21.15.1:0/3273530910 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7f8dc805c9d0 con 0x7f8dd00cf180 2024-05-09T06:30:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:17.440 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:30:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:30:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 msgr2=0x7f8db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 0x7f8db006e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0034000 tx=0x7f8dc002e000 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 msgr2=0x7f8dd00c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c780 tx=0x7f8dc800cc40 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 shutdown_connections 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd00cf180 0x7f8dd00c5480 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dd00cdab0 0x7f8dd01671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 --2- 172.21.15.1:0/3273530910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dd006e6d0 0x7f8dd015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 >> 172.21.15.1:0/3273530910 conn(0x7f8dd00cc4b0 msgr2=0x7f8dd00b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.460+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 shutdown_connections 2024-05-09T06:30:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.461+0000 7f8dd48cd640 1 -- 172.21.15.1:0/3273530910 wait complete. 2024-05-09T06:30:17.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T06:30:17.578 INFO:teuthology.orchestra.run.smithi001.stdout:{"message":"","return_code":0} 2024-05-09T06:30:17.579 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:30:17.579 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-09T06:30:17.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.642+0000 7fdfa0a2af40 1 Processor -- start 2024-05-09T06:30:17.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.642+0000 7fdfa0a2af40 1 -- start start 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.642+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d44411950 0x555d44410be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.642+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d4443f2d0 0x555d444c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555d44442370 con 0x555d44411d30 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555d443b9c80 con 0x555d4443f2d0 2024-05-09T06:30:17.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555d4443b590 con 0x555d44411950 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38188/0 (socket says 172.21.15.1:38188) 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/2660369707 learned_addr learned my addr 172.21.15.1:0/2660369707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d44411950 msgr2=0x555d44410be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d44411950 0x555d44410be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d4443f2d0 msgr2=0x555d444c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d4443f2d0 0x555d444c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/2660369707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4433ecb0 con 0x555d44411d30 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009790 tx=0x7fdf88009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4ccf13fa2bcb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9cd28640 1 -- 172.21.15.1:0/2660369707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88017070 con 0x555d44411d30 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9cd28640 1 -- 172.21.15.1:0/2660369707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880090d0 con 0x555d44411d30 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.643+0000 7fdf9cd28640 1 -- 172.21.15.1:0/2660369707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88006b60 con 0x555d44411d30 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/2660369707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 msgr2=0x555d444c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009790 tx=0x7fdf88009c50 comp rx=0 tx=0).stop 2024-05-09T06:30:17.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/2660369707 shutdown_connections 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d4443f2d0 0x555d444c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d444c7e10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/2660369707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d44411950 0x555d44410be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/2660369707 >> 172.21.15.1:0/2660369707 conn(0x555d44412d10 msgr2=0x555d444e19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/2660369707 shutdown_connections 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/2660369707 wait complete. 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 Processor -- start 2024-05-09T06:30:17.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.644+0000 7fdfa0a2af40 1 -- start start 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d44411950 0x555d444cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d44441b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d4443f2d0 0x555d444e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555d444c8db0 con 0x555d44411d30 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555d444e57c0 con 0x555d44411950 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdfa0a2af40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555d444e5a80 con 0x555d4443f2d0 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdf9dd2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d44441b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdf9dd2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d44441b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38200/0 (socket says 172.21.15.1:38200) 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/3421413587 learned_addr learned my addr 172.21.15.1:0/3421413587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdf9e52b640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d44411950 0x555d444cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.645+0000 7fdf9ed2c640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d4443f2d0 0x555d444e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d4443f2d0 msgr2=0x555d444e2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555d4443f2d0 0x555d444e2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d44411950 msgr2=0x555d444cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555d44411950 0x555d444cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 -- 172.21.15.1:0/3421413587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4443bf10 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf9dd2a640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d44441b20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdf88015040 tx=0x7fdf88003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf977fe640 1 -- 172.21.15.1:0/3421413587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88017070 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf977fe640 1 -- 172.21.15.1:0/3421413587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88018040 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d444e8c50 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdf977fe640 1 -- 172.21.15.1:0/3421413587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88012890 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.646+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4443c490 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.647+0000 7fdf977fe640 1 -- 172.21.15.1:0/3421413587 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf88012a30 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf977fe640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf8006a840 0x7fdf8006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf977fe640 1 -- 172.21.15.1:0/3421413587 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fdf8808a820 con 0x555d44411d30 2024-05-09T06:30:17.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf9e52b640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf8006a840 0x7fdf8006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x555d44457620 0x555d444e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53) -- 0x555d444e2fc0 con 0x555d44457620 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf9ed2c640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x555d44457620 0x555d444e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf9e52b640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf8006a840 0x7fdf8006cd00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf98057000 tx=0x7fdf98051000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.648+0000 7fdf9ed2c640 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x555d44457620 0x555d444e96d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdf9ed2c640 1 -- 172.21.15.1:0/3421413587 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fdf90003450 con 0x555d44457620 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x555d44457620 msgr2=0x555d444e96d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x555d44457620 0x555d444e96d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf8006a840 msgr2=0x7fdf8006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf8006a840 0x7fdf8006cd00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf98057000 tx=0x7fdf98051000 comp rx=0 tx=0).stop 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 msgr2=0x555d44441b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 --2- 172.21.15.1:0/3421413587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d44411d30 0x555d44441b20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdf88015040 tx=0x7fdf88003020 comp rx=0 tx=0).stop 2024-05-09T06:30:17.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdf9ed2c640 1 -- 172.21.15.1:0/3421413587 reap_dead start 2024-05-09T06:30:17.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 shutdown_connections 2024-05-09T06:30:17.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 >> 172.21.15.1:0/3421413587 conn(0x555d44412d10 msgr2=0x555d444e1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:17.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 shutdown_connections 2024-05-09T06:30:17.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:17.649+0000 7fdfa0a2af40 1 -- 172.21.15.1:0/3421413587 wait complete. 2024-05-09T06:30:17.636 INFO:teuthology.orchestra.run.smithi001.stdout:Yalphaparent_a 2024-05-09T06:30:17.636 DEBUG:teuthology.orchestra.run.smithi001:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T06:30:17.684 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ancestors": [ 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1099511627776, 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "alpha", 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1, 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "parent_a", 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 7 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T06:30:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 8, 2024-05-09T06:30:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: "old_pools": [] 2024-05-09T06:30:17.686 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:17.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:30:17.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:30:17.769 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:30:17.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:30:17.823 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:30:17.823 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:30:17.823 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:17.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:17.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:30:17.925 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:30:17.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.001+0000 7f1a38da5640 1 Processor -- start 2024-05-09T06:30:17.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.001+0000 7f1a38da5640 1 -- start start 2024-05-09T06:30:17.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a38da5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a34164ce0 0x7f1a341670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a38da5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a34071c20 con 0x7f1a341454b0 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a38da5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3406eb40 con 0x7f1a340b2a60 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a38da5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3406ee20 con 0x7f1a34164ce0 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a3259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1a341454b0 0x7f1a340b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59352/0 (socket says 172.21.15.1:59352) 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.002+0000 7f1a3259c640 1 -- 172.21.15.1:0/2527497205 learned_addr learned my addr 172.21.15.1:0/2527497205 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a32d9d640 1 --2- 172.21.15.1:0/2527497205 >> v2:172.21.15.6:3300/0 conn(0x7f1a34164ce0 0x7f1a341670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a3159a640 1 -- 172.21.15.1:0/2527497205 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336117130 0 0) 0x7f1a34071c20 con 0x7f1a341454b0 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a3159a640 1 -- 172.21.15.1:0/2527497205 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a20003520 con 0x7f1a341454b0 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a32d9d640 1 -- 172.21.15.1:0/2527497205 >> v1:172.21.15.6:6789/0 conn(0x7f1a340b2a60 legacy=0x7f1a340b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a32d9d640 1 -- 172.21.15.1:0/2527497205 >> v1:172.21.15.1:6789/0 conn(0x7f1a341454b0 legacy=0x7f1a340b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.003+0000 7f1a32d9d640 1 -- 172.21.15.1:0/2527497205 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3406d8e0 con 0x7f1a34164ce0 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a32d9d640 1 --2- 172.21.15.1:0/2527497205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a341670d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0097a0 tx=0x7f1a2c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5505461249ba51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a3159a640 1 -- 172.21.15.1:0/2527497205 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c037070 con 0x7f1a34164ce0 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a3159a640 1 -- 172.21.15.1:0/2527497205 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c032440 con 0x7f1a34164ce0 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a3159a640 1 -- 172.21.15.1:0/2527497205 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c032810 con 0x7f1a34164ce0 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a38da5640 1 -- 172.21.15.1:0/2527497205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 msgr2=0x7f1a341670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.004+0000 7f1a38da5640 1 --2- 172.21.15.1:0/2527497205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a341670d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0097a0 tx=0x7f1a2c02a230 comp rx=0 tx=0).stop 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.005+0000 7f1a38da5640 1 -- 172.21.15.1:0/2527497205 shutdown_connections 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.005+0000 7f1a38da5640 1 --2- 172.21.15.1:0/2527497205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a341670d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.005+0000 7f1a38da5640 1 -- 172.21.15.1:0/2527497205 >> 172.21.15.1:0/2527497205 conn(0x7f1a340cc4b0 msgr2=0x7f1a340cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.005+0000 7f1a38da5640 1 -- 172.21.15.1:0/2527497205 shutdown_connections 2024-05-09T06:30:17.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.005+0000 7f1a38da5640 1 -- 172.21.15.1:0/2527497205 wait complete. 2024-05-09T06:30:17.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.006+0000 7f1a38da5640 1 Processor -- start 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.006+0000 7f1a38da5640 1 -- start start 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 0x7f1a3413de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a340ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b5140 con 0x7f1a341454b0 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34145890 con 0x7f1a34164ce0 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a38da5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3406e0e0 con 0x7f1a340b2a60 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42046/0 (socket says 172.21.15.1:42046) 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 -- 172.21.15.1:0/1426476710 learned_addr learned my addr 172.21.15.1:0/1426476710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a32d9d640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a340ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a3259c640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 0x7f1a3413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 -- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 msgr2=0x7f1a3413de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 0x7f1a3413de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 -- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 msgr2=0x7f1a340ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a340ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 -- 172.21.15.1:0/1426476710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3413cdb0 con 0x7f1a341454b0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a3259c640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 0x7f1a3413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a32d9d640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a340ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a31d9b640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c480 tx=0x7f1a2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.007+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a240052d0 con 0x7f1a341454b0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.008+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a24005470 con 0x7f1a341454b0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.008+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3413d050 con 0x7f1a341454b0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.008+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24012870 con 0x7f1a341454b0 2024-05-09T06:30:17.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.008+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3413d610 con 0x7f1a341454b0 2024-05-09T06:30:17.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.009+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a340752b0 con 0x7f1a341454b0 2024-05-09T06:30:17.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.009+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a24012a10 con 0x7f1a341454b0 2024-05-09T06:30:17.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.010+0000 7f1a2affd640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:17.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.010+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f1a2408ac40 con 0x7f1a341454b0 2024-05-09T06:30:17.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.010+0000 7f1a3259c640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:17.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.010+0000 7f1a3259c640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 0x7f1a1406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001560 tx=0x7f1a1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:17.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.014+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a240594c0 con 0x7f1a341454b0 2024-05-09T06:30:18.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.199+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a340b5b00 con 0x7f1a341454b0 2024-05-09T06:30:18.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.200+0000 7f1a2affd640 1 -- 172.21.15.1:0/1426476710 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16850 (secure 0 0 0) 0x7f1a24058dc0 con 0x7f1a341454b0 2024-05-09T06:30:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:18.182 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:03.192204+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 msgr2=0x7f1a1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 0x7f1a1406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001560 tx=0x7f1a1c009040 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 msgr2=0x7f1a341411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c480 tx=0x7f1a2400c940 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 shutdown_connections 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a34164ce0 0x7f1a340ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a341454b0 0x7f1a341411c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 --2- 172.21.15.1:0/1426476710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a340b2a60 0x7f1a3413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 >> 172.21.15.1:0/1426476710 conn(0x7f1a340cc4b0 msgr2=0x7f1a341425f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 shutdown_connections 2024-05-09T06:30:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.202+0000 7f1a38da5640 1 -- 172.21.15.1:0/1426476710 wait complete. 2024-05-09T06:30:18.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:29:31.502757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:29:32.177695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:30:18.195 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:30:18.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.359+0000 7f25072df640 1 Processor -- start 2024-05-09T06:30:18.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.360+0000 7f25072df640 1 -- start start 2024-05-09T06:30:18.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.360+0000 7f25072df640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f25000d1c70 0x7f250015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.360+0000 7f25072df640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2500161fe0 con 0x7f250015e190 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.360+0000 7f25072df640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25001622c0 con 0x7f25000d45f0 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.360+0000 7f25072df640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f250015f580 con 0x7f25000d1c70 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250487a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f25000d1c70 0x7f250015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250507b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f250015e190 0x7f25000d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59362/0 (socket says 172.21.15.1:59362) 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250507b640 1 -- 172.21.15.1:0/2733913582 learned_addr learned my addr 172.21.15.1:0/2733913582 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250487a640 1 -- 172.21.15.1:0/2733913582 >> v1:172.21.15.6:6789/0 conn(0x7f25000d45f0 legacy=0x7f25000b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250487a640 1 -- 172.21.15.1:0/2733913582 >> v1:172.21.15.1:6789/0 conn(0x7f250015e190 legacy=0x7f25000d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f250487a640 1 -- 172.21.15.1:0/2733913582 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25001625a0 con 0x7f25000d1c70 2024-05-09T06:30:18.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.361+0000 7f24fffff640 1 -- 172.21.15.1:0/2733913582 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1280319869 0 0) 0x7f2500161fe0 con 0x7f250015e190 2024-05-09T06:30:18.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f250487a640 1 --2- 172.21.15.1:0/2733913582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f250015f780 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24f40097a0 tx=0x7f24f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1d6eb01c26acf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f24fffff640 1 -- 172.21.15.1:0/2733913582 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4037070 con 0x7f25000d1c70 2024-05-09T06:30:18.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f24fffff640 1 -- 172.21.15.1:0/2733913582 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4032440 con 0x7f25000d1c70 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f24fffff640 1 -- 172.21.15.1:0/2733913582 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4032810 con 0x7f25000d1c70 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 -- 172.21.15.1:0/2733913582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 msgr2=0x7f250015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 --2- 172.21.15.1:0/2733913582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f250015f780 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24f40097a0 tx=0x7f24f402a230 comp rx=0 tx=0).stop 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 -- 172.21.15.1:0/2733913582 shutdown_connections 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 --2- 172.21.15.1:0/2733913582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f250015f780 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 -- 172.21.15.1:0/2733913582 >> 172.21.15.1:0/2733913582 conn(0x7f2500073340 msgr2=0x7f250009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 -- 172.21.15.1:0/2733913582 shutdown_connections 2024-05-09T06:30:18.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.362+0000 7f25072df640 1 -- 172.21.15.1:0/2733913582 wait complete. 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.363+0000 7f25072df640 1 Processor -- start 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 -- start start 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25000d45f0 0x7f25000cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f250015e190 0x7f25000d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25000b56d0 con 0x7f25000d1c70 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2500161c60 con 0x7f25000d45f0 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f25072df640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f25000b5850 con 0x7f250015e190 2024-05-09T06:30:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f250507b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f250507b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42062/0 (socket says 172.21.15.1:42062) 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f250507b640 1 -- 172.21.15.1:0/2009281955 learned_addr learned my addr 172.21.15.1:0/2009281955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f250487a640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25000d45f0 0x7f25000cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.364+0000 7f250587c640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f250015e190 0x7f25000d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 -- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f250015e190 msgr2=0x7f25000d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f250015e190 0x7f25000d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 -- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25000d45f0 msgr2=0x7f25000cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25000d45f0 0x7f25000cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 -- 172.21.15.1:0/2009281955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25000d0d40 con 0x7f25000d1c70 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f250507b640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c3f0 tx=0x7f24f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80074e0 con 0x7f25000d1c70 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f80047e0 con 0x7f25000d1c70 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25000d1020 con 0x7f25000d1c70 2024-05-09T06:30:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8004c10 con 0x7f25000d1c70 2024-05-09T06:30:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.365+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25000d15e0 con 0x7f25000d1c70 2024-05-09T06:30:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.366+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000752b0 con 0x7f25000d1c70 2024-05-09T06:30:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.367+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24f8011070 con 0x7f25000d1c70 2024-05-09T06:30:18.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.367+0000 7f24fdffb640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 0x7f24ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.367+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f24f808aa50 con 0x7f25000d1c70 2024-05-09T06:30:18.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.367+0000 7f250487a640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 0x7f24ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.368+0000 7f250487a640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 0x7f24ec06e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24f4034000 tx=0x7f24f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.372+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24f8058c80 con 0x7f25000d1c70 2024-05-09T06:30:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.552+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f250008e1c0 con 0x7f25000d1c70 2024-05-09T06:30:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.553+0000 7f24fdffb640 1 -- 172.21.15.1:0/2009281955 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f24f805cf80 con 0x7f25000d1c70 2024-05-09T06:30:18.534 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:30:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 msgr2=0x7f24ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 0x7f24ec06e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24f4034000 tx=0x7f24f402e000 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 msgr2=0x7f2500151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c3f0 tx=0x7f24f800c8b0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 shutdown_connections 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24ec06be60 0x7f24ec06e320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f250015e190 0x7f25000d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25000d45f0 0x7f25000cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 --2- 172.21.15.1:0/2009281955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25000d1c70 0x7f2500151350 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 >> 172.21.15.1:0/2009281955 conn(0x7f2500073340 msgr2=0x7f250015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 shutdown_connections 2024-05-09T06:30:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.555+0000 7f25072df640 1 -- 172.21.15.1:0/2009281955 wait complete. 2024-05-09T06:30:18.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:30:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.673+0000 7fdd5f109640 1 Processor -- start 2024-05-09T06:30:18.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.673+0000 7fdd5f109640 1 -- start start 2024-05-09T06:30:18.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5f109640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd5815e190 0x7fdd5815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5f109640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 0x7fdd580b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5f109640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd58161fe0 con 0x7fdd580d1c70 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5f109640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5815f580 con 0x7fdd5815e190 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5f109640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdd581622c0 con 0x7fdd580d45f0 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd5815e190 0x7fdd5815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd5815e190 0x7fdd5815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42064/0 (socket says 172.21.15.1:42064) 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd57fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdd580d1c70 0x7fdd580d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59374/0 (socket says 172.21.15.1:59374) 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 -- 172.21.15.1:0/200161216 learned_addr learned my addr 172.21.15.1:0/200161216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5d6a6640 1 --2- 172.21.15.1:0/200161216 >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 0x7fdd580b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 -- 172.21.15.1:0/200161216 >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 msgr2=0x7fdd580b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 --2- 172.21.15.1:0/200161216 >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 0x7fdd580b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 -- 172.21.15.1:0/200161216 >> v1:172.21.15.1:6789/0 conn(0x7fdd580d1c70 legacy=0x7fdd580d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5cea5640 1 -- 172.21.15.1:0/200161216 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581625a0 con 0x7fdd5815e190 2024-05-09T06:30:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.674+0000 7fdd5d6a6640 1 --2- 172.21.15.1:0/200161216 >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 0x7fdd580b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd5cea5640 1 --2- 172.21.15.1:0/200161216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd5815f780 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdd580ce7f0 tx=0x7fdd4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417fa3a93fa613a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd577fe640 1 -- 172.21.15.1:0/200161216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48037070 con 0x7fdd5815e190 2024-05-09T06:30:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd577fe640 1 -- 172.21.15.1:0/200161216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd48032440 con 0x7fdd5815e190 2024-05-09T06:30:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd577fe640 1 -- 172.21.15.1:0/200161216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48032810 con 0x7fdd5815e190 2024-05-09T06:30:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd5f109640 1 -- 172.21.15.1:0/200161216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 msgr2=0x7fdd5815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.675+0000 7fdd5f109640 1 --2- 172.21.15.1:0/200161216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd5815f780 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdd580ce7f0 tx=0x7fdd4802a270 comp rx=0 tx=0).stop 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 -- 172.21.15.1:0/200161216 shutdown_connections 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 --2- 172.21.15.1:0/200161216 >> v2:172.21.15.6:3300/0 conn(0x7fdd580d45f0 0x7fdd580b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 --2- 172.21.15.1:0/200161216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd5815f780 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 -- 172.21.15.1:0/200161216 >> 172.21.15.1:0/200161216 conn(0x7fdd58073340 msgr2=0x7fdd5809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 -- 172.21.15.1:0/200161216 shutdown_connections 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.676+0000 7fdd5f109640 1 -- 172.21.15.1:0/200161216 wait complete. 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.677+0000 7fdd5f109640 1 Processor -- start 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.677+0000 7fdd5f109640 1 -- start start 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd580d1c70 0x7fdd58156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 0x7fdd580c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580b56d0 con 0x7fdd5815e190 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58161c60 con 0x7fdd580d1c70 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5f109640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdd580b5850 con 0x7fdd580d45f0 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42078/0 (socket says 172.21.15.1:42078) 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 -- 172.21.15.1:0/1707187438 learned_addr learned my addr 172.21.15.1:0/1707187438 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd57fff640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 0x7fdd580c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5cea5640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd580d1c70 0x7fdd58156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 -- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 msgr2=0x7fdd580c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 0x7fdd580c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 -- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd580d1c70 msgr2=0x7fdd58156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd580d1c70 0x7fdd58156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 -- 172.21.15.1:0/1707187438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd580c8a00 con 0x7fdd5815e190 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd57fff640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 0x7fdd580c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:18.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd5d6a6640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000ced0 tx=0x7fdd5000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.678+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003b90 con 0x7fdd5815e190 2024-05-09T06:30:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.679+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd50003d50 con 0x7fdd5815e190 2024-05-09T06:30:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.679+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004260 con 0x7fdd5815e190 2024-05-09T06:30:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.679+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd580c8440 con 0x7fdd5815e190 2024-05-09T06:30:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.679+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd580cb310 con 0x7fdd5815e190 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.680+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580752b0 con 0x7fdd5815e190 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.680+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd50004400 con 0x7fdd5815e190 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.681+0000 7fdd557fa640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.681+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fdd5008ba10 con 0x7fdd5815e190 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.681+0000 7fdd5cea5640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.681+0000 7fdd5cea5640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd48034000 tx=0x7fdd4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.685+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd580c8a00 con 0x7fdd5815e190 2024-05-09T06:30:18.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.867+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd5808e1c0 con 0x7fdd5815e190 2024-05-09T06:30:18.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.868+0000 7fdd557fa640 1 -- 172.21.15.1:0/1707187438 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7fdd5005a2c0 con 0x7fdd5815e190 2024-05-09T06:30:18.854 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 msgr2=0x7fdd3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd48034000 tx=0x7fdd4802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 msgr2=0x7fdd580cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000ced0 tx=0x7fdd5000a710 comp rx=0 tx=0).stop 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 shutdown_connections 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd5815e190 0x7fdd580cbb10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd580d45f0 0x7fdd580c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 --2- 172.21.15.1:0/1707187438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd580d1c70 0x7fdd58156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 >> 172.21.15.1:0/1707187438 conn(0x7fdd58073340 msgr2=0x7fdd580b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 shutdown_connections 2024-05-09T06:30:18.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.870+0000 7fdd5f109640 1 -- 172.21.15.1:0/1707187438 wait complete. 2024-05-09T06:30:18.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:30:18.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.988+0000 7f9252510640 1 Processor -- start 2024-05-09T06:30:18.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.988+0000 7f9252510640 1 -- start start 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9252510640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 0x7f924c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9252510640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f924c0d08b0 0x7f924c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9252510640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c0b1340 con 0x7f924c15e270 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9252510640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0cc260 con 0x7f924c0d08b0 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9252510640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0d1020 con 0x7f924c0ceba0 2024-05-09T06:30:18.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f924bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f924c15e270 0x7f924c0ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59388/0 (socket says 172.21.15.1:59388) 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9250aad640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f924c0d08b0 0x7f924c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f924bfff640 1 -- 172.21.15.1:0/2393825287 learned_addr learned my addr 172.21.15.1:0/2393825287 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f924b7fe640 1 --2- 172.21.15.1:0/2393825287 >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 0x7f924c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9250aad640 1 -- 172.21.15.1:0/2393825287 >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 msgr2=0x7f924c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9250aad640 1 --2- 172.21.15.1:0/2393825287 >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 0x7f924c0cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9250aad640 1 -- 172.21.15.1:0/2393825287 >> v1:172.21.15.1:6789/0 conn(0x7f924c15e270 legacy=0x7f924c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f9250aad640 1 -- 172.21.15.1:0/2393825287 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0cc4b0 con 0x7f924c0d08b0 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f924affd640 1 -- 172.21.15.1:0/2393825287 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458254076 0 0) 0x7f923c001110 con 0x7f924c15e270 2024-05-09T06:30:18.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.989+0000 7f924b7fe640 1 --2- 172.21.15.1:0/2393825287 >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 0x7f924c0cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.990+0000 7f9250aad640 1 --2- 172.21.15.1:0/2393825287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c0d08b0 0x7f924c0ccf90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f924c06e4b0 tx=0x7f924402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2c054d8973e7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.990+0000 7f924affd640 1 -- 172.21.15.1:0/2393825287 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244007740 con 0x7f924c0d08b0 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.990+0000 7f924affd640 1 -- 172.21.15.1:0/2393825287 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92440078e0 con 0x7f924c0d08b0 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.990+0000 7f924affd640 1 -- 172.21.15.1:0/2393825287 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244007cb0 con 0x7f924c0d08b0 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 -- 172.21.15.1:0/2393825287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c0d08b0 msgr2=0x7f924c0ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 --2- 172.21.15.1:0/2393825287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c0d08b0 0x7f924c0ccf90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f924c06e4b0 tx=0x7f924402a710 comp rx=0 tx=0).stop 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 -- 172.21.15.1:0/2393825287 shutdown_connections 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 --2- 172.21.15.1:0/2393825287 >> v2:172.21.15.6:3300/0 conn(0x7f924c0ceba0 0x7f924c0cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 --2- 172.21.15.1:0/2393825287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c0d08b0 0x7f924c0ccf90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 -- 172.21.15.1:0/2393825287 >> 172.21.15.1:0/2393825287 conn(0x7f924c0d1c10 msgr2=0x7f924c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 -- 172.21.15.1:0/2393825287 shutdown_connections 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.991+0000 7f9252510640 1 -- 172.21.15.1:0/2393825287 wait complete. 2024-05-09T06:30:18.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.992+0000 7f9252510640 1 Processor -- start 2024-05-09T06:30:18.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.992+0000 7f9252510640 1 -- start start 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f924c0ceba0 0x7f924c0c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 0x7f924c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f924c15e650 con 0x7f924c15e270 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0cca70 con 0x7f924c0d08b0 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9252510640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f924c0d0cc0 con 0x7f924c0ceba0 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42094/0 (socket says 172.21.15.1:42094) 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 -- 172.21.15.1:0/2614368730 learned_addr learned my addr 172.21.15.1:0/2614368730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 -- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f924c0ceba0 msgr2=0x7f924c0c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f924c0ceba0 0x7f924c0c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 -- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 msgr2=0x7f924c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f924b7fe640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 0x7f924c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.993+0000 7f9250aad640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 0x7f924c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9250aad640 1 -- 172.21.15.1:0/2614368730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c158640 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f924b7fe640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 0x7f924c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9250aad640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f924402ad00 tx=0x7f9244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244040070 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244031c90 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244045680 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c157880 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.994+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c157e40 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.995+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c0c6de0 con 0x7f924c15e270 2024-05-09T06:30:18.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.995+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9244039040 con 0x7f924c15e270 2024-05-09T06:30:18.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.996+0000 7f9248ff9640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 0x7f923806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.996+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f92440aca10 con 0x7f924c15e270 2024-05-09T06:30:18.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.996+0000 7f924bfff640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 0x7f923806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:18.996+0000 7f924bfff640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 0x7f923806e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f923c007fd0 tx=0x7f923c007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.000+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9244043360 con 0x7f924c15e270 2024-05-09T06:30:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.183+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f924c0752b0 con 0x7f924c15e270 2024-05-09T06:30:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.184+0000 7f9248ff9640 1 -- 172.21.15.1:0/2614368730 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7f924407b350 con 0x7f924c15e270 2024-05-09T06:30:19.166 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 msgr2=0x7f923806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 0x7f923806e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f923c007fd0 tx=0x7f923c007960 comp rx=0 tx=0).stop 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 msgr2=0x7f924c0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f924402ad00 tx=0x7f9244004a40 comp rx=0 tx=0).stop 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 shutdown_connections 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f923806be30 0x7f923806e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f924c15e270 0x7f924c0cac00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f924c0d08b0 0x7f924c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 --2- 172.21.15.1:0/2614368730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f924c0ceba0 0x7f924c0c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 >> 172.21.15.1:0/2614368730 conn(0x7f924c0d1c10 msgr2=0x7f924c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 shutdown_connections 2024-05-09T06:30:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.186+0000 7f9252510640 1 -- 172.21.15.1:0/2614368730 wait complete. 2024-05-09T06:30:19.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:30:19.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.302+0000 7f2390189640 1 Processor -- start 2024-05-09T06:30:19.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.302+0000 7f2390189640 1 -- start start 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f2390189640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f23881565e0 0x7f2388074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f2390189640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f238815c680 0x7f23880afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f2390189640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2388074a70 con 0x7f23880b0170 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f2390189640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f238815fd30 con 0x7f238815c680 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f2390189640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f238815cc40 con 0x7f23881565e0 2024-05-09T06:30:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f238815c680 0x7f23880afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f238815c680 0x7f23880afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42100/0 (socket says 172.21.15.1:42100) 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 -- 172.21.15.1:0/2054607002 learned_addr learned my addr 172.21.15.1:0/2054607002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 -- 172.21.15.1:0/2054607002 >> v2:172.21.15.6:3300/0 conn(0x7f23881565e0 msgr2=0x7f2388074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 --2- 172.21.15.1:0/2054607002 >> v2:172.21.15.6:3300/0 conn(0x7f23881565e0 0x7f2388074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 -- 172.21.15.1:0/2054607002 >> v1:172.21.15.6:6789/0 conn(0x7f23880b0170 legacy=0x7f23881601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.303+0000 7f238d724640 1 -- 172.21.15.1:0/2054607002 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238815cf20 con 0x7f238815c680 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.304+0000 7f238d724640 1 --2- 172.21.15.1:0/2054607002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f23880afc30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f23880d13d0 tx=0x7f237802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2e1010ed8abc0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.304+0000 7f238cf23640 1 -- 172.21.15.1:0/2054607002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378037070 con 0x7f238815c680 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.304+0000 7f238cf23640 1 -- 172.21.15.1:0/2054607002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2378032440 con 0x7f238815c680 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.304+0000 7f238cf23640 1 -- 172.21.15.1:0/2054607002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378032810 con 0x7f238815c680 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 -- 172.21.15.1:0/2054607002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 msgr2=0x7f23880afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 --2- 172.21.15.1:0/2054607002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f23880afc30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f23880d13d0 tx=0x7f237802a230 comp rx=0 tx=0).stop 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 -- 172.21.15.1:0/2054607002 shutdown_connections 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 --2- 172.21.15.1:0/2054607002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f23880afc30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 --2- 172.21.15.1:0/2054607002 >> v2:172.21.15.6:3300/0 conn(0x7f23881565e0 0x7f2388074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 -- 172.21.15.1:0/2054607002 >> 172.21.15.1:0/2054607002 conn(0x7f23880b1fb0 msgr2=0x7f23880b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:19.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 -- 172.21.15.1:0/2054607002 shutdown_connections 2024-05-09T06:30:19.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.305+0000 7f2390189640 1 -- 172.21.15.1:0/2054607002 wait complete. 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.306+0000 7f2390189640 1 Processor -- start 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.306+0000 7f2390189640 1 -- start start 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 0x7f238814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23881565e0 0x7f238813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23881569c0 con 0x7f238815c680 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f238815d4e0 con 0x7f23880b0170 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f2390189640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f23881608d0 con 0x7f23881565e0 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42114/0 (socket says 172.21.15.1:42114) 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 -- 172.21.15.1:0/4192649724 learned_addr learned my addr 172.21.15.1:0/4192649724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238df25640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 0x7f238814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238d724640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23881565e0 0x7f238813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 -- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23881565e0 msgr2=0x7f238813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23881565e0 0x7f238813cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 -- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 msgr2=0x7f238814c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 0x7f238814c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238e726640 1 -- 172.21.15.1:0/4192649724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238813d140 con 0x7f238815c680 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.307+0000 7f238df25640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 0x7f238814c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f238e726640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f238000b990 tx=0x7f238000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380019070 con 0x7f238815c680 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f238000cba0 con 0x7f238815c680 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2388159830 con 0x7f238815c680 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380014440 con 0x7f238815c680 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.308+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238813a470 con 0x7f238815c680 2024-05-09T06:30:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.309+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23880b1030 con 0x7f238815c680 2024-05-09T06:30:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.309+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f238001c070 con 0x7f238815c680 2024-05-09T06:30:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.310+0000 7f237e7fc640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 0x7f236c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.310+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f238008ad70 con 0x7f238815c680 2024-05-09T06:30:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.310+0000 7f238df25640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 0x7f236c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.310+0000 7f238df25640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 0x7f236c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.314+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23880b1030 con 0x7f238815c680 2024-05-09T06:30:19.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.508+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23880752b0 con 0x7f238815c680 2024-05-09T06:30:19.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.509+0000 7f237e7fc640 1 -- 172.21.15.1:0/4192649724 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f2380059560 con 0x7f238815c680 2024-05-09T06:30:19.490 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:30:19.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.510+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 msgr2=0x7f236c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.510+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 0x7f236c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384008040 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.510+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 msgr2=0x7f238815b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.510+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f238000b990 tx=0x7f238000be50 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 shutdown_connections 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f236c06be60 0x7f236c06e320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f238815c680 0x7f238815b830 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23881565e0 0x7f238813cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 --2- 172.21.15.1:0/4192649724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23880b0170 0x7f238814c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 >> 172.21.15.1:0/4192649724 conn(0x7f23880b1fb0 msgr2=0x7f23880b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 shutdown_connections 2024-05-09T06:30:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.511+0000 7f2390189640 1 -- 172.21.15.1:0/4192649724 wait complete. 2024-05-09T06:30:19.503 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:30:19.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:19.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:19.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[55766]: fuse finished with error 0 and tester_r 0 2024-05-09T06:30:19.530 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:30:19.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:19.530 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:30:19.626 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:30:19.626 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:30:19.627 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:19.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:30:19.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:30:19.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:19.645 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:30:19.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:19.703 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:30:19.704 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:30:19.704 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:30:19.704 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:19.721 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:30:19.721 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:30:19.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:19.722 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:30:19.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:30:19.778 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:30:19.778 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:30:19.778 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:19.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:30:19.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:30:19.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:19.797 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:30:19.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:19.855 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:30:19.856 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:19.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 Processor -- start 2024-05-09T06:30:19.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 -- start start 2024-05-09T06:30:19.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f34980c6860 0x7f3498099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34980b2aa0 0x7f349815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3498099930 con 0x7f34980c6c40 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f34980b1e20 con 0x7f34980b2aa0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.979+0000 7f349e309640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f349815ee90 con 0x7f34980c6860 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34980b2aa0 0x7f349815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f349c8a6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f34980c6c40 0x7f349815f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59390/0 (socket says 172.21.15.1:59390) 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f34980b2aa0 0x7f349815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42128/0 (socket says 172.21.15.1:42128) 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 -- 172.21.15.1:0/3152143241 learned_addr learned my addr 172.21.15.1:0/3152143241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f3497fff640 1 --2- 172.21.15.1:0/3152143241 >> v2:172.21.15.6:3300/0 conn(0x7f34980c6860 0x7f3498099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 -- 172.21.15.1:0/3152143241 >> v2:172.21.15.6:3300/0 conn(0x7f34980c6860 msgr2=0x7f3498099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 --2- 172.21.15.1:0/3152143241 >> v2:172.21.15.6:3300/0 conn(0x7f34980c6860 0x7f3498099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 -- 172.21.15.1:0/3152143241 >> v1:172.21.15.1:6789/0 conn(0x7f34980c6c40 legacy=0x7f349815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f34977fe640 1 -- 172.21.15.1:0/3152143241 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498162930 con 0x7f34980b2aa0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.980+0000 7f3496ffd640 1 -- 172.21.15.1:0/3152143241 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085181045 0 0) 0x7f3490001490 con 0x7f34980c6c40 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f34977fe640 1 --2- 172.21.15.1:0/3152143241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980b2aa0 0x7f349815e4b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34980722c0 tx=0x7f348c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3686ab142c79aca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f3496ffd640 1 -- 172.21.15.1:0/3152143241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c037070 con 0x7f34980b2aa0 2024-05-09T06:30:19.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f3496ffd640 1 -- 172.21.15.1:0/3152143241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c032440 con 0x7f34980b2aa0 2024-05-09T06:30:19.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f3496ffd640 1 -- 172.21.15.1:0/3152143241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c032810 con 0x7f34980b2aa0 2024-05-09T06:30:19.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 -- 172.21.15.1:0/3152143241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980b2aa0 msgr2=0x7f349815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 --2- 172.21.15.1:0/3152143241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980b2aa0 0x7f349815e4b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34980722c0 tx=0x7f348c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 -- 172.21.15.1:0/3152143241 shutdown_connections 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 --2- 172.21.15.1:0/3152143241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980b2aa0 0x7f349815e4b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 --2- 172.21.15.1:0/3152143241 >> v2:172.21.15.6:3300/0 conn(0x7f34980c6860 0x7f3498099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 -- 172.21.15.1:0/3152143241 >> 172.21.15.1:0/3152143241 conn(0x7f34980b1fb0 msgr2=0x7f34980b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 -- 172.21.15.1:0/3152143241 shutdown_connections 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.981+0000 7f349e309640 1 -- 172.21.15.1:0/3152143241 wait complete. 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.983+0000 7f349e309640 1 Processor -- start 2024-05-09T06:30:19.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.983+0000 7f349e309640 1 -- start start 2024-05-09T06:30:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.983+0000 7f349e309640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34980b2aa0 0x7f3498148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34980c6c40 0x7f3498146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f349815eb20 con 0x7f34980c6860 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f349815fa30 con 0x7f34980b2aa0 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f34980c70d0 con 0x7f34980c6c40 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42130/0 (socket says 172.21.15.1:42130) 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 -- 172.21.15.1:0/4147787395 learned_addr learned my addr 172.21.15.1:0/4147787395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f3497fff640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34980b2aa0 0x7f3498148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349c8a6640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34980c6c40 0x7f3498146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 -- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34980c6c40 msgr2=0x7f3498146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34980c6c40 0x7f3498146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 -- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34980b2aa0 msgr2=0x7f3498148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34980b2aa0 0x7f3498148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 -- 172.21.15.1:0/4147787395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498148dc0 con 0x7f34980c6860 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f34977fe640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f348c02a220 tx=0x7f348c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c041020 con 0x7f34980c6860 2024-05-09T06:30:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c0306c0 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34981490a0 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c037040 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.984+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3498149660 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.985+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34980752b0 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.986+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f348c044020 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.986+0000 7f3494ff9640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 0x7f348406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.986+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f348c0ab690 con 0x7f34980c6860 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.986+0000 7f3497fff640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 0x7f348406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:19.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.987+0000 7f3497fff640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 0x7f348406e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:19.991+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f348c04b030 con 0x7f34980c6860 2024-05-09T06:30:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.211+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f349806ce30 con 0x7f34980c6860 2024-05-09T06:30:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.212+0000 7f3494ff9640 1 -- 172.21.15.1:0/4147787395 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7f348c0a9e20 con 0x7f34980c6860 2024-05-09T06:30:20.194 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:20.194 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:30:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 msgr2=0x7f348406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 0x7f348406e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488008040 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 msgr2=0x7f34981f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f348c02a220 tx=0x7f348c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 shutdown_connections 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f348406be30 0x7f348406e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f34980c6c40 0x7f3498146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34980c6860 0x7f34981f77b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 --2- 172.21.15.1:0/4147787395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f34980b2aa0 0x7f3498148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 >> 172.21.15.1:0/4147787395 conn(0x7f34980b1fb0 msgr2=0x7f349806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 shutdown_connections 2024-05-09T06:30:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.214+0000 7f349e309640 1 -- 172.21.15.1:0/4147787395 wait complete. 2024-05-09T06:30:20.207 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:20.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.328+0000 7ff61125d640 1 Processor -- start 2024-05-09T06:30:20.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.328+0000 7ff61125d640 1 -- start start 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff61125d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff60c0d4940 0x7ff60c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff61125d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c0b33c0 con 0x7ff60c161950 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff61125d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c0b3660 con 0x7ff60c0d2b10 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff61125d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff60c0d4dd0 con 0x7ff60c0d4940 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff60ad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff60c161950 0x7ff60c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59398/0 (socket says 172.21.15.1:59398) 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.329+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3158135711 learned_addr learned my addr 172.21.15.1:0/3158135711 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:20.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3158135711 >> v2:172.21.15.6:3300/0 conn(0x7ff60c0d4940 0x7ff60c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280663409 0 0) 0x7ff60c0b33c0 con 0x7ff60c161950 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f8003520 con 0x7ff60c161950 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 622517449 0 0) 0x7ff5f8003520 con 0x7ff60c161950 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 >> v2:172.21.15.6:3300/0 conn(0x7ff60c0d4940 msgr2=0x7ff60c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 --2- 172.21.15.1:0/3158135711 >> v2:172.21.15.6:3300/0 conn(0x7ff60c0d4940 0x7ff60c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 >> v1:172.21.15.6:6789/0 conn(0x7ff60c0d2b10 legacy=0x7ff60c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c0b3940 con 0x7ff60c161950 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7ff5fc001670 con 0x7ff60c161950 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 >> v1:172.21.15.1:6789/0 conn(0x7ff60c161950 legacy=0x7ff60c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff5f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0d4dd0 con 0x7ff5f80039b0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f800c060 con 0x7ff5f8004800 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f800c260 con 0x7ff5f80051c0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff5f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60b59e640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80051c0 msgr2=0x7ff5f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8004800 msgr2=0x7ff5f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3158135711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8003520 con 0x7ff5f80039b0 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:20.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.330+0000 7ff60b59e640 1 -- 172.21.15.1:0/3158135711 reap_dead start 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.331+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff5f8003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c240 tx=0x7ff5f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ec354aa98fa30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.331+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007d10 con 0x7ff5f80039b0 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.331+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc005ad0 con 0x7ff5f80039b0 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.331+0000 7ff609d9b640 1 -- 172.21.15.1:0/3158135711 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007070 con 0x7ff5f80039b0 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 -- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 msgr2=0x7ff5f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff5f8003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c240 tx=0x7ff5f8005830 comp rx=0 tx=0).stop 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 -- 172.21.15.1:0/3158135711 shutdown_connections 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 --2- 172.21.15.1:0/3158135711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff5f8003dd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 -- 172.21.15.1:0/3158135711 >> 172.21.15.1:0/3158135711 conn(0x7ff60c0d3e20 msgr2=0x7ff60c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 -- 172.21.15.1:0/3158135711 shutdown_connections 2024-05-09T06:30:20.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.332+0000 7ff61125d640 1 -- 172.21.15.1:0/3158135711 wait complete. 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.333+0000 7ff61125d640 1 Processor -- start 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.333+0000 7ff61125d640 1 -- start start 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff60c0cc370 0x7ff60c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff60c0cc7d0 0x7ff60c0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0d1100 con 0x7ff5f80039b0 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0d0f80 con 0x7ff60c0cc370 2024-05-09T06:30:20.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff61125d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c0d1c00 con 0x7ff60c0cc7d0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42144/0 (socket says 172.21.15.1:42144) 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3175913273 learned_addr learned my addr 172.21.15.1:0/3175913273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff60c0cc370 0x7ff60c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff60c0cc7d0 msgr2=0x7ff60c0b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.334+0000 7ff60b59e640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff60c0cc7d0 0x7ff60c0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff60c0cc7d0 0x7ff60c0b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff60c0cc370 msgr2=0x7ff60c150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff60c0cc370 0x7ff60c150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff60ad9d640 1 -- 172.21.15.1:0/3175913273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c0cb2d0 con 0x7ff5f80039b0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff60ad9d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c360 tx=0x7ff60c0b3b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc041070 con 0x7ff5f80039b0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc000ed0 con 0x7ff5f80039b0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007d10 con 0x7ff5f80039b0 2024-05-09T06:30:20.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c0cb5b0 con 0x7ff5f80039b0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.335+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c0cbb70 con 0x7ff5f80039b0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.336+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c0752b0 con 0x7ff5f80039b0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.336+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5fc044070 con 0x7ff5f80039b0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.337+0000 7ff6037fe640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 0x7ff5ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.337+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7ff5fc0ab4a0 con 0x7ff5f80039b0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.337+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 0x7ff5ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.337+0000 7ff60a59c640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 0x7ff5ec06e320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0ccd90 tx=0x7ff5f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.341+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5fc079c80 con 0x7ff5f80039b0 2024-05-09T06:30:20.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.525+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff60c0d21e0 con 0x7ff5f80039b0 2024-05-09T06:30:20.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.527+0000 7ff6037fe640 1 -- 172.21.15.1:0/3175913273 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16850 (secure 0 0 0) 0x7ff5fc079580 con 0x7ff5f80039b0 2024-05-09T06:30:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:20.509 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:03.192204+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 msgr2=0x7ff5ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 0x7ff5ec06e320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0ccd90 tx=0x7ff5f4008040 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 msgr2=0x7ff60c0b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c360 tx=0x7ff60c0b3b10 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 shutdown_connections 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff5ec06be60 0x7ff5ec06e320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff60c0cc7d0 0x7ff60c0b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff60c0cc370 0x7ff60c150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 --2- 172.21.15.1:0/3175913273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f80039b0 0x7ff60c0b7450 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 >> 172.21.15.1:0/3175913273 conn(0x7ff60c0d3e20 msgr2=0x7ff60c15f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 shutdown_connections 2024-05-09T06:30:20.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.529+0000 7ff61125d640 1 -- 172.21.15.1:0/3175913273 wait complete. 2024-05-09T06:30:20.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:29:31.502757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:29:32.177695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:30:20.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:30:20.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 Processor -- start 2024-05-09T06:30:20.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 -- start start 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46e80d45f0 0x7f46e80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e8161fe0 con 0x7f46e80d1c70 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f46e815f580 con 0x7f46e80d45f0 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.648+0000 7f46eee58640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46e81622c0 con 0x7f46e815e190 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46e80d45f0 0x7f46e80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46e7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f46e80d1c70 0x7f46e80d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59414/0 (socket says 172.21.15.1:59414) 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46e80d45f0 0x7f46e80b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42160/0 (socket says 172.21.15.1:42160) 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46e7fff640 1 -- 172.21.15.1:0/4200140256 learned_addr learned my addr 172.21.15.1:0/4200140256 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ecbf4640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 -- 172.21.15.1:0/4200140256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 msgr2=0x7f46e815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 -- 172.21.15.1:0/4200140256 >> v1:172.21.15.1:6789/0 conn(0x7f46e80d1c70 legacy=0x7f46e80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.649+0000 7f46ed3f5640 1 -- 172.21.15.1:0/4200140256 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81625a0 con 0x7f46e80d45f0 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46ed3f5640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d45f0 0x7f46e80b3de0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bd4086d6bc2023 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46e77fe640 1 -- 172.21.15.1:0/4200140256 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0007740 con 0x7f46e80d45f0 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46e77fe640 1 -- 172.21.15.1:0/4200140256 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e00078e0 con 0x7f46e80d45f0 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46e77fe640 1 -- 172.21.15.1:0/4200140256 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0007cb0 con 0x7f46e80d45f0 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46eee58640 1 -- 172.21.15.1:0/4200140256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d45f0 msgr2=0x7f46e80b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.650+0000 7f46eee58640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d45f0 0x7f46e80b3de0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009d90 tx=0x7f46e002a660 comp rx=0 tx=0).stop 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 -- 172.21.15.1:0/4200140256 shutdown_connections 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d45f0 0x7f46e80b3de0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 --2- 172.21.15.1:0/4200140256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 -- 172.21.15.1:0/4200140256 >> 172.21.15.1:0/4200140256 conn(0x7f46e8073340 msgr2=0x7f46e809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 -- 172.21.15.1:0/4200140256 shutdown_connections 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.651+0000 7f46eee58640 1 -- 172.21.15.1:0/4200140256 wait complete. 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.652+0000 7f46eee58640 1 Processor -- start 2024-05-09T06:30:20.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.652+0000 7f46eee58640 1 -- start start 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 0x7f46e8158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b56d0 con 0x7f46e80d1c70 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8161c60 con 0x7f46e80d45f0 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46eee58640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80b5850 con 0x7f46e815e190 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42172/0 (socket says 172.21.15.1:42172) 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 -- 172.21.15.1:0/2860331128 learned_addr learned my addr 172.21.15.1:0/2860331128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ed3f5640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 -- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 msgr2=0x7f46e80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46e7fff640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 0x7f46e8158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 -- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 msgr2=0x7f46e8158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 0x7f46e8158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 -- 172.21.15.1:0/2860331128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81f7c30 con 0x7f46e80d1c70 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ed3f5640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46e7fff640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 0x7f46e8158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.653+0000 7f46ecbf4640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e870 tx=0x7f46dc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.654+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc017070 con 0x7f46e80d1c70 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.654+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc00c940 con 0x7f46e80d1c70 2024-05-09T06:30:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.654+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81f7eb0 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.654+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00cd30 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.654+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81f8470 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.655+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e80752b0 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.655+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46dc01a070 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.656+0000 7f46e57fa640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.656+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f46dc08ae40 con 0x7f46e80d1c70 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.656+0000 7f46e7fff640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.656+0000 7f46e7fff640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 0x7f46cc06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46e81f7650 tx=0x7f46d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.660+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46e81f7c30 con 0x7f46e80d1c70 2024-05-09T06:30:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.844+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46e808e1c0 con 0x7f46e80d1c70 2024-05-09T06:30:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.845+0000 7f46e57fa640 1 -- 172.21.15.1:0/2860331128 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f46dc0596c0 con 0x7f46e80d1c70 2024-05-09T06:30:20.827 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 msgr2=0x7f46cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 0x7f46cc06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46e81f7650 tx=0x7f46d4009040 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 msgr2=0x7f46e81f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e870 tx=0x7f46dc00ed30 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 shutdown_connections 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46cc06be30 0x7f46cc06e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46e815e190 0x7f46e80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e80d45f0 0x7f46e8158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 --2- 172.21.15.1:0/2860331128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46e80d1c70 0x7f46e81f76f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 >> 172.21.15.1:0/2860331128 conn(0x7f46e8073340 msgr2=0x7f46e806d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 shutdown_connections 2024-05-09T06:30:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.847+0000 7f46eee58640 1 -- 172.21.15.1:0/2860331128 wait complete. 2024-05-09T06:30:20.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:30:20.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.965+0000 7f69f9709640 1 Processor -- start 2024-05-09T06:30:20.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.965+0000 7f69f9709640 1 -- start start 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f9709640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69f40d1c70 0x7f69f415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f9709640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f9709640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f4161fe0 con 0x7f69f415e190 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f9709640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69f415f580 con 0x7f69f40d1c70 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f9709640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69f41622c0 con 0x7f69f40d45f0 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f27fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69f40d1c70 0x7f69f415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f2ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69f415e190 0x7f69f40d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59428/0 (socket says 172.21.15.1:59428) 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.966+0000 7f69f2ffd640 1 -- 172.21.15.1:0/3914394168 learned_addr learned my addr 172.21.15.1:0/3914394168 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f37fe640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f27fc640 1 -- 172.21.15.1:0/3914394168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 msgr2=0x7f69f40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f27fc640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f40b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f27fc640 1 -- 172.21.15.1:0/3914394168 >> v1:172.21.15.1:6789/0 conn(0x7f69f415e190 legacy=0x7f69f40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f1ffb640 1 -- 172.21.15.1:0/3914394168 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256224234 0 0) 0x7f69f4161fe0 con 0x7f69f415e190 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f27fc640 1 -- 172.21.15.1:0/3914394168 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f41625a0 con 0x7f69f40d1c70 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f37fe640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f40b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.967+0000 7f69f27fc640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f40d1c70 0x7f69f415f780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0097a0 tx=0x7f69dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9b93c8ebb85e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f1ffb640 1 -- 172.21.15.1:0/3914394168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc037070 con 0x7f69f40d1c70 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f1ffb640 1 -- 172.21.15.1:0/3914394168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69dc032440 con 0x7f69f40d1c70 2024-05-09T06:30:20.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f1ffb640 1 -- 172.21.15.1:0/3914394168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc032810 con 0x7f69f40d1c70 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 -- 172.21.15.1:0/3914394168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f40d1c70 msgr2=0x7f69f415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f40d1c70 0x7f69f415f780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0097a0 tx=0x7f69dc02a180 comp rx=0 tx=0).stop 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 -- 172.21.15.1:0/3914394168 shutdown_connections 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f40b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 --2- 172.21.15.1:0/3914394168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f40d1c70 0x7f69f415f780 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 -- 172.21.15.1:0/3914394168 >> 172.21.15.1:0/3914394168 conn(0x7f69f4073340 msgr2=0x7f69f409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 -- 172.21.15.1:0/3914394168 shutdown_connections 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.968+0000 7f69f9709640 1 -- 172.21.15.1:0/3914394168 wait complete. 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.969+0000 7f69f9709640 1 Processor -- start 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 -- start start 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69f40d1c70 0x7f69f4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69f40b56d0 con 0x7f69f415e190 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4161c60 con 0x7f69f40d1c70 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f9709640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69f40b5850 con 0x7f69f40d45f0 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42190/0 (socket says 172.21.15.1:42190) 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.970+0000 7f69f37fe640 1 -- 172.21.15.1:0/425932937 learned_addr learned my addr 172.21.15.1:0/425932937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f2ffd640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69f40d1c70 0x7f69f4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f27fc640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 -- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 msgr2=0x7f69f415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 -- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69f40d1c70 msgr2=0x7f69f4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69f40d1c70 0x7f69f4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 -- 172.21.15.1:0/425932937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f40ccc40 con 0x7f69f415e190 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f37fe640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f69e400e5e0 tx=0x7f69e400eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e400f040 con 0x7f69f415e190 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e4004230 con 0x7f69f415e190 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f40ccf20 con 0x7f69f415e190 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4003610 con 0x7f69f415e190 2024-05-09T06:30:20.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.971+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f40cd4e0 con 0x7f69f415e190 2024-05-09T06:30:20.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.972+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f40752b0 con 0x7f69f415e190 2024-05-09T06:30:20.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.973+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69e40037b0 con 0x7f69f415e190 2024-05-09T06:30:20.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.973+0000 7f69eb7fe640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 0x7f69d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:20.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.973+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f69e408aca0 con 0x7f69f415e190 2024-05-09T06:30:20.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.973+0000 7f69f2ffd640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 0x7f69d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.974+0000 7f69f2ffd640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 0x7f69d406e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0012e0 tx=0x7f69ec001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:20.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:20.978+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f69e4017020 con 0x7f69f415e190 2024-05-09T06:30:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.160+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69f408e1c0 con 0x7f69f415e190 2024-05-09T06:30:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.160+0000 7f69eb7fe640 1 -- 172.21.15.1:0/425932937 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) ==== 104+0+11 (secure 0 0 0) 0x7f69e4058e30 con 0x7f69f415e190 2024-05-09T06:30:21.142 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 msgr2=0x7f69d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 0x7f69d406e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0012e0 tx=0x7f69ec001630 comp rx=0 tx=0).stop 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 msgr2=0x7f69f415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f69e400e5e0 tx=0x7f69e400eaa0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 shutdown_connections 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f69d406be30 0x7f69d406e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69f415e190 0x7f69f415ce70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69f40d45f0 0x7f69f415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 --2- 172.21.15.1:0/425932937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69f40d1c70 0x7f69f4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 >> 172.21.15.1:0/425932937 conn(0x7f69f4073340 msgr2=0x7f69f40d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 shutdown_connections 2024-05-09T06:30:21.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.162+0000 7f69f9709640 1 -- 172.21.15.1:0/425932937 wait complete. 2024-05-09T06:30:21.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:30:21.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.282+0000 7fb17619e640 1 Processor -- start 2024-05-09T06:30:21.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.282+0000 7fb17619e640 1 -- start start 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17619e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb1700d2950 0x7fb1700d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17619e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb1700b2b60 0x7fb1700b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17619e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1700cfbd0 con 0x7fb17015e230 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17619e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb1700d1c50 con 0x7fb1700b2b60 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17619e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb1700b0930 con 0x7fb1700d2950 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17599d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb1700b2b60 0x7fb1700b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17519c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb17015e230 0x7fb1700d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59430/0 (socket says 172.21.15.1:59430) 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17519c640 1 -- 172.21.15.1:0/1430763977 learned_addr learned my addr 172.21.15.1:0/1430763977 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17599d640 1 -- 172.21.15.1:0/1430763977 >> v2:172.21.15.6:3300/0 conn(0x7fb1700d2950 msgr2=0x7fb1700d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:21.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17599d640 1 --2- 172.21.15.1:0/1430763977 >> v2:172.21.15.6:3300/0 conn(0x7fb1700d2950 0x7fb1700d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17599d640 1 -- 172.21.15.1:0/1430763977 >> v1:172.21.15.1:6789/0 conn(0x7fb17015e230 legacy=0x7fb1700d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb167fff640 1 -- 172.21.15.1:0/1430763977 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157894008 0 0) 0x7fb1700cfbd0 con 0x7fb17015e230 2024-05-09T06:30:21.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.283+0000 7fb17599d640 1 -- 172.21.15.1:0/1430763977 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1700d1f30 con 0x7fb1700b2b60 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.284+0000 7fb17599d640 1 --2- 172.21.15.1:0/1430763977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700b2b60 0x7fb1700b0e50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb17006e3a0 tx=0x7fb16802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86973b244b29373 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.284+0000 7fb167fff640 1 -- 172.21.15.1:0/1430763977 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168007740 con 0x7fb1700b2b60 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.284+0000 7fb167fff640 1 -- 172.21.15.1:0/1430763977 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1680078e0 con 0x7fb1700b2b60 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.284+0000 7fb167fff640 1 -- 172.21.15.1:0/1430763977 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168007cb0 con 0x7fb1700b2b60 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 -- 172.21.15.1:0/1430763977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700b2b60 msgr2=0x7fb1700b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 --2- 172.21.15.1:0/1430763977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700b2b60 0x7fb1700b0e50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb17006e3a0 tx=0x7fb16802a680 comp rx=0 tx=0).stop 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 -- 172.21.15.1:0/1430763977 shutdown_connections 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 --2- 172.21.15.1:0/1430763977 >> v2:172.21.15.6:3300/0 conn(0x7fb1700d2950 0x7fb1700d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 --2- 172.21.15.1:0/1430763977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700b2b60 0x7fb1700b0e50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 -- 172.21.15.1:0/1430763977 >> 172.21.15.1:0/1430763977 conn(0x7fb1700d3c60 msgr2=0x7fb1700d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 -- 172.21.15.1:0/1430763977 shutdown_connections 2024-05-09T06:30:21.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.285+0000 7fb17619e640 1 -- 172.21.15.1:0/1430763977 wait complete. 2024-05-09T06:30:21.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.286+0000 7fb17619e640 1 Processor -- start 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.286+0000 7fb17619e640 1 -- start start 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1700b2b60 0x7fb17015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb17015e230 0x7fb1700b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb1700b1510 con 0x7fb1700d2950 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb1700b1390 con 0x7fb1700b2b60 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17619e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb1700b1690 con 0x7fb17015e230 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42216/0 (socket says 172.21.15.1:42216) 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 -- 172.21.15.1:0/1851484267 learned_addr learned my addr 172.21.15.1:0/1851484267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 -- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb17015e230 msgr2=0x7fb1700b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb17015e230 0x7fb1700b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 -- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1700b2b60 msgr2=0x7fb17015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17519c640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1700b2b60 0x7fb17015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1700b2b60 0x7fb17015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 -- 172.21.15.1:0/1851484267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1700cb260 con 0x7fb1700d2950 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.287+0000 7fb17499b640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb16000b480 tx=0x7fb16000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.288+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160014070 con 0x7fb1700d2950 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.288+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb160010870 con 0x7fb1700d2950 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.288+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1700cb540 con 0x7fb1700d2950 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.288+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160010c60 con 0x7fb1700d2950 2024-05-09T06:30:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.288+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1700cbb00 con 0x7fb1700d2950 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.289+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1700752b0 con 0x7fb1700d2950 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.289+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb1700d2950 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.290+0000 7fb165ffb640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.290+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fb16008ae60 con 0x7fb1700d2950 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.290+0000 7fb17519c640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.290+0000 7fb17519c640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 0x7fb15406e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f60 tx=0x7fb16c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.294+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb1600596e0 con 0x7fb1700d2950 2024-05-09T06:30:21.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.475+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb17015c270 con 0x7fb1700d2950 2024-05-09T06:30:21.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.476+0000 7fb165ffb640 1 -- 172.21.15.1:0/1851484267 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) ==== 100+0+11 (secure 0 0 0) 0x7fb160058fe0 con 0x7fb1700d2950 2024-05-09T06:30:21.457 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:30:21.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.477+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 msgr2=0x7fb15406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.477+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 0x7fb15406e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f60 tx=0x7fb16c009040 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.477+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 msgr2=0x7fb1700b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.477+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb16000b480 tx=0x7fb16000b940 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 shutdown_connections 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb15406be30 0x7fb15406e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb17015e230 0x7fb1700b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1700d2950 0x7fb1700b5a40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 --2- 172.21.15.1:0/1851484267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1700b2b60 0x7fb17015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 >> 172.21.15.1:0/1851484267 conn(0x7fb1700d3c60 msgr2=0x7fb17006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 shutdown_connections 2024-05-09T06:30:21.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.478+0000 7fb17619e640 1 -- 172.21.15.1:0/1851484267 wait complete. 2024-05-09T06:30:21.470 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.592+0000 7ff652ff4640 1 Processor -- start 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.592+0000 7ff652ff4640 1 -- start start 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff652ff4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff64c0d4940 0x7ff64c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff652ff4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff64c0d2b10 0x7ff64c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff652ff4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c0b33c0 con 0x7ff64c161950 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff652ff4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff64c0d4dd0 con 0x7ff64c0d2b10 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff652ff4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff64c0b3660 con 0x7ff64c0d4940 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff650d90640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff64c161950 0x7ff64c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59442/0 (socket says 172.21.15.1:59442) 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff650d90640 1 -- 172.21.15.1:0/3582627722 learned_addr learned my addr 172.21.15.1:0/3582627722 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.593+0000 7ff651591640 1 --2- 172.21.15.1:0/3582627722 >> v2:172.21.15.1:3300/0 conn(0x7ff64c0d2b10 0x7ff64c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff64bfff640 1 --2- 172.21.15.1:0/3582627722 >> v2:172.21.15.6:3300/0 conn(0x7ff64c0d4940 0x7ff64c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff64b7fe640 1 -- 172.21.15.1:0/3582627722 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828715536 0 0) 0x7ff64c0b33c0 con 0x7ff64c161950 2024-05-09T06:30:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff64b7fe640 1 -- 172.21.15.1:0/3582627722 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff634003520 con 0x7ff64c161950 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff651591640 1 -- 172.21.15.1:0/3582627722 >> v2:172.21.15.6:3300/0 conn(0x7ff64c0d4940 msgr2=0x7ff64c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff651591640 1 --2- 172.21.15.1:0/3582627722 >> v2:172.21.15.6:3300/0 conn(0x7ff64c0d4940 0x7ff64c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff651591640 1 -- 172.21.15.1:0/3582627722 >> v1:172.21.15.1:6789/0 conn(0x7ff64c161950 legacy=0x7ff64c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.594+0000 7ff651591640 1 -- 172.21.15.1:0/3582627722 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c0b3940 con 0x7ff64c0d2b10 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff651591640 1 --2- 172.21.15.1:0/3582627722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0d2b10 0x7ff64c0b4680 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff64c06e3d0 tx=0x7ff64402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980227d18b021e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff64b7fe640 1 -- 172.21.15.1:0/3582627722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644007740 con 0x7ff64c0d2b10 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff64b7fe640 1 -- 172.21.15.1:0/3582627722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6440078e0 con 0x7ff64c0d2b10 2024-05-09T06:30:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff64b7fe640 1 -- 172.21.15.1:0/3582627722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644007cb0 con 0x7ff64c0d2b10 2024-05-09T06:30:21.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 -- 172.21.15.1:0/3582627722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0d2b10 msgr2=0x7ff64c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 --2- 172.21.15.1:0/3582627722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0d2b10 0x7ff64c0b4680 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff64c06e3d0 tx=0x7ff64402a680 comp rx=0 tx=0).stop 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 -- 172.21.15.1:0/3582627722 shutdown_connections 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 --2- 172.21.15.1:0/3582627722 >> v2:172.21.15.6:3300/0 conn(0x7ff64c0d4940 0x7ff64c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 --2- 172.21.15.1:0/3582627722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0d2b10 0x7ff64c0b4680 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 -- 172.21.15.1:0/3582627722 >> 172.21.15.1:0/3582627722 conn(0x7ff64c0d3e20 msgr2=0x7ff64c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 -- 172.21.15.1:0/3582627722 shutdown_connections 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.595+0000 7ff652ff4640 1 -- 172.21.15.1:0/3582627722 wait complete. 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.596+0000 7ff652ff4640 1 Processor -- start 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 -- start start 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff64c0d2b10 0x7ff64c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff64c161950 0x7ff64c0b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c0d1100 con 0x7ff64c0cd020 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c0d0f80 con 0x7ff64c0d2b10 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.597+0000 7ff652ff4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c0d1c00 con 0x7ff64c161950 2024-05-09T06:30:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42244/0 (socket says 172.21.15.1:42244) 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 -- 172.21.15.1:0/1450533485 learned_addr learned my addr 172.21.15.1:0/1450533485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff650d90640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff64c0d2b10 0x7ff64c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff64bfff640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff64c161950 0x7ff64c0b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 -- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff64c161950 msgr2=0x7ff64c0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff64c161950 0x7ff64c0b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 -- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff64c0d2b10 msgr2=0x7ff64c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff64c0d2b10 0x7ff64c154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 -- 172.21.15.1:0/1450533485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c154790 con 0x7ff64c0cd020 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff651591640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff64402ac70 tx=0x7ff644004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644040070 con 0x7ff64c0cd020 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff644031bc0 con 0x7ff64c0cd020 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c0a0070 con 0x7ff64c0cd020 2024-05-09T06:30:21.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644009450 con 0x7ff64c0cd020 2024-05-09T06:30:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.598+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c0a0630 con 0x7ff64c0cd020 2024-05-09T06:30:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.599+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c0752b0 con 0x7ff64c0cd020 2024-05-09T06:30:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.600+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff644039040 con 0x7ff64c0cd020 2024-05-09T06:30:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.600+0000 7ff6497fa640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 0x7ff63806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.600+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7ff6440ac720 con 0x7ff64c0cd020 2024-05-09T06:30:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.600+0000 7ff650d90640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 0x7ff63806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.601+0000 7ff650d90640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 0x7ff63806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005880 tx=0x7ff63c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.605+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff64407a8c0 con 0x7ff64c0cd020 2024-05-09T06:30:21.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.828+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff64c0d26c0 con 0x7ff64c0cd020 2024-05-09T06:30:21.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.828+0000 7ff6497fa640 1 -- 172.21.15.1:0/1450533485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7ff64407ebc0 con 0x7ff64c0cd020 2024-05-09T06:30:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:21.811 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:30:21.811 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 msgr2=0x7ff63806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 0x7ff63806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005880 tx=0x7ff63c009040 comp rx=0 tx=0).stop 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 msgr2=0x7ff64c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff64402ac70 tx=0x7ff644004a40 comp rx=0 tx=0).stop 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 shutdown_connections 2024-05-09T06:30:21.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63806be30 0x7ff63806e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff64c0cd020 0x7ff64c0cd4b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff64c161950 0x7ff64c0b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 --2- 172.21.15.1:0/1450533485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff64c0d2b10 0x7ff64c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 >> 172.21.15.1:0/1450533485 conn(0x7ff64c0d3e20 msgr2=0x7ff64c0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 shutdown_connections 2024-05-09T06:30:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.830+0000 7ff652ff4640 1 -- 172.21.15.1:0/1450533485 wait complete. 2024-05-09T06:30:21.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.947+0000 7fcf16fc1640 1 Processor -- start 2024-05-09T06:30:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.947+0000 7fcf16fc1640 1 -- start start 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf16fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcf100b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf16fc1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1006eb40 con 0x7fcf100b2a60 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf16fc1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1006ee20 con 0x7fcf100b4f70 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf16fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1006d8e0 con 0x7fcf101454b0 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf0ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcf100b2a60 0x7fcf100b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59454/0 (socket says 172.21.15.1:59454) 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.948+0000 7fcf0ffff640 1 -- 172.21.15.1:0/1499276323 learned_addr learned my addr 172.21.15.1:0/1499276323 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018786423 0 0) 0x7fcf1006eb40 con 0x7fcf100b2a60 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcf100b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcefc003520 con 0x7fcf100b2a60 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849532359 0 0) 0x7fcefc003520 con 0x7fcf100b2a60 2024-05-09T06:30:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 msgr2=0x7fcf100b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcf100b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 >> v1:172.21.15.6:6789/0 conn(0x7fcf100b4f70 legacy=0x7fcf10070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1006dbc0 con 0x7fcf100b2a60 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fcf000019f0 con 0x7fcf100b2a60 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 >> v1:172.21.15.1:6789/0 conn(0x7fcf100b2a60 legacy=0x7fcf100b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcefc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcefc004d40 0x7fcefc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1006d8e0 con 0x7fcefc004d40 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc00bb50 con 0x7fcefc0039e0 2024-05-09T06:30:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc00bd20 con 0x7fcf101454b0 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcefc004d40 0x7fcefc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf0ffff640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcefc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 -- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 msgr2=0x7fcefc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf101454b0 0x7fcefc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 -- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcefc0039e0 msgr2=0x7fcefc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.949+0000 7fcf1555e640 1 -- 172.21.15.1:0/1499276323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc003520 con 0x7fcefc004d40 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.950+0000 7fcf1555e640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcefc004d40 0x7fcefc0051c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcef8001750 tx=0x7fcefc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5760bcf1c683a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.950+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8001370 con 0x7fcefc004d40 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.950+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef80052d0 con 0x7fcefc004d40 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.950+0000 7fcf0f7fe640 1 -- 172.21.15.1:0/1499276323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef802aba0 con 0x7fcefc004d40 2024-05-09T06:30:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 -- 172.21.15.1:0/1499276323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcefc004d40 msgr2=0x7fcefc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/1499276323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcefc004d40 0x7fcefc0051c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcef8001750 tx=0x7fcefc005830 comp rx=0 tx=0).stop 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf1555e640 1 -- 172.21.15.1:0/1499276323 reap_dead start 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 -- 172.21.15.1:0/1499276323 shutdown_connections 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 -- 172.21.15.1:0/1499276323 >> 172.21.15.1:0/1499276323 conn(0x7fcf100cc4b0 msgr2=0x7fcf100cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 -- 172.21.15.1:0/1499276323 shutdown_connections 2024-05-09T06:30:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.951+0000 7fcf16fc1640 1 -- 172.21.15.1:0/1499276323 wait complete. 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.952+0000 7fcf16fc1640 1 Processor -- start 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.952+0000 7fcf16fc1640 1 -- start start 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 0x7fcf1015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf100c9750 0x7fcf100c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1006e260 con 0x7fcf100c49d0 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1006e0e0 con 0x7fcf100c9750 2024-05-09T06:30:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf16fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf100b4320 con 0x7fcefc004d40 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42264/0 (socket says 172.21.15.1:42264) 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 -- 172.21.15.1:0/3411469046 learned_addr learned my addr 172.21.15.1:0/3411469046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 0x7fcf1015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf0ffff640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf100c9750 0x7fcf100c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 -- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 msgr2=0x7fcf1015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 0x7fcf1015b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 -- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf100c9750 msgr2=0x7fcf100c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf100c9750 0x7fcf100c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf1555e640 1 -- 172.21.15.1:0/3411469046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf100c66c0 con 0x7fcf100c49d0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.953+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 0x7fcf1015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf1555e640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcef802a590 tx=0x7fcef8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8037040 con 0x7fcf100c49d0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef8040070 con 0x7fcf100c49d0 2024-05-09T06:30:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf100c5390 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8007a30 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.954+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf100c5900 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.955+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf100752b0 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.955+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcef80052d0 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.956+0000 7fcf0d7fa640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 0x7fcef006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.956+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fcef80aa900 con 0x7fcf100c49d0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.956+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 0x7fcef006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:21.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.956+0000 7fcf14d5d640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 0x7fcef006e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf08006fd0 tx=0x7fcf08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:21.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:21.960+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcef8079060 con 0x7fcf100c49d0 2024-05-09T06:30:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.144+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf10072fa0 con 0x7fcf100c49d0 2024-05-09T06:30:22.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.145+0000 7fcf0d7fa640 1 -- 172.21.15.1:0/3411469046 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16850 (secure 0 0 0) 0x7fcef8078960 con 0x7fcf100c49d0 2024-05-09T06:30:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:22.127 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:03.192204+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 msgr2=0x7fcef006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 0x7fcef006e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf08006fd0 tx=0x7fcf08008040 comp rx=0 tx=0).stop 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 msgr2=0x7fcf100c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcef802a590 tx=0x7fcef8001120 comp rx=0 tx=0).stop 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 shutdown_connections 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcef006be60 0x7fcef006e320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf100c49d0 0x7fcf100c4e50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf100c9750 0x7fcf100c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 --2- 172.21.15.1:0/3411469046 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcefc004d40 0x7fcf1015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 >> 172.21.15.1:0/3411469046 conn(0x7fcf100cc4b0 msgr2=0x7fcf100b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 shutdown_connections 2024-05-09T06:30:22.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.147+0000 7fcf16fc1640 1 -- 172.21.15.1:0/3411469046 wait complete. 2024-05-09T06:30:22.140 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:30:22.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.263+0000 7ffa33b14640 1 Processor -- start 2024-05-09T06:30:22.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.263+0000 7ffa33b14640 1 -- start start 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa33b14640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffa2c06e4f0 0x7ffa2c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa33b14640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa33b14640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2c15eb40 con 0x7ffa2c15e050 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa33b14640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2c0b3b80 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa33b14640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2c0aef90 con 0x7ffa2c06e4f0 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42280/0 (socket says 172.21.15.1:42280) 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 -- 172.21.15.1:0/2568457727 learned_addr learned my addr 172.21.15.1:0/2568457727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa310af640 1 --2- 172.21.15.1:0/2568457727 >> v2:172.21.15.6:3300/0 conn(0x7ffa2c06e4f0 0x7ffa2c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 -- 172.21.15.1:0/2568457727 >> v2:172.21.15.6:3300/0 conn(0x7ffa2c06e4f0 msgr2=0x7ffa2c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 --2- 172.21.15.1:0/2568457727 >> v2:172.21.15.6:3300/0 conn(0x7ffa2c06e4f0 0x7ffa2c06e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 -- 172.21.15.1:0/2568457727 >> v1:172.21.15.6:6789/0 conn(0x7ffa2c15e050 legacy=0x7ffa2c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.264+0000 7ffa320b1640 1 -- 172.21.15.1:0/2568457727 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c0b3530 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.265+0000 7ffa320b1640 1 --2- 172.21.15.1:0/2568457727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0af5e0 tx=0x7ffa2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2ac00f82a633d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.265+0000 7ffa308ae640 1 -- 172.21.15.1:0/2568457727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24037070 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.265+0000 7ffa308ae640 1 -- 172.21.15.1:0/2568457727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24032440 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.265+0000 7ffa308ae640 1 -- 172.21.15.1:0/2568457727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24032810 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 -- 172.21.15.1:0/2568457727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c0ae2a0 msgr2=0x7ffa2c0ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 --2- 172.21.15.1:0/2568457727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0af5e0 tx=0x7ffa2402a230 comp rx=0 tx=0).stop 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 -- 172.21.15.1:0/2568457727 shutdown_connections 2024-05-09T06:30:22.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 --2- 172.21.15.1:0/2568457727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0ae6a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 --2- 172.21.15.1:0/2568457727 >> v2:172.21.15.6:3300/0 conn(0x7ffa2c06e4f0 0x7ffa2c06e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 -- 172.21.15.1:0/2568457727 >> 172.21.15.1:0/2568457727 conn(0x7ffa2c06d270 msgr2=0x7ffa2c06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 -- 172.21.15.1:0/2568457727 shutdown_connections 2024-05-09T06:30:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.266+0000 7ffa33b14640 1 -- 172.21.15.1:0/2568457727 wait complete. 2024-05-09T06:30:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.267+0000 7ffa33b14640 1 Processor -- start 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.267+0000 7ffa33b14640 1 -- start start 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 0x7ffa2c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c0aebe0 con 0x7ffa2c15e050 2024-05-09T06:30:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c0b4010 con 0x7ffa2c0ae2a0 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa33b14640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c0ce240 con 0x7ffa2c06e4f0 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42284/0 (socket says 172.21.15.1:42284) 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 -- 172.21.15.1:0/187564943 learned_addr learned my addr 172.21.15.1:0/187564943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa318b0640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 0x7ffa2c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa310af640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 -- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 msgr2=0x7ffa2c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 0x7ffa2c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 -- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 msgr2=0x7ffa2c0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 -- 172.21.15.1:0/187564943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c0cd910 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa310af640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa318b0640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 0x7ffa2c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.268+0000 7ffa320b1640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ffa240090b0 tx=0x7ffa240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.269+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24041020 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.269+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa240306c0 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.269+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c0cdbf0 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.269+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24037040 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.269+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c156820 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.270+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2c0752b0 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.270+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa24044020 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.271+0000 7ffa2a7fc640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 0x7ffa1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.271+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7ffa240ab860 con 0x7ffa2c15e050 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.271+0000 7ffa318b0640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 0x7ffa1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.271+0000 7ffa318b0640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 0x7ffa1806e320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009fc0 tx=0x7ffa1c009550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.275+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa2404b030 con 0x7ffa2c15e050 2024-05-09T06:30:22.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.497+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa2c0b4520 con 0x7ffa2c15e050 2024-05-09T06:30:22.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.497+0000 7ffa2a7fc640 1 -- 172.21.15.1:0/187564943 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7ffa24079fb0 con 0x7ffa2c15e050 2024-05-09T06:30:22.479 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:22.479 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:30:22.480 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:30:22.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 msgr2=0x7ffa1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 0x7ffa1806e320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009fc0 tx=0x7ffa1c009550 comp rx=0 tx=0).stop 2024-05-09T06:30:22.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 msgr2=0x7ffa2c0ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ffa240090b0 tx=0x7ffa240046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 shutdown_connections 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffa1806be60 0x7ffa1806e320 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa2c15e050 0x7ffa2c0ce6d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa2c0ae2a0 0x7ffa2c0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 --2- 172.21.15.1:0/187564943 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa2c06e4f0 0x7ffa2c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 >> 172.21.15.1:0/187564943 conn(0x7ffa2c06d270 msgr2=0x7ffa2c072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 shutdown_connections 2024-05-09T06:30:22.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.499+0000 7ffa33b14640 1 -- 172.21.15.1:0/187564943 wait complete. 2024-05-09T06:30:22.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.614+0000 7fc93c9d5640 1 Processor -- start 2024-05-09T06:30:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.614+0000 7fc93c9d5640 1 -- start start 2024-05-09T06:30:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93c9d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc93806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93c9d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc9380cf180 0x7fc9380cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93c9d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9380ce0e0 con 0x7fc9380cdab0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93c9d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc9380ce3c0 con 0x7fc9380cf180 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93c9d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9380ce6a0 con 0x7fc93806e6d0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc936d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc9380cdab0 0x7fc93815e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59458/0 (socket says 172.21.15.1:59458) 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc936d9d640 1 -- 172.21.15.1:0/4144082822 learned_addr learned my addr 172.21.15.1:0/4144082822 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc93806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc92ffff640 1 --2- 172.21.15.1:0/4144082822 >> v2:172.21.15.6:3300/0 conn(0x7fc9380cf180 0x7fc9380cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248176192 0 0) 0x7fc9380ce0e0 con 0x7fc9380cdab0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.615+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc924003520 con 0x7fc9380cdab0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3141408267 0 0) 0x7fc924003520 con 0x7fc9380cdab0 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 msgr2=0x7fc93806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc93806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 >> v2:172.21.15.6:3300/0 conn(0x7fc9380cf180 msgr2=0x7fc9380cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 --2- 172.21.15.1:0/4144082822 >> v2:172.21.15.6:3300/0 conn(0x7fc9380cf180 0x7fc9380cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380ce980 con 0x7fc9380cdab0 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fc9300019f0 con 0x7fc9380cdab0 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc93806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc92ffff640 1 --2- 172.21.15.1:0/4144082822 >> v2:172.21.15.6:3300/0 conn(0x7fc9380cf180 0x7fc9380cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 >> v1:172.21.15.1:6789/0 conn(0x7fc9380cdab0 legacy=0x7fc93815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc924004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc924004d40 0x7fc9240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380ce3c0 con 0x7fc9240039b0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380ce6a0 con 0x7fc924004d40 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc92400b9c0 con 0x7fc93806e6d0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc92ffff640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc924004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc936d9d640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc924004d40 0x7fc9240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 -- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 msgr2=0x7fc924004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc93806e6d0 0x7fc924004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 -- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc924004d40 msgr2=0x7fc9240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc924004d40 0x7fc9240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.616+0000 7fc93659c640 1 -- 172.21.15.1:0/4144082822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc924003520 con 0x7fc9240039b0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.617+0000 7fc93659c640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc928009c90 tx=0x7fc924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2675741b3afdeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.617+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928036070 con 0x7fc9240039b0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.617+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928031400 con 0x7fc9240039b0 2024-05-09T06:30:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.617+0000 7fc935d9b640 1 -- 172.21.15.1:0/4144082822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928009860 con 0x7fc9240039b0 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 -- 172.21.15.1:0/4144082822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9240039b0 msgr2=0x7fc924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/4144082822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc928009c90 tx=0x7fc924005830 comp rx=0 tx=0).stop 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc936d9d640 1 -- 172.21.15.1:0/4144082822 reap_dead start 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 -- 172.21.15.1:0/4144082822 shutdown_connections 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 -- 172.21.15.1:0/4144082822 >> 172.21.15.1:0/4144082822 conn(0x7fc9380cc4b0 msgr2=0x7fc9380cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 -- 172.21.15.1:0/4144082822 shutdown_connections 2024-05-09T06:30:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.618+0000 7fc93c9d5640 1 -- 172.21.15.1:0/4144082822 wait complete. 2024-05-09T06:30:22.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.619+0000 7fc93c9d5640 1 Processor -- start 2024-05-09T06:30:22.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.619+0000 7fc93c9d5640 1 -- start start 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9240039b0 0x7fc938155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 0x7fc93815e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381447c0 con 0x7fc93806d5c0 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc938070ed0 con 0x7fc93815bf00 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93c9d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc938144940 con 0x7fc9240039b0 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42306/0 (socket says 172.21.15.1:42306) 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 -- 172.21.15.1:0/3053782526 learned_addr learned my addr 172.21.15.1:0/3053782526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc92ffff640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 0x7fc93815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc93659c640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9240039b0 0x7fc938155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 -- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9240039b0 msgr2=0x7fc938155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9240039b0 0x7fc938155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 -- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 msgr2=0x7fc93815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 0x7fc93815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 -- 172.21.15.1:0/3053782526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93815e890 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc92ffff640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 0x7fc93815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.620+0000 7fc936d9d640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc930002230 tx=0x7fc93000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.621+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93000ee20 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.621+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc930018070 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.621+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930013660 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.621+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93815eb10 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.621+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93815f0d0 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.622+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93815f2d0 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.622+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc930013800 con 0x7fc93806d5c0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.623+0000 7fc92effd640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 0x7fc91806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.623+0000 7fc93659c640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 0x7fc91806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.623+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fc93008b4a0 con 0x7fc93806d5c0 2024-05-09T06:30:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.623+0000 7fc93659c640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 0x7fc91806e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc928033000 tx=0x7fc928008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.627+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc93815e890 con 0x7fc93806d5c0 2024-05-09T06:30:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.847+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc93806df80 con 0x7fc93806d5c0 2024-05-09T06:30:22.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.847+0000 7fc92effd640 1 -- 172.21.15.1:0/3053782526 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5166 (secure 0 0 0) 0x7fc930059c00 con 0x7fc93806d5c0 2024-05-09T06:30:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:22.830 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:29:33.204942+0000","modified":"2024-05-09T06:29:36.218023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6045":{"gid":6045,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6836/3096552929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3096552929},{"type":"v1","addr":"172.21.15.1:6836","nonce":3096552929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6045,"qdb_cluster":[6045]},"id":3}]} 2024-05-09T06:30:22.830 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-05-09T06:30:22.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 msgr2=0x7fc91806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 0x7fc91806e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc928033000 tx=0x7fc928008030 comp rx=0 tx=0).stop 2024-05-09T06:30:22.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 msgr2=0x7fc93806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc930002230 tx=0x7fc93000e530 comp rx=0 tx=0).stop 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 shutdown_connections 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91806be30 0x7fc91806e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93806d5c0 0x7fc93806da40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc93815bf00 0x7fc93815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 --2- 172.21.15.1:0/3053782526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9240039b0 0x7fc938155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.849+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 >> 172.21.15.1:0/3053782526 conn(0x7fc9380cc4b0 msgr2=0x7fc9381439c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.850+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 shutdown_connections 2024-05-09T06:30:22.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.850+0000 7fc93c9d5640 1 -- 172.21.15.1:0/3053782526 wait complete. 2024-05-09T06:30:22.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.964+0000 7f87aa3f9640 1 Processor -- start 2024-05-09T06:30:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.964+0000 7f87aa3f9640 1 -- start start 2024-05-09T06:30:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87aa3f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f87a41454b0 0x7f87a40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87aa3f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f87a40b2a60 0x7f87a40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87aa3f9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a4071c20 con 0x7f87a40b4da0 2024-05-09T06:30:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87aa3f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f87a406eb40 con 0x7f87a40b2a60 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87aa3f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f87a406ee20 con 0x7f87a41454b0 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f87a40b2a60 0x7f87a40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f87a40b2a60 0x7f87a40b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42308/0 (socket says 172.21.15.1:42308) 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 -- 172.21.15.1:0/3264875332 learned_addr learned my addr 172.21.15.1:0/3264875332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a3fff640 1 --2- 172.21.15.1:0/3264875332 >> v2:172.21.15.6:3300/0 conn(0x7f87a41454b0 0x7f87a40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 -- 172.21.15.1:0/3264875332 >> v2:172.21.15.6:3300/0 conn(0x7f87a41454b0 msgr2=0x7f87a40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 --2- 172.21.15.1:0/3264875332 >> v2:172.21.15.6:3300/0 conn(0x7f87a41454b0 0x7f87a40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 -- 172.21.15.1:0/3264875332 >> v1:172.21.15.6:6789/0 conn(0x7f87a40b4da0 legacy=0x7f87a415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.965+0000 7f87a37fe640 1 -- 172.21.15.1:0/3264875332 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a406d8e0 con 0x7f87a40b2a60 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.966+0000 7f87a37fe640 1 --2- 172.21.15.1:0/3264875332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a40b4860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f87a40689d0 tx=0x7f879802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2c857fc33298b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.966+0000 7f87a2ffd640 1 -- 172.21.15.1:0/3264875332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798037070 con 0x7f87a40b2a60 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.966+0000 7f87a2ffd640 1 -- 172.21.15.1:0/3264875332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8798032440 con 0x7f87a40b2a60 2024-05-09T06:30:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.966+0000 7f87a2ffd640 1 -- 172.21.15.1:0/3264875332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798032810 con 0x7f87a40b2a60 2024-05-09T06:30:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 -- 172.21.15.1:0/3264875332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 msgr2=0x7f87a40b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/3264875332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a40b4860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f87a40689d0 tx=0x7f879802a230 comp rx=0 tx=0).stop 2024-05-09T06:30:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 -- 172.21.15.1:0/3264875332 shutdown_connections 2024-05-09T06:30:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/3264875332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a40b4860 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/3264875332 >> v2:172.21.15.6:3300/0 conn(0x7f87a41454b0 0x7f87a40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 -- 172.21.15.1:0/3264875332 >> 172.21.15.1:0/3264875332 conn(0x7f87a40cc4b0 msgr2=0x7f87a40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 -- 172.21.15.1:0/3264875332 shutdown_connections 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.967+0000 7f87aa3f9640 1 -- 172.21.15.1:0/3264875332 wait complete. 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.968+0000 7f87aa3f9640 1 Processor -- start 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.968+0000 7f87aa3f9640 1 -- start start 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 0x7f87a41f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 0x7f87a413e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4145890 con 0x7f87a40b2a60 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f87a40b51f0 con 0x7f87a40b4da0 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87aa3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f87a406e0e0 con 0x7f87a41454b0 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42314/0 (socket says 172.21.15.1:42314) 2024-05-09T06:30:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87a3fff640 1 -- 172.21.15.1:0/567664097 learned_addr learned my addr 172.21.15.1:0/567664097 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87a8996640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 0x7f87a413e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.969+0000 7f87a37fe640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 0x7f87a41f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 -- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 msgr2=0x7f87a413e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 0x7f87a413e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 -- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 msgr2=0x7f87a41f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 0x7f87a41f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 -- 172.21.15.1:0/567664097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a41f2ad0 con 0x7f87a40b2a60 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a8996640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 0x7f87a413e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a37fe640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 0x7f87a41f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a3fff640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f879400c7f0 tx=0x7f879400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794015070 con 0x7f87a40b2a60 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794004240 con 0x7f87a40b2a60 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794007620 con 0x7f87a40b2a60 2024-05-09T06:30:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a413d7e0 con 0x7f87a40b2a60 2024-05-09T06:30:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.970+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a413dda0 con 0x7f87a40b2a60 2024-05-09T06:30:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.971+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a40752b0 con 0x7f87a40b2a60 2024-05-09T06:30:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.971+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87940079d0 con 0x7f87a40b2a60 2024-05-09T06:30:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.972+0000 7f87a0ff9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 0x7f879006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.972+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7f879408b4c0 con 0x7f87a40b2a60 2024-05-09T06:30:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.972+0000 7f87a37fe640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 0x7f879006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.972+0000 7f87a37fe640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 0x7f879006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8798034000 tx=0x7f879802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:22.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:22.976+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8794059c10 con 0x7f87a40b2a60 2024-05-09T06:30:23.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.162+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87a4072fa0 con 0x7f87a40b2a60 2024-05-09T06:30:23.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.163+0000 7f87a0ff9640 1 -- 172.21.15.1:0/567664097 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16850 (secure 0 0 0) 0x7f8794059510 con 0x7f87a40b2a60 2024-05-09T06:30:23.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:23.145 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:03.192204+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:29:31.502757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T06:29:32.177695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:23.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 msgr2=0x7f879006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 0x7f879006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8798034000 tx=0x7f879802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:23.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 msgr2=0x7f87a41556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f879400c7f0 tx=0x7f879400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 shutdown_connections 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f879006be30 0x7f879006e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87a41454b0 0x7f87a413e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a40b4da0 0x7f87a41f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 --2- 172.21.15.1:0/567664097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f87a40b2a60 0x7f87a41556a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 >> 172.21.15.1:0/567664097 conn(0x7f87a40cc4b0 msgr2=0x7f87a40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 shutdown_connections 2024-05-09T06:30:23.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.165+0000 7f87aa3f9640 1 -- 172.21.15.1:0/567664097 wait complete. 2024-05-09T06:30:23.159 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:30:23.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.283+0000 7fc8d4e76640 1 Processor -- start 2024-05-09T06:30:23.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.283+0000 7fc8d4e76640 1 -- start start 2024-05-09T06:30:23.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8d4e76640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8d00b2a60 0x7fc8d00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8d4e76640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc8d00b4f70 0x7fc8d0070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8d4e76640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d006eb40 con 0x7fc8d01454b0 2024-05-09T06:30:23.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8d4e76640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d006ee20 con 0x7fc8d00b2a60 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8d4e76640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d006d8e0 con 0x7fc8d00b4f70 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8d00b2a60 0x7fc8d00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8d00b2a60 0x7fc8d00b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42326/0 (socket says 172.21.15.1:42326) 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 -- 172.21.15.1:0/1233673358 learned_addr learned my addr 172.21.15.1:0/1233673358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 -- 172.21.15.1:0/1233673358 >> v2:172.21.15.6:3300/0 conn(0x7fc8d00b4f70 msgr2=0x7fc8d0070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 --2- 172.21.15.1:0/1233673358 >> v2:172.21.15.6:3300/0 conn(0x7fc8d00b4f70 0x7fc8d0070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 -- 172.21.15.1:0/1233673358 >> v1:172.21.15.6:6789/0 conn(0x7fc8d01454b0 legacy=0x7fc8d00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:23.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.284+0000 7fc8cdd9b640 1 -- 172.21.15.1:0/1233673358 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d006dbc0 con 0x7fc8d00b2a60 2024-05-09T06:30:23.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.285+0000 7fc8cdd9b640 1 --2- 172.21.15.1:0/1233673358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00b4a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc8d006c2a0 tx=0x7fc8b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea56c7322507ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.285+0000 7fc8cd59a640 1 -- 172.21.15.1:0/1233673358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8037070 con 0x7fc8d00b2a60 2024-05-09T06:30:23.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.285+0000 7fc8cd59a640 1 -- 172.21.15.1:0/1233673358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8032440 con 0x7fc8d00b2a60 2024-05-09T06:30:23.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.285+0000 7fc8cd59a640 1 -- 172.21.15.1:0/1233673358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8032810 con 0x7fc8d00b2a60 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 -- 172.21.15.1:0/1233673358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 msgr2=0x7fc8d00b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/1233673358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00b4a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc8d006c2a0 tx=0x7fc8b802a230 comp rx=0 tx=0).stop 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 -- 172.21.15.1:0/1233673358 shutdown_connections 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/1233673358 >> v2:172.21.15.6:3300/0 conn(0x7fc8d00b4f70 0x7fc8d0070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/1233673358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00b4a30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 -- 172.21.15.1:0/1233673358 >> 172.21.15.1:0/1233673358 conn(0x7fc8d00cc4b0 msgr2=0x7fc8d00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 -- 172.21.15.1:0/1233673358 shutdown_connections 2024-05-09T06:30:23.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.286+0000 7fc8d4e76640 1 -- 172.21.15.1:0/1233673358 wait complete. 2024-05-09T06:30:23.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.287+0000 7fc8d4e76640 1 Processor -- start 2024-05-09T06:30:23.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.287+0000 7fc8d4e76640 1 -- start start 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8d00b4f70 0x7fc8d01f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 0x7fc8d01f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d006e260 con 0x7fc8d00b2a60 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d006e0e0 con 0x7fc8d00b4f70 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8d4e76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d0071150 con 0x7fc8d01454b0 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42332/0 (socket says 172.21.15.1:42332) 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8ce59c640 1 -- 172.21.15.1:0/3958585520 learned_addr learned my addr 172.21.15.1:0/3958585520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.288+0000 7fc8cdd9b640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8d00b4f70 0x7fc8d01f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ced9d640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 0x7fc8d01f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 -- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 msgr2=0x7fc8d01f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 0x7fc8d01f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 -- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8d00b4f70 msgr2=0x7fc8d01f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8d00b4f70 0x7fc8d01f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 -- 172.21.15.1:0/3958585520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d00cdfc0 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ced9d640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 0x7fc8d01f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8ce59c640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800b3e0 tx=0x7fc8c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8015020 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c8011850 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d00ce2a0 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011c40 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.289+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d00ce860 con 0x7fc8d00b2a60 2024-05-09T06:30:23.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.290+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d00752b0 con 0x7fc8d00b2a60 2024-05-09T06:30:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.290+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8c8018020 con 0x7fc8d00b2a60 2024-05-09T06:30:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.291+0000 7fc8c6ffd640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 0x7fc8b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.291+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6159+0+0 (secure 0 0 0) 0x7fc8c808b040 con 0x7fc8d00b2a60 2024-05-09T06:30:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.291+0000 7fc8cdd9b640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 0x7fc8b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.291+0000 7fc8cdd9b640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 0x7fc8b006e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8034000 tx=0x7fc8b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.295+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8c801f020 con 0x7fc8d00b2a60 2024-05-09T06:30:23.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.517+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8d00aee50 con 0x7fc8d00b2a60 2024-05-09T06:30:23.499 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:30:23.518+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:30:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.519+0000 7fc8c6ffd640 1 -- 172.21.15.1:0/3958585520 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fc8c80172c0 con 0x7fc8d00b2a60 2024-05-09T06:30:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 msgr2=0x7fc8b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 0x7fc8b006e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8034000 tx=0x7fc8b802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 msgr2=0x7fc8d00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800b3e0 tx=0x7fc8c800b8a0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 shutdown_connections 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc8b006be30 0x7fc8b006e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8d01454b0 0x7fc8d01f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8d00b4f70 0x7fc8d01f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 --2- 172.21.15.1:0/3958585520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d00b2a60 0x7fc8d00cd570 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 >> 172.21.15.1:0/3958585520 conn(0x7fc8d00cc4b0 msgr2=0x7fc8d0071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 shutdown_connections 2024-05-09T06:30:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.522+0000 7fc8d4e76640 1 -- 172.21.15.1:0/3958585520 wait complete. 2024-05-09T06:30:23.506 INFO:tasks.ceph.mds.c.smithi001.stderr: -4924> 2024-05-09T06:29:39.811+0000 7f1f54836640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:30:23.508 INFO:tasks.ceph.mds.c.smithi001.stderr: -3181> 2024-05-09T06:29:46.670+0000 7f1f5082e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6240=0-4194304@1} caps={6240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6240 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ed483c2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:30:23.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:30:23.537 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:30:23.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.641+0000 7fe6cc3a5640 1 Processor -- start 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.641+0000 7fe6cc3a5640 1 -- start start 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6cc3a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c40718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6cc3a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6c415c4a0 0x7fe6c4071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6cc3a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6c415b420 con 0x7fe6c406e6d0 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6cc3a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c4158e30 con 0x7fe6c415c4a0 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6cc3a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c4072480 con 0x7fe6c4158a50 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6c415c4a0 0x7fe6c4071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6c415c4a0 0x7fe6c4071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42352/0 (socket says 172.21.15.1:42352) 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 -- 172.21.15.1:0/3918966255 learned_addr learned my addr 172.21.15.1:0/3918966255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6ca942640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe6c406e6d0 0x7fe6c40cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59470/0 (socket says 172.21.15.1:59470) 2024-05-09T06:30:23.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6ca141640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c40718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 -- 172.21.15.1:0/3918966255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 msgr2=0x7fe6c40718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c40718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 -- 172.21.15.1:0/3918966255 >> v1:172.21.15.1:6789/0 conn(0x7fe6c406e6d0 legacy=0x7fe6c40cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c9940640 1 -- 172.21.15.1:0/3918966255 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c415b6e0 con 0x7fe6c415c4a0 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6c913f640 1 -- 172.21.15.1:0/3918966255 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919824121 0 0) 0x7fe6c415b420 con 0x7fe6c406e6d0 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.642+0000 7fe6ca141640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c40718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.643+0000 7fe6c9940640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4071e00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009870 tx=0x7fe6b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f9a69b126f33bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.643+0000 7fe6c913f640 1 -- 172.21.15.1:0/3918966255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4037070 con 0x7fe6c415c4a0 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.643+0000 7fe6c913f640 1 -- 172.21.15.1:0/3918966255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b4032440 con 0x7fe6c415c4a0 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.643+0000 7fe6c913f640 1 -- 172.21.15.1:0/3918966255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4032810 con 0x7fe6c415c4a0 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/3918966255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c415c4a0 msgr2=0x7fe6c4071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4071e00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009870 tx=0x7fe6b402a270 comp rx=0 tx=0).stop 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/3918966255 shutdown_connections 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4071e00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/3918966255 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c40718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/3918966255 >> 172.21.15.1:0/3918966255 conn(0x7fe6c40cc4d0 msgr2=0x7fe6c40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/3918966255 shutdown_connections 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.644+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/3918966255 wait complete. 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.645+0000 7fe6cc3a5640 1 Processor -- start 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.645+0000 7fe6cc3a5640 1 -- start start 2024-05-09T06:30:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4168200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c415bf80 con 0x7fe6c406e6d0 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c415bca0 con 0x7fe6c415c4a0 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6cc3a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c4070ea0 con 0x7fe6c4158a50 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42358/0 (socket says 172.21.15.1:42358) 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 -- 172.21.15.1:0/959190578 learned_addr learned my addr 172.21.15.1:0/959190578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 -- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 msgr2=0x7fe6c4150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 -- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6c415c4a0 msgr2=0x7fe6c4168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 -- 172.21.15.1:0/959190578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c4167160 con 0x7fe6c406e6d0 2024-05-09T06:30:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.646+0000 7fe6ca141640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e810 tx=0x7fe6c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.647+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000f040 con 0x7fe6c406e6d0 2024-05-09T06:30:23.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.647+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0004030 con 0x7fe6c406e6d0 2024-05-09T06:30:23.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.647+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c4167440 con 0x7fe6c406e6d0 2024-05-09T06:30:23.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.647+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0004400 con 0x7fe6c406e6d0 2024-05-09T06:30:23.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.647+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c4167a00 con 0x7fe6c406e6d0 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.648+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c40752b0 con 0x7fe6c406e6d0 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.648+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6c000c6d0 con 0x7fe6c406e6d0 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.648+0000 7fe6baffd640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 0x7fe6a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.649+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6245+0+0 (secure 0 0 0) 0x7fe6c008ade0 con 0x7fe6c406e6d0 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.649+0000 7fe6c9940640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 0x7fe6a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.649+0000 7fe6c9940640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 0x7fe6a806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4034000 tx=0x7fe6b402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.653+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6c0017020 con 0x7fe6c406e6d0 2024-05-09T06:30:23.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:23.874+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe6c4072b70 con 0x7fe6c406e6d0 2024-05-09T06:30:24.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.520+0000 7fe6baffd640 1 -- 172.21.15.1:0/959190578 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fe6c0059600 con 0x7fe6c406e6d0 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 msgr2=0x7fe6a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 0x7fe6a806e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4034000 tx=0x7fe6b402c000 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 msgr2=0x7fe6c4166b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e810 tx=0x7fe6c000ecd0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 shutdown_connections 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6a806be30 0x7fe6a806e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6c415c4a0 0x7fe6c4168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6c4158a50 0x7fe6c4150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 --2- 172.21.15.1:0/959190578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6c406e6d0 0x7fe6c4166b80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 >> 172.21.15.1:0/959190578 conn(0x7fe6c40cc4d0 msgr2=0x7fe6c415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:24.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 shutdown_connections 2024-05-09T06:30:24.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.523+0000 7fe6cc3a5640 1 -- 172.21.15.1:0/959190578 wait complete. 2024-05-09T06:30:24.519 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:30:24.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 Processor -- start 2024-05-09T06:30:24.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 -- start start 2024-05-09T06:30:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54f00b2ed0 0x7f54f015e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54f00d2a10 0x7f54f00b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.696+0000 7f54f6e6b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f54f015e870 con 0x7f54f00b2ed0 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54f6e6b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f54f00d1cc0 con 0x7f54f00d2a10 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54f6e6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54f00d4d10 con 0x7f54f00d2630 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54f00b2ed0 0x7f54f015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54f00b2ed0 0x7f54f015e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42368/0 (socket says 172.21.15.1:42368) 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 -- 172.21.15.1:0/2924445581 learned_addr learned my addr 172.21.15.1:0/2924445581 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54f5408640 1 --2- 172.21.15.1:0/2924445581 >> v2:172.21.15.6:3300/0 conn(0x7f54f00d2a10 0x7f54f00b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54f4c07640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 -- 172.21.15.1:0/2924445581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 msgr2=0x7f54f015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 -- 172.21.15.1:0/2924445581 >> v2:172.21.15.6:3300/0 conn(0x7f54f00d2a10 msgr2=0x7f54f00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 --2- 172.21.15.1:0/2924445581 >> v2:172.21.15.6:3300/0 conn(0x7f54f00d2a10 0x7f54f00b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54effff640 1 -- 172.21.15.1:0/2924445581 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f00d1f80 con 0x7f54f00b2ed0 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.697+0000 7f54f4c07640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54effff640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f015e330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f54f008c6b0 tx=0x7f54dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed3db47b277e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54ef7fe640 1 -- 172.21.15.1:0/2924445581 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc037070 con 0x7f54f00b2ed0 2024-05-09T06:30:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54ef7fe640 1 -- 172.21.15.1:0/2924445581 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc032440 con 0x7f54f00b2ed0 2024-05-09T06:30:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54ef7fe640 1 -- 172.21.15.1:0/2924445581 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc032810 con 0x7f54f00b2ed0 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54f6e6b640 1 -- 172.21.15.1:0/2924445581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 msgr2=0x7f54f015e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.698+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f015e330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f54f008c6b0 tx=0x7f54dc02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 -- 172.21.15.1:0/2924445581 shutdown_connections 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/2924445581 >> v2:172.21.15.6:3300/0 conn(0x7f54f00d2a10 0x7f54f00b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f015e330 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/2924445581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 -- 172.21.15.1:0/2924445581 >> 172.21.15.1:0/2924445581 conn(0x7f54f00d3ee0 msgr2=0x7f54f00d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 -- 172.21.15.1:0/2924445581 shutdown_connections 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.699+0000 7f54f6e6b640 1 -- 172.21.15.1:0/2924445581 wait complete. 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.700+0000 7f54f6e6b640 1 Processor -- start 2024-05-09T06:30:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.700+0000 7f54f6e6b640 1 -- start start 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54f00d2a10 0x7f54f006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00b1640 con 0x7f54f00b2ed0 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00d4a00 con 0x7f54f00d2a10 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f6e6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54f00b17c0 con 0x7f54f00d2630 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42370/0 (socket says 172.21.15.1:42370) 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 -- 172.21.15.1:0/4213653908 learned_addr learned my addr 172.21.15.1:0/4213653908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f5408640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54f00d2a10 0x7f54f006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 -- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 msgr2=0x7f54f015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 -- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54f00d2a10 msgr2=0x7f54f006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54f00d2a10 0x7f54f006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:24.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 -- 172.21.15.1:0/4213653908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01f4a50 con 0x7f54f00b2ed0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.701+0000 7f54f4c07640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c7c0 tx=0x7f54e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.702+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4015070 con 0x7f54f00b2ed0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.702+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4007a30 con 0x7f54f00b2ed0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.702+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4007e00 con 0x7f54f00b2ed0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.702+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f01f4770 con 0x7f54f00b2ed0 2024-05-09T06:30:24.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.702+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f009dfd0 con 0x7f54f00b2ed0 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.703+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f0075320 con 0x7f54f00b2ed0 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.703+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54e4018070 con 0x7f54f00b2ed0 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.704+0000 7f54ed7fa640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 0x7f54d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.704+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6245+0+0 (secure 0 0 0) 0x7f54e408aff0 con 0x7f54f00b2ed0 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.704+0000 7f54effff640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 0x7f54d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:24.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.704+0000 7f54effff640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 0x7f54d406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f54dc034000 tx=0x7f54dc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:24.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.708+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54e4059780 con 0x7f54f00b2ed0 2024-05-09T06:30:24.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:24.893+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f54f009e3d0 con 0x7f54f00b2ed0 2024-05-09T06:30:25.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.526+0000 7f54ed7fa640 1 -- 172.21.15.1:0/4213653908 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) ==== 181+0+0 (secure 0 0 0) 0x7f54e4059080 con 0x7f54f00b2ed0 2024-05-09T06:30:25.508 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:30:25.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 msgr2=0x7f54d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 0x7f54d406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f54dc034000 tx=0x7f54dc02c000 comp rx=0 tx=0).stop 2024-05-09T06:30:25.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 msgr2=0x7f54f009e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c7c0 tx=0x7f54e400cc80 comp rx=0 tx=0).stop 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 shutdown_connections 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54d406be30 0x7f54d406e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54f00d2a10 0x7f54f006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54f00d2630 0x7f54f015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 --2- 172.21.15.1:0/4213653908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f00b2ed0 0x7f54f009e7b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.530+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 >> 172.21.15.1:0/4213653908 conn(0x7f54f00d3ee0 msgr2=0x7f54f015ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.531+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 shutdown_connections 2024-05-09T06:30:25.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.531+0000 7f54f6e6b640 1 -- 172.21.15.1:0/4213653908 wait complete. 2024-05-09T06:30:25.527 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:30:25.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.659+0000 7f251a0e1640 1 Processor -- start 2024-05-09T06:30:25.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.659+0000 7f251a0e1640 1 -- start start 2024-05-09T06:30:25.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f251a0e1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f25140b3820 0x7f25140ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f251a0e1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25140b3cc0 con 0x7f25140ced00 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f251a0e1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25140b3fa0 con 0x7f2514145880 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f251a0e1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f25140b4280 con 0x7f25140b3820 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f2513fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f25140ced00 0x7f2514070d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59482/0 (socket says 172.21.15.1:59482) 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f2513fff640 1 -- 172.21.15.1:0/512707438 learned_addr learned my addr 172.21.15.1:0/512707438 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.660+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> v2:172.21.15.6:3300/0 conn(0x7f25140b3820 0x7f25140ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4115047531 0 0) 0x7f25140b3cc0 con 0x7f25140ced00 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24fc003520 con 0x7f25140ced00 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1846749124 0 0) 0x7f24fc003520 con 0x7f25140ced00 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 >> v2:172.21.15.6:3300/0 conn(0x7f25140b3820 msgr2=0x7f25140ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:25.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 --2- 172.21.15.1:0/512707438 >> v2:172.21.15.6:3300/0 conn(0x7f25140b3820 0x7f25140ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 >> v1:172.21.15.6:6789/0 conn(0x7f2514145880 legacy=0x7f251415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25140b4560 con 0x7f25140ced00 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f250c0019f0 con 0x7f25140ced00 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> v2:172.21.15.6:3300/0 conn(0x7f25140b3820 0x7f25140ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 >> v1:172.21.15.1:6789/0 conn(0x7f25140ced00 legacy=0x7f2514070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24fc0039b0 0x7f24fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f24fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24fc0051c0 0x7f24fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25140b4280 con 0x7f24fc004800 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc00c060 con 0x7f24fc0039b0 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc00c260 con 0x7f24fc0051c0 2024-05-09T06:30:25.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f24fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f25137fe640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24fc0039b0 0x7f24fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2513fff640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24fc0051c0 0x7f24fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 -- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24fc0051c0 msgr2=0x7f24fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24fc0051c0 0x7f24fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 -- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24fc0039b0 msgr2=0x7f24fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24fc0039b0 0x7f24fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2512ffd640 1 -- 172.21.15.1:0/512707438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc003520 con 0x7f24fc004800 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.661+0000 7f2513fff640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24fc0051c0 0x7f24fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.662+0000 7f2513fff640 1 -- 172.21.15.1:0/512707438 reap_dead start 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.662+0000 7f2512ffd640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f24fc004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2508009c90 tx=0x7f24fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb39546454da19e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.662+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508036070 con 0x7f24fc004800 2024-05-09T06:30:25.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.662+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508031400 con 0x7f24fc004800 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.662+0000 7f25127fc640 1 -- 172.21.15.1:0/512707438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508009860 con 0x7f24fc004800 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 -- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 msgr2=0x7f24fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f24fc004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2508009c90 tx=0x7f24fc005830 comp rx=0 tx=0).stop 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 -- 172.21.15.1:0/512707438 shutdown_connections 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 --2- 172.21.15.1:0/512707438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f24fc004c80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 -- 172.21.15.1:0/512707438 >> 172.21.15.1:0/512707438 conn(0x7f251406b3c0 msgr2=0x7f251406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 -- 172.21.15.1:0/512707438 shutdown_connections 2024-05-09T06:30:25.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.663+0000 7f251a0e1640 1 -- 172.21.15.1:0/512707438 wait complete. 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.664+0000 7f251a0e1640 1 Processor -- start 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.664+0000 7f251a0e1640 1 -- start start 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2514137380 0x7f2514142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2514144280 0x7f2514144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25140ce7c0 con 0x7f24fc004800 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25140cfee0 con 0x7f2514144280 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f251a0e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f25140ce940 con 0x7f2514137380 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f25137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f25137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42388/0 (socket says 172.21.15.1:42388) 2024-05-09T06:30:25.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.665+0000 7f25137fe640 1 -- 172.21.15.1:0/651343631 learned_addr learned my addr 172.21.15.1:0/651343631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 -- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2514137380 msgr2=0x7f2514142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2514137380 0x7f2514142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 -- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2514144280 msgr2=0x7f2514144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f2513fff640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2514144280 0x7f2514144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2514144280 0x7f2514144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 -- 172.21.15.1:0/651343631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141377e0 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f25137fe640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f250400f820 tx=0x7f250400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25040050d0 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2504018070 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141431e0 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504005490 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.666+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141437a0 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.667+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2514072ec0 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.667+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f250400d730 con 0x7f24fc004800 2024-05-09T06:30:25.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.668+0000 7f24fbfff640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 0x7f250006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:25.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.668+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5807+0+0 (secure 0 0 0) 0x7f250408ad50 con 0x7f24fc004800 2024-05-09T06:30:25.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.668+0000 7f2512ffd640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 0x7f250006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:25.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.668+0000 7f2512ffd640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 0x7f250006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2508033000 tx=0x7f250802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:25.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.672+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2514072ec0 con 0x7f24fc004800 2024-05-09T06:30:25.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:25.856+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25140b2bc0 con 0x7f24fc004800 2024-05-09T06:30:26.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.528+0000 7f24fbfff640 1 -- 172.21.15.1:0/651343631 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) ==== 169+0+0 (secure 0 0 0) 0x7f25140b2bc0 con 0x7f24fc004800 2024-05-09T06:30:26.510 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:30:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.532+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 msgr2=0x7f250006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.532+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 0x7f250006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2508033000 tx=0x7f250802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.532+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 msgr2=0x7f251406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.532+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f250400f820 tx=0x7f250400fce0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 shutdown_connections 2024-05-09T06:30:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f250006be30 0x7f250006e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2514144280 0x7f2514144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2514137380 0x7f2514142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 --2- 172.21.15.1:0/651343631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24fc004800 0x7f251406e8d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.533+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 >> 172.21.15.1:0/651343631 conn(0x7f251406b3c0 msgr2=0x7f251413fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.534+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 shutdown_connections 2024-05-09T06:30:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.536+0000 7f251a0e1640 1 -- 172.21.15.1:0/651343631 wait complete. 2024-05-09T06:30:26.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:30:26.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.665+0000 7f3ee337e640 1 Processor -- start 2024-05-09T06:30:26.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.665+0000 7f3ee337e640 1 -- start start 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee337e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3edc0b11f0 0x7f3edc0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee337e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 0x7f3edc0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee337e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc15e900 con 0x7f3edc15e390 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee337e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc15ebe0 con 0x7f3edc0b11f0 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee337e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc15eec0 con 0x7f3edc0cc8e0 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3edc0b11f0 0x7f3edc0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3edc0b11f0 0x7f3edc0b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42398/0 (socket says 172.21.15.1:42398) 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 -- 172.21.15.1:0/2749607128 learned_addr learned my addr 172.21.15.1:0/2749607128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee191b640 1 --2- 172.21.15.1:0/2749607128 >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 0x7f3edc0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 -- 172.21.15.1:0/2749607128 >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 msgr2=0x7f3edc0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 --2- 172.21.15.1:0/2749607128 >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 0x7f3edc0ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 -- 172.21.15.1:0/2749607128 >> v1:172.21.15.6:6789/0 conn(0x7f3edc15e390 legacy=0x7f3edc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee0919640 1 -- 172.21.15.1:0/2749607128 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc15f1a0 con 0x7f3edc0b11f0 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.666+0000 7f3ee191b640 1 --2- 172.21.15.1:0/2749607128 >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 0x7f3edc0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.667+0000 7f3ee0919640 1 --2- 172.21.15.1:0/2749607128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc0b11f0 0x7f3edc0b1680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0721e0 tx=0x7f3ed002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c82d91892eca641 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.667+0000 7f3ecffff640 1 -- 172.21.15.1:0/2749607128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0037070 con 0x7f3edc0b11f0 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.667+0000 7f3ecffff640 1 -- 172.21.15.1:0/2749607128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed0032440 con 0x7f3edc0b11f0 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.667+0000 7f3ecffff640 1 -- 172.21.15.1:0/2749607128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0032810 con 0x7f3edc0b11f0 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 -- 172.21.15.1:0/2749607128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc0b11f0 msgr2=0x7f3edc0b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 --2- 172.21.15.1:0/2749607128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc0b11f0 0x7f3edc0b1680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0721e0 tx=0x7f3ed002a270 comp rx=0 tx=0).stop 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 -- 172.21.15.1:0/2749607128 shutdown_connections 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 --2- 172.21.15.1:0/2749607128 >> v2:172.21.15.6:3300/0 conn(0x7f3edc0cc8e0 0x7f3edc0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 --2- 172.21.15.1:0/2749607128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc0b11f0 0x7f3edc0b1680 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 -- 172.21.15.1:0/2749607128 >> 172.21.15.1:0/2749607128 conn(0x7f3edc0b1da0 msgr2=0x7f3edc0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:26.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 -- 172.21.15.1:0/2749607128 shutdown_connections 2024-05-09T06:30:26.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.668+0000 7f3ee337e640 1 -- 172.21.15.1:0/2749607128 wait complete. 2024-05-09T06:30:26.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.669+0000 7f3ee337e640 1 Processor -- start 2024-05-09T06:30:26.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.669+0000 7f3ee337e640 1 -- start start 2024-05-09T06:30:26.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3edc0b11f0 0x7f3edc0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 0x7f3edc157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0cffb0 con 0x7f3edc15e390 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc162ba0 con 0x7f3edc0cc8e0 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0cfb60 con 0x7f3edc0b11f0 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42414/0 (socket says 172.21.15.1:42414) 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 -- 172.21.15.1:0/657445135 learned_addr learned my addr 172.21.15.1:0/657445135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:26.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee0919640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 0x7f3edc157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee111a640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3edc0b11f0 0x7f3edc0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 -- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3edc0b11f0 msgr2=0x7f3edc0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3edc0b11f0 0x7f3edc0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 -- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 msgr2=0x7f3edc157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 0x7f3edc157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 -- 172.21.15.1:0/657445135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0b3d00 con 0x7f3edc15e390 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee0919640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 0x7f3edc157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee191b640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007ef0 tx=0x7f3ed400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed401a070 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4003b90 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc1597b0 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.670+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4004030 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.671+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc159d70 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.672+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc0752b0 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.672+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ed401d070 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.672+0000 7f3ecdffb640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.673+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f3ed408ae80 con 0x7f3edc15e390 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.673+0000 7f3ee111a640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.673+0000 7f3ee111a640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 0x7f3ec006e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8007fd0 tx=0x7f3ed8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.677+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ed4010070 con 0x7f3edc15e390 2024-05-09T06:30:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.860+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3edc0b2ae0 con 0x7f3edc15e390 2024-05-09T06:30:26.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.863+0000 7f3ecdffb640 1 -- 172.21.15.1:0/657445135 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f3ed40593e0 con 0x7f3edc15e390 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 msgr2=0x7f3ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 0x7f3ec006e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8007fd0 tx=0x7f3ed8009040 comp rx=0 tx=0).stop 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 msgr2=0x7f3edc1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007ef0 tx=0x7f3ed400d4a0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 shutdown_connections 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3edc15e390 0x7f3edc1f4390 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3edc0cc8e0 0x7f3edc157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 --2- 172.21.15.1:0/657445135 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3edc0b11f0 0x7f3edc0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 >> 172.21.15.1:0/657445135 conn(0x7f3edc0b1da0 msgr2=0x7f3edc06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 shutdown_connections 2024-05-09T06:30:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.865+0000 7f3ee337e640 1 -- 172.21.15.1:0/657445135 wait complete. 2024-05-09T06:30:26.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-09T06:30:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.990+0000 7fdef4d7d640 1 Processor -- start 2024-05-09T06:30:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.990+0000 7fdef4d7d640 1 -- start start 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdef4d7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef006e6d0 0x7fdef006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdef4d7d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef00cf180 0x7fdef00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdef4d7d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef00ce0e0 con 0x7fdef00cdab0 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdef4d7d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdef00ce3c0 con 0x7fdef00cf180 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdef4d7d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdef00ce6a0 con 0x7fdef006e6d0 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef00cf180 0x7fdef00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef00cf180 0x7fdef00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42430/0 (socket says 172.21.15.1:42430) 2024-05-09T06:30:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 -- 172.21.15.1:0/1953352045 learned_addr learned my addr 172.21.15.1:0/1953352045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeee59c640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef006e6d0 0x7fdef006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 -- 172.21.15.1:0/1953352045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef006e6d0 msgr2=0x7fdef006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef006e6d0 0x7fdef006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 -- 172.21.15.1:0/1953352045 >> v1:172.21.15.6:6789/0 conn(0x7fdef00cdab0 legacy=0x7fdef015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.992+0000 7fdeedd9b640 1 -- 172.21.15.1:0/1953352045 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef00ce980 con 0x7fdef00cf180 2024-05-09T06:30:26.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.993+0000 7fdeedd9b640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef00cf180 0x7fdef00cd570 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009870 tx=0x7fdedc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e86b244604b63a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.993+0000 7fdeed59a640 1 -- 172.21.15.1:0/1953352045 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc037070 con 0x7fdef00cf180 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdeed59a640 1 -- 172.21.15.1:0/1953352045 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc032440 con 0x7fdef00cf180 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdeed59a640 1 -- 172.21.15.1:0/1953352045 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc032810 con 0x7fdef00cf180 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 -- 172.21.15.1:0/1953352045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef00cf180 msgr2=0x7fdef00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef00cf180 0x7fdef00cd570 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009870 tx=0x7fdedc02a270 comp rx=0 tx=0).stop 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 -- 172.21.15.1:0/1953352045 shutdown_connections 2024-05-09T06:30:26.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef00cf180 0x7fdef00cd570 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/1953352045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef006e6d0 0x7fdef006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 -- 172.21.15.1:0/1953352045 >> 172.21.15.1:0/1953352045 conn(0x7fdef00cc4b0 msgr2=0x7fdef00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 -- 172.21.15.1:0/1953352045 shutdown_connections 2024-05-09T06:30:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.994+0000 7fdef4d7d640 1 -- 172.21.15.1:0/1953352045 wait complete. 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 Processor -- start 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 -- start start 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef00cdab0 0x7fdef0159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef00cf180 0x7fdef0140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0070ed0 con 0x7fdef006e6d0 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdef006eff0 con 0x7fdef00cf180 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.996+0000 7fdef4d7d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdef01447c0 con 0x7fdef00cdab0 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42446/0 (socket says 172.21.15.1:42446) 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 -- 172.21.15.1:0/3745377984 learned_addr learned my addr 172.21.15.1:0/3745377984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeeed9d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef00cf180 0x7fdef0140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeedd9b640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef00cdab0 0x7fdef0159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 -- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef00cdab0 msgr2=0x7fdef0159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef00cdab0 0x7fdef0159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 -- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef00cf180 msgr2=0x7fdef0140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:26.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef00cf180 0x7fdef0140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 -- 172.21.15.1:0/3745377984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef013d140 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdeee59c640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e8a0 tx=0x7fdee000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0017070 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee000c940 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef013d420 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee000cd30 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.997+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef013d9e0 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.998+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef00752b0 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:26.999+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdee001a070 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.000+0000 7fdee6ffd640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 0x7fded806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.000+0000 7fdeedd9b640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 0x7fded806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.000+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fdee008abb0 con 0x7fdef006e6d0 2024-05-09T06:30:26.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.000+0000 7fdeedd9b640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 0x7fded806e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034000 tx=0x7fdedc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:26.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.004+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdee00596d0 con 0x7fdef006e6d0 2024-05-09T06:30:27.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.257+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fdef0141b70 con 0x7fdef006e6d0 2024-05-09T06:30:27.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.534+0000 7fdee6ffd640 1 -- 172.21.15.1:0/3745377984 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v180) ==== 149+0+0 (secure 0 0 0) 0x7fdee0058fd0 con 0x7fdef006e6d0 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 msgr2=0x7fded806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 0x7fded806e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034000 tx=0x7fdedc02e000 comp rx=0 tx=0).stop 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 msgr2=0x7fdef015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e8a0 tx=0x7fdee000ed60 comp rx=0 tx=0).stop 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 shutdown_connections 2024-05-09T06:30:27.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fded806be30 0x7fded806e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdef00cf180 0x7fdef0140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdef00cdab0 0x7fdef0159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 --2- 172.21.15.1:0/3745377984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef006e6d0 0x7fdef015c080 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.537+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 >> 172.21.15.1:0/3745377984 conn(0x7fdef00cc4b0 msgr2=0x7fdef00b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.538+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 shutdown_connections 2024-05-09T06:30:27.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.538+0000 7fdef4d7d640 1 -- 172.21.15.1:0/3745377984 wait complete. 2024-05-09T06:30:27.531 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-09T06:30:27.531 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-09T06:30:27.531 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:30:27.532 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.656+0000 7fc97e8a5640 1 Processor -- start 2024-05-09T06:30:27.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.656+0000 7fc97e8a5640 1 -- start start 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97e8a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc97806e6d0 0x7fc97806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97e8a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97e8a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9780ce0e0 con 0x7fc9780cf180 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97e8a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc9780ce3c0 con 0x7fc97806e6d0 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97e8a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9780ce6a0 con 0x7fc9780b3300 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc97806e6d0 0x7fc97806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc9777fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc9780cf180 0x7fc9780cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59498/0 (socket says 172.21.15.1:59498) 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc97806e6d0 0x7fc97806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42448/0 (socket says 172.21.15.1:42448) 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc9777fe640 1 -- 172.21.15.1:0/4137858551 learned_addr learned my addr 172.21.15.1:0/4137858551 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc97ce42640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 -- 172.21.15.1:0/4137858551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 msgr2=0x7fc9780b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 -- 172.21.15.1:0/4137858551 >> v1:172.21.15.1:6789/0 conn(0x7fc9780cf180 legacy=0x7fc9780cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.657+0000 7fc977fff640 1 -- 172.21.15.1:0/4137858551 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9780ce980 con 0x7fc97806e6d0 2024-05-09T06:30:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.658+0000 7fc977fff640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc97806e6d0 0x7fc97806eab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc9680097a0 tx=0x7fc96802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a225e68dc3f1c0ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.658+0000 7fc976ffd640 1 -- 172.21.15.1:0/4137858551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968037070 con 0x7fc97806e6d0 2024-05-09T06:30:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.658+0000 7fc976ffd640 1 -- 172.21.15.1:0/4137858551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc968032440 con 0x7fc97806e6d0 2024-05-09T06:30:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.658+0000 7fc976ffd640 1 -- 172.21.15.1:0/4137858551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968032810 con 0x7fc97806e6d0 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 -- 172.21.15.1:0/4137858551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc97806e6d0 msgr2=0x7fc97806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc97806e6d0 0x7fc97806eab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc9680097a0 tx=0x7fc96802a180 comp rx=0 tx=0).stop 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 -- 172.21.15.1:0/4137858551 shutdown_connections 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/4137858551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc97806e6d0 0x7fc97806eab0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 -- 172.21.15.1:0/4137858551 >> 172.21.15.1:0/4137858551 conn(0x7fc9780cc4b0 msgr2=0x7fc9780cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 -- 172.21.15.1:0/4137858551 shutdown_connections 2024-05-09T06:30:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.659+0000 7fc97e8a5640 1 -- 172.21.15.1:0/4137858551 wait complete. 2024-05-09T06:30:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.660+0000 7fc97e8a5640 1 Processor -- start 2024-05-09T06:30:27.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.660+0000 7fc97e8a5640 1 -- start start 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 0x7fc97815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc978070ed0 con 0x7fc9780cf180 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc97806eff0 con 0x7fc97806e6d0 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97e8a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9781447c0 con 0x7fc9780b3300 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42458/0 (socket says 172.21.15.1:42458) 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 -- 172.21.15.1:0/1199508060 learned_addr learned my addr 172.21.15.1:0/1199508060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:27.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc9777fe640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc977fff640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 0x7fc97815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 -- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 msgr2=0x7fc9780ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 -- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 msgr2=0x7fc97815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 0x7fc97815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc97ce42640 1 -- 172.21.15.1:0/1199508060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97815d650 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.661+0000 7fc977fff640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 0x7fc97815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc97ce42640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc970007e40 tx=0x7fc97000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970014620 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970019070 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97815d930 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970014a20 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.662+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9780a98e0 con 0x7fc9780cf180 2024-05-09T06:30:27.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.663+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9780a9ae0 con 0x7fc9780cf180 2024-05-09T06:30:27.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.663+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc970014bc0 con 0x7fc9780cf180 2024-05-09T06:30:27.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.664+0000 7fc974ff9640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.664+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fc97008b6a0 con 0x7fc9780cf180 2024-05-09T06:30:27.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.664+0000 7fc977fff640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.664+0000 7fc977fff640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 0x7fc96406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc968034000 tx=0x7fc96802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.668+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc9780a9ae0 con 0x7fc9780cf180 2024-05-09T06:30:27.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.852+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9780752b0 con 0x7fc9780cf180 2024-05-09T06:30:27.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.853+0000 7fc974ff9640 1 -- 172.21.15.1:0/1199508060 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13159 (secure 0 0 0) 0x7fc97005a120 con 0x7fc9780cf180 2024-05-09T06:30:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:27.835 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":56,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:26.527698+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/3096552929":"2024-05-10T06:30:23.518878+0000","172.21.15.1:6834/3096552929":"2024-05-10T06:30:23.518878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 msgr2=0x7fc96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 0x7fc96406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc968034000 tx=0x7fc96802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 msgr2=0x7fc9780ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc970007e40 tx=0x7fc97000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 shutdown_connections 2024-05-09T06:30:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9780cf180 0x7fc9780ad370 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9780b3300 0x7fc9780ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 --2- 172.21.15.1:0/1199508060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc97806e6d0 0x7fc97815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 >> 172.21.15.1:0/1199508060 conn(0x7fc9780cc4b0 msgr2=0x7fc9780b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 shutdown_connections 2024-05-09T06:30:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.855+0000 7fc97e8a5640 1 -- 172.21.15.1:0/1199508060 wait complete. 2024-05-09T06:30:27.848 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:30:27.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:30:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.974+0000 7f8bcc015640 1 Processor -- start 2024-05-09T06:30:27.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.974+0000 7f8bcc015640 1 -- start start 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bcc015640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8bc4158a50 0x7f8bc40718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bcc015640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc415b420 con 0x7f8bc406ede0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bcc015640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc415b6e0 con 0x7f8bc415c4a0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bcc015640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc4158e30 con 0x7f8bc4158a50 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bca5b2640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8bc406ede0 0x7f8bc415ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38224/0 (socket says 172.21.15.1:38224) 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bca5b2640 1 -- 172.21.15.1:0/1320633904 learned_addr learned my addr 172.21.15.1:0/1320633904 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bc9db1640 1 --2- 172.21.15.1:0/1320633904 >> v2:172.21.15.6:3300/0 conn(0x7f8bc4158a50 0x7f8bc40718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727595926 0 0) 0x7f8bc415b420 con 0x7f8bc406ede0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.975+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb0003520 con 0x7f8bc406ede0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3889493532 0 0) 0x7f8bb0003520 con 0x7f8bc406ede0 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 >> v2:172.21.15.6:3300/0 conn(0x7f8bc4158a50 msgr2=0x7f8bc40718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 --2- 172.21.15.1:0/1320633904 >> v2:172.21.15.6:3300/0 conn(0x7f8bc4158a50 0x7f8bc40718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 >> v1:172.21.15.6:6789/0 conn(0x7f8bc415c4a0 legacy=0x7f8bc406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc415b9c0 con 0x7f8bc406ede0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8bbc001a10 con 0x7f8bc406ede0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc9db1640 1 --2- 172.21.15.1:0/1320633904 >> v2:172.21.15.6:3300/0 conn(0x7f8bc4158a50 0x7f8bc40718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 >> v1:172.21.15.1:6789/0 conn(0x7f8bc406ede0 legacy=0x7f8bc415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 0x7f8bb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4158e30 con 0x7f8bb00051c0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb000c060 con 0x7f8bb00039b0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb000c260 con 0x7f8bb0004800 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 0x7f8bb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc95b0640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bc9db1640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 -- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb0004800 msgr2=0x7f8bb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 -- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00039b0 msgr2=0x7f8bb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.976+0000 7f8bca5b2640 1 -- 172.21.15.1:0/1320633904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb0003520 con 0x7f8bb00051c0 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.977+0000 7f8bca5b2640 1 -- 172.21.15.1:0/1320633904 reap_dead start 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.977+0000 7f8bca5b2640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 0x7f8bb000b9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406c2c0 tx=0x7f8bb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55112f6ce23dda5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.977+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00a4d0 con 0x7f8bb00051c0 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.977+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc02f070 con 0x7f8bb00051c0 2024-05-09T06:30:27.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.977+0000 7f8bc8daf640 1 -- 172.21.15.1:0/1320633904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02bc70 con 0x7f8bb00051c0 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 -- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 msgr2=0x7f8bb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 0x7f8bb000b9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406c2c0 tx=0x7f8bb0005830 comp rx=0 tx=0).stop 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 -- 172.21.15.1:0/1320633904 shutdown_connections 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 --2- 172.21.15.1:0/1320633904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb00051c0 0x7f8bb000b9c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 -- 172.21.15.1:0/1320633904 >> 172.21.15.1:0/1320633904 conn(0x7f8bc40cc4d0 msgr2=0x7f8bc40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 -- 172.21.15.1:0/1320633904 shutdown_connections 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.978+0000 7f8bcc015640 1 -- 172.21.15.1:0/1320633904 wait complete. 2024-05-09T06:30:27.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.979+0000 7f8bcc015640 1 Processor -- start 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.979+0000 7f8bcc015640 1 -- start start 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00051c0 0x7f8bc40b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bc40b4d80 0x7f8bc40ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc415bf80 con 0x7f8bc41f4420 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4072480 con 0x7f8bb00051c0 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc4070ea0 con 0x7f8bc40b4d80 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50330/0 (socket says 172.21.15.1:50330) 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 -- 172.21.15.1:0/48090620 learned_addr learned my addr 172.21.15.1:0/48090620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 -- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bc40b4d80 msgr2=0x7f8bc40ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bc40b4d80 0x7f8bc40ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 -- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00051c0 msgr2=0x7f8bc40b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00051c0 0x7f8bc40b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 -- 172.21.15.1:0/48090620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc40ae6a0 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bc95b0640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400f620 tx=0x7f8bb400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40052e0 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb4018070 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc40b34b0 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.980+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4003810 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.981+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc40b3a70 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.981+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc40752b0 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.982+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bb40039b0 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.982+0000 7f8bba7fc640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 0x7f8ba806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.982+0000 7f8bc9db1640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 0x7f8ba806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.982+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f8bb408aa70 con 0x7f8bc41f4420 2024-05-09T06:30:27.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.983+0000 7f8bc9db1640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 0x7f8ba806e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009bd0 tx=0x7f8bc0006a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:27.987+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bb40595a0 con 0x7f8bc41f4420 2024-05-09T06:30:28.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.168+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bc40ce630 con 0x7f8bc41f4420 2024-05-09T06:30:28.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.169+0000 7f8bba7fc640 1 -- 172.21.15.1:0/48090620 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f8bb4058ea0 con 0x7f8bc41f4420 2024-05-09T06:30:28.150 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 msgr2=0x7f8ba806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 0x7f8ba806e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009bd0 tx=0x7f8bc0006a60 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 msgr2=0x7f8bc40b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400f620 tx=0x7f8bb400fae0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 shutdown_connections 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8ba806be60 0x7f8ba806e320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bc40b4d80 0x7f8bc40ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc41f4420 0x7f8bc40b4840 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 --2- 172.21.15.1:0/48090620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bb00051c0 0x7f8bc40b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 >> 172.21.15.1:0/48090620 conn(0x7f8bc40cc4d0 msgr2=0x7f8bc40cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 shutdown_connections 2024-05-09T06:30:28.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.171+0000 7f8bcc015640 1 -- 172.21.15.1:0/48090620 wait complete. 2024-05-09T06:30:28.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-09T06:30:28.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.286+0000 7f904c0a5640 1 Processor -- start 2024-05-09T06:30:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.286+0000 7f904c0a5640 1 -- start start 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904c0a5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90441454b0 0x7f90440b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904c0a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90440b2a60 0x7f90440b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904c0a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9044071c20 con 0x7f90440b4da0 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904c0a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f904406eb40 con 0x7f90440b2a60 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904c0a5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f904406ee20 con 0x7f90441454b0 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049640640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90440b2a60 0x7f90440b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904a642640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f90440b4da0 0x7f904415f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38236/0 (socket says 172.21.15.1:38236) 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f904a642640 1 -- 172.21.15.1:0/818738142 learned_addr learned my addr 172.21.15.1:0/818738142 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049e41640 1 --2- 172.21.15.1:0/818738142 >> v2:172.21.15.6:3300/0 conn(0x7f90441454b0 0x7f90440b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049640640 1 -- 172.21.15.1:0/818738142 >> v2:172.21.15.6:3300/0 conn(0x7f90441454b0 msgr2=0x7f90440b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049640640 1 --2- 172.21.15.1:0/818738142 >> v2:172.21.15.6:3300/0 conn(0x7f90441454b0 0x7f90440b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049640640 1 -- 172.21.15.1:0/818738142 >> v1:172.21.15.1:6789/0 conn(0x7f90440b4da0 legacy=0x7f904415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9048e3f640 1 -- 172.21.15.1:0/818738142 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918206292 0 0) 0x7f9044071c20 con 0x7f90440b4da0 2024-05-09T06:30:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.287+0000 7f9049640640 1 -- 172.21.15.1:0/818738142 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904406d8e0 con 0x7f90440b2a60 2024-05-09T06:30:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.288+0000 7f9049640640 1 --2- 172.21.15.1:0/818738142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f90440b4860 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f90440689d0 tx=0x7f903402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b474e8bffce84dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.288+0000 7f9048e3f640 1 -- 172.21.15.1:0/818738142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034037070 con 0x7f90440b2a60 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.288+0000 7f9048e3f640 1 -- 172.21.15.1:0/818738142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9034032440 con 0x7f90440b2a60 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.288+0000 7f9048e3f640 1 -- 172.21.15.1:0/818738142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034032810 con 0x7f90440b2a60 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 -- 172.21.15.1:0/818738142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 msgr2=0x7f90440b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 --2- 172.21.15.1:0/818738142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f90440b4860 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f90440689d0 tx=0x7f903402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 -- 172.21.15.1:0/818738142 shutdown_connections 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 --2- 172.21.15.1:0/818738142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f90440b4860 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 --2- 172.21.15.1:0/818738142 >> v2:172.21.15.6:3300/0 conn(0x7f90441454b0 0x7f90440b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 -- 172.21.15.1:0/818738142 >> 172.21.15.1:0/818738142 conn(0x7f90440cc4b0 msgr2=0x7f90440cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 -- 172.21.15.1:0/818738142 shutdown_connections 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.289+0000 7f904c0a5640 1 -- 172.21.15.1:0/818738142 wait complete. 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.290+0000 7f904c0a5640 1 Processor -- start 2024-05-09T06:30:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.290+0000 7f904c0a5640 1 -- start start 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90440b4da0 0x7f90440acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90441454b0 0x7f9044155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9044145890 con 0x7f90440b2a60 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f90440b51f0 con 0x7f90441454b0 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f904c0a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f904406e0e0 con 0x7f90440b4da0 2024-05-09T06:30:28.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50358/0 (socket says 172.21.15.1:50358) 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 -- 172.21.15.1:0/574697386 learned_addr learned my addr 172.21.15.1:0/574697386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 -- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90440b4da0 msgr2=0x7f90440acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90440b4da0 0x7f90440acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 -- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90441454b0 msgr2=0x7f9044155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90441454b0 0x7f9044155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 -- 172.21.15.1:0/574697386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904413fad0 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.291+0000 7f9049e41640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f904000e9b0 tx=0x7f904000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.292+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040017070 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.292+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904000c9a0 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.292+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904413ea30 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.292+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904000cd90 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.292+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904413eff0 con 0x7f90440b2a60 2024-05-09T06:30:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.293+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90440752b0 con 0x7f90440b2a60 2024-05-09T06:30:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.293+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f904001a070 con 0x7f90440b2a60 2024-05-09T06:30:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.294+0000 7f903a7fc640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 0x7f902806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.294+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f904008ad00 con 0x7f90440b2a60 2024-05-09T06:30:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.294+0000 7f9049640640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 0x7f902806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:28.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.294+0000 7f9049640640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 0x7f902806e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9034034000 tx=0x7f903402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:28.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.298+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9040059820 con 0x7f90440b2a60 2024-05-09T06:30:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:28.554+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f90440cd1a0 con 0x7f90440b2a60 2024-05-09T06:30:29.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.540+0000 7f903a7fc640 1 -- 172.21.15.1:0/574697386 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v182) ==== 172+0+0 (secure 0 0 0) 0x7f9040059120 con 0x7f90440b2a60 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 msgr2=0x7f902806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 0x7f902806e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9034034000 tx=0x7f903402c000 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 msgr2=0x7f904413e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f904000e9b0 tx=0x7f904000ee70 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 shutdown_connections 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f902806be60 0x7f902806e320 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90441454b0 0x7f9044155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90440b4da0 0x7f90440acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 --2- 172.21.15.1:0/574697386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90440b2a60 0x7f904413e130 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 >> 172.21.15.1:0/574697386 conn(0x7f90440cc4b0 msgr2=0x7f9044070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:29.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 shutdown_connections 2024-05-09T06:30:29.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.543+0000 7f904c0a5640 1 -- 172.21.15.1:0/574697386 wait complete. 2024-05-09T06:30:29.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:30:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.670+0000 7f4a6cf9e640 1 Processor -- start 2024-05-09T06:30:29.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.670+0000 7f4a6cf9e640 1 -- start start 2024-05-09T06:30:29.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a6cf9e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a680d2950 0x7f4a680d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a6cf9e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4a680b2b60 0x7f4a680b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a6cf9e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a680cfbd0 con 0x7f4a6815e230 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a6cf9e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4a680d1c50 con 0x7f4a680b2b60 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a6cf9e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4a680b0930 con 0x7f4a680d2950 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a67fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4a680b2b60 0x7f4a680b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a677fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4a6815e230 0x7f4a680d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38244/0 (socket says 172.21.15.1:38244) 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.671+0000 7f4a677fe640 1 -- 172.21.15.1:0/1679044398 learned_addr learned my addr 172.21.15.1:0/1679044398 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a66ffd640 1 --2- 172.21.15.1:0/1679044398 >> v2:172.21.15.6:3300/0 conn(0x7f4a680d2950 0x7f4a680d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a67fff640 1 -- 172.21.15.1:0/1679044398 >> v2:172.21.15.6:3300/0 conn(0x7f4a680d2950 msgr2=0x7f4a680d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a67fff640 1 --2- 172.21.15.1:0/1679044398 >> v2:172.21.15.6:3300/0 conn(0x7f4a680d2950 0x7f4a680d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a67fff640 1 -- 172.21.15.1:0/1679044398 >> v1:172.21.15.1:6789/0 conn(0x7f4a6815e230 legacy=0x7f4a680d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a67fff640 1 -- 172.21.15.1:0/1679044398 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a680d1f30 con 0x7f4a680b2b60 2024-05-09T06:30:29.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.672+0000 7f4a67fff640 1 --2- 172.21.15.1:0/1679044398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a680b0e50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4a6806e3a0 tx=0x7f4a6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2edab93f7aa248 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a667fc640 1 -- 172.21.15.1:0/1679044398 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60007740 con 0x7f4a680b2b60 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a667fc640 1 -- 172.21.15.1:0/1679044398 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a600078e0 con 0x7f4a680b2b60 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a667fc640 1 -- 172.21.15.1:0/1679044398 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60007cb0 con 0x7f4a680b2b60 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/1679044398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 msgr2=0x7f4a680b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/1679044398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a680b0e50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4a6806e3a0 tx=0x7f4a6002a680 comp rx=0 tx=0).stop 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/1679044398 shutdown_connections 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/1679044398 >> v2:172.21.15.6:3300/0 conn(0x7f4a680d2950 0x7f4a680d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/1679044398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a680b0e50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/1679044398 >> 172.21.15.1:0/1679044398 conn(0x7f4a680d3c60 msgr2=0x7f4a680d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/1679044398 shutdown_connections 2024-05-09T06:30:29.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.673+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/1679044398 wait complete. 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.675+0000 7f4a6cf9e640 1 Processor -- start 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.675+0000 7f4a6cf9e640 1 -- start start 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a680d2950 0x7f4a68160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a6815e230 0x7f4a681510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680b1510 con 0x7f4a680b2b60 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680b1390 con 0x7f4a680d2950 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4a680b1690 con 0x7f4a6815e230 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50370/0 (socket says 172.21.15.1:50370) 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 -- 172.21.15.1:0/88730486 learned_addr learned my addr 172.21.15.1:0/88730486 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:29.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a66ffd640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a680d2950 0x7f4a68160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a67fff640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a6815e230 0x7f4a681510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 -- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a6815e230 msgr2=0x7f4a681510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a6815e230 0x7f4a681510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 -- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a680d2950 msgr2=0x7f4a68160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a680d2950 0x7f4a68160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 -- 172.21.15.1:0/88730486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a680caeb0 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a677fe640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c3c0 tx=0x7f4a5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58017070 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a58005ce0 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a680cb170 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.676+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a580072f0 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.677+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681603f0 con 0x7f4a680b2b60 2024-05-09T06:30:29.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.677+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a680752b0 con 0x7f4a680b2b60 2024-05-09T06:30:29.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.678+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a5801a070 con 0x7f4a680b2b60 2024-05-09T06:30:29.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.678+0000 7f4a4ffff640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 0x7f4a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:29.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.679+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f4a5808a0c0 con 0x7f4a680b2b60 2024-05-09T06:30:29.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.679+0000 7f4a66ffd640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 0x7f4a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:29.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.679+0000 7f4a66ffd640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 0x7f4a5406e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a6815a5c0 tx=0x7f4a5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:29.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.683+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a58058be0 con 0x7f4a680b2b60 2024-05-09T06:30:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.866+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4a680d0c30 con 0x7f4a680b2b60 2024-05-09T06:30:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.869+0000 7f4a4ffff640 1 -- 172.21.15.1:0/88730486 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f4a580584e0 con 0x7f4a680b2b60 2024-05-09T06:30:29.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.871+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 msgr2=0x7f4a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.871+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 0x7f4a5406e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a6815a5c0 tx=0x7f4a5c008040 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 msgr2=0x7f4a681fae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c3c0 tx=0x7f4a5800c880 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 shutdown_connections 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a5406be30 0x7f4a5406e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a6815e230 0x7f4a681510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a680d2950 0x7f4a68160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 --2- 172.21.15.1:0/88730486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a680b2b60 0x7f4a681fae30 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 >> 172.21.15.1:0/88730486 conn(0x7f4a680d3c60 msgr2=0x7f4a680d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 shutdown_connections 2024-05-09T06:30:29.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:29.872+0000 7f4a6cf9e640 1 -- 172.21.15.1:0/88730486 wait complete. 2024-05-09T06:30:29.865 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:30:29.865 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:30:29.907 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:30:29.959 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:30:29.959 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:30:29.959 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:29.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:30:29.975 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:30:29.975 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:29.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:30:29.994 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:30:29.995 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:30.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:30:30.032 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:30:30.032 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:30:30.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:30:30.053 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:30:30.053 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:30.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.193+0000 7ffa20916640 1 Processor -- start 2024-05-09T06:30:30.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.193+0000 7ffa20916640 1 -- start start 2024-05-09T06:30:30.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa20916640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 0x7ffa1c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa20916640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa20916640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0ce0e0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa20916640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0ce3c0 con 0x7ffa1c0cf180 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa20916640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0ce6a0 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa1a59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ffa1c06e6d0 0x7ffa1c06eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38252/0 (socket says 172.21.15.1:38252) 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa1a59c640 1 -- 172.21.15.1:0/2799853779 learned_addr learned my addr 172.21.15.1:0/2799853779 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa1ad9d640 1 --2- 172.21.15.1:0/2799853779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa13fff640 1 --2- 172.21.15.1:0/2799853779 >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 0x7ffa1c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472834977 0 0) 0x7ffa1c0ce0e0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.194+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa08003520 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3903136107 0 0) 0x7ffa08003520 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa1ad9d640 1 -- 172.21.15.1:0/2799853779 >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 msgr2=0x7ffa1c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:30.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa1ad9d640 1 --2- 172.21.15.1:0/2799853779 >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 0x7ffa1c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa1ad9d640 1 -- 172.21.15.1:0/2799853779 >> v1:172.21.15.1:6789/0 conn(0x7ffa1c06e6d0 legacy=0x7ffa1c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa1ad9d640 1 -- 172.21.15.1:0/2799853779 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0ce980 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.195+0000 7ffa13fff640 1 --2- 172.21.15.1:0/2799853779 >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 0x7ffa1c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa1ad9d640 1 --2- 172.21.15.1:0/2799853779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c0b3300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009d90 tx=0x7ffa1402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a56c14b41ff480f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c001670 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14007540 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa19d9b640 1 -- 172.21.15.1:0/2799853779 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140078f0 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 -- 172.21.15.1:0/2799853779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 msgr2=0x7ffa1c0b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 --2- 172.21.15.1:0/2799853779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c0b3300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009d90 tx=0x7ffa1402a5f0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 -- 172.21.15.1:0/2799853779 shutdown_connections 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 --2- 172.21.15.1:0/2799853779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c0b3300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 --2- 172.21.15.1:0/2799853779 >> v2:172.21.15.6:3300/0 conn(0x7ffa1c0cf180 0x7ffa1c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 -- 172.21.15.1:0/2799853779 >> 172.21.15.1:0/2799853779 conn(0x7ffa1c0cc4b0 msgr2=0x7ffa1c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 -- 172.21.15.1:0/2799853779 shutdown_connections 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.196+0000 7ffa20916640 1 -- 172.21.15.1:0/2799853779 wait complete. 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 Processor -- start 2024-05-09T06:30:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 -- start start 2024-05-09T06:30:30.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c1e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1c0cf180 0x7ffa1c1e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1447c0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c070ed0 con 0x7ffa1c0cf180 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.198+0000 7ffa20916640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c144940 con 0x7ffa1c0cdab0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50380/0 (socket says 172.21.15.1:50380) 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 -- 172.21.15.1:0/2580849825 learned_addr learned my addr 172.21.15.1:0/2580849825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa13fff640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c1e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1ad9d640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1c0cf180 0x7ffa1c1e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 -- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 msgr2=0x7ffa1c1e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c1e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 -- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1c0cf180 msgr2=0x7ffa1c1e74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1c0cf180 0x7ffa1c1e74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 -- 172.21.15.1:0/2580849825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c1e55e0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa1a59c640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c700 tx=0x7ffa0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c017070 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c005ad0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007070 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c1e58c0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.199+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c1e5e80 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.200+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c0752b0 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.201+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa0c01a070 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.201+0000 7ffa12ffd640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 0x7ff9fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.201+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7ffa0c08a060 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.202+0000 7ffa13fff640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 0x7ff9fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.202+0000 7ffa13fff640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 0x7ff9fc06e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009560 tx=0x7ffa04009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.206+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa0c058540 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.427+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa1c06cf50 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.427+0000 7ffa12ffd640 1 -- 172.21.15.1:0/2580849825 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3854 (secure 0 0 0) 0x7ffa0c05c840 con 0x7ffa1c06e6d0 2024-05-09T06:30:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:30.409 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3722953627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3722953627},{"type":"v1","addr":"172.21.15.6:6834","nonce":3722953627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5497,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/662810375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":662810375},{"type":"v1","addr":"172.21.15.1:6837","nonce":662810375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3747283466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3747283466},{"type":"v1","addr":"172.21.15.6:6835","nonce":3747283466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":6477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1370563853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1370563853},{"type":"v1","addr":"172.21.15.1:6836","nonce":1370563853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-05-09T06:30:30.409 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 29 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 msgr2=0x7ff9fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 0x7ff9fc06e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009560 tx=0x7ffa04009190 comp rx=0 tx=0).stop 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 msgr2=0x7ffa1c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c700 tx=0x7ffa0c00cbc0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 shutdown_connections 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9fc06be30 0x7ff9fc06e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa1c0cf180 0x7ffa1c1e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa1c0cdab0 0x7ffa1c1e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 --2- 172.21.15.1:0/2580849825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa1c06e6d0 0x7ffa1c159ff0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 >> 172.21.15.1:0/2580849825 conn(0x7ffa1c0cc4b0 msgr2=0x7ffa1c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 shutdown_connections 2024-05-09T06:30:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.429+0000 7ffa20916640 1 -- 172.21.15.1:0/2580849825 wait complete. 2024-05-09T06:30:30.422 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:30:30.422 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:30:30.422 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:30:30.423 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:30:30.423 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:30:30.423 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:30:30.423 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:30:30.424 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:30:30.424 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:30:30.424 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:30:30.424 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:30:30.425 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:30:30.425 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:30:30.425 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:30:30.425 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:30:30.425 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:30:30.432 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:30:30.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:30:30.433 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:30:30.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:30:30.440 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:30:30.440 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:30:30.441 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:30:30.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:30:30.476 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:30:30.476 DEBUG:teuthology.parallel:result is None 2024-05-09T06:30:30.477 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:30:30.477 DEBUG:teuthology.parallel:result is None 2024-05-09T06:30:30.483 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:30:30.483 DEBUG:teuthology.parallel:result is None 2024-05-09T06:30:30.484 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:30:30.484 DEBUG:teuthology.parallel:result is None 2024-05-09T06:30:30.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:30:30.554 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:30:30.554 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:30:30.573 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:30:30.573 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:30:30.574 INFO:tasks.ceph.mds.b.smithi006.stderr:2024-05-09T06:30:30.592+0000 7ff0d7734640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-09T06:30:30.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.613+0000 7f9323631640 1 Processor -- start 2024-05-09T06:30:30.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.613+0000 7f9323631640 1 -- start start 2024-05-09T06:30:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9323631640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9323631640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f931c0d0f80 0x7f931c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9323631640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f931c0b33c0 con 0x7f931c0d4940 2024-05-09T06:30:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9323631640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f931c0d4dd0 con 0x7f931c0d0f80 2024-05-09T06:30:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9323631640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f931c0b3660 con 0x7f931c161950 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9320bcc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f931c0d4940 0x7f931c0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38264/0 (socket says 172.21.15.1:38264) 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9321bce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f931c0d0f80 0x7f931c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f93213cd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f9320bcc640 1 -- 172.21.15.1:0/716131155 learned_addr learned my addr 172.21.15.1:0/716131155 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f93213cd640 1 -- 172.21.15.1:0/716131155 >> v2:172.21.15.1:3300/0 conn(0x7f931c0d0f80 msgr2=0x7f931c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f93213cd640 1 --2- 172.21.15.1:0/716131155 >> v2:172.21.15.1:3300/0 conn(0x7f931c0d0f80 0x7f931c0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f93213cd640 1 -- 172.21.15.1:0/716131155 >> v1:172.21.15.1:6789/0 conn(0x7f931c0d4940 legacy=0x7f931c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.614+0000 7f93213cd640 1 -- 172.21.15.1:0/716131155 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c0b3940 con 0x7f931c161950 2024-05-09T06:30:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.615+0000 7f93213cd640 1 --2- 172.21.15.1:0/716131155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c0d2400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f931c0b3590 tx=0x7f931802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ce66369ff921fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.615+0000 7f9313fff640 1 -- 172.21.15.1:0/716131155 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318037070 con 0x7f931c161950 2024-05-09T06:30:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.615+0000 7f9313fff640 1 -- 172.21.15.1:0/716131155 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9318032440 con 0x7f931c161950 2024-05-09T06:30:30.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.615+0000 7f9313fff640 1 -- 172.21.15.1:0/716131155 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318032810 con 0x7f931c161950 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 -- 172.21.15.1:0/716131155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 msgr2=0x7f931c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 --2- 172.21.15.1:0/716131155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c0d2400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f931c0b3590 tx=0x7f931802a180 comp rx=0 tx=0).stop 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 -- 172.21.15.1:0/716131155 shutdown_connections 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 --2- 172.21.15.1:0/716131155 >> v2:172.21.15.1:3300/0 conn(0x7f931c0d0f80 0x7f931c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 --2- 172.21.15.1:0/716131155 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c0d2400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 -- 172.21.15.1:0/716131155 >> 172.21.15.1:0/716131155 conn(0x7f931c0d3e20 msgr2=0x7f931c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 -- 172.21.15.1:0/716131155 shutdown_connections 2024-05-09T06:30:30.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.616+0000 7f9323631640 1 -- 172.21.15.1:0/716131155 wait complete. 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.617+0000 7f9323631640 1 Processor -- start 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.617+0000 7f9323631640 1 -- start start 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 0x7f931c0b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f931c0b52d0 con 0x7f931c0d0f80 2024-05-09T06:30:30.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f931c0b5150 con 0x7f931c0cc0c0 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9323631640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f931c0b5450 con 0x7f931c161950 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50438/0 (socket says 172.21.15.1:50438) 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 -- 172.21.15.1:0/659767937 learned_addr learned my addr 172.21.15.1:0/659767937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9320bcc640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f9321bce640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 0x7f931c0b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 -- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 msgr2=0x7f931c1f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c1f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 -- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 msgr2=0x7f931c0b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.618+0000 7f93213cd640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 0x7f931c0b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f93213cd640 1 -- 172.21.15.1:0/659767937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c1f57b0 con 0x7f931c0d0f80 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9321bce640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 0x7f931c0b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f93213cd640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318037040 con 0x7f931c0d0f80 2024-05-09T06:30:30.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9318042070 con 0x7f931c0d0f80 2024-05-09T06:30:30.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c0cb020 con 0x7f931c0d0f80 2024-05-09T06:30:30.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318030900 con 0x7f931c0d0f80 2024-05-09T06:30:30.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.619+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c0cb5e0 con 0x7f931c0d0f80 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.620+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931c0752b0 con 0x7f931c0d0f80 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.621+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9318030aa0 con 0x7f931c0d0f80 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.621+0000 7f9311ffb640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 0x7f930006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.621+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f93180ab640 con 0x7f931c0d0f80 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.622+0000 7f9320bcc640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 0x7f930006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:30.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.622+0000 7f9320bcc640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 0x7f930006e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f931c06d250 tx=0x7f930c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:30.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.626+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9318045360 con 0x7f931c0d0f80 2024-05-09T06:30:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:30.824+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f931c0d21e0 con 0x7f931c0d0f80 2024-05-09T06:30:31.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.550+0000 7f9311ffb640 1 -- 172.21.15.1:0/659767937 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) ==== 97+0+0 (secure 0 0 0) 0x7f931807a1f0 con 0x7f931c0d0f80 2024-05-09T06:30:31.532 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 msgr2=0x7f930006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 0x7f930006e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f931c06d250 tx=0x7f930c009040 comp rx=0 tx=0).stop 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 msgr2=0x7f931c0cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 shutdown_connections 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f930006be30 0x7f930006e2f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f931c0cc0c0 0x7f931c0b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f931c161950 0x7f931c1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 --2- 172.21.15.1:0/659767937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f931c0d0f80 0x7f931c0cd9d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 >> 172.21.15.1:0/659767937 conn(0x7f931c0d3e20 msgr2=0x7f931c15ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 shutdown_connections 2024-05-09T06:30:31.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.554+0000 7f9323631640 1 -- 172.21.15.1:0/659767937 wait complete. 2024-05-09T06:30:31.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:30:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.679+0000 7f5aa764d640 1 Processor -- start 2024-05-09T06:30:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.679+0000 7f5aa764d640 1 -- start start 2024-05-09T06:30:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa764d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5aa0163c70 0x7f5aa0166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa764d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00ce0e0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa764d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00ce3c0 con 0x7f5aa00cf180 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa764d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa00ce6a0 con 0x7f5aa0163c70 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa53e9640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5aa006e6d0 0x7f5aa006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38278/0 (socket says 172.21.15.1:38278) 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa53e9640 1 -- 172.21.15.1:0/2242566226 learned_addr learned my addr 172.21.15.1:0/2242566226 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.680+0000 7f5aa5bea640 1 --2- 172.21.15.1:0/2242566226 >> v2:172.21.15.1:3300/0 conn(0x7f5aa0163c70 0x7f5aa0166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.681+0000 7f5aa5bea640 1 -- 172.21.15.1:0/2242566226 >> v1:172.21.15.6:6789/0 conn(0x7f5aa00cf180 legacy=0x7f5aa00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.681+0000 7f5aa5bea640 1 -- 172.21.15.1:0/2242566226 >> v1:172.21.15.1:6789/0 conn(0x7f5aa006e6d0 legacy=0x7f5aa006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.681+0000 7f5aa5bea640 1 -- 172.21.15.1:0/2242566226 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa00ce980 con 0x7f5aa0163c70 2024-05-09T06:30:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.681+0000 7f5a9ffff640 1 -- 172.21.15.1:0/2242566226 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810036108 0 0) 0x7f5aa00ce0e0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.681+0000 7f5aa5bea640 1 --2- 172.21.15.1:0/2242566226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa0163c70 0x7f5aa0166060 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009d90 tx=0x7f5a9002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e65afda127626f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5a9ffff640 1 -- 172.21.15.1:0/2242566226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007740 con 0x7f5aa0163c70 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5a9ffff640 1 -- 172.21.15.1:0/2242566226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a900078e0 con 0x7f5aa0163c70 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5a9ffff640 1 -- 172.21.15.1:0/2242566226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007cb0 con 0x7f5aa0163c70 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 -- 172.21.15.1:0/2242566226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa0163c70 msgr2=0x7f5aa0166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 --2- 172.21.15.1:0/2242566226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa0163c70 0x7f5aa0166060 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009d90 tx=0x7f5a9002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 -- 172.21.15.1:0/2242566226 shutdown_connections 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 --2- 172.21.15.1:0/2242566226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa0163c70 0x7f5aa0166060 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 -- 172.21.15.1:0/2242566226 >> 172.21.15.1:0/2242566226 conn(0x7f5aa00cc4b0 msgr2=0x7f5aa00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 -- 172.21.15.1:0/2242566226 shutdown_connections 2024-05-09T06:30:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.682+0000 7f5aa764d640 1 -- 172.21.15.1:0/2242566226 wait complete. 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 Processor -- start 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 -- start start 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aa00cf180 0x7f5aa015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aa0163c70 0x7f5aa015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa01447c0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0070ed0 con 0x7f5aa00cf180 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa764d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa0144940 con 0x7f5aa0163c70 2024-05-09T06:30:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa53e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa53e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50466/0 (socket says 172.21.15.1:50466) 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.684+0000 7f5aa53e9640 1 -- 172.21.15.1:0/395945485 learned_addr learned my addr 172.21.15.1:0/395945485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa4be8640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aa00cf180 0x7f5aa015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 -- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aa0163c70 msgr2=0x7f5aa015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aa0163c70 0x7f5aa015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 -- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aa00cf180 msgr2=0x7f5aa015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aa00cf180 0x7f5aa015b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 -- 172.21.15.1:0/395945485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa015d730 con 0x7f5aa006e6d0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa53e9640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800c430 tx=0x7f5a9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a980074e0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a980047e0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa015da10 con 0x7f5aa006e6d0 2024-05-09T06:30:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98004bd0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.685+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa015d350 con 0x7f5aa006e6d0 2024-05-09T06:30:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.686+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa00752b0 con 0x7f5aa006e6d0 2024-05-09T06:30:31.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.687+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a98011070 con 0x7f5aa006e6d0 2024-05-09T06:30:31.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.687+0000 7f5a9dffb640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 0x7f5a8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.687+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f5a9808a840 con 0x7f5aa006e6d0 2024-05-09T06:30:31.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.687+0000 7f5aa4be8640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 0x7f5a8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.688+0000 7f5aa4be8640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 0x7f5a8c06e320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5aa015d140 tx=0x7f5a94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.692+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a9800e280 con 0x7f5aa006e6d0 2024-05-09T06:30:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.947+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5aa006cf50 con 0x7f5aa006e6d0 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.947+0000 7f5a9dffb640 1 -- 172.21.15.1:0/395945485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f5a98059440 con 0x7f5aa006e6d0 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:30:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:30:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:30:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:30:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:30:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:30:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:30:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:30:31.942 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 msgr2=0x7f5a8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 0x7f5a8c06e320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5aa015d140 tx=0x7f5a94009040 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 msgr2=0x7f5aa012ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800c430 tx=0x7f5a9800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 shutdown_connections 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a8c06be60 0x7f5a8c06e320 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aa0163c70 0x7f5aa015b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aa00cf180 0x7f5aa015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 --2- 172.21.15.1:0/395945485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa006e6d0 0x7f5aa012ade0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 >> 172.21.15.1:0/395945485 conn(0x7f5aa00cc4b0 msgr2=0x7f5aa00b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 shutdown_connections 2024-05-09T06:30:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:31.949+0000 7f5aa764d640 1 -- 172.21.15.1:0/395945485 wait complete. 2024-05-09T06:30:31.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:32.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.068+0000 7fbb8e438640 1 Processor -- start 2024-05-09T06:30:32.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.068+0000 7fbb8e438640 1 -- start start 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8e438640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbb8815db10 0x7fbb880b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8e438640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb88070f40 con 0x7fbb8815e6d0 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8e438640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb880cf170 con 0x7fbb880715d0 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8e438640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbb880cf450 con 0x7fbb8815db10 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb87fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbb8815db10 0x7fbb880b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8c9d5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbb8815e6d0 0x7fbb8806e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38290/0 (socket says 172.21.15.1:38290) 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb8c9d5640 1 -- 172.21.15.1:0/3439496155 learned_addr learned my addr 172.21.15.1:0/3439496155 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb877fe640 1 -- 172.21.15.1:0/3439496155 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936151284 0 0) 0x7fbb88070f40 con 0x7fbb8815e6d0 2024-05-09T06:30:32.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb877fe640 1 -- 172.21.15.1:0/3439496155 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb74005730 con 0x7fbb8815e6d0 2024-05-09T06:30:32.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb87fff640 1 -- 172.21.15.1:0/3439496155 >> v1:172.21.15.6:6789/0 conn(0x7fbb880715d0 legacy=0x7fbb8815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb87fff640 1 -- 172.21.15.1:0/3439496155 >> v1:172.21.15.1:6789/0 conn(0x7fbb8815e6d0 legacy=0x7fbb8806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.069+0000 7fbb87fff640 1 -- 172.21.15.1:0/3439496155 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb88070c10 con 0x7fbb8815db10 2024-05-09T06:30:32.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.070+0000 7fbb87fff640 1 --2- 172.21.15.1:0/3439496155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb8815db10 0x7fbb880b2a60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097a0 tx=0x7fbb7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e57316cc3812c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.070+0000 7fbb877fe640 1 -- 172.21.15.1:0/3439496155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037070 con 0x7fbb8815db10 2024-05-09T06:30:32.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.070+0000 7fbb877fe640 1 -- 172.21.15.1:0/3439496155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78032440 con 0x7fbb8815db10 2024-05-09T06:30:32.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.070+0000 7fbb877fe640 1 -- 172.21.15.1:0/3439496155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78032810 con 0x7fbb8815db10 2024-05-09T06:30:32.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 -- 172.21.15.1:0/3439496155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb8815db10 msgr2=0x7fbb880b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 --2- 172.21.15.1:0/3439496155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb8815db10 0x7fbb880b2a60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097a0 tx=0x7fbb7802a230 comp rx=0 tx=0).stop 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 -- 172.21.15.1:0/3439496155 shutdown_connections 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 --2- 172.21.15.1:0/3439496155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb8815db10 0x7fbb880b2a60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 -- 172.21.15.1:0/3439496155 >> 172.21.15.1:0/3439496155 conn(0x7fbb880cc4b0 msgr2=0x7fbb880cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 -- 172.21.15.1:0/3439496155 shutdown_connections 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.071+0000 7fbb8e438640 1 -- 172.21.15.1:0/3439496155 wait complete. 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.072+0000 7fbb8e438640 1 Processor -- start 2024-05-09T06:30:32.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.072+0000 7fbb8e438640 1 -- start start 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb8815db10 0x7fbb880cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb8815e6d0 0x7fbb880ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8815edc0 con 0x7fbb880715d0 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8815ec40 con 0x7fbb8815e6d0 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8806ea50 con 0x7fbb8815db10 2024-05-09T06:30:32.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50488/0 (socket says 172.21.15.1:50488) 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 -- 172.21.15.1:0/258851196 learned_addr learned my addr 172.21.15.1:0/258851196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 -- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb8815db10 msgr2=0x7fbb880cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb8815db10 0x7fbb880cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 -- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb8815e6d0 msgr2=0x7fbb880ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb8815e6d0 0x7fbb880ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 -- 172.21.15.1:0/258851196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8814c4b0 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb87fff640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbb78035040 tx=0x7fbb780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037040 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78030500 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb88157c50 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78030910 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.073+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb88158210 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.074+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb880cd550 con 0x7fbb880715d0 2024-05-09T06:30:32.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.075+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb78044020 con 0x7fbb880715d0 2024-05-09T06:30:32.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.075+0000 7fbb857fa640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.075+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7fbb780aaf10 con 0x7fbb880715d0 2024-05-09T06:30:32.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.075+0000 7fbb7ffff640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.076+0000 7fbb7ffff640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000f60 tx=0x7fbb700079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.080+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb78079b10 con 0x7fbb880715d0 2024-05-09T06:30:32.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.263+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb880752b0 con 0x7fbb880715d0 2024-05-09T06:30:32.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.264+0000 7fbb857fa640 1 -- 172.21.15.1:0/258851196 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13032 (secure 0 0 0) 0x7fbb78079410 con 0x7fbb880715d0 2024-05-09T06:30:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:32.246 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":60,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:31.558790+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 msgr2=0x7fbb6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000f60 tx=0x7fbb700079d0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 msgr2=0x7fbb8814bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbb78035040 tx=0x7fbb780046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 shutdown_connections 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb8815e6d0 0x7fbb880ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb8815db10 0x7fbb880cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 --2- 172.21.15.1:0/258851196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb880715d0 0x7fbb8814bf70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.266+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 >> 172.21.15.1:0/258851196 conn(0x7fbb880cc4b0 msgr2=0x7fbb880cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.267+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 shutdown_connections 2024-05-09T06:30:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.267+0000 7fbb8e438640 1 -- 172.21.15.1:0/258851196 wait complete. 2024-05-09T06:30:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:30:32.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:30:32.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.385+0000 7efe64431640 1 Processor -- start 2024-05-09T06:30:32.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.385+0000 7efe64431640 1 -- start start 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe64431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0d06f0 0x7efe5c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe64431640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe5c0c41e0 0x7efe5c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe64431640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c072050 con 0x7efe5c06b1e0 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe64431640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efe5c0cd160 con 0x7efe5c0c41e0 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe64431640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c072330 con 0x7efe5c0d06f0 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe5c0c41e0 0x7efe5c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe5c0c41e0 0x7efe5c0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50498/0 (socket says 172.21.15.1:50498) 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 -- 172.21.15.1:0/1445485139 learned_addr learned my addr 172.21.15.1:0/1445485139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe621cd640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0d06f0 0x7efe5c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 -- 172.21.15.1:0/1445485139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0d06f0 msgr2=0x7efe5c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0d06f0 0x7efe5c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 -- 172.21.15.1:0/1445485139 >> v1:172.21.15.6:6789/0 conn(0x7efe5c06b1e0 legacy=0x7efe5c161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.386+0000 7efe619cc640 1 -- 172.21.15.1:0/1445485139 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c072610 con 0x7efe5c0c41e0 2024-05-09T06:30:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.387+0000 7efe619cc640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c0c41e0 0x7efe5c0ccc20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009870 tx=0x7efe4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d56154b6800c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.387+0000 7efe611cb640 1 -- 172.21.15.1:0/1445485139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c037070 con 0x7efe5c0c41e0 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.387+0000 7efe611cb640 1 -- 172.21.15.1:0/1445485139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c032440 con 0x7efe5c0c41e0 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.387+0000 7efe611cb640 1 -- 172.21.15.1:0/1445485139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c032810 con 0x7efe5c0c41e0 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 -- 172.21.15.1:0/1445485139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c0c41e0 msgr2=0x7efe5c0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c0c41e0 0x7efe5c0ccc20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009870 tx=0x7efe4c02a210 comp rx=0 tx=0).stop 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 -- 172.21.15.1:0/1445485139 shutdown_connections 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c0c41e0 0x7efe5c0ccc20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 --2- 172.21.15.1:0/1445485139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0d06f0 0x7efe5c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 -- 172.21.15.1:0/1445485139 >> 172.21.15.1:0/1445485139 conn(0x7efe5c0d0b50 msgr2=0x7efe5c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 -- 172.21.15.1:0/1445485139 shutdown_connections 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.388+0000 7efe64431640 1 -- 172.21.15.1:0/1445485139 wait complete. 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.389+0000 7efe64431640 1 Processor -- start 2024-05-09T06:30:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.389+0000 7efe64431640 1 -- start start 2024-05-09T06:30:32.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 0x7efe5c0a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 0x7efe5c0a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c0cf8e0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c071ce0 con 0x7efe5c0d06f0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe64431640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c0cfa60 con 0x7efe5c0c41e0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50514/0 (socket says 172.21.15.1:50514) 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 -- 172.21.15.1:0/1917681724 learned_addr learned my addr 172.21.15.1:0/1917681724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe619cc640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 0x7efe5c0a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe629ce640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 0x7efe5c0a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 -- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 msgr2=0x7efe5c0a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 0x7efe5c0a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 -- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 msgr2=0x7efe5c0a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 0x7efe5c0a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 -- 172.21.15.1:0/1917681724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c0c92d0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe619cc640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 0x7efe5c0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe629ce640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 0x7efe5c0a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.390+0000 7efe621cd640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efe5800d5f0 tx=0x7efe5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.391+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580125f0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.391+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe58019070 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.391+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c0c95b0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.391+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580129d0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.391+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c0c9b70 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.392+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c0752b0 con 0x7efe5c06b1e0 2024-05-09T06:30:32.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.392+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe58012b70 con 0x7efe5c06b1e0 2024-05-09T06:30:32.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.393+0000 7efe52ffd640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.393+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7efe5808b020 con 0x7efe5c06b1e0 2024-05-09T06:30:32.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.393+0000 7efe619cc640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.393+0000 7efe619cc640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efe4c034000 tx=0x7efe4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.397+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe58059c20 con 0x7efe5c06b1e0 2024-05-09T06:30:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.585+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe5c072f30 con 0x7efe5c06b1e0 2024-05-09T06:30:32.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.585+0000 7efe52ffd640 1 -- 172.21.15.1:0/1917681724 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7efe58059520 con 0x7efe5c06b1e0 2024-05-09T06:30:32.567 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:30:32.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 msgr2=0x7efe4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efe4c034000 tx=0x7efe4c02e000 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 msgr2=0x7efe5c1e38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efe5800d5f0 tx=0x7efe5800dab0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 shutdown_connections 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe5c0d06f0 0x7efe5c0a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe5c0c41e0 0x7efe5c0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 --2- 172.21.15.1:0/1917681724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe5c06b1e0 0x7efe5c1e38f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.587+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 >> 172.21.15.1:0/1917681724 conn(0x7efe5c0d0b50 msgr2=0x7efe5c0c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.588+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 shutdown_connections 2024-05-09T06:30:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.588+0000 7efe64431640 1 -- 172.21.15.1:0/1917681724 wait complete. 2024-05-09T06:30:32.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:30:32.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 Processor -- start 2024-05-09T06:30:32.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 -- start start 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f86a80d4940 0x7f86a80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86a80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a80b33c0 con 0x7f86a8161950 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f86a80d4dd0 con 0x7f86a80d4940 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.703+0000 7f86ae57d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f86a80b3660 con 0x7f86a80d2b10 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f86a8161950 0x7f86a80d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38300/0 (socket says 172.21.15.1:38300) 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a7fff640 1 -- 172.21.15.1:0/2387961299 learned_addr learned my addr 172.21.15.1:0/2387961299 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a77fe640 1 --2- 172.21.15.1:0/2387961299 >> v2:172.21.15.6:3300/0 conn(0x7f86a80d4940 0x7f86a80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86acb1a640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86a80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909744647 0 0) 0x7f86a80b33c0 con 0x7f86a8161950 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8690003520 con 0x7f86a8161950 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2082913212 0 0) 0x7f8690003520 con 0x7f86a8161950 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 msgr2=0x7f86a80b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86a80b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 >> v2:172.21.15.6:3300/0 conn(0x7f86a80d4940 msgr2=0x7f86a80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 --2- 172.21.15.1:0/2387961299 >> v2:172.21.15.6:3300/0 conn(0x7f86a80d4940 0x7f86a80b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a80b3940 con 0x7f86a8161950 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8698001690 con 0x7f86a8161950 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 >> v1:172.21.15.1:6789/0 conn(0x7f86a8161950 legacy=0x7f86a80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f8690004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8690004d40 0x7f86900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80d4dd0 con 0x7f8690003f30 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80b3660 con 0x7f8690004d40 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f869000b9c0 con 0x7f86a80d2b10 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.704+0000 7f86a77fe640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f8690004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a7fff640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86acb1a640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8690004d40 0x7f86900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 -- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 msgr2=0x7f86900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a80d2b10 0x7f86900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 -- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8690004d40 msgr2=0x7f86900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8690004d40 0x7f86900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 -- 172.21.15.1:0/2387961299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8690003520 con 0x7f8690003f30 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.705+0000 7f86a77fe640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f8690004800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f869c009ac0 tx=0x7f8690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbbe3e1f1f04d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c036070 con 0x7f8690003f30 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869c031400 con 0x7f8690003f30 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86a6ffd640 1 -- 172.21.15.1:0/2387961299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c009690 con 0x7f8690003f30 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 -- 172.21.15.1:0/2387961299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 msgr2=0x7f8690004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 --2- 172.21.15.1:0/2387961299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f8690004800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f869c009ac0 tx=0x7f8690005830 comp rx=0 tx=0).stop 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86acb1a640 1 -- 172.21.15.1:0/2387961299 reap_dead start 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 -- 172.21.15.1:0/2387961299 shutdown_connections 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 -- 172.21.15.1:0/2387961299 >> 172.21.15.1:0/2387961299 conn(0x7f86a80d3e20 msgr2=0x7f86a80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 -- 172.21.15.1:0/2387961299 shutdown_connections 2024-05-09T06:30:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.706+0000 7f86ae57d640 1 -- 172.21.15.1:0/2387961299 wait complete. 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.707+0000 7f86ae57d640 1 Processor -- start 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 -- start start 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a81f44f0 0x7f86a80cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 0x7f86a809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80d1100 con 0x7f8690003f30 2024-05-09T06:30:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80d0f80 con 0x7f86a80cd610 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.708+0000 7f86ae57d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f86a80d1c00 con 0x7f86a81f44f0 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50536/0 (socket says 172.21.15.1:50536) 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 -- 172.21.15.1:0/973559814 learned_addr learned my addr 172.21.15.1:0/973559814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86acb1a640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 0x7f86a809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a77fe640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a81f44f0 0x7f86a80cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 -- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a81f44f0 msgr2=0x7f86a80cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a81f44f0 0x7f86a80cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 -- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 msgr2=0x7f86a809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 0x7f86a809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 -- 172.21.15.1:0/973559814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a8151960 con 0x7f8690003f30 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86acb1a640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 0x7f86a809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a7fff640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f86980072c0 tx=0x7f8698007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869800f9c0 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8698018070 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869800fde0 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a80cc070 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.709+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a80cc5f0 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.710+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a80752b0 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.711+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f869801d460 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.711+0000 7f86a4ff9640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.711+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f869808ad00 con 0x7f8690003f30 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.711+0000 7f86a77fe640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.712+0000 7f86a77fe640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 0x7f869406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f869c033000 tx=0x7f869c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.716+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86a8151960 con 0x7f8690003f30 2024-05-09T06:30:32.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.900+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f86a80d21e0 con 0x7f8690003f30 2024-05-09T06:30:32.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.900+0000 7f86a4ff9640 1 -- 172.21.15.1:0/973559814 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) ==== 84+0+65 (secure 0 0 0) 0x7f8698059900 con 0x7f8690003f30 2024-05-09T06:30:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:32.882 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:30:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:30:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:30:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:32.883 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:30:32.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 msgr2=0x7f869406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 0x7f869406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f869c033000 tx=0x7f869c008030 comp rx=0 tx=0).stop 2024-05-09T06:30:32.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 msgr2=0x7f86a8151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:32.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f86980072c0 tx=0x7f8698007780 comp rx=0 tx=0).stop 2024-05-09T06:30:32.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 shutdown_connections 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86a80cd610 0x7f86a809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f86a81f44f0 0x7f86a80cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 --2- 172.21.15.1:0/973559814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8690003f30 0x7f86a8151420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 >> 172.21.15.1:0/973559814 conn(0x7f86a80d3e20 msgr2=0x7f86a815f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 shutdown_connections 2024-05-09T06:30:32.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:32.902+0000 7f86ae57d640 1 -- 172.21.15.1:0/973559814 wait complete. 2024-05-09T06:30:32.895 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:30:32.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:30:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.022+0000 7fb92e4f5640 1 Processor -- start 2024-05-09T06:30:33.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.022+0000 7fb92e4f5640 1 -- start start 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92e4f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9280d03c0 0x7fb92806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92e4f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb9281446d0 0x7fb92806d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92e4f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9280cdd50 con 0x7fb928143bb0 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92e4f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb9280ce8d0 con 0x7fb9281446d0 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92e4f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb9280ce030 con 0x7fb9280d03c0 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb9281446d0 0x7fb92806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb9281446d0 0x7fb92806d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50538/0 (socket says 172.21.15.1:50538) 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 -- 172.21.15.1:0/1790742621 learned_addr learned my addr 172.21.15.1:0/1790742621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb9277fe640 1 --2- 172.21.15.1:0/1790742621 >> v2:172.21.15.6:3300/0 conn(0x7fb9280d03c0 0x7fb92806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 -- 172.21.15.1:0/1790742621 >> v2:172.21.15.6:3300/0 conn(0x7fb9280d03c0 msgr2=0x7fb92806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 --2- 172.21.15.1:0/1790742621 >> v2:172.21.15.6:3300/0 conn(0x7fb9280d03c0 0x7fb92806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 -- 172.21.15.1:0/1790742621 >> v1:172.21.15.6:6789/0 conn(0x7fb928143bb0 legacy=0x7fb9280cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.023+0000 7fb92ca92640 1 -- 172.21.15.1:0/1790742621 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9280ce310 con 0x7fb9281446d0 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.024+0000 7fb92ca92640 1 --2- 172.21.15.1:0/1790742621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9281446d0 0x7fb92806d850 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb9280cd800 tx=0x7fb92002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938435be4bba676d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.024+0000 7fb926ffd640 1 -- 172.21.15.1:0/1790742621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920036070 con 0x7fb9281446d0 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.024+0000 7fb926ffd640 1 -- 172.21.15.1:0/1790742621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb920031600 con 0x7fb9281446d0 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.024+0000 7fb926ffd640 1 -- 172.21.15.1:0/1790742621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200319d0 con 0x7fb9281446d0 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1790742621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9281446d0 msgr2=0x7fb92806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1790742621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9281446d0 0x7fb92806d850 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb9280cd800 tx=0x7fb92002a660 comp rx=0 tx=0).stop 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1790742621 shutdown_connections 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1790742621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9281446d0 0x7fb92806d850 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1790742621 >> v2:172.21.15.6:3300/0 conn(0x7fb9280d03c0 0x7fb92806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1790742621 >> 172.21.15.1:0/1790742621 conn(0x7fb92806b3c0 msgr2=0x7fb92806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1790742621 shutdown_connections 2024-05-09T06:30:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.025+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1790742621 wait complete. 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.026+0000 7fb92e4f5640 1 Processor -- start 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.026+0000 7fb92e4f5640 1 -- start start 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb928143bb0 0x7fb9281401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9281446d0 0x7fb928140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280cdaf0 con 0x7fb9280d03c0 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280cd970 con 0x7fb9281446d0 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb92e4f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb928144110 con 0x7fb928143bb0 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50552/0 (socket says 172.21.15.1:50552) 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 -- 172.21.15.1:0/1235772351 learned_addr learned my addr 172.21.15.1:0/1235772351 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 -- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb928143bb0 msgr2=0x7fb9281401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb928143bb0 0x7fb9281401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 -- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9281446d0 msgr2=0x7fb928140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9281446d0 0x7fb928140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 -- 172.21.15.1:0/1235772351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9281e53c0 con 0x7fb9280d03c0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb927fff640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c430 tx=0x7fb91800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.027+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918011790 con 0x7fb9280d03c0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.028+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb918017070 con 0x7fb9280d03c0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.028+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918011bb0 con 0x7fb9280d03c0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.028+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9281e56a0 con 0x7fb9280d03c0 2024-05-09T06:30:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.028+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281e5c60 con 0x7fb9280d03c0 2024-05-09T06:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.029+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928075320 con 0x7fb9280d03c0 2024-05-09T06:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.029+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb91801c460 con 0x7fb9280d03c0 2024-05-09T06:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.029+0000 7fb924ff9640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 0x7fb91406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.030+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7fb91808ab60 con 0x7fb9280d03c0 2024-05-09T06:30:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.030+0000 7fb9277fe640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 0x7fb91406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.030+0000 7fb9277fe640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 0x7fb91406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb928142fa0 tx=0x7fb91c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.034+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9180596d0 con 0x7fb9280d03c0 2024-05-09T06:30:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.217+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb928071300 con 0x7fb9280d03c0 2024-05-09T06:30:33.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.556+0000 7fb924ff9640 1 -- 172.21.15.1:0/1235772351 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) ==== 138+0+0 (secure 0 0 0) 0x7fb918058fd0 con 0x7fb9280d03c0 2024-05-09T06:30:33.538 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 msgr2=0x7fb91406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 0x7fb91406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb928142fa0 tx=0x7fb91c008040 comp rx=0 tx=0).stop 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 msgr2=0x7fb92813d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c430 tx=0x7fb91800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 shutdown_connections 2024-05-09T06:30:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb91406be30 0x7fb91406e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9281446d0 0x7fb928140710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb928143bb0 0x7fb9281401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 --2- 172.21.15.1:0/1235772351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9280d03c0 0x7fb92813d4e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.559+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 >> 172.21.15.1:0/1235772351 conn(0x7fb92806b3c0 msgr2=0x7fb92806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.560+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 shutdown_connections 2024-05-09T06:30:33.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.560+0000 7fb92e4f5640 1 -- 172.21.15.1:0/1235772351 wait complete. 2024-05-09T06:30:33.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:30:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.688+0000 7f46def90640 1 Processor -- start 2024-05-09T06:30:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.688+0000 7f46def90640 1 -- start start 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46def90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46d80cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46def90640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d80d2060 con 0x7f46d815e210 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46def90640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d80d4ac0 con 0x7f46d80d0fb0 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46def90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46d80b3170 con 0x7f46d80d29b0 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46dcd2c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f46d815e210 0x7f46d80d22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38312/0 (socket says 172.21.15.1:38312) 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.689+0000 7f46dcd2c640 1 -- 172.21.15.1:0/2182016400 learned_addr learned my addr 172.21.15.1:0/2182016400 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d7fff640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46d80cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1279561996 0 0) 0x7f46d80d2060 con 0x7f46d815e210 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46c4003520 con 0x7f46d815e210 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104662127 0 0) 0x7f46c4003520 con 0x7f46d815e210 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 msgr2=0x7f46d80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46d80cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 >> v1:172.21.15.6:6789/0 conn(0x7f46d80d0fb0 legacy=0x7f46d80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d80d1d80 con 0x7f46d815e210 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f46d0001670 con 0x7f46d815e210 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 >> v1:172.21.15.1:6789/0 conn(0x7f46d815e210 legacy=0x7f46d80d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c40039e0 0x7f46c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46d80b3170 con 0x7f46c40039e0 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46c400baf0 con 0x7f46c4004800 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46c400bcf0 con 0x7f46d80d29b0 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c40039e0 0x7f46c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46d7fff640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.690+0000 7f46dd52d640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 -- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 msgr2=0x7f46c4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80d29b0 0x7f46c4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 -- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c4004800 msgr2=0x7f46c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 -- 172.21.15.1:0/2182016400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c4003520 con 0x7f46c40039e0 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c40039e0 0x7f46c4003dc0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c190 tx=0x7f46c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a24e172ee3b2dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.691+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0007cc0 con 0x7f46c40039e0 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d0005ad0 con 0x7f46c40039e0 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46d77fe640 1 -- 172.21.15.1:0/2182016400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0007070 con 0x7f46c40039e0 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 -- 172.21.15.1:0/2182016400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c40039e0 msgr2=0x7f46c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 --2- 172.21.15.1:0/2182016400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46c40039e0 0x7f46c4003dc0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c190 tx=0x7f46c4005830 comp rx=0 tx=0).stop 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46dd52d640 1 -- 172.21.15.1:0/2182016400 reap_dead start 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 -- 172.21.15.1:0/2182016400 shutdown_connections 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 -- 172.21.15.1:0/2182016400 >> 172.21.15.1:0/2182016400 conn(0x7f46d80d3e70 msgr2=0x7f46d80d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 -- 172.21.15.1:0/2182016400 shutdown_connections 2024-05-09T06:30:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.692+0000 7f46def90640 1 -- 172.21.15.1:0/2182016400 wait complete. 2024-05-09T06:30:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.693+0000 7f46def90640 1 Processor -- start 2024-05-09T06:30:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.693+0000 7f46def90640 1 -- start start 2024-05-09T06:30:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 0x7f46d8161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 0x7f46d81f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46d815e5f0 con 0x7f46d81f7bd0 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46d80d4da0 con 0x7f46c40039e0 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46def90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46d815e770 con 0x7f46d80cd880 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46dd52d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46dd52d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50570/0 (socket says 172.21.15.1:50570) 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.694+0000 7f46dd52d640 1 -- 172.21.15.1:0/3104479374 learned_addr learned my addr 172.21.15.1:0/3104479374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 0x7f46d8161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46d7fff640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 0x7f46d81f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 -- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 msgr2=0x7f46d81f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 0x7f46d81f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 -- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 msgr2=0x7f46d8161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 0x7f46d8161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 -- 172.21.15.1:0/3104479374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d8160460 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 0x7f46d8161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46d7fff640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 0x7f46d81f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46dd52d640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f46c800fae0 tx=0x7f46c8002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8003b80 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8018070 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d8160740 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c800d7d0 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.695+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d8160d00 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.696+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d80752b0 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.697+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46c800d970 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.697+0000 7f46d57fa640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 0x7f46b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.697+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5619+0+0 (secure 0 0 0) 0x7f46c808ae40 con 0x7f46d81f7bd0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.697+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 0x7f46b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.697+0000 7f46dcd2c640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 0x7f46b806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f46d0032000 tx=0x7f46d002d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.702+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46c801a310 con 0x7f46d81f7bd0 2024-05-09T06:30:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:33.886+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f46d80b2130 con 0x7f46d81f7bd0 2024-05-09T06:30:34.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.559+0000 7f46d57fa640 1 -- 172.21.15.1:0/3104479374 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) ==== 170+0+0 (secure 0 0 0) 0x7f46c8059840 con 0x7f46d81f7bd0 2024-05-09T06:30:34.541 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 msgr2=0x7f46b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 0x7f46b806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f46d0032000 tx=0x7f46d002d000 comp rx=0 tx=0).stop 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 msgr2=0x7f46d815cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f46c800fae0 tx=0x7f46c8002140 comp rx=0 tx=0).stop 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 shutdown_connections 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f46b806be30 0x7f46b806e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46d81f7bd0 0x7f46d815cb20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46d80cd880 0x7f46d81f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 --2- 172.21.15.1:0/3104479374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46c40039e0 0x7f46d8161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.562+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 >> 172.21.15.1:0/3104479374 conn(0x7f46d80d3e70 msgr2=0x7f46d80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:34.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.563+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 shutdown_connections 2024-05-09T06:30:34.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.563+0000 7f46def90640 1 -- 172.21.15.1:0/3104479374 wait complete. 2024-05-09T06:30:34.571 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:30:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.702+0000 7fba95130640 1 Processor -- start 2024-05-09T06:30:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.702+0000 7fba95130640 1 -- start start 2024-05-09T06:30:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba95130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 0x7fba9006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba95130640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 0x7fba9015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba95130640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9006cf40 con 0x7fba900b5260 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba95130640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fba90156980 con 0x7fba900b4de0 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba95130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fba9006d870 con 0x7fba9015c4c0 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba8ed9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 0x7fba9006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba87fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 0x7fba9015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba8f59e640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fba900b5260 0x7fba90070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:32898/0 (socket says 172.21.15.1:32898) 2024-05-09T06:30:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba8f59e640 1 -- 172.21.15.1:0/3167067501 learned_addr learned my addr 172.21.15.1:0/3167067501 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba8e59c640 1 -- 172.21.15.1:0/3167067501 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024789061 0 0) 0x7fba9006cf40 con 0x7fba900b5260 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.703+0000 7fba8e59c640 1 -- 172.21.15.1:0/3167067501 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba7c003520 con 0x7fba900b5260 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8ed9d640 1 -- 172.21.15.1:0/3167067501 >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 msgr2=0x7fba9015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8ed9d640 1 --2- 172.21.15.1:0/3167067501 >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 0x7fba9015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8ed9d640 1 -- 172.21.15.1:0/3167067501 >> v1:172.21.15.6:6789/0 conn(0x7fba900b5260 legacy=0x7fba90070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8ed9d640 1 -- 172.21.15.1:0/3167067501 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9006d220 con 0x7fba9015c4c0 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba87fff640 1 --2- 172.21.15.1:0/3167067501 >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 0x7fba9015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8ed9d640 1 --2- 172.21.15.1:0/3167067501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 0x7fba9006dd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba800097a0 tx=0x7fba8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2519a7e85a56ea6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.704+0000 7fba8e59c640 1 -- 172.21.15.1:0/3167067501 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80037070 con 0x7fba9015c4c0 2024-05-09T06:30:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba8e59c640 1 -- 172.21.15.1:0/3167067501 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba80032440 con 0x7fba9015c4c0 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba8e59c640 1 -- 172.21.15.1:0/3167067501 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800327f0 con 0x7fba9015c4c0 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 -- 172.21.15.1:0/3167067501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 msgr2=0x7fba9006dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 --2- 172.21.15.1:0/3167067501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 0x7fba9006dd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba800097a0 tx=0x7fba8002a180 comp rx=0 tx=0).stop 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 -- 172.21.15.1:0/3167067501 shutdown_connections 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 --2- 172.21.15.1:0/3167067501 >> v2:172.21.15.6:3300/0 conn(0x7fba900b4de0 0x7fba9015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 --2- 172.21.15.1:0/3167067501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba9015c4c0 0x7fba9006dd00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 -- 172.21.15.1:0/3167067501 >> 172.21.15.1:0/3167067501 conn(0x7fba900cc4d0 msgr2=0x7fba900cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 -- 172.21.15.1:0/3167067501 shutdown_connections 2024-05-09T06:30:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.705+0000 7fba95130640 1 -- 172.21.15.1:0/3167067501 wait complete. 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.706+0000 7fba95130640 1 Processor -- start 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 -- start start 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 0x7fba90154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba900b5260 0x7fba900a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba900b2dc0 con 0x7fba9015c4c0 2024-05-09T06:30:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba900b2c40 con 0x7fba900b5260 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba95130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fba9015cde0 con 0x7fba900b4de0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba8f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.707+0000 7fba8ed9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 0x7fba90154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba87fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba900b5260 0x7fba900a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50574/0 (socket says 172.21.15.1:50574) 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 -- 172.21.15.1:0/1992970509 learned_addr learned my addr 172.21.15.1:0/1992970509 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 -- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 msgr2=0x7fba90154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 0x7fba90154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 -- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba900b5260 msgr2=0x7fba900a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba900b5260 0x7fba900a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 -- 172.21.15.1:0/1992970509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba90159c10 con 0x7fba9015c4c0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8ed9d640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 0x7fba90154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba8f59e640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fba8800cb40 tx=0x7fba8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88017070 con 0x7fba9015c4c0 2024-05-09T06:30:34.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8800aca0 con 0x7fba9015c4c0 2024-05-09T06:30:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88012620 con 0x7fba9015c4c0 2024-05-09T06:30:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba90159ef0 con 0x7fba9015c4c0 2024-05-09T06:30:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.708+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9015a4b0 con 0x7fba9015c4c0 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.709+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba900752b0 con 0x7fba9015c4c0 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.710+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba8801a070 con 0x7fba9015c4c0 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.710+0000 7fba877fe640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 0x7fba7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.711+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6001+0+0 (secure 0 0 0) 0x7fba8808ad50 con 0x7fba9015c4c0 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.711+0000 7fba8ed9d640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 0x7fba7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.711+0000 7fba8ed9d640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 0x7fba7006e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.715+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba8808e030 con 0x7fba9015c4c0 2024-05-09T06:30:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:34.936+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fba9006cfc0 con 0x7fba9015c4c0 2024-05-09T06:30:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.578+0000 7fba877fe640 1 -- 172.21.15.1:0/1992970509 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) ==== 173+0+0 (secure 0 0 0) 0x7fba88058f80 con 0x7fba9015c4c0 2024-05-09T06:30:35.560 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 9 and data pool 10 2024-05-09T06:30:35.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.586+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 msgr2=0x7fba7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.586+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 0x7fba7006e320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).stop 2024-05-09T06:30:35.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 msgr2=0x7fba900abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fba8800cb40 tx=0x7fba8800a060 comp rx=0 tx=0).stop 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 shutdown_connections 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba7006be60 0x7fba7006e320 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).stop 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9015c4c0 0x7fba900abac0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba900b5260 0x7fba900a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 --2- 172.21.15.1:0/1992970509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba900b4de0 0x7fba90154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.587+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 >> 172.21.15.1:0/1992970509 conn(0x7fba900cc4d0 msgr2=0x7fba900691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:35.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.588+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 shutdown_connections 2024-05-09T06:30:35.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.588+0000 7fba95130640 1 -- 172.21.15.1:0/1992970509 wait complete. 2024-05-09T06:30:35.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:35.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.717+0000 7f2b4eab3640 1 Processor -- start 2024-05-09T06:30:35.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.717+0000 7f2b4eab3640 1 -- start start 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4eab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b4815c680 0x7f2b480afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4eab3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b480b0170 0x7f2b481601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4eab3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4815cc40 con 0x7f2b481565e0 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4eab3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4815fd30 con 0x7f2b480b0170 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4eab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4815cf20 con 0x7f2b4815c680 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b480b0170 0x7f2b481601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b480b0170 0x7f2b481601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50580/0 (socket says 172.21.15.1:50580) 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 -- 172.21.15.1:0/3492779493 learned_addr learned my addr 172.21.15.1:0/3492779493 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 -- 172.21.15.1:0/3492779493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b4815c680 msgr2=0x7f2b480afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b47fff640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b4815c680 0x7f2b480afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b4815c680 0x7f2b480afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 -- 172.21.15.1:0/3492779493 >> v1:172.21.15.6:6789/0 conn(0x7f2b481565e0 legacy=0x7f2b48074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:35.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.718+0000 7f2b4d050640 1 -- 172.21.15.1:0/3492779493 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4815d200 con 0x7f2b480b0170 2024-05-09T06:30:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.719+0000 7f2b4d050640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b481601c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009870 tx=0x7f2b4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3744cc9c2f8946 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.719+0000 7f2b477fe640 1 -- 172.21.15.1:0/3492779493 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40037070 con 0x7f2b480b0170 2024-05-09T06:30:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.719+0000 7f2b477fe640 1 -- 172.21.15.1:0/3492779493 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40032440 con 0x7f2b480b0170 2024-05-09T06:30:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.719+0000 7f2b477fe640 1 -- 172.21.15.1:0/3492779493 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40032810 con 0x7f2b480b0170 2024-05-09T06:30:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 -- 172.21.15.1:0/3492779493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 msgr2=0x7f2b481601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b481601c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009870 tx=0x7f2b4002a210 comp rx=0 tx=0).stop 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 -- 172.21.15.1:0/3492779493 shutdown_connections 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b481601c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/3492779493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b4815c680 0x7f2b480afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 -- 172.21.15.1:0/3492779493 >> 172.21.15.1:0/3492779493 conn(0x7f2b480b1fb0 msgr2=0x7f2b480b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 -- 172.21.15.1:0/3492779493 shutdown_connections 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.720+0000 7f2b4eab3640 1 -- 172.21.15.1:0/3492779493 wait complete. 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.721+0000 7f2b4eab3640 1 Processor -- start 2024-05-09T06:30:35.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.721+0000 7f2b4eab3640 1 -- start start 2024-05-09T06:30:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 0x7f2b480ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 0x7f2b481e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48160700 con 0x7f2b480b0170 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2b481569c0 con 0x7f2b4815c680 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4eab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2b48160880 con 0x7f2b481565e0 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50594/0 (socket says 172.21.15.1:50594) 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 -- 172.21.15.1:0/4075180594 learned_addr learned my addr 172.21.15.1:0/4075180594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4d050640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 0x7f2b481e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b47fff640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 0x7f2b480ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 -- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 msgr2=0x7f2b480ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 0x7f2b480ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 -- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 msgr2=0x7f2b481e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 0x7f2b481e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 -- 172.21.15.1:0/4075180594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b480cc910 con 0x7f2b480b0170 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4d050640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 0x7f2b481e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b47fff640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 0x7f2b480ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.722+0000 7f2b4c84f640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c430 tx=0x7f2b3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.723+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30011790 con 0x7f2b480b0170 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.723+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b30017070 con 0x7f2b480b0170 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.723+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b480b30a0 con 0x7f2b480b0170 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.723+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30011bb0 con 0x7f2b480b0170 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.723+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b480b59b0 con 0x7f2b480b0170 2024-05-09T06:30:35.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.724+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b480752b0 con 0x7f2b480b0170 2024-05-09T06:30:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.724+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b3001c460 con 0x7f2b480b0170 2024-05-09T06:30:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.725+0000 7f2b457fa640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.725+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f2b3008aec0 con 0x7f2b480b0170 2024-05-09T06:30:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.725+0000 7f2b47fff640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.725+0000 7f2b47fff640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 0x7f2b2806e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b480b5470 tx=0x7f2b38007400 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:35.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.729+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b300596e0 con 0x7f2b480b0170 2024-05-09T06:30:35.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.915+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b480ccae0 con 0x7f2b480b0170 2024-05-09T06:30:35.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.916+0000 7f2b457fa640 1 -- 172.21.15.1:0/4075180594 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16765 (secure 0 0 0) 0x7f2b30058fe0 con 0x7f2b480b0170 2024-05-09T06:30:35.898 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:35.898 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":63,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:34.938419+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:35.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 msgr2=0x7f2b2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 0x7f2b2806e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b480b5470 tx=0x7f2b38007400 comp rx=0 tx=0).stop 2024-05-09T06:30:35.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 msgr2=0x7f2b480c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c430 tx=0x7f2b3000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 shutdown_connections 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2b4815c680 0x7f2b481e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2b481565e0 0x7f2b480ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 --2- 172.21.15.1:0/4075180594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b480b0170 0x7f2b480c6ee0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 >> 172.21.15.1:0/4075180594 conn(0x7f2b480b1fb0 msgr2=0x7f2b480b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 shutdown_connections 2024-05-09T06:30:35.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:35.918+0000 7f2b4eab3640 1 -- 172.21.15.1:0/4075180594 wait complete. 2024-05-09T06:30:35.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:36.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.033+0000 7fe60644d640 1 Processor -- start 2024-05-09T06:30:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.033+0000 7fe60644d640 1 -- start start 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe60644d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 0x7fe600071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe60644d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6000cbb60 0x7fe60015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe60644d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6000d04a0 con 0x7fe60015fd70 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe60644d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe6000cbfe0 con 0x7fe6000cbb60 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe60644d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6000722c0 con 0x7fe6000b0080 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6000cbb60 0x7fe60015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6000cbb60 0x7fe60015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50600/0 (socket says 172.21.15.1:50600) 2024-05-09T06:30:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 -- 172.21.15.1:0/2185339024 learned_addr learned my addr 172.21.15.1:0/2185339024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5fffff640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 0x7fe600071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 -- 172.21.15.1:0/2185339024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 msgr2=0x7fe600071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 0x7fe600071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 -- 172.21.15.1:0/2185339024 >> v1:172.21.15.6:6789/0 conn(0x7fe60015fd70 legacy=0x7fe60015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5ff7fe640 1 -- 172.21.15.1:0/2185339024 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6000725a0 con 0x7fe6000cbb60 2024-05-09T06:30:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.034+0000 7fe5fffff640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 0x7fe600071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.035+0000 7fe5ff7fe640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000cbb60 0x7fe60015f830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009870 tx=0x7fe5f402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612b09309c26256f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.035+0000 7fe5feffd640 1 -- 172.21.15.1:0/2185339024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4037070 con 0x7fe6000cbb60 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.035+0000 7fe5feffd640 1 -- 172.21.15.1:0/2185339024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f4032440 con 0x7fe6000cbb60 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.035+0000 7fe5feffd640 1 -- 172.21.15.1:0/2185339024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4032810 con 0x7fe6000cbb60 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 -- 172.21.15.1:0/2185339024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000cbb60 msgr2=0x7fe60015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000cbb60 0x7fe60015f830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009870 tx=0x7fe5f402a250 comp rx=0 tx=0).stop 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 -- 172.21.15.1:0/2185339024 shutdown_connections 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000cbb60 0x7fe60015f830 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 --2- 172.21.15.1:0/2185339024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6000b0080 0x7fe600071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 -- 172.21.15.1:0/2185339024 >> 172.21.15.1:0/2185339024 conn(0x7fe6000d0b50 msgr2=0x7fe6000d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 -- 172.21.15.1:0/2185339024 shutdown_connections 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.036+0000 7fe60644d640 1 -- 172.21.15.1:0/2185339024 wait complete. 2024-05-09T06:30:36.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.037+0000 7fe60644d640 1 Processor -- start 2024-05-09T06:30:36.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.037+0000 7fe60644d640 1 -- start start 2024-05-09T06:30:36.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6000cbb60 0x7fe6000c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe60015fd70 0x7fe600158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe60015e7c0 con 0x7fe6000b0080 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe600072b60 con 0x7fe6000cbb60 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe60015f3f0 con 0x7fe60015fd70 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50616/0 (socket says 172.21.15.1:50616) 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 -- 172.21.15.1:0/3820043197 learned_addr learned my addr 172.21.15.1:0/3820043197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 -- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe60015fd70 msgr2=0x7fe600158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe60015fd70 0x7fe600158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 -- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6000cbb60 msgr2=0x7fe6000c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6000cbb60 0x7fe6000c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 -- 172.21.15.1:0/3820043197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe600159160 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fffff640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d7d0 tx=0x7fe5f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0004030 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f00041d0 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6000c8390 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00096a0 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.038+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6000c8950 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.039+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6000752b0 con 0x7fe6000b0080 2024-05-09T06:30:36.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.040+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5f0009840 con 0x7fe6000b0080 2024-05-09T06:30:36.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.040+0000 7fe5fcff9640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 0x7fe5ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.041+0000 7fe5ff7fe640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 0x7fe5ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.041+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fe5f008b280 con 0x7fe6000b0080 2024-05-09T06:30:36.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.041+0000 7fe5ff7fe640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 0x7fe5ec06e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4034000 tx=0x7fe5f402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.045+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5f0059aa0 con 0x7fe6000b0080 2024-05-09T06:30:36.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.227+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6000cec80 con 0x7fe6000b0080 2024-05-09T06:30:36.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.229+0000 7fe5fcff9640 1 -- 172.21.15.1:0/3820043197 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16765 (secure 0 0 0) 0x7fe5f00593a0 con 0x7fe6000b0080 2024-05-09T06:30:36.210 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:36.211 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":63,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:34.938419+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:36.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 msgr2=0x7fe5ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 0x7fe5ec06e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4034000 tx=0x7fe5f402c000 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 msgr2=0x7fe6000b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d7d0 tx=0x7fe5f000dc90 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 shutdown_connections 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe5ec06be30 0x7fe5ec06e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe60015fd70 0x7fe600158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6000cbb60 0x7fe6000c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 --2- 172.21.15.1:0/3820043197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6000b0080 0x7fe6000b52d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 >> 172.21.15.1:0/3820043197 conn(0x7fe6000d0b50 msgr2=0x7fe6000b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 shutdown_connections 2024-05-09T06:30:36.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.231+0000 7fe60644d640 1 -- 172.21.15.1:0/3820043197 wait complete. 2024-05-09T06:30:36.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:30:36.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.349+0000 7fae7aa3f640 1 Processor -- start 2024-05-09T06:30:36.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.349+0000 7fae7aa3f640 1 -- start start 2024-05-09T06:30:36.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae7aa3f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae740d1c70 0x7fae7415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae7aa3f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae74161fe0 con 0x7fae740d45f0 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae7aa3f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae741622c0 con 0x7fae7415e190 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae7aa3f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fae7415f580 con 0x7fae740d1c70 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae740d1c70 0x7fae7415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fae740d1c70 0x7fae7415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50628/0 (socket says 172.21.15.1:50628) 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae78fdc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fae740d45f0 0x7fae740b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38320/0 (socket says 172.21.15.1:38320) 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 -- 172.21.15.1:0/1726802832 learned_addr learned my addr 172.21.15.1:0/1726802832 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 -- 172.21.15.1:0/1726802832 >> v1:172.21.15.6:6789/0 conn(0x7fae7415e190 legacy=0x7fae740d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 -- 172.21.15.1:0/1726802832 >> v1:172.21.15.1:6789/0 conn(0x7fae740d45f0 legacy=0x7fae740b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae6bfff640 1 -- 172.21.15.1:0/1726802832 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae741625a0 con 0x7fae740d1c70 2024-05-09T06:30:36.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.350+0000 7fae737fe640 1 -- 172.21.15.1:0/1726802832 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439062394 0 0) 0x7fae74161fe0 con 0x7fae740d45f0 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.351+0000 7fae6bfff640 1 --2- 172.21.15.1:0/1726802832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae740d1c70 0x7fae7415f780 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedb12e4e60209de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.351+0000 7fae737fe640 1 -- 172.21.15.1:0/1726802832 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037070 con 0x7fae740d1c70 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.351+0000 7fae737fe640 1 -- 172.21.15.1:0/1726802832 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c032440 con 0x7fae740d1c70 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.351+0000 7fae737fe640 1 -- 172.21.15.1:0/1726802832 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c032810 con 0x7fae740d1c70 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 -- 172.21.15.1:0/1726802832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae740d1c70 msgr2=0x7fae7415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/1726802832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae740d1c70 0x7fae7415f780 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a2a0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 -- 172.21.15.1:0/1726802832 shutdown_connections 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/1726802832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae740d1c70 0x7fae7415f780 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 -- 172.21.15.1:0/1726802832 >> 172.21.15.1:0/1726802832 conn(0x7fae74073340 msgr2=0x7fae7409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 -- 172.21.15.1:0/1726802832 shutdown_connections 2024-05-09T06:30:36.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.352+0000 7fae7aa3f640 1 -- 172.21.15.1:0/1726802832 wait complete. 2024-05-09T06:30:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.353+0000 7fae7aa3f640 1 Processor -- start 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.353+0000 7fae7aa3f640 1 -- start start 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae740d1c70 0x7fae74159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 0x7fae740cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae740b56d0 con 0x7fae7415e190 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fae74161c60 con 0x7fae740d45f0 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae7aa3f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fae740b5850 con 0x7fae740d1c70 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50634/0 (socket says 172.21.15.1:50634) 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 -- 172.21.15.1:0/623424817 learned_addr learned my addr 172.21.15.1:0/623424817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae73fff640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae740d1c70 0x7fae74159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae6bfff640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 0x7fae740cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 -- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae740d1c70 msgr2=0x7fae74159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae740d1c70 0x7fae74159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 -- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 msgr2=0x7fae740cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 0x7fae740cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 -- 172.21.15.1:0/623424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae740cc8a0 con 0x7fae7415e190 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae6bfff640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 0x7fae740cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.354+0000 7fae78fdc640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c7b0 tx=0x7fae6c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.355+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c018070 con 0x7fae7415e190 2024-05-09T06:30:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.355+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c00a640 con 0x7fae7415e190 2024-05-09T06:30:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.355+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae740ccb80 con 0x7fae7415e190 2024-05-09T06:30:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.355+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00aa70 con 0x7fae7415e190 2024-05-09T06:30:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.355+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae740cd140 con 0x7fae7415e190 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.356+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae740752b0 con 0x7fae7415e190 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.356+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae6c01b070 con 0x7fae7415e190 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.357+0000 7fae717fa640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 0x7fae5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.357+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fae6c08a650 con 0x7fae7415e190 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.357+0000 7fae73fff640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 0x7fae5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.357+0000 7fae73fff640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 0x7fae5406e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae64006fd0 tx=0x7fae64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.361+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fae6c058e60 con 0x7fae7415e190 2024-05-09T06:30:36.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:36.586+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fae7408e1c0 con 0x7fae7415e190 2024-05-09T06:30:37.540 INFO:tasks.ceph.mds.b.smithi006.stderr:2024-05-09T06:30:37.559+0000 7ff0d1728640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:30:37.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.589+0000 7fae717fa640 1 -- 172.21.15.1:0/623424817 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7fae6c058760 con 0x7fae7415e190 2024-05-09T06:30:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 msgr2=0x7fae5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 0x7fae5406e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae64006fd0 tx=0x7fae64008040 comp rx=0 tx=0).stop 2024-05-09T06:30:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 msgr2=0x7fae740c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c7b0 tx=0x7fae6c00cc70 comp rx=0 tx=0).stop 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 shutdown_connections 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fae5406be30 0x7fae5406e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae7415e190 0x7fae740c8d00 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae740d45f0 0x7fae740cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 --2- 172.21.15.1:0/623424817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fae740d1c70 0x7fae74159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 >> 172.21.15.1:0/623424817 conn(0x7fae74073340 msgr2=0x7fae740d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:37.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 shutdown_connections 2024-05-09T06:30:37.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.592+0000 7fae7aa3f640 1 -- 172.21.15.1:0/623424817 wait complete. 2024-05-09T06:30:37.587 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:30:37.587 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:30:37.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.720+0000 7f09c65ce640 1 Processor -- start 2024-05-09T06:30:37.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.720+0000 7f09c65ce640 1 -- start start 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.720+0000 7f09c65ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09c015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c65ce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f09c00d1c70 0x7f09c00d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c65ce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c0161fe0 con 0x7f09c00b56d0 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c65ce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f09c015f580 con 0x7f09c00d1c70 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c65ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09c01622c0 con 0x7f09c015e190 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c4b6b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f09c00b56d0 0x7f09c00b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59106/0 (socket says 172.21.15.1:59106) 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09c4b6b640 1 -- 172.21.15.1:0/3164495210 learned_addr learned my addr 172.21.15.1:0/3164495210 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:37.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09b7fff640 1 --2- 172.21.15.1:0/3164495210 >> v2:172.21.15.6:3300/0 conn(0x7f09c00d1c70 0x7f09c00d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09c015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869592946 0 0) 0x7f09c0161fe0 con 0x7f09c00b56d0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09ac003520 con 0x7f09c00b56d0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 346866250 0 0) 0x7f09ac003520 con 0x7f09c00b56d0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 msgr2=0x7f09c015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09c015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 >> v2:172.21.15.6:3300/0 conn(0x7f09c00d1c70 msgr2=0x7f09c00d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3164495210 >> v2:172.21.15.6:3300/0 conn(0x7f09c00d1c70 0x7f09c00d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01625a0 con 0x7f09c00b56d0 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09b7fff640 1 --2- 172.21.15.1:0/3164495210 >> v2:172.21.15.6:3300/0 conn(0x7f09c00d1c70 0x7f09c00d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f09b80019f0 con 0x7f09c00b56d0 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09c015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 >> v1:172.21.15.1:6789/0 conn(0x7f09c00b56d0 legacy=0x7f09c00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09ac0039b0 0x7f09ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09ac004d40 0x7f09ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01622c0 con 0x7f09ac0039b0 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac00b9c0 con 0x7f09ac004d40 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.721+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac00bb40 con 0x7f09c015e190 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09ac0039b0 0x7f09ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09c4b6b640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09ac004d40 0x7f09ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09b7fff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 -- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 msgr2=0x7f09ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09c015e190 0x7f09ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 -- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09ac004d40 msgr2=0x7f09ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:37.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09ac004d40 0x7f09ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09bffff640 1 -- 172.21.15.1:0/3164495210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac003520 con 0x7f09ac0039b0 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.722+0000 7f09c4b6b640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09ac004d40 0x7f09ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09bffff640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09ac0039b0 0x7f09ac003dd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009c90 tx=0x7f09ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a050f2a1c6e51c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0036070 con 0x7f09ac0039b0 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b0031400 con 0x7f09ac0039b0 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09bf7fe640 1 -- 172.21.15.1:0/3164495210 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0009860 con 0x7f09ac0039b0 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 -- 172.21.15.1:0/3164495210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09ac0039b0 msgr2=0x7f09ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 --2- 172.21.15.1:0/3164495210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09ac0039b0 0x7f09ac003dd0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009c90 tx=0x7f09ac005830 comp rx=0 tx=0).stop 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c4b6b640 1 -- 172.21.15.1:0/3164495210 reap_dead start 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 -- 172.21.15.1:0/3164495210 shutdown_connections 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 -- 172.21.15.1:0/3164495210 >> 172.21.15.1:0/3164495210 conn(0x7f09c0073340 msgr2=0x7f09c009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 -- 172.21.15.1:0/3164495210 shutdown_connections 2024-05-09T06:30:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.723+0000 7f09c65ce640 1 -- 172.21.15.1:0/3164495210 wait complete. 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 Processor -- start 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 -- start start 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09ac0039b0 0x7f09c00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09c01f39d0 0x7f09c015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09c015f780 con 0x7f09c00d12b0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0161c60 con 0x7f09c01f39d0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09c65ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09c015f900 con 0x7f09ac0039b0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33264/0 (socket says 172.21.15.1:33264) 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09b7fff640 1 -- 172.21.15.1:0/1009045381 learned_addr learned my addr 172.21.15.1:0/1009045381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.725+0000 7f09bffff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09ac0039b0 0x7f09c00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09c4b6b640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09c01f39d0 0x7f09c015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 -- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09ac0039b0 msgr2=0x7f09c00b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09ac0039b0 0x7f09c00b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 -- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09c01f39d0 msgr2=0x7f09c015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09c01f39d0 0x7f09c015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 -- 172.21.15.1:0/1009045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01f4340 con 0x7f09c00d12b0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09b7fff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f09a8011900 tx=0x7f09a8011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800f9c0 con 0x7f09c00d12b0 2024-05-09T06:30:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a800fb60 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c01f4620 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8015460 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.726+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c00cf7b0 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.727+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c00752b0 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.728+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09a8015600 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.728+0000 7f09bd7fa640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 0x7f09a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.728+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f09a808b930 con 0x7f09c00d12b0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.728+0000 7f09bffff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 0x7f09a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.729+0000 7f09bffff640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 0x7f09a006e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f09b0033000 tx=0x7f09b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.733+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09a8019030 con 0x7f09c00d12b0 2024-05-09T06:30:37.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:37.961+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f09c008e1c0 con 0x7f09c00d12b0 2024-05-09T06:30:38.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.590+0000 7f09bd7fa640 1 -- 172.21.15.1:0/1009045381 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f09a805a150 con 0x7f09c00d12b0 2024-05-09T06:30:38.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 msgr2=0x7f09a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 0x7f09a006e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f09b0033000 tx=0x7f09b0008030 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 msgr2=0x7f09c00cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f09a8011900 tx=0x7f09a8011d00 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 shutdown_connections 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09a006be30 0x7f09a006e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09c01f39d0 0x7f09c015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09c00d12b0 0x7f09c00cd7e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 --2- 172.21.15.1:0/1009045381 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09ac0039b0 0x7f09c00b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.592+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 >> 172.21.15.1:0/1009045381 conn(0x7f09c0073340 msgr2=0x7f09c00b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.593+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 shutdown_connections 2024-05-09T06:30:38.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.593+0000 7f09c65ce640 1 -- 172.21.15.1:0/1009045381 wait complete. 2024-05-09T06:30:38.587 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:30:38.587 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.713+0000 7f3b517dc640 1 Processor -- start 2024-05-09T06:30:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.713+0000 7f3b517dc640 1 -- start start 2024-05-09T06:30:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b517dc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b4c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b517dc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c06eb40 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b517dc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c06ee20 con 0x7f3b4c0b4f70 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b517dc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c06d8e0 con 0x7f3b4c1454b0 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b43fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3b4c0b2a60 0x7f3b4c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59120/0 (socket says 172.21.15.1:59120) 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b43fff640 1 -- 172.21.15.1:0/1622015911 learned_addr learned my addr 172.21.15.1:0/1622015911 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.714+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b4c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472878844 0 0) 0x7f3b4c06eb40 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b38003520 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678674802 0 0) 0x7f3b4c06ee20 con 0x7f3b4c0b4f70 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b4c06eb40 con 0x7f3b4c0b4f70 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1735386913 0 0) 0x7f3b38003520 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 msgr2=0x7f3b4c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b4c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 >> v1:172.21.15.6:6789/0 conn(0x7f3b4c0b4f70 legacy=0x7f3b4c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c06dbc0 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3b34001670 con 0x7f3b4c0b2a60 2024-05-09T06:30:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 >> v1:172.21.15.1:6789/0 conn(0x7f3b4c0b2a60 legacy=0x7f3b4c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38003b70 0x7f3b38003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b38004800 0x7f3b38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c06d8e0 con 0x7f3b38004800 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3800ba70 con 0x7f3b38003b70 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3800bc40 con 0x7f3b4c1454b0 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b38004800 0x7f3b38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38003b70 0x7f3b38003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4b7fe640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 -- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 msgr2=0x7f3b380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 -- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38003b70 msgr2=0x7f3b38003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38003b70 0x7f3b38003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b43fff640 1 -- 172.21.15.1:0/1622015911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38003520 con 0x7f3b38004800 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.715+0000 7f3b4b7fe640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1454b0 0x7f3b380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.716+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b38004800 0x7f3b38004c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c190 tx=0x7f3b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87150b6550ec504c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.716+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34007cc0 con 0x7f3b38004800 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.716+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34005af0 con 0x7f3b38004800 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.716+0000 7f3b4a7fc640 1 -- 172.21.15.1:0/1622015911 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34007070 con 0x7f3b38004800 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 -- 172.21.15.1:0/1622015911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b38004800 msgr2=0x7f3b38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1622015911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b38004800 0x7f3b38004c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c190 tx=0x7f3b38005830 comp rx=0 tx=0).stop 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b4b7fe640 1 -- 172.21.15.1:0/1622015911 reap_dead start 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 -- 172.21.15.1:0/1622015911 shutdown_connections 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 -- 172.21.15.1:0/1622015911 >> 172.21.15.1:0/1622015911 conn(0x7f3b4c0cc4b0 msgr2=0x7f3b4c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 -- 172.21.15.1:0/1622015911 shutdown_connections 2024-05-09T06:30:38.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.717+0000 7f3b517dc640 1 -- 172.21.15.1:0/1622015911 wait complete. 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.718+0000 7f3b517dc640 1 Processor -- start 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.718+0000 7f3b517dc640 1 -- start start 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 0x7f3b4c1f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 0x7f3b4c0ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c06e260 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c06e0e0 con 0x7f3b38004800 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b517dc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c0b4320 con 0x7f3b4c1e2ad0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33288/0 (socket says 172.21.15.1:33288) 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b43fff640 1 -- 172.21.15.1:0/1600840660 learned_addr learned my addr 172.21.15.1:0/1600840660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 0x7f3b4c1f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.719+0000 7f3b4b7fe640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 0x7f3b4c0ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 -- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 msgr2=0x7f3b4c0ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 0x7f3b4c0ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 -- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 msgr2=0x7f3b4c1f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 0x7f3b4c1f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 -- 172.21.15.1:0/1600840660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c0ae6e0 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 0x7f3b4c1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b4b7fe640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 0x7f3b4c0ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b43fff640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c190 tx=0x7f3b4c15c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34001670 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34038040 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1f6550 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34007290 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.720+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c0adc00 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.721+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c0752b0 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.722+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b34007430 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.722+0000 7f3b437fe640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 0x7f3b2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.722+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f3b340ab0e0 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.722+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 0x7f3b2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.722+0000 7f3b4affd640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 0x7f3b2c06e320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0099e0 tx=0x7f3b3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.726+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b3404b020 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.949+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b4c072fa0 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.949+0000 7f3b437fe640 1 -- 172.21.15.1:0/1600840660 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5167 (secure 0 0 0) 0x7f3b34079990 con 0x7f3b4c0aefd0 2024-05-09T06:30:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:38.931 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:30:37.962593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:30:38.931 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-05-09T06:30:38.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 msgr2=0x7f3b2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 0x7f3b2c06e320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0099e0 tx=0x7f3b3c008040 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 msgr2=0x7f3b4c1e2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400c190 tx=0x7f3b4c15c370 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 shutdown_connections 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2c06be60 0x7f3b2c06e320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b4c1e2ad0 0x7f3b4c0ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b4c0aefd0 0x7f3b4c1e2590 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 --2- 172.21.15.1:0/1600840660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b38004800 0x7f3b4c1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 >> 172.21.15.1:0/1600840660 conn(0x7f3b4c0cc4b0 msgr2=0x7f3b4c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 shutdown_connections 2024-05-09T06:30:38.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:38.952+0000 7f3b517dc640 1 -- 172.21.15.1:0/1600840660 wait complete. 2024-05-09T06:30:38.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:30:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.067+0000 7f8541b09640 1 Processor -- start 2024-05-09T06:30:39.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.067+0000 7f8541b09640 1 -- start start 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f8541b09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f853c1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f8541b09640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c071c20 con 0x7f853c0b2a60 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f8541b09640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c06eb40 con 0x7f853c1454b0 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f8541b09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f853c06ee20 con 0x7f853c164ce0 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f853affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f853c0b2a60 0x7f853c0b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59132/0 (socket says 172.21.15.1:59132) 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f853affd640 1 -- 172.21.15.1:0/1912805310 learned_addr learned my addr 172.21.15.1:0/1912805310 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f853bfff640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f853c1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123094441 0 0) 0x7f853c071c20 con 0x7f853c0b2a60 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.068+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8524003520 con 0x7f853c0b2a60 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519682131 0 0) 0x7f853c06eb40 con 0x7f853c1454b0 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f853c071c20 con 0x7f853c1454b0 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1417430686 0 0) 0x7f8524003520 con 0x7f853c0b2a60 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 msgr2=0x7f853c1670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f853c1670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 >> v1:172.21.15.6:6789/0 conn(0x7f853c1454b0 legacy=0x7f853c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c06d8e0 con 0x7f853c0b2a60 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8530001670 con 0x7f853c0b2a60 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 >> v1:172.21.15.1:6789/0 conn(0x7f853c0b2a60 legacy=0x7f853c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f8524003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f8524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8524004d40 0x7f85240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f853c06ee20 con 0x7f8524003b70 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f852400ba70 con 0x7f8524004d40 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f852400bc20 con 0x7f853c164ce0 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f8524003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853affd640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f8524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853bfff640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8524004d40 0x7f85240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 -- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 msgr2=0x7f8524004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c164ce0 0x7f8524004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 -- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8524004d40 msgr2=0x7f85240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8524004d40 0x7f85240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853b7fe640 1 -- 172.21.15.1:0/1912805310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8524003520 con 0x7f8524003b70 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.069+0000 7f853bfff640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8524004d40 0x7f85240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.070+0000 7f853b7fe640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f8524003f70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f852c0018d0 tx=0x7f8524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8971e5bd03af731 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.070+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0014f0 con 0x7f8524003b70 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.070+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0052d0 con 0x7f8524003b70 2024-05-09T06:30:39.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.070+0000 7f853a7fc640 1 -- 172.21.15.1:0/1912805310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c02aca0 con 0x7f8524003b70 2024-05-09T06:30:39.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 -- 172.21.15.1:0/1912805310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 msgr2=0x7f8524003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 --2- 172.21.15.1:0/1912805310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f8524003f70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f852c0018d0 tx=0x7f8524005830 comp rx=0 tx=0).stop 2024-05-09T06:30:39.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f853bfff640 1 -- 172.21.15.1:0/1912805310 reap_dead start 2024-05-09T06:30:39.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 -- 172.21.15.1:0/1912805310 shutdown_connections 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 -- 172.21.15.1:0/1912805310 >> 172.21.15.1:0/1912805310 conn(0x7f853c0cc4b0 msgr2=0x7f853c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 -- 172.21.15.1:0/1912805310 shutdown_connections 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.071+0000 7f8541b09640 1 -- 172.21.15.1:0/1912805310 wait complete. 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.072+0000 7f8541b09640 1 Processor -- start 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.072+0000 7f8541b09640 1 -- start start 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f853c15b860 0x7f853c13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 0x7f853c144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0b5140 con 0x7f8524003b70 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f853c145890 con 0x7f853c15b860 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f8541b09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f853c06e0e0 con 0x7f853c13b2b0 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33310/0 (socket says 172.21.15.1:33310) 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 -- 172.21.15.1:0/2118830389 learned_addr learned my addr 172.21.15.1:0/2118830389 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853affd640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f853c15b860 0x7f853c13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853bfff640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 0x7f853c144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 -- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 msgr2=0x7f853c144070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 0x7f853c144070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 -- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f853c15b860 msgr2=0x7f853c13ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f853c15b860 0x7f853c13ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 -- 172.21.15.1:0/2118830389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c13b760 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853bfff640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 0x7f853c144070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.073+0000 7f853b7fe640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f852c005040 tx=0x7f852c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.074+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c03f020 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.074+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c006e50 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.074+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c13ba40 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.074+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c037040 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.074+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c13c6c0 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.075+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c13bf60 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.075+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f852c042020 con 0x7f8524003b70 2024-05-09T06:30:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.076+0000 7f8523fff640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 0x7f852806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.076+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f852c0aac70 con 0x7f8524003b70 2024-05-09T06:30:39.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.076+0000 7f853affd640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 0x7f852806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.076+0000 7f853affd640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 0x7f852806e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f853c15bf00 tx=0x7f8530001e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.080+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f852c049030 con 0x7f8524003b70 2024-05-09T06:30:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.263+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f853c0752b0 con 0x7f8524003b70 2024-05-09T06:30:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.264+0000 7f8523fff640 1 -- 172.21.15.1:0/2118830389 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16752 (secure 0 0 0) 0x7f852c079490 con 0x7f8524003b70 2024-05-09T06:30:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:39.246 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":64,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:30:36.576447+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 msgr2=0x7f852806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 0x7f852806e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f853c15bf00 tx=0x7f8530001e30 comp rx=0 tx=0).stop 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 msgr2=0x7f853c15ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f852c005040 tx=0x7f852c0012d0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 shutdown_connections 2024-05-09T06:30:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f852806be30 0x7f852806e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f853c13b2b0 0x7f853c144070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f853c15b860 0x7f853c13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 --2- 172.21.15.1:0/2118830389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8524003b70 0x7f853c15ab50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 >> 172.21.15.1:0/2118830389 conn(0x7f853c0cc4b0 msgr2=0x7f853c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 shutdown_connections 2024-05-09T06:30:39.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.266+0000 7f8541b09640 1 -- 172.21.15.1:0/2118830389 wait complete. 2024-05-09T06:30:39.259 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:30:39.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:30:39.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.381+0000 7fd7edeba640 1 Processor -- start 2024-05-09T06:30:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 -- start start 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7e8161950 0x7fd7e80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd7e80d4940 0x7fd7e80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e80b33c0 con 0x7fd7e80d0f80 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e80d4dd0 con 0x7fd7e8161950 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.382+0000 7fd7edeba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e80b3660 con 0x7fd7e80d4940 2024-05-09T06:30:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7e8161950 0x7fd7e80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd7e80d0f80 0x7fd7e80cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59134/0 (socket says 172.21.15.1:59134) 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e7fff640 1 -- 172.21.15.1:0/1874842736 learned_addr learned my addr 172.21.15.1:0/1874842736 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7e8161950 0x7fd7e80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33318/0 (socket says 172.21.15.1:33318) 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e6ffd640 1 --2- 172.21.15.1:0/1874842736 >> v2:172.21.15.6:3300/0 conn(0x7fd7e80d4940 0x7fd7e80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 -- 172.21.15.1:0/1874842736 >> v2:172.21.15.6:3300/0 conn(0x7fd7e80d4940 msgr2=0x7fd7e80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 --2- 172.21.15.1:0/1874842736 >> v2:172.21.15.6:3300/0 conn(0x7fd7e80d4940 0x7fd7e80d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 -- 172.21.15.1:0/1874842736 >> v1:172.21.15.1:6789/0 conn(0x7fd7e80d0f80 legacy=0x7fd7e80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e67fc640 1 -- 172.21.15.1:0/1874842736 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460743644 0 0) 0x7fd7e80b33c0 con 0x7fd7e80d0f80 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.383+0000 7fd7e77fe640 1 -- 172.21.15.1:0/1874842736 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e80b3940 con 0x7fd7e8161950 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7e77fe640 1 --2- 172.21.15.1:0/1874842736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80d2400 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806e3d0 tx=0x7fd7d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ae2a0051d87389 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7e67fc640 1 -- 172.21.15.1:0/1874842736 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8037070 con 0x7fd7e8161950 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7e67fc640 1 -- 172.21.15.1:0/1874842736 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d8032440 con 0x7fd7e8161950 2024-05-09T06:30:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7e67fc640 1 -- 172.21.15.1:0/1874842736 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8032810 con 0x7fd7e8161950 2024-05-09T06:30:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 -- 172.21.15.1:0/1874842736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 msgr2=0x7fd7e80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 --2- 172.21.15.1:0/1874842736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80d2400 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806e3d0 tx=0x7fd7d802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 -- 172.21.15.1:0/1874842736 shutdown_connections 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 --2- 172.21.15.1:0/1874842736 >> v2:172.21.15.6:3300/0 conn(0x7fd7e80d4940 0x7fd7e80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 --2- 172.21.15.1:0/1874842736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80d2400 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 -- 172.21.15.1:0/1874842736 >> 172.21.15.1:0/1874842736 conn(0x7fd7e80d3e20 msgr2=0x7fd7e80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 -- 172.21.15.1:0/1874842736 shutdown_connections 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.384+0000 7fd7edeba640 1 -- 172.21.15.1:0/1874842736 wait complete. 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.386+0000 7fd7edeba640 1 Processor -- start 2024-05-09T06:30:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.386+0000 7fd7edeba640 1 -- start start 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.386+0000 7fd7edeba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7e80d0f80 0x7fd7e80b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7e80cce50 0x7fd7e81f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e80b52d0 con 0x7fd7e8161950 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e80b5150 con 0x7fd7e80d0f80 2024-05-09T06:30:39.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e80b5450 con 0x7fd7e80cce50 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33322/0 (socket says 172.21.15.1:33322) 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 -- 172.21.15.1:0/3816998495 learned_addr learned my addr 172.21.15.1:0/3816998495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e77fe640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7e80d0f80 0x7fd7e80b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 -- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7e80cce50 msgr2=0x7fd7e81f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7e80cce50 0x7fd7e81f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 -- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7e80d0f80 msgr2=0x7fd7e80b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7e80d0f80 0x7fd7e80b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 -- 172.21.15.1:0/3816998495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e81f7360 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7e6ffd640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00e930 tx=0x7fd7dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc017070 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7dc00ca00 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00cdf0 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.387+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e81f7640 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.388+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81f7c00 con 0x7fd7e8161950 2024-05-09T06:30:39.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.388+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e80752b0 con 0x7fd7e8161950 2024-05-09T06:30:39.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.389+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7dc01a070 con 0x7fd7e8161950 2024-05-09T06:30:39.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.390+0000 7fd7cffff640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 0x7fd7d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.390+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fd7dc08b400 con 0x7fd7e8161950 2024-05-09T06:30:39.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.390+0000 7fd7e77fe640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 0x7fd7d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.390+0000 7fd7e77fe640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 0x7fd7d406e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8034000 tx=0x7fd7d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.394+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7dc059ce0 con 0x7fd7e8161950 2024-05-09T06:30:39.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.572+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7e80d21e0 con 0x7fd7d406be30 2024-05-09T06:30:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.578+0000 7fd7cffff640 1 -- 172.21.15.1:0/3816998495 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7fd7e80d21e0 con 0x7fd7d406be30 2024-05-09T06:30:39.563 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:39.564 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.581+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 msgr2=0x7fd7d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.581+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 0x7fd7d406e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8034000 tx=0x7fd7d802e000 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.581+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 msgr2=0x7fd7e80cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.581+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00e930 tx=0x7fd7dc00edf0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 shutdown_connections 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd7d406be30 0x7fd7d406e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7e80cce50 0x7fd7e81f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7e8161950 0x7fd7e80cf3d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 --2- 172.21.15.1:0/3816998495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7e80d0f80 0x7fd7e80b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 >> 172.21.15.1:0/3816998495 conn(0x7fd7e80d3e20 msgr2=0x7fd7e815edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 shutdown_connections 2024-05-09T06:30:39.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.582+0000 7fd7edeba640 1 -- 172.21.15.1:0/3816998495 wait complete. 2024-05-09T06:30:39.574 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-09T06:30:39.109311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244280,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730440,"statfs":{"total":773094113280,"available":772843970560,"internally_reserved":0,"allocated":7446528,"data_stored":3991684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550883"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815679+0000","last_change":"2024-05-09T06:30:35.589750+0000","last_active":"2024-05-09T06:30:36.815679+0000","last_peered":"2024-05-09T06:30:36.815679+0000","last_clean":"2024-05-09T06:30:36.815679+0000","last_became_active":"2024-05-09T06:30:35.589385+0000","last_became_peered":"2024-05-09T06:30:35.589385+0000","last_unstale":"2024-05-09T06:30:36.815679+0000","last_undegraded":"2024-05-09T06:30:36.815679+0000","last_fullsized":"2024-05-09T06:30:36.815679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.386784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815241+0000","last_change":"2024-05-09T06:30:35.587930+0000","last_active":"2024-05-09T06:30:36.815241+0000","last_peered":"2024-05-09T06:30:36.815241+0000","last_clean":"2024-05-09T06:30:36.815241+0000","last_became_active":"2024-05-09T06:30:35.587662+0000","last_became_peered":"2024-05-09T06:30:35.587662+0000","last_unstale":"2024-05-09T06:30:36.815241+0000","last_undegraded":"2024-05-09T06:30:36.815241+0000","last_fullsized":"2024-05-09T06:30:36.815241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:01.765676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815446+0000","last_change":"2024-05-09T06:30:35.599075+0000","last_active":"2024-05-09T06:30:36.815446+0000","last_peered":"2024-05-09T06:30:36.815446+0000","last_clean":"2024-05-09T06:30:36.815446+0000","last_became_active":"2024-05-09T06:30:35.598755+0000","last_became_peered":"2024-05-09T06:30:35.598755+0000","last_unstale":"2024-05-09T06:30:36.815446+0000","last_undegraded":"2024-05-09T06:30:36.815446+0000","last_fullsized":"2024-05-09T06:30:36.815446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:53.676819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583013+0000","last_change":"2024-05-09T06:30:35.594930+0000","last_active":"2024-05-09T06:30:36.583013+0000","last_peered":"2024-05-09T06:30:36.583013+0000","last_clean":"2024-05-09T06:30:36.583013+0000","last_became_active":"2024-05-09T06:30:35.594589+0000","last_became_peered":"2024-05-09T06:30:35.594589+0000","last_unstale":"2024-05-09T06:30:36.583013+0000","last_undegraded":"2024-05-09T06:30:36.583013+0000","last_fullsized":"2024-05-09T06:30:36.583013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:32.975202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733561+0000","last_change":"2024-05-09T06:30:35.589147+0000","last_active":"2024-05-09T06:30:36.733561+0000","last_peered":"2024-05-09T06:30:36.733561+0000","last_clean":"2024-05-09T06:30:36.733561+0000","last_became_active":"2024-05-09T06:30:35.588807+0000","last_became_peered":"2024-05-09T06:30:35.588807+0000","last_unstale":"2024-05-09T06:30:36.733561+0000","last_undegraded":"2024-05-09T06:30:36.733561+0000","last_fullsized":"2024-05-09T06:30:36.733561+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:56.681261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814953+0000","last_change":"2024-05-09T06:30:35.602433+0000","last_active":"2024-05-09T06:30:36.814953+0000","last_peered":"2024-05-09T06:30:36.814953+0000","last_clean":"2024-05-09T06:30:36.814953+0000","last_became_active":"2024-05-09T06:30:35.602174+0000","last_became_peered":"2024-05-09T06:30:35.602174+0000","last_unstale":"2024-05-09T06:30:36.814953+0000","last_undegraded":"2024-05-09T06:30:36.814953+0000","last_fullsized":"2024-05-09T06:30:36.814953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:42.173195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589969+0000","last_change":"2024-05-09T06:30:35.601000+0000","last_active":"2024-05-09T06:30:36.589969+0000","last_peered":"2024-05-09T06:30:36.589969+0000","last_clean":"2024-05-09T06:30:36.589969+0000","last_became_active":"2024-05-09T06:30:35.600712+0000","last_became_peered":"2024-05-09T06:30:35.600712+0000","last_unstale":"2024-05-09T06:30:36.589969+0000","last_undegraded":"2024-05-09T06:30:36.589969+0000","last_fullsized":"2024-05-09T06:30:36.589969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:05.959852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587612+0000","last_change":"2024-05-09T06:30:35.602409+0000","last_active":"2024-05-09T06:30:36.587612+0000","last_peered":"2024-05-09T06:30:36.587612+0000","last_clean":"2024-05-09T06:30:36.587612+0000","last_became_active":"2024-05-09T06:30:35.602138+0000","last_became_peered":"2024-05-09T06:30:35.602138+0000","last_unstale":"2024-05-09T06:30:36.587612+0000","last_undegraded":"2024-05-09T06:30:36.587612+0000","last_fullsized":"2024-05-09T06:30:36.587612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:01.297028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815643+0000","last_change":"2024-05-09T06:30:35.599617+0000","last_active":"2024-05-09T06:30:36.815643+0000","last_peered":"2024-05-09T06:30:36.815643+0000","last_clean":"2024-05-09T06:30:36.815643+0000","last_became_active":"2024-05-09T06:30:35.599320+0000","last_became_peered":"2024-05-09T06:30:35.599320+0000","last_unstale":"2024-05-09T06:30:36.815643+0000","last_undegraded":"2024-05-09T06:30:36.815643+0000","last_fullsized":"2024-05-09T06:30:36.815643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:15.099298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815510+0000","last_change":"2024-05-09T06:30:35.600441+0000","last_active":"2024-05-09T06:30:36.815510+0000","last_peered":"2024-05-09T06:30:36.815510+0000","last_clean":"2024-05-09T06:30:36.815510+0000","last_became_active":"2024-05-09T06:30:35.600157+0000","last_became_peered":"2024-05-09T06:30:35.600157+0000","last_unstale":"2024-05-09T06:30:36.815510+0000","last_undegraded":"2024-05-09T06:30:36.815510+0000","last_fullsized":"2024-05-09T06:30:36.815510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:31.264391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582786+0000","last_change":"2024-05-09T06:30:35.596162+0000","last_active":"2024-05-09T06:30:36.582786+0000","last_peered":"2024-05-09T06:30:36.582786+0000","last_clean":"2024-05-09T06:30:36.582786+0000","last_became_active":"2024-05-09T06:30:35.595860+0000","last_became_peered":"2024-05-09T06:30:35.595860+0000","last_unstale":"2024-05-09T06:30:36.582786+0000","last_undegraded":"2024-05-09T06:30:36.582786+0000","last_fullsized":"2024-05-09T06:30:36.582786+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:51.680007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587912+0000","last_change":"2024-05-09T06:30:35.603174+0000","last_active":"2024-05-09T06:30:36.587912+0000","last_peered":"2024-05-09T06:30:36.587912+0000","last_clean":"2024-05-09T06:30:36.587912+0000","last_became_active":"2024-05-09T06:30:35.602852+0000","last_became_peered":"2024-05-09T06:30:35.602852+0000","last_unstale":"2024-05-09T06:30:36.587912+0000","last_undegraded":"2024-05-09T06:30:36.587912+0000","last_fullsized":"2024-05-09T06:30:36.587912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:53.676819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587996+0000","last_change":"2024-05-09T06:30:35.599007+0000","last_active":"2024-05-09T06:30:36.587996+0000","last_peered":"2024-05-09T06:30:36.587996+0000","last_clean":"2024-05-09T06:30:36.587996+0000","last_became_active":"2024-05-09T06:30:35.598617+0000","last_became_peered":"2024-05-09T06:30:35.598617+0000","last_unstale":"2024-05-09T06:30:36.587996+0000","last_undegraded":"2024-05-09T06:30:36.587996+0000","last_fullsized":"2024-05-09T06:30:36.587996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:44.024185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815290+0000","last_change":"2024-05-09T06:30:35.599066+0000","last_active":"2024-05-09T06:30:36.815290+0000","last_peered":"2024-05-09T06:30:36.815290+0000","last_clean":"2024-05-09T06:30:36.815290+0000","last_became_active":"2024-05-09T06:30:35.598726+0000","last_became_peered":"2024-05-09T06:30:35.598726+0000","last_unstale":"2024-05-09T06:30:36.815290+0000","last_undegraded":"2024-05-09T06:30:36.815290+0000","last_fullsized":"2024-05-09T06:30:36.815290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:24.617044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815078+0000","last_change":"2024-05-09T06:30:35.604711+0000","last_active":"2024-05-09T06:30:36.815078+0000","last_peered":"2024-05-09T06:30:36.815078+0000","last_clean":"2024-05-09T06:30:36.815078+0000","last_became_active":"2024-05-09T06:30:35.604392+0000","last_became_peered":"2024-05-09T06:30:35.604392+0000","last_unstale":"2024-05-09T06:30:36.815078+0000","last_undegraded":"2024-05-09T06:30:36.815078+0000","last_fullsized":"2024-05-09T06:30:36.815078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:22.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589975+0000","last_change":"2024-05-09T06:30:35.589521+0000","last_active":"2024-05-09T06:30:36.589975+0000","last_peered":"2024-05-09T06:30:36.589975+0000","last_clean":"2024-05-09T06:30:36.589975+0000","last_became_active":"2024-05-09T06:30:35.588794+0000","last_became_peered":"2024-05-09T06:30:35.588794+0000","last_unstale":"2024-05-09T06:30:36.589975+0000","last_undegraded":"2024-05-09T06:30:36.589975+0000","last_fullsized":"2024-05-09T06:30:36.589975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:13.171268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815309+0000","last_change":"2024-05-09T06:30:35.599410+0000","last_active":"2024-05-09T06:30:36.815309+0000","last_peered":"2024-05-09T06:30:36.815309+0000","last_clean":"2024-05-09T06:30:36.815309+0000","last_became_active":"2024-05-09T06:30:35.599073+0000","last_became_peered":"2024-05-09T06:30:35.599073+0000","last_unstale":"2024-05-09T06:30:36.815309+0000","last_undegraded":"2024-05-09T06:30:36.815309+0000","last_fullsized":"2024-05-09T06:30:36.815309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:26.303772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582929+0000","last_change":"2024-05-09T06:30:35.595848+0000","last_active":"2024-05-09T06:30:36.582929+0000","last_peered":"2024-05-09T06:30:36.582929+0000","last_clean":"2024-05-09T06:30:36.582929+0000","last_became_active":"2024-05-09T06:30:35.595562+0000","last_became_peered":"2024-05-09T06:30:35.595562+0000","last_unstale":"2024-05-09T06:30:36.582929+0000","last_undegraded":"2024-05-09T06:30:36.582929+0000","last_fullsized":"2024-05-09T06:30:36.582929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:58.128009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733821+0000","last_change":"2024-05-09T06:30:35.579805+0000","last_active":"2024-05-09T06:30:36.733821+0000","last_peered":"2024-05-09T06:30:36.733821+0000","last_clean":"2024-05-09T06:30:36.733821+0000","last_became_active":"2024-05-09T06:30:35.579480+0000","last_became_peered":"2024-05-09T06:30:35.579480+0000","last_unstale":"2024-05-09T06:30:36.733821+0000","last_undegraded":"2024-05-09T06:30:36.733821+0000","last_fullsized":"2024-05-09T06:30:36.733821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:09.630054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579401+0000","last_change":"2024-05-09T06:30:35.583903+0000","last_active":"2024-05-09T06:30:36.579401+0000","last_peered":"2024-05-09T06:30:36.579401+0000","last_clean":"2024-05-09T06:30:36.579401+0000","last_became_active":"2024-05-09T06:30:35.583582+0000","last_became_peered":"2024-05-09T06:30:35.583582+0000","last_unstale":"2024-05-09T06:30:36.579401+0000","last_undegraded":"2024-05-09T06:30:36.579401+0000","last_fullsized":"2024-05-09T06:30:36.579401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:24.757216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588172+0000","last_change":"2024-05-09T06:30:35.594510+0000","last_active":"2024-05-09T06:30:36.588172+0000","last_peered":"2024-05-09T06:30:36.588172+0000","last_clean":"2024-05-09T06:30:36.588172+0000","last_became_active":"2024-05-09T06:30:35.594231+0000","last_became_peered":"2024-05-09T06:30:35.594231+0000","last_unstale":"2024-05-09T06:30:36.588172+0000","last_undegraded":"2024-05-09T06:30:36.588172+0000","last_fullsized":"2024-05-09T06:30:36.588172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:32.185578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579301+0000","last_change":"2024-05-09T06:30:35.600502+0000","last_active":"2024-05-09T06:30:36.579301+0000","last_peered":"2024-05-09T06:30:36.579301+0000","last_clean":"2024-05-09T06:30:36.579301+0000","last_became_active":"2024-05-09T06:30:35.600169+0000","last_became_peered":"2024-05-09T06:30:35.600169+0000","last_unstale":"2024-05-09T06:30:36.579301+0000","last_undegraded":"2024-05-09T06:30:36.579301+0000","last_fullsized":"2024-05-09T06:30:36.579301+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:18.100098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579295+0000","last_change":"2024-05-09T06:30:35.600134+0000","last_active":"2024-05-09T06:30:36.579295+0000","last_peered":"2024-05-09T06:30:36.579295+0000","last_clean":"2024-05-09T06:30:36.579295+0000","last_became_active":"2024-05-09T06:30:35.598796+0000","last_became_peered":"2024-05-09T06:30:35.598796+0000","last_unstale":"2024-05-09T06:30:36.579295+0000","last_undegraded":"2024-05-09T06:30:36.579295+0000","last_fullsized":"2024-05-09T06:30:36.579295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:41.043668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815273+0000","last_change":"2024-05-09T06:30:35.598841+0000","last_active":"2024-05-09T06:30:36.815273+0000","last_peered":"2024-05-09T06:30:36.815273+0000","last_clean":"2024-05-09T06:30:36.815273+0000","last_became_active":"2024-05-09T06:30:35.598550+0000","last_became_peered":"2024-05-09T06:30:35.598550+0000","last_unstale":"2024-05-09T06:30:36.815273+0000","last_undegraded":"2024-05-09T06:30:36.815273+0000","last_fullsized":"2024-05-09T06:30:36.815273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:53.660591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588179+0000","last_change":"2024-05-09T06:30:35.591723+0000","last_active":"2024-05-09T06:30:36.588179+0000","last_peered":"2024-05-09T06:30:36.588179+0000","last_clean":"2024-05-09T06:30:36.588179+0000","last_became_active":"2024-05-09T06:30:35.591371+0000","last_became_peered":"2024-05-09T06:30:35.591371+0000","last_unstale":"2024-05-09T06:30:36.588179+0000","last_undegraded":"2024-05-09T06:30:36.588179+0000","last_fullsized":"2024-05-09T06:30:36.588179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:15.099298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588111+0000","last_change":"2024-05-09T06:30:35.590430+0000","last_active":"2024-05-09T06:30:36.588111+0000","last_peered":"2024-05-09T06:30:36.588111+0000","last_clean":"2024-05-09T06:30:36.588111+0000","last_became_active":"2024-05-09T06:30:35.590124+0000","last_became_peered":"2024-05-09T06:30:35.590124+0000","last_unstale":"2024-05-09T06:30:36.588111+0000","last_undegraded":"2024-05-09T06:30:36.588111+0000","last_fullsized":"2024-05-09T06:30:36.588111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:31.264391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582797+0000","last_change":"2024-05-09T06:30:35.581626+0000","last_active":"2024-05-09T06:30:36.582797+0000","last_peered":"2024-05-09T06:30:36.582797+0000","last_clean":"2024-05-09T06:30:36.582797+0000","last_became_active":"2024-05-09T06:30:35.581262+0000","last_became_peered":"2024-05-09T06:30:35.581262+0000","last_unstale":"2024-05-09T06:30:36.582797+0000","last_undegraded":"2024-05-09T06:30:36.582797+0000","last_fullsized":"2024-05-09T06:30:36.582797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:55.011652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815598+0000","last_change":"2024-05-09T06:30:35.598169+0000","last_active":"2024-05-09T06:30:36.815598+0000","last_peered":"2024-05-09T06:30:36.815598+0000","last_clean":"2024-05-09T06:30:36.815598+0000","last_became_active":"2024-05-09T06:30:35.597789+0000","last_became_peered":"2024-05-09T06:30:35.597789+0000","last_unstale":"2024-05-09T06:30:36.815598+0000","last_undegraded":"2024-05-09T06:30:36.815598+0000","last_fullsized":"2024-05-09T06:30:36.815598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:11.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815158+0000","last_change":"2024-05-09T06:30:35.606177+0000","last_active":"2024-05-09T06:30:36.815158+0000","last_peered":"2024-05-09T06:30:36.815158+0000","last_clean":"2024-05-09T06:30:36.815158+0000","last_became_active":"2024-05-09T06:30:35.605683+0000","last_became_peered":"2024-05-09T06:30:35.605683+0000","last_unstale":"2024-05-09T06:30:36.815158+0000","last_undegraded":"2024-05-09T06:30:36.815158+0000","last_fullsized":"2024-05-09T06:30:36.815158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:01.297028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587673+0000","last_change":"2024-05-09T06:30:35.602388+0000","last_active":"2024-05-09T06:30:36.587673+0000","last_peered":"2024-05-09T06:30:36.587673+0000","last_clean":"2024-05-09T06:30:36.587673+0000","last_became_active":"2024-05-09T06:30:35.602090+0000","last_became_peered":"2024-05-09T06:30:35.602090+0000","last_unstale":"2024-05-09T06:30:36.587673+0000","last_undegraded":"2024-05-09T06:30:36.587673+0000","last_fullsized":"2024-05-09T06:30:36.587673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:53.660591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582472+0000","last_change":"2024-05-09T06:30:35.580345+0000","last_active":"2024-05-09T06:30:36.582472+0000","last_peered":"2024-05-09T06:30:36.582472+0000","last_clean":"2024-05-09T06:30:36.582472+0000","last_became_active":"2024-05-09T06:30:35.580072+0000","last_became_peered":"2024-05-09T06:30:35.580072+0000","last_unstale":"2024-05-09T06:30:36.582472+0000","last_undegraded":"2024-05-09T06:30:36.582472+0000","last_fullsized":"2024-05-09T06:30:36.582472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:05.633077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814744+0000","last_change":"2024-05-09T06:30:35.580194+0000","last_active":"2024-05-09T06:30:36.814744+0000","last_peered":"2024-05-09T06:30:36.814744+0000","last_clean":"2024-05-09T06:30:36.814744+0000","last_became_active":"2024-05-09T06:30:35.579875+0000","last_became_peered":"2024-05-09T06:30:35.579875+0000","last_unstale":"2024-05-09T06:30:36.814744+0000","last_undegraded":"2024-05-09T06:30:36.814744+0000","last_fullsized":"2024-05-09T06:30:36.814744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:04.454302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733762+0000","last_change":"2024-05-09T06:30:35.595138+0000","last_active":"2024-05-09T06:30:36.733762+0000","last_peered":"2024-05-09T06:30:36.733762+0000","last_clean":"2024-05-09T06:30:36.733762+0000","last_became_active":"2024-05-09T06:30:35.593256+0000","last_became_peered":"2024-05-09T06:30:35.593256+0000","last_unstale":"2024-05-09T06:30:36.733762+0000","last_undegraded":"2024-05-09T06:30:36.733762+0000","last_fullsized":"2024-05-09T06:30:36.733762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:01.765676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815476+0000","last_change":"2024-05-09T06:30:34.586203+0000","last_active":"2024-05-09T06:30:36.815476+0000","last_peered":"2024-05-09T06:30:36.815476+0000","last_clean":"2024-05-09T06:30:36.815476+0000","last_became_active":"2024-05-09T06:30:34.585885+0000","last_became_peered":"2024-05-09T06:30:34.585885+0000","last_unstale":"2024-05-09T06:30:36.815476+0000","last_undegraded":"2024-05-09T06:30:36.815476+0000","last_fullsized":"2024-05-09T06:30:36.815476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:54.007366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583049+0000","last_change":"2024-05-09T06:30:35.584528+0000","last_active":"2024-05-09T06:30:36.583049+0000","last_peered":"2024-05-09T06:30:36.583049+0000","last_clean":"2024-05-09T06:30:36.583049+0000","last_became_active":"2024-05-09T06:30:35.584202+0000","last_became_peered":"2024-05-09T06:30:35.584202+0000","last_unstale":"2024-05-09T06:30:36.583049+0000","last_undegraded":"2024-05-09T06:30:36.583049+0000","last_fullsized":"2024-05-09T06:30:36.583049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:35.172202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815547+0000","last_change":"2024-05-09T06:30:34.584707+0000","last_active":"2024-05-09T06:30:36.815547+0000","last_peered":"2024-05-09T06:30:36.815547+0000","last_clean":"2024-05-09T06:30:36.815547+0000","last_became_active":"2024-05-09T06:30:34.584350+0000","last_became_peered":"2024-05-09T06:30:34.584350+0000","last_unstale":"2024-05-09T06:30:36.815547+0000","last_undegraded":"2024-05-09T06:30:36.815547+0000","last_fullsized":"2024-05-09T06:30:36.815547+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.095011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815384+0000","last_change":"2024-05-09T06:30:35.580204+0000","last_active":"2024-05-09T06:30:36.815384+0000","last_peered":"2024-05-09T06:30:36.815384+0000","last_clean":"2024-05-09T06:30:36.815384+0000","last_became_active":"2024-05-09T06:30:35.579888+0000","last_became_peered":"2024-05-09T06:30:35.579888+0000","last_unstale":"2024-05-09T06:30:36.815384+0000","last_undegraded":"2024-05-09T06:30:36.815384+0000","last_fullsized":"2024-05-09T06:30:36.815384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:23.661621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815425+0000","last_change":"2024-05-09T06:30:34.579101+0000","last_active":"2024-05-09T06:30:36.815425+0000","last_peered":"2024-05-09T06:30:36.815425+0000","last_clean":"2024-05-09T06:30:36.815425+0000","last_became_active":"2024-05-09T06:30:34.578809+0000","last_became_peered":"2024-05-09T06:30:34.578809+0000","last_unstale":"2024-05-09T06:30:36.815425+0000","last_undegraded":"2024-05-09T06:30:36.815425+0000","last_fullsized":"2024-05-09T06:30:36.815425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:43.019898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815500+0000","last_change":"2024-05-09T06:30:35.604960+0000","last_active":"2024-05-09T06:30:36.815500+0000","last_peered":"2024-05-09T06:30:36.815500+0000","last_clean":"2024-05-09T06:30:36.815500+0000","last_became_active":"2024-05-09T06:30:35.604530+0000","last_became_peered":"2024-05-09T06:30:35.604530+0000","last_unstale":"2024-05-09T06:30:36.815500+0000","last_undegraded":"2024-05-09T06:30:36.815500+0000","last_fullsized":"2024-05-09T06:30:36.815500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:44.024185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815187+0000","last_change":"2024-05-09T06:30:34.578459+0000","last_active":"2024-05-09T06:30:36.815187+0000","last_peered":"2024-05-09T06:30:36.815187+0000","last_clean":"2024-05-09T06:30:36.815187+0000","last_became_active":"2024-05-09T06:30:34.578161+0000","last_became_peered":"2024-05-09T06:30:34.578161+0000","last_unstale":"2024-05-09T06:30:36.815187+0000","last_undegraded":"2024-05-09T06:30:36.815187+0000","last_fullsized":"2024-05-09T06:30:36.815187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:39.265772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589992+0000","last_change":"2024-05-09T06:30:35.583897+0000","last_active":"2024-05-09T06:30:36.589992+0000","last_peered":"2024-05-09T06:30:36.589992+0000","last_clean":"2024-05-09T06:30:36.589992+0000","last_became_active":"2024-05-09T06:30:35.583628+0000","last_became_peered":"2024-05-09T06:30:35.583628+0000","last_unstale":"2024-05-09T06:30:36.589992+0000","last_undegraded":"2024-05-09T06:30:36.589992+0000","last_fullsized":"2024-05-09T06:30:36.589992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:23.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579636+0000","last_change":"2024-05-09T06:30:34.578866+0000","last_active":"2024-05-09T06:30:36.579636+0000","last_peered":"2024-05-09T06:30:36.579636+0000","last_clean":"2024-05-09T06:30:36.579636+0000","last_became_active":"2024-05-09T06:30:34.578584+0000","last_became_peered":"2024-05-09T06:30:34.578584+0000","last_unstale":"2024-05-09T06:30:36.579636+0000","last_undegraded":"2024-05-09T06:30:36.579636+0000","last_fullsized":"2024-05-09T06:30:36.579636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:03.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815290+0000","last_change":"2024-05-09T06:30:35.599294+0000","last_active":"2024-05-09T06:30:36.815290+0000","last_peered":"2024-05-09T06:30:36.815290+0000","last_clean":"2024-05-09T06:30:36.815290+0000","last_became_active":"2024-05-09T06:30:35.598955+0000","last_became_peered":"2024-05-09T06:30:35.598955+0000","last_unstale":"2024-05-09T06:30:36.815290+0000","last_undegraded":"2024-05-09T06:30:36.815290+0000","last_fullsized":"2024-05-09T06:30:36.815290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:09.630054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579414+0000","last_change":"2024-05-09T06:30:34.573669+0000","last_active":"2024-05-09T06:30:36.579414+0000","last_peered":"2024-05-09T06:30:36.579414+0000","last_clean":"2024-05-09T06:30:36.579414+0000","last_became_active":"2024-05-09T06:30:34.573372+0000","last_became_peered":"2024-05-09T06:30:34.573372+0000","last_unstale":"2024-05-09T06:30:36.579414+0000","last_undegraded":"2024-05-09T06:30:36.579414+0000","last_fullsized":"2024-05-09T06:30:36.579414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:00:48.303315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733414+0000","last_change":"2024-05-09T06:30:35.583522+0000","last_active":"2024-05-09T06:30:36.733414+0000","last_peered":"2024-05-09T06:30:36.733414+0000","last_clean":"2024-05-09T06:30:36.733414+0000","last_became_active":"2024-05-09T06:30:35.582877+0000","last_became_peered":"2024-05-09T06:30:35.582877+0000","last_unstale":"2024-05-09T06:30:36.733414+0000","last_undegraded":"2024-05-09T06:30:36.733414+0000","last_fullsized":"2024-05-09T06:30:36.733414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:43.552330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579299+0000","last_change":"2024-05-09T06:30:34.585694+0000","last_active":"2024-05-09T06:30:36.579299+0000","last_peered":"2024-05-09T06:30:36.579299+0000","last_clean":"2024-05-09T06:30:36.579299+0000","last_became_active":"2024-05-09T06:30:34.585395+0000","last_became_peered":"2024-05-09T06:30:34.585395+0000","last_unstale":"2024-05-09T06:30:36.579299+0000","last_undegraded":"2024-05-09T06:30:36.579299+0000","last_fullsized":"2024-05-09T06:30:36.579299+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:02.835777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579579+0000","last_change":"2024-05-09T06:30:35.584730+0000","last_active":"2024-05-09T06:30:36.579579+0000","last_peered":"2024-05-09T06:30:36.579579+0000","last_clean":"2024-05-09T06:30:36.579579+0000","last_became_active":"2024-05-09T06:30:35.584401+0000","last_became_peered":"2024-05-09T06:30:35.584401+0000","last_unstale":"2024-05-09T06:30:36.579579+0000","last_undegraded":"2024-05-09T06:30:36.579579+0000","last_fullsized":"2024-05-09T06:30:36.579579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:09.845887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733321+0000","last_change":"2024-05-09T06:30:34.585396+0000","last_active":"2024-05-09T06:30:36.733321+0000","last_peered":"2024-05-09T06:30:36.733321+0000","last_clean":"2024-05-09T06:30:36.733321+0000","last_became_active":"2024-05-09T06:30:34.585042+0000","last_became_peered":"2024-05-09T06:30:34.585042+0000","last_unstale":"2024-05-09T06:30:36.733321+0000","last_undegraded":"2024-05-09T06:30:36.733321+0000","last_fullsized":"2024-05-09T06:30:36.733321+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:05.148173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815714+0000","last_change":"2024-05-09T06:30:35.585358+0000","last_active":"2024-05-09T06:30:36.815714+0000","last_peered":"2024-05-09T06:30:36.815714+0000","last_clean":"2024-05-09T06:30:36.815714+0000","last_became_active":"2024-05-09T06:30:35.584981+0000","last_became_peered":"2024-05-09T06:30:35.584981+0000","last_unstale":"2024-05-09T06:30:36.815714+0000","last_undegraded":"2024-05-09T06:30:36.815714+0000","last_fullsized":"2024-05-09T06:30:36.815714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:11.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815645+0000","last_change":"2024-05-09T06:30:34.585067+0000","last_active":"2024-05-09T06:30:36.815645+0000","last_peered":"2024-05-09T06:30:36.815645+0000","last_clean":"2024-05-09T06:30:36.815645+0000","last_became_active":"2024-05-09T06:30:34.584344+0000","last_became_peered":"2024-05-09T06:30:34.584344+0000","last_unstale":"2024-05-09T06:30:36.815645+0000","last_undegraded":"2024-05-09T06:30:36.815645+0000","last_fullsized":"2024-05-09T06:30:36.815645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:55.676974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579411+0000","last_change":"2024-05-09T06:30:35.586882+0000","last_active":"2024-05-09T06:30:36.579411+0000","last_peered":"2024-05-09T06:30:36.579411+0000","last_clean":"2024-05-09T06:30:36.579411+0000","last_became_active":"2024-05-09T06:30:35.586559+0000","last_became_peered":"2024-05-09T06:30:35.586559+0000","last_unstale":"2024-05-09T06:30:36.579411+0000","last_undegraded":"2024-05-09T06:30:36.579411+0000","last_fullsized":"2024-05-09T06:30:36.579411+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:50:16.115738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583044+0000","last_change":"2024-05-09T06:30:34.581563+0000","last_active":"2024-05-09T06:30:36.583044+0000","last_peered":"2024-05-09T06:30:36.583044+0000","last_clean":"2024-05-09T06:30:36.583044+0000","last_became_active":"2024-05-09T06:30:34.581256+0000","last_became_peered":"2024-05-09T06:30:34.581256+0000","last_unstale":"2024-05-09T06:30:36.583044+0000","last_undegraded":"2024-05-09T06:30:36.583044+0000","last_fullsized":"2024-05-09T06:30:36.583044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:03.008622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733889+0000","last_change":"2024-05-09T06:30:35.593660+0000","last_active":"2024-05-09T06:30:36.733889+0000","last_peered":"2024-05-09T06:30:36.733889+0000","last_clean":"2024-05-09T06:30:36.733889+0000","last_became_active":"2024-05-09T06:30:35.593347+0000","last_became_peered":"2024-05-09T06:30:35.593347+0000","last_unstale":"2024-05-09T06:30:36.733889+0000","last_undegraded":"2024-05-09T06:30:36.733889+0000","last_fullsized":"2024-05-09T06:30:36.733889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.386784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815446+0000","last_change":"2024-05-09T06:30:34.585269+0000","last_active":"2024-05-09T06:30:36.815446+0000","last_peered":"2024-05-09T06:30:36.815446+0000","last_clean":"2024-05-09T06:30:36.815446+0000","last_became_active":"2024-05-09T06:30:34.584966+0000","last_became_peered":"2024-05-09T06:30:34.584966+0000","last_unstale":"2024-05-09T06:30:36.815446+0000","last_undegraded":"2024-05-09T06:30:36.815446+0000","last_fullsized":"2024-05-09T06:30:36.815446+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:41.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583099+0000","last_change":"2024-05-09T06:30:35.592954+0000","last_active":"2024-05-09T06:30:36.583099+0000","last_peered":"2024-05-09T06:30:36.583099+0000","last_clean":"2024-05-09T06:30:36.583099+0000","last_became_active":"2024-05-09T06:30:35.592664+0000","last_became_peered":"2024-05-09T06:30:35.592664+0000","last_unstale":"2024-05-09T06:30:36.583099+0000","last_undegraded":"2024-05-09T06:30:36.583099+0000","last_fullsized":"2024-05-09T06:30:36.583099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:25.535201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815687+0000","last_change":"2024-05-09T06:30:34.584317+0000","last_active":"2024-05-09T06:30:36.815687+0000","last_peered":"2024-05-09T06:30:36.815687+0000","last_clean":"2024-05-09T06:30:36.815687+0000","last_became_active":"2024-05-09T06:30:34.584002+0000","last_became_peered":"2024-05-09T06:30:34.584002+0000","last_unstale":"2024-05-09T06:30:36.815687+0000","last_undegraded":"2024-05-09T06:30:36.815687+0000","last_fullsized":"2024-05-09T06:30:36.815687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:57.123722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582464+0000","last_change":"2024-05-09T06:30:35.601825+0000","last_active":"2024-05-09T06:30:36.582464+0000","last_peered":"2024-05-09T06:30:36.582464+0000","last_clean":"2024-05-09T06:30:36.582464+0000","last_became_active":"2024-05-09T06:30:35.600566+0000","last_became_peered":"2024-05-09T06:30:35.600566+0000","last_unstale":"2024-05-09T06:30:36.582464+0000","last_undegraded":"2024-05-09T06:30:36.582464+0000","last_fullsized":"2024-05-09T06:30:36.582464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:13.066743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815307+0000","last_change":"2024-05-09T06:30:34.586092+0000","last_active":"2024-05-09T06:30:36.815307+0000","last_peered":"2024-05-09T06:30:36.815307+0000","last_clean":"2024-05-09T06:30:36.815307+0000","last_became_active":"2024-05-09T06:30:34.585765+0000","last_became_peered":"2024-05-09T06:30:34.585765+0000","last_unstale":"2024-05-09T06:30:36.815307+0000","last_undegraded":"2024-05-09T06:30:36.815307+0000","last_fullsized":"2024-05-09T06:30:36.815307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:42.548043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587605+0000","last_change":"2024-05-09T06:30:35.595770+0000","last_active":"2024-05-09T06:30:36.587605+0000","last_peered":"2024-05-09T06:30:36.587605+0000","last_clean":"2024-05-09T06:30:36.587605+0000","last_became_active":"2024-05-09T06:30:35.595500+0000","last_became_peered":"2024-05-09T06:30:35.595500+0000","last_unstale":"2024-05-09T06:30:36.587605+0000","last_undegraded":"2024-05-09T06:30:36.587605+0000","last_fullsized":"2024-05-09T06:30:36.587605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:22.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588281+0000","last_change":"2024-05-09T06:30:34.582151+0000","last_active":"2024-05-09T06:30:36.588281+0000","last_peered":"2024-05-09T06:30:36.588281+0000","last_clean":"2024-05-09T06:30:36.588281+0000","last_became_active":"2024-05-09T06:30:34.581849+0000","last_became_peered":"2024-05-09T06:30:34.581849+0000","last_unstale":"2024-05-09T06:30:36.588281+0000","last_undegraded":"2024-05-09T06:30:36.588281+0000","last_fullsized":"2024-05-09T06:30:36.588281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:42.548043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733465+0000","last_change":"2024-05-09T06:30:35.583325+0000","last_active":"2024-05-09T06:30:36.733465+0000","last_peered":"2024-05-09T06:30:36.733465+0000","last_clean":"2024-05-09T06:30:36.733465+0000","last_became_active":"2024-05-09T06:30:35.582132+0000","last_became_peered":"2024-05-09T06:30:35.582132+0000","last_unstale":"2024-05-09T06:30:36.733465+0000","last_undegraded":"2024-05-09T06:30:36.733465+0000","last_fullsized":"2024-05-09T06:30:36.733465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:48.194063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815273+0000","last_change":"2024-05-09T06:30:34.584924+0000","last_active":"2024-05-09T06:30:36.815273+0000","last_peered":"2024-05-09T06:30:36.815273+0000","last_clean":"2024-05-09T06:30:36.815273+0000","last_became_active":"2024-05-09T06:30:34.584194+0000","last_became_peered":"2024-05-09T06:30:34.584194+0000","last_unstale":"2024-05-09T06:30:36.815273+0000","last_undegraded":"2024-05-09T06:30:36.815273+0000","last_fullsized":"2024-05-09T06:30:36.815273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:24.530914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815098+0000","last_change":"2024-05-09T06:30:35.602403+0000","last_active":"2024-05-09T06:30:36.815098+0000","last_peered":"2024-05-09T06:30:36.815098+0000","last_clean":"2024-05-09T06:30:36.815098+0000","last_became_active":"2024-05-09T06:30:35.602105+0000","last_became_peered":"2024-05-09T06:30:35.602105+0000","last_unstale":"2024-05-09T06:30:36.815098+0000","last_undegraded":"2024-05-09T06:30:36.815098+0000","last_fullsized":"2024-05-09T06:30:36.815098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:51.940031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582377+0000","last_change":"2024-05-09T06:30:34.586240+0000","last_active":"2024-05-09T06:30:36.582377+0000","last_peered":"2024-05-09T06:30:36.582377+0000","last_clean":"2024-05-09T06:30:36.582377+0000","last_became_active":"2024-05-09T06:30:34.585916+0000","last_became_peered":"2024-05-09T06:30:34.585916+0000","last_unstale":"2024-05-09T06:30:36.582377+0000","last_undegraded":"2024-05-09T06:30:36.582377+0000","last_fullsized":"2024-05-09T06:30:36.582377+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:51.243721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815598+0000","last_change":"2024-05-09T06:30:35.599942+0000","last_active":"2024-05-09T06:30:36.815598+0000","last_peered":"2024-05-09T06:30:36.815598+0000","last_clean":"2024-05-09T06:30:36.815598+0000","last_became_active":"2024-05-09T06:30:35.599578+0000","last_became_peered":"2024-05-09T06:30:35.599578+0000","last_unstale":"2024-05-09T06:30:36.815598+0000","last_undegraded":"2024-05-09T06:30:36.815598+0000","last_fullsized":"2024-05-09T06:30:36.815598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:32.185578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588166+0000","last_change":"2024-05-09T06:30:34.581592+0000","last_active":"2024-05-09T06:30:36.588166+0000","last_peered":"2024-05-09T06:30:36.588166+0000","last_clean":"2024-05-09T06:30:36.588166+0000","last_became_active":"2024-05-09T06:30:34.581306+0000","last_became_peered":"2024-05-09T06:30:34.581306+0000","last_unstale":"2024-05-09T06:30:36.588166+0000","last_undegraded":"2024-05-09T06:30:36.588166+0000","last_fullsized":"2024-05-09T06:30:36.588166+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:41.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733849+0000","last_change":"2024-05-09T06:30:35.589096+0000","last_active":"2024-05-09T06:30:36.733849+0000","last_peered":"2024-05-09T06:30:36.733849+0000","last_clean":"2024-05-09T06:30:36.733849+0000","last_became_active":"2024-05-09T06:30:35.588760+0000","last_became_peered":"2024-05-09T06:30:35.588760+0000","last_unstale":"2024-05-09T06:30:36.733849+0000","last_undegraded":"2024-05-09T06:30:36.733849+0000","last_fullsized":"2024-05-09T06:30:36.733849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:11.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815576+0000","last_change":"2024-05-09T06:30:34.585960+0000","last_active":"2024-05-09T06:30:36.815576+0000","last_peered":"2024-05-09T06:30:36.815576+0000","last_clean":"2024-05-09T06:30:36.815576+0000","last_became_active":"2024-05-09T06:30:34.585629+0000","last_became_peered":"2024-05-09T06:30:34.585629+0000","last_unstale":"2024-05-09T06:30:36.815576+0000","last_undegraded":"2024-05-09T06:30:36.815576+0000","last_fullsized":"2024-05-09T06:30:36.815576+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:34.167915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582952+0000","last_change":"2024-05-09T06:30:35.582883+0000","last_active":"2024-05-09T06:30:36.582952+0000","last_peered":"2024-05-09T06:30:36.582952+0000","last_clean":"2024-05-09T06:30:36.582952+0000","last_became_active":"2024-05-09T06:30:35.582563+0000","last_became_peered":"2024-05-09T06:30:35.582563+0000","last_unstale":"2024-05-09T06:30:36.582952+0000","last_undegraded":"2024-05-09T06:30:36.582952+0000","last_fullsized":"2024-05-09T06:30:36.582952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:08.940461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815588+0000","last_change":"2024-05-09T06:30:34.584535+0000","last_active":"2024-05-09T06:30:36.815588+0000","last_peered":"2024-05-09T06:30:36.815588+0000","last_clean":"2024-05-09T06:30:36.815588+0000","last_became_active":"2024-05-09T06:30:34.584226+0000","last_became_peered":"2024-05-09T06:30:34.584226+0000","last_unstale":"2024-05-09T06:30:36.815588+0000","last_undegraded":"2024-05-09T06:30:36.815588+0000","last_fullsized":"2024-05-09T06:30:36.815588+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:04.525662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733821+0000","last_change":"2024-05-09T06:30:35.591377+0000","last_active":"2024-05-09T06:30:36.733821+0000","last_peered":"2024-05-09T06:30:36.733821+0000","last_clean":"2024-05-09T06:30:36.733821+0000","last_became_active":"2024-05-09T06:30:35.591115+0000","last_became_peered":"2024-05-09T06:30:35.591115+0000","last_unstale":"2024-05-09T06:30:36.733821+0000","last_undegraded":"2024-05-09T06:30:36.733821+0000","last_fullsized":"2024-05-09T06:30:36.733821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:26.303772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579414+0000","last_change":"2024-05-09T06:30:34.579600+0000","last_active":"2024-05-09T06:30:36.579414+0000","last_peered":"2024-05-09T06:30:36.579414+0000","last_clean":"2024-05-09T06:30:36.579414+0000","last_became_active":"2024-05-09T06:30:34.579332+0000","last_became_peered":"2024-05-09T06:30:34.579332+0000","last_unstale":"2024-05-09T06:30:36.579414+0000","last_undegraded":"2024-05-09T06:30:36.579414+0000","last_fullsized":"2024-05-09T06:30:36.579414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:22.685792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814776+0000","last_change":"2024-05-09T06:30:35.580810+0000","last_active":"2024-05-09T06:30:36.814776+0000","last_peered":"2024-05-09T06:30:36.814776+0000","last_clean":"2024-05-09T06:30:36.814776+0000","last_became_active":"2024-05-09T06:30:35.580084+0000","last_became_peered":"2024-05-09T06:30:35.580084+0000","last_unstale":"2024-05-09T06:30:36.814776+0000","last_undegraded":"2024-05-09T06:30:36.814776+0000","last_fullsized":"2024-05-09T06:30:36.814776+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:14.667295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814922+0000","last_change":"2024-05-09T06:30:34.579697+0000","last_active":"2024-05-09T06:30:36.814922+0000","last_peered":"2024-05-09T06:30:36.814922+0000","last_clean":"2024-05-09T06:30:36.814922+0000","last_became_active":"2024-05-09T06:30:34.579396+0000","last_became_peered":"2024-05-09T06:30:34.579396+0000","last_unstale":"2024-05-09T06:30:36.814922+0000","last_undegraded":"2024-05-09T06:30:36.814922+0000","last_fullsized":"2024-05-09T06:30:36.814922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:52.672533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587632+0000","last_change":"2024-05-09T06:30:35.591336+0000","last_active":"2024-05-09T06:30:36.587632+0000","last_peered":"2024-05-09T06:30:36.587632+0000","last_clean":"2024-05-09T06:30:36.587632+0000","last_became_active":"2024-05-09T06:30:35.591044+0000","last_became_peered":"2024-05-09T06:30:35.591044+0000","last_unstale":"2024-05-09T06:30:36.587632+0000","last_undegraded":"2024-05-09T06:30:36.587632+0000","last_fullsized":"2024-05-09T06:30:36.587632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:24.617044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587797+0000","last_change":"2024-05-09T06:30:34.582245+0000","last_active":"2024-05-09T06:30:36.587797+0000","last_peered":"2024-05-09T06:30:36.587797+0000","last_clean":"2024-05-09T06:30:36.587797+0000","last_became_active":"2024-05-09T06:30:34.581844+0000","last_became_peered":"2024-05-09T06:30:34.581844+0000","last_unstale":"2024-05-09T06:30:36.587797+0000","last_undegraded":"2024-05-09T06:30:36.587797+0000","last_fullsized":"2024-05-09T06:30:36.587797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:47.189776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733638+0000","last_change":"2024-05-09T06:30:35.583250+0000","last_active":"2024-05-09T06:30:36.733638+0000","last_peered":"2024-05-09T06:30:36.733638+0000","last_clean":"2024-05-09T06:30:36.733638+0000","last_became_active":"2024-05-09T06:30:35.582493+0000","last_became_peered":"2024-05-09T06:30:35.582493+0000","last_unstale":"2024-05-09T06:30:36.733638+0000","last_undegraded":"2024-05-09T06:30:36.733638+0000","last_fullsized":"2024-05-09T06:30:36.733638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:51.940031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814883+0000","last_change":"2024-05-09T06:30:34.584692+0000","last_active":"2024-05-09T06:30:36.814883+0000","last_peered":"2024-05-09T06:30:36.814883+0000","last_clean":"2024-05-09T06:30:36.814883+0000","last_became_active":"2024-05-09T06:30:34.584194+0000","last_became_peered":"2024-05-09T06:30:34.584194+0000","last_unstale":"2024-05-09T06:30:36.814883+0000","last_undegraded":"2024-05-09T06:30:36.814883+0000","last_fullsized":"2024-05-09T06:30:36.814883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:31.181292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815156+0000","last_change":"2024-05-09T06:30:35.582542+0000","last_active":"2024-05-09T06:30:36.815156+0000","last_peered":"2024-05-09T06:30:36.815156+0000","last_clean":"2024-05-09T06:30:36.815156+0000","last_became_active":"2024-05-09T06:30:35.582265+0000","last_became_peered":"2024-05-09T06:30:35.582265+0000","last_unstale":"2024-05-09T06:30:36.815156+0000","last_undegraded":"2024-05-09T06:30:36.815156+0000","last_fullsized":"2024-05-09T06:30:36.815156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:35.426267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588313+0000","last_change":"2024-05-09T06:30:34.587377+0000","last_active":"2024-05-09T06:30:36.588313+0000","last_peered":"2024-05-09T06:30:36.588313+0000","last_clean":"2024-05-09T06:30:36.588313+0000","last_became_active":"2024-05-09T06:30:34.587081+0000","last_became_peered":"2024-05-09T06:30:34.587081+0000","last_unstale":"2024-05-09T06:30:36.588313+0000","last_undegraded":"2024-05-09T06:30:36.588313+0000","last_fullsized":"2024-05-09T06:30:36.588313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:04.525662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.590198+0000","last_change":"2024-05-09T06:30:35.581800+0000","last_active":"2024-05-09T06:30:36.590198+0000","last_peered":"2024-05-09T06:30:36.590198+0000","last_clean":"2024-05-09T06:30:36.590198+0000","last_became_active":"2024-05-09T06:30:35.581406+0000","last_became_peered":"2024-05-09T06:30:35.581406+0000","last_unstale":"2024-05-09T06:30:36.590198+0000","last_undegraded":"2024-05-09T06:30:36.590198+0000","last_fullsized":"2024-05-09T06:30:36.590198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:54:49.122341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589624+0000","last_change":"2024-05-09T06:30:34.574936+0000","last_active":"2024-05-09T06:30:36.589624+0000","last_peered":"2024-05-09T06:30:36.589624+0000","last_clean":"2024-05-09T06:30:36.589624+0000","last_became_active":"2024-05-09T06:30:34.574630+0000","last_became_peered":"2024-05-09T06:30:34.574630+0000","last_unstale":"2024-05-09T06:30:36.589624+0000","last_undegraded":"2024-05-09T06:30:36.589624+0000","last_fullsized":"2024-05-09T06:30:36.589624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:24.530914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588247+0000","last_change":"2024-05-09T06:30:35.589778+0000","last_active":"2024-05-09T06:30:36.588247+0000","last_peered":"2024-05-09T06:30:36.588247+0000","last_clean":"2024-05-09T06:30:36.588247+0000","last_became_active":"2024-05-09T06:30:35.589458+0000","last_became_peered":"2024-05-09T06:30:35.589458+0000","last_unstale":"2024-05-09T06:30:36.588247+0000","last_undegraded":"2024-05-09T06:30:36.588247+0000","last_fullsized":"2024-05-09T06:30:36.588247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:40.270059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815520+0000","last_change":"2024-05-09T06:30:34.575398+0000","last_active":"2024-05-09T06:30:36.815520+0000","last_peered":"2024-05-09T06:30:36.815520+0000","last_clean":"2024-05-09T06:30:36.815520+0000","last_became_active":"2024-05-09T06:30:34.574158+0000","last_became_peered":"2024-05-09T06:30:34.574158+0000","last_unstale":"2024-05-09T06:30:36.815520+0000","last_undegraded":"2024-05-09T06:30:36.815520+0000","last_fullsized":"2024-05-09T06:30:36.815520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:30.260104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733912+0000","last_change":"2024-05-09T06:30:35.590668+0000","last_active":"2024-05-09T06:30:36.733912+0000","last_peered":"2024-05-09T06:30:36.733912+0000","last_clean":"2024-05-09T06:30:36.733912+0000","last_became_active":"2024-05-09T06:30:35.590391+0000","last_became_peered":"2024-05-09T06:30:35.590391+0000","last_unstale":"2024-05-09T06:30:36.733912+0000","last_undegraded":"2024-05-09T06:30:36.733912+0000","last_fullsized":"2024-05-09T06:30:36.733912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:11.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815739+0000","last_change":"2024-05-09T06:30:34.581343+0000","last_active":"2024-05-09T06:30:36.815739+0000","last_peered":"2024-05-09T06:30:36.815739+0000","last_clean":"2024-05-09T06:30:36.815739+0000","last_became_active":"2024-05-09T06:30:34.581002+0000","last_became_peered":"2024-05-09T06:30:34.581002+0000","last_unstale":"2024-05-09T06:30:36.815739+0000","last_undegraded":"2024-05-09T06:30:36.815739+0000","last_fullsized":"2024-05-09T06:30:36.815739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:07.936174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733340+0000","last_change":"2024-05-09T06:30:35.592518+0000","last_active":"2024-05-09T06:30:36.733340+0000","last_peered":"2024-05-09T06:30:36.733340+0000","last_clean":"2024-05-09T06:30:36.733340+0000","last_became_active":"2024-05-09T06:30:35.591911+0000","last_became_peered":"2024-05-09T06:30:35.591911+0000","last_unstale":"2024-05-09T06:30:36.733340+0000","last_undegraded":"2024-05-09T06:30:36.733340+0000","last_fullsized":"2024-05-09T06:30:36.733340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:05.529949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815425+0000","last_change":"2024-05-09T06:30:34.585348+0000","last_active":"2024-05-09T06:30:36.815425+0000","last_peered":"2024-05-09T06:30:36.815425+0000","last_clean":"2024-05-09T06:30:36.815425+0000","last_became_active":"2024-05-09T06:30:34.584016+0000","last_became_peered":"2024-05-09T06:30:34.584016+0000","last_unstale":"2024-05-09T06:30:36.815425+0000","last_undegraded":"2024-05-09T06:30:36.815425+0000","last_fullsized":"2024-05-09T06:30:36.815425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:47.189776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582633+0000","last_change":"2024-05-09T06:30:35.601343+0000","last_active":"2024-05-09T06:30:36.582633+0000","last_peered":"2024-05-09T06:30:36.582633+0000","last_clean":"2024-05-09T06:30:36.582633+0000","last_became_active":"2024-05-09T06:30:35.601059+0000","last_became_peered":"2024-05-09T06:30:35.601059+0000","last_unstale":"2024-05-09T06:30:36.582633+0000","last_undegraded":"2024-05-09T06:30:36.582633+0000","last_fullsized":"2024-05-09T06:30:36.582633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:43.563652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579512+0000","last_change":"2024-05-09T06:30:34.581383+0000","last_active":"2024-05-09T06:30:36.579512+0000","last_peered":"2024-05-09T06:30:36.579512+0000","last_clean":"2024-05-09T06:30:36.579512+0000","last_became_active":"2024-05-09T06:30:34.581087+0000","last_became_peered":"2024-05-09T06:30:34.581087+0000","last_unstale":"2024-05-09T06:30:36.579512+0000","last_undegraded":"2024-05-09T06:30:36.579512+0000","last_fullsized":"2024-05-09T06:30:36.579512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:34.548659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815230+0000","last_change":"2024-05-09T06:30:35.598716+0000","last_active":"2024-05-09T06:30:36.815230+0000","last_peered":"2024-05-09T06:30:36.815230+0000","last_clean":"2024-05-09T06:30:36.815230+0000","last_became_active":"2024-05-09T06:30:35.598367+0000","last_became_peered":"2024-05-09T06:30:35.598367+0000","last_unstale":"2024-05-09T06:30:36.815230+0000","last_undegraded":"2024-05-09T06:30:36.815230+0000","last_fullsized":"2024-05-09T06:30:36.815230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:56.848828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579406+0000","last_change":"2024-05-09T06:30:34.579968+0000","last_active":"2024-05-09T06:30:36.579406+0000","last_peered":"2024-05-09T06:30:36.579406+0000","last_clean":"2024-05-09T06:30:36.579406+0000","last_became_active":"2024-05-09T06:30:34.579687+0000","last_became_peered":"2024-05-09T06:30:34.579687+0000","last_unstale":"2024-05-09T06:30:36.579406+0000","last_undegraded":"2024-05-09T06:30:36.579406+0000","last_fullsized":"2024-05-09T06:30:36.579406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:32.965877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589963+0000","last_change":"2024-05-09T06:30:35.588154+0000","last_active":"2024-05-09T06:30:36.589963+0000","last_peered":"2024-05-09T06:30:36.589963+0000","last_clean":"2024-05-09T06:30:36.589963+0000","last_became_active":"2024-05-09T06:30:35.587854+0000","last_became_peered":"2024-05-09T06:30:35.587854+0000","last_unstale":"2024-05-09T06:30:36.589963+0000","last_undegraded":"2024-05-09T06:30:36.589963+0000","last_fullsized":"2024-05-09T06:30:36.589963+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:46.225956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814696+0000","last_change":"2024-05-09T06:30:34.586042+0000","last_active":"2024-05-09T06:30:36.814696+0000","last_peered":"2024-05-09T06:30:36.814696+0000","last_clean":"2024-05-09T06:30:36.814696+0000","last_became_active":"2024-05-09T06:30:34.585688+0000","last_became_peered":"2024-05-09T06:30:34.585688+0000","last_unstale":"2024-05-09T06:30:36.814696+0000","last_undegraded":"2024-05-09T06:30:36.814696+0000","last_fullsized":"2024-05-09T06:30:36.814696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:50.675720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733813+0000","last_change":"2024-05-09T06:30:35.592271+0000","last_active":"2024-05-09T06:30:36.733813+0000","last_peered":"2024-05-09T06:30:36.733813+0000","last_clean":"2024-05-09T06:30:36.733813+0000","last_became_active":"2024-05-09T06:30:35.591953+0000","last_became_peered":"2024-05-09T06:30:35.591953+0000","last_unstale":"2024-05-09T06:30:36.733813+0000","last_undegraded":"2024-05-09T06:30:36.733813+0000","last_fullsized":"2024-05-09T06:30:36.733813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:56.848828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579225+0000","last_change":"2024-05-09T06:30:34.579060+0000","last_active":"2024-05-09T06:30:36.579225+0000","last_peered":"2024-05-09T06:30:36.579225+0000","last_clean":"2024-05-09T06:30:36.579225+0000","last_became_active":"2024-05-09T06:30:34.574824+0000","last_became_peered":"2024-05-09T06:30:34.574824+0000","last_unstale":"2024-05-09T06:30:36.579225+0000","last_undegraded":"2024-05-09T06:30:36.579225+0000","last_fullsized":"2024-05-09T06:30:36.579225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:13.663009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814949+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:30:36.814949+0000","last_peered":"2024-05-09T06:30:36.814949+0000","last_clean":"2024-05-09T06:30:36.814949+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:30:36.814949+0000","last_undegraded":"2024-05-09T06:30:36.814949+0000","last_fullsized":"2024-05-09T06:30:36.814949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1384448,"data_stored":941910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":350800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":350788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":352378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":350766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":350766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":353142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":941134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:30:39.579 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.736+0000 7f1d6fcae640 1 Processor -- start 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.737+0000 7f1d6fcae640 1 -- start start 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.737+0000 7f1d6fcae640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 0x7f1d680d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.737+0000 7f1d6fcae640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d680d4940 0x7f1d680d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.737+0000 7f1d6fcae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d680cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.737+0000 7f1d6fcae640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1d680d4dd0 con 0x7f1d680d4940 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6fcae640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1d680b33c0 con 0x7f1d68161950 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6fcae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680b3660 con 0x7f1d680d0f80 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d680d4940 0x7f1d680d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d680d4940 0x7f1d680d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33324/0 (socket says 172.21.15.1:33324) 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 -- 172.21.15.1:0/3730040825 learned_addr learned my addr 172.21.15.1:0/3730040825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6e24b640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d680cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6da4a640 1 --2- 172.21.15.1:0/3730040825 >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 0x7f1d680d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 -- 172.21.15.1:0/3730040825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 msgr2=0x7f1d680cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d680cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 -- 172.21.15.1:0/3730040825 >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 msgr2=0x7f1d680d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 --2- 172.21.15.1:0/3730040825 >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 0x7f1d680d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6d249640 1 -- 172.21.15.1:0/3730040825 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680b3940 con 0x7f1d680d4940 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.738+0000 7f1d6da4a640 1 --2- 172.21.15.1:0/3730040825 >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 0x7f1d680d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6d249640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d680d4940 0x7f1d680d2940 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1d6806b7f0 tx=0x7f1d5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd92b3ac1e9241b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6ca48640 1 -- 172.21.15.1:0/3730040825 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58037070 con 0x7f1d680d4940 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6ca48640 1 -- 172.21.15.1:0/3730040825 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58032440 con 0x7f1d680d4940 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6ca48640 1 -- 172.21.15.1:0/3730040825 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58032810 con 0x7f1d680d4940 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6fcae640 1 -- 172.21.15.1:0/3730040825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d680d4940 msgr2=0x7f1d680d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.739+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d680d4940 0x7f1d680d2940 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1d6806b7f0 tx=0x7f1d5802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 -- 172.21.15.1:0/3730040825 shutdown_connections 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d680cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/3730040825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d680d4940 0x7f1d680d2940 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/3730040825 >> v2:172.21.15.6:3300/0 conn(0x7f1d68161950 0x7f1d680d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 -- 172.21.15.1:0/3730040825 >> 172.21.15.1:0/3730040825 conn(0x7f1d680d3e20 msgr2=0x7f1d680d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 -- 172.21.15.1:0/3730040825 shutdown_connections 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.740+0000 7f1d6fcae640 1 -- 172.21.15.1:0/3730040825 wait complete. 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.741+0000 7f1d6fcae640 1 Processor -- start 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.741+0000 7f1d6fcae640 1 -- start start 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d681f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d681579c0 0x7f1d680b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680d1d80 con 0x7f1d68161950 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680d1c00 con 0x7f1d681579c0 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6fcae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680d1f00 con 0x7f1d680d0f80 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33326/0 (socket says 172.21.15.1:33326) 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 -- 172.21.15.1:0/2379275697 learned_addr learned my addr 172.21.15.1:0/2379275697 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6da4a640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d681f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6e24b640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d681579c0 0x7f1d680b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 -- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 msgr2=0x7f1d681f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d681f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 -- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d681579c0 msgr2=0x7f1d680b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d681579c0 0x7f1d680b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 -- 172.21.15.1:0/2379275697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d681f8100 con 0x7f1d68161950 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d6d249640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1d5802a220 tx=0x7f1d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58041020 con 0x7f1d68161950 2024-05-09T06:30:39.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d580306c0 con 0x7f1d68161950 2024-05-09T06:30:39.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.742+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58037040 con 0x7f1d68161950 2024-05-09T06:30:39.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.743+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d680caf90 con 0x7f1d68161950 2024-05-09T06:30:39.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.743+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d680cb550 con 0x7f1d68161950 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.744+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d680752b0 con 0x7f1d68161950 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.744+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d58044020 con 0x7f1d68161950 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.744+0000 7f1d5e7fc640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 0x7f1d4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.745+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1d580ab6e0 con 0x7f1d68161950 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.745+0000 7f1d6da4a640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 0x7f1d4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.745+0000 7f1d6da4a640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 0x7f1d4c06e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006e80 tx=0x7f1d64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.749+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d5804b030 con 0x7f1d68161950 2024-05-09T06:30:39.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.928+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d680cd800 con 0x7f1d4c06be60 2024-05-09T06:30:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.934+0000 7f1d5e7fc640 1 -- 172.21.15.1:0/2379275697 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7f1d680cd800 con 0x7f1d4c06be60 2024-05-09T06:30:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.936+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 msgr2=0x7f1d4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.936+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 0x7f1d4c06e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006e80 tx=0x7f1d64008040 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.936+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 msgr2=0x7f1d68157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.936+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1d5802a220 tx=0x7f1d580046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 shutdown_connections 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d4c06be60 0x7f1d4c06e320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d681579c0 0x7f1d680b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d68161950 0x7f1d68157480 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 --2- 172.21.15.1:0/2379275697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d680d0f80 0x7f1d681f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 >> 172.21.15.1:0/2379275697 conn(0x7f1d680d3e20 msgr2=0x7f1d680b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 shutdown_connections 2024-05-09T06:30:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:39.937+0000 7f1d6fcae640 1 -- 172.21.15.1:0/2379275697 wait complete. 2024-05-09T06:30:39.931 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-09T06:30:39.109311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244280,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730440,"statfs":{"total":773094113280,"available":772843970560,"internally_reserved":0,"allocated":7446528,"data_stored":3991684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550883"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815679+0000","last_change":"2024-05-09T06:30:35.589750+0000","last_active":"2024-05-09T06:30:36.815679+0000","last_peered":"2024-05-09T06:30:36.815679+0000","last_clean":"2024-05-09T06:30:36.815679+0000","last_became_active":"2024-05-09T06:30:35.589385+0000","last_became_peered":"2024-05-09T06:30:35.589385+0000","last_unstale":"2024-05-09T06:30:36.815679+0000","last_undegraded":"2024-05-09T06:30:36.815679+0000","last_fullsized":"2024-05-09T06:30:36.815679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.386784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815241+0000","last_change":"2024-05-09T06:30:35.587930+0000","last_active":"2024-05-09T06:30:36.815241+0000","last_peered":"2024-05-09T06:30:36.815241+0000","last_clean":"2024-05-09T06:30:36.815241+0000","last_became_active":"2024-05-09T06:30:35.587662+0000","last_became_peered":"2024-05-09T06:30:35.587662+0000","last_unstale":"2024-05-09T06:30:36.815241+0000","last_undegraded":"2024-05-09T06:30:36.815241+0000","last_fullsized":"2024-05-09T06:30:36.815241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:01.765676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815446+0000","last_change":"2024-05-09T06:30:35.599075+0000","last_active":"2024-05-09T06:30:36.815446+0000","last_peered":"2024-05-09T06:30:36.815446+0000","last_clean":"2024-05-09T06:30:36.815446+0000","last_became_active":"2024-05-09T06:30:35.598755+0000","last_became_peered":"2024-05-09T06:30:35.598755+0000","last_unstale":"2024-05-09T06:30:36.815446+0000","last_undegraded":"2024-05-09T06:30:36.815446+0000","last_fullsized":"2024-05-09T06:30:36.815446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:53.676819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583013+0000","last_change":"2024-05-09T06:30:35.594930+0000","last_active":"2024-05-09T06:30:36.583013+0000","last_peered":"2024-05-09T06:30:36.583013+0000","last_clean":"2024-05-09T06:30:36.583013+0000","last_became_active":"2024-05-09T06:30:35.594589+0000","last_became_peered":"2024-05-09T06:30:35.594589+0000","last_unstale":"2024-05-09T06:30:36.583013+0000","last_undegraded":"2024-05-09T06:30:36.583013+0000","last_fullsized":"2024-05-09T06:30:36.583013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:32.975202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733561+0000","last_change":"2024-05-09T06:30:35.589147+0000","last_active":"2024-05-09T06:30:36.733561+0000","last_peered":"2024-05-09T06:30:36.733561+0000","last_clean":"2024-05-09T06:30:36.733561+0000","last_became_active":"2024-05-09T06:30:35.588807+0000","last_became_peered":"2024-05-09T06:30:35.588807+0000","last_unstale":"2024-05-09T06:30:36.733561+0000","last_undegraded":"2024-05-09T06:30:36.733561+0000","last_fullsized":"2024-05-09T06:30:36.733561+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:56.681261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814953+0000","last_change":"2024-05-09T06:30:35.602433+0000","last_active":"2024-05-09T06:30:36.814953+0000","last_peered":"2024-05-09T06:30:36.814953+0000","last_clean":"2024-05-09T06:30:36.814953+0000","last_became_active":"2024-05-09T06:30:35.602174+0000","last_became_peered":"2024-05-09T06:30:35.602174+0000","last_unstale":"2024-05-09T06:30:36.814953+0000","last_undegraded":"2024-05-09T06:30:36.814953+0000","last_fullsized":"2024-05-09T06:30:36.814953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:42.173195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589969+0000","last_change":"2024-05-09T06:30:35.601000+0000","last_active":"2024-05-09T06:30:36.589969+0000","last_peered":"2024-05-09T06:30:36.589969+0000","last_clean":"2024-05-09T06:30:36.589969+0000","last_became_active":"2024-05-09T06:30:35.600712+0000","last_became_peered":"2024-05-09T06:30:35.600712+0000","last_unstale":"2024-05-09T06:30:36.589969+0000","last_undegraded":"2024-05-09T06:30:36.589969+0000","last_fullsized":"2024-05-09T06:30:36.589969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:05.959852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587612+0000","last_change":"2024-05-09T06:30:35.602409+0000","last_active":"2024-05-09T06:30:36.587612+0000","last_peered":"2024-05-09T06:30:36.587612+0000","last_clean":"2024-05-09T06:30:36.587612+0000","last_became_active":"2024-05-09T06:30:35.602138+0000","last_became_peered":"2024-05-09T06:30:35.602138+0000","last_unstale":"2024-05-09T06:30:36.587612+0000","last_undegraded":"2024-05-09T06:30:36.587612+0000","last_fullsized":"2024-05-09T06:30:36.587612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:01.297028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815643+0000","last_change":"2024-05-09T06:30:35.599617+0000","last_active":"2024-05-09T06:30:36.815643+0000","last_peered":"2024-05-09T06:30:36.815643+0000","last_clean":"2024-05-09T06:30:36.815643+0000","last_became_active":"2024-05-09T06:30:35.599320+0000","last_became_peered":"2024-05-09T06:30:35.599320+0000","last_unstale":"2024-05-09T06:30:36.815643+0000","last_undegraded":"2024-05-09T06:30:36.815643+0000","last_fullsized":"2024-05-09T06:30:36.815643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:15.099298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815510+0000","last_change":"2024-05-09T06:30:35.600441+0000","last_active":"2024-05-09T06:30:36.815510+0000","last_peered":"2024-05-09T06:30:36.815510+0000","last_clean":"2024-05-09T06:30:36.815510+0000","last_became_active":"2024-05-09T06:30:35.600157+0000","last_became_peered":"2024-05-09T06:30:35.600157+0000","last_unstale":"2024-05-09T06:30:36.815510+0000","last_undegraded":"2024-05-09T06:30:36.815510+0000","last_fullsized":"2024-05-09T06:30:36.815510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:31.264391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582786+0000","last_change":"2024-05-09T06:30:35.596162+0000","last_active":"2024-05-09T06:30:36.582786+0000","last_peered":"2024-05-09T06:30:36.582786+0000","last_clean":"2024-05-09T06:30:36.582786+0000","last_became_active":"2024-05-09T06:30:35.595860+0000","last_became_peered":"2024-05-09T06:30:35.595860+0000","last_unstale":"2024-05-09T06:30:36.582786+0000","last_undegraded":"2024-05-09T06:30:36.582786+0000","last_fullsized":"2024-05-09T06:30:36.582786+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:51.680007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587912+0000","last_change":"2024-05-09T06:30:35.603174+0000","last_active":"2024-05-09T06:30:36.587912+0000","last_peered":"2024-05-09T06:30:36.587912+0000","last_clean":"2024-05-09T06:30:36.587912+0000","last_became_active":"2024-05-09T06:30:35.602852+0000","last_became_peered":"2024-05-09T06:30:35.602852+0000","last_unstale":"2024-05-09T06:30:36.587912+0000","last_undegraded":"2024-05-09T06:30:36.587912+0000","last_fullsized":"2024-05-09T06:30:36.587912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:53.676819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587996+0000","last_change":"2024-05-09T06:30:35.599007+0000","last_active":"2024-05-09T06:30:36.587996+0000","last_peered":"2024-05-09T06:30:36.587996+0000","last_clean":"2024-05-09T06:30:36.587996+0000","last_became_active":"2024-05-09T06:30:35.598617+0000","last_became_peered":"2024-05-09T06:30:35.598617+0000","last_unstale":"2024-05-09T06:30:36.587996+0000","last_undegraded":"2024-05-09T06:30:36.587996+0000","last_fullsized":"2024-05-09T06:30:36.587996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:44.024185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815290+0000","last_change":"2024-05-09T06:30:35.599066+0000","last_active":"2024-05-09T06:30:36.815290+0000","last_peered":"2024-05-09T06:30:36.815290+0000","last_clean":"2024-05-09T06:30:36.815290+0000","last_became_active":"2024-05-09T06:30:35.598726+0000","last_became_peered":"2024-05-09T06:30:35.598726+0000","last_unstale":"2024-05-09T06:30:36.815290+0000","last_undegraded":"2024-05-09T06:30:36.815290+0000","last_fullsized":"2024-05-09T06:30:36.815290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:24.617044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815078+0000","last_change":"2024-05-09T06:30:35.604711+0000","last_active":"2024-05-09T06:30:36.815078+0000","last_peered":"2024-05-09T06:30:36.815078+0000","last_clean":"2024-05-09T06:30:36.815078+0000","last_became_active":"2024-05-09T06:30:35.604392+0000","last_became_peered":"2024-05-09T06:30:35.604392+0000","last_unstale":"2024-05-09T06:30:36.815078+0000","last_undegraded":"2024-05-09T06:30:36.815078+0000","last_fullsized":"2024-05-09T06:30:36.815078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:22.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589975+0000","last_change":"2024-05-09T06:30:35.589521+0000","last_active":"2024-05-09T06:30:36.589975+0000","last_peered":"2024-05-09T06:30:36.589975+0000","last_clean":"2024-05-09T06:30:36.589975+0000","last_became_active":"2024-05-09T06:30:35.588794+0000","last_became_peered":"2024-05-09T06:30:35.588794+0000","last_unstale":"2024-05-09T06:30:36.589975+0000","last_undegraded":"2024-05-09T06:30:36.589975+0000","last_fullsized":"2024-05-09T06:30:36.589975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:13.171268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815309+0000","last_change":"2024-05-09T06:30:35.599410+0000","last_active":"2024-05-09T06:30:36.815309+0000","last_peered":"2024-05-09T06:30:36.815309+0000","last_clean":"2024-05-09T06:30:36.815309+0000","last_became_active":"2024-05-09T06:30:35.599073+0000","last_became_peered":"2024-05-09T06:30:35.599073+0000","last_unstale":"2024-05-09T06:30:36.815309+0000","last_undegraded":"2024-05-09T06:30:36.815309+0000","last_fullsized":"2024-05-09T06:30:36.815309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:26.303772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582929+0000","last_change":"2024-05-09T06:30:35.595848+0000","last_active":"2024-05-09T06:30:36.582929+0000","last_peered":"2024-05-09T06:30:36.582929+0000","last_clean":"2024-05-09T06:30:36.582929+0000","last_became_active":"2024-05-09T06:30:35.595562+0000","last_became_peered":"2024-05-09T06:30:35.595562+0000","last_unstale":"2024-05-09T06:30:36.582929+0000","last_undegraded":"2024-05-09T06:30:36.582929+0000","last_fullsized":"2024-05-09T06:30:36.582929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:58.128009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733821+0000","last_change":"2024-05-09T06:30:35.579805+0000","last_active":"2024-05-09T06:30:36.733821+0000","last_peered":"2024-05-09T06:30:36.733821+0000","last_clean":"2024-05-09T06:30:36.733821+0000","last_became_active":"2024-05-09T06:30:35.579480+0000","last_became_peered":"2024-05-09T06:30:35.579480+0000","last_unstale":"2024-05-09T06:30:36.733821+0000","last_undegraded":"2024-05-09T06:30:36.733821+0000","last_fullsized":"2024-05-09T06:30:36.733821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:09.630054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579401+0000","last_change":"2024-05-09T06:30:35.583903+0000","last_active":"2024-05-09T06:30:36.579401+0000","last_peered":"2024-05-09T06:30:36.579401+0000","last_clean":"2024-05-09T06:30:36.579401+0000","last_became_active":"2024-05-09T06:30:35.583582+0000","last_became_peered":"2024-05-09T06:30:35.583582+0000","last_unstale":"2024-05-09T06:30:36.579401+0000","last_undegraded":"2024-05-09T06:30:36.579401+0000","last_fullsized":"2024-05-09T06:30:36.579401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:24.757216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588172+0000","last_change":"2024-05-09T06:30:35.594510+0000","last_active":"2024-05-09T06:30:36.588172+0000","last_peered":"2024-05-09T06:30:36.588172+0000","last_clean":"2024-05-09T06:30:36.588172+0000","last_became_active":"2024-05-09T06:30:35.594231+0000","last_became_peered":"2024-05-09T06:30:35.594231+0000","last_unstale":"2024-05-09T06:30:36.588172+0000","last_undegraded":"2024-05-09T06:30:36.588172+0000","last_fullsized":"2024-05-09T06:30:36.588172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:32.185578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579301+0000","last_change":"2024-05-09T06:30:35.600502+0000","last_active":"2024-05-09T06:30:36.579301+0000","last_peered":"2024-05-09T06:30:36.579301+0000","last_clean":"2024-05-09T06:30:36.579301+0000","last_became_active":"2024-05-09T06:30:35.600169+0000","last_became_peered":"2024-05-09T06:30:35.600169+0000","last_unstale":"2024-05-09T06:30:36.579301+0000","last_undegraded":"2024-05-09T06:30:36.579301+0000","last_fullsized":"2024-05-09T06:30:36.579301+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:18.100098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579295+0000","last_change":"2024-05-09T06:30:35.600134+0000","last_active":"2024-05-09T06:30:36.579295+0000","last_peered":"2024-05-09T06:30:36.579295+0000","last_clean":"2024-05-09T06:30:36.579295+0000","last_became_active":"2024-05-09T06:30:35.598796+0000","last_became_peered":"2024-05-09T06:30:35.598796+0000","last_unstale":"2024-05-09T06:30:36.579295+0000","last_undegraded":"2024-05-09T06:30:36.579295+0000","last_fullsized":"2024-05-09T06:30:36.579295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:41.043668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815273+0000","last_change":"2024-05-09T06:30:35.598841+0000","last_active":"2024-05-09T06:30:36.815273+0000","last_peered":"2024-05-09T06:30:36.815273+0000","last_clean":"2024-05-09T06:30:36.815273+0000","last_became_active":"2024-05-09T06:30:35.598550+0000","last_became_peered":"2024-05-09T06:30:35.598550+0000","last_unstale":"2024-05-09T06:30:36.815273+0000","last_undegraded":"2024-05-09T06:30:36.815273+0000","last_fullsized":"2024-05-09T06:30:36.815273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:53.660591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588179+0000","last_change":"2024-05-09T06:30:35.591723+0000","last_active":"2024-05-09T06:30:36.588179+0000","last_peered":"2024-05-09T06:30:36.588179+0000","last_clean":"2024-05-09T06:30:36.588179+0000","last_became_active":"2024-05-09T06:30:35.591371+0000","last_became_peered":"2024-05-09T06:30:35.591371+0000","last_unstale":"2024-05-09T06:30:36.588179+0000","last_undegraded":"2024-05-09T06:30:36.588179+0000","last_fullsized":"2024-05-09T06:30:36.588179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:15.099298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588111+0000","last_change":"2024-05-09T06:30:35.590430+0000","last_active":"2024-05-09T06:30:36.588111+0000","last_peered":"2024-05-09T06:30:36.588111+0000","last_clean":"2024-05-09T06:30:36.588111+0000","last_became_active":"2024-05-09T06:30:35.590124+0000","last_became_peered":"2024-05-09T06:30:35.590124+0000","last_unstale":"2024-05-09T06:30:36.588111+0000","last_undegraded":"2024-05-09T06:30:36.588111+0000","last_fullsized":"2024-05-09T06:30:36.588111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:31.264391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582797+0000","last_change":"2024-05-09T06:30:35.581626+0000","last_active":"2024-05-09T06:30:36.582797+0000","last_peered":"2024-05-09T06:30:36.582797+0000","last_clean":"2024-05-09T06:30:36.582797+0000","last_became_active":"2024-05-09T06:30:35.581262+0000","last_became_peered":"2024-05-09T06:30:35.581262+0000","last_unstale":"2024-05-09T06:30:36.582797+0000","last_undegraded":"2024-05-09T06:30:36.582797+0000","last_fullsized":"2024-05-09T06:30:36.582797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:55.011652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815598+0000","last_change":"2024-05-09T06:30:35.598169+0000","last_active":"2024-05-09T06:30:36.815598+0000","last_peered":"2024-05-09T06:30:36.815598+0000","last_clean":"2024-05-09T06:30:36.815598+0000","last_became_active":"2024-05-09T06:30:35.597789+0000","last_became_peered":"2024-05-09T06:30:35.597789+0000","last_unstale":"2024-05-09T06:30:36.815598+0000","last_undegraded":"2024-05-09T06:30:36.815598+0000","last_fullsized":"2024-05-09T06:30:36.815598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:11.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815158+0000","last_change":"2024-05-09T06:30:35.606177+0000","last_active":"2024-05-09T06:30:36.815158+0000","last_peered":"2024-05-09T06:30:36.815158+0000","last_clean":"2024-05-09T06:30:36.815158+0000","last_became_active":"2024-05-09T06:30:35.605683+0000","last_became_peered":"2024-05-09T06:30:35.605683+0000","last_unstale":"2024-05-09T06:30:36.815158+0000","last_undegraded":"2024-05-09T06:30:36.815158+0000","last_fullsized":"2024-05-09T06:30:36.815158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:01.297028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587673+0000","last_change":"2024-05-09T06:30:35.602388+0000","last_active":"2024-05-09T06:30:36.587673+0000","last_peered":"2024-05-09T06:30:36.587673+0000","last_clean":"2024-05-09T06:30:36.587673+0000","last_became_active":"2024-05-09T06:30:35.602090+0000","last_became_peered":"2024-05-09T06:30:35.602090+0000","last_unstale":"2024-05-09T06:30:36.587673+0000","last_undegraded":"2024-05-09T06:30:36.587673+0000","last_fullsized":"2024-05-09T06:30:36.587673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:53.660591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582472+0000","last_change":"2024-05-09T06:30:35.580345+0000","last_active":"2024-05-09T06:30:36.582472+0000","last_peered":"2024-05-09T06:30:36.582472+0000","last_clean":"2024-05-09T06:30:36.582472+0000","last_became_active":"2024-05-09T06:30:35.580072+0000","last_became_peered":"2024-05-09T06:30:35.580072+0000","last_unstale":"2024-05-09T06:30:36.582472+0000","last_undegraded":"2024-05-09T06:30:36.582472+0000","last_fullsized":"2024-05-09T06:30:36.582472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:05.633077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814744+0000","last_change":"2024-05-09T06:30:35.580194+0000","last_active":"2024-05-09T06:30:36.814744+0000","last_peered":"2024-05-09T06:30:36.814744+0000","last_clean":"2024-05-09T06:30:36.814744+0000","last_became_active":"2024-05-09T06:30:35.579875+0000","last_became_peered":"2024-05-09T06:30:35.579875+0000","last_unstale":"2024-05-09T06:30:36.814744+0000","last_undegraded":"2024-05-09T06:30:36.814744+0000","last_fullsized":"2024-05-09T06:30:36.814744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:04.454302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733762+0000","last_change":"2024-05-09T06:30:35.595138+0000","last_active":"2024-05-09T06:30:36.733762+0000","last_peered":"2024-05-09T06:30:36.733762+0000","last_clean":"2024-05-09T06:30:36.733762+0000","last_became_active":"2024-05-09T06:30:35.593256+0000","last_became_peered":"2024-05-09T06:30:35.593256+0000","last_unstale":"2024-05-09T06:30:36.733762+0000","last_undegraded":"2024-05-09T06:30:36.733762+0000","last_fullsized":"2024-05-09T06:30:36.733762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:01.765676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815476+0000","last_change":"2024-05-09T06:30:34.586203+0000","last_active":"2024-05-09T06:30:36.815476+0000","last_peered":"2024-05-09T06:30:36.815476+0000","last_clean":"2024-05-09T06:30:36.815476+0000","last_became_active":"2024-05-09T06:30:34.585885+0000","last_became_peered":"2024-05-09T06:30:34.585885+0000","last_unstale":"2024-05-09T06:30:36.815476+0000","last_undegraded":"2024-05-09T06:30:36.815476+0000","last_fullsized":"2024-05-09T06:30:36.815476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:54.007366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583049+0000","last_change":"2024-05-09T06:30:35.584528+0000","last_active":"2024-05-09T06:30:36.583049+0000","last_peered":"2024-05-09T06:30:36.583049+0000","last_clean":"2024-05-09T06:30:36.583049+0000","last_became_active":"2024-05-09T06:30:35.584202+0000","last_became_peered":"2024-05-09T06:30:35.584202+0000","last_unstale":"2024-05-09T06:30:36.583049+0000","last_undegraded":"2024-05-09T06:30:36.583049+0000","last_fullsized":"2024-05-09T06:30:36.583049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:35.172202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815547+0000","last_change":"2024-05-09T06:30:34.584707+0000","last_active":"2024-05-09T06:30:36.815547+0000","last_peered":"2024-05-09T06:30:36.815547+0000","last_clean":"2024-05-09T06:30:36.815547+0000","last_became_active":"2024-05-09T06:30:34.584350+0000","last_became_peered":"2024-05-09T06:30:34.584350+0000","last_unstale":"2024-05-09T06:30:36.815547+0000","last_undegraded":"2024-05-09T06:30:36.815547+0000","last_fullsized":"2024-05-09T06:30:36.815547+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:14.095011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815384+0000","last_change":"2024-05-09T06:30:35.580204+0000","last_active":"2024-05-09T06:30:36.815384+0000","last_peered":"2024-05-09T06:30:36.815384+0000","last_clean":"2024-05-09T06:30:36.815384+0000","last_became_active":"2024-05-09T06:30:35.579888+0000","last_became_peered":"2024-05-09T06:30:35.579888+0000","last_unstale":"2024-05-09T06:30:36.815384+0000","last_undegraded":"2024-05-09T06:30:36.815384+0000","last_fullsized":"2024-05-09T06:30:36.815384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:23.661621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815425+0000","last_change":"2024-05-09T06:30:34.579101+0000","last_active":"2024-05-09T06:30:36.815425+0000","last_peered":"2024-05-09T06:30:36.815425+0000","last_clean":"2024-05-09T06:30:36.815425+0000","last_became_active":"2024-05-09T06:30:34.578809+0000","last_became_peered":"2024-05-09T06:30:34.578809+0000","last_unstale":"2024-05-09T06:30:36.815425+0000","last_undegraded":"2024-05-09T06:30:36.815425+0000","last_fullsized":"2024-05-09T06:30:36.815425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:43.019898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815500+0000","last_change":"2024-05-09T06:30:35.604960+0000","last_active":"2024-05-09T06:30:36.815500+0000","last_peered":"2024-05-09T06:30:36.815500+0000","last_clean":"2024-05-09T06:30:36.815500+0000","last_became_active":"2024-05-09T06:30:35.604530+0000","last_became_peered":"2024-05-09T06:30:35.604530+0000","last_unstale":"2024-05-09T06:30:36.815500+0000","last_undegraded":"2024-05-09T06:30:36.815500+0000","last_fullsized":"2024-05-09T06:30:36.815500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:44.024185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815187+0000","last_change":"2024-05-09T06:30:34.578459+0000","last_active":"2024-05-09T06:30:36.815187+0000","last_peered":"2024-05-09T06:30:36.815187+0000","last_clean":"2024-05-09T06:30:36.815187+0000","last_became_active":"2024-05-09T06:30:34.578161+0000","last_became_peered":"2024-05-09T06:30:34.578161+0000","last_unstale":"2024-05-09T06:30:36.815187+0000","last_undegraded":"2024-05-09T06:30:36.815187+0000","last_fullsized":"2024-05-09T06:30:36.815187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:39.265772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589992+0000","last_change":"2024-05-09T06:30:35.583897+0000","last_active":"2024-05-09T06:30:36.589992+0000","last_peered":"2024-05-09T06:30:36.589992+0000","last_clean":"2024-05-09T06:30:36.589992+0000","last_became_active":"2024-05-09T06:30:35.583628+0000","last_became_peered":"2024-05-09T06:30:35.583628+0000","last_unstale":"2024-05-09T06:30:36.589992+0000","last_undegraded":"2024-05-09T06:30:36.589992+0000","last_fullsized":"2024-05-09T06:30:36.589992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:23.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579636+0000","last_change":"2024-05-09T06:30:34.578866+0000","last_active":"2024-05-09T06:30:36.579636+0000","last_peered":"2024-05-09T06:30:36.579636+0000","last_clean":"2024-05-09T06:30:36.579636+0000","last_became_active":"2024-05-09T06:30:34.578584+0000","last_became_peered":"2024-05-09T06:30:34.578584+0000","last_unstale":"2024-05-09T06:30:36.579636+0000","last_undegraded":"2024-05-09T06:30:36.579636+0000","last_fullsized":"2024-05-09T06:30:36.579636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:03.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815290+0000","last_change":"2024-05-09T06:30:35.599294+0000","last_active":"2024-05-09T06:30:36.815290+0000","last_peered":"2024-05-09T06:30:36.815290+0000","last_clean":"2024-05-09T06:30:36.815290+0000","last_became_active":"2024-05-09T06:30:35.598955+0000","last_became_peered":"2024-05-09T06:30:35.598955+0000","last_unstale":"2024-05-09T06:30:36.815290+0000","last_undegraded":"2024-05-09T06:30:36.815290+0000","last_fullsized":"2024-05-09T06:30:36.815290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:09.630054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579414+0000","last_change":"2024-05-09T06:30:34.573669+0000","last_active":"2024-05-09T06:30:36.579414+0000","last_peered":"2024-05-09T06:30:36.579414+0000","last_clean":"2024-05-09T06:30:36.579414+0000","last_became_active":"2024-05-09T06:30:34.573372+0000","last_became_peered":"2024-05-09T06:30:34.573372+0000","last_unstale":"2024-05-09T06:30:36.579414+0000","last_undegraded":"2024-05-09T06:30:36.579414+0000","last_fullsized":"2024-05-09T06:30:36.579414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:00:48.303315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733414+0000","last_change":"2024-05-09T06:30:35.583522+0000","last_active":"2024-05-09T06:30:36.733414+0000","last_peered":"2024-05-09T06:30:36.733414+0000","last_clean":"2024-05-09T06:30:36.733414+0000","last_became_active":"2024-05-09T06:30:35.582877+0000","last_became_peered":"2024-05-09T06:30:35.582877+0000","last_unstale":"2024-05-09T06:30:36.733414+0000","last_undegraded":"2024-05-09T06:30:36.733414+0000","last_fullsized":"2024-05-09T06:30:36.733414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:43.552330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579299+0000","last_change":"2024-05-09T06:30:34.585694+0000","last_active":"2024-05-09T06:30:36.579299+0000","last_peered":"2024-05-09T06:30:36.579299+0000","last_clean":"2024-05-09T06:30:36.579299+0000","last_became_active":"2024-05-09T06:30:34.585395+0000","last_became_peered":"2024-05-09T06:30:34.585395+0000","last_unstale":"2024-05-09T06:30:36.579299+0000","last_undegraded":"2024-05-09T06:30:36.579299+0000","last_fullsized":"2024-05-09T06:30:36.579299+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:02.835777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579579+0000","last_change":"2024-05-09T06:30:35.584730+0000","last_active":"2024-05-09T06:30:36.579579+0000","last_peered":"2024-05-09T06:30:36.579579+0000","last_clean":"2024-05-09T06:30:36.579579+0000","last_became_active":"2024-05-09T06:30:35.584401+0000","last_became_peered":"2024-05-09T06:30:35.584401+0000","last_unstale":"2024-05-09T06:30:36.579579+0000","last_undegraded":"2024-05-09T06:30:36.579579+0000","last_fullsized":"2024-05-09T06:30:36.579579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:09.845887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733321+0000","last_change":"2024-05-09T06:30:34.585396+0000","last_active":"2024-05-09T06:30:36.733321+0000","last_peered":"2024-05-09T06:30:36.733321+0000","last_clean":"2024-05-09T06:30:36.733321+0000","last_became_active":"2024-05-09T06:30:34.585042+0000","last_became_peered":"2024-05-09T06:30:34.585042+0000","last_unstale":"2024-05-09T06:30:36.733321+0000","last_undegraded":"2024-05-09T06:30:36.733321+0000","last_fullsized":"2024-05-09T06:30:36.733321+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:05.148173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815714+0000","last_change":"2024-05-09T06:30:35.585358+0000","last_active":"2024-05-09T06:30:36.815714+0000","last_peered":"2024-05-09T06:30:36.815714+0000","last_clean":"2024-05-09T06:30:36.815714+0000","last_became_active":"2024-05-09T06:30:35.584981+0000","last_became_peered":"2024-05-09T06:30:35.584981+0000","last_unstale":"2024-05-09T06:30:36.815714+0000","last_undegraded":"2024-05-09T06:30:36.815714+0000","last_fullsized":"2024-05-09T06:30:36.815714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:11.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815645+0000","last_change":"2024-05-09T06:30:34.585067+0000","last_active":"2024-05-09T06:30:36.815645+0000","last_peered":"2024-05-09T06:30:36.815645+0000","last_clean":"2024-05-09T06:30:36.815645+0000","last_became_active":"2024-05-09T06:30:34.584344+0000","last_became_peered":"2024-05-09T06:30:34.584344+0000","last_unstale":"2024-05-09T06:30:36.815645+0000","last_undegraded":"2024-05-09T06:30:36.815645+0000","last_fullsized":"2024-05-09T06:30:36.815645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:55.676974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579411+0000","last_change":"2024-05-09T06:30:35.586882+0000","last_active":"2024-05-09T06:30:36.579411+0000","last_peered":"2024-05-09T06:30:36.579411+0000","last_clean":"2024-05-09T06:30:36.579411+0000","last_became_active":"2024-05-09T06:30:35.586559+0000","last_became_peered":"2024-05-09T06:30:35.586559+0000","last_unstale":"2024-05-09T06:30:36.579411+0000","last_undegraded":"2024-05-09T06:30:36.579411+0000","last_fullsized":"2024-05-09T06:30:36.579411+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:50:16.115738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583044+0000","last_change":"2024-05-09T06:30:34.581563+0000","last_active":"2024-05-09T06:30:36.583044+0000","last_peered":"2024-05-09T06:30:36.583044+0000","last_clean":"2024-05-09T06:30:36.583044+0000","last_became_active":"2024-05-09T06:30:34.581256+0000","last_became_peered":"2024-05-09T06:30:34.581256+0000","last_unstale":"2024-05-09T06:30:36.583044+0000","last_undegraded":"2024-05-09T06:30:36.583044+0000","last_fullsized":"2024-05-09T06:30:36.583044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:03.008622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733889+0000","last_change":"2024-05-09T06:30:35.593660+0000","last_active":"2024-05-09T06:30:36.733889+0000","last_peered":"2024-05-09T06:30:36.733889+0000","last_clean":"2024-05-09T06:30:36.733889+0000","last_became_active":"2024-05-09T06:30:35.593347+0000","last_became_peered":"2024-05-09T06:30:35.593347+0000","last_unstale":"2024-05-09T06:30:36.733889+0000","last_undegraded":"2024-05-09T06:30:36.733889+0000","last_fullsized":"2024-05-09T06:30:36.733889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.386784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815446+0000","last_change":"2024-05-09T06:30:34.585269+0000","last_active":"2024-05-09T06:30:36.815446+0000","last_peered":"2024-05-09T06:30:36.815446+0000","last_clean":"2024-05-09T06:30:36.815446+0000","last_became_active":"2024-05-09T06:30:34.584966+0000","last_became_peered":"2024-05-09T06:30:34.584966+0000","last_unstale":"2024-05-09T06:30:36.815446+0000","last_undegraded":"2024-05-09T06:30:36.815446+0000","last_fullsized":"2024-05-09T06:30:36.815446+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:41.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.583099+0000","last_change":"2024-05-09T06:30:35.592954+0000","last_active":"2024-05-09T06:30:36.583099+0000","last_peered":"2024-05-09T06:30:36.583099+0000","last_clean":"2024-05-09T06:30:36.583099+0000","last_became_active":"2024-05-09T06:30:35.592664+0000","last_became_peered":"2024-05-09T06:30:35.592664+0000","last_unstale":"2024-05-09T06:30:36.583099+0000","last_undegraded":"2024-05-09T06:30:36.583099+0000","last_fullsized":"2024-05-09T06:30:36.583099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:25.535201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815687+0000","last_change":"2024-05-09T06:30:34.584317+0000","last_active":"2024-05-09T06:30:36.815687+0000","last_peered":"2024-05-09T06:30:36.815687+0000","last_clean":"2024-05-09T06:30:36.815687+0000","last_became_active":"2024-05-09T06:30:34.584002+0000","last_became_peered":"2024-05-09T06:30:34.584002+0000","last_unstale":"2024-05-09T06:30:36.815687+0000","last_undegraded":"2024-05-09T06:30:36.815687+0000","last_fullsized":"2024-05-09T06:30:36.815687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:57.123722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582464+0000","last_change":"2024-05-09T06:30:35.601825+0000","last_active":"2024-05-09T06:30:36.582464+0000","last_peered":"2024-05-09T06:30:36.582464+0000","last_clean":"2024-05-09T06:30:36.582464+0000","last_became_active":"2024-05-09T06:30:35.600566+0000","last_became_peered":"2024-05-09T06:30:35.600566+0000","last_unstale":"2024-05-09T06:30:36.582464+0000","last_undegraded":"2024-05-09T06:30:36.582464+0000","last_fullsized":"2024-05-09T06:30:36.582464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:13.066743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815307+0000","last_change":"2024-05-09T06:30:34.586092+0000","last_active":"2024-05-09T06:30:36.815307+0000","last_peered":"2024-05-09T06:30:36.815307+0000","last_clean":"2024-05-09T06:30:36.815307+0000","last_became_active":"2024-05-09T06:30:34.585765+0000","last_became_peered":"2024-05-09T06:30:34.585765+0000","last_unstale":"2024-05-09T06:30:36.815307+0000","last_undegraded":"2024-05-09T06:30:36.815307+0000","last_fullsized":"2024-05-09T06:30:36.815307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:42.548043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587605+0000","last_change":"2024-05-09T06:30:35.595770+0000","last_active":"2024-05-09T06:30:36.587605+0000","last_peered":"2024-05-09T06:30:36.587605+0000","last_clean":"2024-05-09T06:30:36.587605+0000","last_became_active":"2024-05-09T06:30:35.595500+0000","last_became_peered":"2024-05-09T06:30:35.595500+0000","last_unstale":"2024-05-09T06:30:36.587605+0000","last_undegraded":"2024-05-09T06:30:36.587605+0000","last_fullsized":"2024-05-09T06:30:36.587605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:22.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588281+0000","last_change":"2024-05-09T06:30:34.582151+0000","last_active":"2024-05-09T06:30:36.588281+0000","last_peered":"2024-05-09T06:30:36.588281+0000","last_clean":"2024-05-09T06:30:36.588281+0000","last_became_active":"2024-05-09T06:30:34.581849+0000","last_became_peered":"2024-05-09T06:30:34.581849+0000","last_unstale":"2024-05-09T06:30:36.588281+0000","last_undegraded":"2024-05-09T06:30:36.588281+0000","last_fullsized":"2024-05-09T06:30:36.588281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:42.548043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733465+0000","last_change":"2024-05-09T06:30:35.583325+0000","last_active":"2024-05-09T06:30:36.733465+0000","last_peered":"2024-05-09T06:30:36.733465+0000","last_clean":"2024-05-09T06:30:36.733465+0000","last_became_active":"2024-05-09T06:30:35.582132+0000","last_became_peered":"2024-05-09T06:30:35.582132+0000","last_unstale":"2024-05-09T06:30:36.733465+0000","last_undegraded":"2024-05-09T06:30:36.733465+0000","last_fullsized":"2024-05-09T06:30:36.733465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:48.194063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815273+0000","last_change":"2024-05-09T06:30:34.584924+0000","last_active":"2024-05-09T06:30:36.815273+0000","last_peered":"2024-05-09T06:30:36.815273+0000","last_clean":"2024-05-09T06:30:36.815273+0000","last_became_active":"2024-05-09T06:30:34.584194+0000","last_became_peered":"2024-05-09T06:30:34.584194+0000","last_unstale":"2024-05-09T06:30:36.815273+0000","last_undegraded":"2024-05-09T06:30:36.815273+0000","last_fullsized":"2024-05-09T06:30:36.815273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:24.530914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815098+0000","last_change":"2024-05-09T06:30:35.602403+0000","last_active":"2024-05-09T06:30:36.815098+0000","last_peered":"2024-05-09T06:30:36.815098+0000","last_clean":"2024-05-09T06:30:36.815098+0000","last_became_active":"2024-05-09T06:30:35.602105+0000","last_became_peered":"2024-05-09T06:30:35.602105+0000","last_unstale":"2024-05-09T06:30:36.815098+0000","last_undegraded":"2024-05-09T06:30:36.815098+0000","last_fullsized":"2024-05-09T06:30:36.815098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:51.940031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582377+0000","last_change":"2024-05-09T06:30:34.586240+0000","last_active":"2024-05-09T06:30:36.582377+0000","last_peered":"2024-05-09T06:30:36.582377+0000","last_clean":"2024-05-09T06:30:36.582377+0000","last_became_active":"2024-05-09T06:30:34.585916+0000","last_became_peered":"2024-05-09T06:30:34.585916+0000","last_unstale":"2024-05-09T06:30:36.582377+0000","last_undegraded":"2024-05-09T06:30:36.582377+0000","last_fullsized":"2024-05-09T06:30:36.582377+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:51.243721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815598+0000","last_change":"2024-05-09T06:30:35.599942+0000","last_active":"2024-05-09T06:30:36.815598+0000","last_peered":"2024-05-09T06:30:36.815598+0000","last_clean":"2024-05-09T06:30:36.815598+0000","last_became_active":"2024-05-09T06:30:35.599578+0000","last_became_peered":"2024-05-09T06:30:35.599578+0000","last_unstale":"2024-05-09T06:30:36.815598+0000","last_undegraded":"2024-05-09T06:30:36.815598+0000","last_fullsized":"2024-05-09T06:30:36.815598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:32.185578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588166+0000","last_change":"2024-05-09T06:30:34.581592+0000","last_active":"2024-05-09T06:30:36.588166+0000","last_peered":"2024-05-09T06:30:36.588166+0000","last_clean":"2024-05-09T06:30:36.588166+0000","last_became_active":"2024-05-09T06:30:34.581306+0000","last_became_peered":"2024-05-09T06:30:34.581306+0000","last_unstale":"2024-05-09T06:30:36.588166+0000","last_undegraded":"2024-05-09T06:30:36.588166+0000","last_fullsized":"2024-05-09T06:30:36.588166+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:58:41.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733849+0000","last_change":"2024-05-09T06:30:35.589096+0000","last_active":"2024-05-09T06:30:36.733849+0000","last_peered":"2024-05-09T06:30:36.733849+0000","last_clean":"2024-05-09T06:30:36.733849+0000","last_became_active":"2024-05-09T06:30:35.588760+0000","last_became_peered":"2024-05-09T06:30:35.588760+0000","last_unstale":"2024-05-09T06:30:36.733849+0000","last_undegraded":"2024-05-09T06:30:36.733849+0000","last_fullsized":"2024-05-09T06:30:36.733849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:11.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815576+0000","last_change":"2024-05-09T06:30:34.585960+0000","last_active":"2024-05-09T06:30:36.815576+0000","last_peered":"2024-05-09T06:30:36.815576+0000","last_clean":"2024-05-09T06:30:36.815576+0000","last_became_active":"2024-05-09T06:30:34.585629+0000","last_became_peered":"2024-05-09T06:30:34.585629+0000","last_unstale":"2024-05-09T06:30:36.815576+0000","last_undegraded":"2024-05-09T06:30:36.815576+0000","last_fullsized":"2024-05-09T06:30:36.815576+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:34.167915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582952+0000","last_change":"2024-05-09T06:30:35.582883+0000","last_active":"2024-05-09T06:30:36.582952+0000","last_peered":"2024-05-09T06:30:36.582952+0000","last_clean":"2024-05-09T06:30:36.582952+0000","last_became_active":"2024-05-09T06:30:35.582563+0000","last_became_peered":"2024-05-09T06:30:35.582563+0000","last_unstale":"2024-05-09T06:30:36.582952+0000","last_undegraded":"2024-05-09T06:30:36.582952+0000","last_fullsized":"2024-05-09T06:30:36.582952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:08.940461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815588+0000","last_change":"2024-05-09T06:30:34.584535+0000","last_active":"2024-05-09T06:30:36.815588+0000","last_peered":"2024-05-09T06:30:36.815588+0000","last_clean":"2024-05-09T06:30:36.815588+0000","last_became_active":"2024-05-09T06:30:34.584226+0000","last_became_peered":"2024-05-09T06:30:34.584226+0000","last_unstale":"2024-05-09T06:30:36.815588+0000","last_undegraded":"2024-05-09T06:30:36.815588+0000","last_fullsized":"2024-05-09T06:30:36.815588+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:04.525662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733821+0000","last_change":"2024-05-09T06:30:35.591377+0000","last_active":"2024-05-09T06:30:36.733821+0000","last_peered":"2024-05-09T06:30:36.733821+0000","last_clean":"2024-05-09T06:30:36.733821+0000","last_became_active":"2024-05-09T06:30:35.591115+0000","last_became_peered":"2024-05-09T06:30:35.591115+0000","last_unstale":"2024-05-09T06:30:36.733821+0000","last_undegraded":"2024-05-09T06:30:36.733821+0000","last_fullsized":"2024-05-09T06:30:36.733821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:38:26.303772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579414+0000","last_change":"2024-05-09T06:30:34.579600+0000","last_active":"2024-05-09T06:30:36.579414+0000","last_peered":"2024-05-09T06:30:36.579414+0000","last_clean":"2024-05-09T06:30:36.579414+0000","last_became_active":"2024-05-09T06:30:34.579332+0000","last_became_peered":"2024-05-09T06:30:34.579332+0000","last_unstale":"2024-05-09T06:30:36.579414+0000","last_undegraded":"2024-05-09T06:30:36.579414+0000","last_fullsized":"2024-05-09T06:30:36.579414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:22.685792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814776+0000","last_change":"2024-05-09T06:30:35.580810+0000","last_active":"2024-05-09T06:30:36.814776+0000","last_peered":"2024-05-09T06:30:36.814776+0000","last_clean":"2024-05-09T06:30:36.814776+0000","last_became_active":"2024-05-09T06:30:35.580084+0000","last_became_peered":"2024-05-09T06:30:35.580084+0000","last_unstale":"2024-05-09T06:30:36.814776+0000","last_undegraded":"2024-05-09T06:30:36.814776+0000","last_fullsized":"2024-05-09T06:30:36.814776+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:14.667295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814922+0000","last_change":"2024-05-09T06:30:34.579697+0000","last_active":"2024-05-09T06:30:36.814922+0000","last_peered":"2024-05-09T06:30:36.814922+0000","last_clean":"2024-05-09T06:30:36.814922+0000","last_became_active":"2024-05-09T06:30:34.579396+0000","last_became_peered":"2024-05-09T06:30:34.579396+0000","last_unstale":"2024-05-09T06:30:36.814922+0000","last_undegraded":"2024-05-09T06:30:36.814922+0000","last_fullsized":"2024-05-09T06:30:36.814922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:52.672533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587632+0000","last_change":"2024-05-09T06:30:35.591336+0000","last_active":"2024-05-09T06:30:36.587632+0000","last_peered":"2024-05-09T06:30:36.587632+0000","last_clean":"2024-05-09T06:30:36.587632+0000","last_became_active":"2024-05-09T06:30:35.591044+0000","last_became_peered":"2024-05-09T06:30:35.591044+0000","last_unstale":"2024-05-09T06:30:36.587632+0000","last_undegraded":"2024-05-09T06:30:36.587632+0000","last_fullsized":"2024-05-09T06:30:36.587632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:24.617044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.587797+0000","last_change":"2024-05-09T06:30:34.582245+0000","last_active":"2024-05-09T06:30:36.587797+0000","last_peered":"2024-05-09T06:30:36.587797+0000","last_clean":"2024-05-09T06:30:36.587797+0000","last_became_active":"2024-05-09T06:30:34.581844+0000","last_became_peered":"2024-05-09T06:30:34.581844+0000","last_unstale":"2024-05-09T06:30:36.587797+0000","last_undegraded":"2024-05-09T06:30:36.587797+0000","last_fullsized":"2024-05-09T06:30:36.587797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:47.189776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733638+0000","last_change":"2024-05-09T06:30:35.583250+0000","last_active":"2024-05-09T06:30:36.733638+0000","last_peered":"2024-05-09T06:30:36.733638+0000","last_clean":"2024-05-09T06:30:36.733638+0000","last_became_active":"2024-05-09T06:30:35.582493+0000","last_became_peered":"2024-05-09T06:30:35.582493+0000","last_unstale":"2024-05-09T06:30:36.733638+0000","last_undegraded":"2024-05-09T06:30:36.733638+0000","last_fullsized":"2024-05-09T06:30:36.733638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:51.940031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814883+0000","last_change":"2024-05-09T06:30:34.584692+0000","last_active":"2024-05-09T06:30:36.814883+0000","last_peered":"2024-05-09T06:30:36.814883+0000","last_clean":"2024-05-09T06:30:36.814883+0000","last_became_active":"2024-05-09T06:30:34.584194+0000","last_became_peered":"2024-05-09T06:30:34.584194+0000","last_unstale":"2024-05-09T06:30:36.814883+0000","last_undegraded":"2024-05-09T06:30:36.814883+0000","last_fullsized":"2024-05-09T06:30:36.814883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:31.181292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815156+0000","last_change":"2024-05-09T06:30:35.582542+0000","last_active":"2024-05-09T06:30:36.815156+0000","last_peered":"2024-05-09T06:30:36.815156+0000","last_clean":"2024-05-09T06:30:36.815156+0000","last_became_active":"2024-05-09T06:30:35.582265+0000","last_became_peered":"2024-05-09T06:30:35.582265+0000","last_unstale":"2024-05-09T06:30:36.815156+0000","last_undegraded":"2024-05-09T06:30:36.815156+0000","last_fullsized":"2024-05-09T06:30:36.815156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:35.426267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588313+0000","last_change":"2024-05-09T06:30:34.587377+0000","last_active":"2024-05-09T06:30:36.588313+0000","last_peered":"2024-05-09T06:30:36.588313+0000","last_clean":"2024-05-09T06:30:36.588313+0000","last_became_active":"2024-05-09T06:30:34.587081+0000","last_became_peered":"2024-05-09T06:30:34.587081+0000","last_unstale":"2024-05-09T06:30:36.588313+0000","last_undegraded":"2024-05-09T06:30:36.588313+0000","last_fullsized":"2024-05-09T06:30:36.588313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:04.525662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.590198+0000","last_change":"2024-05-09T06:30:35.581800+0000","last_active":"2024-05-09T06:30:36.590198+0000","last_peered":"2024-05-09T06:30:36.590198+0000","last_clean":"2024-05-09T06:30:36.590198+0000","last_became_active":"2024-05-09T06:30:35.581406+0000","last_became_peered":"2024-05-09T06:30:35.581406+0000","last_unstale":"2024-05-09T06:30:36.590198+0000","last_undegraded":"2024-05-09T06:30:36.590198+0000","last_fullsized":"2024-05-09T06:30:36.590198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:54:49.122341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589624+0000","last_change":"2024-05-09T06:30:34.574936+0000","last_active":"2024-05-09T06:30:36.589624+0000","last_peered":"2024-05-09T06:30:36.589624+0000","last_clean":"2024-05-09T06:30:36.589624+0000","last_became_active":"2024-05-09T06:30:34.574630+0000","last_became_peered":"2024-05-09T06:30:34.574630+0000","last_unstale":"2024-05-09T06:30:36.589624+0000","last_undegraded":"2024-05-09T06:30:36.589624+0000","last_fullsized":"2024-05-09T06:30:36.589624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:24.530914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.588247+0000","last_change":"2024-05-09T06:30:35.589778+0000","last_active":"2024-05-09T06:30:36.588247+0000","last_peered":"2024-05-09T06:30:36.588247+0000","last_clean":"2024-05-09T06:30:36.588247+0000","last_became_active":"2024-05-09T06:30:35.589458+0000","last_became_peered":"2024-05-09T06:30:35.589458+0000","last_unstale":"2024-05-09T06:30:36.588247+0000","last_undegraded":"2024-05-09T06:30:36.588247+0000","last_fullsized":"2024-05-09T06:30:36.588247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:40.270059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815520+0000","last_change":"2024-05-09T06:30:34.575398+0000","last_active":"2024-05-09T06:30:36.815520+0000","last_peered":"2024-05-09T06:30:36.815520+0000","last_clean":"2024-05-09T06:30:36.815520+0000","last_became_active":"2024-05-09T06:30:34.574158+0000","last_became_peered":"2024-05-09T06:30:34.574158+0000","last_unstale":"2024-05-09T06:30:36.815520+0000","last_undegraded":"2024-05-09T06:30:36.815520+0000","last_fullsized":"2024-05-09T06:30:36.815520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:30.260104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733912+0000","last_change":"2024-05-09T06:30:35.590668+0000","last_active":"2024-05-09T06:30:36.733912+0000","last_peered":"2024-05-09T06:30:36.733912+0000","last_clean":"2024-05-09T06:30:36.733912+0000","last_became_active":"2024-05-09T06:30:35.590391+0000","last_became_peered":"2024-05-09T06:30:35.590391+0000","last_unstale":"2024-05-09T06:30:36.733912+0000","last_undegraded":"2024-05-09T06:30:36.733912+0000","last_fullsized":"2024-05-09T06:30:36.733912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:11.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815739+0000","last_change":"2024-05-09T06:30:34.581343+0000","last_active":"2024-05-09T06:30:36.815739+0000","last_peered":"2024-05-09T06:30:36.815739+0000","last_clean":"2024-05-09T06:30:36.815739+0000","last_became_active":"2024-05-09T06:30:34.581002+0000","last_became_peered":"2024-05-09T06:30:34.581002+0000","last_unstale":"2024-05-09T06:30:36.815739+0000","last_undegraded":"2024-05-09T06:30:36.815739+0000","last_fullsized":"2024-05-09T06:30:36.815739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:07.936174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733340+0000","last_change":"2024-05-09T06:30:35.592518+0000","last_active":"2024-05-09T06:30:36.733340+0000","last_peered":"2024-05-09T06:30:36.733340+0000","last_clean":"2024-05-09T06:30:36.733340+0000","last_became_active":"2024-05-09T06:30:35.591911+0000","last_became_peered":"2024-05-09T06:30:35.591911+0000","last_unstale":"2024-05-09T06:30:36.733340+0000","last_undegraded":"2024-05-09T06:30:36.733340+0000","last_fullsized":"2024-05-09T06:30:36.733340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:05.529949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815425+0000","last_change":"2024-05-09T06:30:34.585348+0000","last_active":"2024-05-09T06:30:36.815425+0000","last_peered":"2024-05-09T06:30:36.815425+0000","last_clean":"2024-05-09T06:30:36.815425+0000","last_became_active":"2024-05-09T06:30:34.584016+0000","last_became_peered":"2024-05-09T06:30:34.584016+0000","last_unstale":"2024-05-09T06:30:36.815425+0000","last_undegraded":"2024-05-09T06:30:36.815425+0000","last_fullsized":"2024-05-09T06:30:36.815425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:47.189776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.582633+0000","last_change":"2024-05-09T06:30:35.601343+0000","last_active":"2024-05-09T06:30:36.582633+0000","last_peered":"2024-05-09T06:30:36.582633+0000","last_clean":"2024-05-09T06:30:36.582633+0000","last_became_active":"2024-05-09T06:30:35.601059+0000","last_became_peered":"2024-05-09T06:30:35.601059+0000","last_unstale":"2024-05-09T06:30:36.582633+0000","last_undegraded":"2024-05-09T06:30:36.582633+0000","last_fullsized":"2024-05-09T06:30:36.582633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:43.563652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579512+0000","last_change":"2024-05-09T06:30:34.581383+0000","last_active":"2024-05-09T06:30:36.579512+0000","last_peered":"2024-05-09T06:30:36.579512+0000","last_clean":"2024-05-09T06:30:36.579512+0000","last_became_active":"2024-05-09T06:30:34.581087+0000","last_became_peered":"2024-05-09T06:30:34.581087+0000","last_unstale":"2024-05-09T06:30:36.579512+0000","last_undegraded":"2024-05-09T06:30:36.579512+0000","last_fullsized":"2024-05-09T06:30:36.579512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:34.548659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.815230+0000","last_change":"2024-05-09T06:30:35.598716+0000","last_active":"2024-05-09T06:30:36.815230+0000","last_peered":"2024-05-09T06:30:36.815230+0000","last_clean":"2024-05-09T06:30:36.815230+0000","last_became_active":"2024-05-09T06:30:35.598367+0000","last_became_peered":"2024-05-09T06:30:35.598367+0000","last_unstale":"2024-05-09T06:30:36.815230+0000","last_undegraded":"2024-05-09T06:30:36.815230+0000","last_fullsized":"2024-05-09T06:30:36.815230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:56.848828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579406+0000","last_change":"2024-05-09T06:30:34.579968+0000","last_active":"2024-05-09T06:30:36.579406+0000","last_peered":"2024-05-09T06:30:36.579406+0000","last_clean":"2024-05-09T06:30:36.579406+0000","last_became_active":"2024-05-09T06:30:34.579687+0000","last_became_peered":"2024-05-09T06:30:34.579687+0000","last_unstale":"2024-05-09T06:30:36.579406+0000","last_undegraded":"2024-05-09T06:30:36.579406+0000","last_fullsized":"2024-05-09T06:30:36.579406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:32.965877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.589963+0000","last_change":"2024-05-09T06:30:35.588154+0000","last_active":"2024-05-09T06:30:36.589963+0000","last_peered":"2024-05-09T06:30:36.589963+0000","last_clean":"2024-05-09T06:30:36.589963+0000","last_became_active":"2024-05-09T06:30:35.587854+0000","last_became_peered":"2024-05-09T06:30:35.587854+0000","last_unstale":"2024-05-09T06:30:36.589963+0000","last_undegraded":"2024-05-09T06:30:36.589963+0000","last_fullsized":"2024-05-09T06:30:36.589963+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:46.225956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814696+0000","last_change":"2024-05-09T06:30:34.586042+0000","last_active":"2024-05-09T06:30:36.814696+0000","last_peered":"2024-05-09T06:30:36.814696+0000","last_clean":"2024-05-09T06:30:36.814696+0000","last_became_active":"2024-05-09T06:30:34.585688+0000","last_became_peered":"2024-05-09T06:30:34.585688+0000","last_unstale":"2024-05-09T06:30:36.814696+0000","last_undegraded":"2024-05-09T06:30:36.814696+0000","last_fullsized":"2024-05-09T06:30:36.814696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:50.675720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.733813+0000","last_change":"2024-05-09T06:30:35.592271+0000","last_active":"2024-05-09T06:30:36.733813+0000","last_peered":"2024-05-09T06:30:36.733813+0000","last_clean":"2024-05-09T06:30:36.733813+0000","last_became_active":"2024-05-09T06:30:35.591953+0000","last_became_peered":"2024-05-09T06:30:35.591953+0000","last_unstale":"2024-05-09T06:30:36.733813+0000","last_undegraded":"2024-05-09T06:30:36.733813+0000","last_fullsized":"2024-05-09T06:30:36.733813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:34.557984+0000","last_clean_scrub_stamp":"2024-05-09T06:30:34.557984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:56.848828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.579225+0000","last_change":"2024-05-09T06:30:34.579060+0000","last_active":"2024-05-09T06:30:36.579225+0000","last_peered":"2024-05-09T06:30:36.579225+0000","last_clean":"2024-05-09T06:30:36.579225+0000","last_became_active":"2024-05-09T06:30:34.574824+0000","last_became_peered":"2024-05-09T06:30:34.574824+0000","last_unstale":"2024-05-09T06:30:36.579225+0000","last_undegraded":"2024-05-09T06:30:36.579225+0000","last_fullsized":"2024-05-09T06:30:36.579225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:30:33.553697+0000","last_clean_scrub_stamp":"2024-05-09T06:30:33.553697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:13.663009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-09T06:30:36.814949+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:30:36.814949+0000","last_peered":"2024-05-09T06:30:36.814949+0000","last_clean":"2024-05-09T06:30:36.814949+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:30:36.814949+0000","last_undegraded":"2024-05-09T06:30:36.814949+0000","last_fullsized":"2024-05-09T06:30:36.814949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1384448,"data_stored":941910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":350800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":350788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":352378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":350766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":350766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":353142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":941134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:30:39.935 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:30:39.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:30:40.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.101+0000 7fe56e458640 1 Processor -- start 2024-05-09T06:30:40.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.101+0000 7fe56e458640 1 -- start start 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56e458640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe56815a4d0 0x7fe5680cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56e458640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe568071610 0x7fe56806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56e458640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5680cf1b0 con 0x7fe5680cdb90 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56e458640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe56815a8b0 con 0x7fe568071610 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56e458640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe56806dc10 con 0x7fe56815a4d0 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe568071610 0x7fe56806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe568071610 0x7fe56806e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33332/0 (socket says 172.21.15.1:33332) 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe55ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe5680cdb90 0x7fe568070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59146/0 (socket says 172.21.15.1:59146) 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 -- 172.21.15.1:0/470492039 learned_addr learned my addr 172.21.15.1:0/470492039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:40.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe567fff640 1 --2- 172.21.15.1:0/470492039 >> v2:172.21.15.6:3300/0 conn(0x7fe56815a4d0 0x7fe5680cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 -- 172.21.15.1:0/470492039 >> v2:172.21.15.6:3300/0 conn(0x7fe56815a4d0 msgr2=0x7fe5680cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 --2- 172.21.15.1:0/470492039 >> v2:172.21.15.6:3300/0 conn(0x7fe56815a4d0 0x7fe5680cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 -- 172.21.15.1:0/470492039 >> v1:172.21.15.1:6789/0 conn(0x7fe5680cdb90 legacy=0x7fe568070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe56c9f5640 1 -- 172.21.15.1:0/470492039 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5680cf490 con 0x7fe568071610 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.102+0000 7fe5677fe640 1 -- 172.21.15.1:0/470492039 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052196836 0 0) 0x7fe5680cf1b0 con 0x7fe5680cdb90 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.103+0000 7fe56c9f5640 1 --2- 172.21.15.1:0/470492039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe568071610 0x7fe56806e0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe56806c810 tx=0x7fe56002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5bf5f50d72cab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.103+0000 7fe5677fe640 1 -- 172.21.15.1:0/470492039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56000b2e0 con 0x7fe568071610 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.103+0000 7fe5677fe640 1 -- 172.21.15.1:0/470492039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe560037070 con 0x7fe568071610 2024-05-09T06:30:40.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.103+0000 7fe5677fe640 1 -- 172.21.15.1:0/470492039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56000b480 con 0x7fe568071610 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 -- 172.21.15.1:0/470492039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe568071610 msgr2=0x7fe56806e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 --2- 172.21.15.1:0/470492039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe568071610 0x7fe56806e0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe56806c810 tx=0x7fe56002c940 comp rx=0 tx=0).stop 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 -- 172.21.15.1:0/470492039 shutdown_connections 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 --2- 172.21.15.1:0/470492039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe568071610 0x7fe56806e0a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 --2- 172.21.15.1:0/470492039 >> v2:172.21.15.6:3300/0 conn(0x7fe56815a4d0 0x7fe5680cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 -- 172.21.15.1:0/470492039 >> 172.21.15.1:0/470492039 conn(0x7fe5680cc4d0 msgr2=0x7fe5680cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 -- 172.21.15.1:0/470492039 shutdown_connections 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.104+0000 7fe56e458640 1 -- 172.21.15.1:0/470492039 wait complete. 2024-05-09T06:30:40.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.105+0000 7fe56e458640 1 Processor -- start 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.105+0000 7fe56e458640 1 -- start start 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 0x7fe5681ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 0x7fe5680b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680b2be0 con 0x7fe56815a4d0 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680b2a60 con 0x7fe568071610 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56e458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe56806cd30 con 0x7fe5680cdb90 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33346/0 (socket says 172.21.15.1:33346) 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 -- 172.21.15.1:0/2593263673 learned_addr learned my addr 172.21.15.1:0/2593263673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe567fff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 0x7fe5681ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe55ffff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 0x7fe5680b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 -- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 msgr2=0x7fe5680b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 0x7fe5680b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 -- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 msgr2=0x7fe5681ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 0x7fe5681ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe56c9f5640 1 -- 172.21.15.1:0/2593263673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681efbb0 con 0x7fe56815a4d0 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe567fff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 0x7fe5681ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.106+0000 7fe55ffff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 0x7fe5680b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe56c9f5640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe560034040 tx=0x7fe560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56000b720 con 0x7fe56815a4d0 2024-05-09T06:30:40.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe560032840 con 0x7fe56815a4d0 2024-05-09T06:30:40.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560032c30 con 0x7fe56815a4d0 2024-05-09T06:30:40.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe568158c70 con 0x7fe56815a4d0 2024-05-09T06:30:40.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.107+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568159230 con 0x7fe56815a4d0 2024-05-09T06:30:40.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.108+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5680752b0 con 0x7fe56815a4d0 2024-05-09T06:30:40.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.108+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe56003b400 con 0x7fe56815a4d0 2024-05-09T06:30:40.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.109+0000 7fe5657fa640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.109+0000 7fe567fff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.109+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fe5600ae5c0 con 0x7fe56815a4d0 2024-05-09T06:30:40.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.109+0000 7fe567fff640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 0x7fe54806e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b820 tx=0x7fe558008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.113+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe56007cdf0 con 0x7fe56815a4d0 2024-05-09T06:30:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.370+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe568159c10 con 0x7fe56815a4d0 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.372+0000 7fe5657fa640 1 -- 172.21.15.1:0/2593263673 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fe56007c6f0 con 0x7fe56815a4d0 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:30:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 msgr2=0x7fe54806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 0x7fe54806e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b820 tx=0x7fe558008040 comp rx=0 tx=0).stop 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 msgr2=0x7fe5680b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe560034040 tx=0x7fe560004a40 comp rx=0 tx=0).stop 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 shutdown_connections 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe56815a4d0 0x7fe5680b4450 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe5680cdb90 0x7fe5680b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 --2- 172.21.15.1:0/2593263673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe568071610 0x7fe5681ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 >> 172.21.15.1:0/2593263673 conn(0x7fe5680cc4d0 msgr2=0x7fe5680b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:40.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 shutdown_connections 2024-05-09T06:30:40.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.374+0000 7fe56e458640 1 -- 172.21.15.1:0/2593263673 wait complete. 2024-05-09T06:30:40.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:40.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.489+0000 7fb15b3f9640 1 Processor -- start 2024-05-09T06:30:40.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.489+0000 7fb15b3f9640 1 -- start start 2024-05-09T06:30:40.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 0x7fb15406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb15415e5b0 0x7fb15406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1540d2890 0x7fb1540cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb1540cc3c0 con 0x7fb15415e5b0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb1540ccc60 con 0x7fb15415e1d0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb15b3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb15406c2f0 con 0x7fb1540d2890 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb15415e5b0 0x7fb15406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb15415e5b0 0x7fb15406fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33362/0 (socket says 172.21.15.1:33362) 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 -- 172.21.15.1:0/2582269356 learned_addr learned my addr 172.21.15.1:0/2582269356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 -- 172.21.15.1:0/2582269356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1540d2890 msgr2=0x7fb1540cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb159195640 1 --2- 172.21.15.1:0/2582269356 >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 0x7fb15406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb159996640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1540d2890 0x7fb1540cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1540d2890 0x7fb1540cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 -- 172.21.15.1:0/2582269356 >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 msgr2=0x7fb15406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 --2- 172.21.15.1:0/2582269356 >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 0x7fb15406f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb158994640 1 -- 172.21.15.1:0/2582269356 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15406f500 con 0x7fb15415e5b0 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.490+0000 7fb159195640 1 --2- 172.21.15.1:0/2582269356 >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 0x7fb15406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:40.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.491+0000 7fb158994640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb15415e5b0 0x7fb15406fdd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb1540704a0 tx=0x7fb14402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d5036b785568e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.491+0000 7fb14bfff640 1 -- 172.21.15.1:0/2582269356 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144037070 con 0x7fb15415e5b0 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.491+0000 7fb14bfff640 1 -- 172.21.15.1:0/2582269356 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb144032440 con 0x7fb15415e5b0 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.491+0000 7fb14bfff640 1 -- 172.21.15.1:0/2582269356 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144032810 con 0x7fb15415e5b0 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 -- 172.21.15.1:0/2582269356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb15415e5b0 msgr2=0x7fb15406fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb15415e5b0 0x7fb15406fdd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb1540704a0 tx=0x7fb14402a140 comp rx=0 tx=0).stop 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 -- 172.21.15.1:0/2582269356 shutdown_connections 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1540d2890 0x7fb1540cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/2582269356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb15415e5b0 0x7fb15406fdd0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/2582269356 >> v2:172.21.15.6:3300/0 conn(0x7fb15415e1d0 0x7fb15406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 -- 172.21.15.1:0/2582269356 >> 172.21.15.1:0/2582269356 conn(0x7fb1540d1d70 msgr2=0x7fb1540d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 -- 172.21.15.1:0/2582269356 shutdown_connections 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.492+0000 7fb15b3f9640 1 -- 172.21.15.1:0/2582269356 wait complete. 2024-05-09T06:30:40.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.493+0000 7fb15b3f9640 1 Processor -- start 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.493+0000 7fb15b3f9640 1 -- start start 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb15415e1d0 0x7fb15414a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb15415e5b0 0x7fb154160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb15406f1b0 con 0x7fb1540d2890 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb1540cc8f0 con 0x7fb15415e1d0 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb15b3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb1540cd630 con 0x7fb15415e5b0 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33378/0 (socket says 172.21.15.1:33378) 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 -- 172.21.15.1:0/383729000 learned_addr learned my addr 172.21.15.1:0/383729000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 -- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb15415e5b0 msgr2=0x7fb154160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb15415e5b0 0x7fb154160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 -- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb15415e1d0 msgr2=0x7fb15414a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb15415e1d0 0x7fb15414a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 -- 172.21.15.1:0/383729000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15406db20 con 0x7fb1540d2890 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.494+0000 7fb159195640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c7f0 tx=0x7fb15000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.495+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150015070 con 0x7fb1540d2890 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.495+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb150007a30 con 0x7fb1540d2890 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.495+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1541545b0 con 0x7fb1540d2890 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.495+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150007de0 con 0x7fb1540d2890 2024-05-09T06:30:40.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.495+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb154154b70 con 0x7fb1540d2890 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.496+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb154067ac0 con 0x7fb1540d2890 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.497+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb150018070 con 0x7fb1540d2890 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.497+0000 7fb149ffb640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.497+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fb15008af20 con 0x7fb1540d2890 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.497+0000 7fb158994640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:40.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.498+0000 7fb158994640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 0x7fb13806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb144034000 tx=0x7fb14402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:40.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.501+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb154067ac0 con 0x7fb1540d2890 2024-05-09T06:30:40.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.721+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1540752b0 con 0x7fb1540d2890 2024-05-09T06:30:40.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.721+0000 7fb149ffb640 1 -- 172.21.15.1:0/383729000 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5167 (secure 0 0 0) 0x7fb1500597d0 con 0x7fb1540d2890 2024-05-09T06:30:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:40.703 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:30:37.962593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:30:40.704 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 msgr2=0x7fb13806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 0x7fb13806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb144034000 tx=0x7fb14402c000 comp rx=0 tx=0).stop 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 msgr2=0x7fb154153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c7f0 tx=0x7fb15000cbf0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 shutdown_connections 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb15415e5b0 0x7fb154160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb15415e1d0 0x7fb15414a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 --2- 172.21.15.1:0/383729000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1540d2890 0x7fb154153f20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 >> 172.21.15.1:0/383729000 conn(0x7fb1540d1d70 msgr2=0x7fb1540cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:40.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 shutdown_connections 2024-05-09T06:30:40.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:40.723+0000 7fb15b3f9640 1 -- 172.21.15.1:0/383729000 wait complete. 2024-05-09T06:30:41.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:30:34.938735+0000', 'modified': '2024-05-09T06:30:37.962593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6576}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6576': {'gid': 6576, 'name': 'b', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6835/531134827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6833', 'nonce': 531134827}, {'type': 'v1', 'addr': '172.21.15.6:6835', 'nonce': 531134827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6576, 'qdb_cluster': [6576]} 2024-05-09T06:30:41.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:30:41.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T06:30:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.247+0000 7f5c5b5f9640 1 Processor -- start 2024-05-09T06:30:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 -- start start 2024-05-09T06:30:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c540b2a60 0x7f5c5415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415e500 0x7f5c5406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c54070f40 con 0x7f5c5415db10 2024-05-09T06:30:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5c540cf170 con 0x7f5c540b2a60 2024-05-09T06:30:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.248+0000 7f5c5b5f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c540cf450 con 0x7f5c5415e500 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c540b2a60 0x7f5c5415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415e500 0x7f5c5406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c540b2a60 0x7f5c5415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:36196/0 (socket says 172.21.15.1:36196) 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 -- 172.21.15.1:0/2375905449 learned_addr learned my addr 172.21.15.1:0/2375905449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c59395640 1 --1- 172.21.15.1:0/2375905449 >> v1:172.21.15.6:6789/0 conn(0x7f5c5415db10 0x7f5c540715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:60508/0 (socket says 172.21.15.1:60508) 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 -- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415e500 msgr2=0x7f5c5406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 --2- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415e500 0x7f5c5406e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 -- 172.21.15.1:0/2375905449 >> v1:172.21.15.6:6789/0 conn(0x7f5c5415db10 legacy=0x7f5c540715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.249+0000 7f5c58b94640 1 -- 172.21.15.1:0/2375905449 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54070c10 con 0x7f5c540b2a60 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.250+0000 7f5c58b94640 1 --2- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5415dfc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009870 tx=0x7f5c4802c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e0495cb06040b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.250+0000 7f5c53fff640 1 -- 172.21.15.1:0/2375905449 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48038070 con 0x7f5c540b2a60 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.250+0000 7f5c53fff640 1 -- 172.21.15.1:0/2375905449 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c48031710 con 0x7f5c540b2a60 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.250+0000 7f5c53fff640 1 -- 172.21.15.1:0/2375905449 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48031ae0 con 0x7f5c540b2a60 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 msgr2=0x7f5c5415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5415dfc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009870 tx=0x7f5c4802c3d0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/2375905449 shutdown_connections 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415e500 0x7f5c5406e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/2375905449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5415dfc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/2375905449 >> 172.21.15.1:0/2375905449 conn(0x7f5c540cc4b0 msgr2=0x7f5c540cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/2375905449 shutdown_connections 2024-05-09T06:30:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.251+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/2375905449 wait complete. 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.252+0000 7f5c5b5f9640 1 Processor -- start 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.252+0000 7f5c5b5f9640 1 -- start start 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 0x7f5c54154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5406ceb0 con 0x7f5c5415e500 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5406cd30 con 0x7f5c540b2a60 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c5b5f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5406d030 con 0x7f5c5415db10 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33394/0 (socket says 172.21.15.1:33394) 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 -- 172.21.15.1:0/3834051686 learned_addr learned my addr 172.21.15.1:0/3834051686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59395640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c58b94640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 0x7f5c54154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 -- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 msgr2=0x7f5c54154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 0x7f5c54154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 -- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 msgr2=0x7f5c5414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c59b96640 1 -- 172.21.15.1:0/3834051686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c541559f0 con 0x7f5c5415e500 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.253+0000 7f5c58b94640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 0x7f5c54154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c59b96640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e7f0 tx=0x7f5c4400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c51ffb640 1 -- 172.21.15.1:0/3834051686 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44017070 con 0x7f5c5415e500 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c51ffb640 1 -- 172.21.15.1:0/3834051686 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4400ee40 con 0x7f5c5415e500 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c54154b20 con 0x7f5c5415e500 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c51ffb640 1 -- 172.21.15.1:0/3834051686 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400cb80 con 0x7f5c5415e500 2024-05-09T06:30:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.254+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541550e0 con 0x7f5c5415e500 2024-05-09T06:30:41.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.256+0000 7f5c51ffb640 1 -- 172.21.15.1:0/3834051686 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c4401a070 con 0x7f5c5415e500 2024-05-09T06:30:41.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.256+0000 7f5c51ffb640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 0x7f5c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.256+0000 7f5c51ffb640 1 -- 172.21.15.1:0/3834051686 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5c4408ae10 con 0x7f5c5415e500 2024-05-09T06:30:41.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.256+0000 7f5c59395640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 0x7f5c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.256+0000 7f5c59395640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 0x7f5c4006e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0012e0 tx=0x7f5c4c005b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.257+0000 7f5c5e3e8740 1 Processor -- start 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.257+0000 7f5c5e3e8740 1 -- start start 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d0da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 0x561610d295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561610c464d0 con 0x561610d2af10 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561610bee3e0 con 0x561610ce3350 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561610c42490 con 0x561610d0d670 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33406/0 (socket says 172.21.15.1:33406) 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 -- 172.21.15.1:0/1538899309 learned_addr learned my addr 172.21.15.1:0/1538899309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c59395640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d0da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c59b96640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 0x561610d295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 -- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 msgr2=0x561610d0da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d0da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 -- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 msgr2=0x561610d295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 0x561610d295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 -- 172.21.15.1:0/1538899309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56161095a2d0 con 0x561610d2af10 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c59b96640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 0x561610d295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c59395640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d0da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c58b94640 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5c4802c410 tx=0x7f5c48030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c84b3a928901c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c3affd640 1 -- 172.21.15.1:0/1538899309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48038040 con 0x561610d2af10 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c3affd640 1 -- 172.21.15.1:0/1538899309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c48043070 con 0x561610d2af10 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.258+0000 7f5c3affd640 1 -- 172.21.15.1:0/1538899309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48031910 con 0x561610d2af10 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1538899309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 msgr2=0x561610d2b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5c4802c410 tx=0x7f5c48030000 comp rx=0 tx=0).stop 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1538899309 shutdown_connections 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610ce3350 0x561610d295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d2b460 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1538899309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d0da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1538899309 >> 172.21.15.1:0/1538899309 conn(0x561610cdfc80 msgr2=0x561610d28480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1538899309 shutdown_connections 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.259+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1538899309 wait complete. 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.260+0000 7f5c5e3e8740 1 -- start start 2024-05-09T06:30:41.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.260+0000 7f5c5e3e8740 1 Processor -- start 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d0d670 0x561610d09ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d2af10 0x561610d39170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561610ce40f0 con 0x561610d3bd70 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561610c018e0 con 0x561610d0d670 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561610cddbb0 con 0x561610d2af10 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c5e3e8740 10 client.0 _ll_register_callbacks cb 0x561610bce9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33410/0 (socket says 172.21.15.1:33410) 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 -- 172.21.15.1:0/933035249 learned_addr learned my addr 172.21.15.1:0/933035249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59395640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d0d670 0x561610d09ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c58b94640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d2af10 0x561610d39170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 -- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d2af10 msgr2=0x561610d39170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d2af10 0x561610d39170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 -- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d0d670 msgr2=0x561610d09ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d0d670 0x561610d09ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 -- 172.21.15.1:0/933035249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561610a246b0 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c59b96640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004770 tx=0x7f5c4405d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a159f7a5b0b33d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c33fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44071020 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.261+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5c44058410 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c48043070 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c44021030 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 10 client.6711 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561610d38970 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c48043070 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f5c4405b070 con 0x561610d3bd70 2024-05-09T06:30:41.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 10 client.6711 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 10 client.6711 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 20 client.6711 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d0ea80 0x561610d0ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 4 client.6711 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561610c53c00 con 0x561610d0ea80 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.262+0000 7f5c59395640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d0ea80 0x561610d0ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c33fff640 10 client.6711 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c59395640 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d0ea80 0x561610d0ee60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d1514b631d93da server_cookie=59221d90b8ff20f1 in_seq=0 out_seq=0 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c33fff640 1 -- 172.21.15.1:0/933035249 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561610c53c00 con 0x561610d0ea80 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c33fff640 10 client.6711 handle_command_reply: tid=0 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c5e3e8740 1 client.6711 shutdown 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 msgr2=0x561610d10710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.263+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004770 tx=0x7f5c4405d370 comp rx=0 tx=0).stop 2024-05-09T06:30:41.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 shutdown_connections 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610d10710 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d0ea80 0x561610d0ee60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d2af10 0x561610d39170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/933035249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d0d670 0x561610d09ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c33fff640 0 client.6711 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 >> 172.21.15.1:0/933035249 conn(0x561610ce3350 msgr2=0x561610d2c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 shutdown_connections 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/933035249 wait complete. 2024-05-09T06:30:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.264+0000 7f5c5e3e8740 20 client.6711 trim_cache size 0 max 16384 2024-05-09T06:30:41.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.282+0000 7f5c5e3e8740 1 Processor -- start 2024-05-09T06:30:41.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.282+0000 7f5c5e3e8740 1 -- start start 2024-05-09T06:30:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610cdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 0x561610d37e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ffb00 con 0x561610d3bd70 2024-05-09T06:30:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c54156b30 con 0x561610d2af10 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c54157180 con 0x561610d0d670 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33418/0 (socket says 172.21.15.1:33418) 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 -- 172.21.15.1:0/1482264194 learned_addr learned my addr 172.21.15.1:0/1482264194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 -- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 msgr2=0x561610cdd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59395640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610cdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c58b94640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 0x561610d37e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610cdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 -- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 msgr2=0x561610d37e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 0x561610d37e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 -- 172.21.15.1:0/1482264194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561610cdf140 con 0x561610d3bd70 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c58b94640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 0x561610d37e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c59b96640 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5c44058f50 tx=0x7f5c4405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771b5debeefaaca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c3a7fc640 1 -- 172.21.15.1:0/1482264194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44071020 con 0x561610d3bd70 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c3a7fc640 1 -- 172.21.15.1:0/1482264194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44082ba0 con 0x561610d3bd70 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.283+0000 7f5c3a7fc640 1 -- 172.21.15.1:0/1482264194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44054d10 con 0x561610d3bd70 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1482264194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 msgr2=0x561610ce1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5c44058f50 tx=0x7f5c4405b000 comp rx=0 tx=0).stop 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1482264194 shutdown_connections 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d3bd70 0x561610ce1220 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d2af10 0x561610d37e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/1482264194 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610cdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1482264194 >> 172.21.15.1:0/1482264194 conn(0x561610ce3350 msgr2=0x561610d1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1482264194 shutdown_connections 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.284+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/1482264194 wait complete. 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.285+0000 7f5c5e3e8740 1 -- start start 2024-05-09T06:30:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.285+0000 7f5c5e3e8740 1 Processor -- start 2024-05-09T06:30:41.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d2a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d3bd70 0x561610d9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561610d26970 con 0x561610d2af10 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561610d110d0 con 0x561610d3bd70 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561610d10e40 con 0x561610d0d670 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 10 client.0 _ll_register_callbacks cb 0x7f5c44058e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33422/0 (socket says 172.21.15.1:33422) 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 -- 172.21.15.1:0/3340952485 learned_addr learned my addr 172.21.15.1:0/3340952485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c59395640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d2a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c59b96640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d3bd70 0x561610d9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 -- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 msgr2=0x561610d2a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d2a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 -- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d3bd70 msgr2=0x561610d9c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d3bd70 0x561610d9c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 -- 172.21.15.1:0/3340952485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561610d4c950 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c58b94640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5c4802c410 tx=0x7f5c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ef2e3f13cef5da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c33fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4804e060 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f5c48003a20 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.286+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c44082ba0 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c48054050 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 10 client.6717 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561610d0c700 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44082ba0 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f5c48057020 con 0x561610d2af10 2024-05-09T06:30:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 10 client.6717 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 10 client.6717 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 20 client.6717 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d9c760 0x561610d4b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 4 client.6717 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "status"}) -- 0x561610cdeaa0 con 0x561610d9c760 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c59395640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d9c760 0x561610d4b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c59395640 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d9c760 0x561610d4b220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0fa31c66ff2863 server_cookie=413852eaacb740c8 in_seq=0 out_seq=0 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.287+0000 7f5c33fff640 10 client.6717 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c33fff640 1 -- 172.21.15.1:0/3340952485 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561610cdeaa0 con 0x561610d9c760 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c33fff640 10 client.6717 handle_command_reply: tid=0 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 client.6717 shutdown 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 msgr2=0x561610d0bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5c4802c410 tx=0x7f5c48004730 comp rx=0 tx=0).stop 2024-05-09T06:30:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 shutdown_connections 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561610d3bd70 0x561610d9c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561610d2af10 0x561610d0bce0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x561610d9c760 0x561610d4b220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 --2- 172.21.15.1:0/3340952485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561610d0d670 0x561610d2a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c33fff640 0 client.6717 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 >> 172.21.15.1:0/3340952485 conn(0x561610ce3350 msgr2=0x561610ce4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 shutdown_connections 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 1 -- 172.21.15.1:0/3340952485 wait complete. 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.288+0000 7f5c5e3e8740 20 client.6717 trim_cache size 0 max 16384 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 6576, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 5.7113089129999999, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 35, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 64, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 64, 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 10.694568558 2024-05-09T06:30:41.277 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 msgr2=0x7f5c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 0x7f5c4006e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0012e0 tx=0x7f5c4c005b70 comp rx=0 tx=0).stop 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 msgr2=0x7f5c540cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e7f0 tx=0x7f5c4400ebf0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 shutdown_connections 2024-05-09T06:30:41.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5c4006be30 0x7f5c4006e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c5415e500 0x7f5c540cda90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c5415db10 0x7f5c54154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 --2- 172.21.15.1:0/3834051686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c540b2a60 0x7f5c5414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 >> 172.21.15.1:0/3834051686 conn(0x7f5c540cc4b0 msgr2=0x7f5c5415b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 shutdown_connections 2024-05-09T06:30:41.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.289+0000 7f5c5b5f9640 1 -- 172.21.15.1:0/3834051686 wait complete. 2024-05-09T06:30:41.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:30:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.404+0000 7f740f3f4640 1 Processor -- start 2024-05-09T06:30:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.404+0000 7f740f3f4640 1 -- start start 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740f3f4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74080b2aa0 0x7f740815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740f3f4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7408099930 con 0x7f7408099b30 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740f3f4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7408162930 con 0x7f74080c6860 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740f3f4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f74080b1e20 con 0x7f74080b2aa0 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74080b2aa0 0x7f740815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74080b2aa0 0x7f740815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33434/0 (socket says 172.21.15.1:33434) 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740c98f640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7408099b30 0x7f740815f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59160/0 (socket says 172.21.15.1:59160) 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 -- 172.21.15.1:0/1431184172 learned_addr learned my addr 172.21.15.1:0/1431184172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 -- 172.21.15.1:0/1431184172 >> v1:172.21.15.6:6789/0 conn(0x7f74080c6860 legacy=0x7f74080c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f73fffff640 1 -- 172.21.15.1:0/1431184172 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397699608 0 0) 0x7f73f8001110 con 0x7f7408099b30 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 -- 172.21.15.1:0/1431184172 >> v1:172.21.15.1:6789/0 conn(0x7f7408099b30 legacy=0x7f740815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.405+0000 7f740d991640 1 -- 172.21.15.1:0/1431184172 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7408162c10 con 0x7f74080b2aa0 2024-05-09T06:30:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.406+0000 7f740d991640 1 --2- 172.21.15.1:0/1431184172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080b2aa0 0x7f740815e4b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f740002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1022e9b48b573f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.406+0000 7f73fffff640 1 -- 172.21.15.1:0/1431184172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400036070 con 0x7f74080b2aa0 2024-05-09T06:30:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.406+0000 7f73fffff640 1 -- 172.21.15.1:0/1431184172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7400031600 con 0x7f74080b2aa0 2024-05-09T06:30:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.406+0000 7f73fffff640 1 -- 172.21.15.1:0/1431184172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000319d0 con 0x7f74080b2aa0 2024-05-09T06:30:41.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 -- 172.21.15.1:0/1431184172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080b2aa0 msgr2=0x7f740815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 --2- 172.21.15.1:0/1431184172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080b2aa0 0x7f740815e4b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f740002a730 comp rx=0 tx=0).stop 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 -- 172.21.15.1:0/1431184172 shutdown_connections 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 --2- 172.21.15.1:0/1431184172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080b2aa0 0x7f740815e4b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 -- 172.21.15.1:0/1431184172 >> 172.21.15.1:0/1431184172 conn(0x7f74080b1fb0 msgr2=0x7f74080b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 -- 172.21.15.1:0/1431184172 shutdown_connections 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.407+0000 7f740f3f4640 1 -- 172.21.15.1:0/1431184172 wait complete. 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.408+0000 7f740f3f4640 1 Processor -- start 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.408+0000 7f740f3f4640 1 -- start start 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7408099b30 0x7f7408156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 0x7f74080b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f740815fa30 con 0x7f74080c6860 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f740815ee90 con 0x7f7408099b30 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740f3f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f74080cdf50 con 0x7f74080b2aa0 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33442/0 (socket says 172.21.15.1:33442) 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 -- 172.21.15.1:0/199358016 learned_addr learned my addr 172.21.15.1:0/199358016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d190640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7408099b30 0x7f7408156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740c98f640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 0x7f74080b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 -- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 msgr2=0x7f74080b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 0x7f74080b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 -- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7408099b30 msgr2=0x7f7408156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7408099b30 0x7f7408156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740d991640 1 -- 172.21.15.1:0/199358016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74080cab50 con 0x7f74080c6860 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.409+0000 7f740c98f640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 0x7f74080b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f740d991640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f7400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400036050 con 0x7f74080c6860 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7400042070 con 0x7f74080c6860 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400009510 con 0x7f74080c6860 2024-05-09T06:30:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74080cae30 con 0x7f74080c6860 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.410+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74080cb3f0 con 0x7f74080c6860 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.411+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74080752b0 con 0x7f74080c6860 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.411+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7400007bf0 con 0x7f74080c6860 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.412+0000 7f73fdffb640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 0x7f73ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.412+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f74000abd80 con 0x7f74080c6860 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.412+0000 7f740d190640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 0x7f73ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.412+0000 7f740d190640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 0x7f73ec06e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7404000f60 tx=0x7f7404009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.416+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f740007a630 con 0x7f74080c6860 2024-05-09T06:30:41.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.642+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f740806ce30 con 0x7f74080c6860 2024-05-09T06:30:41.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.642+0000 7f73fdffb640 1 -- 172.21.15.1:0/199358016 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f7400079f30 con 0x7f74080c6860 2024-05-09T06:30:41.624 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 msgr2=0x7f73ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 0x7f73ec06e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7404000f60 tx=0x7f7404009040 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 msgr2=0x7f74080b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f7400004a40 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 shutdown_connections 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73ec06be30 0x7f73ec06e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74080c6860 0x7f74080b35e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74080b2aa0 0x7f74080b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 --2- 172.21.15.1:0/199358016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7408099b30 0x7f7408156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 >> 172.21.15.1:0/199358016 conn(0x7f74080b1fb0 msgr2=0x7f740815d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 shutdown_connections 2024-05-09T06:30:41.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41.645+0000 7f740f3f4640 1 -- 172.21.15.1:0/199358016 wait complete. 2024-05-09T06:30:41.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:30:41.638 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:30:41.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:41.638 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:30:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:30:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.657 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:30:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:30:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:30:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:30:41.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:41.658 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:30:41.720 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:30:41.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:41.722 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:30:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:30:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:30:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:30:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:30:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:30:41.779 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:30:41.780 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:30:41.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:30:41.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:30:41.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:30:41.780 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:30:41.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:30:41.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:41.910 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:30:41.926 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:30:41.926 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:30:41.926 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:30:41.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:41.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:30:41.992 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:30:41.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:30:41.993 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:42.048 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:30:42.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:30:42.049 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:42.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:30:42.170 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:30:42.228 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:30:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:30:42.230 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:30:42.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.231 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:30:42.284 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:30:42.284 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:30:42.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:30:42.352 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:30:42.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:30:42.411 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:30:42.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.412 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:30:42.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:30:42.429+0000 7fb7dff22480 -1 init, newargv = 0x55f8aa599f70 newargc=15 2024-05-09T06:30:42.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[58951]: starting ceph client 2024-05-09T06:30:42.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[58951]: starting fuse 2024-05-09T06:30:42.429 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:30:42.429 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:30:42.430 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:30:42.431 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:30:42.550 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.58951.asok 2024-05-09T06:30:42.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:30:42.561 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58951.asok 2024-05-09T06:30:42.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58951.asok status 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "58951", 2024-05-09T06:30:42.700 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 6732, 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6732 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 426229859 2024-05-09T06:30:42.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 426229859 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.6732 192.168.0.1:0/426229859", 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/426229859", 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 35, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 64, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:30:42.702 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:30:42.703 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:42.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.708 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:30:42.709 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:42.725 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:30:42.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:42.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.725 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:30:42.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.793 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-09T06:30:42.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:42.856 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-09T06:30:42.916 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:30:43.024 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:30:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:30:43.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:30:43.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.205+0000 7f416fc2a640 1 Processor -- start 2024-05-09T06:30:43.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.205+0000 7f416fc2a640 1 -- start start 2024-05-09T06:30:43.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416fc2a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f416815db10 0x7f41680b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416fc2a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815dfc0 0x7f416815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416fc2a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4168070f40 con 0x7f41680715d0 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416fc2a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f416806d440 con 0x7f416815db10 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416fc2a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f41680cf170 con 0x7f416815dfc0 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f416815db10 0x7f41680b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f416815db10 0x7f41680b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33454/0 (socket says 172.21.15.1:33454) 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 -- 172.21.15.1:0/2911044205 learned_addr learned my addr 172.21.15.1:0/2911044205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416e1c7640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815dfc0 0x7f416815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 -- 172.21.15.1:0/2911044205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815dfc0 msgr2=0x7f416815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815dfc0 0x7f416815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 -- 172.21.15.1:0/2911044205 >> v1:172.21.15.6:6789/0 conn(0x7f41680715d0 legacy=0x7f416806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.206+0000 7f416d9c6640 1 -- 172.21.15.1:0/2911044205 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41680cf450 con 0x7f416815db10 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.207+0000 7f416d9c6640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815db10 0x7f41680b2a60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4164009870 tx=0x7f416402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06449578e8a2ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.207+0000 7f416c9c4640 1 -- 172.21.15.1:0/2911044205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164037070 con 0x7f416815db10 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.207+0000 7f416c9c4640 1 -- 172.21.15.1:0/2911044205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4164032440 con 0x7f416815db10 2024-05-09T06:30:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.207+0000 7f416c9c4640 1 -- 172.21.15.1:0/2911044205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164032810 con 0x7f416815db10 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 -- 172.21.15.1:0/2911044205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815db10 msgr2=0x7f41680b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815db10 0x7f41680b2a60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4164009870 tx=0x7f416402a210 comp rx=0 tx=0).stop 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 -- 172.21.15.1:0/2911044205 shutdown_connections 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815dfc0 0x7f416815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 --2- 172.21.15.1:0/2911044205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815db10 0x7f41680b2a60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 -- 172.21.15.1:0/2911044205 >> 172.21.15.1:0/2911044205 conn(0x7f41680cc4b0 msgr2=0x7f41680cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 -- 172.21.15.1:0/2911044205 shutdown_connections 2024-05-09T06:30:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.208+0000 7f416fc2a640 1 -- 172.21.15.1:0/2911044205 wait complete. 2024-05-09T06:30:43.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.209+0000 7f416fc2a640 1 Processor -- start 2024-05-09T06:30:43.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.209+0000 7f416fc2a640 1 -- start start 2024-05-09T06:30:43.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41680715d0 0x7f4168147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815db10 0x7f416815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4168071bd0 con 0x7f416815dfc0 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4168071a50 con 0x7f41680715d0 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416fc2a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f416815ec40 con 0x7f416815db10 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33464/0 (socket says 172.21.15.1:33464) 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 -- 172.21.15.1:0/758757954 learned_addr learned my addr 172.21.15.1:0/758757954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416d1c5640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815db10 0x7f416815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416d9c6640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41680715d0 0x7f4168147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 -- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815db10 msgr2=0x7f416815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815db10 0x7f416815b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 -- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41680715d0 msgr2=0x7f4168147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41680715d0 0x7f4168147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 -- 172.21.15.1:0/758757954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681531f0 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f416e1c7640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f416000cc30 tx=0x7f416000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.210+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000ce40 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.211+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160003990 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.211+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160003d40 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.211+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681534d0 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.211+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4168153a90 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.211+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41680752b0 con 0x7f416815dfc0 2024-05-09T06:30:43.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.212+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4160003ee0 con 0x7f416815dfc0 2024-05-09T06:30:43.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.212+0000 7f415e7fc640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 0x7f414c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.213+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f416008b420 con 0x7f416815dfc0 2024-05-09T06:30:43.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.213+0000 7f416d9c6640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 0x7f414c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.213+0000 7f416d9c6640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 0x7f414c06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.217+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4160014030 con 0x7f416815dfc0 2024-05-09T06:30:43.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.437+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f416806d070 con 0x7f416815dfc0 2024-05-09T06:30:43.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.438+0000 7f415e7fc640 1 -- 172.21.15.1:0/758757954 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5167 (secure 0 0 0) 0x7f41600595f0 con 0x7f416815dfc0 2024-05-09T06:30:43.420 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:43.420 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:30:37.962593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:30:43.420 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-05-09T06:30:43.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 msgr2=0x7f414c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 0x7f414c06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).stop 2024-05-09T06:30:43.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 msgr2=0x7f416815b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f416000cc30 tx=0x7f416000a5c0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 shutdown_connections 2024-05-09T06:30:43.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.439+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f414c06be30 0x7f414c06e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f416815dfc0 0x7f416815b8a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f416815db10 0x7f416815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 --2- 172.21.15.1:0/758757954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41680715d0 0x7f4168147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 >> 172.21.15.1:0/758757954 conn(0x7f41680cc4b0 msgr2=0x7f41680cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 shutdown_connections 2024-05-09T06:30:43.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.440+0000 7f416fc2a640 1 -- 172.21.15.1:0/758757954 wait complete. 2024-05-09T06:30:43.433 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-09T06:30:43.580 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2024-05-09T06:30:43.580 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:30:43.580 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-09T06:30:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.649+0000 7efdc771bf40 1 Processor -- start 2024-05-09T06:30:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.649+0000 7efdc771bf40 1 -- start start 2024-05-09T06:30:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb7c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb731ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e8bb6e7790 con 0x55e8bb7bf500 2024-05-09T06:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8bb68ec80 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc771bf40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e8bb6e7510 con 0x55e8bb732f80 2024-05-09T06:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33480/0 (socket says 172.21.15.1:33480) 2024-05-09T06:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 -- 172.21.15.1:0/306072080 learned_addr learned my addr 172.21.15.1:0/306072080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc4fa7640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb731ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdc47a6640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb7c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 -- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 msgr2=0x55e8bb7c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb7c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 -- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 msgr2=0x55e8bb731ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.650+0000 7efdbbfff640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb731ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdbbfff640 1 -- 172.21.15.1:0/306072080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bb613cb0 con 0x55e8bb7bf500 2024-05-09T06:30:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc4fa7640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdbbfff640 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efdac045f70 tx=0x7efdac051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c43949f00981aab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc37a4640 1 -- 172.21.15.1:0/306072080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac053720 con 0x55e8bb7bf500 2024-05-09T06:30:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc37a4640 1 -- 172.21.15.1:0/306072080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac0538c0 con 0x55e8bb7bf500 2024-05-09T06:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc37a4640 1 -- 172.21.15.1:0/306072080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac053c90 con 0x55e8bb7bf500 2024-05-09T06:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 -- 172.21.15.1:0/306072080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 msgr2=0x55e8bb6e8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efdac045f70 tx=0x7efdac051000 comp rx=0 tx=0).stop 2024-05-09T06:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 -- 172.21.15.1:0/306072080 shutdown_connections 2024-05-09T06:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf500 0x55e8bb6e8360 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 --2- 172.21.15.1:0/306072080 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb7c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 -- 172.21.15.1:0/306072080 >> 172.21.15.1:0/306072080 conn(0x55e8bb6e5b00 msgr2=0x55e8bb6e5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 -- 172.21.15.1:0/306072080 shutdown_connections 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.651+0000 7efdc771bf40 1 -- 172.21.15.1:0/306072080 wait complete. 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.652+0000 7efdc771bf40 1 Processor -- start 2024-05-09T06:30:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.652+0000 7efdc771bf40 1 -- start start 2024-05-09T06:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb72e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf500 0x55e8bb7ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb7bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e8bb7b86a0 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8bb7b88f0 con 0x55e8bb7bf500 2024-05-09T06:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e8bb7c1550 con 0x55e8bb732f80 2024-05-09T06:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb7bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb7bdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33484/0 (socket says 172.21.15.1:33484) 2024-05-09T06:30:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 -- 172.21.15.1:0/3962702989 learned_addr learned my addr 172.21.15.1:0/3962702989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc47a6640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb72e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdbbfff640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf500 0x55e8bb7ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 -- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 msgr2=0x55e8bb72e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e8bb732f80 0x55e8bb72e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 -- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf500 msgr2=0x55e8bb7ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8bb7bf500 0x55e8bb7ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 -- 172.21.15.1:0/3962702989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bb72eee0 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc4fa7640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb7bdec0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efdbc004270 tx=0x7efdbc00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc27a2640 1 -- 172.21.15.1:0/3962702989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc018070 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc27a2640 1 -- 172.21.15.1:0/3962702989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdbc003110 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc27a2640 1 -- 172.21.15.1:0/3962702989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc006030 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.653+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e8bb7ad060 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.654+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8bb7acb80 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.655+0000 7efdc27a2640 1 -- 172.21.15.1:0/3962702989 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdbc001540 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.655+0000 7efdc27a2640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efda406a810 0x7efda406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.655+0000 7efdc27a2640 1 -- 172.21.15.1:0/3962702989 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7efdbc0891c0 con 0x55e8bb7bf8e0 2024-05-09T06:30:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.655+0000 7efdc47a6640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efda406a810 0x7efda406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.656+0000 7efdc771bf40 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e8bb6e8360 0x55e8bb7aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.656+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) -- 0x55e8bb7af120 con 0x55e8bb6e8360 2024-05-09T06:30:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.656+0000 7efdc47a6640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efda406a810 0x7efda406ccd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005400 tx=0x7efdb400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.656+0000 7efdbbfff640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e8bb6e8360 0x55e8bb7aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.656+0000 7efdbbfff640 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e8bb6e8360 0x55e8bb7aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdbbfff640 1 -- 172.21.15.1:0/3962702989 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7efdac053620 con 0x55e8bb6e8360 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e8bb6e8360 msgr2=0x55e8bb7aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e8bb6e8360 0x55e8bb7aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efda406a810 msgr2=0x7efda406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efda406a810 0x7efda406ccd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005400 tx=0x7efdb400c040 comp rx=0 tx=0).stop 2024-05-09T06:30:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 msgr2=0x55e8bb7bdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 --2- 172.21.15.1:0/3962702989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e8bb7bf8e0 0x55e8bb7bdec0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efdbc004270 tx=0x7efdbc00d5f0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc4fa7640 1 -- 172.21.15.1:0/3962702989 reap_dead start 2024-05-09T06:30:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 shutdown_connections 2024-05-09T06:30:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 >> 172.21.15.1:0/3962702989 conn(0x55e8bb6e5b00 msgr2=0x55e8bb6e5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 shutdown_connections 2024-05-09T06:30:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.657+0000 7efdc771bf40 1 -- 172.21.15.1:0/3962702989 wait complete. 2024-05-09T06:30:43.665 INFO:teuthology.orchestra.run.smithi001.stdout:Xfiledir_test 2024-05-09T06:30:43.666 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:43.666 DEBUG:teuthology.orchestra.run.smithi001:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T06:30:43.758 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "ancestors": [ 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1099511627776, 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "file", 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dirino": 1, 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "dir_test", 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 10, 2024-05-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "old_pools": [] 2024-05-09T06:30:43.760 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:30:43.761 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-09T06:30:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.881+0000 7f4fd1797f40 1 Processor -- start 2024-05-09T06:30:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 -- start start 2024-05-09T06:30:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a247f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a2cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55617a247670 con 0x55617a24b530 2024-05-09T06:30:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55617a24b940 con 0x55617a2d4540 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55617a245ee0 con 0x55617a22ef10 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fcea99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fcea99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33510/0 (socket says 172.21.15.1:33510) 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fcea99640 1 -- 172.21.15.1:0/1167559827 learned_addr learned my addr 172.21.15.1:0/1167559827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fcf29a640 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a247f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.882+0000 7f4fcfa9b640 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a2cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 -- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 msgr2=0x55617a247f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a247f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 -- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 msgr2=0x55617a2cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a2cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 -- 172.21.15.1:0/1167559827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a13fdf0 con 0x55617a24b530 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800a770 tx=0x7f4fb800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeb59d397008515 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcda97640 1 -- 172.21.15.1:0/1167559827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018070 con 0x55617a24b530 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcda97640 1 -- 172.21.15.1:0/1167559827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb8007600 con 0x55617a24b530 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fcda97640 1 -- 172.21.15.1:0/1167559827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80079b0 con 0x55617a24b530 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1167559827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 msgr2=0x55617a201c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.883+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800a770 tx=0x7f4fb800a860 comp rx=0 tx=0).stop 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1167559827 shutdown_connections 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a2cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a201c60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1167559827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a247f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1167559827 >> 172.21.15.1:0/1167559827 conn(0x55617a1fdb00 msgr2=0x55617a1fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1167559827 shutdown_connections 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1167559827 wait complete. 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 Processor -- start 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.884+0000 7f4fd1797f40 1 -- start start 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a2cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a36d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a23fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55617a2d0a90 con 0x55617a24b530 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55617a2d1010 con 0x55617a2d4540 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fd1797f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55617a2d3060 con 0x55617a22ef10 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a36d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a36d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33520/0 (socket says 172.21.15.1:33520) 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 -- 172.21.15.1:0/1027969826 learned_addr learned my addr 172.21.15.1:0/1027969826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcfa9b640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a23fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcf29a640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a2cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 -- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 msgr2=0x55617a2cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a2cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 -- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 msgr2=0x55617a23fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55617a2d4540 0x55617a23fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 -- 172.21.15.1:0/1027969826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a12bcb0 con 0x55617a24b530 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcf29a640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55617a22ef10 0x55617a2cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.885+0000 7f4fcea99640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a36d1d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016040 tx=0x7f4fb8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.886+0000 7f4fcca95640 1 -- 172.21.15.1:0/1027969826 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018040 con 0x55617a24b530 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.886+0000 7f4fcca95640 1 -- 172.21.15.1:0/1027969826 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb8022070 con 0x55617a24b530 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.886+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55617a2445c0 con 0x55617a24b530 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.886+0000 7f4fcca95640 1 -- 172.21.15.1:0/1027969826 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8007bf0 con 0x55617a24b530 2024-05-09T06:30:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.886+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617a244000 con 0x55617a24b530 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.887+0000 7f4fcca95640 1 -- 172.21.15.1:0/1027969826 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fb8013410 con 0x55617a24b530 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.887+0000 7f4fcca95640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fb006a810 0x7f4fb006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fcca95640 1 -- 172.21.15.1:0/1027969826 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f4fb808a530 con 0x55617a24b530 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fcf29a640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fb006a810 0x7f4fb006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55617a242ea0 0x55617a1ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) -- 0x55617a36d710 con 0x55617a242ea0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fcf29a640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fb006a810 0x7f4fb006ccd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8057000 tx=0x7f4fc8052000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fcfa9b640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55617a242ea0 0x55617a1ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.888+0000 7f4fcfa9b640 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55617a242ea0 0x55617a1ffe80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fcfa9b640 1 -- 172.21.15.1:0/1027969826 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v64'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4fc000d940 con 0x55617a242ea0 2024-05-09T06:30:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55617a242ea0 msgr2=0x55617a1ffe80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55617a242ea0 0x55617a1ffe80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fb006a810 msgr2=0x7f4fb006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fb006a810 0x7f4fb006ccd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8057000 tx=0x7f4fc8052000 comp rx=0 tx=0).stop 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 msgr2=0x55617a36d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 --2- 172.21.15.1:0/1027969826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55617a24b530 0x55617a36d1d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8016040 tx=0x7f4fb8003020 comp rx=0 tx=0).stop 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fcfa9b640 1 -- 172.21.15.1:0/1027969826 reap_dead start 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 shutdown_connections 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 >> 172.21.15.1:0/1027969826 conn(0x55617a1fdb00 msgr2=0x55617a1fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 shutdown_connections 2024-05-09T06:30:43.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:43.890+0000 7f4fd1797f40 1 -- 172.21.15.1:0/1027969826 wait complete. 2024-05-09T06:30:43.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:43.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-09T06:30:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.010+0000 7feda78d4640 1 Processor -- start 2024-05-09T06:30:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.011+0000 7feda78d4640 1 -- start start 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.011+0000 7feda78d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.011+0000 7feda78d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda78d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda00d1cc0 con 0x7feda00d2630 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda78d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7feda00d4d10 con 0x7feda00b2ed0 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda78d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7feda00d1f80 con 0x7feda00b2110 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda4e6f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda4e6f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:36314/0 (socket says 172.21.15.1:36314) 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:44872/0 (socket says 172.21.15.1:44872) 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 -- 172.21.15.1:0/3164980586 learned_addr learned my addr 172.21.15.1:0/3164980586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5670640 1 --1- 172.21.15.1:0/3164980586 >> v1:172.21.15.6:6789/0 conn(0x7feda00d2630 0x7feda015ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:60534/0 (socket says 172.21.15.1:60534) 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 -- 172.21.15.1:0/3164980586 >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 msgr2=0x7feda00ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 --2- 172.21.15.1:0/3164980586 >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 -- 172.21.15.1:0/3164980586 >> v1:172.21.15.6:6789/0 conn(0x7feda00d2630 legacy=0x7feda015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda5e71640 1 -- 172.21.15.1:0/3164980586 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda00b0c00 con 0x7feda00b2110 2024-05-09T06:30:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.012+0000 7feda4e6f640 1 --2- 172.21.15.1:0/3164980586 >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.013+0000 7feda5e71640 1 --2- 172.21.15.1:0/3164980586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed9002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37631bc6b9dd98a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.013+0000 7fed9ffff640 1 -- 172.21.15.1:0/3164980586 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90036070 con 0x7feda00b2110 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.013+0000 7fed9ffff640 1 -- 172.21.15.1:0/3164980586 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed90031600 con 0x7feda00b2110 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.013+0000 7fed9ffff640 1 -- 172.21.15.1:0/3164980586 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed900319d0 con 0x7feda00b2110 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 -- 172.21.15.1:0/3164980586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 msgr2=0x7feda00cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 --2- 172.21.15.1:0/3164980586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed90009e50 tx=0x7fed9002a710 comp rx=0 tx=0).stop 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 -- 172.21.15.1:0/3164980586 shutdown_connections 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 --2- 172.21.15.1:0/3164980586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00b2110 0x7feda00cf0e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 --2- 172.21.15.1:0/3164980586 >> v2:172.21.15.6:3300/0 conn(0x7feda00b2ed0 0x7feda00ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 -- 172.21.15.1:0/3164980586 >> 172.21.15.1:0/3164980586 conn(0x7feda00d3ee0 msgr2=0x7feda00d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 -- 172.21.15.1:0/3164980586 shutdown_connections 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.014+0000 7feda78d4640 1 -- 172.21.15.1:0/3164980586 wait complete. 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.015+0000 7feda78d4640 1 Processor -- start 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.015+0000 7feda78d4640 1 -- start start 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 0x7feda014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00d2630 0x7feda014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feda00d2b90 con 0x7feda00b2110 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7feda00d2a10 con 0x7feda00b2ed0 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda78d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7feda00d2d10 con 0x7feda00d2630 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33528/0 (socket says 172.21.15.1:33528) 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 -- 172.21.15.1:0/196441580 learned_addr learned my addr 172.21.15.1:0/196441580 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5e71640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00d2630 0x7feda014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda4e6f640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 0x7feda014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 -- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00d2630 msgr2=0x7feda014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00d2630 0x7feda014ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 -- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 msgr2=0x7feda014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 0x7feda014e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 -- 172.21.15.1:0/196441580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda00cb9c0 con 0x7feda00b2110 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda4e6f640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 0x7feda014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.016+0000 7feda5670640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c430 tx=0x7fed9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.017+0000 7fed9dffb640 1 -- 172.21.15.1:0/196441580 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980074e0 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.017+0000 7fed9dffb640 1 -- 172.21.15.1:0/196441580 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed98007680 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.017+0000 7fed9dffb640 1 -- 172.21.15.1:0/196441580 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98004a00 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.017+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda00ce1a0 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.017+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda00cbf40 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.018+0000 7fed9dffb640 1 -- 172.21.15.1:0/196441580 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed9801a020 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.019+0000 7fed9dffb640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.019+0000 7fed9dffb640 1 -- 172.21.15.1:0/196441580 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fed9808a890 con 0x7feda00b2110 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.019+0000 7feda4e6f640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.019+0000 7feda4e6f640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 0x7fed8c06e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed94009a20 tx=0x7fed94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 Processor -- start 2024-05-09T06:30:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 -- start start 2024-05-09T06:30:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559436474ef0 0x5594364784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5594363dd880 con 0x5594364a2430 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559436385520 con 0x559436474ef0 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.020+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5594363d9840 con 0x5594364a2050 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33530/0 (socket says 172.21.15.1:33530) 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 -- 172.21.15.1:0/1149842385 learned_addr learned my addr 172.21.15.1:0/1149842385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda5670640 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda5e71640 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559436474ef0 0x5594364784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 -- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 msgr2=0x5594364a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 -- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559436474ef0 msgr2=0x5594364784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559436474ef0 0x5594364784c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 -- 172.21.15.1:0/1149842385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559436379b40 con 0x5594364a2430 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fed94010ad0 tx=0x7fed94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3446cdfb5b55f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7fed86ffd640 1 -- 172.21.15.1:0/1149842385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9401c050 con 0x5594364a2430 2024-05-09T06:30:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7fed86ffd640 1 -- 172.21.15.1:0/1149842385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94026070 con 0x5594364a2430 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.021+0000 7fed86ffd640 1 -- 172.21.15.1:0/1149842385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9400f040 con 0x5594364a2430 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1149842385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 msgr2=0x5594364a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fed94010ad0 tx=0x7fed94013000 comp rx=0 tx=0).stop 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1149842385 shutdown_connections 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559436474ef0 0x5594364784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a2430 0x5594364a4da0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1149842385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1149842385 >> 172.21.15.1:0/1149842385 conn(0x5594364771b0 msgr2=0x5594364bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1149842385 shutdown_connections 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1149842385 wait complete. 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.022+0000 7fedaa5e7740 1 -- start start 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 Processor -- start 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55943647a0d0 con 0x5594364a0d10 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559436398ab0 con 0x5594364a2430 2024-05-09T06:30:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55943647a3b0 con 0x5594364a2050 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.023+0000 7fedaa5e7740 10 client.0 _ll_register_callbacks cb 0x5594360f0b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33538/0 (socket says 172.21.15.1:33538) 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 -- 172.21.15.1:0/3410628189 learned_addr learned my addr 172.21.15.1:0/3410628189 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5670640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda4e6f640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 -- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 msgr2=0x5594364a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 -- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 msgr2=0x5594364c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 -- 172.21.15.1:0/3410628189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594361bd7b0 con 0x5594364a0d10 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7feda5e71640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fed90009f70 tx=0x7fed90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a581ce97c43ff540 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fed7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90043070 con 0x5594364a0d10 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed94026070 con 0x5594364a0d10 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fed9003d870 con 0x5594364a0d10 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed9004d050 con 0x5594364a0d10 2024-05-09T06:30:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fedaa5e7740 10 client.6765 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.024+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55943621df40 con 0x5594364a0d10 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94026070 con 0x5594364a0d10 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fed90050020 con 0x5594364a0d10 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 10 client.6765 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 10 client.6765 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 20 client.6765 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5594364a1170 0x5594364c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 4 client.6765 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594363eafb0 con 0x5594364a1170 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7feda5670640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5594364a1170 0x5594364c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7fed7bfff640 10 client.6765 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.025+0000 7feda5670640 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5594364a1170 0x5594364c23e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e433895058a684 server_cookie=814604eb32c3fd13 in_seq=0 out_seq=0 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fed7bfff640 1 -- 172.21.15.1:0/3410628189 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594363eafb0 con 0x5594364a1170 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fed7bfff640 10 client.6765 handle_command_reply: tid=0 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 client.6765 shutdown 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 msgr2=0x5594364c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fed90009f70 tx=0x7fed90004ab0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 shutdown_connections 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c1ea0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5594364a1170 0x5594364c23e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/3410628189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 >> 172.21.15.1:0/3410628189 conn(0x559436474ef0 msgr2=0x5594364a4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fed7bfff640 0 client.6765 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 shutdown_connections 2024-05-09T06:30:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 1 -- 172.21.15.1:0/3410628189 wait complete. 2024-05-09T06:30:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.026+0000 7fedaa5e7740 20 client.6765 trim_cache size 0 max 16384 2024-05-09T06:30:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.044+0000 7fedaa5e7740 1 Processor -- start 2024-05-09T06:30:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.044+0000 7fedaa5e7740 1 -- start start 2024-05-09T06:30:44.034 INFO:tasks.ceph.mds.b.smithi006.stderr:2024-05-09T06:30:44.051+0000 7ff0cd720640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6732=0-4194304@1} caps={6732=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6732 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562048534c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2050 0x5594364c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2430 0x5594364c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feda0200bf0 con 0x5594364a0d10 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7feda01ffbf0 con 0x5594364a2050 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7feda00cd8c0 con 0x5594364a2430 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33542/0 (socket says 172.21.15.1:33542) 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 -- 172.21.15.1:0/746401818 learned_addr learned my addr 172.21.15.1:0/746401818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 -- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2430 msgr2=0x5594364c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2430 0x5594364c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 -- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2050 msgr2=0x5594364c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2050 0x5594364c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 -- 172.21.15.1:0/746401818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594360b10a0 con 0x5594364a0d10 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7feda5670640 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fed9800cdf0 tx=0x7fed98088b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8163c997772ca7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.045+0000 7fed867fc640 1 -- 172.21.15.1:0/746401818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98056c30 con 0x5594364a0d10 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fed867fc640 1 -- 172.21.15.1:0/746401818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed98019da0 con 0x5594364a0d10 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fed867fc640 1 -- 172.21.15.1:0/746401818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98054fa0 con 0x5594364a0d10 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 -- 172.21.15.1:0/746401818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 msgr2=0x5594364c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fed9800cdf0 tx=0x7fed98088b00 comp rx=0 tx=0).stop 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 -- 172.21.15.1:0/746401818 shutdown_connections 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2430 0x5594364c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2050 0x5594364c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/746401818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364c4710 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 -- 172.21.15.1:0/746401818 >> 172.21.15.1:0/746401818 conn(0x559436474ef0 msgr2=0x5594364e3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 -- 172.21.15.1:0/746401818 shutdown_connections 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.046+0000 7fedaa5e7740 1 -- 172.21.15.1:0/746401818 wait complete. 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.047+0000 7fedaa5e7740 1 -- start start 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.047+0000 7fedaa5e7740 1 Processor -- start 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55943647a0d0 con 0x5594364a0d10 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7feda00cdf10 con 0x5594364a2430 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5594364c3380 con 0x5594364a2050 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 10 client.0 _ll_register_callbacks cb 0x7fed94009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33550/0 (socket says 172.21.15.1:33550) 2024-05-09T06:30:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 -- 172.21.15.1:0/1284998469 learned_addr learned my addr 172.21.15.1:0/1284998469 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5e71640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 -- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 msgr2=0x5594364753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 -- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 msgr2=0x5594364c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 -- 172.21.15.1:0/1284998469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943653d9d0 con 0x5594364a0d10 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7feda5670640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fed980070a0 tx=0x7fed98015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c57a572cb53c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fed7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98076370 con 0x5594364a0d10 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fed98058b90 con 0x5594364a0d10 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed98019da0 con 0x5594364a0d10 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed9807e050 con 0x5594364a0d10 2024-05-09T06:30:44.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 10 client.6771 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.048+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55943653a370 con 0x5594364a0d10 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed98019da0 con 0x5594364a0d10 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fed98081020 con 0x5594364a0d10 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 10 client.6771 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 10 client.6771 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 20 client.6771 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x559436475910 0x5594364c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 4 client.6771 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x5594363e5300 con 0x559436475910 2024-05-09T06:30:44.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x559436475910 0x5594364c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7fed7bfff640 10 client.6771 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.049+0000 7feda4e6f640 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x559436475910 0x5594364c4b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e98f1485bd9f29 server_cookie=f7d5f0831dee5f38 in_seq=0 out_seq=0 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fed7bfff640 1 -- 172.21.15.1:0/1284998469 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5594363e5300 con 0x559436475910 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fed7bfff640 10 client.6771 handle_command_reply: tid=0 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 client.6771 shutdown 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 msgr2=0x5594364b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fed980070a0 tx=0x7fed98015040 comp rx=0 tx=0).stop 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 shutdown_connections 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5594364a2430 0x5594364c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x559436475910 0x5594364c4b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5594364a2050 0x5594364753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 --2- 172.21.15.1:0/1284998469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5594364a0d10 0x5594364b5e70 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fed7bfff640 0 client.6771 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 >> 172.21.15.1:0/1284998469 conn(0x559436474ef0 msgr2=0x5594364e3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 shutdown_connections 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 1 -- 172.21.15.1:0/1284998469 wait complete. 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.050+0000 7fedaa5e7740 20 client.6771 trim_cache size 0 max 16384 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "efb96016-0a47-4e90-bef3-fca7e233d923", 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:30:44.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 msgr2=0x7fed8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 0x7fed8c06e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed94009a20 tx=0x7fed94008040 comp rx=0 tx=0).stop 2024-05-09T06:30:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 msgr2=0x7feda0160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c430 tx=0x7fed9800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 shutdown_connections 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feda00d2630 0x7feda014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda00b2ed0 0x7feda014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 --2- 172.21.15.1:0/196441580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feda00b2110 0x7feda0160f90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 >> 172.21.15.1:0/196441580 conn(0x7feda00d3ee0 msgr2=0x7feda00d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 shutdown_connections 2024-05-09T06:30:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.051+0000 7feda78d4640 1 -- 172.21.15.1:0/196441580 wait complete. 2024-05-09T06:30:44.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:30:44.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-09T06:30:44.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.171+0000 7f59efc15640 1 Processor -- start 2024-05-09T06:30:44.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 -- start start 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f59e806dd00 0x7f59e815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e806cf40 con 0x7f59e815c4c0 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f59e8156980 con 0x7f59e806dd00 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.172+0000 7f59efc15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f59e806e180 con 0x7f59e815cde0 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed9b1640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f59e815c4c0 0x7f59e80b4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59172/0 (socket says 172.21.15.1:59172) 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed9b1640 1 -- 172.21.15.1:0/1329138497 learned_addr learned my addr 172.21.15.1:0/1329138497 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1329138497 >> v2:172.21.15.6:3300/0 conn(0x7f59e806dd00 0x7f59e815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e8070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ec9af640 1 -- 172.21.15.1:0/1329138497 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826008094 0 0) 0x7f59e806cf40 con 0x7f59e815c4c0 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ec9af640 1 -- 172.21.15.1:0/1329138497 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d4003520 con 0x7f59e815c4c0 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 msgr2=0x7f59e8070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e8070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1329138497 >> v1:172.21.15.1:6789/0 conn(0x7f59e815c4c0 legacy=0x7f59e80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1329138497 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e806d220 con 0x7f59e806dd00 2024-05-09T06:30:44.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.173+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e8070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.174+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e815c8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f59d80097a0 tx=0x7f59d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95cb9d91fe7fa766 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.174+0000 7f59ec9af640 1 -- 172.21.15.1:0/1329138497 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8037070 con 0x7f59e806dd00 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.174+0000 7f59ec9af640 1 -- 172.21.15.1:0/1329138497 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8032440 con 0x7f59e806dd00 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.174+0000 7f59ec9af640 1 -- 172.21.15.1:0/1329138497 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d80327f0 con 0x7f59e806dd00 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 -- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 msgr2=0x7f59e815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e815c8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f59d80097a0 tx=0x7f59d802a180 comp rx=0 tx=0).stop 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 -- 172.21.15.1:0/1329138497 shutdown_connections 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e8070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 --2- 172.21.15.1:0/1329138497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e815c8a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 -- 172.21.15.1:0/1329138497 >> 172.21.15.1:0/1329138497 conn(0x7f59e80cc4d0 msgr2=0x7f59e80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 -- 172.21.15.1:0/1329138497 shutdown_connections 2024-05-09T06:30:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.175+0000 7f59efc15640 1 -- 172.21.15.1:0/1329138497 wait complete. 2024-05-09T06:30:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.176+0000 7f59efc15640 1 Processor -- start 2024-05-09T06:30:44.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.176+0000 7f59efc15640 1 -- start start 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e80ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59e815d270 con 0x7f59e815c4c0 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e806d870 con 0x7f59e806dd00 2024-05-09T06:30:44.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59efc15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f59e80b54f0 con 0x7f59e815cde0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33558/0 (socket says 172.21.15.1:33558) 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 -- 172.21.15.1:0/2562687183 learned_addr learned my addr 172.21.15.1:0/2562687183 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e80ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 -- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 msgr2=0x7f59e813fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e813fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 -- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 msgr2=0x7f59e80ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e80ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 -- 172.21.15.1:0/2562687183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e813f620 con 0x7f59e815c4c0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f59d8035040 tx=0x7f59d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59de7fc640 1 -- 172.21.15.1:0/2562687183 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8037040 con 0x7f59e815c4c0 2024-05-09T06:30:44.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.177+0000 7f59de7fc640 1 -- 172.21.15.1:0/2562687183 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d80044c0 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.178+0000 7f59de7fc640 1 -- 172.21.15.1:0/2562687183 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8006e10 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.178+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81f6ea0 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.178+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81f7460 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.179+0000 7f59de7fc640 1 -- 172.21.15.1:0/2562687183 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59d8044020 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.180+0000 7f59de7fc640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 0x7f59cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.180+0000 7f59de7fc640 1 -- 172.21.15.1:0/2562687183 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f59d80ab110 con 0x7f59e815c4c0 2024-05-09T06:30:44.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.180+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 0x7f59cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.180+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 0x7f59cc06e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005880 tx=0x7f59e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 Processor -- start 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 -- start start 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef198b9650 0x55ef1986bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c030 0x55ef19870e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef197d5d00 con 0x55ef1989c770 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef1977d4c0 con 0x55ef198b9650 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ef197d1cc0 con 0x55ef1989c030 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33566/0 (socket says 172.21.15.1:33566) 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.181+0000 7f59ed1b0640 1 -- 172.21.15.1:0/3414460627 learned_addr learned my addr 172.21.15.1:0/3414460627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c030 0x55ef19870e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef198b9650 0x55ef1986bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 -- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c030 msgr2=0x55ef19870e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c030 0x55ef19870e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 -- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef198b9650 msgr2=0x55ef1986bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef198b9650 0x55ef1986bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 -- 172.21.15.1:0/3414460627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef195b37d0 con 0x55ef1989c770 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f59d80a9420 tx=0x7f59d8078e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e29659ce719b59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59cb7fe640 1 -- 172.21.15.1:0/3414460627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d807d5e0 con 0x55ef1989c770 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59cb7fe640 1 -- 172.21.15.1:0/3414460627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8081e10 con 0x55ef1989c770 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.182+0000 7f59cb7fe640 1 -- 172.21.15.1:0/3414460627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8091120 con 0x55ef1989c770 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- 172.21.15.1:0/3414460627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 msgr2=0x55ef1989baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f59d80a9420 tx=0x7f59d8078e30 comp rx=0 tx=0).stop 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- 172.21.15.1:0/3414460627 shutdown_connections 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c030 0x55ef19870e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef1989c770 0x55ef1989baf0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 --2- 172.21.15.1:0/3414460627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef198b9650 0x55ef1986bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- 172.21.15.1:0/3414460627 >> 172.21.15.1:0/3414460627 conn(0x55ef1986e250 msgr2=0x55ef198b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- 172.21.15.1:0/3414460627 shutdown_connections 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- 172.21.15.1:0/3414460627 wait complete. 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 -- start start 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.183+0000 7f59f2a04740 1 Processor -- start 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef198b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef19897ac0 0x55ef1989b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef19870400 con 0x55ef198b9650 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef19790b90 con 0x55ef19897ac0 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ef1986f7f0 con 0x55ef1989c770 2024-05-09T06:30:44.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59f2a04740 10 client.0 _ll_register_callbacks cb 0x55ef197ccda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59ed1b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33582/0 (socket says 172.21.15.1:33582) 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1975200074 learned_addr learned my addr 172.21.15.1:0/1975200074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef19897ac0 0x55ef1989b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.184+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef198b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 msgr2=0x55ef198b89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef198b89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef19897ac0 msgr2=0x55ef1989b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef19897ac0 0x55ef1989b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 -- 172.21.15.1:0/1975200074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef194e93f0 con 0x55ef198b9650 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f59d8000c00 tx=0x7f59d8008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e66c6a8019912a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d807d5e0 con 0x55ef198b9650 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d8081e10 con 0x55ef198b9650 2024-05-09T06:30:44.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f59d807d170 con 0x55ef198b9650 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59d807d370 con 0x55ef198b9650 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 10 client.6783 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef194a90f0 con 0x55ef198b9650 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8081e10 con 0x55ef198b9650 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f59d803a030 con 0x55ef198b9650 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 10 client.6783 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 10 client.6783 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.185+0000 7f59f2a04740 20 client.6783 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef1989d510 0x55ef1989d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59f2a04740 4 client.6783 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef197e3430 con 0x55ef1989d510 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef1989d510 0x55ef1989d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef1989d510 0x55ef1989d920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da5c2c6a497eba7 server_cookie=2ecc30a483eaf05a in_seq=0 out_seq=0 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.186+0000 7f59c8ff9640 10 client.6783 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1975200074 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef197e3430 con 0x55ef1989d510 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59c8ff9640 10 client.6783 handle_command_reply: tid=0 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 client.6783 shutdown 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 msgr2=0x55ef1989b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f59d8000c00 tx=0x7f59d8008000 comp rx=0 tx=0).stop 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 shutdown_connections 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef19897ac0 0x55ef1989b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef198b9650 0x55ef1989b0c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef1989d510 0x55ef1989d920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1975200074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef198b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59c8ff9640 0 client.6783 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 >> 172.21.15.1:0/1975200074 conn(0x55ef1989c030 msgr2=0x55ef19870800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 shutdown_connections 2024-05-09T06:30:44.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 1 -- 172.21.15.1:0/1975200074 wait complete. 2024-05-09T06:30:44.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.187+0000 7f59f2a04740 20 client.6783 trim_cache size 0 max 16384 2024-05-09T06:30:44.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.205+0000 7f59f2a04740 1 Processor -- start 2024-05-09T06:30:44.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.205+0000 7f59f2a04740 1 -- start start 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef1992aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef1986ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8200a50 con 0x55ef19897ac0 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ffa90 con 0x55ef1989d510 2024-05-09T06:30:44.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f59e8133b50 con 0x55ef1989c770 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33590/0 (socket says 172.21.15.1:33590) 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 -- 172.21.15.1:0/4252755597 learned_addr learned my addr 172.21.15.1:0/4252755597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef1986ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef1992aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 -- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 msgr2=0x55ef1992aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef1992aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 -- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 msgr2=0x55ef1986ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef1986ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 -- 172.21.15.1:0/4252755597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1986b850 con 0x55ef19897ac0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f59e400f8b0 tx=0x7f59e4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879bc18b6712ab78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59caffd640 1 -- 172.21.15.1:0/4252755597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e402d070 con 0x55ef19897ac0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59caffd640 1 -- 172.21.15.1:0/4252755597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e40159e0 con 0x55ef19897ac0 2024-05-09T06:30:44.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.206+0000 7f59caffd640 1 -- 172.21.15.1:0/4252755597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4015bd0 con 0x55ef19897ac0 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- 172.21.15.1:0/4252755597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 msgr2=0x55ef198daa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f59e400f8b0 tx=0x7f59e4000ed0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- 172.21.15.1:0/4252755597 shutdown_connections 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef1986ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef1992aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 --2- 172.21.15.1:0/4252755597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198daa30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- 172.21.15.1:0/4252755597 >> 172.21.15.1:0/4252755597 conn(0x55ef1989c030 msgr2=0x55ef1989d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- 172.21.15.1:0/4252755597 shutdown_connections 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- 172.21.15.1:0/4252755597 wait complete. 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.207+0000 7f59f2a04740 1 -- start start 2024-05-09T06:30:44.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.208+0000 7f59f2a04740 1 Processor -- start 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.208+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.208+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef199269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.208+0000 7f59f2a04740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef19926f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.208+0000 7f59f2a04740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef1989f740 con 0x55ef19897ac0 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ff440 con 0x55ef1989d510 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ef19898f60 con 0x55ef1989c770 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 10 client.0 _ll_register_callbacks cb 0x7f59d8091070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:30:44.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33604/0 (socket says 172.21.15.1:33604) 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 -- 172.21.15.1:0/1618850600 learned_addr learned my addr 172.21.15.1:0/1618850600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef19926f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef199269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 -- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 msgr2=0x55ef199269e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef199269e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 -- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 msgr2=0x55ef19926f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef19926f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 -- 172.21.15.1:0/1618850600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef198d9f20 con 0x55ef19897ac0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ee1b2640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef19926f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef199269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59ed9b1640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f59e402b040 tx=0x7f59e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e5324f3b059b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e402d070 con 0x55ef19897ac0 2024-05-09T06:30:44.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f59e4004cb0 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e40159e0 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59e4005030 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 10 client.6789 fetch_fsmap learned FSMap version 35 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.209+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef198c55f0 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e40159e0 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f59e40051d0 con 0x55ef19897ac0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 10 client.6789 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 10 client.6789 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 20 client.6789 populate_metadata read hostname 'smithi001' 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef198b9650 0x55ef19897f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 4 client.6789 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef198b71f0 con 0x55ef198b9650 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef198b9650 0x55ef19897f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59c8ff9640 10 client.6789 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59ed1b0640 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef198b9650 0x55ef19897f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55af94181d8a756d server_cookie=956f39e09cc70138 in_seq=0 out_seq=0 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59c8ff9640 1 -- 172.21.15.1:0/1618850600 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x55ef198b71f0 con 0x55ef198b9650 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59c8ff9640 10 client.6789 handle_command_reply: tid=0 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.210+0000 7f59f2a04740 1 client.6789 shutdown 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 msgr2=0x55ef198d99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f59e402b040 tx=0x7f59e4000bc0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 shutdown_connections 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef1989d510 0x55ef19926f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ef198b9650 0x55ef19897f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ef1989c770 0x55ef199269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 --2- 172.21.15.1:0/1618850600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19897ac0 0x55ef198d99e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59c8ff9640 0 client.6789 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 >> 172.21.15.1:0/1618850600 conn(0x55ef1989c030 msgr2=0x55ef19871980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 shutdown_connections 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 1 -- 172.21.15.1:0/1618850600 wait complete. 2024-05-09T06:30:44.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59f2a04740 20 client.6789 trim_cache size 0 max 16384 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "efb96016-0a47-4e90-bef3-fca7e233d923": { 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "efb96016-0a47-4e90-bef3-fca7e233d923", 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive" 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:30:44.199 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:30:44.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 msgr2=0x7f59cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 0x7f59cc06e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005880 tx=0x7f59e4009040 comp rx=0 tx=0).stop 2024-05-09T06:30:44.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 msgr2=0x7f59e813f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:44.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f59d8035040 tx=0x7f59d80046c0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 shutdown_connections 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f59cc06be30 0x7f59cc06e2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59e815cde0 0x7f59e813fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59e815c4c0 0x7f59e813f850 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 --2- 172.21.15.1:0/2562687183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59e806dd00 0x7f59e80ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 >> 172.21.15.1:0/2562687183 conn(0x7f59e80cc4d0 msgr2=0x7f59e80cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 shutdown_connections 2024-05-09T06:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:30:44.211+0000 7f59efc15640 1 -- 172.21.15.1:0/2562687183 wait complete. 2024-05-09T06:30:44.204 INFO:tasks.cephfs.filesystem:scrub status for tag:efb96016-0a47-4e90-bef3-fca7e233d923 - {'path': '/', 'tag': 'efb96016-0a47-4e90-bef3-fca7e233d923', 'options': 'recursive'} 2024-05-09T06:30:47.592 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:30:47.611+0000 7f5fea9bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T06:30:47.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:30:47.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:31:14.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:14.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-09T06:31:14.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.342+0000 7f6aa16bd640 1 Processor -- start 2024-05-09T06:31:14.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.342+0000 7f6aa16bd640 1 -- start start 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.343+0000 7f6aa16bd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a9c15c4c0 0x7f6a9c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.343+0000 7f6aa16bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 0x7f6a9c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.343+0000 7f6aa16bd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c06cf40 con 0x7f6a9c0b4de0 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.343+0000 7f6aa16bd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9c156980 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.343+0000 7f6aa16bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c06d870 con 0x7f6a9c15cfb0 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9a7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6a9c0b4de0 0x7f6a9c15c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39056/0 (socket says 172.21.15.1:39056) 2024-05-09T06:31:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/2081741402 learned_addr learned my addr 172.21.15.1:0/2081741402 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 0x7f6a9c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2081741402 >> v2:172.21.15.6:3300/0 conn(0x7f6a9c15c4c0 0x7f6a9c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217851527 0 0) 0x7f6a9c06cf40 con 0x7f6a9c0b4de0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a84003520 con 0x7f6a9c0b4de0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289192177 0 0) 0x7f6a84003520 con 0x7f6a9c0b4de0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9affd640 1 -- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 msgr2=0x7f6a9c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 0x7f6a9c070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9affd640 1 -- 172.21.15.1:0/2081741402 >> v1:172.21.15.1:6789/0 conn(0x7f6a9c0b4de0 legacy=0x7f6a9c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9affd640 1 -- 172.21.15.1:0/2081741402 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c06d220 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.344+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 0x7f6a9c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.345+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c06dd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0097a0 tx=0x7f6a8c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda939de6b8a658d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.345+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c037070 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.345+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8c02ab10 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.345+0000 7f6a99ffb640 1 -- 172.21.15.1:0/2081741402 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900019f0 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 -- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15c4c0 msgr2=0x7f6a9c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c06dd00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0097a0 tx=0x7f6a8c02a080 comp rx=0 tx=0).stop 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 -- 172.21.15.1:0/2081741402 shutdown_connections 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c15cfb0 0x7f6a9c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/2081741402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c06dd00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 -- 172.21.15.1:0/2081741402 >> 172.21.15.1:0/2081741402 conn(0x7f6a9c0cc4d0 msgr2=0x7f6a9c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 -- 172.21.15.1:0/2081741402 shutdown_connections 2024-05-09T06:31:14.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.346+0000 7f6aa16bd640 1 -- 172.21.15.1:0/2081741402 wait complete. 2024-05-09T06:31:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.347+0000 7f6aa16bd640 1 Processor -- start 2024-05-09T06:31:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.347+0000 7f6aa16bd640 1 -- start start 2024-05-09T06:31:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c0b4de0 0x7f6a9c150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 0x7f6a9c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c0b2dc0 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c0b2c40 con 0x7f6a9c15cfb0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6aa16bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c0b5260 con 0x7f6a9c0b4de0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45186/0 (socket says 172.21.15.1:45186) 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.348+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/1112684769 learned_addr learned my addr 172.21.15.1:0/1112684769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c0b4de0 0x7f6a9c150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 0x7f6a9c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c0b4de0 msgr2=0x7f6a9c150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c0b4de0 0x7f6a9c150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 msgr2=0x7f6a9c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 0x7f6a9c06dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/1112684769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c156bb0 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 0x7f6a9c06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001560 tx=0x7f6a9000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a837fe640 1 -- 172.21.15.1:0/1112684769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017070 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a837fe640 1 -- 172.21.15.1:0/1112684769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90012400 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6a837fe640 1 -- 172.21.15.1:0/1112684769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900127f0 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c156e50 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.349+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c157410 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.351+0000 7f6a837fe640 1 -- 172.21.15.1:0/1112684769 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a9001a070 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.351+0000 7f6a837fe640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 0x7f6a8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.351+0000 7f6a837fe640 1 -- 172.21.15.1:0/1112684769 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6a9008b090 con 0x7f6a9c15c4c0 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.352+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 0x7f6a8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.352+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 0x7f6a8806e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c034000 tx=0x7f6a8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.352+0000 7f6aa43d2740 1 Processor -- start 2024-05-09T06:31:14.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 -- start start 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9ec83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 0x55efb9ecd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55efb9e2dd00 con 0x55efb9ef8b80 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55efb9dd54c0 con 0x55efb9ef8440 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55efb9e29cc0 con 0x55efb9f15a60 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45194/0 (socket says 172.21.15.1:45194) 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.353+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/363056539 learned_addr learned my addr 172.21.15.1:0/363056539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 0x55efb9ecd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9affd640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9ec83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 msgr2=0x55efb9ec83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9ec83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 msgr2=0x55efb9ecd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 0x55efb9ecd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/363056539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb9c0b7d0 con 0x55efb9ef8b80 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 0x55efb9ecd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9affd640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9ec83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6a900590c0 tx=0x7f6a90059130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6dc4f44f8468f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a817fa640 1 -- 172.21.15.1:0/363056539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90071170 con 0x55efb9ef8b80 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a817fa640 1 -- 172.21.15.1:0/363056539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90057430 con 0x55efb9ef8b80 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.354+0000 7f6a817fa640 1 -- 172.21.15.1:0/363056539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900557a0 con 0x55efb9ef8b80 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- 172.21.15.1:0/363056539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 msgr2=0x55efb9ef7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6a900590c0 tx=0x7f6a90059130 comp rx=0 tx=0).stop 2024-05-09T06:31:14.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- 172.21.15.1:0/363056539 shutdown_connections 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef8440 0x55efb9ecd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef7f00 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/363056539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9ec83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- 172.21.15.1:0/363056539 >> 172.21.15.1:0/363056539 conn(0x55efb9eca660 msgr2=0x55efb9f12f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- 172.21.15.1:0/363056539 shutdown_connections 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- 172.21.15.1:0/363056539 wait complete. 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 -- start start 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.355+0000 7f6aa43d2740 1 Processor -- start 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9f15a60 0x55efb9f14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef7590 0x55efb9ef7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55efb9ecc810 con 0x55efb9ef8b80 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55efb9de8b90 con 0x55efb9f15a60 2024-05-09T06:31:14.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55efb9ecbc00 con 0x55efb9ef7590 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6a9affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6aa43d2740 10 client.0 _ll_register_callbacks cb 0x55efb9e24da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6a9affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45204/0 (socket says 172.21.15.1:45204) 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6a9affd640 1 -- 172.21.15.1:0/2703997424 learned_addr learned my addr 172.21.15.1:0/2703997424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef7590 0x55efb9ef7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.356+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9f15a60 0x55efb9f14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 -- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef7590 msgr2=0x55efb9ef7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef7590 0x55efb9ef7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 -- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9f15a60 msgr2=0x55efb9f14cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9f15a60 0x55efb9f14cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 -- 172.21.15.1:0/2703997424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb9b413f0 con 0x55efb9ef8b80 2024-05-09T06:31:14.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a9affd640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009890 tx=0x7f6a8c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25646b273212f57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c037050 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6a8c003ba0 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a90057430 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c057050 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6aa43d2740 10 client.6801 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55efb9b010f0 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90057430 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.357+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f6a8c05a020 con 0x55efb9ef8b80 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 10 client.6801 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 10 client.6801 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 20 client.6801 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:14.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef9620 0x55efb9ef9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 4 client.6801 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:14.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efb9e3b430 con 0x55efb9ef9620 2024-05-09T06:31:14.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef9620 0x55efb9ef9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef9620 0x55efb9ef9a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f012dc3d0cafc1 server_cookie=79131c7bc71b7448 in_seq=0 out_seq=0 2024-05-09T06:31:14.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.358+0000 7f6a79ffb640 10 client.6801 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6a79ffb640 1 -- 172.21.15.1:0/2703997424 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efb9e3b430 con 0x55efb9ef9620 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6a79ffb640 10 client.6801 handle_command_reply: tid=0 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 client.6801 shutdown 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 msgr2=0x55efb9ef3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009890 tx=0x7f6a8c02c070 comp rx=0 tx=0).stop 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 shutdown_connections 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef9620 0x55efb9ef9a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef7590 0x55efb9ef7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6a79ffb640 0 client.6801 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9f15a60 0x55efb9f14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2703997424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9ef3850 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 >> 172.21.15.1:0/2703997424 conn(0x55efb9ef8440 msgr2=0x55efb9f16320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 shutdown_connections 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2703997424 wait complete. 2024-05-09T06:31:14.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.359+0000 7f6aa43d2740 20 client.6801 trim_cache size 0 max 16384 2024-05-09T06:31:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.377+0000 7f6aa43d2740 1 Processor -- start 2024-05-09T06:31:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.377+0000 7f6aa43d2740 1 -- start start 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9ec3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9f36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1fc7a0 con 0x55efb9ef8b80 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c0cf0b0 con 0x55efb9ef7590 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c158e60 con 0x55efb9f15a60 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45206/0 (socket says 172.21.15.1:45206) 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/2619944340 learned_addr learned my addr 172.21.15.1:0/2619944340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 msgr2=0x55efb9f36e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9f36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 msgr2=0x55efb9ec3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9ec3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 -- 172.21.15.1:0/2619944340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb9f13600 con 0x55efb9ef8b80 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6a90059170 tx=0x7f6a9001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ab5f74f36323d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a80ff9640 1 -- 172.21.15.1:0/2619944340 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900557a0 con 0x55efb9ef8b80 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a80ff9640 1 -- 172.21.15.1:0/2619944340 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90061880 con 0x55efb9ef8b80 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.378+0000 7f6a80ff9640 1 -- 172.21.15.1:0/2619944340 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90055940 con 0x55efb9ef8b80 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2619944340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 msgr2=0x55efb9efa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6a90059170 tx=0x7f6a9001e000 comp rx=0 tx=0).stop 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2619944340 shutdown_connections 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9f15a60 0x55efb9f36e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9ef8b80 0x55efb9efa670 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/2619944340 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9ec3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2619944340 >> 172.21.15.1:0/2619944340 conn(0x55efb9ef8440 msgr2=0x55efb9ef9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2619944340 shutdown_connections 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- 172.21.15.1:0/2619944340 wait complete. 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.379+0000 7f6aa43d2740 1 -- start start 2024-05-09T06:31:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 Processor -- start 2024-05-09T06:31:14.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9f16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef8b80 0x55efb9f13e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c0cea60 con 0x55efb9f15a60 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55efb9dad5c0 con 0x55efb9ef7590 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55efb9ef4320 con 0x55efb9ef8b80 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6aa43d2740 10 client.0 _ll_register_callbacks cb 0x7f6a8c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6a9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.380+0000 7f6a9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45216/0 (socket says 172.21.15.1:45216) 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 -- 172.21.15.1:0/1294289379 learned_addr learned my addr 172.21.15.1:0/1294289379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9f16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9a7fc640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef8b80 0x55efb9f13e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 -- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef8b80 msgr2=0x55efb9f13e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef8b80 0x55efb9f13e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 -- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 msgr2=0x55efb9f16720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9f16720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 -- 172.21.15.1:0/1294289379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb9f16c60 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a9b7fe640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009bf0 tx=0x7f6a940136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc41b91b4cd7774 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401c070 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6a94016460 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a90061880 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6a94027050 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6aa43d2740 10 client.6807 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55efb9f26480 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.381+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90061880 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f6a9402a020 con 0x55efb9f15a60 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 10 client.6807 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 10 client.6807 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 20 client.6807 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef6b10 0x55efb9f14910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 4 client.6807 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efb9e35780 con 0x55efb9ef6b10 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef6b10 0x55efb9f14910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a79ffb640 10 client.6807 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a9affd640 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef6b10 0x55efb9f14910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300673da2f26bf93 server_cookie=6241b76887db2abd in_seq=0 out_seq=0 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a79ffb640 1 -- 172.21.15.1:0/1294289379 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55efb9e35780 con 0x55efb9ef6b10 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6a79ffb640 10 client.6807 handle_command_reply: tid=0 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.382+0000 7f6aa43d2740 1 client.6807 shutdown 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 msgr2=0x55efb9f143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009bf0 tx=0x7f6a940136b0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 shutdown_connections 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55efb9f15a60 0x55efb9f143d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55efb9ef6b10 0x55efb9f14910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55efb9ef8b80 0x55efb9f13e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 --2- 172.21.15.1:0/1294289379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55efb9ef7590 0x55efb9f16720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6a79ffb640 0 client.6807 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 >> 172.21.15.1:0/1294289379 conn(0x55efb9ef8440 msgr2=0x55efb9ef4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 shutdown_connections 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 1 -- 172.21.15.1:0/1294289379 wait complete. 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa43d2740 20 client.6807 trim_cache size 0 max 16384 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:31:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:31:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:31:14.371 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:31:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 msgr2=0x7f6a8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 0x7f6a8806e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c034000 tx=0x7f6a8c02e000 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 msgr2=0x7f6a9c15a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001560 tx=0x7f6a9000a430 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 shutdown_connections 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6a8806be30 0x7f6a8806e2f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a9c15cfb0 0x7f6a9c06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9c15c4c0 0x7f6a9c15a840 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 --2- 172.21.15.1:0/1112684769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a9c0b4de0 0x7f6a9c150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 >> 172.21.15.1:0/1112684769 conn(0x7f6a9c0cc4d0 msgr2=0x7f6a9c15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 shutdown_connections 2024-05-09T06:31:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.383+0000 7f6aa16bd640 1 -- 172.21.15.1:0/1112684769 wait complete. 2024-05-09T06:31:14.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:31:14.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:14.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.498+0000 7f5de8363640 1 Processor -- start 2024-05-09T06:31:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.498+0000 7f5de8363640 1 -- start start 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.499+0000 7f5de8363640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5de00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de8363640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de006eb40 con 0x7f5de00b2a60 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de8363640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de006ee20 con 0x7f5de00b4f70 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de8363640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5de006d8e0 con 0x7f5de01454b0 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de58fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5de00b2a60 0x7f5de00b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39062/0 (socket says 172.21.15.1:39062) 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de58fe640 1 -- 172.21.15.1:0/116430503 learned_addr learned my addr 172.21.15.1:0/116430503 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:14.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731793394 0 0) 0x7f5de006eb40 con 0x7f5de00b2a60 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de60ff640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5de00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dc8003520 con 0x7f5de00b2a60 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396805673 0 0) 0x7f5dc8003520 con 0x7f5de00b2a60 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 msgr2=0x7f5de00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5de00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 >> v1:172.21.15.6:6789/0 conn(0x7f5de00b4f70 legacy=0x7f5de0070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de006dbc0 con 0x7f5de00b2a60 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f5dd4001670 con 0x7f5de00b2a60 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 >> v1:172.21.15.1:6789/0 conn(0x7f5de00b2a60 legacy=0x7f5de00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5dc80039e0 0x7f5dc8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5dc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8004d40 0x7f5dc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5de006d8e0 con 0x7f5dc8004d40 2024-05-09T06:31:14.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc800ba70 con 0x7f5dc80039e0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc800bc70 con 0x7f5de01454b0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de60ff640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5dc80039e0 0x7f5dc8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.500+0000 7f5de6900640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8004d40 0x7f5dc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de58fe640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5dc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de60ff640 1 -- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 msgr2=0x7f5dc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de60ff640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5de01454b0 0x7f5dc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de60ff640 1 -- 172.21.15.1:0/116430503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8004d40 msgr2=0x7f5dc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de60ff640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8004d40 0x7f5dc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de60ff640 1 -- 172.21.15.1:0/116430503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc8003520 con 0x7f5dc80039e0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.501+0000 7f5de6900640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8004d40 0x7f5dc80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de60ff640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5dc80039e0 0x7f5dc8003e00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009a90 tx=0x7f5dc8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a550a79f8f19c95e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8036070 con 0x7f5dc80039e0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd8031400 con 0x7f5dc80039e0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de50fd640 1 -- 172.21.15.1:0/116430503 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8009690 con 0x7f5dc80039e0 2024-05-09T06:31:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 -- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5dc80039e0 msgr2=0x7f5dc8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 --2- 172.21.15.1:0/116430503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5dc80039e0 0x7f5dc8003e00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009a90 tx=0x7f5dc8005830 comp rx=0 tx=0).stop 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de6900640 1 -- 172.21.15.1:0/116430503 reap_dead start 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 -- 172.21.15.1:0/116430503 shutdown_connections 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 -- 172.21.15.1:0/116430503 >> 172.21.15.1:0/116430503 conn(0x7f5de00cc4b0 msgr2=0x7f5de00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 -- 172.21.15.1:0/116430503 shutdown_connections 2024-05-09T06:31:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.502+0000 7f5de8363640 1 -- 172.21.15.1:0/116430503 wait complete. 2024-05-09T06:31:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 Processor -- start 2024-05-09T06:31:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 -- start start 2024-05-09T06:31:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5dc80039e0 0x7f5de015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5de013eb60 0x7f5de00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5de006e260 con 0x7f5de013efe0 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5de006e0e0 con 0x7f5de013eb60 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.504+0000 7f5de8363640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00b4320 con 0x7f5dc80039e0 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45240/0 (socket says 172.21.15.1:45240) 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 -- 172.21.15.1:0/379227491 learned_addr learned my addr 172.21.15.1:0/379227491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de60ff640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5dc80039e0 0x7f5de015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de58fe640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5de013eb60 0x7f5de00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 -- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5dc80039e0 msgr2=0x7f5de015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5dc80039e0 0x7f5de015b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 -- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5de013eb60 msgr2=0x7f5de00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5de013eb60 0x7f5de00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 -- 172.21.15.1:0/379227491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00ce070 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de6900640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b5400 tx=0x7f5dd000feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0001a70 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd0018070 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de00ce350 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0003da0 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.505+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de013e360 con 0x7f5de013efe0 2024-05-09T06:31:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.506+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de00752b0 con 0x7f5de013efe0 2024-05-09T06:31:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.507+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5dd00050d0 con 0x7f5de013efe0 2024-05-09T06:31:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.507+0000 7f5ddeffd640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 0x7f5dcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.508+0000 7f5de60ff640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 0x7f5dcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.508+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f5dd008acc0 con 0x7f5de013efe0 2024-05-09T06:31:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.508+0000 7f5de60ff640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 0x7f5dcc06e320 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8033000 tx=0x7f5dd8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.512+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5dd0059500 con 0x7f5de013efe0 2024-05-09T06:31:14.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.735+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de0072fa0 con 0x7f5de013efe0 2024-05-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.735+0000 7f5ddeffd640 1 -- 172.21.15.1:0/379227491 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5167 (secure 0 0 0) 0x7f5dd0058e00 con 0x7f5de013efe0 2024-05-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:30:47.607710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":6,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 msgr2=0x7f5dcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 0x7f5dcc06e320 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8033000 tx=0x7f5dd8008030 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 msgr2=0x7f5de00addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b5400 tx=0x7f5dd000feb0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 shutdown_connections 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5dcc06be60 0x7f5dcc06e320 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de013efe0 0x7f5de00addb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5de013eb60 0x7f5de00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 --2- 172.21.15.1:0/379227491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5dc80039e0 0x7f5de015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 >> 172.21.15.1:0/379227491 conn(0x7f5de00cc4b0 msgr2=0x7f5de00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 shutdown_connections 2024-05-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.737+0000 7f5de8363640 1 -- 172.21.15.1:0/379227491 wait complete. 2024-05-09T06:31:14.730 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-09T06:31:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.854+0000 7f99540ba640 1 Processor -- start 2024-05-09T06:31:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.854+0000 7f99540ba640 1 -- start start 2024-05-09T06:31:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f99540ba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f994c06ffb0 0x7f994c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f99540ba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c06c420 con 0x7f994c0ae9c0 2024-05-09T06:31:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f99540ba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c06c700 con 0x7f994c15e300 2024-05-09T06:31:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f99540ba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f994c06f3f0 con 0x7f994c06ffb0 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f9952657640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f994c06ffb0 0x7f994c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f9951655640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f994c0ae9c0 0x7f994c06f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39072/0 (socket says 172.21.15.1:39072) 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.855+0000 7f9951655640 1 -- 172.21.15.1:0/3373275724 learned_addr learned my addr 172.21.15.1:0/3373275724 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.856+0000 7f9952657640 1 -- 172.21.15.1:0/3373275724 >> v1:172.21.15.6:6789/0 conn(0x7f994c15e300 legacy=0x7f994c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.856+0000 7f9952657640 1 -- 172.21.15.1:0/3373275724 >> v1:172.21.15.1:6789/0 conn(0x7f994c0ae9c0 legacy=0x7f994c06f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.856+0000 7f9952657640 1 -- 172.21.15.1:0/3373275724 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c06c9e0 con 0x7f994c06ffb0 2024-05-09T06:31:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.856+0000 7f9950e54640 1 -- 172.21.15.1:0/3373275724 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861433706 0 0) 0x7f994c06c420 con 0x7f994c0ae9c0 2024-05-09T06:31:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.856+0000 7f9952657640 1 --2- 172.21.15.1:0/3373275724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c06ffb0 0x7f994c06d4a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272306ef3cacf987 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f9950e54640 1 -- 172.21.15.1:0/3373275724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944031600 con 0x7f994c06ffb0 2024-05-09T06:31:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f9950e54640 1 -- 172.21.15.1:0/3373275724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99440317a0 con 0x7f994c06ffb0 2024-05-09T06:31:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f9950e54640 1 -- 172.21.15.1:0/3373275724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944031b70 con 0x7f994c06ffb0 2024-05-09T06:31:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 -- 172.21.15.1:0/3373275724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c06ffb0 msgr2=0x7f994c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 --2- 172.21.15.1:0/3373275724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c06ffb0 0x7f994c06d4a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 -- 172.21.15.1:0/3373275724 shutdown_connections 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 --2- 172.21.15.1:0/3373275724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c06ffb0 0x7f994c06d4a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 -- 172.21.15.1:0/3373275724 >> 172.21.15.1:0/3373275724 conn(0x7f994c0d1c50 msgr2=0x7f994c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 -- 172.21.15.1:0/3373275724 shutdown_connections 2024-05-09T06:31:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.857+0000 7f99540ba640 1 -- 172.21.15.1:0/3373275724 wait complete. 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.858+0000 7f99540ba640 1 Processor -- start 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.858+0000 7f99540ba640 1 -- start start 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f994c06ffb0 0x7f994c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994c0ae9c0 0x7f994c09d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0d1040 con 0x7f994c15e300 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f994c06cfa0 con 0x7f994c0ae9c0 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f99540ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0d11c0 con 0x7f994c06ffb0 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45252/0 (socket says 172.21.15.1:45252) 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.859+0000 7f9952657640 1 -- 172.21.15.1:0/1665683573 learned_addr learned my addr 172.21.15.1:0/1665683573 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9951655640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994c0ae9c0 0x7f994c09d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9951e56640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f994c06ffb0 0x7f994c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 -- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f994c06ffb0 msgr2=0x7f994c159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f994c06ffb0 0x7f994c159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 -- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994c0ae9c0 msgr2=0x7f994c09d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994c0ae9c0 0x7f994c09d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 -- 172.21.15.1:0/1665683573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c09eed0 con 0x7f994c15e300 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f9952657640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f9944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f99427fc640 1 -- 172.21.15.1:0/1665683573 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994403f070 con 0x7f994c15e300 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f99427fc640 1 -- 172.21.15.1:0/1665683573 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9944030b60 con 0x7f994c15e300 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c09de30 con 0x7f994c15e300 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f99427fc640 1 -- 172.21.15.1:0/1665683573 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944043410 con 0x7f994c15e300 2024-05-09T06:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.860+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c09e3f0 con 0x7f994c15e300 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.862+0000 7f99427fc640 1 -- 172.21.15.1:0/1665683573 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f994403b410 con 0x7f994c15e300 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.862+0000 7f99427fc640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 0x7f993006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.862+0000 7f99427fc640 1 -- 172.21.15.1:0/1665683573 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f99440ac3f0 con 0x7f994c15e300 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.863+0000 7f9951e56640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 0x7f993006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.863+0000 7f9951e56640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 0x7f993006e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f60 tx=0x7f9948009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.863+0000 7f9956dca740 1 Processor -- start 2024-05-09T06:31:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.863+0000 7f9956dca740 1 -- start start 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea87410620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea87410b60 0x55ea873e5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ea87349a30 con 0x55ea8742e180 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ea872e7590 con 0x55ea87410b60 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ea873459f0 con 0x55ea874112a0 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9951e56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9951e56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45264/0 (socket says 172.21.15.1:45264) 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9951e56640 1 -- 172.21.15.1:0/3408356011 learned_addr learned my addr 172.21.15.1:0/3408356011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.864+0000 7f9951e56640 1 -- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 msgr2=0x55ea87410620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9951e56640 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea87410620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9951e56640 1 -- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea87410b60 msgr2=0x55ea873e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9951e56640 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea87410b60 0x55ea873e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9951e56640 1 -- 172.21.15.1:0/3408356011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea871297c0 con 0x55ea8742e180 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9951e56640 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9948001ea0 tx=0x7f9948011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5ac57339499bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f992f7fe640 1 -- 172.21.15.1:0/3408356011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948003800 con 0x55ea8742e180 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f992f7fe640 1 -- 172.21.15.1:0/3408356011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480039a0 con 0x55ea8742e180 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f992f7fe640 1 -- 172.21.15.1:0/3408356011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948003d70 con 0x55ea8742e180 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9956dca740 1 -- 172.21.15.1:0/3408356011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 msgr2=0x55ea873e0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.865+0000 7f9956dca740 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9948001ea0 tx=0x7f9948011000 comp rx=0 tx=0).stop 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 -- 172.21.15.1:0/3408356011 shutdown_connections 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea87410b60 0x55ea873e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea87410620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 --2- 172.21.15.1:0/3408356011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea873e0b10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 -- 172.21.15.1:0/3408356011 >> 172.21.15.1:0/3408356011 conn(0x55ea873e2d80 msgr2=0x55ea8742b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 -- 172.21.15.1:0/3408356011 shutdown_connections 2024-05-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 -- 172.21.15.1:0/3408356011 wait complete. 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 -- start start 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.866+0000 7f9956dca740 1 Processor -- start 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea8740c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea8742e180 0x55ea87422cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ea873e4f30 con 0x55ea87411790 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ea872e5060 con 0x55ea874112a0 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ea873e4320 con 0x55ea8742e180 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9956dca740 10 client.0 _ll_register_callbacks cb 0x55ea87309dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45276/0 (socket says 172.21.15.1:45276) 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9952657640 1 -- 172.21.15.1:0/1246007066 learned_addr learned my addr 172.21.15.1:0/1246007066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9951e56640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea8740c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.867+0000 7f9951655640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea8742e180 0x55ea87422cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 -- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea8742e180 msgr2=0x55ea87422cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea8742e180 0x55ea87422cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 -- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 msgr2=0x55ea8740c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea8740c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 -- 172.21.15.1:0/1246007066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea87189f50 con 0x55ea87411790 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9952657640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f994407a3a0 tx=0x7f994407a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235a221cd783194c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99440831d0 con 0x55ea87411790 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f99440ac0c0 con 0x55ea87411790 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99480039a0 con 0x55ea87411790 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9944099050 con 0x55ea87411790 2024-05-09T06:31:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 10 client.6828 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55ea8701d0b0 con 0x55ea87411790 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480039a0 con 0x55ea87411790 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f994407a900 con 0x55ea87411790 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 10 client.6828 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 10 client.6828 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 20 client.6828 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8743f5b0 0x55ea8743f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 4 client.6828 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.868+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea87357160 con 0x55ea8743f5b0 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.869+0000 7f9951e56640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8743f5b0 0x55ea8743f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.869+0000 7f992cff9640 10 client.6828 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.869+0000 7f9951e56640 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8743f5b0 0x55ea8743f990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f4727c0340f341 server_cookie=5b50b4ca31d9464d in_seq=0 out_seq=0 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f992cff9640 1 -- 172.21.15.1:0/1246007066 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea87357160 con 0x55ea8743f5b0 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f992cff9640 10 client.6828 handle_command_reply: tid=0 2024-05-09T06:31:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 client.6828 shutdown 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 msgr2=0x55ea87412f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f994407a3a0 tx=0x7f994407a6a0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 shutdown_connections 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea87412f90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8743f5b0 0x55ea8743f990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea8742e180 0x55ea87422cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 --2- 172.21.15.1:0/1246007066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea8740c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f992cff9640 0 client.6828 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 >> 172.21.15.1:0/1246007066 conn(0x55ea87410b60 msgr2=0x55ea8742b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 shutdown_connections 2024-05-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 1 -- 172.21.15.1:0/1246007066 wait complete. 2024-05-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.870+0000 7f9956dca740 20 client.6828 trim_cache size 0 max 16384 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 Processor -- start 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 -- start start 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea873e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea87411790 0x55ea87413ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1f5960 con 0x55ea8742e180 2024-05-09T06:31:14.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f994c09c5e0 con 0x55ea874112a0 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.884+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f994c1f5310 con 0x55ea87411790 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45280/0 (socket says 172.21.15.1:45280) 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 -- 172.21.15.1:0/489923055 learned_addr learned my addr 172.21.15.1:0/489923055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9951655640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea87411790 0x55ea87413ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9951e56640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea873e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 -- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea87411790 msgr2=0x55ea87413ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea87411790 0x55ea87413ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 -- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 msgr2=0x55ea873e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea873e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 -- 172.21.15.1:0/489923055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea873e7050 con 0x55ea8742e180 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9951e56640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea873e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:14.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f9952657640 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f994402abc0 tx=0x7f9944044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de66225a8d34850 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f992effd640 1 -- 172.21.15.1:0/489923055 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994409d020 con 0x55ea8742e180 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f992effd640 1 -- 172.21.15.1:0/489923055 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994407a900 con 0x55ea8742e180 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.885+0000 7f992effd640 1 -- 172.21.15.1:0/489923055 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99440831d0 con 0x55ea8742e180 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- 172.21.15.1:0/489923055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 msgr2=0x55ea87414010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f994402abc0 tx=0x7f9944044000 comp rx=0 tx=0).stop 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- 172.21.15.1:0/489923055 shutdown_connections 2024-05-09T06:31:14.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea8742e180 0x55ea87414010 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea87411790 0x55ea87413ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 --2- 172.21.15.1:0/489923055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea874112a0 0x55ea873e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- 172.21.15.1:0/489923055 >> 172.21.15.1:0/489923055 conn(0x55ea87410b60 msgr2=0x55ea87422ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- 172.21.15.1:0/489923055 shutdown_connections 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- 172.21.15.1:0/489923055 wait complete. 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 -- start start 2024-05-09T06:31:14.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.886+0000 7f9956dca740 1 Processor -- start 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea874a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea8742e180 0x55ea8743d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ea87411cb0 con 0x55ea87411790 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ea87413950 con 0x55ea8742e180 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ea873e69b0 con 0x55ea874112a0 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9956dca740 10 client.0 _ll_register_callbacks cb 0x7f9944009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9951655640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9951655640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45292/0 (socket says 172.21.15.1:45292) 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9951655640 1 -- 172.21.15.1:0/2246318104 learned_addr learned my addr 172.21.15.1:0/2246318104 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9952657640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea8742e180 0x55ea8743d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.887+0000 7f9951e56640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea874a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 -- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 msgr2=0x55ea874a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea874a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 -- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea8742e180 msgr2=0x55ea8743d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea8742e180 0x55ea8743d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 -- 172.21.15.1:0/2246318104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea8740dd10 con 0x55ea87411790 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9951655640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f994c09f300 tx=0x7f993c005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43451cfb4fd01d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c01a020 con 0x55ea87411790 2024-05-09T06:31:14.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f993c012b20 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994407a900 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f993c024050 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9956dca740 10 client.6834 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55ea873e57c0 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994407a900 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f993c027020 con 0x55ea87411790 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9956dca740 10 client.6834 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.888+0000 7f9956dca740 10 client.6834 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 20 client.6834 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8742a040 0x55ea8742ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 4 client.6834 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ea873514b0 con 0x55ea8742a040 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9951e56640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8742a040 0x55ea8742ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f992cff9640 10 client.6834 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9951e56640 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8742a040 0x55ea8742ea40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bca2d611db3aca server_cookie=6e04d7324786838c in_seq=0 out_seq=0 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f992cff9640 1 -- 172.21.15.1:0/2246318104 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x55ea873514b0 con 0x55ea8742a040 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f992cff9640 10 client.6834 handle_command_reply: tid=0 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 client.6834 shutdown 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 msgr2=0x55ea874a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f994c09f300 tx=0x7f993c005f20 comp rx=0 tx=0).stop 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 shutdown_connections 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea8742e180 0x55ea8743d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea87411790 0x55ea874a8a70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55ea8742a040 0x55ea8742ea40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 --2- 172.21.15.1:0/2246318104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ea874112a0 0x55ea874a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f992cff9640 0 client.6834 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.889+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 >> 172.21.15.1:0/2246318104 conn(0x55ea87410b60 msgr2=0x55ea8742bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 shutdown_connections 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f9956dca740 1 -- 172.21.15.1:0/2246318104 wait complete. 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f9956dca740 20 client.6834 trim_cache size 0 max 16384 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "backtrace", 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 2539680266, 2024-05-09T06:31:14.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:31:14.882 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir_test/file" 2024-05-09T06:31:14.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:31:14.882 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 msgr2=0x7f993006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 0x7f993006e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9948000f60 tx=0x7f9948009040 comp rx=0 tx=0).stop 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 msgr2=0x7f994c0b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f9944004a40 comp rx=0 tx=0).stop 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 shutdown_connections 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f993006be30 0x7f993006e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f994c15e300 0x7f994c0b6c70 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f994c0ae9c0 0x7f994c09d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 --2- 172.21.15.1:0/1665683573 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f994c06ffb0 0x7f994c159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 >> 172.21.15.1:0/1665683573 conn(0x7f994c0d1c50 msgr2=0x7f994c15bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 shutdown_connections 2024-05-09T06:31:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:14.890+0000 7f99540ba640 1 -- 172.21.15.1:0/1665683573 wait complete. 2024-05-09T06:31:14.884 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:31:14.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.005+0000 7f45d6d41640 1 Processor -- start 2024-05-09T06:31:14.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.005+0000 7f45d6d41640 1 -- start start 2024-05-09T06:31:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d6d41640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f45d00d4940 0x7f45d00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d6d41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d6d41640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d00b33c0 con 0x7f45d0161950 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d6d41640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f45d00d4dd0 con 0x7f45d00d4940 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d6d41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00b3660 con 0x7f45d00d2b10 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d4add640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f45d0161950 0x7f45d00d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39088/0 (socket says 172.21.15.1:39088) 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.006+0000 7f45d4add640 1 -- 172.21.15.1:0/3945295209 learned_addr learned my addr 172.21.15.1:0/3945295209 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45cffff640 1 --2- 172.21.15.1:0/3945295209 >> v2:172.21.15.6:3300/0 conn(0x7f45d00d4940 0x7f45d00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45d52de640 1 --2- 172.21.15.1:0/3945295209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45cf7fe640 1 -- 172.21.15.1:0/3945295209 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179066248 0 0) 0x7f45d00b33c0 con 0x7f45d0161950 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45cf7fe640 1 -- 172.21.15.1:0/3945295209 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45bc003520 con 0x7f45d0161950 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45d52de640 1 -- 172.21.15.1:0/3945295209 >> v2:172.21.15.6:3300/0 conn(0x7f45d00d4940 msgr2=0x7f45d00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45d52de640 1 --2- 172.21.15.1:0/3945295209 >> v2:172.21.15.6:3300/0 conn(0x7f45d00d4940 0x7f45d00b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45d52de640 1 -- 172.21.15.1:0/3945295209 >> v1:172.21.15.1:6789/0 conn(0x7f45d0161950 legacy=0x7f45d00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.007+0000 7f45d52de640 1 -- 172.21.15.1:0/3945295209 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00b3940 con 0x7f45d00d2b10 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d52de640 1 --2- 172.21.15.1:0/3945295209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d00b4680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097a0 tx=0x7f45c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a99f457188adbbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45cf7fe640 1 -- 172.21.15.1:0/3945295209 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8037070 con 0x7f45d00d2b10 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45cf7fe640 1 -- 172.21.15.1:0/3945295209 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8032440 con 0x7f45d00d2b10 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45cf7fe640 1 -- 172.21.15.1:0/3945295209 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8032810 con 0x7f45d00d2b10 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 -- 172.21.15.1:0/3945295209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 msgr2=0x7f45d00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 --2- 172.21.15.1:0/3945295209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d00b4680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097a0 tx=0x7f45c802a180 comp rx=0 tx=0).stop 2024-05-09T06:31:14.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 -- 172.21.15.1:0/3945295209 shutdown_connections 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 --2- 172.21.15.1:0/3945295209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d00b4680 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 --2- 172.21.15.1:0/3945295209 >> v2:172.21.15.6:3300/0 conn(0x7f45d00d4940 0x7f45d00b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 -- 172.21.15.1:0/3945295209 >> 172.21.15.1:0/3945295209 conn(0x7f45d00d3e20 msgr2=0x7f45d00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 -- 172.21.15.1:0/3945295209 shutdown_connections 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.008+0000 7f45d6d41640 1 -- 172.21.15.1:0/3945295209 wait complete. 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.010+0000 7f45d6d41640 1 Processor -- start 2024-05-09T06:31:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.010+0000 7f45d6d41640 1 -- start start 2024-05-09T06:31:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d00cc950 0x7f45d015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00d1100 con 0x7f45d0161950 2024-05-09T06:31:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00d0f80 con 0x7f45d00cc950 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00d1c00 con 0x7f45d00d2b10 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45308/0 (socket says 172.21.15.1:45308) 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 -- 172.21.15.1:0/1451903112 learned_addr learned my addr 172.21.15.1:0/1451903112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d4add640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d52de640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d00cc950 0x7f45d015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 -- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 msgr2=0x7f45d015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 -- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d00cc950 msgr2=0x7f45d015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d00cc950 0x7f45d015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 -- 172.21.15.1:0/1451903112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d015c750 con 0x7f45d0161950 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cffff640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9e0 tx=0x7f45b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8017070 con 0x7f45d0161950 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b800c9d0 con 0x7f45d0161950 2024-05-09T06:31:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d00b6b40 con 0x7f45d0161950 2024-05-09T06:31:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.011+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800cdc0 con 0x7f45d0161950 2024-05-09T06:31:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.012+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d00b7100 con 0x7f45d0161950 2024-05-09T06:31:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.013+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d00752b0 con 0x7f45d0161950 2024-05-09T06:31:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.013+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b801a070 con 0x7f45d0161950 2024-05-09T06:31:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.014+0000 7f45cd7fa640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 0x7f45b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.014+0000 7f45d4add640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 0x7f45b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.014+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f45b808af90 con 0x7f45d0161950 2024-05-09T06:31:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.014+0000 7f45d4add640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 0x7f45b006e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45c0001580 tx=0x7f45c0000f40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.018+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45b80590f0 con 0x7f45d0161950 2024-05-09T06:31:15.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.246+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f45d00ca6b0 con 0x7f45d0161950 2024-05-09T06:31:15.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.246+0000 7f45cd7fa640 1 -- 172.21.15.1:0/1451903112 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f45b805d3f0 con 0x7f45d0161950 2024-05-09T06:31:15.228 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:15.228 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.248+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 msgr2=0x7f45b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.248+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 0x7f45b006e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45c0001580 tx=0x7f45c0000f40 comp rx=0 tx=0).stop 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.248+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 msgr2=0x7f45d015c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.248+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9e0 tx=0x7f45b800eea0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 shutdown_connections 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f45b006be60 0x7f45b006e320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45d00cc950 0x7f45d015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45d0161950 0x7f45d015c210 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 --2- 172.21.15.1:0/1451903112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45d00d2b10 0x7f45d015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 >> 172.21.15.1:0/1451903112 conn(0x7f45d00d3e20 msgr2=0x7f45d00b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 shutdown_connections 2024-05-09T06:31:15.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.249+0000 7f45d6d41640 1 -- 172.21.15.1:0/1451903112 wait complete. 2024-05-09T06:31:15.242 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:31:15.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:15.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-09T06:31:15.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.369+0000 7f6720a15640 1 Processor -- start 2024-05-09T06:31:15.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.369+0000 7f6720a15640 1 -- start start 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6720a15640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f671c145880 0x7f671c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6720a15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6720a15640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c15d470 con 0x7f671c0ced00 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6720a15640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f671c0cf160 con 0x7f671c145880 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6720a15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f671c0b3cc0 con 0x7f671c0b3820 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f671c145880 0x7f671c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f671c145880 0x7f671c15cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45318/0 (socket says 172.21.15.1:45318) 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 -- 172.21.15.1:0/1576788981 learned_addr learned my addr 172.21.15.1:0/1576788981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6719d9b640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 -- 172.21.15.1:0/1576788981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 msgr2=0x7f671c0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c0ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 -- 172.21.15.1:0/1576788981 >> v1:172.21.15.1:6789/0 conn(0x7f671c0ced00 legacy=0x7f671c15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671a59c640 1 -- 172.21.15.1:0/1576788981 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c0b3fa0 con 0x7f671c145880 2024-05-09T06:31:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f671959a640 1 -- 172.21.15.1:0/1576788981 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145212674 0 0) 0x7f671c15d470 con 0x7f671c0ced00 2024-05-09T06:31:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.370+0000 7f6719d9b640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f671a59c640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c15cf30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f670c009870 tx=0x7f670c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ae70f7683e67a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f671959a640 1 -- 172.21.15.1:0/1576788981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037070 con 0x7f671c145880 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f671959a640 1 -- 172.21.15.1:0/1576788981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c032440 con 0x7f671c145880 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f671959a640 1 -- 172.21.15.1:0/1576788981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c032810 con 0x7f671c145880 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f6720a15640 1 -- 172.21.15.1:0/1576788981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 msgr2=0x7f671c15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.371+0000 7f6720a15640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c15cf30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f670c009870 tx=0x7f670c02a270 comp rx=0 tx=0).stop 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 -- 172.21.15.1:0/1576788981 shutdown_connections 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 --2- 172.21.15.1:0/1576788981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c15cf30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 -- 172.21.15.1:0/1576788981 >> 172.21.15.1:0/1576788981 conn(0x7f671c06b3c0 msgr2=0x7f671c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 -- 172.21.15.1:0/1576788981 shutdown_connections 2024-05-09T06:31:15.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.372+0000 7f6720a15640 1 -- 172.21.15.1:0/1576788981 wait complete. 2024-05-09T06:31:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.373+0000 7f6720a15640 1 Processor -- start 2024-05-09T06:31:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.373+0000 7f6720a15640 1 -- start start 2024-05-09T06:31:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f671c0ced00 0x7f671c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0b4840 con 0x7f671c145880 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0b4560 con 0x7f671c0ced00 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f6720a15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f671c0cfee0 con 0x7f671c0b3820 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45328/0 (socket says 172.21.15.1:45328) 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 -- 172.21.15.1:0/1638766488 learned_addr learned my addr 172.21.15.1:0/1638766488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 -- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 msgr2=0x7f671c139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 -- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f671c0ced00 msgr2=0x7f671c15b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f671c0ced00 0x7f671c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 -- 172.21.15.1:0/1638766488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c06cf10 con 0x7f671c145880 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.374+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f671400c760 tx=0x7f671400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.375+0000 7f670affd640 1 -- 172.21.15.1:0/1638766488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714003da0 con 0x7f671c145880 2024-05-09T06:31:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.375+0000 7f670affd640 1 -- 172.21.15.1:0/1638766488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6714005070 con 0x7f671c145880 2024-05-09T06:31:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.375+0000 7f670affd640 1 -- 172.21.15.1:0/1638766488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714005460 con 0x7f671c145880 2024-05-09T06:31:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.375+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c137120 con 0x7f671c145880 2024-05-09T06:31:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.375+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c06d490 con 0x7f671c145880 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.376+0000 7f670affd640 1 -- 172.21.15.1:0/1638766488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6714005600 con 0x7f671c145880 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.377+0000 7f670affd640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 0x7f66fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.377+0000 7f670affd640 1 -- 172.21.15.1:0/1638766488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f671408afa0 con 0x7f671c145880 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.377+0000 7f671a59c640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 0x7f66fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.377+0000 7f671a59c640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 0x7f66fc06e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f670c034000 tx=0x7f670c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.377+0000 7f67237ff740 1 Processor -- start 2024-05-09T06:31:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 -- start start 2024-05-09T06:31:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a25ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28ae70 0x55605a25fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55605a1c2960 con 0x55605a28b5b0 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55605a16a5f0 con 0x55605a28ae70 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55605a1be920 con 0x55605a2a8490 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45330/0 (socket says 172.21.15.1:45330) 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.378+0000 7f6719d9b640 1 -- 172.21.15.1:0/3024443524 learned_addr learned my addr 172.21.15.1:0/3024443524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f671a59c640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a25ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f671ad9d640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28ae70 0x55605a25fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 -- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 msgr2=0x55605a25ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a25ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 -- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28ae70 msgr2=0x55605a25fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28ae70 0x55605a25fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 -- 172.21.15.1:0/3024443524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556059fa2940 con 0x55605a28b5b0 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f671a59c640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a25ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f6719d9b640 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f671000a9f0 tx=0x7f671000aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ac150b5c84b9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f66fbfff640 1 -- 172.21.15.1:0/3024443524 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710015070 con 0x55605a28b5b0 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f66fbfff640 1 -- 172.21.15.1:0/3024443524 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671000cc60 con 0x55605a28b5b0 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.379+0000 7f66fbfff640 1 -- 172.21.15.1:0/3024443524 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100113f0 con 0x55605a28b5b0 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- 172.21.15.1:0/3024443524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 msgr2=0x55605a28a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f671000a9f0 tx=0x7f671000aeb0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- 172.21.15.1:0/3024443524 shutdown_connections 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28ae70 0x55605a25fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a28b5b0 0x55605a28a930 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 --2- 172.21.15.1:0/3024443524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a25ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- 172.21.15.1:0/3024443524 >> 172.21.15.1:0/3024443524 conn(0x55605a25d090 msgr2=0x55605a2a5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- 172.21.15.1:0/3024443524 shutdown_connections 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- 172.21.15.1:0/3024443524 wait complete. 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 -- start start 2024-05-09T06:31:15.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.380+0000 7f67237ff740 1 Processor -- start 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55605a25f240 con 0x55605a29c740 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55605a17db80 con 0x55605a28b5b0 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55605a25e630 con 0x55605a2a8490 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f67237ff740 10 client.0 _ll_register_callbacks cb 0x55605a160b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f671ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f671ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45338/0 (socket says 172.21.15.1:45338) 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.381+0000 7f671ad9d640 1 -- 172.21.15.1:0/1935937326 learned_addr learned my addr 172.21.15.1:0/1935937326 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f6719d9b640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671a59c640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 -- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 msgr2=0x55605a28e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 -- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 msgr2=0x55605a2783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 -- 172.21.15.1:0/1935937326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605a0030d0 con 0x55605a29c740 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671a59c640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f671ad9d640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6714061700 tx=0x7f6714059810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95896cc48c9bc75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714071030 con 0x55605a29c740 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f67140592b0 con 0x55605a29c740 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671000cc60 con 0x55605a29c740 2024-05-09T06:31:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6714061c30 con 0x55605a29c740 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 10 client.6855 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x556059e96230 con 0x55605a29c740 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671000cc60 con 0x55605a29c740 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f67140617a0 con 0x55605a29c740 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 10 client.6855 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 10 client.6855 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 20 client.6855 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a3580 0x55605a2a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 4 client.6855 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.382+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55605a1d0090 con 0x55605a2a3580 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.383+0000 7f671a59c640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a3580 0x55605a2a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.383+0000 7f66f97fa640 10 client.6855 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.383+0000 7f671a59c640 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a3580 0x55605a2a3990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4994aa685bad49 server_cookie=dc5ab25e1b815d56 in_seq=0 out_seq=0 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f66f97fa640 1 -- 172.21.15.1:0/1935937326 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55605a1d0090 con 0x55605a2a3580 2024-05-09T06:31:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f66f97fa640 10 client.6855 handle_command_reply: tid=0 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 client.6855 shutdown 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 msgr2=0x55605a2a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6714061700 tx=0x7f6714059810 comp rx=0 tx=0).stop 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 shutdown_connections 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a9130 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a3580 0x55605a2a3990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 --2- 172.21.15.1:0/1935937326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f66f97fa640 0 client.6855 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 >> 172.21.15.1:0/1935937326 conn(0x55605a28ae70 msgr2=0x55605a2a5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 shutdown_connections 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 1 -- 172.21.15.1:0/1935937326 wait complete. 2024-05-09T06:31:15.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.384+0000 7f67237ff740 20 client.6855 trim_cache size 0 max 16384 2024-05-09T06:31:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.402+0000 7f67237ff740 1 Processor -- start 2024-05-09T06:31:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.402+0000 7f67237ff740 1 -- start start 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f671c200c40 con 0x55605a29c740 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1ffc60 con 0x55605a28b5b0 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f671c06ebf0 con 0x55605a2a8490 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45348/0 (socket says 172.21.15.1:45348) 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 -- 172.21.15.1:0/2677378212 learned_addr learned my addr 172.21.15.1:0/2677378212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 -- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 msgr2=0x55605a28e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 -- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 msgr2=0x55605a2c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 -- 172.21.15.1:0/2677378212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605a29cbc0 con 0x55605a29c740 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f671000a990 tx=0x7f6710011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65905928395aedfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f66fb7fe640 1 -- 172.21.15.1:0/2677378212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710015040 con 0x55605a29c740 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f66fb7fe640 1 -- 172.21.15.1:0/2677378212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671000cbb0 con 0x55605a29c740 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.403+0000 7f66fb7fe640 1 -- 172.21.15.1:0/2677378212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710025460 con 0x55605a29c740 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- 172.21.15.1:0/2677378212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 msgr2=0x55605a2a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f671000a990 tx=0x7f6710011000 comp rx=0 tx=0).stop 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- 172.21.15.1:0/2677378212 shutdown_connections 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a2a8490 0x55605a28e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a2a3960 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 --2- 172.21.15.1:0/2677378212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a28b5b0 0x55605a2c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- 172.21.15.1:0/2677378212 >> 172.21.15.1:0/2677378212 conn(0x55605a28ae70 msgr2=0x55605a2a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- 172.21.15.1:0/2677378212 shutdown_connections 2024-05-09T06:31:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- 172.21.15.1:0/2677378212 wait complete. 2024-05-09T06:31:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.404+0000 7f67237ff740 1 -- start start 2024-05-09T06:31:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 Processor -- start 2024-05-09T06:31:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a28b5b0 0x55605a2b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a2a8490 0x55605a323ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55605a2b8790 con 0x55605a29c740 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1ff5f0 con 0x55605a2a8490 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55605a1cdf70 con 0x55605a28b5b0 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.405+0000 7f67237ff740 10 client.0 _ll_register_callbacks cb 0x7f6714058c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45360/0 (socket says 172.21.15.1:45360) 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 -- 172.21.15.1:0/2013797364 learned_addr learned my addr 172.21.15.1:0/2013797364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f671a59c640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a28b5b0 0x55605a2b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f671ad9d640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a2a8490 0x55605a323ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 -- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a28b5b0 msgr2=0x55605a2b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a28b5b0 0x55605a2b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 -- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a2a8490 msgr2=0x55605a323ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a2a8490 0x55605a323ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 -- 172.21.15.1:0/2013797364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605a25ad90 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f6719d9b640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f671001d040 tx=0x7f6710006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57efde0f29f97fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f66f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710015050 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6710025cc0 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671000cbb0 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6710039050 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f67237ff740 10 client.6861 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.406+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55605a2b4660 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671000cbb0 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f671003c020 con 0x55605a29c740 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 10 client.6861 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 10 client.6861 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 20 client.6861 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a77c0 0x55605a2a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 4 client.6861 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-09T06:31:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55605a1ca3e0 con 0x55605a2a77c0 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f671a59c640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a77c0 0x55605a2a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f66f97fa640 10 client.6861 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.407+0000 7f671a59c640 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a77c0 0x55605a2a6c20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daf9ba3f169501a server_cookie=f45cc9569ddd434b in_seq=0 out_seq=0 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f66f97fa640 1 -- 172.21.15.1:0/2013797364 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55605a1ca3e0 con 0x55605a2a77c0 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f66f97fa640 10 client.6861 handle_command_reply: tid=0 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 client.6861 shutdown 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 msgr2=0x55605a323590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f671001d040 tx=0x7f6710006c90 comp rx=0 tx=0).stop 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 shutdown_connections 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55605a2a8490 0x55605a323ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55605a29c740 0x55605a323590 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55605a2a77c0 0x55605a2a6c20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 --2- 172.21.15.1:0/2013797364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55605a28b5b0 0x55605a2b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f66f97fa640 0 client.6861 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 >> 172.21.15.1:0/2013797364 conn(0x55605a28ae70 msgr2=0x55605a259ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 shutdown_connections 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 1 -- 172.21.15.1:0/2013797364 wait complete. 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.408+0000 7f67237ff740 20 client.6861 trim_cache size 0 max 16384 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "43ed2925-f20c-4195-97f4-d17a9ec995ac", 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:31:15.395 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:31:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 msgr2=0x7f66fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 0x7f66fc06e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f670c034000 tx=0x7f670c02c000 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 msgr2=0x7f671c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f671400c760 tx=0x7f671400cc20 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 shutdown_connections 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f66fc06be30 0x7f66fc06e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f671c145880 0x7f671c0b1740 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f671c0ced00 0x7f671c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 --2- 172.21.15.1:0/1638766488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f671c0b3820 0x7f671c139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 >> 172.21.15.1:0/1638766488 conn(0x7f671c06b3c0 msgr2=0x7f671c15be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 shutdown_connections 2024-05-09T06:31:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.409+0000 7f6720a15640 1 -- 172.21.15.1:0/1638766488 wait complete. 2024-05-09T06:31:15.397 INFO:tasks.ceph.mds.b.smithi006.stderr:2024-05-09T06:31:15.409+0000 7ff0cd720640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-09T06:30:42.935028+0000 1=1+0) caps={6732=pAsLsXsFs/-@2},l=6732(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x562048534c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:31:15.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:15.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-09T06:31:15.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.526+0000 7f2217e38640 1 Processor -- start 2024-05-09T06:31:15.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.526+0000 7f2217e38640 1 -- start start 2024-05-09T06:31:15.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2217e38640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f221015c4c0 0x7f221006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2217e38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221006cf40 con 0x7f22100b4de0 2024-05-09T06:31:15.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2217e38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221006d220 con 0x7f221015cfb0 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2217e38640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2210156980 con 0x7f221015c4c0 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2215bd4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f221015c4c0 0x7f221006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2215bd4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f221015c4c0 0x7f221006dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45376/0 (socket says 172.21.15.1:45376) 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.527+0000 7f2215bd4640 1 -- 172.21.15.1:0/512198285 learned_addr learned my addr 172.21.15.1:0/512198285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.528+0000 7f2215bd4640 1 -- 172.21.15.1:0/512198285 >> v1:172.21.15.6:6789/0 conn(0x7f221015cfb0 legacy=0x7f2210070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.528+0000 7f2215bd4640 1 -- 172.21.15.1:0/512198285 >> v1:172.21.15.1:6789/0 conn(0x7f22100b4de0 legacy=0x7f221015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.528+0000 7f2215bd4640 1 -- 172.21.15.1:0/512198285 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221006d500 con 0x7f221015c4c0 2024-05-09T06:31:15.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.528+0000 7f2214bd2640 1 -- 172.21.15.1:0/512198285 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780597422 0 0) 0x7f221006cf40 con 0x7f22100b4de0 2024-05-09T06:31:15.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.528+0000 7f2215bd4640 1 --2- 172.21.15.1:0/512198285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015c4c0 0x7f221006dd00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2208009e50 tx=0x7f220802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9239990e3dfb0b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2214bd2640 1 -- 172.21.15.1:0/512198285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208036070 con 0x7f221015c4c0 2024-05-09T06:31:15.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2214bd2640 1 -- 172.21.15.1:0/512198285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208031600 con 0x7f221015c4c0 2024-05-09T06:31:15.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2214bd2640 1 -- 172.21.15.1:0/512198285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080319d0 con 0x7f221015c4c0 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 -- 172.21.15.1:0/512198285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015c4c0 msgr2=0x7f221006dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 --2- 172.21.15.1:0/512198285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015c4c0 0x7f221006dd00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2208009e50 tx=0x7f220802a730 comp rx=0 tx=0).stop 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 -- 172.21.15.1:0/512198285 shutdown_connections 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 --2- 172.21.15.1:0/512198285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015c4c0 0x7f221006dd00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 -- 172.21.15.1:0/512198285 >> 172.21.15.1:0/512198285 conn(0x7f22100cc4d0 msgr2=0x7f22100cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 -- 172.21.15.1:0/512198285 shutdown_connections 2024-05-09T06:31:15.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.529+0000 7f2217e38640 1 -- 172.21.15.1:0/512198285 wait complete. 2024-05-09T06:31:15.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.530+0000 7f2217e38640 1 Processor -- start 2024-05-09T06:31:15.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.530+0000 7f2217e38640 1 -- start start 2024-05-09T06:31:15.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22100b4de0 0x7f22101446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 0x7f22100aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22100b5260 con 0x7f221015cfb0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f221006d870 con 0x7f22100b4de0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2217e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f22100b53e0 con 0x7f221015c4c0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f22163d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f22163d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45384/0 (socket says 172.21.15.1:45384) 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f22163d5640 1 -- 172.21.15.1:0/2632927069 learned_addr learned my addr 172.21.15.1:0/2632927069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.531+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22100b4de0 0x7f22101446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22153d3640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 0x7f22100aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 -- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 msgr2=0x7f22100aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 0x7f22100aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 -- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22100b4de0 msgr2=0x7f22101446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22100b4de0 0x7f22101446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 -- 172.21.15.1:0/2632927069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100af440 con 0x7f221015cfb0 2024-05-09T06:31:15.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22153d3640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 0x7f22100aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f22163d5640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f220000c390 tx=0x7f220000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f220e7fc640 1 -- 172.21.15.1:0/2632927069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200017070 con 0x7f221015cfb0 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f220e7fc640 1 -- 172.21.15.1:0/2632927069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000052d0 con 0x7f221015cfb0 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f220e7fc640 1 -- 172.21.15.1:0/2632927069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200003810 con 0x7f221015cfb0 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22100ac030 con 0x7f221015cfb0 2024-05-09T06:31:15.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.532+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22100ae700 con 0x7f221015cfb0 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.534+0000 7f220e7fc640 1 -- 172.21.15.1:0/2632927069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f220001a070 con 0x7f221015cfb0 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.534+0000 7f220e7fc640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 0x7f21fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.534+0000 7f220e7fc640 1 -- 172.21.15.1:0/2632927069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f220008a250 con 0x7f221015cfb0 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.534+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 0x7f21fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.535+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 0x7f21fc06e320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2208033000 tx=0x7f2208009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.535+0000 7f221ac24740 1 Processor -- start 2024-05-09T06:31:15.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.535+0000 7f221ac24740 1 -- start start 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b48357efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af030 0x55b483583e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b4834e8d00 con 0x55b4835af770 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b4834904c0 con 0x55b4835cc650 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b4834e4cc0 con 0x55b4835af030 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45400/0 (socket says 172.21.15.1:45400) 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f22153d3640 1 -- 172.21.15.1:0/1597451566 learned_addr learned my addr 172.21.15.1:0/1597451566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f2215bd4640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b48357efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.536+0000 7f22163d5640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af030 0x55b483583e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 -- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af030 msgr2=0x55b483583e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af030 0x55b483583e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 -- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 msgr2=0x55b48357efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b48357efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 -- 172.21.15.1:0/1597451566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4832c67d0 con 0x55b4835af770 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f2215bd4640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b48357efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f22153d3640 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2204001700 tx=0x7f2204001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b1cef960503e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f21f77fe640 1 -- 172.21.15.1:0/1597451566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204019070 con 0x55b4835af770 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f21f77fe640 1 -- 172.21.15.1:0/1597451566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22040052e0 con 0x55b4835af770 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.537+0000 7f21f77fe640 1 -- 172.21.15.1:0/1597451566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204005490 con 0x55b4835af770 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- 172.21.15.1:0/1597451566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 msgr2=0x55b4835aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2204001700 tx=0x7f2204001bc0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- 172.21.15.1:0/1597451566 shutdown_connections 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af030 0x55b483583e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835af770 0x55b4835aeaf0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 --2- 172.21.15.1:0/1597451566 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b48357efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- 172.21.15.1:0/1597451566 >> 172.21.15.1:0/1597451566 conn(0x55b483581250 msgr2=0x55b4835c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- 172.21.15.1:0/1597451566 shutdown_connections 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- 172.21.15.1:0/1597451566 wait complete. 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 -- start start 2024-05-09T06:31:15.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.538+0000 7f221ac24740 1 Processor -- start 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835aa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835c8c60 0x55b4835b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b483583400 con 0x55b4835cc650 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b4834a3b90 con 0x55b4835c8c60 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b4835827f0 con 0x55b4835af770 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f221ac24740 10 client.0 _ll_register_callbacks cb 0x55b4834dfda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45406/0 (socket says 172.21.15.1:45406) 2024-05-09T06:31:15.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f22153d3640 1 -- 172.21.15.1:0/1899042463 learned_addr learned my addr 172.21.15.1:0/1899042463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f2215bd4640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.539+0000 7f22163d5640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835c8c60 0x55b4835b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 -- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 msgr2=0x55b4835aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 -- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835c8c60 msgr2=0x55b4835b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835c8c60 0x55b4835b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 -- 172.21.15.1:0/1899042463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4831fc3f0 con 0x55b4835cc650 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f22153d3640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2204017040 tx=0x7f2204007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c173f331c0f9f7bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204019070 con 0x55b4835cc650 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f22040035f0 con 0x55b4835cc650 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22040052e0 con 0x55b4835cc650 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f220402b050 con 0x55b4835cc650 2024-05-09T06:31:15.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 10 client.6876 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55b4831bc0f0 con 0x55b4835cc650 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22040052e0 con 0x55b4835cc650 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f220402e020 con 0x55b4835cc650 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 10 client.6876 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 10 client.6876 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.540+0000 7f221ac24740 20 client.6876 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835c7b70 0x55b4835c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f221ac24740 4 client.6876 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4834f6430 con 0x55b4835c7b70 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f2215bd4640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835c7b70 0x55b4835c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f21f4ff9640 10 client.6876 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.541+0000 7f2215bd4640 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835c7b70 0x55b4835c7f50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b56f7ee8819740 server_cookie=2358bca4202f1af6 in_seq=0 out_seq=0 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f21f4ff9640 1 -- 172.21.15.1:0/1899042463 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4834f6430 con 0x55b4835c7b70 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f21f4ff9640 10 client.6876 handle_command_reply: tid=0 2024-05-09T06:31:15.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 client.6876 shutdown 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 msgr2=0x55b4835c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2204017040 tx=0x7f2204007a00 comp rx=0 tx=0).stop 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 shutdown_connections 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835c8c60 0x55b4835b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835cc650 0x55b4835c8720 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835c7b70 0x55b4835c7f50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 --2- 172.21.15.1:0/1899042463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f21f4ff9640 0 client.6876 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 >> 172.21.15.1:0/1899042463 conn(0x55b4835af030 msgr2=0x55b4835c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 shutdown_connections 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 1 -- 172.21.15.1:0/1899042463 wait complete. 2024-05-09T06:31:15.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.542+0000 7f221ac24740 20 client.6876 trim_cache size 0 max 16384 2024-05-09T06:31:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 Processor -- start 2024-05-09T06:31:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 -- start start 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22101feaf0 con 0x55b4835c8c60 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f22101fdad0 con 0x55b4835cc650 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f22101f5b10 con 0x55b4835af770 2024-05-09T06:31:15.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.560+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45412/0 (socket says 172.21.15.1:45412) 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 -- 172.21.15.1:0/3877294454 learned_addr learned my addr 172.21.15.1:0/3877294454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f2215bd4640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22163d5640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 -- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 msgr2=0x55b4835c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 -- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 msgr2=0x55b4835c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 -- 172.21.15.1:0/3877294454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4835dcde0 con 0x55b4835c8c60 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22163d5640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f2215bd4640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f22153d3640 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2204001c00 tx=0x7f220401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6acca8299960961 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f21f6ffd640 1 -- 172.21.15.1:0/3877294454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040052e0 con 0x55b4835c8c60 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f21f6ffd640 1 -- 172.21.15.1:0/3877294454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204023070 con 0x55b4835c8c60 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.561+0000 7f21f6ffd640 1 -- 172.21.15.1:0/3877294454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040039d0 con 0x55b4835c8c60 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- 172.21.15.1:0/3877294454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 msgr2=0x55b4835848f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2204001c00 tx=0x7f220401c000 comp rx=0 tx=0).stop 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- 172.21.15.1:0/3877294454 shutdown_connections 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835848f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 --2- 172.21.15.1:0/3877294454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b4835c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- 172.21.15.1:0/3877294454 >> 172.21.15.1:0/3877294454 conn(0x55b4835af030 msgr2=0x55b48357df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- 172.21.15.1:0/3877294454 shutdown_connections 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- 172.21.15.1:0/3877294454 wait complete. 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 -- start start 2024-05-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.562+0000 7f221ac24740 1 Processor -- start 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b48357f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b483583a80 con 0x55b4835c8c60 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f22101fd440 con 0x55b4835cc650 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b4835dcb80 con 0x55b4835af770 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f221ac24740 10 client.0 _ll_register_callbacks cb 0x7f2204017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.563+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45424/0 (socket says 172.21.15.1:45424) 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 -- 172.21.15.1:0/2345957475 learned_addr learned my addr 172.21.15.1:0/2345957475 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b48357f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22163d5640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 -- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 msgr2=0x55b48357f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b48357f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 -- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 msgr2=0x55b4835c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 -- 172.21.15.1:0/2345957475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4835ec9a0 con 0x55b4835c8c60 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f22153d3640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f220400e530 tx=0x7f2204009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd068225f87de011 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f21f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220403b070 con 0x55b4835c8c60 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2204023070 con 0x55b4835c8c60 2024-05-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f22040035f0 con 0x55b4835c8c60 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2204046050 con 0x55b4835c8c60 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f221ac24740 10 client.6882 fetch_fsmap learned FSMap version 36 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.564+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55b4835c9630 con 0x55b4835c8c60 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204023070 con 0x55b4835c8c60 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f2204049020 con 0x55b4835c8c60 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 10 client.6882 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 10 client.6882 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 20 client.6882 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835ac1a0 0x55b4835ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 4 client.6882 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4834f0780 con 0x55b4835ac1a0 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835ac1a0 0x55b4835ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f21f4ff9640 10 client.6882 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f2215bd4640 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835ac1a0 0x55b4835ab200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab54be15e3ca052 server_cookie=e99f09ea6a0f9f0f in_seq=0 out_seq=0 2024-05-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f21f4ff9640 1 -- 172.21.15.1:0/2345957475 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55b4834f0780 con 0x55b4835ac1a0 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f21f4ff9640 10 client.6882 handle_command_reply: tid=0 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.565+0000 7f221ac24740 1 client.6882 shutdown 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 msgr2=0x55b4835c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f220400e530 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 shutdown_connections 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b4835cc650 0x55b4835c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4835c8c60 0x55b4835c6410 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55b4835ac1a0 0x55b4835ab200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 --2- 172.21.15.1:0/2345957475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b4835af770 0x55b48357f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f21f4ff9640 0 client.6882 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 >> 172.21.15.1:0/2345957475 conn(0x55b4835af030 msgr2=0x55b483584860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 shutdown_connections 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 1 -- 172.21.15.1:0/2345957475 wait complete. 2024-05-09T06:31:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f221ac24740 20 client.6882 trim_cache size 0 max 16384 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "43ed2925-f20c-4195-97f4-d17a9ec995ac": { 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "43ed2925-f20c-4195-97f4-d17a9ec995ac", 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair,force" 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:31:15.555 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:31:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 msgr2=0x7f21fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 0x7f21fc06e320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2208033000 tx=0x7f2208009000 comp rx=0 tx=0).stop 2024-05-09T06:31:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 msgr2=0x7f221015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f220000c390 tx=0x7f220000c850 comp rx=0 tx=0).stop 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 shutdown_connections 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f21fc06be60 0x7f21fc06e320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f221015cfb0 0x7f221015e580 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f221015c4c0 0x7f22100aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 --2- 172.21.15.1:0/2632927069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22100b4de0 0x7f22101446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 >> 172.21.15.1:0/2632927069 conn(0x7f22100cc4d0 msgr2=0x7f221015c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.566+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 shutdown_connections 2024-05-09T06:31:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:15.567+0000 7f2217e38640 1 -- 172.21.15.1:0/2632927069 wait complete. 2024-05-09T06:31:15.559 INFO:tasks.cephfs.filesystem:scrub status for tag:43ed2925-f20c-4195-97f4-d17a9ec995ac - {'path': '/', 'tag': '43ed2925-f20c-4195-97f4-d17a9ec995ac', 'options': 'recursive,repair,force'} 2024-05-09T06:31:18.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:31:18.059 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:31:45.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:45.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-09T06:31:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.698+0000 7fc813ea5640 1 Processor -- start 2024-05-09T06:31:45.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.698+0000 7fc813ea5640 1 -- start start 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc813ea5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc80c15e390 0x7fc80c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc813ea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 0x7fc80c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc813ea5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80c15e900 con 0x7fc80c0b1730 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc813ea5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc80c0b1ba0 con 0x7fc80c15e390 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc813ea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c15ebe0 con 0x7fc80c0cf990 2024-05-09T06:31:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc80c15e390 0x7fc80c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc812442640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc80c0b1730 0x7fc80c072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51950/0 (socket says 172.21.15.1:51950) 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc812442640 1 -- 172.21.15.1:0/517009304 learned_addr learned my addr 172.21.15.1:0/517009304 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc80c15e390 0x7fc80c0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57592/0 (socket says 172.21.15.1:57592) 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811440640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 0x7fc80c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 -- 172.21.15.1:0/517009304 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 msgr2=0x7fc80c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 0x7fc80c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 -- 172.21.15.1:0/517009304 >> v1:172.21.15.1:6789/0 conn(0x7fc80c0b1730 legacy=0x7fc80c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811c41640 1 -- 172.21.15.1:0/517009304 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c15eec0 con 0x7fc80c15e390 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc810c3f640 1 -- 172.21.15.1:0/517009304 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424035500 0 0) 0x7fc80c15e900 con 0x7fc80c0b1730 2024-05-09T06:31:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.699+0000 7fc811440640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 0x7fc80c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.700+0000 7fc811c41640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c15e390 0x7fc80c0cf450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc8080097a0 tx=0x7fc80802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c79208336a815f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.700+0000 7fc810c3f640 1 -- 172.21.15.1:0/517009304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808037070 con 0x7fc80c15e390 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.700+0000 7fc810c3f640 1 -- 172.21.15.1:0/517009304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc808032440 con 0x7fc80c15e390 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.700+0000 7fc810c3f640 1 -- 172.21.15.1:0/517009304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808032810 con 0x7fc80c15e390 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 -- 172.21.15.1:0/517009304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c15e390 msgr2=0x7fc80c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c15e390 0x7fc80c0cf450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc8080097a0 tx=0x7fc80802a180 comp rx=0 tx=0).stop 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 -- 172.21.15.1:0/517009304 shutdown_connections 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c0cf990 0x7fc80c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 --2- 172.21.15.1:0/517009304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c15e390 0x7fc80c0cf450 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 -- 172.21.15.1:0/517009304 >> 172.21.15.1:0/517009304 conn(0x7fc80c0b1da0 msgr2=0x7fc80c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 -- 172.21.15.1:0/517009304 shutdown_connections 2024-05-09T06:31:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.701+0000 7fc813ea5640 1 -- 172.21.15.1:0/517009304 wait complete. 2024-05-09T06:31:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.702+0000 7fc813ea5640 1 Processor -- start 2024-05-09T06:31:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.702+0000 7fc813ea5640 1 -- start start 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc80c0cf990 0x7fc80c1e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c15e390 0x7fc80c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0cd8f0 con 0x7fc80c0b1730 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c15f6c0 con 0x7fc80c0cf990 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc813ea5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c162ba0 con 0x7fc80c15e390 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc811c41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc811c41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57598/0 (socket says 172.21.15.1:57598) 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc811c41640 1 -- 172.21.15.1:0/318418948 learned_addr learned my addr 172.21.15.1:0/318418948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc812442640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c15e390 0x7fc80c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.703+0000 7fc811440640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc80c0cf990 0x7fc80c1e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 -- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c15e390 msgr2=0x7fc80c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c15e390 0x7fc80c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 -- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc80c0cf990 msgr2=0x7fc80c1e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc80c0cf990 0x7fc80c1e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 -- 172.21.15.1:0/318418948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c0b5bf0 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc811c41640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc808035040 tx=0x7fc8080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc8027fc640 1 -- 172.21.15.1:0/318418948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808037040 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc8027fc640 1 -- 172.21.15.1:0/318418948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc808030770 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc8027fc640 1 -- 172.21.15.1:0/318418948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808030b80 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1e7bc0 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.704+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1e8180 con 0x7fc80c0b1730 2024-05-09T06:31:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.706+0000 7fc8027fc640 1 -- 172.21.15.1:0/318418948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc808044020 con 0x7fc80c0b1730 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.706+0000 7fc8027fc640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 0x7fc7f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.706+0000 7fc8027fc640 1 -- 172.21.15.1:0/318418948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7fc8080ab690 con 0x7fc80c0b1730 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.706+0000 7fc811440640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 0x7fc7f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.707+0000 7fc811440640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 0x7fc7f006e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ad5c0 tx=0x7fc7fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.707+0000 7fc816bb5740 1 Processor -- start 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.707+0000 7fc816bb5740 1 -- start start 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177031ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350030 0x561770324e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561770289d00 con 0x561770350770 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5617702314c0 con 0x561770350030 2024-05-09T06:31:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561770285cc0 con 0x56177036d650 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57602/0 (socket says 172.21.15.1:57602) 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 -- 172.21.15.1:0/2762511325 learned_addr learned my addr 172.21.15.1:0/2762511325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 -- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 msgr2=0x56177031ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177031ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 -- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350030 msgr2=0x561770324e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350030 0x561770324e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 -- 172.21.15.1:0/2762511325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617700677d0 con 0x561770350770 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.708+0000 7fc811440640 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0109a0 tx=0x7fc7fc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211b40e004140602 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc7ef7fe640 1 -- 172.21.15.1:0/2762511325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01c050 con 0x561770350770 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc7ef7fe640 1 -- 172.21.15.1:0/2762511325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc026070 con 0x561770350770 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc7ef7fe640 1 -- 172.21.15.1:0/2762511325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00f040 con 0x561770350770 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 -- 172.21.15.1:0/2762511325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 msgr2=0x56177034faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0109a0 tx=0x7fc7fc013000 comp rx=0 tx=0).stop 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 -- 172.21.15.1:0/2762511325 shutdown_connections 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350030 0x561770324e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770350770 0x56177034faf0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2762511325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177031ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 -- 172.21.15.1:0/2762511325 >> 172.21.15.1:0/2762511325 conn(0x561770322250 msgr2=0x56177036aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 -- 172.21.15.1:0/2762511325 shutdown_connections 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.709+0000 7fc816bb5740 1 -- 172.21.15.1:0/2762511325 wait complete. 2024-05-09T06:31:45.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.710+0000 7fc816bb5740 1 -- start start 2024-05-09T06:31:45.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.710+0000 7fc816bb5740 1 Processor -- start 2024-05-09T06:31:45.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x5617703408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770351640 0x56177034c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561770324400 con 0x56177036d650 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561770244b90 con 0x561770350770 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5617703237f0 con 0x561770351640 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc816bb5740 10 client.0 _ll_register_callbacks cb 0x561770280da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57606/0 (socket says 172.21.15.1:57606) 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 -- 172.21.15.1:0/2318512594 learned_addr learned my addr 172.21.15.1:0/2318512594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 -- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770351640 msgr2=0x56177034c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770351640 0x56177034c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811c41640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x5617703408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 -- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 msgr2=0x5617703408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x5617703408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.711+0000 7fc811440640 1 -- 172.21.15.1:0/2318512594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176ff9d3f0 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc811440640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc022000 tx=0x7fc7fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1005d4c59e6e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc02f070 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7fc7fc004440 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc026070 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc03a050 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 10 client.6897 fetch_fsmap learned FSMap version 37 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x56176ff5d0f0 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc026070 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7fc7fc03d020 con 0x56177036d650 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 10 client.6897 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 10 client.6897 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 20 client.6897 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x56177033e220 0x56177034ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 4 client.6897 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.712+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561770297430 con 0x56177033e220 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.713+0000 7fc811c41640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x56177033e220 0x56177034ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.713+0000 7fc7ecff9640 10 client.6897 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.713+0000 7fc811c41640 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x56177033e220 0x56177034ed80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b38a1f259735a7 server_cookie=ad9330b9fe7e0300 in_seq=0 out_seq=0 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc7ecff9640 1 -- 172.21.15.1:0/2318512594 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561770297430 con 0x56177033e220 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc7ecff9640 10 client.6897 handle_command_reply: tid=0 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 client.6897 shutdown 2024-05-09T06:31:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 msgr2=0x5617703538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc022000 tx=0x7fc7fc004730 comp rx=0 tx=0).stop 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 shutdown_connections 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x56177033e220 0x56177034ed80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770351640 0x56177034c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc7ecff9640 0 client.6897 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703538b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 --2- 172.21.15.1:0/2318512594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x5617703408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 >> 172.21.15.1:0/2318512594 conn(0x561770350030 msgr2=0x56177036cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 shutdown_connections 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 1 -- 172.21.15.1:0/2318512594 wait complete. 2024-05-09T06:31:45.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.714+0000 7fc816bb5740 20 client.6897 trim_cache size 0 max 16384 2024-05-09T06:31:45.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.732+0000 7fc816bb5740 1 Processor -- start 2024-05-09T06:31:45.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.732+0000 7fc816bb5740 1 -- start start 2024-05-09T06:31:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x561770361490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177036aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561770384f60 con 0x561770351640 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c200c20 con 0x561770350770 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c1ffc20 con 0x56177036d650 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57614/0 (socket says 172.21.15.1:57614) 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 -- 172.21.15.1:0/51835430 learned_addr learned my addr 172.21.15.1:0/51835430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc812442640 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177036aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811c41640 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x561770361490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 -- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 msgr2=0x56177036aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177036aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 -- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 msgr2=0x561770361490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x561770361490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 -- 172.21.15.1:0/51835430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56177036e580 con 0x561770351640 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.733+0000 7fc811440640 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00c6e0 tx=0x7fc7fc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216f7629b3b00a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc7eeffd640 1 -- 172.21.15.1:0/51835430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc043070 con 0x561770351640 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc7eeffd640 1 -- 172.21.15.1:0/51835430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc026070 con 0x561770351640 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc7eeffd640 1 -- 172.21.15.1:0/51835430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01c030 con 0x561770351640 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 -- 172.21.15.1:0/51835430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 msgr2=0x5617703619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00c6e0 tx=0x7fc7fc02c000 comp rx=0 tx=0).stop 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 -- 172.21.15.1:0/51835430 shutdown_connections 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56177036d650 0x56177036aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561770351640 0x5617703619d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 --2- 172.21.15.1:0/51835430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770350770 0x561770361490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 -- 172.21.15.1:0/51835430 >> 172.21.15.1:0/51835430 conn(0x561770350030 msgr2=0x561770352fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 -- 172.21.15.1:0/51835430 shutdown_connections 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.734+0000 7fc816bb5740 1 -- 172.21.15.1:0/51835430 wait complete. 2024-05-09T06:31:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.735+0000 7fc816bb5740 1 -- start start 2024-05-09T06:31:45.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.735+0000 7fc816bb5740 1 Processor -- start 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770350770 0x56177031fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770351640 0x5617703e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1ff5d0 con 0x56177036d650 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1fefe0 con 0x561770351640 2024-05-09T06:31:45.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5617703e5660 con 0x561770350770 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc816bb5740 10 client.0 _ll_register_callbacks cb 0x7fc7fc010840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57620/0 (socket says 172.21.15.1:57620) 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 -- 172.21.15.1:0/1450495892 learned_addr learned my addr 172.21.15.1:0/1450495892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc811c41640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770350770 0x56177031fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc811440640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770351640 0x5617703e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 -- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770350770 msgr2=0x56177031fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770350770 0x56177031fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 -- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770351640 msgr2=0x5617703e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770351640 0x5617703e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 -- 172.21.15.1:0/1450495892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561770368130 con 0x56177036d650 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc812442640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc804007920 tx=0x7fc804001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f0e74fc801dad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.736+0000 7fc7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804002090 con 0x56177036d650 2024-05-09T06:31:45.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7fc804013570 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc026070 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc804021050 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 10 client.6903 fetch_fsmap learned FSMap version 37 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x5617703e53a0 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc026070 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7fc804024020 con 0x56177036d650 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 10 client.6903 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 10 client.6903 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 20 client.6903 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5617703e4f90 0x56177034a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 4 client.6903 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561770291780 con 0x5617703e4f90 2024-05-09T06:31:45.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.737+0000 7fc811c41640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5617703e4f90 0x56177034a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc811c41640 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5617703e4f90 0x56177034a740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73894c032797409f server_cookie=c56e2f8c7431ff1e in_seq=0 out_seq=0 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc7ecff9640 10 client.6903 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc7ecff9640 1 -- 172.21.15.1:0/1450495892 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561770291780 con 0x5617703e4f90 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc7ecff9640 10 client.6903 handle_command_reply: tid=0 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 client.6903 shutdown 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 msgr2=0x5617703e4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc804007920 tx=0x7fc804001a00 comp rx=0 tx=0).stop 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 shutdown_connections 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177036d650 0x5617703e4a50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561770351640 0x5617703e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x5617703e4f90 0x56177034a740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 --2- 172.21.15.1:0/1450495892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561770350770 0x56177031fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 >> 172.21.15.1:0/1450495892 conn(0x561770350030 msgr2=0x56177036ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc7ecff9640 0 client.6903 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 shutdown_connections 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 1 -- 172.21.15.1:0/1450495892 wait complete. 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.738+0000 7fc816bb5740 20 client.6903 trim_cache size 0 max 16384 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:31:45.726 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:31:45.727 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:31:45.727 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:31:45.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 msgr2=0x7fc7f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 0x7fc7f006e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ad5c0 tx=0x7fc7fc008040 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 msgr2=0x7fc80c0ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc808035040 tx=0x7fc8080046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 shutdown_connections 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc7f006be30 0x7fc7f006e2f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc80c15e390 0x7fc80c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc80c0cf990 0x7fc80c1e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 --2- 172.21.15.1:0/318418948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc80c0b1730 0x7fc80c0ad080 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 >> 172.21.15.1:0/318418948 conn(0x7fc80c0b1da0 msgr2=0x7fc80c0cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 shutdown_connections 2024-05-09T06:31:45.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.739+0000 7fc813ea5640 1 -- 172.21.15.1:0/318418948 wait complete. 2024-05-09T06:31:45.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:31:45.732 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:45.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.854+0000 7f9bb19b0640 1 Processor -- start 2024-05-09T06:31:45.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.854+0000 7f9bb19b0640 1 -- start start 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bb19b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 0x7f9bac0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bb19b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9bac0d2b10 0x7f9bac0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bb19b0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac0b33c0 con 0x7f9bac161950 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bb19b0640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9bac0d4dd0 con 0x7f9bac0d2b10 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bb19b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0b3660 con 0x7f9bac0d4940 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bab7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9bac0d2b10 0x7f9bac0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bab7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9bac0d2b10 0x7f9bac0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57628/0 (socket says 172.21.15.1:57628) 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.855+0000 7f9bab7fe640 1 -- 172.21.15.1:0/2692309202 learned_addr learned my addr 172.21.15.1:0/2692309202 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9baa7fc640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 0x7f9bac0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9bab7fe640 1 -- 172.21.15.1:0/2692309202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 msgr2=0x7f9bac0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9bab7fe640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 0x7f9bac0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9bab7fe640 1 -- 172.21.15.1:0/2692309202 >> v1:172.21.15.6:6789/0 conn(0x7f9bac161950 legacy=0x7f9bac0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9bab7fe640 1 -- 172.21.15.1:0/2692309202 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac0b3940 con 0x7f9bac0d2b10 2024-05-09T06:31:45.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9baa7fc640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 0x7f9bac0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:45.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.856+0000 7f9bab7fe640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b4680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06b540 tx=0x7f9b9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39434a65fc86f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9ba9ffb640 1 -- 172.21.15.1:0/2692309202 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c036070 con 0x7f9bac0d2b10 2024-05-09T06:31:45.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9ba9ffb640 1 -- 172.21.15.1:0/2692309202 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c031600 con 0x7f9bac0d2b10 2024-05-09T06:31:45.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9ba9ffb640 1 -- 172.21.15.1:0/2692309202 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0319d0 con 0x7f9bac0d2b10 2024-05-09T06:31:45.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 -- 172.21.15.1:0/2692309202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac0d2b10 msgr2=0x7f9bac0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b4680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06b540 tx=0x7f9b9c02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 -- 172.21.15.1:0/2692309202 shutdown_connections 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b4680 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/2692309202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac0d4940 0x7f9bac0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 -- 172.21.15.1:0/2692309202 >> 172.21.15.1:0/2692309202 conn(0x7f9bac0d3e20 msgr2=0x7f9bac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 -- 172.21.15.1:0/2692309202 shutdown_connections 2024-05-09T06:31:45.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.857+0000 7f9bb19b0640 1 -- 172.21.15.1:0/2692309202 wait complete. 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.858+0000 7f9bb19b0640 1 Processor -- start 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 -- start start 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac161950 0x7f9bac153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0d1100 con 0x7f9bac15c550 2024-05-09T06:31:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0d0f80 con 0x7f9bac0d2b10 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.859+0000 7f9bb19b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0d1c00 con 0x7f9bac161950 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57640/0 (socket says 172.21.15.1:57640) 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 -- 172.21.15.1:0/3077666545 learned_addr learned my addr 172.21.15.1:0/3077666545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9baaffd640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9baa7fc640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac161950 0x7f9bac153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 -- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac161950 msgr2=0x7f9bac153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac161950 0x7f9bac153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 -- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bac0d2b10 msgr2=0x7f9bac0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 -- 172.21.15.1:0/3077666545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac0b7310 con 0x7f9bac15c550 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bab7fe640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009f40 tx=0x7f9b9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c036050 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c042070 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac0b75f0 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c009510 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.860+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac0b7bb0 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.861+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac0752b0 con 0x7f9bac15c550 2024-05-09T06:31:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.862+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b9c007bd0 con 0x7f9bac15c550 2024-05-09T06:31:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.862+0000 7f9b93fff640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.862+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f9b9c0abf60 con 0x7f9bac15c550 2024-05-09T06:31:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.862+0000 7f9baaffd640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.863+0000 7f9baaffd640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 0x7f9b9806e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4006fd0 tx=0x7f9ba4009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:45.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:45.867+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b9c07a020 con 0x7f9bac15c550 2024-05-09T06:31:46.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.088+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bac0b5370 con 0x7f9bac15c550 2024-05-09T06:31:46.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.088+0000 7f9b93fff640 1 -- 172.21.15.1:0/3077666545 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5168 (secure 0 0 0) 0x7f9b9c07e320 con 0x7f9bac15c550 2024-05-09T06:31:46.070 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:46.070 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:31:15.612251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:46.071 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 37 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 msgr2=0x7f9b9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 0x7f9b9806e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4006fd0 tx=0x7f9ba4009060 comp rx=0 tx=0).stop 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 msgr2=0x7f9bac0b83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009f40 tx=0x7f9b9c004a40 comp rx=0 tx=0).stop 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 shutdown_connections 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bac15c550 0x7f9bac0b83b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bac161950 0x7f9bac153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 --2- 172.21.15.1:0/3077666545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bac0d2b10 0x7f9bac0b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 >> 172.21.15.1:0/3077666545 conn(0x7f9bac0d3e20 msgr2=0x7f9bac0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 shutdown_connections 2024-05-09T06:31:46.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.090+0000 7f9bb19b0640 1 -- 172.21.15.1:0/3077666545 wait complete. 2024-05-09T06:31:46.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-09T06:31:46.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.208+0000 7f2d19053640 1 Processor -- start 2024-05-09T06:31:46.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.208+0000 7f2d19053640 1 -- start start 2024-05-09T06:31:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2d14158a50 0x7f2d140718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2d1415c4a0 0x7f2d14071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d1406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2d14158e30 con 0x7f2d14158a50 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2d14072480 con 0x7f2d1415c4a0 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d19053640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1415b420 con 0x7f2d1406e6d0 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d12d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2d14158a50 0x7f2d140718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d12d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2d14158a50 0x7f2d140718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57642/0 (socket says 172.21.15.1:57642) 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.209+0000 7f2d12d9d640 1 -- 172.21.15.1:0/1650591104 learned_addr learned my addr 172.21.15.1:0/1650591104 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d1259c640 1 --2- 172.21.15.1:0/1650591104 >> v2:172.21.15.6:3300/0 conn(0x7f2d1415c4a0 0x7f2d14071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d1359e640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d1406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 -- 172.21.15.1:0/1650591104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 msgr2=0x7f2d1406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d1406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 -- 172.21.15.1:0/1650591104 >> v2:172.21.15.6:3300/0 conn(0x7f2d1415c4a0 msgr2=0x7f2d14071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/1650591104 >> v2:172.21.15.6:3300/0 conn(0x7f2d1415c4a0 0x7f2d14071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 -- 172.21.15.1:0/1650591104 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1415b6e0 con 0x7f2d14158a50 2024-05-09T06:31:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.210+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d14158a50 0x7f2d140718c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2d14068a10 tx=0x7f2d0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383efeb6fe8e02e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d11d9b640 1 -- 172.21.15.1:0/1650591104 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c036070 con 0x7f2d14158a50 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d11d9b640 1 -- 172.21.15.1:0/1650591104 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c031600 con 0x7f2d14158a50 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d11d9b640 1 -- 172.21.15.1:0/1650591104 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0319d0 con 0x7f2d14158a50 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 -- 172.21.15.1:0/1650591104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d14158a50 msgr2=0x7f2d140718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d14158a50 0x7f2d140718c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2d14068a10 tx=0x7f2d0c02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 -- 172.21.15.1:0/1650591104 shutdown_connections 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d1406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 --2- 172.21.15.1:0/1650591104 >> v2:172.21.15.6:3300/0 conn(0x7f2d1415c4a0 0x7f2d14071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 --2- 172.21.15.1:0/1650591104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d14158a50 0x7f2d140718c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 -- 172.21.15.1:0/1650591104 >> 172.21.15.1:0/1650591104 conn(0x7f2d140cc4d0 msgr2=0x7f2d140cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 -- 172.21.15.1:0/1650591104 shutdown_connections 2024-05-09T06:31:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.211+0000 7f2d19053640 1 -- 172.21.15.1:0/1650591104 wait complete. 2024-05-09T06:31:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.213+0000 7f2d19053640 1 Processor -- start 2024-05-09T06:31:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.213+0000 7f2d19053640 1 -- start start 2024-05-09T06:31:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.213+0000 7f2d19053640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d14151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.213+0000 7f2d19053640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d14158a50 0x7f2d14147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1415bf80 con 0x7f2d1415c4a0 2024-05-09T06:31:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1415bca0 con 0x7f2d14158a50 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d14070ea0 con 0x7f2d1406e6d0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57656/0 (socket says 172.21.15.1:57656) 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 -- 172.21.15.1:0/798222020 learned_addr learned my addr 172.21.15.1:0/798222020 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d14151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1259c640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d14158a50 0x7f2d14147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 -- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 msgr2=0x7f2d14151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d14151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 -- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d14158a50 msgr2=0x7f2d14147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d14158a50 0x7f2d14147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 -- 172.21.15.1:0/798222020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1415cdd0 con 0x7f2d1415c4a0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d1359e640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400e7f0 tx=0x7f2d0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d0b7fe640 1 -- 172.21.15.1:0/798222020 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04017070 con 0x7f2d1415c4a0 2024-05-09T06:31:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d0b7fe640 1 -- 172.21.15.1:0/798222020 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0400c730 con 0x7f2d1415c4a0 2024-05-09T06:31:46.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d0b7fe640 1 -- 172.21.15.1:0/798222020 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400cb00 con 0x7f2d1415c4a0 2024-05-09T06:31:46.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1415d0b0 con 0x7f2d1415c4a0 2024-05-09T06:31:46.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.214+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1415d670 con 0x7f2d1415c4a0 2024-05-09T06:31:46.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.216+0000 7f2d0b7fe640 1 -- 172.21.15.1:0/798222020 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d0401a070 con 0x7f2d1415c4a0 2024-05-09T06:31:46.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.217+0000 7f2d0b7fe640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 0x7f2cf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.217+0000 7f2d0b7fe640 1 -- 172.21.15.1:0/798222020 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f2d0408b190 con 0x7f2d1415c4a0 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.217+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 0x7f2cf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.217+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 0x7f2cf406e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c033000 tx=0x7f2d0c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.218+0000 7f2d1be33740 1 Processor -- start 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.218+0000 7f2d1be33740 1 -- start start 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c7feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c82eb60 0x55e18c803970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e18c767a30 con 0x55e18c82f2a0 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e18c705590 con 0x55e18c82eb60 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e18c7639f0 con 0x55e18c84c180 2024-05-09T06:31:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57668/0 (socket says 172.21.15.1:57668) 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 -- 172.21.15.1:0/2980141887 learned_addr learned my addr 172.21.15.1:0/2980141887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c7feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1359e640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c82eb60 0x55e18c803970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 -- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 msgr2=0x55e18c7feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c7feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 -- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c82eb60 msgr2=0x55e18c803970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c82eb60 0x55e18c803970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 -- 172.21.15.1:0/2980141887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18c5477c0 con 0x55e18c82f2a0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c7feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009c30 tx=0x7f2cfc011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb9c89bf58a478 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.219+0000 7f2d08ff9640 1 -- 172.21.15.1:0/2980141887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc019070 con 0x55e18c82f2a0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d08ff9640 1 -- 172.21.15.1:0/2980141887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc004030 con 0x55e18c82f2a0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d08ff9640 1 -- 172.21.15.1:0/2980141887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0043e0 con 0x55e18c82f2a0 2024-05-09T06:31:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 -- 172.21.15.1:0/2980141887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 msgr2=0x55e18c82e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009c30 tx=0x7f2cfc011690 comp rx=0 tx=0).stop 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 -- 172.21.15.1:0/2980141887 shutdown_connections 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c82eb60 0x55e18c803970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c82e620 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2980141887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c7feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 -- 172.21.15.1:0/2980141887 >> 172.21.15.1:0/2980141887 conn(0x55e18c800d80 msgr2=0x55e18c849620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 -- 172.21.15.1:0/2980141887 shutdown_connections 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.220+0000 7f2d1be33740 1 -- 172.21.15.1:0/2980141887 wait complete. 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.221+0000 7f2d1be33740 1 -- start start 2024-05-09T06:31:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.221+0000 7f2d1be33740 1 Processor -- start 2024-05-09T06:31:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c81b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c8487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e18c802f30 con 0x55e18c82f2a0 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e18c703060 con 0x55e18c81ae80 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e18c802320 con 0x55e18c84c180 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1be33740 10 client.0 _ll_register_callbacks cb 0x55e18c727dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57680/0 (socket says 172.21.15.1:57680) 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 -- 172.21.15.1:0/2962388145 learned_addr learned my addr 172.21.15.1:0/2962388145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c81b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1359e640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c8487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 -- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 msgr2=0x55e18c81b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c81b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 -- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 msgr2=0x55e18c8487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c8487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 -- 172.21.15.1:0/2962388145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18c5a7f50 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d1359e640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c8487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c03d460 tx=0x7f2d0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481da1a58249bae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2cf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c036040 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.222+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f2d0c0316d0 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc004030 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c003990 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 10 client.6921 fetch_fsmap learned FSMap version 37 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55e18c43b0b0 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc004030 con 0x55e18c82f2a0 2024-05-09T06:31:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f2d0c003b30 con 0x55e18c82f2a0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 10 client.6921 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 10 client.6921 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 20 client.6921 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c8039c0 0x55e18c803da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 4 client.6921 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18c775160 con 0x55e18c8039c0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.223+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c8039c0 0x55e18c803da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.224+0000 7f2cf1ffb640 10 client.6921 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.224+0000 7f2d1259c640 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c8039c0 0x55e18c803da0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c650b634a0b90e server_cookie=88c8625e1448368 in_seq=0 out_seq=0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.224+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/2962388145 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e18c775160 con 0x55e18c8039c0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.224+0000 7f2cf1ffb640 10 client.6921 handle_command_reply: tid=0 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.224+0000 7f2d1be33740 1 client.6921 shutdown 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 msgr2=0x55e18c830ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c03d460 tx=0x7f2d0c004ab0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 shutdown_connections 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c8487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c8039c0 0x55e18c803da0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c84c180 0x55e18c81b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2cf1ffb640 0 client.6921 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 --2- 172.21.15.1:0/2962388145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c830ab0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 >> 172.21.15.1:0/2962388145 conn(0x55e18c82eb60 msgr2=0x55e18c7fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 shutdown_connections 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 1 -- 172.21.15.1:0/2962388145 wait complete. 2024-05-09T06:31:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.225+0000 7f2d1be33740 20 client.6921 trim_cache size 0 max 16384 2024-05-09T06:31:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.238+0000 7f2d1be33740 1 Processor -- start 2024-05-09T06:31:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.238+0000 7f2d1be33740 1 -- start start 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c7febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c82f2a0 0x55e18c8486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141fb940 con 0x55e18c84c180 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1406e1e0 con 0x55e18c81ae80 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d140ce2f0 con 0x55e18c82f2a0 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57692/0 (socket says 172.21.15.1:57692) 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 -- 172.21.15.1:0/934997157 learned_addr learned my addr 172.21.15.1:0/934997157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 -- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c82f2a0 msgr2=0x55e18c8486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c82f2a0 0x55e18c8486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c7febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 -- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 msgr2=0x55e18c7febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c7febd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 -- 172.21.15.1:0/934997157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18c86d8a0 con 0x55e18c84c180 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2d1359e640 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400c520 tx=0x7f2d04078040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287ec0629116b0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2cf3fff640 1 -- 172.21.15.1:0/934997157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04021040 con 0x55e18c84c180 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2cf3fff640 1 -- 172.21.15.1:0/934997157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04057600 con 0x55e18c84c180 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.239+0000 7f2cf3fff640 1 -- 172.21.15.1:0/934997157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04055970 con 0x55e18c84c180 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 -- 172.21.15.1:0/934997157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 msgr2=0x55e18c849420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400c520 tx=0x7f2d04078040 comp rx=0 tx=0).stop 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 -- 172.21.15.1:0/934997157 shutdown_connections 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c84c180 0x55e18c849420 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c82f2a0 0x55e18c8486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 --2- 172.21.15.1:0/934997157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c81ae80 0x55e18c7febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 -- 172.21.15.1:0/934997157 >> 172.21.15.1:0/934997157 conn(0x55e18c82eb60 msgr2=0x55e18c846550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 -- 172.21.15.1:0/934997157 shutdown_connections 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.240+0000 7f2d1be33740 1 -- 172.21.15.1:0/934997157 wait complete. 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 -- start start 2024-05-09T06:31:46.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 Processor -- start 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 0x55e18c86ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 0x55e18c802560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e18c82f950 con 0x55e18c82f2a0 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e18c81b300 con 0x55e18c84c180 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e18c84ac70 con 0x55e18c81ae80 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.241+0000 7f2d1be33740 10 client.0 _ll_register_callbacks cb 0x7f2cfc009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57700/0 (socket says 172.21.15.1:57700) 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 -- 172.21.15.1:0/3291648197 learned_addr learned my addr 172.21.15.1:0/3291648197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 0x55e18c86ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1359e640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 0x55e18c802560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 -- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 msgr2=0x55e18c86ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 0x55e18c86ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 -- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 msgr2=0x55e18c802560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 0x55e18c802560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 -- 172.21.15.1:0/3291648197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18c847ad0 con 0x55e18c82f2a0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1359e640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 0x55e18c802560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 0x55e18c86ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1259c640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011b90 tx=0x7f2cfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561ce3d7acf2c35e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2cf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc019040 con 0x55e18c82f2a0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f2cfc01e450 con 0x55e18c82f2a0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d04057600 con 0x55e18c82f2a0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2cfc009710 con 0x55e18c82f2a0 2024-05-09T06:31:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1be33740 10 client.6927 fetch_fsmap learned FSMap version 37 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55e18c849fd0 con 0x55e18c82f2a0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.242+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04057600 con 0x55e18c82f2a0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f2cfc036080 con 0x55e18c82f2a0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 10 client.6927 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 10 client.6927 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 20 client.6927 populate_metadata read hostname 'smithi001' 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c802aa0 0x55e18c7fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 4 client.6927 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 --> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e18c76f4b0 con 0x55e18c802aa0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c802aa0 0x55e18c7fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2cf1ffb640 10 client.6927 ms_handle_connect on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d12d9d640 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c802aa0 0x55e18c7fd9b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63161428f0b43b16 server_cookie=c53abdac120d3284 in_seq=0 out_seq=0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2cf1ffb640 1 -- 172.21.15.1:0/3291648197 <== mds.0 v2:172.21.15.6:6833/531134827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e18c76f4b0 con 0x55e18c802aa0 2024-05-09T06:31:46.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2cf1ffb640 10 client.6927 handle_command_reply: tid=0 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 1 client.6927 shutdown 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 msgr2=0x55e18c866d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.243+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc011b90 tx=0x7f2cfc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 shutdown_connections 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e18c84c180 0x55e18c802560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e18c82f2a0 0x55e18c866d40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:6833/531134827,v1:172.21.15.6:6835/531134827] conn(0x55e18c802aa0 0x55e18c7fd9b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 --2- 172.21.15.1:0/3291648197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e18c81ae80 0x55e18c86ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2cf1ffb640 0 client.6927 ms_handle_reset on v2:172.21.15.6:6833/531134827 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 >> 172.21.15.1:0/3291648197 conn(0x55e18c82eb60 msgr2=0x55e18c82fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 shutdown_connections 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 1 -- 172.21.15.1:0/3291648197 wait complete. 2024-05-09T06:31:46.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d1be33740 20 client.6927 trim_cache size 0 max 16384 2024-05-09T06:31:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:46.233 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 msgr2=0x7f2cf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 0x7f2cf406e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c033000 tx=0x7f2d0c009000 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 msgr2=0x7f2d1415de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400e7f0 tx=0x7f2d0400ecb0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 shutdown_connections 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2cf406be30 0x7f2cf406e2f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1415c4a0 0x7f2d1415de70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d14158a50 0x7f2d14147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 --2- 172.21.15.1:0/798222020 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d1406e6d0 0x7f2d14151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 >> 172.21.15.1:0/798222020 conn(0x7f2d140cc4d0 msgr2=0x7f2d1415a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 shutdown_connections 2024-05-09T06:31:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.244+0000 7f2d19053640 1 -- 172.21.15.1:0/798222020 wait complete. 2024-05-09T06:31:46.236 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:31:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 Processor -- start 2024-05-09T06:31:46.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 -- start start 2024-05-09T06:31:46.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 0x7f62840b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 0x7f62840b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62840b33c0 con 0x7f6284161950 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f62840d4dd0 con 0x7f62840d2b10 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.359+0000 7f628a968640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f62840b3660 con 0x7f62840d4940 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f6283fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6284161950 0x7f62840d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51964/0 (socket says 172.21.15.1:51964) 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f6283fff640 1 -- 172.21.15.1:0/762328650 learned_addr learned my addr 172.21.15.1:0/762328650 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f627bfff640 1 --2- 172.21.15.1:0/762328650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 0x7f62840b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f6288f05640 1 --2- 172.21.15.1:0/762328650 >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 0x7f62840b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f62837fe640 1 -- 172.21.15.1:0/762328650 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064692794 0 0) 0x7f62840b33c0 con 0x7f6284161950 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f62837fe640 1 -- 172.21.15.1:0/762328650 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6270003520 con 0x7f6284161950 2024-05-09T06:31:46.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f627bfff640 1 -- 172.21.15.1:0/762328650 >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 msgr2=0x7f62840b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f627bfff640 1 --2- 172.21.15.1:0/762328650 >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 0x7f62840b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f627bfff640 1 -- 172.21.15.1:0/762328650 >> v1:172.21.15.1:6789/0 conn(0x7f6284161950 legacy=0x7f62840d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f627bfff640 1 -- 172.21.15.1:0/762328650 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62840b3940 con 0x7f62840d4940 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.360+0000 7f6288f05640 1 --2- 172.21.15.1:0/762328650 >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 0x7f62840b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.361+0000 7f627bfff640 1 --2- 172.21.15.1:0/762328650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 0x7f62840b4140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f626c0097a0 tx=0x7f626c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769d9b9af0310db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.361+0000 7f62837fe640 1 -- 172.21.15.1:0/762328650 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c037070 con 0x7f62840d4940 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.361+0000 7f62837fe640 1 -- 172.21.15.1:0/762328650 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f626c032440 con 0x7f62840d4940 2024-05-09T06:31:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.361+0000 7f62837fe640 1 -- 172.21.15.1:0/762328650 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c032810 con 0x7f62840d4940 2024-05-09T06:31:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 -- 172.21.15.1:0/762328650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 msgr2=0x7f62840b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 --2- 172.21.15.1:0/762328650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 0x7f62840b4140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f626c0097a0 tx=0x7f626c02a180 comp rx=0 tx=0).stop 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 -- 172.21.15.1:0/762328650 shutdown_connections 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 --2- 172.21.15.1:0/762328650 >> v2:172.21.15.6:3300/0 conn(0x7f62840d2b10 0x7f62840b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 --2- 172.21.15.1:0/762328650 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f62840d4940 0x7f62840b4140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 -- 172.21.15.1:0/762328650 >> 172.21.15.1:0/762328650 conn(0x7f62840d3e20 msgr2=0x7f62840d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 -- 172.21.15.1:0/762328650 shutdown_connections 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.362+0000 7f628a968640 1 -- 172.21.15.1:0/762328650 wait complete. 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.363+0000 7f628a968640 1 Processor -- start 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.363+0000 7f628a968640 1 -- start start 2024-05-09T06:31:46.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6284161950 0x7f628409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f628409dfe0 0x7f628414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62840d1100 con 0x7f62840d2b10 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f62840d0f80 con 0x7f628409dfe0 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f628a968640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f62840d1c00 con 0x7f6284161950 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57714/0 (socket says 172.21.15.1:57714) 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 -- 172.21.15.1:0/3057432836 learned_addr learned my addr 172.21.15.1:0/3057432836 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 -- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6284161950 msgr2=0x7f628409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6284161950 0x7f628409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 -- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f628409dfe0 msgr2=0x7f628414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f628409dfe0 0x7f628414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 -- 172.21.15.1:0/3057432836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628414ea80 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.364+0000 7f6283fff640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f627400c400 tx=0x7f627400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.365+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274017070 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.365+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6274005d00 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.365+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628414ed60 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.365+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274007310 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.365+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628414f320 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.366+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62840b46f0 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.366+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f627401a070 con 0x7f62840d2b10 2024-05-09T06:31:46.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.367+0000 7f62817fa640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 0x7f626406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.367+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f627408a430 con 0x7f62840d2b10 2024-05-09T06:31:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.367+0000 7f627bfff640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 0x7f626406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.367+0000 7f627bfff640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 0x7f626406e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f626c034000 tx=0x7f626c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.371+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f62840b46f0 con 0x7f62840d2b10 2024-05-09T06:31:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.598+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f628415ec10 con 0x7f62840d2b10 2024-05-09T06:31:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.599+0000 7f62817fa640 1 -- 172.21.15.1:0/3057432836 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f628415ec10 con 0x7f62840d2b10 2024-05-09T06:31:46.581 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:46.581 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 msgr2=0x7f626406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 0x7f626406e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f626c034000 tx=0x7f626c02c000 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 msgr2=0x7f628409f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f627400c400 tx=0x7f627400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 shutdown_connections 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f626406be30 0x7f626406e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f628409dfe0 0x7f628414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6284161950 0x7f628409daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 --2- 172.21.15.1:0/3057432836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62840d2b10 0x7f628409f970 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 >> 172.21.15.1:0/3057432836 conn(0x7f62840d3e20 msgr2=0x7f62840b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 shutdown_connections 2024-05-09T06:31:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.601+0000 7f628a968640 1 -- 172.21.15.1:0/3057432836 wait complete. 2024-05-09T06:31:46.593 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:31:46.594 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:31:46.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:31:46.633 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:31:46.686 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:31:46.686 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:31:46.687 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:31:46.687 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:46.704 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:31:46.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.866+0000 7f8a38768640 1 Processor -- start 2024-05-09T06:31:46.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.866+0000 7f8a38768640 1 -- start start 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a38768640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8a301454b0 0x7f8a300b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a38768640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a300b2a60 0x7f8a300b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a38768640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a30071c20 con 0x7f8a300b4da0 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a38768640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3006eb40 con 0x7f8a300b2a60 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a38768640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3006ee20 con 0x7f8a301454b0 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35d03640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8a300b2a60 0x7f8a300b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a36d05640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8a300b4da0 0x7f8a3015f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51978/0 (socket says 172.21.15.1:51978) 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a36d05640 1 -- 172.21.15.1:0/2057398493 learned_addr learned my addr 172.21.15.1:0/2057398493 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35d03640 1 -- 172.21.15.1:0/2057398493 >> v2:172.21.15.6:3300/0 conn(0x7f8a301454b0 msgr2=0x7f8a300b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:46.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35d03640 1 --2- 172.21.15.1:0/2057398493 >> v2:172.21.15.6:3300/0 conn(0x7f8a301454b0 0x7f8a300b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35d03640 1 -- 172.21.15.1:0/2057398493 >> v1:172.21.15.1:6789/0 conn(0x7f8a300b4da0 legacy=0x7f8a3015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35d03640 1 -- 172.21.15.1:0/2057398493 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3006d8e0 con 0x7f8a300b2a60 2024-05-09T06:31:46.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.867+0000 7f8a35502640 1 -- 172.21.15.1:0/2057398493 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001581397 0 0) 0x7f8a30071c20 con 0x7f8a300b4da0 2024-05-09T06:31:46.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.868+0000 7f8a35d03640 1 --2- 172.21.15.1:0/2057398493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a300b2a60 0x7f8a300b4860 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8a300689d0 tx=0x7f8a2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e0e300c52b7c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.868+0000 7f8a35502640 1 -- 172.21.15.1:0/2057398493 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20037070 con 0x7f8a300b2a60 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.868+0000 7f8a35502640 1 -- 172.21.15.1:0/2057398493 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20032440 con 0x7f8a300b2a60 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.868+0000 7f8a35502640 1 -- 172.21.15.1:0/2057398493 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20032810 con 0x7f8a300b2a60 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 -- 172.21.15.1:0/2057398493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a300b2a60 msgr2=0x7f8a300b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 --2- 172.21.15.1:0/2057398493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a300b2a60 0x7f8a300b4860 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8a300689d0 tx=0x7f8a2002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 -- 172.21.15.1:0/2057398493 shutdown_connections 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 --2- 172.21.15.1:0/2057398493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a300b2a60 0x7f8a300b4860 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 --2- 172.21.15.1:0/2057398493 >> v2:172.21.15.6:3300/0 conn(0x7f8a301454b0 0x7f8a300b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 -- 172.21.15.1:0/2057398493 >> 172.21.15.1:0/2057398493 conn(0x7f8a300cc4b0 msgr2=0x7f8a300cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:46.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 -- 172.21.15.1:0/2057398493 shutdown_connections 2024-05-09T06:31:46.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.869+0000 7f8a38768640 1 -- 172.21.15.1:0/2057398493 wait complete. 2024-05-09T06:31:46.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.870+0000 7f8a38768640 1 Processor -- start 2024-05-09T06:31:46.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.870+0000 7f8a38768640 1 -- start start 2024-05-09T06:31:46.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 0x7f8a301545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a300b4da0 0x7f8a30139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30145890 con 0x7f8a301454b0 2024-05-09T06:31:46.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300b51f0 con 0x7f8a300b4da0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a38768640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3006e0e0 con 0x7f8a300b2a60 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57722/0 (socket says 172.21.15.1:57722) 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 -- 172.21.15.1:0/1354606049 learned_addr learned my addr 172.21.15.1:0/1354606049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a35d03640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a300b4da0 0x7f8a30139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36504640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 0x7f8a301545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 -- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 msgr2=0x7f8a301545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 0x7f8a301545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 -- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a300b4da0 msgr2=0x7f8a30139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a300b4da0 0x7f8a30139c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 -- 172.21.15.1:0/1354606049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a30154b00 con 0x7f8a301454b0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36504640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 0x7f8a301545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.871+0000 7f8a36d05640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c7a0 tx=0x7f8a2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.872+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28018070 con 0x7f8a301454b0 2024-05-09T06:31:46.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.872+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2800a640 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.872+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301f4230 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.872+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800aab0 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.872+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301f47f0 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.873+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a300752b0 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.873+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a2801b070 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.874+0000 7f8a26ffd640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 0x7f8a1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.874+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f8a2808a550 con 0x7f8a301454b0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.874+0000 7f8a36504640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 0x7f8a1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:46.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.874+0000 7f8a36504640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 0x7f8a1406e320 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8a3015bf00 tx=0x7f8a2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:46.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:46.878+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a280586b0 con 0x7f8a301454b0 2024-05-09T06:31:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.065+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a30072fa0 con 0x7f8a301454b0 2024-05-09T06:31:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.066+0000 7f8a26ffd640 1 -- 172.21.15.1:0/1354606049 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16944 (secure 0 0 0) 0x7f8a2805c9b0 con 0x7f8a301454b0 2024-05-09T06:31:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:47.048 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":66,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:31:03.690223+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 msgr2=0x7f8a1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 0x7f8a1406e320 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8a3015bf00 tx=0x7f8a2c008040 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 msgr2=0x7f8a3013a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c7a0 tx=0x7f8a2800cc60 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 shutdown_connections 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a1406be60 0x7f8a1406e320 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a301454b0 0x7f8a3013a170 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a300b4da0 0x7f8a30139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 --2- 172.21.15.1:0/1354606049 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a300b2a60 0x7f8a301545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 >> 172.21.15.1:0/1354606049 conn(0x7f8a300cc4b0 msgr2=0x7f8a30144200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 shutdown_connections 2024-05-09T06:31:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.068+0000 7f8a38768640 1 -- 172.21.15.1:0/1354606049 wait complete. 2024-05-09T06:31:47.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:30:33.248885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:30:33.918243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:31:47.061 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:31:47.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.188+0000 7ff759e38640 1 Processor -- start 2024-05-09T06:31:47.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.188+0000 7ff759e38640 1 -- start start 2024-05-09T06:31:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff759e38640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff7540cd450 0x7ff75415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff759e38640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff754074660 0x7ff754160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff759e38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75415c920 con 0x7ff7541605d0 2024-05-09T06:31:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff759e38640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff75415fc00 con 0x7ff7540cd450 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff759e38640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff75415cba0 con 0x7ff754074660 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff7540cd450 0x7ff75415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff753fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff7541605d0 0x7ff75415d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51986/0 (socket says 172.21.15.1:51986) 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff753fff640 1 -- 172.21.15.1:0/2414445702 learned_addr learned my addr 172.21.15.1:0/2414445702 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 --2- 172.21.15.1:0/2414445702 >> v2:172.21.15.1:3300/0 conn(0x7ff7540cd450 0x7ff75415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57732/0 (socket says 172.21.15.1:57732) 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 -- 172.21.15.1:0/2414445702 >> v2:172.21.15.6:3300/0 conn(0x7ff754074660 msgr2=0x7ff754160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 --2- 172.21.15.1:0/2414445702 >> v2:172.21.15.6:3300/0 conn(0x7ff754074660 0x7ff754160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 -- 172.21.15.1:0/2414445702 >> v1:172.21.15.1:6789/0 conn(0x7ff7541605d0 legacy=0x7ff75415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7527fc640 1 -- 172.21.15.1:0/2414445702 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660086167 0 0) 0x7ff74c001490 con 0x7ff7541605d0 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.189+0000 7ff7537fe640 1 -- 172.21.15.1:0/2414445702 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75415ce80 con 0x7ff7540cd450 2024-05-09T06:31:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff7537fe640 1 --2- 172.21.15.1:0/2414445702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff75415c3e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff7540cb890 tx=0x7ff74402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6a9cc5850e36a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff7527fc640 1 -- 172.21.15.1:0/2414445702 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744037070 con 0x7ff7540cd450 2024-05-09T06:31:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff7527fc640 1 -- 172.21.15.1:0/2414445702 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff744032440 con 0x7ff7540cd450 2024-05-09T06:31:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff7527fc640 1 -- 172.21.15.1:0/2414445702 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744032810 con 0x7ff7540cd450 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff759e38640 1 -- 172.21.15.1:0/2414445702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 msgr2=0x7ff75415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.190+0000 7ff759e38640 1 --2- 172.21.15.1:0/2414445702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff75415c3e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff7540cb890 tx=0x7ff74402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 -- 172.21.15.1:0/2414445702 shutdown_connections 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 --2- 172.21.15.1:0/2414445702 >> v2:172.21.15.6:3300/0 conn(0x7ff754074660 0x7ff754160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 --2- 172.21.15.1:0/2414445702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff75415c3e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 -- 172.21.15.1:0/2414445702 >> 172.21.15.1:0/2414445702 conn(0x7ff7540b1da0 msgr2=0x7ff7540b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 -- 172.21.15.1:0/2414445702 shutdown_connections 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.191+0000 7ff759e38640 1 -- 172.21.15.1:0/2414445702 wait complete. 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.192+0000 7ff759e38640 1 Processor -- start 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.192+0000 7ff759e38640 1 -- start start 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff754074660 0x7ff754099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 0x7ff754154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff75415dcb0 con 0x7ff7540cd450 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff7540cd830 con 0x7ff7541605d0 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff759e38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff75415de30 con 0x7ff754074660 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57744/0 (socket says 172.21.15.1:57744) 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 -- 172.21.15.1:0/7835474 learned_addr learned my addr 172.21.15.1:0/7835474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff753fff640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 0x7ff754154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff7537fe640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff754074660 0x7ff754099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 -- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff754074660 msgr2=0x7ff754099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff754074660 0x7ff754099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 -- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 msgr2=0x7ff754154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 0x7ff754154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 -- 172.21.15.1:0/7835474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff754099eb0 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff753fff640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 0x7ff754154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.193+0000 7ff752ffd640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b4a0 tx=0x7ff74800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.194+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748014070 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.194+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff748010610 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.194+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7540b41d0 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.194+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748010a00 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.194+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7540b4790 con 0x7ff7540cd450 2024-05-09T06:31:47.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.195+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7540752b0 con 0x7ff7540cd450 2024-05-09T06:31:47.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.195+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff748017070 con 0x7ff7540cd450 2024-05-09T06:31:47.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.196+0000 7ff73bfff640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.196+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7ff74808aa00 con 0x7ff7540cd450 2024-05-09T06:31:47.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.196+0000 7ff7537fe640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.196+0000 7ff7537fe640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 0x7ff74006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff744034000 tx=0x7ff74402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.200+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff754099eb0 con 0x7ff7540cd450 2024-05-09T06:31:47.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.383+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff75406d930 con 0x7ff7540cd450 2024-05-09T06:31:47.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.384+0000 7ff73bfff640 1 -- 172.21.15.1:0/7835474 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7ff748059240 con 0x7ff7540cd450 2024-05-09T06:31:47.366 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:31:47.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 msgr2=0x7ff74006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 0x7ff74006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff744034000 tx=0x7ff74402e000 comp rx=0 tx=0).stop 2024-05-09T06:31:47.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 msgr2=0x7ff7540b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b4a0 tx=0x7ff74800b960 comp rx=0 tx=0).stop 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 shutdown_connections 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff74006be30 0x7ff74006e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7541605d0 0x7ff754154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7540cd450 0x7ff7540b3860 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 --2- 172.21.15.1:0/7835474 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff754074660 0x7ff754099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 >> 172.21.15.1:0/7835474 conn(0x7ff7540b1da0 msgr2=0x7ff75406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 shutdown_connections 2024-05-09T06:31:47.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.386+0000 7ff759e38640 1 -- 172.21.15.1:0/7835474 wait complete. 2024-05-09T06:31:47.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:31:47.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.503+0000 7f1b6919d640 1 Processor -- start 2024-05-09T06:31:47.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.503+0000 7f1b6919d640 1 -- start start 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6919d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b64158a50 0x7f1b640718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6919d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 0x7f1b64071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6919d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6415b420 con 0x7f1b6406e6d0 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6919d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1b64158e30 con 0x7f1b64158a50 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6919d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1b64072480 con 0x7f1b6415c4a0 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b62d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b64158a50 0x7f1b640718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b62d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b64158a50 0x7f1b640718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57746/0 (socket says 172.21.15.1:57746) 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b62d9d640 1 -- 172.21.15.1:0/3091451162 learned_addr learned my addr 172.21.15.1:0/3091451162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.504+0000 7f1b6359e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1b6406e6d0 0x7f1b640cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51994/0 (socket says 172.21.15.1:51994) 2024-05-09T06:31:47.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b6259c640 1 --2- 172.21.15.1:0/3091451162 >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 0x7f1b64071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b62d9d640 1 -- 172.21.15.1:0/3091451162 >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 msgr2=0x7f1b64071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/3091451162 >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 0x7f1b64071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b62d9d640 1 -- 172.21.15.1:0/3091451162 >> v1:172.21.15.1:6789/0 conn(0x7f1b6406e6d0 legacy=0x7f1b640cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b61d9b640 1 -- 172.21.15.1:0/3091451162 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026490949 0 0) 0x7f1b6415b420 con 0x7f1b6406e6d0 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b62d9d640 1 -- 172.21.15.1:0/3091451162 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6415b6e0 con 0x7f1b64158a50 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b6259c640 1 --2- 172.21.15.1:0/3091451162 >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 0x7f1b64071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.505+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/3091451162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b640718c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1b64068a10 tx=0x7f1b5c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2738719f38ac3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b61d9b640 1 -- 172.21.15.1:0/3091451162 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c036070 con 0x7f1b64158a50 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b61d9b640 1 -- 172.21.15.1:0/3091451162 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c031600 con 0x7f1b64158a50 2024-05-09T06:31:47.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b61d9b640 1 -- 172.21.15.1:0/3091451162 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0319d0 con 0x7f1b64158a50 2024-05-09T06:31:47.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 -- 172.21.15.1:0/3091451162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 msgr2=0x7f1b640718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 --2- 172.21.15.1:0/3091451162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b640718c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1b64068a10 tx=0x7f1b5c02a730 comp rx=0 tx=0).stop 2024-05-09T06:31:47.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 -- 172.21.15.1:0/3091451162 shutdown_connections 2024-05-09T06:31:47.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 --2- 172.21.15.1:0/3091451162 >> v2:172.21.15.6:3300/0 conn(0x7f1b6415c4a0 0x7f1b64071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 --2- 172.21.15.1:0/3091451162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b640718c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 -- 172.21.15.1:0/3091451162 >> 172.21.15.1:0/3091451162 conn(0x7f1b640cc4d0 msgr2=0x7f1b640cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 -- 172.21.15.1:0/3091451162 shutdown_connections 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.506+0000 7f1b6919d640 1 -- 172.21.15.1:0/3091451162 wait complete. 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.508+0000 7f1b6919d640 1 Processor -- start 2024-05-09T06:31:47.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.508+0000 7f1b6919d640 1 -- start start 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.508+0000 7f1b6919d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 0x7f1b64156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b6415c4a0 0x7f1b6413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6415bf80 con 0x7f1b64158a50 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6415bca0 con 0x7f1b6415c4a0 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1b64070ea0 con 0x7f1b6406e6d0 2024-05-09T06:31:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57748/0 (socket says 172.21.15.1:57748) 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 -- 172.21.15.1:0/2538050483 learned_addr learned my addr 172.21.15.1:0/2538050483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6359e640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b6415c4a0 0x7f1b6413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 0x7f1b64156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 -- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 msgr2=0x7f1b64156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 0x7f1b64156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 -- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b6415c4a0 msgr2=0x7f1b6413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b6415c4a0 0x7f1b6413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 -- 172.21.15.1:0/2538050483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b64154e00 con 0x7f1b64158a50 2024-05-09T06:31:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 0x7f1b64156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6259c640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c7f0 tx=0x7f1b4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c015070 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c007820 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b641550e0 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c007bd0 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.509+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b641556a0 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.510+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b640752b0 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.511+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b4c018070 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.511+0000 7f1b5b7fe640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.512+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f1b4c08b200 con 0x7f1b64158a50 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.512+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.512+0000 7f1b62d9d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 0x7f1b4406e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c033000 tx=0x7f1b5c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.516+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b4c059240 con 0x7f1b64158a50 2024-05-09T06:31:47.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.702+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b64072730 con 0x7f1b64158a50 2024-05-09T06:31:47.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.702+0000 7f1b5b7fe640 1 -- 172.21.15.1:0/2538050483 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f1b4c05d540 con 0x7f1b64158a50 2024-05-09T06:31:47.684 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 msgr2=0x7f1b4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 0x7f1b4406e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c033000 tx=0x7f1b5c009000 comp rx=0 tx=0).stop 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 msgr2=0x7f1b6413fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c7f0 tx=0x7f1b4c00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 shutdown_connections 2024-05-09T06:31:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b6415c4a0 0x7f1b6413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b64158a50 0x7f1b6413fec0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 --2- 172.21.15.1:0/2538050483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b6406e6d0 0x7f1b64156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 >> 172.21.15.1:0/2538050483 conn(0x7f1b640cc4d0 msgr2=0x7f1b64158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 shutdown_connections 2024-05-09T06:31:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.704+0000 7f1b6919d640 1 -- 172.21.15.1:0/2538050483 wait complete. 2024-05-09T06:31:47.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:31:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.823+0000 7f1c9e29a640 1 Processor -- start 2024-05-09T06:31:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.823+0000 7f1c9e29a640 1 -- start start 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c9e29a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1c980cd450 0x7f1c9815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c9e29a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c98074660 0x7f1c98160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c9e29a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9815c920 con 0x7f1c981605d0 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c9e29a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9815cba0 con 0x7f1c980cd450 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c9e29a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9815ce80 con 0x7f1c98074660 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1c980cd450 0x7f1c9815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1c980cd450 0x7f1c9815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35108/0 (socket says 172.21.15.1:35108) 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 -- 172.21.15.1:0/76643282 learned_addr learned my addr 172.21.15.1:0/76643282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c977fe640 1 --2- 172.21.15.1:0/76643282 >> v2:172.21.15.6:3300/0 conn(0x7f1c98074660 0x7f1c98160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 -- 172.21.15.1:0/76643282 >> v2:172.21.15.6:3300/0 conn(0x7f1c98074660 msgr2=0x7f1c98160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 --2- 172.21.15.1:0/76643282 >> v2:172.21.15.6:3300/0 conn(0x7f1c98074660 0x7f1c98160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 -- 172.21.15.1:0/76643282 >> v1:172.21.15.6:6789/0 conn(0x7f1c981605d0 legacy=0x7f1c9815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.824+0000 7f1c97fff640 1 -- 172.21.15.1:0/76643282 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9815d160 con 0x7f1c980cd450 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.825+0000 7f1c97fff640 1 --2- 172.21.15.1:0/76643282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c980cd450 0x7f1c9815c3e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1c9809e380 tx=0x7f1c8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3984c3eb90cba90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.825+0000 7f1c96ffd640 1 -- 172.21.15.1:0/76643282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88037070 con 0x7f1c980cd450 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.825+0000 7f1c96ffd640 1 -- 172.21.15.1:0/76643282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c88032440 con 0x7f1c980cd450 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.825+0000 7f1c96ffd640 1 -- 172.21.15.1:0/76643282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88032810 con 0x7f1c980cd450 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 -- 172.21.15.1:0/76643282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c980cd450 msgr2=0x7f1c9815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/76643282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c980cd450 0x7f1c9815c3e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1c9809e380 tx=0x7f1c8802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 -- 172.21.15.1:0/76643282 shutdown_connections 2024-05-09T06:31:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/76643282 >> v2:172.21.15.6:3300/0 conn(0x7f1c98074660 0x7f1c98160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/76643282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c980cd450 0x7f1c9815c3e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 -- 172.21.15.1:0/76643282 >> 172.21.15.1:0/76643282 conn(0x7f1c980b1da0 msgr2=0x7f1c980b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 -- 172.21.15.1:0/76643282 shutdown_connections 2024-05-09T06:31:47.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.826+0000 7f1c9e29a640 1 -- 172.21.15.1:0/76643282 wait complete. 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.827+0000 7f1c9e29a640 1 Processor -- start 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.827+0000 7f1c9e29a640 1 -- start start 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c980cd450 0x7f1c98155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 0x7f1c980c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980cd9b0 con 0x7f1c98074660 2024-05-09T06:31:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980cd830 con 0x7f1c981605d0 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9e29a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9815dcb0 con 0x7f1c980cd450 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35122/0 (socket says 172.21.15.1:35122) 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 -- 172.21.15.1:0/598040157 learned_addr learned my addr 172.21.15.1:0/598040157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c977fe640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c980cd450 0x7f1c98155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c9c837640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 0x7f1c980c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 -- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c980cd450 msgr2=0x7f1c98155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c980cd450 0x7f1c98155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 -- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 msgr2=0x7f1c980c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 0x7f1c980c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.828+0000 7f1c97fff640 1 -- 172.21.15.1:0/598040157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c980c9790 con 0x7f1c98074660 2024-05-09T06:31:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c9c837640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 0x7f1c980c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c97fff640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802a220 tx=0x7f1c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88041020 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c880306c0 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c980c4e90 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88037040 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.829+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c980c9a70 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.830+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c980674d0 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.830+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c88044020 con 0x7f1c98074660 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.831+0000 7f1c94ff9640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 0x7f1c8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.831+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f1c880ab8d0 con 0x7f1c98074660 2024-05-09T06:31:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.831+0000 7f1c977fe640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 0x7f1c8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.831+0000 7f1c977fe640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 0x7f1c8406e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c980d0d50 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:47.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:47.835+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c980674d0 con 0x7f1c98074660 2024-05-09T06:31:47.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.018+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c980752b0 con 0x7f1c98074660 2024-05-09T06:31:47.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.018+0000 7f1c94ff9640 1 -- 172.21.15.1:0/598040157 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f1c8804b030 con 0x7f1c98074660 2024-05-09T06:31:48.000 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 msgr2=0x7f1c8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 0x7f1c8406e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c980d0d50 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 msgr2=0x7f1c980c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802a220 tx=0x7f1c880046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 shutdown_connections 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c8406be60 0x7f1c8406e320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c981605d0 0x7f1c980c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c980cd450 0x7f1c98155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 --2- 172.21.15.1:0/598040157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98074660 0x7f1c980c5970 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 >> 172.21.15.1:0/598040157 conn(0x7f1c980b1da0 msgr2=0x7f1c9815ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:48.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 shutdown_connections 2024-05-09T06:31:48.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.020+0000 7f1c9e29a640 1 -- 172.21.15.1:0/598040157 wait complete. 2024-05-09T06:31:48.013 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:31:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.136+0000 7f6f11f5a640 1 Processor -- start 2024-05-09T06:31:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.136+0000 7f6f11f5a640 1 -- start start 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f11f5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c0b2a60 0x7f6f0c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f11f5a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f0c15e500 0x7f6f0c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f11f5a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0c070f40 con 0x7f6f0c15db10 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f11f5a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c0cf170 con 0x7f6f0c15e500 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f11f5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cf450 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f0c15e500 0x7f6f0c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6f0c15e500 0x7f6f0c06e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35136/0 (socket says 172.21.15.1:35136) 2024-05-09T06:31:48.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f0bfff640 1 -- 172.21.15.1:0/4258373458 learned_addr learned my addr 172.21.15.1:0/4258373458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.137+0000 7f6f0affd640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c0b2a60 0x7f6f0c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0bfff640 1 -- 172.21.15.1:0/4258373458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c0b2a60 msgr2=0x7f6f0c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0bfff640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c0b2a60 0x7f6f0c15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0bfff640 1 -- 172.21.15.1:0/4258373458 >> v1:172.21.15.6:6789/0 conn(0x7f6f0c15db10 legacy=0x7f6f0c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0bfff640 1 -- 172.21.15.1:0/4258373458 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c070c10 con 0x7f6f0c15e500 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0bfff640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c15e500 0x7f6f0c06e340 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009e50 tx=0x7f6f0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4c2f19f4225868 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.138+0000 7f6f0a7fc640 1 -- 172.21.15.1:0/4258373458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04036070 con 0x7f6f0c15e500 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f0a7fc640 1 -- 172.21.15.1:0/4258373458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f04031600 con 0x7f6f0c15e500 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f0a7fc640 1 -- 172.21.15.1:0/4258373458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040319d0 con 0x7f6f0c15e500 2024-05-09T06:31:48.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 -- 172.21.15.1:0/4258373458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c15e500 msgr2=0x7f6f0c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c15e500 0x7f6f0c06e340 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009e50 tx=0x7f6f0402a680 comp rx=0 tx=0).stop 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 -- 172.21.15.1:0/4258373458 shutdown_connections 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c15e500 0x7f6f0c06e340 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/4258373458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c0b2a60 0x7f6f0c15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 -- 172.21.15.1:0/4258373458 >> 172.21.15.1:0/4258373458 conn(0x7f6f0c0cc4b0 msgr2=0x7f6f0c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 -- 172.21.15.1:0/4258373458 shutdown_connections 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.139+0000 7f6f11f5a640 1 -- 172.21.15.1:0/4258373458 wait complete. 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.140+0000 7f6f11f5a640 1 Processor -- start 2024-05-09T06:31:48.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.140+0000 7f6f11f5a640 1 -- start start 2024-05-09T06:31:48.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c15db10 0x7f6f0c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 0x7f6f0c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c06cd30 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c15ee50 con 0x7f6f0c15e500 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f11f5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c06ceb0 con 0x7f6f0c15db10 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35152/0 (socket says 172.21.15.1:35152) 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.141+0000 7f6f0b7fe640 1 -- 172.21.15.1:0/3632691780 learned_addr learned my addr 172.21.15.1:0/3632691780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0affd640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c15db10 0x7f6f0c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0bfff640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 0x7f6f0c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 -- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c15db10 msgr2=0x7f6f0c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c15db10 0x7f6f0c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 -- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 msgr2=0x7f6f0c14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 0x7f6f0c14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 -- 172.21.15.1:0/3632691780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c14e670 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0bfff640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 0x7f6f0c14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f0b7fe640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c400 tx=0x7f6efc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0052d0 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc005490 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc007260 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c14e950 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.142+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c14f1f0 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.143+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c14eb20 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.144+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6efc007400 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.144+0000 7f6ef3fff640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 0x7f6ef806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.144+0000 7f6f0affd640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 0x7f6ef806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.144+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6efc08a5b0 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.145+0000 7f6f0affd640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 0x7f6ef806e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c153060 tx=0x7f6f00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.149+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6efc058d60 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.134 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:31:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.347+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f0c0752b0 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.347+0000 7f6ef3fff640 1 -- 172.21.15.1:0/3632691780 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7f6efc058660 con 0x7f6f0c0b2a60 2024-05-09T06:31:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 msgr2=0x7f6ef806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 0x7f6ef806e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c153060 tx=0x7f6f00008040 comp rx=0 tx=0).stop 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 msgr2=0x7f6f0c14bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c400 tx=0x7f6efc00c8c0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 shutdown_connections 2024-05-09T06:31:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ef806be30 0x7f6ef806e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f0c15e500 0x7f6f0c14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f0c15db10 0x7f6f0c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 --2- 172.21.15.1:0/3632691780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c0b2a60 0x7f6f0c14bcf0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 >> 172.21.15.1:0/3632691780 conn(0x7f6f0c0cc4b0 msgr2=0x7f6f0c0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 shutdown_connections 2024-05-09T06:31:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.349+0000 7f6f11f5a640 1 -- 172.21.15.1:0/3632691780 wait complete. 2024-05-09T06:31:48.342 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:31:48.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:48.342 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:31:48.366 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[58951]: fuse finished with error 0 and tester_r 0 2024-05-09T06:31:48.368 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:31:48.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:48.368 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:31:48.491 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:31:48.492 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:31:48.492 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:48.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:31:48.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:31:48.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:48.511 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:31:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:31:48.569 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:31:48.569 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:31:48.569 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:31:48.570 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:48.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:31:48.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:31:48.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:48.587 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:31:48.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:31:48.642 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:31:48.642 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:31:48.643 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:31:48.643 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:48.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:31:48.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:31:48.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:31:48.662 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:31:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:31:48.720 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:31:48.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.842+0000 7ff65786b640 1 Processor -- start 2024-05-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.843+0000 7ff65786b640 1 -- start start 2024-05-09T06:31:48.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.843+0000 7ff65786b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff650161950 0x7ff6500d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff65786b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 0x7ff6500d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff65786b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6500b33c0 con 0x7ff6500d4940 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff65786b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff6500d4dd0 con 0x7ff650161950 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff65786b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff6500b3660 con 0x7ff6500d0f80 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff650161950 0x7ff6500d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff650161950 0x7ff6500d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35168/0 (socket says 172.21.15.1:35168) 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff654e06640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff6500d4940 0x7ff6500cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55308/0 (socket says 172.21.15.1:55308) 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 -- 172.21.15.1:0/2187030794 learned_addr learned my addr 172.21.15.1:0/2187030794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 -- 172.21.15.1:0/2187030794 >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 msgr2=0x7ff6500d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655e08640 1 --2- 172.21.15.1:0/2187030794 >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 0x7ff6500d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 --2- 172.21.15.1:0/2187030794 >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 0x7ff6500d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff647fff640 1 -- 172.21.15.1:0/2187030794 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696573096 0 0) 0x7ff6500b33c0 con 0x7ff6500d4940 2024-05-09T06:31:48.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 -- 172.21.15.1:0/2187030794 >> v1:172.21.15.1:6789/0 conn(0x7ff6500d4940 legacy=0x7ff6500cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655607640 1 -- 172.21.15.1:0/2187030794 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6500b3940 con 0x7ff650161950 2024-05-09T06:31:48.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.844+0000 7ff655e08640 1 --2- 172.21.15.1:0/2187030794 >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 0x7ff6500d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff655607640 1 --2- 172.21.15.1:0/2187030794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500d2400 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff65008e540 tx=0x7ff64c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bdabbcb9b926c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff647fff640 1 -- 172.21.15.1:0/2187030794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c037070 con 0x7ff650161950 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff647fff640 1 -- 172.21.15.1:0/2187030794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64c032440 con 0x7ff650161950 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff647fff640 1 -- 172.21.15.1:0/2187030794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c032810 con 0x7ff650161950 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 -- 172.21.15.1:0/2187030794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 msgr2=0x7ff6500d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 --2- 172.21.15.1:0/2187030794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500d2400 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff65008e540 tx=0x7ff64c02a270 comp rx=0 tx=0).stop 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 -- 172.21.15.1:0/2187030794 shutdown_connections 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 --2- 172.21.15.1:0/2187030794 >> v2:172.21.15.6:3300/0 conn(0x7ff6500d0f80 0x7ff6500d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 --2- 172.21.15.1:0/2187030794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500d2400 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.845+0000 7ff65786b640 1 -- 172.21.15.1:0/2187030794 >> 172.21.15.1:0/2187030794 conn(0x7ff6500d3e20 msgr2=0x7ff6500d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.846+0000 7ff65786b640 1 -- 172.21.15.1:0/2187030794 shutdown_connections 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.846+0000 7ff65786b640 1 -- 172.21.15.1:0/2187030794 wait complete. 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 Processor -- start 2024-05-09T06:31:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 -- start start 2024-05-09T06:31:48.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6500d0f80 0x7ff6500caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6500cb430 0x7ff6500cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6500b52d0 con 0x7ff650161950 2024-05-09T06:31:48.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff6500b5150 con 0x7ff6500d0f80 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.847+0000 7ff65786b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff6500b5450 con 0x7ff6500cb430 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35182/0 (socket says 172.21.15.1:35182) 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 -- 172.21.15.1:0/943112120 learned_addr learned my addr 172.21.15.1:0/943112120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff655607640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6500d0f80 0x7ff6500caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff655e08640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6500cb430 0x7ff6500cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 -- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6500cb430 msgr2=0x7ff6500cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6500cb430 0x7ff6500cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 -- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6500d0f80 msgr2=0x7ff6500caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6500d0f80 0x7ff6500caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 -- 172.21.15.1:0/943112120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6500cb8d0 con 0x7ff650161950 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff654e06640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c4b0 tx=0x7ff64000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640018070 con 0x7ff650161950 2024-05-09T06:31:48.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff640007480 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6500cbbb0 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.848+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640004820 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.849+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6500cc170 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.850+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6500752b0 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.850+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff64001b070 con 0x7ff650161950 2024-05-09T06:31:48.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.850+0000 7ff645ffb640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 0x7ff63406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:48.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.851+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7ff64008a410 con 0x7ff650161950 2024-05-09T06:31:48.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.851+0000 7ff655607640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 0x7ff63406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:48.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.851+0000 7ff655607640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 0x7ff63406e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff64c034000 tx=0x7ff64c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:48.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:48.855+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff640058be0 con 0x7ff650161950 2024-05-09T06:31:49.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.081+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6500d21e0 con 0x7ff650161950 2024-05-09T06:31:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.081+0000 7ff645ffb640 1 -- 172.21.15.1:0/943112120 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5168 (secure 0 0 0) 0x7ff6400584e0 con 0x7ff650161950 2024-05-09T06:31:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:49.063 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:31:15.612251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 37 2024-05-09T06:31:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 msgr2=0x7ff63406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 0x7ff63406e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff64c034000 tx=0x7ff64c02e000 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 msgr2=0x7ff6500cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c4b0 tx=0x7ff64000c970 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 shutdown_connections 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff63406be30 0x7ff63406e2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6500cb430 0x7ff6500cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff650161950 0x7ff6500cc970 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 --2- 172.21.15.1:0/943112120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6500d0f80 0x7ff6500caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 >> 172.21.15.1:0/943112120 conn(0x7ff6500d3e20 msgr2=0x7ff6500d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 shutdown_connections 2024-05-09T06:31:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.084+0000 7ff65786b640 1 -- 172.21.15.1:0/943112120 wait complete. 2024-05-09T06:31:49.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:31:49.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.199+0000 7f7b9488c640 1 Processor -- start 2024-05-09T06:31:49.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.199+0000 7f7b9488c640 1 -- start start 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b9488c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b90071ce0 0x7f7b900cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b9488c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b900d04a0 con 0x7f7b9015ece0 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b9488c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b900722c0 con 0x7f7b900b0080 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b9488c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7b900725a0 con 0x7f7b90071ce0 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8ed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7b9015ece0 0x7f7b9015f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55318/0 (socket says 172.21.15.1:55318) 2024-05-09T06:31:49.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8ed9d640 1 -- 172.21.15.1:0/935365226 learned_addr learned my addr 172.21.15.1:0/935365226 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040619420 0 0) 0x7f7b900d04a0 con 0x7f7b9015ece0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/935365226 >> v2:172.21.15.6:3300/0 conn(0x7f7b90071ce0 0x7f7b900cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b78003520 con 0x7f7b9015ece0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1451881860 0 0) 0x7f7b78003520 con 0x7f7b9015ece0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 >> v2:172.21.15.6:3300/0 conn(0x7f7b90071ce0 msgr2=0x7f7b900cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 --2- 172.21.15.1:0/935365226 >> v2:172.21.15.6:3300/0 conn(0x7f7b90071ce0 0x7f7b900cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 >> v1:172.21.15.6:6789/0 conn(0x7f7b900b0080 legacy=0x7f7b9015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b90072880 con 0x7f7b9015ece0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f7b880019f0 con 0x7f7b9015ece0 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.200+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 >> v1:172.21.15.1:6789/0 conn(0x7f7b9015ece0 legacy=0x7f7b9015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b780039b0 0x7f7b78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b780051c0 0x7f7b7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900725a0 con 0x7f7b78004800 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7800c060 con 0x7f7b780051c0 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7800c260 con 0x7f7b780039b0 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8e59c640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b780039b0 0x7f7b78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8ed9d640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b780051c0 0x7f7b7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b780039b0 msgr2=0x7f7b78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b780039b0 0x7f7b78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b780051c0 msgr2=0x7f7b7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b780051c0 0x7f7b7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/935365226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b78003520 con 0x7f7b78004800 2024-05-09T06:31:49.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8ed9d640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b780051c0 0x7f7b7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.201+0000 7f7b8ed9d640 1 -- 172.21.15.1:0/935365226 reap_dead start 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009ac0 tx=0x7f7b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8680b360bd2a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84036070 con 0x7f7b78004800 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84031400 con 0x7f7b78004800 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b8d59a640 1 -- 172.21.15.1:0/935365226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84009690 con 0x7f7b78004800 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b9488c640 1 -- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 msgr2=0x7f7b78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b9488c640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009ac0 tx=0x7f7b78005830 comp rx=0 tx=0).stop 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b9488c640 1 -- 172.21.15.1:0/935365226 shutdown_connections 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b9488c640 1 --2- 172.21.15.1:0/935365226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.202+0000 7f7b9488c640 1 -- 172.21.15.1:0/935365226 >> 172.21.15.1:0/935365226 conn(0x7f7b900d0b50 msgr2=0x7f7b900d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.203+0000 7f7b9488c640 1 -- 172.21.15.1:0/935365226 shutdown_connections 2024-05-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.203+0000 7f7b9488c640 1 -- 172.21.15.1:0/935365226 wait complete. 2024-05-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.204+0000 7f7b9488c640 1 Processor -- start 2024-05-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.204+0000 7f7b9488c640 1 -- start start 2024-05-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b78004800 0x7f7b9015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b900c4960 0x7f7b900c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900cbb60 con 0x7f7b9015cc30 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9015e7c0 con 0x7f7b78004800 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b9488c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b900cbce0 con 0x7f7b900c4960 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35200/0 (socket says 172.21.15.1:35200) 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/2924646921 learned_addr learned my addr 172.21.15.1:0/2924646921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b900c4960 msgr2=0x7f7b900c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b900c4960 0x7f7b900c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8e59c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b78004800 0x7f7b9015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b78004800 msgr2=0x7f7b9015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b78004800 0x7f7b9015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 -- 172.21.15.1:0/2924646921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9015d4a0 con 0x7f7b9015cc30 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.205+0000 7f7b8dd9b640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7b8402abb0 tx=0x7f7b84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.206+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8403f020 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.206+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84037040 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.206+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84036070 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.206+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b900b4060 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.206+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b900b4620 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.207+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b900752b0 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.207+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b84042020 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.208+0000 7f7b7effd640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 0x7f7b7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.208+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f7b840ab3f0 con 0x7f7b9015cc30 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.208+0000 7f7b8e59c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 0x7f7b7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.208+0000 7f7b8e59c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 0x7f7b7006e320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005590 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.212+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b84049020 con 0x7f7b9015cc30 2024-05-09T06:31:49.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.396+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b90072f90 con 0x7f7b9015cc30 2024-05-09T06:31:49.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.397+0000 7f7b7effd640 1 -- 172.21.15.1:0/2924646921 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16944 (secure 0 0 0) 0x7f7b84079bd0 con 0x7f7b9015cc30 2024-05-09T06:31:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:49.379 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":66,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:31:03.690223+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:31:49.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 msgr2=0x7f7b7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 0x7f7b7006e320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005590 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2024-05-09T06:31:49.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 msgr2=0x7f7b900b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7b8402abb0 tx=0x7f7b84004730 comp rx=0 tx=0).stop 2024-05-09T06:31:49.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 shutdown_connections 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b7006be60 0x7f7b7006e320 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b900c4960 0x7f7b900c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9015cc30 0x7f7b900b5100 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 --2- 172.21.15.1:0/2924646921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b78004800 0x7f7b9015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 >> 172.21.15.1:0/2924646921 conn(0x7f7b900d0b50 msgr2=0x7f7b900adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 shutdown_connections 2024-05-09T06:31:49.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.399+0000 7f7b9488c640 1 -- 172.21.15.1:0/2924646921 wait complete. 2024-05-09T06:31:49.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:30:33.248885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:30:33.918243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:31:49.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:31:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.517+0000 7fd616170640 1 Processor -- start 2024-05-09T06:31:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.517+0000 7fd616170640 1 -- start start 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd616170640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd6100c41e0 0x7fd6100ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd616170640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd616170640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd610072050 con 0x7fd6100d06f0 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd616170640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd610072330 con 0x7fd6100c41e0 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd616170640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd610072610 con 0x7fd61006b1e0 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd60effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd6100c41e0 0x7fd6100ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd60effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd6100c41e0 0x7fd6100ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35214/0 (socket says 172.21.15.1:35214) 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd60effd640 1 -- 172.21.15.1:0/3999462351 learned_addr learned my addr 172.21.15.1:0/3999462351 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.518+0000 7fd60ffff640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.519+0000 7fd60effd640 1 -- 172.21.15.1:0/3999462351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 msgr2=0x7fd6100cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.519+0000 7fd60effd640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.519+0000 7fd60effd640 1 -- 172.21.15.1:0/3999462351 >> v1:172.21.15.6:6789/0 conn(0x7fd6100d06f0 legacy=0x7fd6100cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.519+0000 7fd60effd640 1 -- 172.21.15.1:0/3999462351 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6100728f0 con 0x7fd6100c41e0 2024-05-09T06:31:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.519+0000 7fd60effd640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd6100ce8d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd604009870 tx=0x7fd60402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279622b4c5291194 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd60e7fc640 1 -- 172.21.15.1:0/3999462351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604037070 con 0x7fd6100c41e0 2024-05-09T06:31:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd60e7fc640 1 -- 172.21.15.1:0/3999462351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604032440 con 0x7fd6100c41e0 2024-05-09T06:31:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd60e7fc640 1 -- 172.21.15.1:0/3999462351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604032810 con 0x7fd6100c41e0 2024-05-09T06:31:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 -- 172.21.15.1:0/3999462351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 msgr2=0x7fd6100ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd6100ce8d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd604009870 tx=0x7fd60402a270 comp rx=0 tx=0).stop 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 -- 172.21.15.1:0/3999462351 shutdown_connections 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 --2- 172.21.15.1:0/3999462351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd6100ce8d0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 -- 172.21.15.1:0/3999462351 >> 172.21.15.1:0/3999462351 conn(0x7fd6100d0b50 msgr2=0x7fd6100d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 -- 172.21.15.1:0/3999462351 shutdown_connections 2024-05-09T06:31:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.520+0000 7fd616170640 1 -- 172.21.15.1:0/3999462351 wait complete. 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.521+0000 7fd616170640 1 Processor -- start 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 -- start start 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6100d06f0 0x7fd61015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6100ccf70 con 0x7fd6100c41e0 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd6100ccdf0 con 0x7fd6100d06f0 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd616170640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd6100cd0f0 con 0x7fd61006b1e0 2024-05-09T06:31:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd60effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd60effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35218/0 (socket says 172.21.15.1:35218) 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.522+0000 7fd60effd640 1 -- 172.21.15.1:0/573369985 learned_addr learned my addr 172.21.15.1:0/573369985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60f7fe640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60ffff640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6100d06f0 0x7fd61015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 -- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 msgr2=0x7fd6100afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 -- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6100d06f0 msgr2=0x7fd61015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6100d06f0 0x7fd61015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 -- 172.21.15.1:0/573369985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6101f7580 con 0x7fd6100c41e0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60f7fe640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd60effd640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd6040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604041020 con 0x7fd6100c41e0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6040306c0 con 0x7fd6100c41e0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6101f7860 con 0x7fd6100c41e0 2024-05-09T06:31:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604037040 con 0x7fd6100c41e0 2024-05-09T06:31:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.523+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101f7e20 con 0x7fd6100c41e0 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.524+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6100752b0 con 0x7fd6100c41e0 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.525+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd604044020 con 0x7fd6100c41e0 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.525+0000 7fd5f7fff640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 0x7fd5fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.525+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7fd6040ab8a0 con 0x7fd6100c41e0 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.525+0000 7fd60f7fe640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 0x7fd5fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.526+0000 7fd60f7fe640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 0x7fd5fc06e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd6100cb390 tx=0x7fd600009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.530+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd60404b030 con 0x7fd6100c41e0 2024-05-09T06:31:49.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.712+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6100c9d20 con 0x7fd6100c41e0 2024-05-09T06:31:49.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.712+0000 7fd5f7fff640 1 -- 172.21.15.1:0/573369985 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7fd60407a050 con 0x7fd6100c41e0 2024-05-09T06:31:49.694 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 msgr2=0x7fd5fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 0x7fd5fc06e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd6100cb390 tx=0x7fd600009040 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 msgr2=0x7fd61015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd6040046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 shutdown_connections 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd5fc06be30 0x7fd5fc06e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6100d06f0 0x7fd61015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6100c41e0 0x7fd61015b160 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 --2- 172.21.15.1:0/573369985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd61006b1e0 0x7fd6100afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 >> 172.21.15.1:0/573369985 conn(0x7fd6100d0b50 msgr2=0x7fd61006f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 shutdown_connections 2024-05-09T06:31:49.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.714+0000 7fd616170640 1 -- 172.21.15.1:0/573369985 wait complete. 2024-05-09T06:31:49.707 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:31:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.833+0000 7f071c0d4640 1 Processor -- start 2024-05-09T06:31:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.833+0000 7f071c0d4640 1 -- start start 2024-05-09T06:31:49.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071c0d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071c0d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f07140b4de0 0x7f071415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071c0d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f071406cf40 con 0x7f07140b5260 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071c0d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0714156980 con 0x7f07140b4de0 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071c0d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f071406d870 con 0x7f071415c4c0 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f07140b4de0 0x7f071415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f07140b4de0 0x7f071415c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35230/0 (socket says 172.21.15.1:35230) 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 -- 172.21.15.1:0/4125583083 learned_addr learned my addr 172.21.15.1:0/4125583083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 -- 172.21.15.1:0/4125583083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 msgr2=0x7f071406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 --2- 172.21.15.1:0/4125583083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 -- 172.21.15.1:0/4125583083 >> v1:172.21.15.6:6789/0 conn(0x7f07140b5260 legacy=0x7f0714070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.834+0000 7f071966f640 1 -- 172.21.15.1:0/4125583083 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071406d220 con 0x7f07140b4de0 2024-05-09T06:31:49.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.835+0000 7f071966f640 1 --2- 172.21.15.1:0/4125583083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f071415c8a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0704009870 tx=0x7f070402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2828b0a1ee26a152 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.835+0000 7f0718e6e640 1 -- 172.21.15.1:0/4125583083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704037070 con 0x7f07140b4de0 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.835+0000 7f0718e6e640 1 -- 172.21.15.1:0/4125583083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0704032440 con 0x7f07140b4de0 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.835+0000 7f0718e6e640 1 -- 172.21.15.1:0/4125583083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704032810 con 0x7f07140b4de0 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 -- 172.21.15.1:0/4125583083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 msgr2=0x7f071415c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 --2- 172.21.15.1:0/4125583083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f071415c8a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0704009870 tx=0x7f070402a210 comp rx=0 tx=0).stop 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 -- 172.21.15.1:0/4125583083 shutdown_connections 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 --2- 172.21.15.1:0/4125583083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f071415c8a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 --2- 172.21.15.1:0/4125583083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 -- 172.21.15.1:0/4125583083 >> 172.21.15.1:0/4125583083 conn(0x7f07140cc4d0 msgr2=0x7f07140cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 -- 172.21.15.1:0/4125583083 shutdown_connections 2024-05-09T06:31:49.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.836+0000 7f071c0d4640 1 -- 172.21.15.1:0/4125583083 wait complete. 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.837+0000 7f071c0d4640 1 Processor -- start 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.837+0000 7f071c0d4640 1 -- start start 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 0x7f0714155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f07140b2dc0 con 0x7f07140b4de0 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f07140b2c40 con 0x7f07140b5260 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071c0d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f071415cde0 con 0x7f071415c4c0 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35236/0 (socket says 172.21.15.1:35236) 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 -- 172.21.15.1:0/775101936 learned_addr learned my addr 172.21.15.1:0/775101936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071a671640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071966f640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 0x7f0714155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 -- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 msgr2=0x7f071415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 -- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 msgr2=0x7f0714155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 0x7f0714155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 -- 172.21.15.1:0/775101936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07140a9520 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f071966f640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 0x7f0714155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f0719e70640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f071000b410 tx=0x7f071000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710015020 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.838+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710011850 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.839+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07140a9800 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.839+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710011c40 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.839+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07140a9dc0 con 0x7f07140b4de0 2024-05-09T06:31:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.840+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07140752b0 con 0x7f07140b4de0 2024-05-09T06:31:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.840+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0710018020 con 0x7f07140b4de0 2024-05-09T06:31:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.841+0000 7f070a7fc640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 0x7f06f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.841+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f071008b000 con 0x7f07140b4de0 2024-05-09T06:31:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.841+0000 7f071966f640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 0x7f06f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:49.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.841+0000 7f071966f640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 0x7f06f806e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f070402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:49.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:49.845+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f071001f020 con 0x7f07140b4de0 2024-05-09T06:31:50.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.026+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f071415e170 con 0x7f07140b4de0 2024-05-09T06:31:50.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.026+0000 7f070a7fc640 1 -- 172.21.15.1:0/775101936 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f07100172c0 con 0x7f07140b4de0 2024-05-09T06:31:50.008 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 msgr2=0x7f06f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 0x7f06f806e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f070402e000 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 msgr2=0x7f07140aa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f071000b410 tx=0x7f071000b8d0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 shutdown_connections 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06f806be30 0x7f06f806e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f071415c4c0 0x7f071415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07140b5260 0x7f0714155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 --2- 172.21.15.1:0/775101936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07140b4de0 0x7f07140aa5c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 >> 172.21.15.1:0/775101936 conn(0x7f07140cc4d0 msgr2=0x7f07140cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 shutdown_connections 2024-05-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.029+0000 7f071c0d4640 1 -- 172.21.15.1:0/775101936 wait complete. 2024-05-09T06:31:50.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:31:50.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.146+0000 7f818dab3640 1 Processor -- start 2024-05-09T06:31:50.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.146+0000 7f818dab3640 1 -- start start 2024-05-09T06:31:50.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.147+0000 7f818dab3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f81880cd450 0x7f818815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.147+0000 7f818dab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.147+0000 7f818dab3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f818815c920 con 0x7f81881605d0 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.147+0000 7f818dab3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f818815fc00 con 0x7f81880cd450 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.147+0000 7f818dab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f818815cba0 con 0x7f8188074660 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81877fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f81880cd450 0x7f818815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f8187fff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f81881605d0 0x7f818815d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:56022/0 (socket says 172.21.15.1:56022) 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:57536/0 (socket says 172.21.15.1:57536) 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f8187fff640 1 -- 172.21.15.1:0/261993634 learned_addr learned my addr 172.21.15.1:0/261993634 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81877fe640 1 -- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 msgr2=0x7f8188160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81877fe640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81877fe640 1 -- 172.21.15.1:0/261993634 >> v1:172.21.15.6:6789/0 conn(0x7f81881605d0 legacy=0x7f818815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81877fe640 1 -- 172.21.15.1:0/261993634 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818815ce80 con 0x7f81880cd450 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f8186ffd640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:50.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.148+0000 7f81867fc640 1 -- 172.21.15.1:0/261993634 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241972225 0 0) 0x7f8180001490 con 0x7f81881605d0 2024-05-09T06:31:50.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.149+0000 7f81877fe640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81880cd450 0x7f818815c3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81780097a0 tx=0x7f817802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b5d39dd9db5e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.149+0000 7f81867fc640 1 -- 172.21.15.1:0/261993634 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178037070 con 0x7f81880cd450 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.149+0000 7f81867fc640 1 -- 172.21.15.1:0/261993634 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8178032440 con 0x7f81880cd450 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.149+0000 7f81867fc640 1 -- 172.21.15.1:0/261993634 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178032810 con 0x7f81880cd450 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 -- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81880cd450 msgr2=0x7f818815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81880cd450 0x7f818815c3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f81780097a0 tx=0x7f817802a180 comp rx=0 tx=0).stop 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 -- 172.21.15.1:0/261993634 shutdown_connections 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8188074660 0x7f8188160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 --2- 172.21.15.1:0/261993634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81880cd450 0x7f818815c3e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 -- 172.21.15.1:0/261993634 >> 172.21.15.1:0/261993634 conn(0x7f81880b1da0 msgr2=0x7f81880b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 -- 172.21.15.1:0/261993634 shutdown_connections 2024-05-09T06:31:50.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.150+0000 7f818dab3640 1 -- 172.21.15.1:0/261993634 wait complete. 2024-05-09T06:31:50.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.151+0000 7f818dab3640 1 Processor -- start 2024-05-09T06:31:50.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.151+0000 7f818dab3640 1 -- start start 2024-05-09T06:31:50.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8188074660 0x7f81880c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81881605d0 0x7f8188158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f818815dcb0 con 0x7f81880cd450 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f81880cd830 con 0x7f8188074660 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f818dab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f818815de30 con 0x7f81881605d0 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35246/0 (socket says 172.21.15.1:35246) 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 -- 172.21.15.1:0/2900366803 learned_addr learned my addr 172.21.15.1:0/2900366803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8187fff640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81881605d0 0x7f8188158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f81877fe640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8188074660 0x7f81880c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 -- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81881605d0 msgr2=0x7f8188158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81881605d0 0x7f8188158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 -- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8188074660 msgr2=0x7f81880c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8188074660 0x7f81880c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 -- 172.21.15.1:0/2900366803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81880c6f30 con 0x7f81880cd450 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.152+0000 7f8186ffd640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f817c012fc0 tx=0x7f817c010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.153+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c010da0 con 0x7f81880cd450 2024-05-09T06:31:50.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.153+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f817c01d070 con 0x7f81880cd450 2024-05-09T06:31:50.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.153+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81881594c0 con 0x7f81880cd450 2024-05-09T06:31:50.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.153+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0186a0 con 0x7f81880cd450 2024-05-09T06:31:50.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.153+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81880c6b50 con 0x7f81880cd450 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.154+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81880752b0 con 0x7f81880cd450 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.154+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f817c016050 con 0x7f81880cd450 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.155+0000 7f816ffff640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 0x7f817406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.155+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f817c08ef60 con 0x7f81880cd450 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.155+0000 7f81877fe640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 0x7f817406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.155+0000 7f81877fe640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 0x7f817406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8178034000 tx=0x7f817802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.159+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f817c026020 con 0x7f81880cd450 2024-05-09T06:31:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.340+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f818806e140 con 0x7f81880cd450 2024-05-09T06:31:50.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.340+0000 7f816ffff640 1 -- 172.21.15.1:0/2900366803 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f817c014040 con 0x7f81880cd450 2024-05-09T06:31:50.322 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:31:50.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 msgr2=0x7f817406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 0x7f817406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8178034000 tx=0x7f817802e000 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 msgr2=0x7f81880c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f817c012fc0 tx=0x7f817c010490 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 shutdown_connections 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f817406be30 0x7f817406e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81881605d0 0x7f8188158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81880cd450 0x7f81880c9a50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 --2- 172.21.15.1:0/2900366803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8188074660 0x7f81880c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 >> 172.21.15.1:0/2900366803 conn(0x7f81880b1da0 msgr2=0x7f818815eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 shutdown_connections 2024-05-09T06:31:50.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.343+0000 7f818dab3640 1 -- 172.21.15.1:0/2900366803 wait complete. 2024-05-09T06:31:50.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:50.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.459+0000 7f6cf8f3b640 1 Processor -- start 2024-05-09T06:31:50.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.459+0000 7f6cf8f3b640 1 -- start start 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf8f3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf40b2a60 0x7f6cf40b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf8f3b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6cf40b4f70 0x7f6cf4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf8f3b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf406eb40 con 0x7f6cf41454b0 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf8f3b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf406ee20 con 0x7f6cf40b4f70 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf8f3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf406d8e0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf1d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf40b2a60 0x7f6cf40b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf2d9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6cf40b4f70 0x7f6cf4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf259c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6cf41454b0 0x7f6cf40b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:56032/0 (socket says 172.21.15.1:56032) 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.460+0000 7f6cf259c640 1 -- 172.21.15.1:0/3629565270 learned_addr learned my addr 172.21.15.1:0/3629565270 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.461+0000 7f6cf2d9d640 1 -- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf40b2a60 msgr2=0x7f6cf40b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.461+0000 7f6cf2d9d640 1 --2- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf40b2a60 0x7f6cf40b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.461+0000 7f6cf2d9d640 1 -- 172.21.15.1:0/3629565270 >> v1:172.21.15.6:6789/0 conn(0x7f6cf41454b0 legacy=0x7f6cf40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.461+0000 7f6cf2d9d640 1 -- 172.21.15.1:0/3629565270 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf406dbc0 con 0x7f6cf40b4f70 2024-05-09T06:31:50.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.461+0000 7f6cf159a640 1 -- 172.21.15.1:0/3629565270 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691956808 0 0) 0x7f6cf406eb40 con 0x7f6cf41454b0 2024-05-09T06:31:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf2d9d640 1 --2- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf4070c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40097a0 tx=0x7f6ce402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994d30a95039364a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf159a640 1 -- 172.21.15.1:0/3629565270 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4037070 con 0x7f6cf40b4f70 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf159a640 1 -- 172.21.15.1:0/3629565270 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4032440 con 0x7f6cf40b4f70 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf159a640 1 -- 172.21.15.1:0/3629565270 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4032810 con 0x7f6cf40b4f70 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 msgr2=0x7f6cf4070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf4070c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40097a0 tx=0x7f6ce402a180 comp rx=0 tx=0).stop 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/3629565270 shutdown_connections 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf4070c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/3629565270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf40b2a60 0x7f6cf40b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/3629565270 >> 172.21.15.1:0/3629565270 conn(0x7f6cf40cc4b0 msgr2=0x7f6cf40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/3629565270 shutdown_connections 2024-05-09T06:31:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.462+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/3629565270 wait complete. 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 Processor -- start 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 -- start start 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf41454b0 0x7f6cf40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf406e260 con 0x7f6cf40b2a60 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf406e0e0 con 0x7f6cf40b4f70 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.464+0000 7f6cf8f3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf4071150 con 0x7f6cf41454b0 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35260/0 (socket says 172.21.15.1:35260) 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 -- 172.21.15.1:0/2049975492 learned_addr learned my addr 172.21.15.1:0/2049975492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf1d9b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf2d9d640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf41454b0 0x7f6cf40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 -- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf41454b0 msgr2=0x7f6cf40c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf41454b0 0x7f6cf40c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 -- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 msgr2=0x7f6cf40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf40ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 -- 172.21.15.1:0/2049975492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf413b4e0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf1d9b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf40ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf259c640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c7c0 tx=0x7f6cec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec018070 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cec00a640 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf4135a20 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00aa30 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.465+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf4135fe0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.466+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40752b0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.467+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cec01b070 con 0x7f6cf40b2a60 2024-05-09T06:31:50.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.467+0000 7f6ceaffd640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 0x7f6cd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.468+0000 7f6cf1d9b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 0x7f6cd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.468+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f6cec08a5e0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.468+0000 7f6cf1d9b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 0x7f6cd406e320 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6cf41442d0 tx=0x7f6cdc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.472+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6cec058740 con 0x7f6cf40b2a60 2024-05-09T06:31:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.695+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cf4072fa0 con 0x7f6cf40b2a60 2024-05-09T06:31:50.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.695+0000 7f6ceaffd640 1 -- 172.21.15.1:0/2049975492 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5168 (secure 0 0 0) 0x7f6cec05ca40 con 0x7f6cf40b2a60 2024-05-09T06:31:50.677 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:50.677 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:31:15.612251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:50.677 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 37 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 msgr2=0x7f6cd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 0x7f6cd406e320 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6cf41442d0 tx=0x7f6cdc009190 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 msgr2=0x7f6cf413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c7c0 tx=0x7f6cec00cc80 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 shutdown_connections 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be60 0x7f6cd406e320 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6cf41454b0 0x7f6cf40c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf40b4f70 0x7f6cf40ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 --2- 172.21.15.1:0/2049975492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cf40b2a60 0x7f6cf413afa0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 >> 172.21.15.1:0/2049975492 conn(0x7f6cf40cc4b0 msgr2=0x7f6cf4143a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 shutdown_connections 2024-05-09T06:31:50.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.697+0000 7f6cf8f3b640 1 -- 172.21.15.1:0/2049975492 wait complete. 2024-05-09T06:31:50.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:31:50.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.813+0000 7f748001c640 1 Processor -- start 2024-05-09T06:31:50.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.813+0000 7f748001c640 1 -- start start 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f748001c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74780b2a60 0x7f747815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f748001c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f747815e500 0x7f747806e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f748001c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7478070f40 con 0x7f747815db10 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f748001c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f747815e950 con 0x7f74780b2a60 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f748001c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f74780cf170 con 0x7f747815e500 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74780b2a60 0x7f747815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f74780b2a60 0x7f747815dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35272/0 (socket says 172.21.15.1:35272) 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 -- 172.21.15.1:0/1545592119 learned_addr learned my addr 172.21.15.1:0/1545592119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747e5b9640 1 --2- 172.21.15.1:0/1545592119 >> v2:172.21.15.6:3300/0 conn(0x7f747815e500 0x7f747806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 -- 172.21.15.1:0/1545592119 >> v2:172.21.15.6:3300/0 conn(0x7f747815e500 msgr2=0x7f747806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 --2- 172.21.15.1:0/1545592119 >> v2:172.21.15.6:3300/0 conn(0x7f747815e500 0x7f747806e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 -- 172.21.15.1:0/1545592119 >> v1:172.21.15.6:6789/0 conn(0x7f747815db10 legacy=0x7f74780715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.814+0000 7f747d5b7640 1 -- 172.21.15.1:0/1545592119 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74780cf450 con 0x7f74780b2a60 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.815+0000 7f747d5b7640 1 --2- 172.21.15.1:0/1545592119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74780b2a60 0x7f747815dfc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f74780b2770 tx=0x7f746c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2586b08445433bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.815+0000 7f747cdb6640 1 -- 172.21.15.1:0/1545592119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c033640 con 0x7f74780b2a60 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.815+0000 7f747cdb6640 1 -- 172.21.15.1:0/1545592119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746c038070 con 0x7f74780b2a60 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.815+0000 7f747cdb6640 1 -- 172.21.15.1:0/1545592119 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c033a00 con 0x7f74780b2a60 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 -- 172.21.15.1:0/1545592119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74780b2a60 msgr2=0x7f747815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 --2- 172.21.15.1:0/1545592119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74780b2a60 0x7f747815dfc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f74780b2770 tx=0x7f746c02c8b0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 -- 172.21.15.1:0/1545592119 shutdown_connections 2024-05-09T06:31:50.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 --2- 172.21.15.1:0/1545592119 >> v2:172.21.15.6:3300/0 conn(0x7f747815e500 0x7f747806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 --2- 172.21.15.1:0/1545592119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74780b2a60 0x7f747815dfc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 -- 172.21.15.1:0/1545592119 >> 172.21.15.1:0/1545592119 conn(0x7f74780cc4b0 msgr2=0x7f74780cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 -- 172.21.15.1:0/1545592119 shutdown_connections 2024-05-09T06:31:50.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.816+0000 7f748001c640 1 -- 172.21.15.1:0/1545592119 wait complete. 2024-05-09T06:31:50.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.817+0000 7f748001c640 1 Processor -- start 2024-05-09T06:31:50.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.817+0000 7f748001c640 1 -- start start 2024-05-09T06:31:50.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74780b2a60 0x7f74780b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 0x7f74781f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f747815edc0 con 0x7f747815e500 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f747815ec40 con 0x7f74780b2a60 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f748001c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f747806cd30 con 0x7f747815db10 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35284/0 (socket says 172.21.15.1:35284) 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 -- 172.21.15.1:0/3800749460 learned_addr learned my addr 172.21.15.1:0/3800749460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747ddb8640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74780b2a60 0x7f74780b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747d5b7640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 0x7f74781f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 -- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 msgr2=0x7f74781f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 0x7f74781f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 -- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74780b2a60 msgr2=0x7f74780b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74780b2a60 0x7f74780b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 -- 172.21.15.1:0/3800749460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74781f0260 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747d5b7640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 0x7f74781f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f747e5b9640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f746800efc0 tx=0x7f746800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468018020 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7468014480 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.818+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468014870 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.819+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74781f0540 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.819+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74781eff20 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.819+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74780cc230 con 0x7f747815e500 2024-05-09T06:31:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.820+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f746801b020 con 0x7f747815e500 2024-05-09T06:31:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.821+0000 7f74767fc640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 0x7f746406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.821+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f746808bad0 con 0x7f747815e500 2024-05-09T06:31:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.821+0000 7f747ddb8640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 0x7f746406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.821+0000 7f747ddb8640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 0x7f746406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:50.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:50.825+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74780cc230 con 0x7f747815e500 2024-05-09T06:31:50.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.008+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7478071b00 con 0x7f747815e500 2024-05-09T06:31:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.009+0000 7f74767fc640 1 -- 172.21.15.1:0/3800749460 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16944 (secure 0 0 0) 0x7f7478071b00 con 0x7f747815e500 2024-05-09T06:31:50.991 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:50.991 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":66,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:31:03.690223+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 msgr2=0x7f746406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 0x7f746406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7470007fd0 tx=0x7f7470009040 comp rx=0 tx=0).stop 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 msgr2=0x7f74781ef9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f746800efc0 tx=0x7f746800c490 comp rx=0 tx=0).stop 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 shutdown_connections 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f746406be30 0x7f746406e2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f747815e500 0x7f74781ef9e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f747815db10 0x7f74781f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 --2- 172.21.15.1:0/3800749460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74780b2a60 0x7f74780b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.011+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 >> 172.21.15.1:0/3800749460 conn(0x7f74780cc4b0 msgr2=0x7f747806d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.012+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 shutdown_connections 2024-05-09T06:31:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.012+0000 7f748001c640 1 -- 172.21.15.1:0/3800749460 wait complete. 2024-05-09T06:31:51.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:31:51.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:51.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.126+0000 7f740905d640 1 Processor -- start 2024-05-09T06:31:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.126+0000 7f740905d640 1 -- start start 2024-05-09T06:31:51.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740905d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f74040b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740905d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f74040b4f70 0x7f7404070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740905d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f740406eb40 con 0x7f74041454b0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740905d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f740406ee20 con 0x7f74040b4f70 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740905d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f740406d8e0 con 0x7f74040b2a60 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f7402d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f74041454b0 0x7f74040b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55320/0 (socket says 172.21.15.1:55320) 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f7402d9d640 1 -- 172.21.15.1:0/1865287355 learned_addr learned my addr 172.21.15.1:0/1865287355 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645052118 0 0) 0x7f740406eb40 con 0x7f74041454b0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73f0003520 con 0x7f74041454b0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740359e640 1 --2- 172.21.15.1:0/1865287355 >> v2:172.21.15.6:3300/0 conn(0x7f74040b4f70 0x7f7404070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.127+0000 7f740259c640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f74040b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 310581703 0 0) 0x7f73f0003520 con 0x7f74041454b0 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 msgr2=0x7f74040b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f74040b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 >> v2:172.21.15.6:3300/0 conn(0x7f74040b4f70 msgr2=0x7f7404070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 --2- 172.21.15.1:0/1865287355 >> v2:172.21.15.6:3300/0 conn(0x7f74040b4f70 0x7f7404070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740406dbc0 con 0x7f74041454b0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f73f40019f0 con 0x7f74041454b0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 >> v1:172.21.15.1:6789/0 conn(0x7f74041454b0 legacy=0x7f74040b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f00039b0 0x7f73f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f0004800 0x7f73f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f73f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f740406ee20 con 0x7f73f00039b0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f740406d8e0 con 0x7f73f0004800 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f73f000b9c0 con 0x7f74040b2a60 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f00039b0 0x7f73f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f740259c640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f0004800 0x7f73f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f740359e640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f73f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 -- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 msgr2=0x7f73f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74040b2a60 0x7f73f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 -- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f0004800 msgr2=0x7f73f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f0004800 0x7f73f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f7402d9d640 1 -- 172.21.15.1:0/1865287355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f0003520 con 0x7f73f00039b0 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.128+0000 7f740259c640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f0004800 0x7f73f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f7402d9d640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f00039b0 0x7f73f0003dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f73f400a7a0 tx=0x7f73f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e953692d20f22f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f400a3f0 con 0x7f73f00039b0 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f402f070 con 0x7f73f00039b0 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f7401d9b640 1 -- 172.21.15.1:0/1865287355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f402bc10 con 0x7f73f00039b0 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f740905d640 1 -- 172.21.15.1:0/1865287355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f00039b0 msgr2=0x7f73f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f740905d640 1 --2- 172.21.15.1:0/1865287355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f00039b0 0x7f73f0003dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f73f400a7a0 tx=0x7f73f0005830 comp rx=0 tx=0).stop 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f740359e640 1 -- 172.21.15.1:0/1865287355 reap_dead start 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f740905d640 1 -- 172.21.15.1:0/1865287355 shutdown_connections 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.129+0000 7f740905d640 1 -- 172.21.15.1:0/1865287355 >> 172.21.15.1:0/1865287355 conn(0x7f74040cc4b0 msgr2=0x7f74040cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.130+0000 7f740905d640 1 -- 172.21.15.1:0/1865287355 shutdown_connections 2024-05-09T06:31:51.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.130+0000 7f740905d640 1 -- 172.21.15.1:0/1865287355 wait complete. 2024-05-09T06:31:51.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.131+0000 7f740905d640 1 Processor -- start 2024-05-09T06:31:51.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.131+0000 7f740905d640 1 -- start start 2024-05-09T06:31:51.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 0x7f740413ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74041f1300 0x7f74041f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f740406e260 con 0x7f74040c8ba0 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f740406e0e0 con 0x7f73f00039b0 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740905d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f74040b4a30 con 0x7f74041f1300 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35306/0 (socket says 172.21.15.1:35306) 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 -- 172.21.15.1:0/894204949 learned_addr learned my addr 172.21.15.1:0/894204949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f7402d9d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 0x7f740413ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740359e640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74041f1300 0x7f74041f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 -- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74041f1300 msgr2=0x7f74041f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74041f1300 0x7f74041f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 -- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 msgr2=0x7f740413ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 0x7f740413ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 -- 172.21.15.1:0/894204949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74041f1cc0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f7402d9d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 0x7f740413ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.132+0000 7f740259c640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f73ec011540 tx=0x7f73ec011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.133+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec015870 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.133+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ec015a10 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.133+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74040ca7b0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.133+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec015dc0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.133+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74040c99f0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.134+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040752b0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.134+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73ec00f460 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.135+0000 7f73fb7fe640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.135+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f73ec08b8c0 con 0x7f74040c8ba0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.135+0000 7f7402d9d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.135+0000 7f7402d9d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 0x7f73e406e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f73f4033000 tx=0x7f73f4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.139+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74041f1cc0 con 0x7f74040c8ba0 2024-05-09T06:31:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.364+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7404072fa0 con 0x7f74040c8ba0 2024-05-09T06:31:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.364+0000 7f73fb7fe640 1 -- 172.21.15.1:0/894204949 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5168 (secure 0 0 0) 0x7f73ec019030 con 0x7f74040c8ba0 2024-05-09T06:31:51.346 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:51.346 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:31:15.612251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 37 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.366+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 msgr2=0x7f73e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.366+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 0x7f73e406e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f73f4033000 tx=0x7f73f4007000 comp rx=0 tx=0).stop 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.366+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 msgr2=0x7f74040c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.366+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f73ec011540 tx=0x7f73ec011a00 comp rx=0 tx=0).stop 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 shutdown_connections 2024-05-09T06:31:51.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f74041f1300 0x7f74041f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74040c8ba0 0x7f74040c9020 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 --2- 172.21.15.1:0/894204949 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73f00039b0 0x7f740413ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 >> 172.21.15.1:0/894204949 conn(0x7f74040cc4b0 msgr2=0x7f74040c5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 shutdown_connections 2024-05-09T06:31:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.367+0000 7f740905d640 1 -- 172.21.15.1:0/894204949 wait complete. 2024-05-09T06:31:51.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:51.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.482+0000 7ff1baa22640 1 Processor -- start 2024-05-09T06:31:51.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.482+0000 7ff1baa22640 1 -- start start 2024-05-09T06:31:51.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1baa22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b4071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1baa22640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1b40cbb60 0x7ff1b415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1baa22640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b40d04a0 con 0x7ff1b415fd70 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1baa22640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b40cbfe0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1baa22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b40722c0 con 0x7ff1b40b0080 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1b40cbb60 0x7ff1b415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1b40cbb60 0x7ff1b415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35318/0 (socket says 172.21.15.1:35318) 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 -- 172.21.15.1:0/2063023292 learned_addr learned my addr 172.21.15.1:0/2063023292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b3fff640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b4071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 -- 172.21.15.1:0/2063023292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 msgr2=0x7ff1b4071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b4071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 -- 172.21.15.1:0/2063023292 >> v1:172.21.15.6:6789/0 conn(0x7ff1b415fd70 legacy=0x7ff1b415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b37fe640 1 -- 172.21.15.1:0/2063023292 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b40725a0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.483+0000 7ff1b3fff640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b4071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:51.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.484+0000 7ff1b37fe640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415f830 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009870 tx=0x7ff1a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5248c61b866ab173 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.484+0000 7ff1b2ffd640 1 -- 172.21.15.1:0/2063023292 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8037070 con 0x7ff1b40cbb60 2024-05-09T06:31:51.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.484+0000 7ff1b2ffd640 1 -- 172.21.15.1:0/2063023292 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8032440 con 0x7ff1b40cbb60 2024-05-09T06:31:51.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.484+0000 7ff1b2ffd640 1 -- 172.21.15.1:0/2063023292 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8032810 con 0x7ff1b40cbb60 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 -- 172.21.15.1:0/2063023292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 msgr2=0x7ff1b415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415f830 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009870 tx=0x7ff1a802a270 comp rx=0 tx=0).stop 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 -- 172.21.15.1:0/2063023292 shutdown_connections 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415f830 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 --2- 172.21.15.1:0/2063023292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b4071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 -- 172.21.15.1:0/2063023292 >> 172.21.15.1:0/2063023292 conn(0x7ff1b40d0b50 msgr2=0x7ff1b40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 -- 172.21.15.1:0/2063023292 shutdown_connections 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.485+0000 7ff1baa22640 1 -- 172.21.15.1:0/2063023292 wait complete. 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.486+0000 7ff1baa22640 1 Processor -- start 2024-05-09T06:31:51.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.486+0000 7ff1baa22640 1 -- start start 2024-05-09T06:31:51.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b40b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b415fd70 0x7ff1b40b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b415e7c0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4072b60 con 0x7ff1b415fd70 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1baa22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b415f3f0 con 0x7ff1b40b0080 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35332/0 (socket says 172.21.15.1:35332) 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 -- 172.21.15.1:0/4205376128 learned_addr learned my addr 172.21.15.1:0/4205376128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b3fff640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b40b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b8fbf640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b415fd70 0x7ff1b40b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 -- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 msgr2=0x7ff1b40b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b40b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 -- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b415fd70 msgr2=0x7ff1b40b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b415fd70 0x7ff1b40b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.487+0000 7ff1b37fe640 1 -- 172.21.15.1:0/4205376128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b40b4190 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1b37fe640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8041020 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80306c0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8037040 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b40b4470 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.488+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b40b4a30 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.489+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b40752b0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.489+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1a8044020 con 0x7ff1b40cbb60 2024-05-09T06:31:51.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.490+0000 7ff1b0ff9640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 0x7ff1a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.490+0000 7ff1b3fff640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 0x7ff1a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.490+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7ff1a80ab6a0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.490+0000 7ff1b3fff640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 0x7ff1a006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40c9880 tx=0x7ff1a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.494+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1a804b030 con 0x7ff1b40cbb60 2024-05-09T06:31:51.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.717+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1b40cb3d0 con 0x7ff1b40cbb60 2024-05-09T06:31:51.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.717+0000 7ff1b0ff9640 1 -- 172.21.15.1:0/4205376128 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5168 (secure 0 0 0) 0x7ff1a8079890 con 0x7ff1b40cbb60 2024-05-09T06:31:51.699 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:51.699 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:30:34.938735+0000","modified":"2024-05-09T06:31:15.612251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.6:6835/531134827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":531134827},{"type":"v1","addr":"172.21.15.6:6835","nonce":531134827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":4}]} 2024-05-09T06:31:51.699 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 37 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 msgr2=0x7ff1a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 0x7ff1a006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40c9880 tx=0x7ff1a4008040 comp rx=0 tx=0).stop 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 msgr2=0x7ff1b415c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a80046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 shutdown_connections 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1a006be30 0x7ff1a006e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1b415fd70 0x7ff1b40b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1b40cbb60 0x7ff1b415c330 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 --2- 172.21.15.1:0/4205376128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1b40b0080 0x7ff1b40b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.719+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 >> 172.21.15.1:0/4205376128 conn(0x7ff1b40d0b50 msgr2=0x7ff1b40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:51.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.720+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 shutdown_connections 2024-05-09T06:31:51.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.720+0000 7ff1baa22640 1 -- 172.21.15.1:0/4205376128 wait complete. 2024-05-09T06:31:51.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:31:51.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.835+0000 7f5980c12640 1 Processor -- start 2024-05-09T06:31:51.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.835+0000 7f5980c12640 1 -- start start 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f5980c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c0d4940 0x7f597c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f5980c12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f597c0d2b10 0x7f597c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f5980c12640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c0b33c0 con 0x7f597c161950 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f5980c12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f597c0d4dd0 con 0x7f597c0d2b10 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f5980c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f597c0b3660 con 0x7f597c0d4940 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597a59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f597c161950 0x7f597c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55326/0 (socket says 172.21.15.1:55326) 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597a59c640 1 -- 172.21.15.1:0/2901054408 learned_addr learned my addr 172.21.15.1:0/2901054408 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597ad9d640 1 --2- 172.21.15.1:0/2901054408 >> v2:172.21.15.1:3300/0 conn(0x7f597c0d2b10 0x7f597c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597ad9d640 1 -- 172.21.15.1:0/2901054408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c0d4940 msgr2=0x7f597c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597ad9d640 1 --2- 172.21.15.1:0/2901054408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c0d4940 0x7f597c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597ad9d640 1 -- 172.21.15.1:0/2901054408 >> v1:172.21.15.1:6789/0 conn(0x7f597c161950 legacy=0x7f597c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.836+0000 7f597ad9d640 1 -- 172.21.15.1:0/2901054408 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0b3940 con 0x7f597c0d2b10 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.837+0000 7f597ad9d640 1 --2- 172.21.15.1:0/2901054408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c0b4680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d90 tx=0x7f597402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435b05b28c3b48c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.837+0000 7f5979d9b640 1 -- 172.21.15.1:0/2901054408 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974007740 con 0x7f597c0d2b10 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.837+0000 7f5979d9b640 1 -- 172.21.15.1:0/2901054408 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59740078e0 con 0x7f597c0d2b10 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.837+0000 7f5979d9b640 1 -- 172.21.15.1:0/2901054408 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974007cb0 con 0x7f597c0d2b10 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 -- 172.21.15.1:0/2901054408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 msgr2=0x7f597c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 --2- 172.21.15.1:0/2901054408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c0b4680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d90 tx=0x7f597402a660 comp rx=0 tx=0).stop 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 -- 172.21.15.1:0/2901054408 shutdown_connections 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 --2- 172.21.15.1:0/2901054408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c0d4940 0x7f597c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 --2- 172.21.15.1:0/2901054408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c0b4680 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 -- 172.21.15.1:0/2901054408 >> 172.21.15.1:0/2901054408 conn(0x7f597c0d3e20 msgr2=0x7f597c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 -- 172.21.15.1:0/2901054408 shutdown_connections 2024-05-09T06:31:51.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.838+0000 7f5980c12640 1 -- 172.21.15.1:0/2901054408 wait complete. 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.839+0000 7f5980c12640 1 Processor -- start 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.839+0000 7f5980c12640 1 -- start start 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 0x7f597c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 0x7f597c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0d1100 con 0x7f597c0d2b10 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0d0f80 con 0x7f597c0cb900 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5980c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f597c0d1c00 con 0x7f597c161950 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35350/0 (socket says 172.21.15.1:35350) 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 -- 172.21.15.1:0/848997824 learned_addr learned my addr 172.21.15.1:0/848997824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597ad9d640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 0x7f597c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 -- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 msgr2=0x7f597c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5973fff640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 0x7f597c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 0x7f597c1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 -- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 msgr2=0x7f597c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 0x7f597c0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597a59c640 1 -- 172.21.15.1:0/848997824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0b69d0 con 0x7f597c0d2b10 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f597ad9d640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 0x7f597c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.840+0000 7f5973fff640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 0x7f597c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f597a59c640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c3c0 tx=0x7f596c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c017070 con 0x7f597c0d2b10 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c005ce0 con 0x7f597c0d2b10 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c007310 con 0x7f597c0d2b10 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c0b6cb0 con 0x7f597c0d2b10 2024-05-09T06:31:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.841+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c0b7270 con 0x7f597c0d2b10 2024-05-09T06:31:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.842+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c0752b0 con 0x7f597c0d2b10 2024-05-09T06:31:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.842+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f596c01a070 con 0x7f597c0d2b10 2024-05-09T06:31:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.843+0000 7f5972ffd640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 0x7f595c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.843+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7f596c08a3a0 con 0x7f597c0d2b10 2024-05-09T06:31:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.843+0000 7f5973fff640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 0x7f595c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:51.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.843+0000 7f5973fff640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 0x7f595c06e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5964006fd0 tx=0x7f5964008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:51.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:51.847+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f596c058b40 con 0x7f597c0d2b10 2024-05-09T06:31:52.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.030+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f597c0d21e0 con 0x7f597c0d2b10 2024-05-09T06:31:52.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.031+0000 7f5972ffd640 1 -- 172.21.15.1:0/848997824 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16944 (secure 0 0 0) 0x7f596c058440 con 0x7f597c0d2b10 2024-05-09T06:31:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:52.013 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":66,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:31:03.690223+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:30:33.248885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-09T06:30:33.918243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 msgr2=0x7f595c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 0x7f595c06e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5964006fd0 tx=0x7f5964008040 comp rx=0 tx=0).stop 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 msgr2=0x7f597c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c3c0 tx=0x7f596c00c880 comp rx=0 tx=0).stop 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 shutdown_connections 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f595c06be30 0x7f595c06e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c0cb900 0x7f597c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c161950 0x7f597c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 --2- 172.21.15.1:0/848997824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c0d2b10 0x7f597c15a080 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 >> 172.21.15.1:0/848997824 conn(0x7f597c0d3e20 msgr2=0x7f597c0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:52.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 shutdown_connections 2024-05-09T06:31:52.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.033+0000 7f5980c12640 1 -- 172.21.15.1:0/848997824 wait complete. 2024-05-09T06:31:52.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:31:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 Processor -- start 2024-05-09T06:31:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 -- start start 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb794099b30 0x7fb7940b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb794099930 con 0x7fb79415e4b0 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb794162930 con 0x7fb7940c6860 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.150+0000 7fb799e12640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940b1e20 con 0x7fb794099b30 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb793fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb79415e4b0 0x7fb79415f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55340/0 (socket says 172.21.15.1:55340) 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb793fff640 1 -- 172.21.15.1:0/2767547295 learned_addr learned my addr 172.21.15.1:0/2767547295 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2767547295 >> v2:172.21.15.6:3300/0 conn(0x7fb794099b30 0x7fb7940b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845304926 0 0) 0x7fb78c001490 con 0x7fb79415e4b0 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb77c003520 con 0x7fb79415e4b0 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 12006325 0 0) 0x7fb77c003520 con 0x7fb79415e4b0 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 >> v2:172.21.15.6:3300/0 conn(0x7fb794099b30 msgr2=0x7fb7940b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 --2- 172.21.15.1:0/2767547295 >> v2:172.21.15.6:3300/0 conn(0x7fb794099b30 0x7fb7940b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 >> v1:172.21.15.6:6789/0 conn(0x7fb7940c6860 legacy=0x7fb7940c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb794162c10 con 0x7fb79415e4b0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb78c001bc0 con 0x7fb79415e4b0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 >> v1:172.21.15.1:6789/0 conn(0x7fb79415e4b0 legacy=0x7fb79415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb77c0039b0 0x7fb77c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb77c0051c0 0x7fb77c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c00c030 con 0x7fb77c004800 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c00c1e0 con 0x7fb77c0039b0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c00c3e0 con 0x7fb77c0051c0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb7937fe640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb77c0039b0 0x7fb77c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.151+0000 7fb793fff640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb77c0051c0 0x7fb77c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 -- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb77c0051c0 msgr2=0x7fb77c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb77c0051c0 0x7fb77c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 -- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb77c0039b0 msgr2=0x7fb77c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb77c0039b0 0x7fb77c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 -- 172.21.15.1:0/2767547295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c003520 con 0x7fb77c004800 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb793fff640 1 -- 172.21.15.1:0/2767547295 reap_dead start 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.152+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb788009ac0 tx=0x7fb77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3888e4ace4975e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788036070 con 0x7fb77c004800 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788031400 con 0x7fb77c004800 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb7927fc640 1 -- 172.21.15.1:0/2767547295 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788009690 con 0x7fb77c004800 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 -- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 msgr2=0x7fb77c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb788009ac0 tx=0x7fb77c005830 comp rx=0 tx=0).stop 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 -- 172.21.15.1:0/2767547295 shutdown_connections 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 --2- 172.21.15.1:0/2767547295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb77c004c80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 -- 172.21.15.1:0/2767547295 >> 172.21.15.1:0/2767547295 conn(0x7fb7940b1fb0 msgr2=0x7fb7940b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:52.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 -- 172.21.15.1:0/2767547295 shutdown_connections 2024-05-09T06:31:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.153+0000 7fb799e12640 1 -- 172.21.15.1:0/2767547295 wait complete. 2024-05-09T06:31:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.154+0000 7fb799e12640 1 Processor -- start 2024-05-09T06:31:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 -- start start 2024-05-09T06:31:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 0x7fb7940b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 0x7fb7941e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940b2aa0 con 0x7fb77c004800 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb79415ee90 con 0x7fb7941e7c00 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb799e12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb7940b2c20 con 0x7fb7940b4d70 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb7937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb7937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35374/0 (socket says 172.21.15.1:35374) 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.155+0000 7fb7937fe640 1 -- 172.21.15.1:0/2661227065 learned_addr learned my addr 172.21.15.1:0/2661227065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 0x7fb7940b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb793fff640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 0x7fb7941e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 -- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 msgr2=0x7fb7940b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 0x7fb7940b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 -- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 msgr2=0x7fb7941e8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 0x7fb7941e8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 -- 172.21.15.1:0/2661227065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7941e88f0 con 0x7fb77c004800 2024-05-09T06:31:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 0x7fb7940b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb793fff640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 0x7fb7941e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb7937fe640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb78400f5f0 tx=0x7fb78400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840052e0 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784018070 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784003810 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7941e8bd0 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.156+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7941e9190 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.157+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7940752b0 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.158+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7840039b0 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.158+0000 7fb77bfff640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 0x7fb78006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.158+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6217+0+0 (secure 0 0 0) 0x7fb78408af20 con 0x7fb77c004800 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.158+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 0x7fb78006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.159+0000 7fb792ffd640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 0x7fb78006e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb788033000 tx=0x7fb788008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.163+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb784059640 con 0x7fb77c004800 2024-05-09T06:31:52.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.383+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb79406ce30 con 0x7fb77c004800 2024-05-09T06:31:52.365 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:31:52.384+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:31:52.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.385+0000 7fb77bfff640 1 -- 172.21.15.1:0/2661227065 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fb784058f40 con 0x7fb77c004800 2024-05-09T06:31:52.367 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 msgr2=0x7fb78006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 0x7fb78006e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb788033000 tx=0x7fb788008030 comp rx=0 tx=0).stop 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 msgr2=0x7fb794157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb78400f5f0 tx=0x7fb78400fab0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 shutdown_connections 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb78006be60 0x7fb78006e320 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7941e7c00 0x7fb7941e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7940b4d70 0x7fb7940b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 --2- 172.21.15.1:0/2661227065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb77c004800 0x7fb794157440 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 >> 172.21.15.1:0/2661227065 conn(0x7fb7940b1fb0 msgr2=0x7fb79415cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:52.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 shutdown_connections 2024-05-09T06:31:52.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.388+0000 7fb799e12640 1 -- 172.21.15.1:0/2661227065 wait complete. 2024-05-09T06:31:52.373 INFO:tasks.ceph.mds.b.smithi006.stderr: -7737> 2024-05-09T06:30:30.592+0000 7ff0d7734640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-09T06:31:52.375 INFO:tasks.ceph.mds.b.smithi006.stderr: -6599> 2024-05-09T06:30:37.559+0000 7ff0d1728640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:31:52.377 INFO:tasks.ceph.mds.b.smithi006.stderr: -4821> 2024-05-09T06:30:44.051+0000 7ff0cd720640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6732=0-4194304@1} caps={6732=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6732 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562048534c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:31:52.378 INFO:tasks.ceph.mds.b.smithi006.stderr: -2875> 2024-05-09T06:31:15.409+0000 7ff0cd720640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-09T06:30:42.935028+0000 1=1+0) caps={6732=pAsLsXsFs/-@2},l=6732(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x562048534c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:31:52.381 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:31:52.410 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:31:52.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.506+0000 7f0bc60e7640 1 Processor -- start 2024-05-09T06:31:52.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.506+0000 7f0bc60e7640 1 -- start start 2024-05-09T06:31:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bc60e7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0bc01585d0 0x7f0bc00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bc60e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bc0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bc60e7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0071aa0 con 0x7f0bc0162090 2024-05-09T06:31:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bc60e7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc01624e0 con 0x7f0bc01585d0 2024-05-09T06:31:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bc60e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc0071d80 con 0x7f0bc00715d0 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0bc0162090 0x7f0bc0146fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55356/0 (socket says 172.21.15.1:55356) 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbffff640 1 -- 172.21.15.1:0/1458010414 learned_addr learned my addr 172.21.15.1:0/1458010414 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3234745786 0 0) 0x7f0bc0071aa0 con 0x7f0bc0162090 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbeffd640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bc0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/1458010414 >> v2:172.21.15.6:3300/0 conn(0x7f0bc01585d0 0x7f0bc00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.507+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bac003520 con 0x7f0bc0162090 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2032217705 0 0) 0x7f0bac003520 con 0x7f0bc0162090 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 msgr2=0x7f0bc0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bc0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 >> v2:172.21.15.6:3300/0 conn(0x7f0bc01585d0 msgr2=0x7f0bc00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 --2- 172.21.15.1:0/1458010414 >> v2:172.21.15.6:3300/0 conn(0x7f0bc01585d0 0x7f0bc00b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc0072060 con 0x7f0bc0162090 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f0bb80019f0 con 0x7f0bc0162090 2024-05-09T06:31:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 >> v1:172.21.15.1:6789/0 conn(0x7f0bc0162090 legacy=0x7f0bc0146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac0039b0 0x7f0bac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac004d40 0x7f0bac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01624e0 con 0x7f0bac004d40 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0071d80 con 0x7f0bac0039b0 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0bac00b9c0 con 0x7f0bc00715d0 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac004d40 0x7f0bac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac0039b0 0x7f0bac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbeffd640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 -- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 msgr2=0x7f0bac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc00715d0 0x7f0bac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 -- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac0039b0 msgr2=0x7f0bac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac0039b0 0x7f0bac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.508+0000 7f0bbffff640 1 -- 172.21.15.1:0/1458010414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac003520 con 0x7f0bac004d40 2024-05-09T06:31:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bbffff640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac004d40 0x7f0bac0051c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800a7d0 tx=0x7f0bac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818650d72c33b38d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800a3f0 con 0x7f0bac004d40 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb802f070 con 0x7f0bac004d40 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bbe7fc640 1 -- 172.21.15.1:0/1458010414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb802bc30 con 0x7f0bac004d40 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bc60e7640 1 -- 172.21.15.1:0/1458010414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac004d40 msgr2=0x7f0bac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/1458010414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac004d40 0x7f0bac0051c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800a7d0 tx=0x7f0bac005830 comp rx=0 tx=0).stop 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.509+0000 7f0bbffff640 1 -- 172.21.15.1:0/1458010414 reap_dead start 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.510+0000 7f0bc60e7640 1 -- 172.21.15.1:0/1458010414 shutdown_connections 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.510+0000 7f0bc60e7640 1 -- 172.21.15.1:0/1458010414 >> 172.21.15.1:0/1458010414 conn(0x7f0bc00b1ba0 msgr2=0x7f0bc00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.510+0000 7f0bc60e7640 1 -- 172.21.15.1:0/1458010414 shutdown_connections 2024-05-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.510+0000 7f0bc60e7640 1 -- 172.21.15.1:0/1458010414 wait complete. 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.511+0000 7f0bc60e7640 1 Processor -- start 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.511+0000 7f0bc60e7640 1 -- start start 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 0x7f0bc0153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 0x7f0bc00cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01589b0 con 0x7f0bc00d0350 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0072620 con 0x7f0bac004d40 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc00b4190 con 0x7f0bc01f4950 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35400/0 (socket says 172.21.15.1:35400) 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 -- 172.21.15.1:0/2267118411 learned_addr learned my addr 172.21.15.1:0/2267118411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbeffd640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 0x7f0bc00cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 0x7f0bc0153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 -- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 msgr2=0x7f0bc00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 0x7f0bc00cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 -- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 msgr2=0x7f0bc0153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 0x7f0bc0153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 -- 172.21.15.1:0/2267118411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01f7f40 con 0x7f0bc00d0350 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbeffd640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 0x7f0bc00cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 0x7f0bc0153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bbffff640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800a7d0 tx=0x7f0bc015f570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8001f30 con 0x7f0bc00d0350 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb802bbc0 con 0x7f0bc00d0350 2024-05-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8034470 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.512+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01f3e70 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.513+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc0153da0 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.514+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0074610 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.514+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bb8034610 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.514+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 0x7f0ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.515+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6303+0+0 (secure 0 0 0) 0x7f0bb80ab200 con 0x7f0bc00d0350 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.515+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 0x7f0ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.515+0000 7f0bbf7fe640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 0x7f0ba006e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009b30 tx=0x7f0bb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.519+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bb80799f0 con 0x7f0bc00d0350 2024-05-09T06:31:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:52.743+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0bc00752b0 con 0x7f0bc00d0350 2024-05-09T06:31:53.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.387+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2267118411 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f0bb80792f0 con 0x7f0bc00d0350 2024-05-09T06:31:53.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 msgr2=0x7f0ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 0x7f0ba006e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009b30 tx=0x7f0bb0008040 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 msgr2=0x7f0bc01f7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800a7d0 tx=0x7f0bc015f570 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 shutdown_connections 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ba006be30 0x7f0ba006e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00d0350 0x7f0bc01f7250 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0bc01f4950 0x7f0bc00cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 --2- 172.21.15.1:0/2267118411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0bac004d40 0x7f0bc0153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 >> 172.21.15.1:0/2267118411 conn(0x7f0bc00b1ba0 msgr2=0x7f0bc015da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 shutdown_connections 2024-05-09T06:31:53.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.390+0000 7f0bc60e7640 1 -- 172.21.15.1:0/2267118411 wait complete. 2024-05-09T06:31:53.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:31:53.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.519+0000 7f1673f14640 1 Processor -- start 2024-05-09T06:31:53.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.519+0000 7f1673f14640 1 -- start start 2024-05-09T06:31:53.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f166c15e420 0x7f166c0d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f166c0d2a00 0x7f166c0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f166c15e800 con 0x7f166c15e420 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f166c0cfe10 con 0x7f166c0d2a00 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1673f14640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0cf850 con 0x7f166c0d1d70 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1671cb0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f166c15e420 0x7f166c0d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1671cb0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f166c15e420 0x7f166c0d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35408/0 (socket says 172.21.15.1:35408) 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.520+0000 7f1671cb0640 1 -- 172.21.15.1:0/2351125036 learned_addr learned my addr 172.21.15.1:0/2351125036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f16714af640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f16724b1640 1 --2- 172.21.15.1:0/2351125036 >> v2:172.21.15.6:3300/0 conn(0x7f166c0d2a00 0x7f166c0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 -- 172.21.15.1:0/2351125036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 msgr2=0x7f166c0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 -- 172.21.15.1:0/2351125036 >> v2:172.21.15.6:3300/0 conn(0x7f166c0d2a00 msgr2=0x7f166c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2351125036 >> v2:172.21.15.6:3300/0 conn(0x7f166c0d2a00 0x7f166c0d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 -- 172.21.15.1:0/2351125036 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0cfb30 con 0x7f166c15e420 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.521+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c15e420 0x7f166c0d4a00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f166c0d0150 tx=0x7f165c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c9a11e91bbe4af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1670cae640 1 -- 172.21.15.1:0/2351125036 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c036070 con 0x7f166c15e420 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1670cae640 1 -- 172.21.15.1:0/2351125036 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c031600 con 0x7f166c15e420 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1670cae640 1 -- 172.21.15.1:0/2351125036 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c0319d0 con 0x7f166c15e420 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 -- 172.21.15.1:0/2351125036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c15e420 msgr2=0x7f166c0d4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c15e420 0x7f166c0d4a00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f166c0d0150 tx=0x7f165c02a610 comp rx=0 tx=0).stop 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 -- 172.21.15.1:0/2351125036 shutdown_connections 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 --2- 172.21.15.1:0/2351125036 >> v2:172.21.15.6:3300/0 conn(0x7f166c0d2a00 0x7f166c0d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 --2- 172.21.15.1:0/2351125036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c15e420 0x7f166c0d4a00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 -- 172.21.15.1:0/2351125036 >> 172.21.15.1:0/2351125036 conn(0x7f166c0d3ee0 msgr2=0x7f166c0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 -- 172.21.15.1:0/2351125036 shutdown_connections 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.522+0000 7f1673f14640 1 -- 172.21.15.1:0/2351125036 wait complete. 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 Processor -- start 2024-05-09T06:31:53.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 -- start start 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0b9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f166c15e420 0x7f166c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0cf500 con 0x7f166c0d2a00 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0c7e40 con 0x7f166c15e420 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f1673f14640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0d0a50 con 0x7f166c0d1d70 2024-05-09T06:31:53.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f16714af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f16714af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35418/0 (socket says 172.21.15.1:35418) 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.524+0000 7f16714af640 1 -- 172.21.15.1:0/2681774030 learned_addr learned my addr 172.21.15.1:0/2681774030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16724b1640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f166c15e420 0x7f166c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 -- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 msgr2=0x7f166c0b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 -- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f166c15e420 msgr2=0x7f166c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f166c15e420 0x7f166c0cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 -- 172.21.15.1:0/2681774030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0b8770 con 0x7f166c0d2a00 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f16714af640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f166000d7d0 tx=0x7f166000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16600096a0 con 0x7f166c0d2a00 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1660019070 con 0x7f166c0d2a00 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c0cb440 con 0x7f166c0d2a00 2024-05-09T06:31:53.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660009a60 con 0x7f166c0d2a00 2024-05-09T06:31:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.525+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c0cba00 con 0x7f166c0d2a00 2024-05-09T06:31:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.526+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c075320 con 0x7f166c0d2a00 2024-05-09T06:31:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.527+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1660009c00 con 0x7f166c0d2a00 2024-05-09T06:31:53.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.527+0000 7f166a7fc640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 0x7f165806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.527+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6303+0+0 (secure 0 0 0) 0x7f166008b2d0 con 0x7f166c0d2a00 2024-05-09T06:31:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.527+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 0x7f165806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:53.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.528+0000 7f1671cb0640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 0x7f165806e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f165c033000 tx=0x7f165c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:53.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.531+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1660059470 con 0x7f166c0d2a00 2024-05-09T06:31:53.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:53.716+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f166c0b98c0 con 0x7f166c0d2a00 2024-05-09T06:31:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.389+0000 7f166a7fc640 1 -- 172.21.15.1:0/2681774030 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) ==== 181+0+0 (secure 0 0 0) 0x7f166005d770 con 0x7f166c0d2a00 2024-05-09T06:31:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:31:54.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 msgr2=0x7f165806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:54.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 0x7f165806e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f165c033000 tx=0x7f165c009000 comp rx=0 tx=0).stop 2024-05-09T06:31:54.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 msgr2=0x7f166c1576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:54.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f166000d7d0 tx=0x7f166000dc90 comp rx=0 tx=0).stop 2024-05-09T06:31:54.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 shutdown_connections 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f165806be60 0x7f165806e320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f166c15e420 0x7f166c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f166c0d2a00 0x7f166c1576d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 --2- 172.21.15.1:0/2681774030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f166c0d1d70 0x7f166c0b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.392+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 >> 172.21.15.1:0/2681774030 conn(0x7f166c0d3ee0 msgr2=0x7f166c0c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.393+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 shutdown_connections 2024-05-09T06:31:54.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.393+0000 7f1673f14640 1 -- 172.21.15.1:0/2681774030 wait complete. 2024-05-09T06:31:54.388 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:31:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.524+0000 7f5e1a141640 1 Processor -- start 2024-05-09T06:31:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.524+0000 7f5e1a141640 1 -- start start 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5e14143bb0 0x7f5e140cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5e141446d0 0x7f5e140d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14144110 0x7f5e1415e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5e140ce8d0 con 0x7f5e14143bb0 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5e140cdd50 con 0x7f5e141446d0 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e1a141640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5e140ce030 con 0x7f5e14144110 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e137fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5e14143bb0 0x7f5e140cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e137fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5e14143bb0 0x7f5e140cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35422/0 (socket says 172.21.15.1:35422) 2024-05-09T06:31:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.525+0000 7f5e137fe640 1 -- 172.21.15.1:0/3348391308 learned_addr learned my addr 172.21.15.1:0/3348391308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e13fff640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14144110 0x7f5e1415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e12ffd640 1 --2- 172.21.15.1:0/3348391308 >> v2:172.21.15.6:3300/0 conn(0x7f5e141446d0 0x7f5e140d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 -- 172.21.15.1:0/3348391308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14144110 msgr2=0x7f5e1415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14144110 0x7f5e1415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 -- 172.21.15.1:0/3348391308 >> v2:172.21.15.6:3300/0 conn(0x7f5e141446d0 msgr2=0x7f5e140d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 --2- 172.21.15.1:0/3348391308 >> v2:172.21.15.6:3300/0 conn(0x7f5e141446d0 0x7f5e140d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 -- 172.21.15.1:0/3348391308 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e140ce310 con 0x7f5e14143bb0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.526+0000 7f5e137fe640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14143bb0 0x7f5e140cfcb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e140cd800 tx=0x7f5e0402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c3cebce7819bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e127fc640 1 -- 172.21.15.1:0/3348391308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04032700 con 0x7f5e14143bb0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e127fc640 1 -- 172.21.15.1:0/3348391308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040328a0 con 0x7f5e14143bb0 2024-05-09T06:31:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e127fc640 1 -- 172.21.15.1:0/3348391308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04032c70 con 0x7f5e14143bb0 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 -- 172.21.15.1:0/3348391308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14143bb0 msgr2=0x7f5e140cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14143bb0 0x7f5e140cfcb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e140cd800 tx=0x7f5e0402c800 comp rx=0 tx=0).stop 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 -- 172.21.15.1:0/3348391308 shutdown_connections 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14144110 0x7f5e1415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 --2- 172.21.15.1:0/3348391308 >> v2:172.21.15.6:3300/0 conn(0x7f5e141446d0 0x7f5e140d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 --2- 172.21.15.1:0/3348391308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14143bb0 0x7f5e140cfcb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 -- 172.21.15.1:0/3348391308 >> 172.21.15.1:0/3348391308 conn(0x7f5e1406b3c0 msgr2=0x7f5e1406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 -- 172.21.15.1:0/3348391308 shutdown_connections 2024-05-09T06:31:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.527+0000 7f5e1a141640 1 -- 172.21.15.1:0/3348391308 wait complete. 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.528+0000 7f5e1a141640 1 Processor -- start 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 -- start start 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14143bb0 0x7f5e14132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 0x7f5e141f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e140cdaf0 con 0x7f5e14144110 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5e140cd970 con 0x7f5e141446d0 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e1a141640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1415eea0 con 0x7f5e14143bb0 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e12ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35424/0 (socket says 172.21.15.1:35424) 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.529+0000 7f5e12ffd640 1 -- 172.21.15.1:0/1203311366 learned_addr learned my addr 172.21.15.1:0/1203311366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e137fe640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14143bb0 0x7f5e14132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e13fff640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 0x7f5e141f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 -- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14143bb0 msgr2=0x7f5e14132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14143bb0 0x7f5e14132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 -- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 msgr2=0x7f5e141f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 0x7f5e141f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 -- 172.21.15.1:0/1203311366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e140a97d0 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e13fff640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 0x7f5e141f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e12ffd640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800d5f0 tx=0x7f5e0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080125f0 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e08019070 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e14140440 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080129d0 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.530+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e140a9480 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.531+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1415dbb0 con 0x7f5e14144110 2024-05-09T06:31:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.532+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e08012b70 con 0x7f5e14144110 2024-05-09T06:31:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.532+0000 7f5dfbfff640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 0x7f5e0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.532+0000 7f5e137fe640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 0x7f5e0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.532+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5836+0+0 (secure 0 0 0) 0x7f5e0808b240 con 0x7f5e14144110 2024-05-09T06:31:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.533+0000 7f5e137fe640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 0x7f5e0006e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e04037000 tx=0x7f5e04032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.537+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e1415dbb0 con 0x7f5e14144110 2024-05-09T06:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:54.725+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e140cd420 con 0x7f5e14144110 2024-05-09T06:31:55.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.390+0000 7f5dfbfff640 1 -- 172.21.15.1:0/1203311366 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) ==== 169+0+0 (secure 0 0 0) 0x7f5e140cd420 con 0x7f5e14144110 2024-05-09T06:31:55.372 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:31:55.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 msgr2=0x7f5e0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 0x7f5e0006e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e04037000 tx=0x7f5e04032000 comp rx=0 tx=0).stop 2024-05-09T06:31:55.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 msgr2=0x7f5e14142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800d5f0 tx=0x7f5e0800dab0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 shutdown_connections 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5e0006be60 0x7f5e0006e320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e141446d0 0x7f5e141f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e14144110 0x7f5e14142ba0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 --2- 172.21.15.1:0/1203311366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5e14143bb0 0x7f5e14132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 >> 172.21.15.1:0/1203311366 conn(0x7f5e1406b3c0 msgr2=0x7f5e14069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.395+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 shutdown_connections 2024-05-09T06:31:55.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.396+0000 7f5e1a141640 1 -- 172.21.15.1:0/1203311366 wait complete. 2024-05-09T06:31:55.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:31:55.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.522+0000 7f04ab25d640 1 Processor -- start 2024-05-09T06:31:55.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.522+0000 7f04ab25d640 1 -- start start 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04ab25d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04a40cf180 0x7f04a40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04ab25d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a40b1340 con 0x7f04a40ceba0 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04ab25d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a40d1020 con 0x7f04a415e270 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04ab25d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f04a40d0df0 con 0x7f04a40cf180 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04a3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f04a40ceba0 0x7f04a40ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55370/0 (socket says 172.21.15.1:55370) 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.523+0000 7f04a3fff640 1 -- 172.21.15.1:0/3748665722 learned_addr learned my addr 172.21.15.1:0/3748665722 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a97fa640 1 --2- 172.21.15.1:0/3748665722 >> v2:172.21.15.6:3300/0 conn(0x7f04a40cf180 0x7f04a40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317809862 0 0) 0x7f0498001110 con 0x7f04a40ceba0 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f048c003520 con 0x7f04a40ceba0 2024-05-09T06:31:55.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2086919365 0 0) 0x7f048c003520 con 0x7f04a40ceba0 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 >> v2:172.21.15.6:3300/0 conn(0x7f04a40cf180 msgr2=0x7f04a40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 --2- 172.21.15.1:0/3748665722 >> v2:172.21.15.6:3300/0 conn(0x7f04a40cf180 0x7f04a40d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 >> v1:172.21.15.6:6789/0 conn(0x7f04a415e270 legacy=0x7f04a40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a40cc4b0 con 0x7f04a40ceba0 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f0498001840 con 0x7f04a40ceba0 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 >> v1:172.21.15.1:6789/0 conn(0x7f04a40ceba0 legacy=0x7f04a40ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f048c0039b0 0x7f048c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c0051c0 0x7f048c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f048c00c040 con 0x7f048c004800 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f048c00c1f0 con 0x7f048c0039b0 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f048c00c3f0 con 0x7f048c0051c0 2024-05-09T06:31:55.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a8ff9640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f048c0039b0 0x7f048c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a97fa640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c0051c0 0x7f048c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 -- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c0051c0 msgr2=0x7f048c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c0051c0 0x7f048c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 -- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f048c0039b0 msgr2=0x7f048c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f048c0039b0 0x7f048c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a3fff640 1 -- 172.21.15.1:0/3748665722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c003520 con 0x7f048c004800 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.524+0000 7f04a97fa640 1 -- 172.21.15.1:0/3748665722 reap_dead start 2024-05-09T06:31:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.525+0000 7f04a3fff640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 0x7f048c004c80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f049800c4c0 tx=0x7f048c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b64efea91f3bda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.525+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049800c0e0 con 0x7f048c004800 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.525+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0498034070 con 0x7f048c004800 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.525+0000 7f04a37fe640 1 -- 172.21.15.1:0/3748665722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498005d70 con 0x7f048c004800 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 -- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 msgr2=0x7f048c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 0x7f048c004c80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f049800c4c0 tx=0x7f048c005830 comp rx=0 tx=0).stop 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 -- 172.21.15.1:0/3748665722 shutdown_connections 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 --2- 172.21.15.1:0/3748665722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 -- 172.21.15.1:0/3748665722 >> 172.21.15.1:0/3748665722 conn(0x7f04a40d1c10 msgr2=0x7f04a40d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 -- 172.21.15.1:0/3748665722 shutdown_connections 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.526+0000 7f04ab25d640 1 -- 172.21.15.1:0/3748665722 wait complete. 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.527+0000 7f04ab25d640 1 Processor -- start 2024-05-09T06:31:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.527+0000 7f04ab25d640 1 -- start start 2024-05-09T06:31:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c004800 0x7f04a40c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04a4143e80 0x7f04a4144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04a40cf590 con 0x7f04a41574d0 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f04a40cc260 con 0x7f04a4143e80 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04ab25d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f04a40cca70 con 0x7f048c004800 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04a3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35448/0 (socket says 172.21.15.1:35448) 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.528+0000 7f04a3fff640 1 -- 172.21.15.1:0/2929086892 learned_addr learned my addr 172.21.15.1:0/2929086892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a8ff9640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c004800 0x7f04a40c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a97fa640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04a4143e80 0x7f04a4144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 -- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c004800 msgr2=0x7f04a40c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c004800 0x7f04a40c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 -- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04a4143e80 msgr2=0x7f04a4144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04a4143e80 0x7f04a4144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 -- 172.21.15.1:0/2929086892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a40c6b30 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a3fff640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0498031040 tx=0x7f0498001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498001840 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049803f030 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498007290 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a40c6e10 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.529+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a40c73d0 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.530+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a40752b0 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.531+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0498007430 con 0x7f04a41574d0 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.531+0000 7f04a17fa640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 0x7f049006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.531+0000 7f04a8ff9640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 0x7f049006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.531+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f04980ab3b0 con 0x7f04a41574d0 2024-05-09T06:31:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.531+0000 7f04a8ff9640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 0x7f049006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0494000c00 tx=0x7f0494009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.535+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0498049030 con 0x7f04a41574d0 2024-05-09T06:31:55.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.722+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f04a40d29e0 con 0x7f04a41574d0 2024-05-09T06:31:55.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.725+0000 7f04a17fa640 1 -- 172.21.15.1:0/2929086892 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f04a40b1340 con 0x7f04a41574d0 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 msgr2=0x7f049006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 0x7f049006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0494000c00 tx=0x7f0494009040 comp rx=0 tx=0).stop 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 msgr2=0x7f04a4157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0498031040 tx=0x7f0498001c10 comp rx=0 tx=0).stop 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 shutdown_connections 2024-05-09T06:31:55.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f049006be60 0x7f049006e320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04a4143e80 0x7f04a4144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04a41574d0 0x7f04a4157950 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 --2- 172.21.15.1:0/2929086892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f048c004800 0x7f04a40c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 >> 172.21.15.1:0/2929086892 conn(0x7f04a40d1c10 msgr2=0x7f04a415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 shutdown_connections 2024-05-09T06:31:55.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.728+0000 7f04ab25d640 1 -- 172.21.15.1:0/2929086892 wait complete. 2024-05-09T06:31:55.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-09T06:31:55.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 Processor -- start 2024-05-09T06:31:55.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 -- start start 2024-05-09T06:31:55.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5880cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5880d04a0 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5880722c0 con 0x7ff5880b0080 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.844+0000 7ff58d329640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff5880725a0 con 0x7ff58815f3f0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5867fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff588071ce0 0x7ff58815ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55372/0 (socket says 172.21.15.1:55372) 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5867fc640 1 -- 172.21.15.1:0/1894115487 learned_addr learned my addr 172.21.15.1:0/1894115487 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758355390 0 0) 0x7ff5880d04a0 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff570003520 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5877fe640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5880cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2636880135 0 0) 0x7ff570003520 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 msgr2=0x7ff5880cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5880cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 >> v1:172.21.15.6:6789/0 conn(0x7ff5880b0080 legacy=0x7ff58815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff588072880 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7ff57c001670 con 0x7ff588071ce0 2024-05-09T06:31:55.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5877fe640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5880cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 >> v1:172.21.15.1:6789/0 conn(0x7ff588071ce0 legacy=0x7ff58815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff570004d40 0x7ff5700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5880725a0 con 0x7ff570003fd0 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff57000bb20 con 0x7ff570004d40 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff57000bd20 con 0x7ff58815f3f0 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5867fc640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff586ffd640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.845+0000 7ff5877fe640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff570004d40 0x7ff5700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 -- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 msgr2=0x7ff5700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff58815f3f0 0x7ff5700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 -- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff570004d40 msgr2=0x7ff5700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff570004d40 0x7ff5700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 -- 172.21.15.1:0/1894115487 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff570003520 con 0x7ff570003fd0 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.846+0000 7ff5867fc640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff570004800 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c210 tx=0x7ff570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56819ff156522fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c007d30 con 0x7ff570003fd0 2024-05-09T06:31:55.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c005af0 con 0x7ff570003fd0 2024-05-09T06:31:55.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff585ffb640 1 -- 172.21.15.1:0/1894115487 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c007070 con 0x7ff570003fd0 2024-05-09T06:31:55.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 -- 172.21.15.1:0/1894115487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 msgr2=0x7ff570004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 --2- 172.21.15.1:0/1894115487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff570004800 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c210 tx=0x7ff570005830 comp rx=0 tx=0).stop 2024-05-09T06:31:55.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff5877fe640 1 -- 172.21.15.1:0/1894115487 reap_dead start 2024-05-09T06:31:55.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 -- 172.21.15.1:0/1894115487 shutdown_connections 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 -- 172.21.15.1:0/1894115487 >> 172.21.15.1:0/1894115487 conn(0x7ff5880d0b50 msgr2=0x7ff5880d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 -- 172.21.15.1:0/1894115487 shutdown_connections 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.847+0000 7ff58d329640 1 -- 172.21.15.1:0/1894115487 wait complete. 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.848+0000 7ff58d329640 1 Processor -- start 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 -- start start 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 0x7ff5880c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5881600c0 0x7ff588160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff58815ff40 con 0x7ff570003fd0 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff58815e7c0 con 0x7ff588151080 2024-05-09T06:31:55.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff58d329640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff5880cbb60 con 0x7ff5881600c0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff586ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff586ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35460/0 (socket says 172.21.15.1:35460) 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff586ffd640 1 -- 172.21.15.1:0/586658286 learned_addr learned my addr 172.21.15.1:0/586658286 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff5877fe640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5881600c0 0x7ff588160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.849+0000 7ff5867fc640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 0x7ff5880c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 -- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5881600c0 msgr2=0x7ff588160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5881600c0 0x7ff588160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 -- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 msgr2=0x7ff5880c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 0x7ff5880c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 -- 172.21.15.1:0/586658286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff588151500 con 0x7ff570003fd0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff5867fc640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 0x7ff5880c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff586ffd640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff57800f7d0 tx=0x7ff57800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780050d0 con 0x7ff570003fd0 2024-05-09T06:31:55.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578018070 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5880c6e00 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800d6d0 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.850+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5880c73c0 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.851+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5880752b0 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.852+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff57800d870 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.852+0000 7ff56f7fe640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 0x7ff57406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.852+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7ff57808aa30 con 0x7ff570003fd0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.852+0000 7ff5867fc640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 0x7ff57406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.853+0000 7ff5867fc640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 0x7ff57406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff57c032000 tx=0x7ff57c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:55.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:55.857+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff578059550 con 0x7ff570003fd0 2024-05-09T06:31:56.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.117+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7ff5880caf60 con 0x7ff570003fd0 2024-05-09T06:31:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.392+0000 7ff56f7fe640 1 -- 172.21.15.1:0/586658286 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v240) ==== 169+0+0 (secure 0 0 0) 0x7ff578058e50 con 0x7ff570003fd0 2024-05-09T06:31:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 msgr2=0x7ff57406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 0x7ff57406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff57c032000 tx=0x7ff57c009030 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 msgr2=0x7ff5880caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff57800f7d0 tx=0x7ff57800fc90 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 shutdown_connections 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff57406be30 0x7ff57406e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5881600c0 0x7ff588160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff588151080 0x7ff5880c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 --2- 172.21.15.1:0/586658286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff570003fd0 0x7ff5880caa20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 >> 172.21.15.1:0/586658286 conn(0x7ff5880d0b50 msgr2=0x7ff5880d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 shutdown_connections 2024-05-09T06:31:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.394+0000 7ff58d329640 1 -- 172.21.15.1:0/586658286 wait complete. 2024-05-09T06:31:56.388 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-09T06:31:56.388 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-09T06:31:56.388 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:31:56.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:31:56.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.512+0000 7fc3e65ce640 1 Processor -- start 2024-05-09T06:31:56.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.512+0000 7fc3e65ce640 1 -- start start 2024-05-09T06:31:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e65ce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc3e01454b0 0x7fc3e00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e65ce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc3e00b4f70 0x7fc3e0070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e65ce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e006eb40 con 0x7fc3e00b2a60 2024-05-09T06:31:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e65ce640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e006ee20 con 0x7fc3e00b4f70 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e65ce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e006d8e0 con 0x7fc3e01454b0 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3e4b6b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc3e00b4f70 0x7fc3e0070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3d7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc3e00b2a60 0x7fc3e00b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55374/0 (socket says 172.21.15.1:55374) 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.513+0000 7fc3d7fff640 1 -- 172.21.15.1:0/4072785441 learned_addr learned my addr 172.21.15.1:0/4072785441 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3dffff640 1 --2- 172.21.15.1:0/4072785441 >> v2:172.21.15.6:3300/0 conn(0x7fc3e01454b0 0x7fc3e00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3e4b6b640 1 -- 172.21.15.1:0/4072785441 >> v2:172.21.15.6:3300/0 conn(0x7fc3e01454b0 msgr2=0x7fc3e00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3e4b6b640 1 --2- 172.21.15.1:0/4072785441 >> v2:172.21.15.6:3300/0 conn(0x7fc3e01454b0 0x7fc3e00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3e4b6b640 1 -- 172.21.15.1:0/4072785441 >> v1:172.21.15.1:6789/0 conn(0x7fc3e00b2a60 legacy=0x7fc3e00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3e4b6b640 1 -- 172.21.15.1:0/4072785441 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e006dbc0 con 0x7fc3e00b4f70 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3df7fe640 1 -- 172.21.15.1:0/4072785441 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831232608 0 0) 0x7fc3e006eb40 con 0x7fc3e00b2a60 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3e4b6b640 1 --2- 172.21.15.1:0/4072785441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b4f70 0x7fc3e0070c10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00689d0 tx=0x7fc3d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299680cd896b11dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.514+0000 7fc3df7fe640 1 -- 172.21.15.1:0/4072785441 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8007740 con 0x7fc3e00b4f70 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3df7fe640 1 -- 172.21.15.1:0/4072785441 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d80078e0 con 0x7fc3e00b4f70 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3df7fe640 1 -- 172.21.15.1:0/4072785441 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8007cb0 con 0x7fc3e00b4f70 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 -- 172.21.15.1:0/4072785441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b4f70 msgr2=0x7fc3e0070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/4072785441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b4f70 0x7fc3e0070c10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00689d0 tx=0x7fc3d802a680 comp rx=0 tx=0).stop 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 -- 172.21.15.1:0/4072785441 shutdown_connections 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/4072785441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b4f70 0x7fc3e0070c10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/4072785441 >> v2:172.21.15.6:3300/0 conn(0x7fc3e01454b0 0x7fc3e00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 -- 172.21.15.1:0/4072785441 >> 172.21.15.1:0/4072785441 conn(0x7fc3e00cc4b0 msgr2=0x7fc3e00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 -- 172.21.15.1:0/4072785441 shutdown_connections 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.515+0000 7fc3e65ce640 1 -- 172.21.15.1:0/4072785441 wait complete. 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.516+0000 7fc3e65ce640 1 Processor -- start 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.516+0000 7fc3e65ce640 1 -- start start 2024-05-09T06:31:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 0x7fc3e015b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3e01454b0 0x7fc3e013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e006e260 con 0x7fc3e00b2a60 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e006e0e0 con 0x7fc3e00b4f70 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3e65ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e0071150 con 0x7fc3e01454b0 2024-05-09T06:31:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35478/0 (socket says 172.21.15.1:35478) 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3dffff640 1 -- 172.21.15.1:0/715547808 learned_addr learned my addr 172.21.15.1:0/715547808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.517+0000 7fc3d7fff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 0x7fc3e015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3e4b6b640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3e01454b0 0x7fc3e013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 -- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3e01454b0 msgr2=0x7fc3e013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3e01454b0 0x7fc3e013c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 -- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 msgr2=0x7fc3e015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 0x7fc3e015b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 -- 172.21.15.1:0/715547808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e0139b70 con 0x7fc3e00b2a60 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3d7fff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 0x7fc3e015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dffff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000d620 tx=0x7fc3d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000b620 con 0x7fc3e00b2a60 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0019070 con 0x7fc3e00b2a60 2024-05-09T06:31:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e00abc40 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000b9e0 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.518+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e00ac200 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.519+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e00752b0 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.520+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3d000bb80 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.520+0000 7fc3dd7fa640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 0x7fc3c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.520+0000 7fc3d7fff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 0x7fc3c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.520+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7fc3d008b120 con 0x7fc3e00b2a60 2024-05-09T06:31:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.521+0000 7fc3d7fff640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 0x7fc3c006e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3e01f7180 tx=0x7fc3c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.525+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3d0059c40 con 0x7fc3e00b2a60 2024-05-09T06:31:56.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.708+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3e0072fa0 con 0x7fc3e00b2a60 2024-05-09T06:31:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.709+0000 7fc3dd7fa640 1 -- 172.21.15.1:0/715547808 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+13158 (secure 0 0 0) 0x7fc3d0059540 con 0x7fc3e00b2a60 2024-05-09T06:31:56.690 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:56.690 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":69,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:31:55.390465+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6835/531134827":"2024-05-10T06:31:52.385288+0000","172.21.15.6:6833/531134827":"2024-05-10T06:31:52.385288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:31:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 msgr2=0x7fc3c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 0x7fc3c006e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3e01f7180 tx=0x7fc3c8009040 comp rx=0 tx=0).stop 2024-05-09T06:31:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 msgr2=0x7fc3e0139630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000d620 tx=0x7fc3d000dae0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 shutdown_connections 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc3c006be30 0x7fc3c006e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc3e01454b0 0x7fc3e013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc3e00b4f70 0x7fc3e015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 --2- 172.21.15.1:0/715547808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3e00b2a60 0x7fc3e0139630 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 >> 172.21.15.1:0/715547808 conn(0x7fc3e00cc4b0 msgr2=0x7fc3e00b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 shutdown_connections 2024-05-09T06:31:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.711+0000 7fc3e65ce640 1 -- 172.21.15.1:0/715547808 wait complete. 2024-05-09T06:31:56.704 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:31:56.704 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:31:56.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.830+0000 7f4c8b948640 1 Processor -- start 2024-05-09T06:31:56.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.830+0000 7f4c8b948640 1 -- start start 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c8b948640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 0x7f4c8415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c8b948640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c8415cde0 0x7f4c84070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c8b948640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8406cf40 con 0x7f4c8415c4c0 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c8b948640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4c84156980 con 0x7f4c8415cde0 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c8b948640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8406e180 con 0x7f4c8406dd00 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c8415cde0 0x7f4c84070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c8415cde0 0x7f4c84070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35482/0 (socket says 172.21.15.1:35482) 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 -- 172.21.15.1:0/856826191 learned_addr learned my addr 172.21.15.1:0/856826191 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c88ee3640 1 --2- 172.21.15.1:0/856826191 >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 0x7f4c8415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 -- 172.21.15.1:0/856826191 >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 msgr2=0x7f4c8415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 --2- 172.21.15.1:0/856826191 >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 0x7f4c8415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 -- 172.21.15.1:0/856826191 >> v1:172.21.15.6:6789/0 conn(0x7f4c8415c4c0 legacy=0x7f4c840b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c89ee5640 1 -- 172.21.15.1:0/856826191 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8406d220 con 0x7f4c8415cde0 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.831+0000 7f4c88ee3640 1 --2- 172.21.15.1:0/856826191 >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 0x7f4c8415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.832+0000 7f4c89ee5640 1 --2- 172.21.15.1:0/856826191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415cde0 0x7f4c84070c10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4c840cd190 tx=0x7f4c7402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910f2f8db3bb87bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.832+0000 7f4c83fff640 1 -- 172.21.15.1:0/856826191 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74036070 con 0x7f4c8415cde0 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.832+0000 7f4c83fff640 1 -- 172.21.15.1:0/856826191 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c74031600 con 0x7f4c8415cde0 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.832+0000 7f4c83fff640 1 -- 172.21.15.1:0/856826191 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740319d0 con 0x7f4c8415cde0 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 -- 172.21.15.1:0/856826191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415cde0 msgr2=0x7f4c84070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 --2- 172.21.15.1:0/856826191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415cde0 0x7f4c84070c10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4c840cd190 tx=0x7f4c7402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 -- 172.21.15.1:0/856826191 shutdown_connections 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 --2- 172.21.15.1:0/856826191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415cde0 0x7f4c84070c10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 --2- 172.21.15.1:0/856826191 >> v2:172.21.15.6:3300/0 conn(0x7f4c8406dd00 0x7f4c8415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 -- 172.21.15.1:0/856826191 >> 172.21.15.1:0/856826191 conn(0x7f4c840cc4d0 msgr2=0x7f4c840cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 -- 172.21.15.1:0/856826191 shutdown_connections 2024-05-09T06:31:56.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.833+0000 7f4c8b948640 1 -- 172.21.15.1:0/856826191 wait complete. 2024-05-09T06:31:56.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.834+0000 7f4c8b948640 1 Processor -- start 2024-05-09T06:31:56.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.834+0000 7f4c8b948640 1 -- start start 2024-05-09T06:31:56.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c8406dd00 0x7f4c84155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 0x7f4c840b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8415d270 con 0x7f4c8415c4c0 2024-05-09T06:31:56.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8406d870 con 0x7f4c8415cde0 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c8b948640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c840b54f0 con 0x7f4c8406dd00 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c88ee3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c88ee3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35488/0 (socket says 172.21.15.1:35488) 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c88ee3640 1 -- 172.21.15.1:0/1153678842 learned_addr learned my addr 172.21.15.1:0/1153678842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c89ee5640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 0x7f4c840b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.835+0000 7f4c896e4640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c8406dd00 0x7f4c84155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 -- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c8406dd00 msgr2=0x7f4c84155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c8406dd00 0x7f4c84155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 -- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 msgr2=0x7f4c840b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 0x7f4c840b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 -- 172.21.15.1:0/1153678842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c840af900 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c89ee5640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 0x7f4c840b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c88ee3640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800c7f0 tx=0x7f4c7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78015070 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c78007a30 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c840afbe0 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78007e00 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.836+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c840b01a0 con 0x7f4c8415c4c0 2024-05-09T06:31:56.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.837+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c840752b0 con 0x7f4c8415c4c0 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.838+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c78018070 con 0x7f4c8415c4c0 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.838+0000 7f4c81ffb640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 0x7f4c7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.838+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f4c7808ace0 con 0x7f4c8415c4c0 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.838+0000 7f4c896e4640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 0x7f4c7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.839+0000 7f4c896e4640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 0x7f4c7006e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004ea0 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:56.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:56.843+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c78059800 con 0x7f4c8415c4c0 2024-05-09T06:31:57.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.025+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c84072880 con 0x7f4c8415c4c0 2024-05-09T06:31:57.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.025+0000 7f4c81ffb640 1 -- 172.21.15.1:0/1153678842 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f4c78059100 con 0x7f4c8415c4c0 2024-05-09T06:31:57.007 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 msgr2=0x7f4c7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 0x7f4c7006e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004ea0 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 msgr2=0x7f4c840b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800c7f0 tx=0x7f4c7800cbf0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 shutdown_connections 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c7006be30 0x7f4c7006e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c8415cde0 0x7f4c840b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c8415c4c0 0x7f4c840b09a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 --2- 172.21.15.1:0/1153678842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c8406dd00 0x7f4c84155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 >> 172.21.15.1:0/1153678842 conn(0x7f4c840cc4d0 msgr2=0x7f4c840cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:57.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 shutdown_connections 2024-05-09T06:31:57.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.027+0000 7f4c8b948640 1 -- 172.21.15.1:0/1153678842 wait complete. 2024-05-09T06:31:57.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-09T06:31:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.143+0000 7f1cde815640 1 Processor -- start 2024-05-09T06:31:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 -- start start 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1cd8161950 0x7f1cd80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd80b33c0 con 0x7f1cd80d4940 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd80d4dd0 con 0x7f1cd8161950 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.144+0000 7f1cde815640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b3660 con 0x7f1cd80d0f80 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1cd8161950 0x7f1cd80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1cd8161950 0x7f1cd80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35490/0 (socket says 172.21.15.1:35490) 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 -- 172.21.15.1:0/3714384870 learned_addr learned my addr 172.21.15.1:0/3714384870 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 -- 172.21.15.1:0/3714384870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 msgr2=0x7f1cd80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cdcdb2640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd80d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 -- 172.21.15.1:0/3714384870 >> v1:172.21.15.6:6789/0 conn(0x7f1cd80d4940 legacy=0x7f1cd80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.145+0000 7f1cd7fff640 1 -- 172.21.15.1:0/3714384870 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd80b3940 con 0x7f1cd8161950 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd80d2400 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009870 tx=0x7f1cc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790aafd0f51089d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cd77fe640 1 -- 172.21.15.1:0/3714384870 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1cd8161950 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cd77fe640 1 -- 172.21.15.1:0/3714384870 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8032440 con 0x7f1cd8161950 2024-05-09T06:31:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cd77fe640 1 -- 172.21.15.1:0/3714384870 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032810 con 0x7f1cd8161950 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 -- 172.21.15.1:0/3714384870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 msgr2=0x7f1cd80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd80d2400 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009870 tx=0x7f1cc802a210 comp rx=0 tx=0).stop 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 -- 172.21.15.1:0/3714384870 shutdown_connections 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 --2- 172.21.15.1:0/3714384870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd80d2400 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 -- 172.21.15.1:0/3714384870 >> 172.21.15.1:0/3714384870 conn(0x7f1cd80d3e20 msgr2=0x7f1cd80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:57.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 -- 172.21.15.1:0/3714384870 shutdown_connections 2024-05-09T06:31:57.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.146+0000 7f1cde815640 1 -- 172.21.15.1:0/3714384870 wait complete. 2024-05-09T06:31:57.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 Processor -- start 2024-05-09T06:31:57.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 -- start start 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 0x7f1cd814dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b52d0 con 0x7f1cd8161950 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b5150 con 0x7f1cd814db50 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.148+0000 7f1cde815640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b5450 con 0x7f1cd80d0f80 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35498/0 (socket says 172.21.15.1:35498) 2024-05-09T06:31:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 -- 172.21.15.1:0/254749061 learned_addr learned my addr 172.21.15.1:0/254749061 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cdcdb2640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 0x7f1cd814dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 -- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 msgr2=0x7f1cd806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 -- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 msgr2=0x7f1cd814dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 0x7f1cd814dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 -- 172.21.15.1:0/254749061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd80b6b50 con 0x7f1cd8161950 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cdcdb2640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 0x7f1cd814dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1ccffff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c410 tx=0x7f1cc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0017070 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc00035f0 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd80b91d0 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003a20 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.149+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd80b6dc0 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.150+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd80752b0 con 0x7f1cd8161950 2024-05-09T06:31:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.151+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cc001a070 con 0x7f1cd8161950 2024-05-09T06:31:57.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.151+0000 7f1cd57fa640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 0x7f1cb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:57.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.151+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f1cc008a110 con 0x7f1cd8161950 2024-05-09T06:31:57.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.151+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 0x7f1cb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:57.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.152+0000 7f1cd7fff640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 0x7f1cb806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.156+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1cc0058cc0 con 0x7f1cd8161950 2024-05-09T06:31:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:57.409+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f1cd8074650 con 0x7f1cd8161950 2024-05-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.397+0000 7f1cd57fa640 1 -- 172.21.15.1:0/254749061 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v242) ==== 169+0+0 (secure 0 0 0) 0x7f1cc00010c0 con 0x7f1cd8161950 2024-05-09T06:31:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 msgr2=0x7f1cb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 0x7f1cb806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 msgr2=0x7f1cd806c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c410 tx=0x7f1cc000c8d0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 shutdown_connections 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cb806be30 0x7f1cb806e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd814db50 0x7f1cd814dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd8161950 0x7f1cd806c9e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 --2- 172.21.15.1:0/254749061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd80d0f80 0x7f1cd806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 >> 172.21.15.1:0/254749061 conn(0x7f1cd80d3e20 msgr2=0x7f1cd80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.400+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 shutdown_connections 2024-05-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.401+0000 7f1cde815640 1 -- 172.21.15.1:0/254749061 wait complete. 2024-05-09T06:31:58.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:31:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.527+0000 7fad3c488640 1 Processor -- start 2024-05-09T06:31:58.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.527+0000 7fad3c488640 1 -- start start 2024-05-09T06:31:58.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3c488640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3c488640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3c488640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3415c920 con 0x7fad341605d0 2024-05-09T06:31:58.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3c488640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad3415fc00 con 0x7fad34074660 2024-05-09T06:31:58.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3c488640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad3415cba0 con 0x7fad340cd450 2024-05-09T06:31:58.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad39a23640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3aa25640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fad341605d0 0x7fad3415d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:58698/0 (socket says 172.21.15.1:58698) 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:51864/0 (socket says 172.21.15.1:51864) 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3aa25640 1 -- 172.21.15.1:0/1617196173 learned_addr learned my addr 172.21.15.1:0/1617196173 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad39a23640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:50564/0 (socket says 172.21.15.1:50564) 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 -- 172.21.15.1:0/1617196173 >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 msgr2=0x7fad34160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 --2- 172.21.15.1:0/1617196173 >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 -- 172.21.15.1:0/1617196173 >> v1:172.21.15.6:6789/0 conn(0x7fad341605d0 legacy=0x7fad3415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad3a224640 1 -- 172.21.15.1:0/1617196173 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3415ce80 con 0x7fad340cd450 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.528+0000 7fad39a23640 1 --2- 172.21.15.1:0/1617196173 >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.529+0000 7fad3a224640 1 --2- 172.21.15.1:0/1617196173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ca5f315735d0639 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:58.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.529+0000 7fad39222640 1 -- 172.21.15.1:0/1617196173 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30037070 con 0x7fad340cd450 2024-05-09T06:31:58.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.529+0000 7fad39222640 1 -- 172.21.15.1:0/1617196173 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30032440 con 0x7fad340cd450 2024-05-09T06:31:58.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.529+0000 7fad39222640 1 -- 172.21.15.1:0/1617196173 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30032810 con 0x7fad340cd450 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 -- 172.21.15.1:0/1617196173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 msgr2=0x7fad3415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 --2- 172.21.15.1:0/1617196173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a180 comp rx=0 tx=0).stop 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 -- 172.21.15.1:0/1617196173 shutdown_connections 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 --2- 172.21.15.1:0/1617196173 >> v2:172.21.15.6:3300/0 conn(0x7fad34074660 0x7fad34160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 --2- 172.21.15.1:0/1617196173 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad340cd450 0x7fad3415c3e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 -- 172.21.15.1:0/1617196173 >> 172.21.15.1:0/1617196173 conn(0x7fad340b1da0 msgr2=0x7fad340b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 -- 172.21.15.1:0/1617196173 shutdown_connections 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.530+0000 7fad3c488640 1 -- 172.21.15.1:0/1617196173 wait complete. 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.531+0000 7fad3c488640 1 Processor -- start 2024-05-09T06:31:58.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.531+0000 7fad3c488640 1 -- start start 2024-05-09T06:31:58.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad34074660 0x7fad340c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad341605d0 0x7fad340c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad3415dcb0 con 0x7fad340cd450 2024-05-09T06:31:58.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad340cd830 con 0x7fad34074660 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3c488640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad3415de30 con 0x7fad341605d0 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad39a23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad39a23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56178/0 (socket says 172.21.15.1:56178) 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad39a23640 1 -- 172.21.15.1:0/1250485917 learned_addr learned my addr 172.21.15.1:0/1250485917 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3aa25640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad341605d0 0x7fad340c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.532+0000 7fad3a224640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad34074660 0x7fad340c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 -- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad341605d0 msgr2=0x7fad340c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad341605d0 0x7fad340c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 -- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad34074660 msgr2=0x7fad340c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad34074660 0x7fad340c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 -- 172.21.15.1:0/1250485917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad340b2e50 con 0x7fad340cd450 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad39a23640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fad2400ebe0 tx=0x7fad2400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24017070 con 0x7fad340cd450 2024-05-09T06:31:58.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2400cc30 con 0x7fad340cd450 2024-05-09T06:31:58.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad34158e00 con 0x7fad340cd450 2024-05-09T06:31:58.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24013470 con 0x7fad340cd450 2024-05-09T06:31:58.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.533+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad341593c0 con 0x7fad340cd450 2024-05-09T06:31:58.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.534+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad340752b0 con 0x7fad340cd450 2024-05-09T06:31:58.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.535+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad2401a070 con 0x7fad340cd450 2024-05-09T06:31:58.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.535+0000 7fad2affd640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 0x7fad1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:58.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.535+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7fad2408af30 con 0x7fad340cd450 2024-05-09T06:31:58.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.535+0000 7fad3a224640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 0x7fad1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:58.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.536+0000 7fad3a224640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 0x7fad1806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad30034000 tx=0x7fad3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.539+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad24059410 con 0x7fad340cd450 2024-05-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.722+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fad340b61f0 con 0x7fad340cd450 2024-05-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.728+0000 7fad2affd640 1 -- 172.21.15.1:0/1250485917 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fad2405d710 con 0x7fad340cd450 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 msgr2=0x7fad1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 0x7fad1806e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad30034000 tx=0x7fad3002e000 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 msgr2=0x7fad34158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fad2400ebe0 tx=0x7fad2400c520 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 shutdown_connections 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1806be30 0x7fad1806e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad341605d0 0x7fad340c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340cd450 0x7fad34158480 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 --2- 172.21.15.1:0/1250485917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad34074660 0x7fad340c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 >> 172.21.15.1:0/1250485917 conn(0x7fad340b1da0 msgr2=0x7fad3415b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:58.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 shutdown_connections 2024-05-09T06:31:58.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:58.730+0000 7fad3c488640 1 -- 172.21.15.1:0/1250485917 wait complete. 2024-05-09T06:31:58.723 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:31:58.723 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:31:58.768 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:31:58.816 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:31:58.816 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:31:58.816 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:58.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:31:58.832 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:31:58.832 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:58.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:31:58.851 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:31:58.851 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:58.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:31:58.889 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:31:58.889 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:31:58.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:31:58.910 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:31:58.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.052+0000 7fe1a2688640 1 Processor -- start 2024-05-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.052+0000 7fe1a2688640 1 -- start start 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe19c1565e0 0x7fe19c074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe19c0b0170 0x7fe19c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe19c15fd30 con 0x7fe19c1565e0 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe19c074a70 con 0x7fe19c0b0170 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a2688640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c15cc40 con 0x7fe19c15c680 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe19c1565e0 0x7fe19c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe19c1565e0 0x7fe19c074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56194/0 (socket says 172.21.15.1:56194) 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 -- 172.21.15.1:0/1040214701 learned_addr learned my addr 172.21.15.1:0/1040214701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19b7fe640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe1a0c25640 1 --2- 172.21.15.1:0/1040214701 >> v2:172.21.15.6:3300/0 conn(0x7fe19c0b0170 0x7fe19c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 -- 172.21.15.1:0/1040214701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 msgr2=0x7fe19c0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c0afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 -- 172.21.15.1:0/1040214701 >> v2:172.21.15.6:3300/0 conn(0x7fe19c0b0170 msgr2=0x7fe19c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 --2- 172.21.15.1:0/1040214701 >> v2:172.21.15.6:3300/0 conn(0x7fe19c0b0170 0x7fe19c1601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.053+0000 7fe19bfff640 1 -- 172.21.15.1:0/1040214701 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c15cf20 con 0x7fe19c1565e0 2024-05-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.054+0000 7fe19bfff640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c1565e0 0x7fe19c074530 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0727b0 tx=0x7fe18402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80490167f0d4ad8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.054+0000 7fe19affd640 1 -- 172.21.15.1:0/1040214701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184037070 con 0x7fe19c1565e0 2024-05-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.054+0000 7fe19affd640 1 -- 172.21.15.1:0/1040214701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe184032440 con 0x7fe19c1565e0 2024-05-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.054+0000 7fe19affd640 1 -- 172.21.15.1:0/1040214701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184032810 con 0x7fe19c1565e0 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 -- 172.21.15.1:0/1040214701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c1565e0 msgr2=0x7fe19c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c1565e0 0x7fe19c074530 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0727b0 tx=0x7fe18402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 -- 172.21.15.1:0/1040214701 shutdown_connections 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 --2- 172.21.15.1:0/1040214701 >> v2:172.21.15.6:3300/0 conn(0x7fe19c0b0170 0x7fe19c1601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 --2- 172.21.15.1:0/1040214701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c1565e0 0x7fe19c074530 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 -- 172.21.15.1:0/1040214701 >> 172.21.15.1:0/1040214701 conn(0x7fe19c0b1fb0 msgr2=0x7fe19c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 -- 172.21.15.1:0/1040214701 shutdown_connections 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.055+0000 7fe1a2688640 1 -- 172.21.15.1:0/1040214701 wait complete. 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.056+0000 7fe1a2688640 1 Processor -- start 2024-05-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.056+0000 7fe1a2688640 1 -- start start 2024-05-09T06:31:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe19c1565e0 0x7fe19c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1569c0 con 0x7fe19c0b0170 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c15d4e0 con 0x7fe19c1565e0 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a2688640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c160700 con 0x7fe19c15c680 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56202/0 (socket says 172.21.15.1:56202) 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 -- 172.21.15.1:0/2662070655 learned_addr learned my addr 172.21.15.1:0/2662070655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe1a0c25640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19b7fe640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe19c1565e0 0x7fe19c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 -- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 msgr2=0x7fe19c14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c14d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 -- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe19c1565e0 msgr2=0x7fe19c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe19c1565e0 0x7fe19c157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 -- 172.21.15.1:0/2662070655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c0c5a60 con 0x7fe19c0b0170 2024-05-09T06:31:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.057+0000 7fe19bfff640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe18402a220 tx=0x7fe1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.058+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184041020 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.058+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1840306c0 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.058+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184037040 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.058+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c0c5d40 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.058+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c0c6300 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.059+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0752b0 con 0x7fe19c0b0170 2024-05-09T06:31:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.059+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe184044020 con 0x7fe19c0b0170 2024-05-09T06:31:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.060+0000 7fe198ff9640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 0x7fe17c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.060+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7fe1840ab530 con 0x7fe19c0b0170 2024-05-09T06:31:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.060+0000 7fe19b7fe640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 0x7fe17c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.060+0000 7fe19b7fe640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 0x7fe17c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe18c006fd0 tx=0x7fe18c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.064+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe18404b030 con 0x7fe19c0b0170 2024-05-09T06:31:59.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.285+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19c0cda50 con 0x7fe19c0b0170 2024-05-09T06:31:59.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.286+0000 7fe198ff9640 1 -- 172.21.15.1:0/2662070655 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3857 (secure 0 0 0) 0x7fe18407a050 con 0x7fe19c0b0170 2024-05-09T06:31:59.268 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:31:59.268 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5456,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1349743237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1349743237},{"type":"v1","addr":"172.21.15.6:6834","nonce":1349743237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3525746098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3525746098},{"type":"v1","addr":"172.21.15.6:6835","nonce":3525746098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6558,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/2580089044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2580089044},{"type":"v1","addr":"172.21.15.1:6836","nonce":2580089044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6561,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/3855066171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":3855066171},{"type":"v1","addr":"172.21.15.1:6837","nonce":3855066171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-05-09T06:31:59.268 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 39 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 msgr2=0x7fe17c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 0x7fe17c06e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe18c006fd0 tx=0x7fe18c008040 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 msgr2=0x7fe19c0c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe18402a220 tx=0x7fe1840046c0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 shutdown_connections 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe17c06be30 0x7fe17c06e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe19c15c680 0x7fe19c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe19c1565e0 0x7fe19c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 --2- 172.21.15.1:0/2662070655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe19c0b0170 0x7fe19c0c5410 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 >> 172.21.15.1:0/2662070655 conn(0x7fe19c0b1fb0 msgr2=0x7fe19c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 shutdown_connections 2024-05-09T06:31:59.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.288+0000 7fe1a2688640 1 -- 172.21.15.1:0/2662070655 wait complete. 2024-05-09T06:31:59.281 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:31:59.281 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:31:59.281 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:31:59.281 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:31:59.281 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:31:59.281 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:31:59.281 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:31:59.282 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:31:59.282 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:31:59.282 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:31:59.282 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:31:59.282 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:31:59.282 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:31:59.282 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:31:59.282 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:31:59.283 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:31:59.291 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:31:59.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:31:59.292 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:31:59.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:31:59.295 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:31:59.295 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:31:59.296 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:31:59.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:31:59.334 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:31:59.334 DEBUG:teuthology.parallel:result is None 2024-05-09T06:31:59.335 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:31:59.335 DEBUG:teuthology.parallel:result is None 2024-05-09T06:31:59.337 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:31:59.337 DEBUG:teuthology.parallel:result is None 2024-05-09T06:31:59.338 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:31:59.338 DEBUG:teuthology.parallel:result is None 2024-05-09T06:31:59.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:31:59.414 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:31:59.414 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:31:59.423 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:31:59.424 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:31:59.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.467+0000 7f9a5fa7b640 1 Processor -- start 2024-05-09T06:31:59.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.467+0000 7f9a5fa7b640 1 -- start start 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5fa7b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5fa7b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9a580d0f80 0x7f9a580d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5fa7b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a580b33c0 con 0x7f9a580d4940 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5fa7b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9a580d4dd0 con 0x7f9a580d0f80 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5fa7b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9a580b3660 con 0x7f9a58161950 2024-05-09T06:31:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5e018640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9a580d0f80 0x7f9a580d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5d817640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5d016640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9a580d4940 0x7f9a580cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:58722/0 (socket says 172.21.15.1:58722) 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.468+0000 7f9a5d016640 1 -- 172.21.15.1:0/165782308 learned_addr learned my addr 172.21.15.1:0/165782308 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5e018640 1 -- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 msgr2=0x7f9a580d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5e018640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5e018640 1 -- 172.21.15.1:0/165782308 >> v1:172.21.15.6:6789/0 conn(0x7f9a580d4940 legacy=0x7f9a580cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5e018640 1 -- 172.21.15.1:0/165782308 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a580b3940 con 0x7f9a580d0f80 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5c815640 1 -- 172.21.15.1:0/165782308 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3539233358 0 0) 0x7f9a580b33c0 con 0x7f9a580d4940 2024-05-09T06:31:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.469+0000 7f9a5d817640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:59.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5e018640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580d0f80 0x7f9a580d2940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b3590 tx=0x7f9a5002a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12127cb92f2b0caa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5c815640 1 -- 172.21.15.1:0/165782308 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5002ae60 con 0x7f9a580d0f80 2024-05-09T06:31:59.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5c815640 1 -- 172.21.15.1:0/165782308 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a50007740 con 0x7f9a580d0f80 2024-05-09T06:31:59.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5c815640 1 -- 172.21.15.1:0/165782308 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50007b10 con 0x7f9a580d0f80 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580d0f80 msgr2=0x7f9a580d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580d0f80 0x7f9a580d2940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a580b3590 tx=0x7f9a5002a640 comp rx=0 tx=0).stop 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/165782308 shutdown_connections 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580d0f80 0x7f9a580d2940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/165782308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/165782308 >> 172.21.15.1:0/165782308 conn(0x7f9a580d3e20 msgr2=0x7f9a580d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/165782308 shutdown_connections 2024-05-09T06:31:59.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.470+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/165782308 wait complete. 2024-05-09T06:31:59.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 Processor -- start 2024-05-09T06:31:59.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 -- start start 2024-05-09T06:31:59.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580b7960 0x7f9a58160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580b52d0 con 0x7f9a580d0f80 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580b5150 con 0x7f9a580b7960 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.472+0000 7f9a5fa7b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9a580b5450 con 0x7f9a58161950 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56238/0 (socket says 172.21.15.1:56238) 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d016640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5e018640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580b7960 0x7f9a58160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 -- 172.21.15.1:0/3114126887 learned_addr learned my addr 172.21.15.1:0/3114126887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 -- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 msgr2=0x7f9a580b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 -- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580b7960 msgr2=0x7f9a58160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:59.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580b7960 0x7f9a58160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 -- 172.21.15.1:0/3114126887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a580b7ea0 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d016640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5d817640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400e990 tx=0x7f9a5400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54017070 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5400c9e0 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400cdb0 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a580b8180 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.473+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a580b8740 con 0x7f9a580d0f80 2024-05-09T06:31:59.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.474+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a581f4e20 con 0x7f9a580d0f80 2024-05-09T06:31:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.475+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a5401a070 con 0x7f9a580d0f80 2024-05-09T06:31:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.476+0000 7f9a4e7fc640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.476+0000 7f9a5d016640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.476+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5335+0+0 (secure 0 0 0) 0x7f9a5408af30 con 0x7f9a580d0f80 2024-05-09T06:31:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.476+0000 7f9a5d016640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9a580cd930 tx=0x7f9a48007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:59.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.480+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a540593e0 con 0x7f9a580d0f80 2024-05-09T06:31:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:31:59.674+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9a580d21e0 con 0x7f9a580d0f80 2024-05-09T06:32:00.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.757+0000 7f9a4e7fc640 1 -- 172.21.15.1:0/3114126887 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7f9a5405d6e0 con 0x7f9a580d0f80 2024-05-09T06:32:00.739 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 msgr2=0x7f9a3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9a580cd930 tx=0x7f9a48007d50 comp rx=0 tx=0).stop 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 msgr2=0x7f9a581592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400e990 tx=0x7f9a5400ee50 comp rx=0 tx=0).stop 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 shutdown_connections 2024-05-09T06:32:00.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a580b7960 0x7f9a58160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a58161950 0x7f9a580b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 --2- 172.21.15.1:0/3114126887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a580d0f80 0x7f9a581592a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.760+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 >> 172.21.15.1:0/3114126887 conn(0x7f9a580d3e20 msgr2=0x7f9a580b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.761+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 shutdown_connections 2024-05-09T06:32:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.761+0000 7f9a5fa7b640 1 -- 172.21.15.1:0/3114126887 wait complete. 2024-05-09T06:32:00.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:32:00.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 Processor -- start 2024-05-09T06:32:00.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 -- start start 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 0x7f88d40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88d40d4940 0x7f88d40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d40b33c0 con 0x7f88d40d0f80 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f88d40d4dd0 con 0x7f88d40d4940 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.885+0000 7f88dacee640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f88d40b3660 con 0x7f88d4161950 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88d40d4940 0x7f88d40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d928b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f88d40d0f80 0x7f88d40cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38732/0 (socket says 172.21.15.1:38732) 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88d40d4940 0x7f88d40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56242/0 (socket says 172.21.15.1:56242) 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d928b640 1 -- 172.21.15.1:0/2643360160 learned_addr learned my addr 172.21.15.1:0/2643360160 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d8a8a640 1 --2- 172.21.15.1:0/2643360160 >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 0x7f88d40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 -- 172.21.15.1:0/2643360160 >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 msgr2=0x7f88d40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 --2- 172.21.15.1:0/2643360160 >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 0x7f88d40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 -- 172.21.15.1:0/2643360160 >> v1:172.21.15.1:6789/0 conn(0x7f88d40d0f80 legacy=0x7f88d40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d3fff640 1 -- 172.21.15.1:0/2643360160 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d40b3940 con 0x7f88d40d4940 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d37fe640 1 -- 172.21.15.1:0/2643360160 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652677161 0 0) 0x7f88d40b33c0 con 0x7f88d40d0f80 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.886+0000 7f88d8a8a640 1 --2- 172.21.15.1:0/2643360160 >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 0x7f88d40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88d3fff640 1 --2- 172.21.15.1:0/2643360160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d4940 0x7f88d40d2940 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f88d406b7f0 tx=0x7f88c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7f17b503e054d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88d37fe640 1 -- 172.21.15.1:0/2643360160 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007740 con 0x7f88d40d4940 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88d37fe640 1 -- 172.21.15.1:0/2643360160 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c40078e0 con 0x7f88d40d4940 2024-05-09T06:32:00.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88d37fe640 1 -- 172.21.15.1:0/2643360160 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007cb0 con 0x7f88d40d4940 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 -- 172.21.15.1:0/2643360160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d4940 msgr2=0x7f88d40d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 --2- 172.21.15.1:0/2643360160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d4940 0x7f88d40d2940 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f88d406b7f0 tx=0x7f88c402a680 comp rx=0 tx=0).stop 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 -- 172.21.15.1:0/2643360160 shutdown_connections 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 --2- 172.21.15.1:0/2643360160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d4940 0x7f88d40d2940 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 --2- 172.21.15.1:0/2643360160 >> v2:172.21.15.6:3300/0 conn(0x7f88d4161950 0x7f88d40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 -- 172.21.15.1:0/2643360160 >> 172.21.15.1:0/2643360160 conn(0x7f88d40d3e20 msgr2=0x7f88d40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 -- 172.21.15.1:0/2643360160 shutdown_connections 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.887+0000 7f88dacee640 1 -- 172.21.15.1:0/2643360160 wait complete. 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.889+0000 7f88dacee640 1 Processor -- start 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.889+0000 7f88dacee640 1 -- start start 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.889+0000 7f88dacee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 0x7f88d41f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88d40cc280 0x7f88d41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40b52d0 con 0x7f88d40d0f80 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40b5150 con 0x7f88d40cc280 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88d40b5450 con 0x7f88d4161950 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56248/0 (socket says 172.21.15.1:56248) 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 -- 172.21.15.1:0/134552824 learned_addr learned my addr 172.21.15.1:0/134552824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d928b640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88d40cc280 0x7f88d41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d3fff640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 0x7f88d41f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 -- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 msgr2=0x7f88d41f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 0x7f88d41f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 -- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88d40cc280 msgr2=0x7f88d41f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88d40cc280 0x7f88d41f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 -- 172.21.15.1:0/134552824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d40ccce0 con 0x7f88d40d0f80 2024-05-09T06:32:00.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d3fff640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 0x7f88d41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d8a8a640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00c7f0 tx=0x7f88cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc015070 con 0x7f88d40d0f80 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88cc007820 con 0x7f88d40d0f80 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc007bd0 con 0x7f88d40d0f80 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.890+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d40cc720 con 0x7f88d40d0f80 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.891+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41f52d0 con 0x7f88d40d0f80 2024-05-09T06:32:00.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.892+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d40752b0 con 0x7f88d40d0f80 2024-05-09T06:32:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.892+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88cc018070 con 0x7f88d40d0f80 2024-05-09T06:32:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.892+0000 7f88d17fa640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 0x7f88b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.892+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f88cc08ac60 con 0x7f88d40d0f80 2024-05-09T06:32:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.892+0000 7f88d3fff640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 0x7f88b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.893+0000 7f88d3fff640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 0x7f88b406e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034000 tx=0x7f88c402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:00.897+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f88cc059210 con 0x7f88d40d0f80 2024-05-09T06:32:01.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.148+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f88d40d21e0 con 0x7f88d40d0f80 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.148+0000 7f88d17fa640 1 -- 172.21.15.1:0/134552824 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f88cc05d510 con 0x7f88d40d0f80 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:32:01.130 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.134 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:32:01.135 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:32:01.139 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:32:01.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:32:01.142 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:32:01.145 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 msgr2=0x7f88b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 0x7f88b406e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034000 tx=0x7f88c402f000 comp rx=0 tx=0).stop 2024-05-09T06:32:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 msgr2=0x7f88d40cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00c7f0 tx=0x7f88cc00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 shutdown_connections 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b406be30 0x7f88b406e2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88d40cc280 0x7f88d41f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88d4161950 0x7f88d41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 --2- 172.21.15.1:0/134552824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88d40d0f80 0x7f88d40cbd40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 >> 172.21.15.1:0/134552824 conn(0x7f88d40d3e20 msgr2=0x7f88d40ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 shutdown_connections 2024-05-09T06:32:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.150+0000 7f88dacee640 1 -- 172.21.15.1:0/134552824 wait complete. 2024-05-09T06:32:01.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.271+0000 7fdf8d534640 1 Processor -- start 2024-05-09T06:32:01.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.271+0000 7fdf8d534640 1 -- start start 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf8d534640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf880cf180 0x7fdf880cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf8d534640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880ce0e0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf8d534640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880ce3c0 con 0x7fdf880cdab0 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf8d534640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880cdec0 con 0x7fdf880cf180 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf880cf180 0x7fdf880cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf880cf180 0x7fdf880cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56250/0 (socket says 172.21.15.1:56250) 2024-05-09T06:32:01.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf86ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdf8806e6d0 0x7fdf8806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38748/0 (socket says 172.21.15.1:38748) 2024-05-09T06:32:01.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 -- 172.21.15.1:0/238650952 learned_addr learned my addr 172.21.15.1:0/238650952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:01.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 -- 172.21.15.1:0/238650952 >> v1:172.21.15.6:6789/0 conn(0x7fdf880cdab0 legacy=0x7fdf880b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 -- 172.21.15.1:0/238650952 >> v1:172.21.15.1:6789/0 conn(0x7fdf8806e6d0 legacy=0x7fdf8806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf85ffb640 1 -- 172.21.15.1:0/238650952 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714063020 0 0) 0x7fdf880ce0e0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.272+0000 7fdf867fc640 1 -- 172.21.15.1:0/238650952 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880ce6a0 con 0x7fdf880cf180 2024-05-09T06:32:01.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.273+0000 7fdf867fc640 1 --2- 172.21.15.1:0/238650952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf880cf180 0x7fdf880cd570 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00bb10 tx=0x7fdf7c02c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88da8e72e6145e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.273+0000 7fdf85ffb640 1 -- 172.21.15.1:0/238650952 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c035070 con 0x7fdf880cf180 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.273+0000 7fdf85ffb640 1 -- 172.21.15.1:0/238650952 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c031830 con 0x7fdf880cf180 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.273+0000 7fdf85ffb640 1 -- 172.21.15.1:0/238650952 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c031c00 con 0x7fdf880cf180 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 -- 172.21.15.1:0/238650952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf880cf180 msgr2=0x7fdf880cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 --2- 172.21.15.1:0/238650952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf880cf180 0x7fdf880cd570 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00bb10 tx=0x7fdf7c02c9d0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 -- 172.21.15.1:0/238650952 shutdown_connections 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 --2- 172.21.15.1:0/238650952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf880cf180 0x7fdf880cd570 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 -- 172.21.15.1:0/238650952 >> 172.21.15.1:0/238650952 conn(0x7fdf880cc4b0 msgr2=0x7fdf880cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 -- 172.21.15.1:0/238650952 shutdown_connections 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.274+0000 7fdf8d534640 1 -- 172.21.15.1:0/238650952 wait complete. 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.275+0000 7fdf8d534640 1 Processor -- start 2024-05-09T06:32:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.275+0000 7fdf8d534640 1 -- start start 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf880cdab0 0x7fdf88153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 0x7fdf88155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88070ed0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880cec60 con 0x7fdf880cdab0 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf8d534640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdf881447c0 con 0x7fdf880cf180 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56266/0 (socket says 172.21.15.1:56266) 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 -- 172.21.15.1:0/2095118536 learned_addr learned my addr 172.21.15.1:0/2095118536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf867fc640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf880cdab0 0x7fdf88153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf877fe640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 0x7fdf88155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 -- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 msgr2=0x7fdf88155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 0x7fdf88155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 -- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf880cdab0 msgr2=0x7fdf88153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf880cdab0 0x7fdf88153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.276+0000 7fdf86ffd640 1 -- 172.21.15.1:0/2095118536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf88156870 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf877fe640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 0x7fdf88155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf86ffd640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c3f0 tx=0x7fdf7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780074e0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780047e0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf88156b50 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78004c10 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.277+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf88157110 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.278+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf880752b0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.278+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf78011070 con 0x7fdf8806e6d0 2024-05-09T06:32:01.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.279+0000 7fdf6f7fe640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 0x7fdf7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.279+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7fdf7808a8e0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.279+0000 7fdf867fc640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 0x7fdf7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.280+0000 7fdf867fc640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 0x7fdf7406e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c037000 tx=0x7fdf7c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.283+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf7801a2b0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.466+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf880b5ad0 con 0x7fdf8806e6d0 2024-05-09T06:32:01.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.467+0000 7fdf6f7fe640 1 -- 172.21.15.1:0/2095118536 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13033 (secure 0 0 0) 0x7fdf78059450 con 0x7fdf8806e6d0 2024-05-09T06:32:01.448 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:01.448 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:00.756700+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 msgr2=0x7fdf7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 0x7fdf7406e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c037000 tx=0x7fdf7c03c040 comp rx=0 tx=0).stop 2024-05-09T06:32:01.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 msgr2=0x7fdf8813ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c3f0 tx=0x7fdf7800c8b0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 shutdown_connections 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7406be30 0x7fdf7406e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf880cf180 0x7fdf88155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf880cdab0 0x7fdf88153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 --2- 172.21.15.1:0/2095118536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf8806e6d0 0x7fdf8813ce30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 >> 172.21.15.1:0/2095118536 conn(0x7fdf880cc4b0 msgr2=0x7fdf8806eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 shutdown_connections 2024-05-09T06:32:01.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.469+0000 7fdf8d534640 1 -- 172.21.15.1:0/2095118536 wait complete. 2024-05-09T06:32:01.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:32:01.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:32:01.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 Processor -- start 2024-05-09T06:32:01.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 -- start start 2024-05-09T06:32:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 0x7f27c015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c0161fe0 con 0x7f27c00b56d0 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f27c015f580 con 0x7f27c00d1c70 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.587+0000 7f27c721c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27c01622c0 con 0x7f27c015e190 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27bffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c4fb8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 0x7f27c015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27bffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:50670/0 (socket says 172.21.15.1:50670) 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c57b9640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f27c00b56d0 0x7f27c00b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:58730/0 (socket says 172.21.15.1:58730) 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27bffff640 1 -- 172.21.15.1:0/3704884494 learned_addr learned my addr 172.21.15.1:0/3704884494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c4fb8640 1 -- 172.21.15.1:0/3704884494 >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 msgr2=0x7f27c00d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c4fb8640 1 --2- 172.21.15.1:0/3704884494 >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c4fb8640 1 -- 172.21.15.1:0/3704884494 >> v1:172.21.15.6:6789/0 conn(0x7f27c00b56d0 legacy=0x7f27c00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27c4fb8640 1 -- 172.21.15.1:0/3704884494 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c01625a0 con 0x7f27c015e190 2024-05-09T06:32:01.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.588+0000 7f27bffff640 1 --2- 172.21.15.1:0/3704884494 >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:32:01.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.589+0000 7f27c4fb8640 1 --2- 172.21.15.1:0/3704884494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 0x7f27c015f780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d90 tx=0x7f27b002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea37dd784aa150f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.589+0000 7f27bf7fe640 1 -- 172.21.15.1:0/3704884494 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0007740 con 0x7f27c015e190 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.589+0000 7f27bf7fe640 1 -- 172.21.15.1:0/3704884494 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b00078e0 con 0x7f27c015e190 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.589+0000 7f27bf7fe640 1 -- 172.21.15.1:0/3704884494 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0007cb0 con 0x7f27c015e190 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 -- 172.21.15.1:0/3704884494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 msgr2=0x7f27c015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 --2- 172.21.15.1:0/3704884494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 0x7f27c015f780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d90 tx=0x7f27b002a660 comp rx=0 tx=0).stop 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 -- 172.21.15.1:0/3704884494 shutdown_connections 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 --2- 172.21.15.1:0/3704884494 >> v2:172.21.15.6:3300/0 conn(0x7f27c00d1c70 0x7f27c00d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 --2- 172.21.15.1:0/3704884494 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c015e190 0x7f27c015f780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 -- 172.21.15.1:0/3704884494 >> 172.21.15.1:0/3704884494 conn(0x7f27c0073340 msgr2=0x7f27c009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 -- 172.21.15.1:0/3704884494 shutdown_connections 2024-05-09T06:32:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.590+0000 7f27c721c640 1 -- 172.21.15.1:0/3704884494 wait complete. 2024-05-09T06:32:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.591+0000 7f27c721c640 1 Processor -- start 2024-05-09T06:32:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.591+0000 7f27c721c640 1 -- start start 2024-05-09T06:32:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c00b56d0 0x7f27c0152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c015e190 0x7f27c00b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27c00d4420 con 0x7f27c00d1c70 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0161c60 con 0x7f27c015e190 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c721c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27c00d45a0 con 0x7f27c00b56d0 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56268/0 (socket says 172.21.15.1:56268) 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 -- 172.21.15.1:0/1111186812 learned_addr learned my addr 172.21.15.1:0/1111186812 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c4fb8640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c00b56d0 0x7f27c0152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27c57b9640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c015e190 0x7f27c00b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 -- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c00b56d0 msgr2=0x7f27c0152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c00b56d0 0x7f27c0152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 -- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c015e190 msgr2=0x7f27c00b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c015e190 0x7f27c00b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 -- 172.21.15.1:0/1111186812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c00b1e20 con 0x7f27c00d1c70 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.592+0000 7f27bffff640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f27a800e9b0 tx=0x7f27a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.593+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8017070 con 0x7f27c00d1c70 2024-05-09T06:32:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.593+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a800ca00 con 0x7f27c00d1c70 2024-05-09T06:32:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.593+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a800cdf0 con 0x7f27c00d1c70 2024-05-09T06:32:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.593+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c00b2100 con 0x7f27c00d1c70 2024-05-09T06:32:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.593+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c00cbea0 con 0x7f27c00d1c70 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.594+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c00752b0 con 0x7f27c00d1c70 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.594+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27a801a070 con 0x7f27c00d1c70 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.595+0000 7f27bd7fa640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.595+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f27a808aee0 con 0x7f27c00d1c70 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.595+0000 7f27c4fb8640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.595+0000 7f27c4fb8640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 0x7f27a006e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f27b0034000 tx=0x7f27b002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.599+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f27a8059400 con 0x7f27c00d1c70 2024-05-09T06:32:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.785+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27c008e1c0 con 0x7f27c00d1c70 2024-05-09T06:32:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.785+0000 7f27bd7fa640 1 -- 172.21.15.1:0/1111186812 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f27a805d700 con 0x7f27c00d1c70 2024-05-09T06:32:01.767 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 msgr2=0x7f27a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 0x7f27a006e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f27b0034000 tx=0x7f27b002f000 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 msgr2=0x7f27c00c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f27a800e9b0 tx=0x7f27a800ee70 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 shutdown_connections 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f27a006be30 0x7f27a006e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c015e190 0x7f27c00b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27c00d1c70 0x7f27c00c8070 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 --2- 172.21.15.1:0/1111186812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27c00b56d0 0x7f27c0152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 >> 172.21.15.1:0/1111186812 conn(0x7f27c0073340 msgr2=0x7f27c006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 shutdown_connections 2024-05-09T06:32:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.787+0000 7f27c721c640 1 -- 172.21.15.1:0/1111186812 wait complete. 2024-05-09T06:32:01.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.904+0000 7f885d7d5640 1 Processor -- start 2024-05-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.904+0000 7f885d7d5640 1 -- start start 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f885d7d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 0x7f8858074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f885d7d5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f885815c680 0x7f88580afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f885d7d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8858074a70 con 0x7f88580b0170 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f885d7d5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f885815fd30 con 0x7f885815c680 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f885d7d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f885815cc40 con 0x7f88581565e0 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f885815c680 0x7f88580afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f88577fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f88580b0170 0x7f88581601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38758/0 (socket says 172.21.15.1:38758) 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f885815c680 0x7f88580afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56282/0 (socket says 172.21.15.1:56282) 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f88577fe640 1 -- 172.21.15.1:0/2411860613 learned_addr learned my addr 172.21.15.1:0/2411860613 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f8856ffd640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 0x7f8858074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 -- 172.21.15.1:0/2411860613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 msgr2=0x7f8858074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 0x7f8858074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 -- 172.21.15.1:0/2411860613 >> v1:172.21.15.1:6789/0 conn(0x7f88580b0170 legacy=0x7f88581601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f884e7fc640 1 -- 172.21.15.1:0/2411860613 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885815cf20 con 0x7f885815c680 2024-05-09T06:32:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.905+0000 7f8856ffd640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 0x7f8858074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.906+0000 7f884e7fc640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f885815c680 0x7f88580afc30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f88400097a0 tx=0x7f884002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51af03782b5ec312 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.906+0000 7f88567fc640 1 -- 172.21.15.1:0/2411860613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840037070 con 0x7f885815c680 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.906+0000 7f88567fc640 1 -- 172.21.15.1:0/2411860613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8840032440 con 0x7f885815c680 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.906+0000 7f88567fc640 1 -- 172.21.15.1:0/2411860613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840032810 con 0x7f885815c680 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 -- 172.21.15.1:0/2411860613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f885815c680 msgr2=0x7f88580afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f885815c680 0x7f88580afc30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f88400097a0 tx=0x7f884002a180 comp rx=0 tx=0).stop 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 -- 172.21.15.1:0/2411860613 shutdown_connections 2024-05-09T06:32:01.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f885815c680 0x7f88580afc30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 --2- 172.21.15.1:0/2411860613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88581565e0 0x7f8858074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 -- 172.21.15.1:0/2411860613 >> 172.21.15.1:0/2411860613 conn(0x7f88580b1fb0 msgr2=0x7f88580b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 -- 172.21.15.1:0/2411860613 shutdown_connections 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.907+0000 7f885d7d5640 1 -- 172.21.15.1:0/2411860613 wait complete. 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.908+0000 7f885d7d5640 1 Processor -- start 2024-05-09T06:32:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.908+0000 7f885d7d5640 1 -- start start 2024-05-09T06:32:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88580b0170 0x7f8858156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 0x7f885815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88581569c0 con 0x7f88581565e0 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f885815d4e0 con 0x7f885815c680 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f885d7d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88581608d0 con 0x7f88580b0170 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56298/0 (socket says 172.21.15.1:56298) 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 -- 172.21.15.1:0/1760340521 learned_addr learned my addr 172.21.15.1:0/1760340521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f8856ffd640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88580b0170 0x7f8858156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f88577fe640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 0x7f885815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 -- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88580b0170 msgr2=0x7f8858156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88580b0170 0x7f8858156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 -- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 msgr2=0x7f885815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 0x7f885815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 -- 172.21.15.1:0/1760340521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858157080 con 0x7f88581565e0 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f88577fe640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 0x7f885815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.909+0000 7f884e7fc640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f88400098c0 tx=0x7f88400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.910+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840037040 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.910+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8840030770 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.910+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581f5210 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.910+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840030b40 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.910+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581f57d0 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.911+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88580752b0 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.911+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8840044020 con 0x7f88581565e0 2024-05-09T06:32:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.912+0000 7f884ffff640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 0x7f883806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:01.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.912+0000 7f8856ffd640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 0x7f883806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:01.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.912+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7f88400ab220 con 0x7f88581565e0 2024-05-09T06:32:01.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.912+0000 7f8856ffd640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 0x7f883806e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8848008850 tx=0x7f8848008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:01.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:01.916+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8840032e20 con 0x7f88581565e0 2024-05-09T06:32:02.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.101+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f88580af5b0 con 0x7f88581565e0 2024-05-09T06:32:02.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.101+0000 7f884ffff640 1 -- 172.21.15.1:0/1760340521 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f8840079d90 con 0x7f88581565e0 2024-05-09T06:32:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:02.083 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:32:02.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 msgr2=0x7f883806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 0x7f883806e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8848008850 tx=0x7f8848008040 comp rx=0 tx=0).stop 2024-05-09T06:32:02.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 msgr2=0x7f88580998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f88400098c0 tx=0x7f88400046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 shutdown_connections 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f883806be30 0x7f883806e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f885815c680 0x7f885815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88581565e0 0x7f88580998e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 --2- 172.21.15.1:0/1760340521 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88580b0170 0x7f8858156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 >> 172.21.15.1:0/1760340521 conn(0x7f88580b1fb0 msgr2=0x7f88581601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 shutdown_connections 2024-05-09T06:32:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.103+0000 7f885d7d5640 1 -- 172.21.15.1:0/1760340521 wait complete. 2024-05-09T06:32:02.095 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:32:02.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:32:02.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.221+0000 7fcaf5d63640 1 Processor -- start 2024-05-09T06:32:02.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.221+0000 7fcaf5d63640 1 -- start start 2024-05-09T06:32:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaf5d63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcaf0145880 0x7fcaf015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaf5d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcaf00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaf5d63640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf015d470 con 0x7fcaf00ced00 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaf5d63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf00cf160 con 0x7fcaf0145880 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaf5d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00b3cc0 con 0x7fcaf00b3820 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaeffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcaf00ced00 0x7fcaf015fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38768/0 (socket says 172.21.15.1:38768) 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.222+0000 7fcaeffff640 1 -- 172.21.15.1:0/505717962 learned_addr learned my addr 172.21.15.1:0/505717962 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaef7fe640 1 --2- 172.21.15.1:0/505717962 >> v2:172.21.15.6:3300/0 conn(0x7fcaf0145880 0x7fcaf015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcaf00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1791995384 0 0) 0x7fcaf015d470 con 0x7fcaf00ced00 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad8003520 con 0x7fcaf00ced00 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 554998569 0 0) 0x7fcad8003520 con 0x7fcaf00ced00 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 msgr2=0x7fcaf00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcaf00ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 >> v2:172.21.15.6:3300/0 conn(0x7fcaf0145880 msgr2=0x7fcaf015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 --2- 172.21.15.1:0/505717962 >> v2:172.21.15.6:3300/0 conn(0x7fcaf0145880 0x7fcaf015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00b3fa0 con 0x7fcaf00ced00 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fcae80019f0 con 0x7fcaf00ced00 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 >> v1:172.21.15.1:6789/0 conn(0x7fcaf00ced00 legacy=0x7fcaf015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcad80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcad8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcad8004d40 0x7fcad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00b3cc0 con 0x7fcad8003f30 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcad800b9c0 con 0x7fcad8004d40 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcad800bb40 con 0x7fcaf00b3820 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcad8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaef7fe640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcad80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaeffff640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcad8004d40 0x7fcad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaeeffd640 1 -- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 msgr2=0x7fcad80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:02.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.223+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00b3820 0x7fcad80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaeeffd640 1 -- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcad8004d40 msgr2=0x7fcad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcad8004d40 0x7fcad80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaeeffd640 1 -- 172.21.15.1:0/505717962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8003520 con 0x7fcad8003f30 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcad8004800 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009ac0 tx=0x7fcad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426b59f022122df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4036070 con 0x7fcad8003f30 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4031400 con 0x7fcad8003f30 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.224+0000 7fcaee7fc640 1 -- 172.21.15.1:0/505717962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4009690 con 0x7fcad8003f30 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 -- 172.21.15.1:0/505717962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 msgr2=0x7fcad8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/505717962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcad8004800 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009ac0 tx=0x7fcad8005830 comp rx=0 tx=0).stop 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaeffff640 1 -- 172.21.15.1:0/505717962 reap_dead start 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 -- 172.21.15.1:0/505717962 shutdown_connections 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 -- 172.21.15.1:0/505717962 >> 172.21.15.1:0/505717962 conn(0x7fcaf006b3c0 msgr2=0x7fcaf006b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 -- 172.21.15.1:0/505717962 shutdown_connections 2024-05-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.225+0000 7fcaf5d63640 1 -- 172.21.15.1:0/505717962 wait complete. 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.226+0000 7fcaf5d63640 1 Processor -- start 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.226+0000 7fcaf5d63640 1 -- start start 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00ad300 0x7fcaf00a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf00a99c0 0x7fcaf00aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00b4840 con 0x7fcad8003f30 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00b4560 con 0x7fcaf00a99c0 2024-05-09T06:32:02.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaf5d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00cfee0 con 0x7fcaf00ad300 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56312/0 (socket says 172.21.15.1:56312) 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.227+0000 7fcaef7fe640 1 -- 172.21.15.1:0/704050261 learned_addr learned my addr 172.21.15.1:0/704050261 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaeffff640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf00a99c0 0x7fcaf00aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 -- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00ad300 msgr2=0x7fcaf00a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00ad300 0x7fcaf00a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 -- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf00a99c0 msgr2=0x7fcaf00aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf00a99c0 0x7fcaf00aacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 -- 172.21.15.1:0/704050261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00ab990 con 0x7fcad8003f30 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaef7fe640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcae0011aa0 tx=0x7fcae0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae000fb80 con 0x7fcad8003f30 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae000fd20 con 0x7fcad8003f30 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf00abc70 con 0x7fcad8003f30 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0014660 con 0x7fcad8003f30 2024-05-09T06:32:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.228+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01f1530 con 0x7fcad8003f30 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.229+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00a9dd0 con 0x7fcad8003f30 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.229+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcae0012040 con 0x7fcad8003f30 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.230+0000 7fcad7fff640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 0x7fcadc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.230+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5249+0+0 (secure 0 0 0) 0x7fcae008b3f0 con 0x7fcad8003f30 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.230+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 0x7fcadc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.230+0000 7fcaeeffd640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 0x7fcadc06e320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcae4033000 tx=0x7fcae402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:02.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.234+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcaf00a9dd0 con 0x7fcad8003f30 2024-05-09T06:32:02.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:02.418+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcaf0075320 con 0x7fcad8003f30 2024-05-09T06:32:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.411+0000 7fcad7fff640 1 -- 172.21.15.1:0/704050261 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7fcae0019030 con 0x7fcad8003f30 2024-05-09T06:32:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:32:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 msgr2=0x7fcadc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 0x7fcadc06e320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcae4033000 tx=0x7fcae402e000 comp rx=0 tx=0).stop 2024-05-09T06:32:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 msgr2=0x7fcaf01f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcae0011aa0 tx=0x7fcae0011f60 comp rx=0 tx=0).stop 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 shutdown_connections 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcadc06be60 0x7fcadc06e320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf00a99c0 0x7fcaf00aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcaf00ad300 0x7fcaf00a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 --2- 172.21.15.1:0/704050261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcad8003f30 0x7fcaf01f0ff0 secure :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcae0011aa0 tx=0x7fcae0011f60 comp rx=0 tx=0).stop 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.415+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 >> 172.21.15.1:0/704050261 conn(0x7fcaf006b3c0 msgr2=0x7fcaf015d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.416+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 shutdown_connections 2024-05-09T06:32:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.416+0000 7fcaf5d63640 1 -- 172.21.15.1:0/704050261 wait complete. 2024-05-09T06:32:03.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:32:03.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.550+0000 7fd09bb29640 1 Processor -- start 2024-05-09T06:32:03.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.550+0000 7fd09bb29640 1 -- start start 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.551+0000 7fd09bb29640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd09415a760 0x7fd09406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.551+0000 7fd09bb29640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.551+0000 7fd09bb29640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0940b32b0 con 0x7fd0940b2a60 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.551+0000 7fd09bb29640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd0940b3ec0 con 0x7fd09415a760 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.551+0000 7fd09bb29640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0940b3590 con 0x7fd09406ecd0 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd09a0c6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0990c4640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd0940b2a60 0x7fd09406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:58742/0 (socket says 172.21.15.1:58742) 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0990c4640 1 -- 172.21.15.1:0/30820480 learned_addr learned my addr 172.21.15.1:0/30820480 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:03.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0998c5640 1 --2- 172.21.15.1:0/30820480 >> v2:172.21.15.6:3300/0 conn(0x7fd09415a760 0x7fd09406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0998c5640 1 -- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 msgr2=0x7fd094070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0998c5640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0998c5640 1 -- 172.21.15.1:0/30820480 >> v1:172.21.15.6:6789/0 conn(0x7fd0940b2a60 legacy=0x7fd09406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd0998c5640 1 -- 172.21.15.1:0/30820480 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0940b3870 con 0x7fd09415a760 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.552+0000 7fd09a0c6640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd0998c5640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd09415a760 0x7fd09406cd70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0097a0 tx=0x7fd08c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7bde7d3ea9fea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:03.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd0988c3640 1 -- 172.21.15.1:0/30820480 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c037070 con 0x7fd09415a760 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd0988c3640 1 -- 172.21.15.1:0/30820480 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08c02ae50 con 0x7fd09415a760 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd0988c3640 1 -- 172.21.15.1:0/30820480 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c032660 con 0x7fd09415a760 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 -- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd09415a760 msgr2=0x7fd09406cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd09415a760 0x7fd09406cd70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0097a0 tx=0x7fd08c02a100 comp rx=0 tx=0).stop 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 -- 172.21.15.1:0/30820480 shutdown_connections 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 --2- 172.21.15.1:0/30820480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd09415a760 0x7fd09406cd70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.553+0000 7fd09bb29640 1 -- 172.21.15.1:0/30820480 >> 172.21.15.1:0/30820480 conn(0x7fd0940b1760 msgr2=0x7fd0940b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.554+0000 7fd09bb29640 1 -- 172.21.15.1:0/30820480 shutdown_connections 2024-05-09T06:32:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.554+0000 7fd09bb29640 1 -- 172.21.15.1:0/30820480 wait complete. 2024-05-09T06:32:03.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 Processor -- start 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 -- start start 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0940b2a60 0x7fd0941405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0940b3040 con 0x7fd09415a760 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd0940b2ec0 con 0x7fd0940b2a60 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.555+0000 7fd09bb29640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0940cc4f0 con 0x7fd09406ecd0 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56324/0 (socket says 172.21.15.1:56324) 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 -- 172.21.15.1:0/971843254 learned_addr learned my addr 172.21.15.1:0/971843254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd0998c5640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd0990c4640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0940b2a60 0x7fd0941405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 -- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 msgr2=0x7fd094153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 -- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0940b2a60 msgr2=0x7fd0941405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0940b2a60 0x7fd0941405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 -- 172.21.15.1:0/971843254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09415c4f0 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09a0c6640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd08400efc0 tx=0x7fd084009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084018030 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08400cde0 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084003bb0 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.556+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09415c210 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.557+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0941410b0 con 0x7fd09415a760 2024-05-09T06:32:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.557+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0940752b0 con 0x7fd09415a760 2024-05-09T06:32:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.558+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd084005d10 con 0x7fd09415a760 2024-05-09T06:32:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.558+0000 7fd0927fc640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 0x7fd08006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.559+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5619+0+0 (secure 0 0 0) 0x7fd08408b960 con 0x7fd09415a760 2024-05-09T06:32:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.559+0000 7fd0998c5640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 0x7fd08006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.559+0000 7fd0998c5640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 0x7fd08006e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd08c034000 tx=0x7fd08c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.563+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd084022070 con 0x7fd09415a760 2024-05-09T06:32:03.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:03.746+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd0940cf190 con 0x7fd09415a760 2024-05-09T06:32:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.418+0000 7fd0927fc640 1 -- 172.21.15.1:0/971843254 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7fd084010040 con 0x7fd09415a760 2024-05-09T06:32:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 msgr2=0x7fd08006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 0x7fd08006e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd08c034000 tx=0x7fd08c02e000 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 msgr2=0x7fd09415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd08400efc0 tx=0x7fd084009f90 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 shutdown_connections 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd08006be30 0x7fd08006e2f0 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd08c034000 tx=0x7fd08c02e000 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd09415a760 0x7fd09415b910 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0940b2a60 0x7fd0941405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 --2- 172.21.15.1:0/971843254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd09406ecd0 0x7fd094153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.424+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 >> 172.21.15.1:0/971843254 conn(0x7fd0940b1760 msgr2=0x7fd0940aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.425+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 shutdown_connections 2024-05-09T06:32:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.425+0000 7fd09bb29640 1 -- 172.21.15.1:0/971843254 wait complete. 2024-05-09T06:32:04.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:32:04.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.562+0000 7f77f78c1640 1 Processor -- start 2024-05-09T06:32:04.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.562+0000 7f77f78c1640 1 -- start start 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f78c1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f77f00c41e0 0x7f77f00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f78c1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77f006b1e0 0x7f77f00cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f78c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f0072050 con 0x7f77f00d06f0 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f78c1640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f77f0072330 con 0x7f77f006b1e0 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f78c1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f77f0072610 con 0x7f77f00c41e0 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77f006b1e0 0x7f77f00cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77f006b1e0 0x7f77f00cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56330/0 (socket says 172.21.15.1:56330) 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1489099468 learned_addr learned my addr 172.21.15.1:0/1489099468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:04.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f4e5c640 1 --2- 172.21.15.1:0/1489099468 >> v2:172.21.15.6:3300/0 conn(0x7f77f00c41e0 0x7f77f00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1489099468 >> v2:172.21.15.6:3300/0 conn(0x7f77f00c41e0 msgr2=0x7f77f00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:04.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 --2- 172.21.15.1:0/1489099468 >> v2:172.21.15.6:3300/0 conn(0x7f77f00c41e0 0x7f77f00ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1489099468 >> v1:172.21.15.6:6789/0 conn(0x7f77f00d06f0 legacy=0x7f77f00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.563+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1489099468 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00728f0 con 0x7f77f006b1e0 2024-05-09T06:32:04.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.564+0000 7f77f5e5e640 1 --2- 172.21.15.1:0/1489099468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f006b1e0 0x7f77f00cee10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f77f00b0620 tx=0x7f77e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a9f133a58d6935 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:04.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.564+0000 7f77e7fff640 1 -- 172.21.15.1:0/1489099468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8036070 con 0x7f77f006b1e0 2024-05-09T06:32:04.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.564+0000 7f77e7fff640 1 -- 172.21.15.1:0/1489099468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e8031600 con 0x7f77f006b1e0 2024-05-09T06:32:04.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.564+0000 7f77e7fff640 1 -- 172.21.15.1:0/1489099468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e80319d0 con 0x7f77f006b1e0 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 -- 172.21.15.1:0/1489099468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f006b1e0 msgr2=0x7f77f00cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1489099468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f006b1e0 0x7f77f00cee10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f77f00b0620 tx=0x7f77e802a730 comp rx=0 tx=0).stop 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 -- 172.21.15.1:0/1489099468 shutdown_connections 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1489099468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f006b1e0 0x7f77f00cee10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1489099468 >> v2:172.21.15.6:3300/0 conn(0x7f77f00c41e0 0x7f77f00ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 -- 172.21.15.1:0/1489099468 >> 172.21.15.1:0/1489099468 conn(0x7f77f00d0b50 msgr2=0x7f77f00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 -- 172.21.15.1:0/1489099468 shutdown_connections 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.565+0000 7f77f78c1640 1 -- 172.21.15.1:0/1489099468 wait complete. 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.566+0000 7f77f78c1640 1 Processor -- start 2024-05-09T06:32:04.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.566+0000 7f77f78c1640 1 -- start start 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 0x7f77f014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77f00c41e0 0x7f77f00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00ccf70 con 0x7f77f00d06f0 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00ccdf0 con 0x7f77f00c41e0 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f78c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77f00cd0f0 con 0x7f77f006b1e0 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56334/0 (socket says 172.21.15.1:56334) 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1069757541 learned_addr learned my addr 172.21.15.1:0/1069757541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f4e5c640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77f00c41e0 0x7f77f00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 msgr2=0x7f77f014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f565d640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 0x7f77f014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 0x7f77f014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77f00c41e0 msgr2=0x7f77f00c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77f00c41e0 0x7f77f00c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f5e5e640 1 -- 172.21.15.1:0/1069757541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00c7da0 con 0x7f77f00d06f0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.567+0000 7f77f565d640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 0x7f77f014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77f5e5e640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009f40 tx=0x7f77e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8036050 con 0x7f77f00d06f0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e8042070 con 0x7f77f00d06f0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f00c8080 con 0x7f77f00d06f0 2024-05-09T06:32:04.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8009510 con 0x7f77f00d06f0 2024-05-09T06:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.568+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f014f950 con 0x7f77f00d06f0 2024-05-09T06:32:04.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.569+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f00752b0 con 0x7f77f00d06f0 2024-05-09T06:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.570+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77e8007bf0 con 0x7f77f00d06f0 2024-05-09T06:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.570+0000 7f77e5ffb640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 0x7f77d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.570+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6001+0+0 (secure 0 0 0) 0x7f77e80abd60 con 0x7f77f00d06f0 2024-05-09T06:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.570+0000 7f77f565d640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 0x7f77d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:04.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.571+0000 7f77f565d640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 0x7f77d406e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0015e0 tx=0x7f77ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.575+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77e807a5e0 con 0x7f77f00d06f0 2024-05-09T06:32:04.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:04.796+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f77f0072f30 con 0x7f77f00d06f0 2024-05-09T06:32:05.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.435+0000 7f77e5ffb640 1 -- 172.21.15.1:0/1069757541 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v41) ==== 174+0+0 (secure 0 0 0) 0x7f77e8079ee0 con 0x7f77f00d06f0 2024-05-09T06:32:05.420 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 11 and data pool 12 2024-05-09T06:32:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 msgr2=0x7f77d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 0x7f77d406e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0015e0 tx=0x7f77ec009040 comp rx=0 tx=0).stop 2024-05-09T06:32:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 msgr2=0x7f77f00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009f40 tx=0x7f77e8004a40 comp rx=0 tx=0).stop 2024-05-09T06:32:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 shutdown_connections 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77d406be60 0x7f77d406e320 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77f00d06f0 0x7f77f00b0bc0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77f00c41e0 0x7f77f00c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 --2- 172.21.15.1:0/1069757541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77f006b1e0 0x7f77f014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.447+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 >> 172.21.15.1:0/1069757541 conn(0x7f77f00d0b50 msgr2=0x7f77f00c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.448+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 shutdown_connections 2024-05-09T06:32:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.448+0000 7f77f78c1640 1 -- 172.21.15.1:0/1069757541 wait complete. 2024-05-09T06:32:05.445 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:05.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.569+0000 7f7bd6d63640 1 Processor -- start 2024-05-09T06:32:05.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.569+0000 7f7bd6d63640 1 -- start start 2024-05-09T06:32:05.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bd6d63640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7bd00d4940 0x7f7bd00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bd6d63640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd00b33c0 con 0x7f7bd00d2b10 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bd6d63640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd00b3660 con 0x7f7bd0161950 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bd6d63640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd00d4dd0 con 0x7f7bd00d4940 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bcffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7bd00d4940 0x7f7bd00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.570+0000 7f7bd5300640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7bd00d2b10 0x7f7bd00b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38774/0 (socket says 172.21.15.1:38774) 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.571+0000 7f7bd5300640 1 -- 172.21.15.1:0/372184362 learned_addr learned my addr 172.21.15.1:0/372184362 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.571+0000 7f7bcffff640 1 -- 172.21.15.1:0/372184362 >> v1:172.21.15.6:6789/0 conn(0x7f7bd0161950 legacy=0x7f7bd00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.571+0000 7f7bcffff640 1 -- 172.21.15.1:0/372184362 >> v1:172.21.15.1:6789/0 conn(0x7f7bd00d2b10 legacy=0x7f7bd00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.571+0000 7f7bcffff640 1 -- 172.21.15.1:0/372184362 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd00b3940 con 0x7f7bd00d4940 2024-05-09T06:32:05.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.571+0000 7f7bcffff640 1 --2- 172.21.15.1:0/372184362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00d4940 0x7f7bd00b4140 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7bd006e3d0 tx=0x7f7bc002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23eacc4188a0746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bcf7fe640 1 -- 172.21.15.1:0/372184362 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0007740 con 0x7f7bd00d4940 2024-05-09T06:32:05.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bcf7fe640 1 -- 172.21.15.1:0/372184362 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc00078e0 con 0x7f7bd00d4940 2024-05-09T06:32:05.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bcf7fe640 1 -- 172.21.15.1:0/372184362 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0007cb0 con 0x7f7bd00d4940 2024-05-09T06:32:05.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 -- 172.21.15.1:0/372184362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00d4940 msgr2=0x7f7bd00b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/372184362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00d4940 0x7f7bd00b4140 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7bd006e3d0 tx=0x7f7bc002a570 comp rx=0 tx=0).stop 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 -- 172.21.15.1:0/372184362 shutdown_connections 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/372184362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00d4940 0x7f7bd00b4140 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 -- 172.21.15.1:0/372184362 >> 172.21.15.1:0/372184362 conn(0x7f7bd00d3e20 msgr2=0x7f7bd00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 -- 172.21.15.1:0/372184362 shutdown_connections 2024-05-09T06:32:05.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.572+0000 7f7bd6d63640 1 -- 172.21.15.1:0/372184362 wait complete. 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.573+0000 7f7bd6d63640 1 Processor -- start 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.573+0000 7f7bd6d63640 1 -- start start 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd00d2b10 0x7f7bd00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd0161950 0x7f7bd0158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd00d1100 con 0x7f7bd00caef0 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd00d0f80 con 0x7f7bd0161950 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd6d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd00d1c00 con 0x7f7bd00d2b10 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.574+0000 7f7bd5300640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56344/0 (socket says 172.21.15.1:56344) 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 -- 172.21.15.1:0/496779442 learned_addr learned my addr 172.21.15.1:0/496779442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd4aff640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd00d2b10 0x7f7bd00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bcffff640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd0161950 0x7f7bd0158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 -- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd00d2b10 msgr2=0x7f7bd00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd00d2b10 0x7f7bd00cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 -- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd0161950 msgr2=0x7f7bd0158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd0161950 0x7f7bd0158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 -- 172.21.15.1:0/496779442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd00cb460 con 0x7f7bd00caef0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd5300640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c780 tx=0x7f7bb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8018070 con 0x7f7bd00caef0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb80072d0 con 0x7f7bd00caef0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd00cb740 con 0x7f7bd00caef0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80047e0 con 0x7f7bd00caef0 2024-05-09T06:32:05.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.575+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd00cbd00 con 0x7f7bd00caef0 2024-05-09T06:32:05.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.576+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd00752b0 con 0x7f7bd00caef0 2024-05-09T06:32:05.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.577+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bb801b070 con 0x7f7bd00caef0 2024-05-09T06:32:05.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.577+0000 7f7bcd7fa640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 0x7f7bb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.577+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f7bb808a4e0 con 0x7f7bd00caef0 2024-05-09T06:32:05.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.577+0000 7f7bd4aff640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 0x7f7bb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.578+0000 7f7bd4aff640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 0x7f7bb006e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006fd0 tx=0x7f7bc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.581+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bb8058d00 con 0x7f7bd00caef0 2024-05-09T06:32:05.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.764+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bd00d21e0 con 0x7f7bd00caef0 2024-05-09T06:32:05.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.765+0000 7f7bcd7fa640 1 -- 172.21.15.1:0/496779442 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16826 (secure 0 0 0) 0x7f7bb8058600 con 0x7f7bd00caef0 2024-05-09T06:32:05.747 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:05.747 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:04.797920+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 msgr2=0x7f7bb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 0x7f7bb006e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006fd0 tx=0x7f7bc8008040 comp rx=0 tx=0).stop 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 msgr2=0x7f7bd00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c780 tx=0x7f7bb800cc40 comp rx=0 tx=0).stop 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 shutdown_connections 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bb006be30 0x7f7bb006e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd00caef0 0x7f7bd00cd3f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.767+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bd0161950 0x7f7bd0158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.768+0000 7f7bd6d63640 1 --2- 172.21.15.1:0/496779442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bd00d2b10 0x7f7bd00cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.768+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 >> 172.21.15.1:0/496779442 conn(0x7f7bd00d3e20 msgr2=0x7f7bd00b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:05.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.768+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 shutdown_connections 2024-05-09T06:32:05.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.768+0000 7f7bd6d63640 1 -- 172.21.15.1:0/496779442 wait complete. 2024-05-09T06:32:05.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:05.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.882+0000 7f384ba8c640 1 Processor -- start 2024-05-09T06:32:05.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 -- start start 2024-05-09T06:32:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d4940 0x7f38440b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f38440d2b10 0x7f38440b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38440b33c0 con 0x7f3844161950 2024-05-09T06:32:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f38440d4dd0 con 0x7f38440d2b10 2024-05-09T06:32:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.883+0000 7f384ba8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38440b3660 con 0x7f38440d4940 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f38440d2b10 0x7f38440b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f38440d2b10 0x7f38440b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56352/0 (socket says 172.21.15.1:56352) 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 -- 172.21.15.1:0/2420260884 learned_addr learned my addr 172.21.15.1:0/2420260884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f3849027640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d4940 0x7f38440b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 -- 172.21.15.1:0/2420260884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d4940 msgr2=0x7f38440b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d4940 0x7f38440b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 -- 172.21.15.1:0/2420260884 >> v1:172.21.15.6:6789/0 conn(0x7f3844161950 legacy=0x7f38440d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.884+0000 7f384a029640 1 -- 172.21.15.1:0/2420260884 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38440b3940 con 0x7f38440d2b10 2024-05-09T06:32:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384a029640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38440d2b10 0x7f38440b4680 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f383c009e50 tx=0x7f383c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8cf5ddb8a1490 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f3848826640 1 -- 172.21.15.1:0/2420260884 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c036070 con 0x7f38440d2b10 2024-05-09T06:32:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f3848826640 1 -- 172.21.15.1:0/2420260884 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c031600 con 0x7f38440d2b10 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f3848826640 1 -- 172.21.15.1:0/2420260884 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0319d0 con 0x7f38440d2b10 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 -- 172.21.15.1:0/2420260884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38440d2b10 msgr2=0x7f38440b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38440d2b10 0x7f38440b4680 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f383c009e50 tx=0x7f383c02a710 comp rx=0 tx=0).stop 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 -- 172.21.15.1:0/2420260884 shutdown_connections 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38440d2b10 0x7f38440b4680 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 --2- 172.21.15.1:0/2420260884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d4940 0x7f38440b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.885+0000 7f384ba8c640 1 -- 172.21.15.1:0/2420260884 >> 172.21.15.1:0/2420260884 conn(0x7f38440d3e20 msgr2=0x7f38440d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.886+0000 7f384ba8c640 1 -- 172.21.15.1:0/2420260884 shutdown_connections 2024-05-09T06:32:05.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.886+0000 7f384ba8c640 1 -- 172.21.15.1:0/2420260884 wait complete. 2024-05-09T06:32:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.887+0000 7f384ba8c640 1 Processor -- start 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.887+0000 7f384ba8c640 1 -- start start 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d2b10 0x7f384414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384414dbb0 0x7f38440cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38440d1100 con 0x7f3844161950 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38440d0f80 con 0x7f384414dbb0 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38440d1c00 con 0x7f38440d2b10 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56356/0 (socket says 172.21.15.1:56356) 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 -- 172.21.15.1:0/1110730099 learned_addr learned my addr 172.21.15.1:0/1110730099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849828640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d2b10 0x7f384414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384a029640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384414dbb0 0x7f38440cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 -- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d2b10 msgr2=0x7f384414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d2b10 0x7f384414d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 -- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384414dbb0 msgr2=0x7f38440cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384414dbb0 0x7f38440cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 -- 172.21.15.1:0/1110730099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441517c0 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f3849027640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f383400d830 tx=0x7f383400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400b6c0 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3834019070 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38440ccb20 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400ba80 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.888+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38440cd0e0 con 0x7f3844161950 2024-05-09T06:32:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.889+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38440ca8f0 con 0x7f3844161950 2024-05-09T06:32:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.890+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f383401e460 con 0x7f3844161950 2024-05-09T06:32:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.890+0000 7f383a7fc640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 0x7f382806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.890+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f383408b6d0 con 0x7f3844161950 2024-05-09T06:32:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.890+0000 7f3849828640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 0x7f382806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.891+0000 7f3849828640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 0x7f382806e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3840000f60 tx=0x7f3840009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:05.895+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38440ca8f0 con 0x7f3844161950 2024-05-09T06:32:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.079+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38440d21e0 con 0x7f3844161950 2024-05-09T06:32:06.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.080+0000 7f383a7fc640 1 -- 172.21.15.1:0/1110730099 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16826 (secure 0 0 0) 0x7f3834059ef0 con 0x7f3844161950 2024-05-09T06:32:06.062 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:06.062 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:04.797920+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 msgr2=0x7f382806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 0x7f382806e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3840000f60 tx=0x7f3840009040 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 msgr2=0x7f38440cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f383400d830 tx=0x7f383400dcf0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 shutdown_connections 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f382806be30 0x7f382806e2f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f384414dbb0 0x7f38440cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3844161950 0x7f38440cbf40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 --2- 172.21.15.1:0/1110730099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38440d2b10 0x7f384414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 >> 172.21.15.1:0/1110730099 conn(0x7f38440d3e20 msgr2=0x7f38440b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 shutdown_connections 2024-05-09T06:32:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.082+0000 7f384ba8c640 1 -- 172.21.15.1:0/1110730099 wait complete. 2024-05-09T06:32:06.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:32:06.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.201+0000 7f5a795a2640 1 Processor -- start 2024-05-09T06:32:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.201+0000 7f5a795a2640 1 -- start start 2024-05-09T06:32:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.201+0000 7f5a795a2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5a74158a50 0x7f5a740718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a795a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7406ede0 0x7f5a74071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a795a2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a7415b420 con 0x7f5a7415c4a0 2024-05-09T06:32:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a795a2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5a74158e30 con 0x7f5a74158a50 2024-05-09T06:32:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a795a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5a74072480 con 0x7f5a7406ede0 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5a74158a50 0x7f5a740718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5a74158a50 0x7f5a740718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56360/0 (socket says 172.21.15.1:56360) 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2554226233 learned_addr learned my addr 172.21.15.1:0/2554226233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2554226233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7406ede0 msgr2=0x7f5a74071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a737fe640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7406ede0 0x7f5a74071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7406ede0 0x7f5a74071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2554226233 >> v1:172.21.15.6:6789/0 conn(0x7f5a7415c4a0 legacy=0x7f5a7406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.202+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2554226233 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7415b6e0 con 0x7f5a74158a50 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a72ffd640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a74158a50 0x7f5a740718c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a6402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb95685a4dafa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a727fc640 1 -- 172.21.15.1:0/2554226233 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64037070 con 0x7f5a74158a50 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a727fc640 1 -- 172.21.15.1:0/2554226233 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a64032440 con 0x7f5a74158a50 2024-05-09T06:32:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a727fc640 1 -- 172.21.15.1:0/2554226233 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64032810 con 0x7f5a74158a50 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a795a2640 1 -- 172.21.15.1:0/2554226233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a74158a50 msgr2=0x7f5a740718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a74158a50 0x7f5a740718c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a6402a210 comp rx=0 tx=0).stop 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a795a2640 1 -- 172.21.15.1:0/2554226233 shutdown_connections 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7406ede0 0x7f5a74071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.203+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2554226233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a74158a50 0x7f5a740718c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.204+0000 7f5a795a2640 1 -- 172.21.15.1:0/2554226233 >> 172.21.15.1:0/2554226233 conn(0x7f5a740cc4d0 msgr2=0x7f5a740cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.204+0000 7f5a795a2640 1 -- 172.21.15.1:0/2554226233 shutdown_connections 2024-05-09T06:32:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.204+0000 7f5a795a2640 1 -- 172.21.15.1:0/2554226233 wait complete. 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.205+0000 7f5a795a2640 1 Processor -- start 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.205+0000 7f5a795a2640 1 -- start start 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a74158a50 0x7f5a74144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7415c4a0 0x7f5a740addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7415bf80 con 0x7f5a7406ede0 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7415bca0 con 0x7f5a74158a50 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5a74070ea0 con 0x7f5a7415c4a0 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56372/0 (socket says 172.21.15.1:56372) 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2601330867 learned_addr learned my addr 172.21.15.1:0/2601330867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a737fe640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7415c4a0 0x7f5a740addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7415c4a0 msgr2=0x7f5a740addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a6bfff640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a74158a50 0x7f5a74144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7415c4a0 0x7f5a740addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a74158a50 msgr2=0x7f5a74144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a74158a50 0x7f5a74144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 -- 172.21.15.1:0/2601330867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a740b05a0 con 0x7f5a7406ede0 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a72ffd640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64041020 con 0x7f5a7406ede0 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a64030970 con 0x7f5a7406ede0 2024-05-09T06:32:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64037040 con 0x7f5a7406ede0 2024-05-09T06:32:06.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.206+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a740b0880 con 0x7f5a7406ede0 2024-05-09T06:32:06.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.207+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a740b0e40 con 0x7f5a7406ede0 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.208+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a740752b0 con 0x7f5a7406ede0 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.208+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a64044020 con 0x7f5a7406ede0 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.209+0000 7f5a6b7fe640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 0x7f5a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.209+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f5a640ab9f0 con 0x7f5a7406ede0 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.209+0000 7f5a6bfff640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 0x7f5a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.209+0000 7f5a6bfff640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 0x7f5a5406e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a740b1390 tx=0x7f5a5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:06.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.213+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a6404b030 con 0x7f5a7406ede0 2024-05-09T06:32:06.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:06.441+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5a7406e220 con 0x7f5a7406ede0 2024-05-09T06:32:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.447+0000 7f5a6b7fe640 1 -- 172.21.15.1:0/2601330867 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f5a640ab070 con 0x7f5a7406ede0 2024-05-09T06:32:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 msgr2=0x7f5a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 0x7f5a5406e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a740b1390 tx=0x7f5a5c009040 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 msgr2=0x7f5a740afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a640046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 shutdown_connections 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5a5406be30 0x7f5a5406e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a7415c4a0 0x7f5a740addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a74158a50 0x7f5a74144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 --2- 172.21.15.1:0/2601330867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a7406ede0 0x7f5a740afd50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 >> 172.21.15.1:0/2601330867 conn(0x7f5a740cc4d0 msgr2=0x7f5a7406d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 shutdown_connections 2024-05-09T06:32:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.450+0000 7f5a795a2640 1 -- 172.21.15.1:0/2601330867 wait complete. 2024-05-09T06:32:07.445 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:32:07.445 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:32:07.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.577+0000 7f6ce84a5640 1 Processor -- start 2024-05-09T06:32:07.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.577+0000 7f6ce84a5640 1 -- start start 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce84a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce84a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce006b1e0 0x7f6ce00cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce84a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce0072050 con 0x7f6ce00d06f0 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce84a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce0072330 con 0x7f6ce00c41e0 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce84a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce0072610 con 0x7f6ce006b1e0 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce5a40640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce5a40640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56378/0 (socket says 172.21.15.1:56378) 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce5a40640 1 -- 172.21.15.1:0/585341781 learned_addr learned my addr 172.21.15.1:0/585341781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce6a42640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce006b1e0 0x7f6ce00cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.578+0000 7f6ce5a40640 1 -- 172.21.15.1:0/585341781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce006b1e0 msgr2=0x7f6ce00cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce5a40640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce006b1e0 0x7f6ce00cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce5a40640 1 -- 172.21.15.1:0/585341781 >> v1:172.21.15.6:6789/0 conn(0x7f6ce00d06f0 legacy=0x7f6ce00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce5a40640 1 -- 172.21.15.1:0/585341781 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce00728f0 con 0x7f6ce00c41e0 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce5a40640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009870 tx=0x7f6cd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb42fb554a6a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce523f640 1 -- 172.21.15.1:0/585341781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0037070 con 0x7f6ce00c41e0 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce523f640 1 -- 172.21.15.1:0/585341781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd0032440 con 0x7f6ce00c41e0 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.579+0000 7f6ce523f640 1 -- 172.21.15.1:0/585341781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0032810 con 0x7f6ce00c41e0 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 -- 172.21.15.1:0/585341781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce00c41e0 msgr2=0x7f6ce00ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009870 tx=0x7f6cd002a270 comp rx=0 tx=0).stop 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 -- 172.21.15.1:0/585341781 shutdown_connections 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce006b1e0 0x7f6ce00cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/585341781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce00c41e0 0x7f6ce00ce8d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 -- 172.21.15.1:0/585341781 >> 172.21.15.1:0/585341781 conn(0x7f6ce00d0b50 msgr2=0x7f6ce00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 -- 172.21.15.1:0/585341781 shutdown_connections 2024-05-09T06:32:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.580+0000 7f6ce84a5640 1 -- 172.21.15.1:0/585341781 wait complete. 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.581+0000 7f6ce84a5640 1 Processor -- start 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.581+0000 7f6ce84a5640 1 -- start start 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce00c41e0 0x7f6ce00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce00d06f0 0x7f6ce00ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce00ccf70 con 0x7f6ce006b1e0 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce00ccdf0 con 0x7f6ce00d06f0 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce84a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce00cd0f0 con 0x7f6ce00c41e0 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56392/0 (socket says 172.21.15.1:56392) 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 -- 172.21.15.1:0/1010748318 learned_addr learned my addr 172.21.15.1:0/1010748318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce5a40640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce00c41e0 0x7f6ce00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6a42640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce00d06f0 0x7f6ce00ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 -- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce00c41e0 msgr2=0x7f6ce00c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce00c41e0 0x7f6ce00c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 -- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce00d06f0 msgr2=0x7f6ce00ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce00d06f0 0x7f6ce00ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.582+0000 7f6ce6241640 1 -- 172.21.15.1:0/1010748318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce01f4090 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6ce6241640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c3c0 tx=0x7f6cdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0052d0 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc005490 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce015c1f0 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007260 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.583+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce01f4630 con 0x7f6ce006b1e0 2024-05-09T06:32:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.584+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce00752b0 con 0x7f6ce006b1e0 2024-05-09T06:32:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.584+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cdc007400 con 0x7f6ce006b1e0 2024-05-09T06:32:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.585+0000 7f6cd6ffd640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.585+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f6cdc08a390 con 0x7f6ce006b1e0 2024-05-09T06:32:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.585+0000 7f6ce5a40640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.585+0000 7f6ce5a40640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 0x7f6cc406e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0034000 tx=0x7f6cd002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:07.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.589+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6cdc0585f0 con 0x7f6ce006b1e0 2024-05-09T06:32:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:07.810+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6ce0072f30 con 0x7f6ce006b1e0 2024-05-09T06:32:08.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.449+0000 7f6cd6ffd640 1 -- 172.21.15.1:0/1010748318 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f6cdc05c8f0 con 0x7f6ce006b1e0 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 msgr2=0x7f6cc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 0x7f6cc406e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0034000 tx=0x7f6cd002e000 comp rx=0 tx=0).stop 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 msgr2=0x7f6ce015bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c3c0 tx=0x7f6cdc00c880 comp rx=0 tx=0).stop 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 shutdown_connections 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cc406be30 0x7f6cc406e2f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce00d06f0 0x7f6ce00ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce00c41e0 0x7f6ce00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 --2- 172.21.15.1:0/1010748318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce006b1e0 0x7f6ce015bcb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 >> 172.21.15.1:0/1010748318 conn(0x7f6ce00d0b50 msgr2=0x7f6ce006e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:08.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 shutdown_connections 2024-05-09T06:32:08.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.451+0000 7f6ce84a5640 1 -- 172.21.15.1:0/1010748318 wait complete. 2024-05-09T06:32:08.446 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:32:08.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:08.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.575+0000 7f691c47d640 1 Processor -- start 2024-05-09T06:32:08.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.575+0000 7f691c47d640 1 -- start start 2024-05-09T06:32:08.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.576+0000 7f691c47d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f691406ffa0 0x7f69140cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.576+0000 7f691c47d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f691415e5b0 0x7f69140d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.576+0000 7f691c47d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691406c2f0 con 0x7f691415e1d0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.576+0000 7f691c47d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69140cc3c0 con 0x7f691406ffa0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.576+0000 7f691c47d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f69140ccc60 con 0x7f691415e5b0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919a18640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f691406ffa0 0x7f69140cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f691a219640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f691415e1d0 0x7f691406f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55246/0 (socket says 172.21.15.1:55246) 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f691a219640 1 -- 172.21.15.1:0/168537068 learned_addr learned my addr 172.21.15.1:0/168537068 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f691aa1a640 1 --2- 172.21.15.1:0/168537068 >> v2:172.21.15.6:3300/0 conn(0x7f691415e5b0 0x7f69140d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919a18640 1 -- 172.21.15.1:0/168537068 >> v2:172.21.15.6:3300/0 conn(0x7f691415e5b0 msgr2=0x7f69140d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919a18640 1 --2- 172.21.15.1:0/168537068 >> v2:172.21.15.6:3300/0 conn(0x7f691415e5b0 0x7f69140d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919a18640 1 -- 172.21.15.1:0/168537068 >> v1:172.21.15.1:6789/0 conn(0x7f691415e1d0 legacy=0x7f691406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919a18640 1 -- 172.21.15.1:0/168537068 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691406f500 con 0x7f691406ffa0 2024-05-09T06:32:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.577+0000 7f6919217640 1 -- 172.21.15.1:0/168537068 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032444824 0 0) 0x7f6910001110 con 0x7f691415e1d0 2024-05-09T06:32:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f6919a18640 1 --2- 172.21.15.1:0/168537068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691406ffa0 0x7f69140cffb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6914072700 tx=0x7f690402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c48177e70f2e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f6919217640 1 -- 172.21.15.1:0/168537068 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904037070 con 0x7f691406ffa0 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f6919217640 1 -- 172.21.15.1:0/168537068 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6904032440 con 0x7f691406ffa0 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f6919217640 1 -- 172.21.15.1:0/168537068 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690402ae60 con 0x7f691406ffa0 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 -- 172.21.15.1:0/168537068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691406ffa0 msgr2=0x7f69140cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 --2- 172.21.15.1:0/168537068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691406ffa0 0x7f69140cffb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6914072700 tx=0x7f690402a110 comp rx=0 tx=0).stop 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 -- 172.21.15.1:0/168537068 shutdown_connections 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 --2- 172.21.15.1:0/168537068 >> v2:172.21.15.6:3300/0 conn(0x7f691415e5b0 0x7f69140d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 --2- 172.21.15.1:0/168537068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691406ffa0 0x7f69140cffb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.578+0000 7f691c47d640 1 -- 172.21.15.1:0/168537068 >> 172.21.15.1:0/168537068 conn(0x7f69140d1d70 msgr2=0x7f69140d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.579+0000 7f691c47d640 1 -- 172.21.15.1:0/168537068 shutdown_connections 2024-05-09T06:32:08.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.579+0000 7f691c47d640 1 -- 172.21.15.1:0/168537068 wait complete. 2024-05-09T06:32:08.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 Processor -- start 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 -- start start 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f691406ffa0 0x7f69140cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691415e5b0 0x7f6914146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f691406f1b0 con 0x7f691415e1d0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69140cc8f0 con 0x7f691406ffa0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.580+0000 7f691c47d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69140ce100 con 0x7f691415e5b0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52404/0 (socket says 172.21.15.1:52404) 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 -- 172.21.15.1:0/3891972458 learned_addr learned my addr 172.21.15.1:0/3891972458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f691a219640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f691406ffa0 0x7f69140cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f691aa1a640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691415e5b0 0x7f6914146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 -- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691415e5b0 msgr2=0x7f6914146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691415e5b0 0x7f6914146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 -- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f691406ffa0 msgr2=0x7f69140cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f691406ffa0 0x7f69140cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 -- 172.21.15.1:0/3891972458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69141607b0 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f6919a18640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6904000c00 tx=0x7f69040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904037040 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6904042070 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6914160a90 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904030a50 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.581+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6914161050 con 0x7f691415e1d0 2024-05-09T06:32:08.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.582+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69140752b0 con 0x7f691415e1d0 2024-05-09T06:32:08.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.583+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6904030bf0 con 0x7f691415e1d0 2024-05-09T06:32:08.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.583+0000 7f690affd640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.584+0000 7f691a219640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.584+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f69040aba10 con 0x7f691415e1d0 2024-05-09T06:32:08.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.584+0000 7f691a219640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.588+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f690407a2c0 con 0x7f691415e1d0 2024-05-09T06:32:08.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.811+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69140cf4d0 con 0x7f691415e1d0 2024-05-09T06:32:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.812+0000 7f690affd640 1 -- 172.21.15.1:0/3891972458 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5168 (secure 0 0 0) 0x7f6904079bc0 con 0x7f691415e1d0 2024-05-09T06:32:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:08.793 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:07.811817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/2073487547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2073487547},{"type":"v1","addr":"172.21.15.6:6834","nonce":2073487547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7125,"qdb_cluster":[7125]},"id":5}]} 2024-05-09T06:32:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 45 2024-05-09T06:32:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 msgr2=0x7f68f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910007970 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 msgr2=0x7f69140cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6904000c00 tx=0x7f69040046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 shutdown_connections 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691415e5b0 0x7f6914146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f691415e1d0 0x7f69140cb920 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 --2- 172.21.15.1:0/3891972458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f691406ffa0 0x7f69140cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 >> 172.21.15.1:0/3891972458 conn(0x7f69140d1d70 msgr2=0x7f69140aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 shutdown_connections 2024-05-09T06:32:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.814+0000 7f691c47d640 1 -- 172.21.15.1:0/3891972458 wait complete. 2024-05-09T06:32:08.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:08.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.930+0000 7f29ec0f5640 1 Processor -- start 2024-05-09T06:32:08.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.930+0000 7f29ec0f5640 1 -- start start 2024-05-09T06:32:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ec0f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f29e406e6d0 0x7f29e406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ec0f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cf180 0x7f29e40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ec0f5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40ce0e0 con 0x7f29e40cdab0 2024-05-09T06:32:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ec0f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40ce3c0 con 0x7f29e406e6d0 2024-05-09T06:32:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ec0f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f29e40ce6a0 con 0x7f29e40cf180 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29e9e91640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f29e406e6d0 0x7f29e406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ea692640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f29e40cdab0 0x7f29e415e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55256/0 (socket says 172.21.15.1:55256) 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.931+0000 7f29ea692640 1 -- 172.21.15.1:0/4185779713 learned_addr learned my addr 172.21.15.1:0/4185779713 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e9690640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cf180 0x7f29e40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e9e91640 1 -- 172.21.15.1:0/4185779713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cf180 msgr2=0x7f29e40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e9e91640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cf180 0x7f29e40cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e9e91640 1 -- 172.21.15.1:0/4185779713 >> v1:172.21.15.1:6789/0 conn(0x7f29e40cdab0 legacy=0x7f29e415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e9e91640 1 -- 172.21.15.1:0/4185779713 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40ce980 con 0x7f29e406e6d0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.932+0000 7f29e8e8f640 1 -- 172.21.15.1:0/4185779713 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579333366 0 0) 0x7f29e40ce0e0 con 0x7f29e40cdab0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29e9e91640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e406e6d0 0x7f29e406eab0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f29e00097a0 tx=0x7f29e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7fd11650f3c07f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29e8e8f640 1 -- 172.21.15.1:0/4185779713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0037070 con 0x7f29e406e6d0 2024-05-09T06:32:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29e8e8f640 1 -- 172.21.15.1:0/4185779713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e0032440 con 0x7f29e406e6d0 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29e8e8f640 1 -- 172.21.15.1:0/4185779713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0032810 con 0x7f29e406e6d0 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 -- 172.21.15.1:0/4185779713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e406e6d0 msgr2=0x7f29e406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e406e6d0 0x7f29e406eab0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f29e00097a0 tx=0x7f29e002a180 comp rx=0 tx=0).stop 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 -- 172.21.15.1:0/4185779713 shutdown_connections 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cf180 0x7f29e40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/4185779713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e406e6d0 0x7f29e406eab0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 -- 172.21.15.1:0/4185779713 >> 172.21.15.1:0/4185779713 conn(0x7f29e40cc4b0 msgr2=0x7f29e40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 -- 172.21.15.1:0/4185779713 shutdown_connections 2024-05-09T06:32:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.933+0000 7f29ec0f5640 1 -- 172.21.15.1:0/4185779713 wait complete. 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 Processor -- start 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 -- start start 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 0x7f29e415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 0x7f29e415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4070ed0 con 0x7f29e40cf180 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f29e406eff0 con 0x7f29e406e6d0 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ec0f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f29e41447c0 con 0x7f29e40cdab0 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ea692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ea692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52414/0 (socket says 172.21.15.1:52414) 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.935+0000 7f29ea692640 1 -- 172.21.15.1:0/1265111735 learned_addr learned my addr 172.21.15.1:0/1265111735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29e9e91640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 0x7f29e415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29e9690640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 0x7f29e415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 -- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 msgr2=0x7f29e415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 0x7f29e415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 -- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 msgr2=0x7f29e415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 0x7f29e415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 -- 172.21.15.1:0/1265111735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e413d640 con 0x7f29e40cf180 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29e9e91640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 0x7f29e415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29e9690640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 0x7f29e415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ea692640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c7d0 tx=0x7f29dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc018070 con 0x7f29e40cf180 2024-05-09T06:32:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc00a640 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e414b740 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00aa70 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.936+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e414bd00 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.937+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40ce0e0 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.938+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29dc01b070 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.938+0000 7f29da7fc640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 0x7f29c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.938+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f29dc08a620 con 0x7f29e40cf180 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.938+0000 7f29e9e91640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 0x7f29c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.939+0000 7f29e9e91640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 0x7f29c806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f29e0034000 tx=0x7f29e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:08.942+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29e40ce0e0 con 0x7f29e40cf180 2024-05-09T06:32:09.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.126+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29e40752b0 con 0x7f29e40cf180 2024-05-09T06:32:09.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.127+0000 7f29da7fc640 1 -- 172.21.15.1:0/1265111735 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16813 (secure 0 0 0) 0x7f29dc058e40 con 0x7f29e40cf180 2024-05-09T06:32:09.109 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:06.420655+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:09.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 msgr2=0x7f29c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 0x7f29c806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f29e0034000 tx=0x7f29e002e000 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 msgr2=0x7f29e414d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c7d0 tx=0x7f29dc00cc90 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 shutdown_connections 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f29c806be30 0x7f29c806e2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29e40cf180 0x7f29e414d730 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f29e40cdab0 0x7f29e415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 --2- 172.21.15.1:0/1265111735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29e406e6d0 0x7f29e415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 >> 172.21.15.1:0/1265111735 conn(0x7f29e40cc4b0 msgr2=0x7f29e40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 shutdown_connections 2024-05-09T06:32:09.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.130+0000 7f29ec0f5640 1 -- 172.21.15.1:0/1265111735 wait complete. 2024-05-09T06:32:09.122 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:32:09.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:32:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.245+0000 7f4fe9dd5640 1 Processor -- start 2024-05-09T06:32:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.246+0000 7f4fe9dd5640 1 -- start start 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.246+0000 7f4fe9dd5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4fe406e6d0 0x7f4fe406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.246+0000 7f4fe9dd5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe40ce0e0 con 0x7f4fe40cf180 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.246+0000 7f4fe9dd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe40ce3c0 con 0x7f4fe40b3300 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.246+0000 7f4fe9dd5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe40ce6a0 con 0x7f4fe406e6d0 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4fe406e6d0 0x7f4fe406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe2ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4fe40cf180 0x7f4fe40cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55268/0 (socket says 172.21.15.1:55268) 2024-05-09T06:32:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe2ffd640 1 -- 172.21.15.1:0/1399703061 learned_addr learned my addr 172.21.15.1:0/1399703061 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:09.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe37fe640 1 -- 172.21.15.1:0/1399703061 >> v1:172.21.15.6:6789/0 conn(0x7f4fe40b3300 legacy=0x7f4fe40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe37fe640 1 -- 172.21.15.1:0/1399703061 >> v1:172.21.15.1:6789/0 conn(0x7f4fe40cf180 legacy=0x7f4fe40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe37fe640 1 -- 172.21.15.1:0/1399703061 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe40ce980 con 0x7f4fe406e6d0 2024-05-09T06:32:09.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.247+0000 7f4fe27fc640 1 -- 172.21.15.1:0/1399703061 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954218163 0 0) 0x7f4fe40ce0e0 con 0x7f4fe40cf180 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe37fe640 1 --2- 172.21.15.1:0/1399703061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe406e6d0 0x7f4fe406eab0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40097a0 tx=0x7f4fd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bfa010804f1f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe27fc640 1 -- 172.21.15.1:0/1399703061 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4037070 con 0x7f4fe406e6d0 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe27fc640 1 -- 172.21.15.1:0/1399703061 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd4032440 con 0x7f4fe406e6d0 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe27fc640 1 -- 172.21.15.1:0/1399703061 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4032810 con 0x7f4fe406e6d0 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/1399703061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe406e6d0 msgr2=0x7f4fe406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/1399703061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe406e6d0 0x7f4fe406eab0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40097a0 tx=0x7f4fd402a230 comp rx=0 tx=0).stop 2024-05-09T06:32:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/1399703061 shutdown_connections 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/1399703061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe406e6d0 0x7f4fe406eab0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/1399703061 >> 172.21.15.1:0/1399703061 conn(0x7f4fe40cc4b0 msgr2=0x7f4fe40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/1399703061 shutdown_connections 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.248+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/1399703061 wait complete. 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 Processor -- start 2024-05-09T06:32:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 -- start start 2024-05-09T06:32:09.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe406e6d0 0x7f4fe40aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe40cf180 0x7f4fe4134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4070ed0 con 0x7f4fe40b3300 2024-05-09T06:32:09.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe406eff0 con 0x7f4fe406e6d0 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.250+0000 7f4fe9dd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe41447c0 con 0x7f4fe40cf180 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52432/0 (socket says 172.21.15.1:52432) 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 -- 172.21.15.1:0/107802426 learned_addr learned my addr 172.21.15.1:0/107802426 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe37fe640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe406e6d0 0x7f4fe40aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe3fff640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe40cf180 0x7f4fe4134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 -- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe40cf180 msgr2=0x7f4fe4134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe40cf180 0x7f4fe4134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 -- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe406e6d0 msgr2=0x7f4fe40aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe406e6d0 0x7f4fe40aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 -- 172.21.15.1:0/107802426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe415e6b0 con 0x7f4fe40b3300 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe2ffd640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c460 tx=0x7f4fd800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8018070 con 0x7f4fe40b3300 2024-05-09T06:32:09.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd8007520 con 0x7f4fe40b3300 2024-05-09T06:32:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe415e990 con 0x7f4fe40b3300 2024-05-09T06:32:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004a20 con 0x7f4fe40b3300 2024-05-09T06:32:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.251+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe415ef50 con 0x7f4fe40b3300 2024-05-09T06:32:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.252+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe415f190 con 0x7f4fe40b3300 2024-05-09T06:32:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.253+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fd801b070 con 0x7f4fe40b3300 2024-05-09T06:32:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.254+0000 7f4fcbfff640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 0x7f4fd006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.254+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f4fd808a390 con 0x7f4fe40b3300 2024-05-09T06:32:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.254+0000 7f4fe37fe640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 0x7f4fd006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.254+0000 7f4fe37fe640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 0x7f4fd006e320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.258+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4fe415f190 con 0x7f4fe40b3300 2024-05-09T06:32:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.440+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fe40752b0 con 0x7f4fd006be60 2024-05-09T06:32:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.447+0000 7f4fcbfff640 1 -- 172.21.15.1:0/107802426 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252072 (secure 0 0 0) 0x7f4fe40752b0 con 0x7f4fd006be60 2024-05-09T06:32:09.431 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:32:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 msgr2=0x7f4fd006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 0x7f4fd006e320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd402e000 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 msgr2=0x7f4fe4150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c460 tx=0x7f4fd800c920 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 shutdown_connections 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fd006be60 0x7f4fd006e320 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe40cf180 0x7f4fe4134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe40b3300 0x7f4fe4150f60 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 --2- 172.21.15.1:0/107802426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe406e6d0 0x7f4fe40aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 >> 172.21.15.1:0/107802426 conn(0x7f4fe40cc4b0 msgr2=0x7f4fe40afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 shutdown_connections 2024-05-09T06:32:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.450+0000 7f4fe9dd5640 1 -- 172.21.15.1:0/107802426 wait complete. 2024-05-09T06:32:09.444 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-05-09T06:32:09.130732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245704,"kb_used_data":8680,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729016,"statfs":{"total":773094113280,"available":772842512384,"internally_reserved":0,"allocated":8888320,"data_stored":4683332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.717381"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430015+0000","last_change":"2024-05-09T06:32:05.448168+0000","last_active":"2024-05-09T06:32:06.430015+0000","last_peered":"2024-05-09T06:32:06.430015+0000","last_clean":"2024-05-09T06:32:06.430015+0000","last_became_active":"2024-05-09T06:32:05.447857+0000","last_became_peered":"2024-05-09T06:32:05.447857+0000","last_unstale":"2024-05-09T06:32:06.430015+0000","last_undegraded":"2024-05-09T06:32:06.430015+0000","last_fullsized":"2024-05-09T06:32:06.430015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:23.611187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428731+0000","last_change":"2024-05-09T06:32:05.433758+0000","last_active":"2024-05-09T06:32:06.428731+0000","last_peered":"2024-05-09T06:32:06.428731+0000","last_clean":"2024-05-09T06:32:06.428731+0000","last_became_active":"2024-05-09T06:32:05.433478+0000","last_became_peered":"2024-05-09T06:32:05.433478+0000","last_unstale":"2024-05-09T06:32:06.428731+0000","last_undegraded":"2024-05-09T06:32:06.428731+0000","last_fullsized":"2024-05-09T06:32:06.428731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:58.313762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428704+0000","last_change":"2024-05-09T06:32:05.443243+0000","last_active":"2024-05-09T06:32:06.428704+0000","last_peered":"2024-05-09T06:32:06.428704+0000","last_clean":"2024-05-09T06:32:06.428704+0000","last_became_active":"2024-05-09T06:32:05.442955+0000","last_became_peered":"2024-05-09T06:32:05.442955+0000","last_unstale":"2024-05-09T06:32:06.428704+0000","last_undegraded":"2024-05-09T06:32:06.428704+0000","last_fullsized":"2024-05-09T06:32:06.428704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.502528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430061+0000","last_change":"2024-05-09T06:32:05.449877+0000","last_active":"2024-05-09T06:32:06.430061+0000","last_peered":"2024-05-09T06:32:06.430061+0000","last_clean":"2024-05-09T06:32:06.430061+0000","last_became_active":"2024-05-09T06:32:05.449461+0000","last_became_peered":"2024-05-09T06:32:05.449461+0000","last_unstale":"2024-05-09T06:32:06.430061+0000","last_undegraded":"2024-05-09T06:32:06.430061+0000","last_fullsized":"2024-05-09T06:32:06.430061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:39.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423946+0000","last_change":"2024-05-09T06:32:05.440360+0000","last_active":"2024-05-09T06:32:06.423946+0000","last_peered":"2024-05-09T06:32:06.423946+0000","last_clean":"2024-05-09T06:32:06.423946+0000","last_became_active":"2024-05-09T06:32:05.440079+0000","last_became_peered":"2024-05-09T06:32:05.440079+0000","last_unstale":"2024-05-09T06:32:06.423946+0000","last_undegraded":"2024-05-09T06:32:06.423946+0000","last_fullsized":"2024-05-09T06:32:06.423946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:28.313413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429205+0000","last_change":"2024-05-09T06:32:05.451724+0000","last_active":"2024-05-09T06:32:06.429205+0000","last_peered":"2024-05-09T06:32:06.429205+0000","last_clean":"2024-05-09T06:32:06.429205+0000","last_became_active":"2024-05-09T06:32:05.451447+0000","last_became_peered":"2024-05-09T06:32:05.451447+0000","last_unstale":"2024-05-09T06:32:06.429205+0000","last_undegraded":"2024-05-09T06:32:06.429205+0000","last_fullsized":"2024-05-09T06:32:06.429205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:15.947058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425300+0000","last_change":"2024-05-09T06:32:05.437648+0000","last_active":"2024-05-09T06:32:06.425300+0000","last_peered":"2024-05-09T06:32:06.425300+0000","last_clean":"2024-05-09T06:32:06.425300+0000","last_became_active":"2024-05-09T06:32:05.437379+0000","last_became_peered":"2024-05-09T06:32:05.437379+0000","last_unstale":"2024-05-09T06:32:06.425300+0000","last_undegraded":"2024-05-09T06:32:06.425300+0000","last_fullsized":"2024-05-09T06:32:06.425300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:04.000848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423779+0000","last_change":"2024-05-09T06:32:05.448861+0000","last_active":"2024-05-09T06:32:06.423779+0000","last_peered":"2024-05-09T06:32:06.423779+0000","last_clean":"2024-05-09T06:32:06.423779+0000","last_became_active":"2024-05-09T06:32:05.448550+0000","last_became_peered":"2024-05-09T06:32:05.448550+0000","last_unstale":"2024-05-09T06:32:06.423779+0000","last_undegraded":"2024-05-09T06:32:06.423779+0000","last_fullsized":"2024-05-09T06:32:06.423779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:40.242265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429544+0000","last_change":"2024-05-09T06:32:05.445624+0000","last_active":"2024-05-09T06:32:06.429544+0000","last_peered":"2024-05-09T06:32:06.429544+0000","last_clean":"2024-05-09T06:32:06.429544+0000","last_became_active":"2024-05-09T06:32:05.445342+0000","last_became_peered":"2024-05-09T06:32:05.445342+0000","last_unstale":"2024-05-09T06:32:06.429544+0000","last_undegraded":"2024-05-09T06:32:06.429544+0000","last_fullsized":"2024-05-09T06:32:06.429544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:45.500091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428566+0000","last_change":"2024-05-09T06:32:05.447975+0000","last_active":"2024-05-09T06:32:06.428566+0000","last_peered":"2024-05-09T06:32:06.428566+0000","last_clean":"2024-05-09T06:32:06.428566+0000","last_became_active":"2024-05-09T06:32:05.447456+0000","last_became_peered":"2024-05-09T06:32:05.447456+0000","last_unstale":"2024-05-09T06:32:06.428566+0000","last_undegraded":"2024-05-09T06:32:06.428566+0000","last_fullsized":"2024-05-09T06:32:06.428566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:33.213776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428050+0000","last_change":"2024-05-09T06:32:05.440652+0000","last_active":"2024-05-09T06:32:06.428050+0000","last_peered":"2024-05-09T06:32:06.428050+0000","last_clean":"2024-05-09T06:32:06.428050+0000","last_became_active":"2024-05-09T06:32:05.440396+0000","last_became_peered":"2024-05-09T06:32:05.440396+0000","last_unstale":"2024-05-09T06:32:06.428050+0000","last_undegraded":"2024-05-09T06:32:06.428050+0000","last_fullsized":"2024-05-09T06:32:06.428050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:19.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430237+0000","last_change":"2024-05-09T06:32:05.442772+0000","last_active":"2024-05-09T06:32:06.430237+0000","last_peered":"2024-05-09T06:32:06.430237+0000","last_clean":"2024-05-09T06:32:06.430237+0000","last_became_active":"2024-05-09T06:32:05.442240+0000","last_became_peered":"2024-05-09T06:32:05.442240+0000","last_unstale":"2024-05-09T06:32:06.430237+0000","last_undegraded":"2024-05-09T06:32:06.430237+0000","last_fullsized":"2024-05-09T06:32:06.430237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:44.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424080+0000","last_change":"2024-05-09T06:32:05.430314+0000","last_active":"2024-05-09T06:32:06.424080+0000","last_peered":"2024-05-09T06:32:06.424080+0000","last_clean":"2024-05-09T06:32:06.424080+0000","last_became_active":"2024-05-09T06:32:05.430024+0000","last_became_peered":"2024-05-09T06:32:05.430024+0000","last_unstale":"2024-05-09T06:32:06.424080+0000","last_undegraded":"2024-05-09T06:32:06.424080+0000","last_fullsized":"2024-05-09T06:32:06.424080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:31.152509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429260+0000","last_change":"2024-05-09T06:32:05.452453+0000","last_active":"2024-05-09T06:32:06.429260+0000","last_peered":"2024-05-09T06:32:06.429260+0000","last_clean":"2024-05-09T06:32:06.429260+0000","last_became_active":"2024-05-09T06:32:05.452172+0000","last_became_peered":"2024-05-09T06:32:05.452172+0000","last_unstale":"2024-05-09T06:32:06.429260+0000","last_undegraded":"2024-05-09T06:32:06.429260+0000","last_fullsized":"2024-05-09T06:32:06.429260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:58.313762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428234+0000","last_change":"2024-05-09T06:32:05.433432+0000","last_active":"2024-05-09T06:32:06.428234+0000","last_peered":"2024-05-09T06:32:06.428234+0000","last_clean":"2024-05-09T06:32:06.428234+0000","last_became_active":"2024-05-09T06:32:05.433126+0000","last_became_peered":"2024-05-09T06:32:05.433126+0000","last_unstale":"2024-05-09T06:32:06.428234+0000","last_undegraded":"2024-05-09T06:32:06.428234+0000","last_fullsized":"2024-05-09T06:32:06.428234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.617088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429541+0000","last_change":"2024-05-09T06:32:05.452517+0000","last_active":"2024-05-09T06:32:06.429541+0000","last_peered":"2024-05-09T06:32:06.429541+0000","last_clean":"2024-05-09T06:32:06.429541+0000","last_became_active":"2024-05-09T06:32:05.452218+0000","last_became_peered":"2024-05-09T06:32:05.452218+0000","last_unstale":"2024-05-09T06:32:06.429541+0000","last_undegraded":"2024-05-09T06:32:06.429541+0000","last_fullsized":"2024-05-09T06:32:06.429541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:48.323284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424396+0000","last_change":"2024-05-09T06:32:05.450790+0000","last_active":"2024-05-09T06:32:06.424396+0000","last_peered":"2024-05-09T06:32:06.424396+0000","last_clean":"2024-05-09T06:32:06.424396+0000","last_became_active":"2024-05-09T06:32:05.450511+0000","last_became_peered":"2024-05-09T06:32:05.450511+0000","last_unstale":"2024-05-09T06:32:06.424396+0000","last_undegraded":"2024-05-09T06:32:06.424396+0000","last_fullsized":"2024-05-09T06:32:06.424396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:23.532300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428916+0000","last_change":"2024-05-09T06:32:05.447446+0000","last_active":"2024-05-09T06:32:06.428916+0000","last_peered":"2024-05-09T06:32:06.428916+0000","last_clean":"2024-05-09T06:32:06.428916+0000","last_became_active":"2024-05-09T06:32:05.446966+0000","last_became_peered":"2024-05-09T06:32:05.446966+0000","last_unstale":"2024-05-09T06:32:06.428916+0000","last_undegraded":"2024-05-09T06:32:06.428916+0000","last_fullsized":"2024-05-09T06:32:06.428916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:04.000848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428540+0000","last_change":"2024-05-09T06:32:05.435911+0000","last_active":"2024-05-09T06:32:06.428540+0000","last_peered":"2024-05-09T06:32:06.428540+0000","last_clean":"2024-05-09T06:32:06.428540+0000","last_became_active":"2024-05-09T06:32:05.435616+0000","last_became_peered":"2024-05-09T06:32:05.435616+0000","last_unstale":"2024-05-09T06:32:06.428540+0000","last_undegraded":"2024-05-09T06:32:06.428540+0000","last_fullsized":"2024-05-09T06:32:06.428540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:50.352947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424448+0000","last_change":"2024-05-09T06:32:05.436497+0000","last_active":"2024-05-09T06:32:06.424448+0000","last_peered":"2024-05-09T06:32:06.424448+0000","last_clean":"2024-05-09T06:32:06.424448+0000","last_became_active":"2024-05-09T06:32:05.436236+0000","last_became_peered":"2024-05-09T06:32:05.436236+0000","last_unstale":"2024-05-09T06:32:06.424448+0000","last_undegraded":"2024-05-09T06:32:06.424448+0000","last_fullsized":"2024-05-09T06:32:06.424448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:13.879666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429618+0000","last_change":"2024-05-09T06:32:05.450670+0000","last_active":"2024-05-09T06:32:06.429618+0000","last_peered":"2024-05-09T06:32:06.429618+0000","last_clean":"2024-05-09T06:32:06.429618+0000","last_became_active":"2024-05-09T06:32:05.450185+0000","last_became_peered":"2024-05-09T06:32:05.450185+0000","last_unstale":"2024-05-09T06:32:06.429618+0000","last_undegraded":"2024-05-09T06:32:06.429618+0000","last_fullsized":"2024-05-09T06:32:06.429618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:28.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424008+0000","last_change":"2024-05-09T06:32:05.430683+0000","last_active":"2024-05-09T06:32:06.424008+0000","last_peered":"2024-05-09T06:32:06.424008+0000","last_clean":"2024-05-09T06:32:06.424008+0000","last_became_active":"2024-05-09T06:32:05.430396+0000","last_became_peered":"2024-05-09T06:32:05.430396+0000","last_unstale":"2024-05-09T06:32:06.424008+0000","last_undegraded":"2024-05-09T06:32:06.424008+0000","last_fullsized":"2024-05-09T06:32:06.424008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:51.921616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428408+0000","last_change":"2024-05-09T06:32:05.454805+0000","last_active":"2024-05-09T06:32:06.428408+0000","last_peered":"2024-05-09T06:32:06.428408+0000","last_clean":"2024-05-09T06:32:06.428408+0000","last_became_active":"2024-05-09T06:32:05.454400+0000","last_became_peered":"2024-05-09T06:32:05.454400+0000","last_unstale":"2024-05-09T06:32:06.428408+0000","last_undegraded":"2024-05-09T06:32:06.428408+0000","last_fullsized":"2024-05-09T06:32:06.428408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:21.800018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427832+0000","last_change":"2024-05-09T06:32:05.432716+0000","last_active":"2024-05-09T06:32:06.427832+0000","last_peered":"2024-05-09T06:32:06.427832+0000","last_clean":"2024-05-09T06:32:06.427832+0000","last_became_active":"2024-05-09T06:32:05.432408+0000","last_became_peered":"2024-05-09T06:32:05.432408+0000","last_unstale":"2024-05-09T06:32:06.427832+0000","last_undegraded":"2024-05-09T06:32:06.427832+0000","last_fullsized":"2024-05-09T06:32:06.427832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:28.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427798+0000","last_change":"2024-05-09T06:32:05.430193+0000","last_active":"2024-05-09T06:32:06.427798+0000","last_peered":"2024-05-09T06:32:06.427798+0000","last_clean":"2024-05-09T06:32:06.427798+0000","last_became_active":"2024-05-09T06:32:05.429905+0000","last_became_peered":"2024-05-09T06:32:05.429905+0000","last_unstale":"2024-05-09T06:32:06.427798+0000","last_undegraded":"2024-05-09T06:32:06.427798+0000","last_fullsized":"2024-05-09T06:32:06.427798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:33.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428739+0000","last_change":"2024-05-09T06:32:05.447723+0000","last_active":"2024-05-09T06:32:06.428739+0000","last_peered":"2024-05-09T06:32:06.428739+0000","last_clean":"2024-05-09T06:32:06.428739+0000","last_became_active":"2024-05-09T06:32:05.447145+0000","last_became_peered":"2024-05-09T06:32:05.447145+0000","last_unstale":"2024-05-09T06:32:06.428739+0000","last_undegraded":"2024-05-09T06:32:06.428739+0000","last_fullsized":"2024-05-09T06:32:06.428739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:04.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430508+0000","last_change":"2024-05-09T06:32:05.452424+0000","last_active":"2024-05-09T06:32:06.430508+0000","last_peered":"2024-05-09T06:32:06.430508+0000","last_clean":"2024-05-09T06:32:06.430508+0000","last_became_active":"2024-05-09T06:32:05.452125+0000","last_became_peered":"2024-05-09T06:32:05.452125+0000","last_unstale":"2024-05-09T06:32:06.430508+0000","last_undegraded":"2024-05-09T06:32:06.430508+0000","last_fullsized":"2024-05-09T06:32:06.430508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:11.194247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424352+0000","last_change":"2024-05-09T06:32:05.451263+0000","last_active":"2024-05-09T06:32:06.424352+0000","last_peered":"2024-05-09T06:32:06.424352+0000","last_clean":"2024-05-09T06:32:06.424352+0000","last_became_active":"2024-05-09T06:32:05.450978+0000","last_became_peered":"2024-05-09T06:32:05.450978+0000","last_unstale":"2024-05-09T06:32:06.424352+0000","last_undegraded":"2024-05-09T06:32:06.424352+0000","last_fullsized":"2024-05-09T06:32:06.424352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:23.516072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428126+0000","last_change":"2024-05-09T06:32:05.441851+0000","last_active":"2024-05-09T06:32:06.428126+0000","last_peered":"2024-05-09T06:32:06.428126+0000","last_clean":"2024-05-09T06:32:06.428126+0000","last_became_active":"2024-05-09T06:32:05.441559+0000","last_became_peered":"2024-05-09T06:32:05.441559+0000","last_unstale":"2024-05-09T06:32:06.428126+0000","last_undegraded":"2024-05-09T06:32:06.428126+0000","last_fullsized":"2024-05-09T06:32:06.428126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:37.044372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429555+0000","last_change":"2024-05-09T06:32:05.444309+0000","last_active":"2024-05-09T06:32:06.429555+0000","last_peered":"2024-05-09T06:32:06.429555+0000","last_clean":"2024-05-09T06:32:06.429555+0000","last_became_active":"2024-05-09T06:32:05.444022+0000","last_became_peered":"2024-05-09T06:32:05.444022+0000","last_unstale":"2024-05-09T06:32:06.429555+0000","last_undegraded":"2024-05-09T06:32:06.429555+0000","last_fullsized":"2024-05-09T06:32:06.429555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:44.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428061+0000","last_change":"2024-05-09T06:32:05.444428+0000","last_active":"2024-05-09T06:32:06.428061+0000","last_peered":"2024-05-09T06:32:06.428061+0000","last_clean":"2024-05-09T06:32:06.428061+0000","last_became_active":"2024-05-09T06:32:05.444146+0000","last_became_peered":"2024-05-09T06:32:05.444146+0000","last_unstale":"2024-05-09T06:32:06.428061+0000","last_undegraded":"2024-05-09T06:32:06.428061+0000","last_fullsized":"2024-05-09T06:32:06.428061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:30.829218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429709+0000","last_change":"2024-05-09T06:32:05.448116+0000","last_active":"2024-05-09T06:32:06.429709+0000","last_peered":"2024-05-09T06:32:06.429709+0000","last_clean":"2024-05-09T06:32:06.429709+0000","last_became_active":"2024-05-09T06:32:05.447820+0000","last_became_peered":"2024-05-09T06:32:05.447820+0000","last_unstale":"2024-05-09T06:32:06.429709+0000","last_undegraded":"2024-05-09T06:32:06.429709+0000","last_fullsized":"2024-05-09T06:32:06.429709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:21.257194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427845+0000","last_change":"2024-05-09T06:32:05.446752+0000","last_active":"2024-05-09T06:32:06.427845+0000","last_peered":"2024-05-09T06:32:06.427845+0000","last_clean":"2024-05-09T06:32:06.427845+0000","last_became_active":"2024-05-09T06:32:05.446494+0000","last_became_peered":"2024-05-09T06:32:05.446494+0000","last_unstale":"2024-05-09T06:32:06.427845+0000","last_undegraded":"2024-05-09T06:32:06.427845+0000","last_fullsized":"2024-05-09T06:32:06.427845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:35.507566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424407+0000","last_change":"2024-05-09T06:32:04.441881+0000","last_active":"2024-05-09T06:32:06.424407+0000","last_peered":"2024-05-09T06:32:06.424407+0000","last_clean":"2024-05-09T06:32:06.424407+0000","last_became_active":"2024-05-09T06:32:04.441564+0000","last_became_peered":"2024-05-09T06:32:04.441564+0000","last_unstale":"2024-05-09T06:32:06.424407+0000","last_undegraded":"2024-05-09T06:32:06.424407+0000","last_fullsized":"2024-05-09T06:32:06.424407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:01.824978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428351+0000","last_change":"2024-05-09T06:32:05.444310+0000","last_active":"2024-05-09T06:32:06.428351+0000","last_peered":"2024-05-09T06:32:06.428351+0000","last_clean":"2024-05-09T06:32:06.428351+0000","last_became_active":"2024-05-09T06:32:05.443546+0000","last_became_peered":"2024-05-09T06:32:05.443546+0000","last_unstale":"2024-05-09T06:32:06.428351+0000","last_undegraded":"2024-05-09T06:32:06.428351+0000","last_fullsized":"2024-05-09T06:32:06.428351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:08.545595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430385+0000","last_change":"2024-05-09T06:32:04.440065+0000","last_active":"2024-05-09T06:32:06.430385+0000","last_peered":"2024-05-09T06:32:06.430385+0000","last_clean":"2024-05-09T06:32:06.430385+0000","last_became_active":"2024-05-09T06:32:04.439727+0000","last_became_peered":"2024-05-09T06:32:04.439727+0000","last_unstale":"2024-05-09T06:32:06.430385+0000","last_undegraded":"2024-05-09T06:32:06.430385+0000","last_fullsized":"2024-05-09T06:32:06.430385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:02.995144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429505+0000","last_change":"2024-05-09T06:32:05.446051+0000","last_active":"2024-05-09T06:32:06.429505+0000","last_peered":"2024-05-09T06:32:06.429505+0000","last_clean":"2024-05-09T06:32:06.429505+0000","last_became_active":"2024-05-09T06:32:05.445752+0000","last_became_peered":"2024-05-09T06:32:05.445752+0000","last_unstale":"2024-05-09T06:32:06.429505+0000","last_undegraded":"2024-05-09T06:32:06.429505+0000","last_fullsized":"2024-05-09T06:32:06.429505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:43.372747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430477+0000","last_change":"2024-05-09T06:32:04.443214+0000","last_active":"2024-05-09T06:32:06.430477+0000","last_peered":"2024-05-09T06:32:06.430477+0000","last_clean":"2024-05-09T06:32:06.430477+0000","last_became_active":"2024-05-09T06:32:04.442700+0000","last_became_peered":"2024-05-09T06:32:04.442700+0000","last_unstale":"2024-05-09T06:32:06.430477+0000","last_undegraded":"2024-05-09T06:32:06.430477+0000","last_fullsized":"2024-05-09T06:32:06.430477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:35.496823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429632+0000","last_change":"2024-05-09T06:32:05.450473+0000","last_active":"2024-05-09T06:32:06.429632+0000","last_peered":"2024-05-09T06:32:06.429632+0000","last_clean":"2024-05-09T06:32:06.429632+0000","last_became_active":"2024-05-09T06:32:05.449824+0000","last_became_peered":"2024-05-09T06:32:05.449824+0000","last_unstale":"2024-05-09T06:32:06.429632+0000","last_undegraded":"2024-05-09T06:32:06.429632+0000","last_fullsized":"2024-05-09T06:32:06.429632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:20:02.491562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425619+0000","last_change":"2024-05-09T06:32:04.443113+0000","last_active":"2024-05-09T06:32:06.425619+0000","last_peered":"2024-05-09T06:32:06.425619+0000","last_clean":"2024-05-09T06:32:06.425619+0000","last_became_active":"2024-05-09T06:32:04.442646+0000","last_became_peered":"2024-05-09T06:32:04.442646+0000","last_unstale":"2024-05-09T06:32:06.425619+0000","last_undegraded":"2024-05-09T06:32:06.425619+0000","last_fullsized":"2024-05-09T06:32:06.425619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:27.405031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427894+0000","last_change":"2024-05-09T06:32:05.441524+0000","last_active":"2024-05-09T06:32:06.427894+0000","last_peered":"2024-05-09T06:32:06.427894+0000","last_clean":"2024-05-09T06:32:06.427894+0000","last_became_active":"2024-05-09T06:32:05.441224+0000","last_became_peered":"2024-05-09T06:32:05.441224+0000","last_unstale":"2024-05-09T06:32:06.427894+0000","last_undegraded":"2024-05-09T06:32:06.427894+0000","last_fullsized":"2024-05-09T06:32:06.427894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:08.574415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429735+0000","last_change":"2024-05-09T06:32:04.440222+0000","last_active":"2024-05-09T06:32:06.429735+0000","last_peered":"2024-05-09T06:32:06.429735+0000","last_clean":"2024-05-09T06:32:06.429735+0000","last_became_active":"2024-05-09T06:32:04.439927+0000","last_became_peered":"2024-05-09T06:32:04.439927+0000","last_unstale":"2024-05-09T06:32:06.429735+0000","last_undegraded":"2024-05-09T06:32:06.429735+0000","last_fullsized":"2024-05-09T06:32:06.429735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:04.276043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428448+0000","last_change":"2024-05-09T06:32:05.439936+0000","last_active":"2024-05-09T06:32:06.428448+0000","last_peered":"2024-05-09T06:32:06.428448+0000","last_clean":"2024-05-09T06:32:06.428448+0000","last_became_active":"2024-05-09T06:32:05.439623+0000","last_became_peered":"2024-05-09T06:32:05.439623+0000","last_unstale":"2024-05-09T06:32:06.428448+0000","last_undegraded":"2024-05-09T06:32:06.428448+0000","last_fullsized":"2024-05-09T06:32:06.428448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:50.352947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429050+0000","last_change":"2024-05-09T06:32:04.438136+0000","last_active":"2024-05-09T06:32:06.429050+0000","last_peered":"2024-05-09T06:32:06.429050+0000","last_clean":"2024-05-09T06:32:06.429050+0000","last_became_active":"2024-05-09T06:32:04.437793+0000","last_became_peered":"2024-05-09T06:32:04.437793+0000","last_unstale":"2024-05-09T06:32:06.429050+0000","last_undegraded":"2024-05-09T06:32:06.429050+0000","last_fullsized":"2024-05-09T06:32:06.429050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:35.496823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429039+0000","last_change":"2024-05-09T06:32:05.446685+0000","last_active":"2024-05-09T06:32:06.429039+0000","last_peered":"2024-05-09T06:32:06.429039+0000","last_clean":"2024-05-09T06:32:06.429039+0000","last_became_active":"2024-05-09T06:32:05.446260+0000","last_became_peered":"2024-05-09T06:32:05.446260+0000","last_unstale":"2024-05-09T06:32:06.429039+0000","last_undegraded":"2024-05-09T06:32:06.429039+0000","last_fullsized":"2024-05-09T06:32:06.429039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:55.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428330+0000","last_change":"2024-05-09T06:32:04.440897+0000","last_active":"2024-05-09T06:32:06.428330+0000","last_peered":"2024-05-09T06:32:06.428330+0000","last_clean":"2024-05-09T06:32:06.428330+0000","last_became_active":"2024-05-09T06:32:04.440603+0000","last_became_peered":"2024-05-09T06:32:04.440603+0000","last_unstale":"2024-05-09T06:32:06.428330+0000","last_undegraded":"2024-05-09T06:32:06.428330+0000","last_fullsized":"2024-05-09T06:32:06.428330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:03.775465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425727+0000","last_change":"2024-05-09T06:32:05.448449+0000","last_active":"2024-05-09T06:32:06.425727+0000","last_peered":"2024-05-09T06:32:06.425727+0000","last_clean":"2024-05-09T06:32:06.425727+0000","last_became_active":"2024-05-09T06:32:05.442306+0000","last_became_peered":"2024-05-09T06:32:05.442306+0000","last_unstale":"2024-05-09T06:32:06.425727+0000","last_undegraded":"2024-05-09T06:32:06.425727+0000","last_fullsized":"2024-05-09T06:32:06.425727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:33.213776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423758+0000","last_change":"2024-05-09T06:32:04.441253+0000","last_active":"2024-05-09T06:32:06.423758+0000","last_peered":"2024-05-09T06:32:06.423758+0000","last_clean":"2024-05-09T06:32:06.423758+0000","last_became_active":"2024-05-09T06:32:04.440770+0000","last_became_peered":"2024-05-09T06:32:04.440770+0000","last_unstale":"2024-05-09T06:32:06.423758+0000","last_undegraded":"2024-05-09T06:32:06.423758+0000","last_fullsized":"2024-05-09T06:32:06.423758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:54.384977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427977+0000","last_change":"2024-05-09T06:32:05.440356+0000","last_active":"2024-05-09T06:32:06.427977+0000","last_peered":"2024-05-09T06:32:06.427977+0000","last_clean":"2024-05-09T06:32:06.427977+0000","last_became_active":"2024-05-09T06:32:05.440069+0000","last_became_peered":"2024-05-09T06:32:05.440069+0000","last_unstale":"2024-05-09T06:32:06.427977+0000","last_undegraded":"2024-05-09T06:32:06.427977+0000","last_fullsized":"2024-05-09T06:32:06.427977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:04.955309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430490+0000","last_change":"2024-05-09T06:32:04.439944+0000","last_active":"2024-05-09T06:32:06.430490+0000","last_peered":"2024-05-09T06:32:06.430490+0000","last_clean":"2024-05-09T06:32:06.430490+0000","last_became_active":"2024-05-09T06:32:04.439443+0000","last_became_peered":"2024-05-09T06:32:04.439443+0000","last_unstale":"2024-05-09T06:32:06.430490+0000","last_undegraded":"2024-05-09T06:32:06.430490+0000","last_fullsized":"2024-05-09T06:32:06.430490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:34.909003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428867+0000","last_change":"2024-05-09T06:32:05.453459+0000","last_active":"2024-05-09T06:32:06.428867+0000","last_peered":"2024-05-09T06:32:06.428867+0000","last_clean":"2024-05-09T06:32:06.428867+0000","last_became_active":"2024-05-09T06:32:05.453155+0000","last_became_peered":"2024-05-09T06:32:05.453155+0000","last_unstale":"2024-05-09T06:32:06.428867+0000","last_undegraded":"2024-05-09T06:32:06.428867+0000","last_fullsized":"2024-05-09T06:32:06.428867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:39:53.867438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428849+0000","last_change":"2024-05-09T06:32:04.438599+0000","last_active":"2024-05-09T06:32:06.428849+0000","last_peered":"2024-05-09T06:32:06.428849+0000","last_clean":"2024-05-09T06:32:06.428849+0000","last_became_active":"2024-05-09T06:32:04.438156+0000","last_became_peered":"2024-05-09T06:32:04.438156+0000","last_unstale":"2024-05-09T06:32:06.428849+0000","last_undegraded":"2024-05-09T06:32:06.428849+0000","last_fullsized":"2024-05-09T06:32:06.428849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:59.105593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429424+0000","last_change":"2024-05-09T06:32:05.432674+0000","last_active":"2024-05-09T06:32:06.429424+0000","last_peered":"2024-05-09T06:32:06.429424+0000","last_clean":"2024-05-09T06:32:06.429424+0000","last_became_active":"2024-05-09T06:32:05.432340+0000","last_became_peered":"2024-05-09T06:32:05.432340+0000","last_unstale":"2024-05-09T06:32:06.429424+0000","last_undegraded":"2024-05-09T06:32:06.429424+0000","last_fullsized":"2024-05-09T06:32:06.429424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:47.662049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424499+0000","last_change":"2024-05-09T06:32:04.444691+0000","last_active":"2024-05-09T06:32:06.424499+0000","last_peered":"2024-05-09T06:32:06.424499+0000","last_clean":"2024-05-09T06:32:06.424499+0000","last_became_active":"2024-05-09T06:32:04.444398+0000","last_became_peered":"2024-05-09T06:32:04.444398+0000","last_unstale":"2024-05-09T06:32:06.424499+0000","last_undegraded":"2024-05-09T06:32:06.424499+0000","last_fullsized":"2024-05-09T06:32:06.424499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:11.610791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428470+0000","last_change":"2024-05-09T06:32:05.441913+0000","last_active":"2024-05-09T06:32:06.428470+0000","last_peered":"2024-05-09T06:32:06.428470+0000","last_clean":"2024-05-09T06:32:06.428470+0000","last_became_active":"2024-05-09T06:32:05.441635+0000","last_became_peered":"2024-05-09T06:32:05.441635+0000","last_unstale":"2024-05-09T06:32:06.428470+0000","last_undegraded":"2024-05-09T06:32:06.428470+0000","last_fullsized":"2024-05-09T06:32:06.428470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:14.868754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425751+0000","last_change":"2024-05-09T06:32:04.432822+0000","last_active":"2024-05-09T06:32:06.425751+0000","last_peered":"2024-05-09T06:32:06.425751+0000","last_clean":"2024-05-09T06:32:06.425751+0000","last_became_active":"2024-05-09T06:32:04.432497+0000","last_became_peered":"2024-05-09T06:32:04.432497+0000","last_unstale":"2024-05-09T06:32:06.425751+0000","last_undegraded":"2024-05-09T06:32:06.425751+0000","last_fullsized":"2024-05-09T06:32:06.425751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:20:01.485858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430229+0000","last_change":"2024-05-09T06:32:05.442529+0000","last_active":"2024-05-09T06:32:06.430229+0000","last_peered":"2024-05-09T06:32:06.430229+0000","last_clean":"2024-05-09T06:32:06.430229+0000","last_became_active":"2024-05-09T06:32:05.441926+0000","last_became_peered":"2024-05-09T06:32:05.441926+0000","last_unstale":"2024-05-09T06:32:06.430229+0000","last_undegraded":"2024-05-09T06:32:06.430229+0000","last_fullsized":"2024-05-09T06:32:06.430229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:48.323284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429626+0000","last_change":"2024-05-09T06:32:04.433926+0000","last_active":"2024-05-09T06:32:06.429626+0000","last_peered":"2024-05-09T06:32:06.429626+0000","last_clean":"2024-05-09T06:32:06.429626+0000","last_became_active":"2024-05-09T06:32:04.433611+0000","last_became_peered":"2024-05-09T06:32:04.433611+0000","last_unstale":"2024-05-09T06:32:06.429626+0000","last_undegraded":"2024-05-09T06:32:06.429626+0000","last_fullsized":"2024-05-09T06:32:06.429626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:29.823513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425562+0000","last_change":"2024-05-09T06:32:05.442937+0000","last_active":"2024-05-09T06:32:06.425562+0000","last_peered":"2024-05-09T06:32:06.425562+0000","last_clean":"2024-05-09T06:32:06.425562+0000","last_became_active":"2024-05-09T06:32:05.442618+0000","last_became_peered":"2024-05-09T06:32:05.442618+0000","last_unstale":"2024-05-09T06:32:06.425562+0000","last_undegraded":"2024-05-09T06:32:06.425562+0000","last_fullsized":"2024-05-09T06:32:06.425562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:44.110160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430263+0000","last_change":"2024-05-09T06:32:04.443631+0000","last_active":"2024-05-09T06:32:06.430263+0000","last_peered":"2024-05-09T06:32:06.430263+0000","last_clean":"2024-05-09T06:32:06.430263+0000","last_became_active":"2024-05-09T06:32:04.442472+0000","last_became_peered":"2024-05-09T06:32:04.442472+0000","last_unstale":"2024-05-09T06:32:06.430263+0000","last_undegraded":"2024-05-09T06:32:06.430263+0000","last_fullsized":"2024-05-09T06:32:06.430263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:13.863049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425671+0000","last_change":"2024-05-09T06:32:05.447261+0000","last_active":"2024-05-09T06:32:06.425671+0000","last_peered":"2024-05-09T06:32:06.425671+0000","last_clean":"2024-05-09T06:32:06.425671+0000","last_became_active":"2024-05-09T06:32:05.446979+0000","last_became_peered":"2024-05-09T06:32:05.446979+0000","last_unstale":"2024-05-09T06:32:06.425671+0000","last_undegraded":"2024-05-09T06:32:06.425671+0000","last_fullsized":"2024-05-09T06:32:06.425671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:43.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423840+0000","last_change":"2024-05-09T06:32:04.443235+0000","last_active":"2024-05-09T06:32:06.423840+0000","last_peered":"2024-05-09T06:32:06.423840+0000","last_clean":"2024-05-09T06:32:06.423840+0000","last_became_active":"2024-05-09T06:32:04.442813+0000","last_became_peered":"2024-05-09T06:32:04.442813+0000","last_unstale":"2024-05-09T06:32:06.423840+0000","last_undegraded":"2024-05-09T06:32:06.423840+0000","last_fullsized":"2024-05-09T06:32:06.423840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:26.977785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425720+0000","last_change":"2024-05-09T06:32:05.448712+0000","last_active":"2024-05-09T06:32:06.425720+0000","last_peered":"2024-05-09T06:32:06.425720+0000","last_clean":"2024-05-09T06:32:06.425720+0000","last_became_active":"2024-05-09T06:32:05.442340+0000","last_became_peered":"2024-05-09T06:32:05.442340+0000","last_unstale":"2024-05-09T06:32:06.425720+0000","last_undegraded":"2024-05-09T06:32:06.425720+0000","last_fullsized":"2024-05-09T06:32:06.425720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:30.875878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428075+0000","last_change":"2024-05-09T06:32:04.432252+0000","last_active":"2024-05-09T06:32:06.428075+0000","last_peered":"2024-05-09T06:32:06.428075+0000","last_clean":"2024-05-09T06:32:06.428075+0000","last_became_active":"2024-05-09T06:32:04.431361+0000","last_became_peered":"2024-05-09T06:32:04.431361+0000","last_unstale":"2024-05-09T06:32:06.428075+0000","last_undegraded":"2024-05-09T06:32:06.428075+0000","last_fullsized":"2024-05-09T06:32:06.428075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:39:52.861734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428568+0000","last_change":"2024-05-09T06:32:05.435262+0000","last_active":"2024-05-09T06:32:06.428568+0000","last_peered":"2024-05-09T06:32:06.428568+0000","last_clean":"2024-05-09T06:32:06.428568+0000","last_became_active":"2024-05-09T06:32:05.434975+0000","last_became_peered":"2024-05-09T06:32:05.434975+0000","last_unstale":"2024-05-09T06:32:06.428568+0000","last_undegraded":"2024-05-09T06:32:06.428568+0000","last_fullsized":"2024-05-09T06:32:06.428568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.617088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430503+0000","last_change":"2024-05-09T06:32:04.444811+0000","last_active":"2024-05-09T06:32:06.430503+0000","last_peered":"2024-05-09T06:32:06.430503+0000","last_clean":"2024-05-09T06:32:06.430503+0000","last_became_active":"2024-05-09T06:32:04.444519+0000","last_became_peered":"2024-05-09T06:32:04.444519+0000","last_unstale":"2024-05-09T06:32:06.430503+0000","last_undegraded":"2024-05-09T06:32:06.430503+0000","last_fullsized":"2024-05-09T06:32:06.430503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:29.001891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423833+0000","last_change":"2024-05-09T06:32:05.443411+0000","last_active":"2024-05-09T06:32:06.423833+0000","last_peered":"2024-05-09T06:32:06.423833+0000","last_clean":"2024-05-09T06:32:06.423833+0000","last_became_active":"2024-05-09T06:32:05.442749+0000","last_became_peered":"2024-05-09T06:32:05.442749+0000","last_unstale":"2024-05-09T06:32:06.423833+0000","last_undegraded":"2024-05-09T06:32:06.423833+0000","last_fullsized":"2024-05-09T06:32:06.423833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:41.345800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429155+0000","last_change":"2024-05-09T06:32:04.432260+0000","last_active":"2024-05-09T06:32:06.429155+0000","last_peered":"2024-05-09T06:32:06.429155+0000","last_clean":"2024-05-09T06:32:06.429155+0000","last_became_active":"2024-05-09T06:32:04.431915+0000","last_became_peered":"2024-05-09T06:32:04.431915+0000","last_unstale":"2024-05-09T06:32:06.429155+0000","last_undegraded":"2024-05-09T06:32:06.429155+0000","last_fullsized":"2024-05-09T06:32:06.429155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:52.511397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424171+0000","last_change":"2024-05-09T06:32:05.435866+0000","last_active":"2024-05-09T06:32:06.424171+0000","last_peered":"2024-05-09T06:32:06.424171+0000","last_clean":"2024-05-09T06:32:06.424171+0000","last_became_active":"2024-05-09T06:32:05.434714+0000","last_became_peered":"2024-05-09T06:32:05.434714+0000","last_unstale":"2024-05-09T06:32:06.424171+0000","last_undegraded":"2024-05-09T06:32:06.424171+0000","last_fullsized":"2024-05-09T06:32:06.424171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:54.472525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424525+0000","last_change":"2024-05-09T06:32:04.440739+0000","last_active":"2024-05-09T06:32:06.424525+0000","last_peered":"2024-05-09T06:32:06.424525+0000","last_clean":"2024-05-09T06:32:06.424525+0000","last_became_active":"2024-05-09T06:32:04.440446+0000","last_became_peered":"2024-05-09T06:32:04.440446+0000","last_unstale":"2024-05-09T06:32:06.424525+0000","last_undegraded":"2024-05-09T06:32:06.424525+0000","last_fullsized":"2024-05-09T06:32:06.424525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:04.021978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428896+0000","last_change":"2024-05-09T06:32:05.435396+0000","last_active":"2024-05-09T06:32:06.428896+0000","last_peered":"2024-05-09T06:32:06.428896+0000","last_clean":"2024-05-09T06:32:06.428896+0000","last_became_active":"2024-05-09T06:32:05.435102+0000","last_became_peered":"2024-05-09T06:32:05.435102+0000","last_unstale":"2024-05-09T06:32:06.428896+0000","last_undegraded":"2024-05-09T06:32:06.428896+0000","last_fullsized":"2024-05-09T06:32:06.428896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:08.545595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428364+0000","last_change":"2024-05-09T06:32:04.441944+0000","last_active":"2024-05-09T06:32:06.428364+0000","last_peered":"2024-05-09T06:32:06.428364+0000","last_clean":"2024-05-09T06:32:06.428364+0000","last_became_active":"2024-05-09T06:32:04.441589+0000","last_became_peered":"2024-05-09T06:32:04.441589+0000","last_unstale":"2024-05-09T06:32:06.428364+0000","last_undegraded":"2024-05-09T06:32:06.428364+0000","last_fullsized":"2024-05-09T06:32:06.428364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:54.435307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428044+0000","last_change":"2024-05-09T06:32:05.449974+0000","last_active":"2024-05-09T06:32:06.428044+0000","last_peered":"2024-05-09T06:32:06.428044+0000","last_clean":"2024-05-09T06:32:06.428044+0000","last_became_active":"2024-05-09T06:32:05.448565+0000","last_became_peered":"2024-05-09T06:32:05.448565+0000","last_unstale":"2024-05-09T06:32:06.428044+0000","last_undegraded":"2024-05-09T06:32:06.428044+0000","last_fullsized":"2024-05-09T06:32:06.428044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:54.655094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429003+0000","last_change":"2024-05-09T06:32:04.434026+0000","last_active":"2024-05-09T06:32:06.429003+0000","last_peered":"2024-05-09T06:32:06.429003+0000","last_clean":"2024-05-09T06:32:06.429003+0000","last_became_active":"2024-05-09T06:32:04.433725+0000","last_became_peered":"2024-05-09T06:32:04.433725+0000","last_unstale":"2024-05-09T06:32:06.429003+0000","last_undegraded":"2024-05-09T06:32:06.429003+0000","last_fullsized":"2024-05-09T06:32:06.429003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:11.863059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428866+0000","last_change":"2024-05-09T06:32:05.451861+0000","last_active":"2024-05-09T06:32:06.428866+0000","last_peered":"2024-05-09T06:32:06.428866+0000","last_clean":"2024-05-09T06:32:06.428866+0000","last_became_active":"2024-05-09T06:32:05.451049+0000","last_became_peered":"2024-05-09T06:32:05.451049+0000","last_unstale":"2024-05-09T06:32:06.428866+0000","last_undegraded":"2024-05-09T06:32:06.428866+0000","last_fullsized":"2024-05-09T06:32:06.428866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:47.662049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428654+0000","last_change":"2024-05-09T06:32:04.433135+0000","last_active":"2024-05-09T06:32:06.428654+0000","last_peered":"2024-05-09T06:32:06.428654+0000","last_clean":"2024-05-09T06:32:06.428654+0000","last_became_active":"2024-05-09T06:32:04.432817+0000","last_became_peered":"2024-05-09T06:32:04.432817+0000","last_unstale":"2024-05-09T06:32:06.428654+0000","last_undegraded":"2024-05-09T06:32:06.428654+0000","last_fullsized":"2024-05-09T06:32:06.428654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:07:42.302696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425847+0000","last_change":"2024-05-09T06:32:05.449545+0000","last_active":"2024-05-09T06:32:06.425847+0000","last_peered":"2024-05-09T06:32:06.425847+0000","last_clean":"2024-05-09T06:32:06.425847+0000","last_became_active":"2024-05-09T06:32:05.449279+0000","last_became_peered":"2024-05-09T06:32:05.449279+0000","last_unstale":"2024-05-09T06:32:06.425847+0000","last_undegraded":"2024-05-09T06:32:06.425847+0000","last_fullsized":"2024-05-09T06:32:06.425847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:04.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428724+0000","last_change":"2024-05-09T06:32:04.433089+0000","last_active":"2024-05-09T06:32:06.428724+0000","last_peered":"2024-05-09T06:32:06.428724+0000","last_clean":"2024-05-09T06:32:06.428724+0000","last_became_active":"2024-05-09T06:32:04.432781+0000","last_became_peered":"2024-05-09T06:32:04.432781+0000","last_unstale":"2024-05-09T06:32:06.428724+0000","last_undegraded":"2024-05-09T06:32:06.428724+0000","last_fullsized":"2024-05-09T06:32:06.428724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:06.529979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428575+0000","last_change":"2024-05-09T06:32:05.448698+0000","last_active":"2024-05-09T06:32:06.428575+0000","last_peered":"2024-05-09T06:32:06.428575+0000","last_clean":"2024-05-09T06:32:06.428575+0000","last_became_active":"2024-05-09T06:32:05.447889+0000","last_became_peered":"2024-05-09T06:32:05.447889+0000","last_unstale":"2024-05-09T06:32:06.428575+0000","last_undegraded":"2024-05-09T06:32:06.428575+0000","last_fullsized":"2024-05-09T06:32:06.428575+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:16.633682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429490+0000","last_change":"2024-05-09T06:32:04.436489+0000","last_active":"2024-05-09T06:32:06.429490+0000","last_peered":"2024-05-09T06:32:06.429490+0000","last_clean":"2024-05-09T06:32:06.429490+0000","last_became_active":"2024-05-09T06:32:04.436148+0000","last_became_peered":"2024-05-09T06:32:04.436148+0000","last_unstale":"2024-05-09T06:32:06.429490+0000","last_undegraded":"2024-05-09T06:32:06.429490+0000","last_fullsized":"2024-05-09T06:32:06.429490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:13.863049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425238+0000","last_change":"2024-05-09T06:32:05.436501+0000","last_active":"2024-05-09T06:32:06.425238+0000","last_peered":"2024-05-09T06:32:06.425238+0000","last_clean":"2024-05-09T06:32:06.425238+0000","last_became_active":"2024-05-09T06:32:05.436196+0000","last_became_peered":"2024-05-09T06:32:05.436196+0000","last_unstale":"2024-05-09T06:32:06.425238+0000","last_undegraded":"2024-05-09T06:32:06.425238+0000","last_fullsized":"2024-05-09T06:32:06.425238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:21:00.111297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.438281+0000","last_change":"2024-05-09T06:32:04.438167+0000","last_active":"2024-05-09T06:32:06.438281+0000","last_peered":"2024-05-09T06:32:06.438281+0000","last_clean":"2024-05-09T06:32:06.438281+0000","last_became_active":"2024-05-09T06:32:04.437870+0000","last_became_peered":"2024-05-09T06:32:04.437870+0000","last_unstale":"2024-05-09T06:32:06.438281+0000","last_undegraded":"2024-05-09T06:32:06.438281+0000","last_fullsized":"2024-05-09T06:32:06.438281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:38.693755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429428+0000","last_change":"2024-05-09T06:32:05.444218+0000","last_active":"2024-05-09T06:32:06.429428+0000","last_peered":"2024-05-09T06:32:06.429428+0000","last_clean":"2024-05-09T06:32:06.429428+0000","last_became_active":"2024-05-09T06:32:05.443930+0000","last_became_peered":"2024-05-09T06:32:05.443930+0000","last_unstale":"2024-05-09T06:32:06.429428+0000","last_undegraded":"2024-05-09T06:32:06.429428+0000","last_fullsized":"2024-05-09T06:32:06.429428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:09.308149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430594+0000","last_change":"2024-05-09T06:32:04.441464+0000","last_active":"2024-05-09T06:32:06.430594+0000","last_peered":"2024-05-09T06:32:06.430594+0000","last_clean":"2024-05-09T06:32:06.430594+0000","last_became_active":"2024-05-09T06:32:04.440040+0000","last_became_peered":"2024-05-09T06:32:04.440040+0000","last_unstale":"2024-05-09T06:32:06.430594+0000","last_undegraded":"2024-05-09T06:32:06.430594+0000","last_fullsized":"2024-05-09T06:32:06.430594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:43.104455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428421+0000","last_change":"2024-05-09T06:32:05.445587+0000","last_active":"2024-05-09T06:32:06.428421+0000","last_peered":"2024-05-09T06:32:06.428421+0000","last_clean":"2024-05-09T06:32:06.428421+0000","last_became_active":"2024-05-09T06:32:05.444735+0000","last_became_peered":"2024-05-09T06:32:05.444735+0000","last_unstale":"2024-05-09T06:32:06.428421+0000","last_undegraded":"2024-05-09T06:32:06.428421+0000","last_fullsized":"2024-05-09T06:32:06.428421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:15.947058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430301+0000","last_change":"2024-05-09T06:32:04.440992+0000","last_active":"2024-05-09T06:32:06.430301+0000","last_peered":"2024-05-09T06:32:06.430301+0000","last_clean":"2024-05-09T06:32:06.430301+0000","last_became_active":"2024-05-09T06:32:04.439755+0000","last_became_peered":"2024-05-09T06:32:04.439755+0000","last_unstale":"2024-05-09T06:32:06.430301+0000","last_undegraded":"2024-05-09T06:32:06.430301+0000","last_fullsized":"2024-05-09T06:32:06.430301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:59.105593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425367+0000","last_change":"2024-05-09T06:32:05.443456+0000","last_active":"2024-05-09T06:32:06.425367+0000","last_peered":"2024-05-09T06:32:06.425367+0000","last_clean":"2024-05-09T06:32:06.425367+0000","last_became_active":"2024-05-09T06:32:05.443146+0000","last_became_peered":"2024-05-09T06:32:05.443146+0000","last_unstale":"2024-05-09T06:32:06.425367+0000","last_undegraded":"2024-05-09T06:32:06.425367+0000","last_fullsized":"2024-05-09T06:32:06.425367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:35.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425105+0000","last_change":"2024-05-09T06:32:04.442472+0000","last_active":"2024-05-09T06:32:06.425105+0000","last_peered":"2024-05-09T06:32:06.425105+0000","last_clean":"2024-05-09T06:32:06.425105+0000","last_became_active":"2024-05-09T06:32:04.442184+0000","last_became_peered":"2024-05-09T06:32:04.442184+0000","last_unstale":"2024-05-09T06:32:06.425105+0000","last_undegraded":"2024-05-09T06:32:06.425105+0000","last_fullsized":"2024-05-09T06:32:06.425105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:44.494387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425532+0000","last_change":"2024-05-09T06:32:05.445778+0000","last_active":"2024-05-09T06:32:06.425532+0000","last_peered":"2024-05-09T06:32:06.425532+0000","last_clean":"2024-05-09T06:32:06.425532+0000","last_became_active":"2024-05-09T06:32:05.442727+0000","last_became_peered":"2024-05-09T06:32:05.442727+0000","last_unstale":"2024-05-09T06:32:06.425532+0000","last_undegraded":"2024-05-09T06:32:06.425532+0000","last_fullsized":"2024-05-09T06:32:06.425532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:30.007595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423809+0000","last_change":"2024-05-09T06:32:04.440551+0000","last_active":"2024-05-09T06:32:06.423809+0000","last_peered":"2024-05-09T06:32:06.423809+0000","last_clean":"2024-05-09T06:32:06.423809+0000","last_became_active":"2024-05-09T06:32:04.439970+0000","last_became_peered":"2024-05-09T06:32:04.439970+0000","last_unstale":"2024-05-09T06:32:06.423809+0000","last_undegraded":"2024-05-09T06:32:06.423809+0000","last_fullsized":"2024-05-09T06:32:06.423809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:37.790237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428950+0000","last_change":"2024-05-09T06:32:05.442577+0000","last_active":"2024-05-09T06:32:06.428950+0000","last_peered":"2024-05-09T06:32:06.428950+0000","last_clean":"2024-05-09T06:32:06.428950+0000","last_became_active":"2024-05-09T06:32:05.442291+0000","last_became_peered":"2024-05-09T06:32:05.442291+0000","last_unstale":"2024-05-09T06:32:06.428950+0000","last_undegraded":"2024-05-09T06:32:06.428950+0000","last_fullsized":"2024-05-09T06:32:06.428950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:35.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425586+0000","last_change":"2024-05-09T06:32:04.434957+0000","last_active":"2024-05-09T06:32:06.425586+0000","last_peered":"2024-05-09T06:32:06.425586+0000","last_clean":"2024-05-09T06:32:06.425586+0000","last_became_active":"2024-05-09T06:32:04.434661+0000","last_became_peered":"2024-05-09T06:32:04.434661+0000","last_unstale":"2024-05-09T06:32:06.425586+0000","last_undegraded":"2024-05-09T06:32:06.425586+0000","last_fullsized":"2024-05-09T06:32:06.425586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:54.435307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428767+0000","last_change":"2024-05-09T06:32:05.446985+0000","last_active":"2024-05-09T06:32:06.428767+0000","last_peered":"2024-05-09T06:32:06.428767+0000","last_clean":"2024-05-09T06:32:06.428767+0000","last_became_active":"2024-05-09T06:32:05.446109+0000","last_became_peered":"2024-05-09T06:32:05.446109+0000","last_unstale":"2024-05-09T06:32:06.428767+0000","last_undegraded":"2024-05-09T06:32:06.428767+0000","last_fullsized":"2024-05-09T06:32:06.428767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:21:00.111297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427822+0000","last_change":"2024-05-09T06:32:04.431721+0000","last_active":"2024-05-09T06:32:06.427822+0000","last_peered":"2024-05-09T06:32:06.427822+0000","last_clean":"2024-05-09T06:32:06.427822+0000","last_became_active":"2024-05-09T06:32:04.431373+0000","last_became_peered":"2024-05-09T06:32:04.431373+0000","last_unstale":"2024-05-09T06:32:06.427822+0000","last_undegraded":"2024-05-09T06:32:06.427822+0000","last_fullsized":"2024-05-09T06:32:06.427822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430598+0000","last_change":"2024-05-09T06:32:05.445098+0000","last_active":"2024-05-09T06:32:06.430598+0000","last_peered":"2024-05-09T06:32:06.430598+0000","last_clean":"2024-05-09T06:32:06.430598+0000","last_became_active":"2024-05-09T06:32:05.444765+0000","last_became_peered":"2024-05-09T06:32:05.444765+0000","last_unstale":"2024-05-09T06:32:06.430598+0000","last_undegraded":"2024-05-09T06:32:06.430598+0000","last_fullsized":"2024-05-09T06:32:06.430598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.643747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429435+0000","last_change":"2024-05-09T06:32:04.438318+0000","last_active":"2024-05-09T06:32:06.429435+0000","last_peered":"2024-05-09T06:32:06.429435+0000","last_clean":"2024-05-09T06:32:06.429435+0000","last_became_active":"2024-05-09T06:32:04.438002+0000","last_became_peered":"2024-05-09T06:32:04.438002+0000","last_unstale":"2024-05-09T06:32:06.429435+0000","last_undegraded":"2024-05-09T06:32:06.429435+0000","last_fullsized":"2024-05-09T06:32:06.429435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:32.855503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429809+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:32:06.429809+0000","last_peered":"2024-05-09T06:32:06.429809+0000","last_clean":"2024-05-09T06:32:06.429809+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:32:06.429809+0000","last_undegraded":"2024-05-09T06:32:06.429809+0000","last_fullsized":"2024-05-09T06:32:06.429809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":1027612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":438834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":437256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":438938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":958464,"data_stored":437804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":437964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":437334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":1027590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:32:09.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:32:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.607+0000 7ff1f4f9e640 1 Processor -- start 2024-05-09T06:32:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.607+0000 7ff1f4f9e640 1 -- start start 2024-05-09T06:32:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1f4f9e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1f00ce710 0x7ff1f00b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1f4f9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1f4f9e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f00d1000 con 0x7ff1f015e190 2024-05-09T06:32:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1f4f9e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f00d0710 con 0x7ff1f00ce710 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1f4f9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00b3320 con 0x7ff1f00ceaf0 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1ef7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1f00ce710 0x7ff1f00b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1eeffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1effff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff1f015e190 0x7ff1f006c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:38736/0 (socket says 172.21.15.1:38736) 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1eeffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:59318/0 (socket says 172.21.15.1:59318) 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1ef7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1f00ce710 0x7ff1f00b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:46602/0 (socket says 172.21.15.1:46602) 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.608+0000 7ff1effff640 1 -- 172.21.15.1:0/3550593132 learned_addr learned my addr 172.21.15.1:0/3550593132 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 msgr2=0x7ff1f015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1ef7fe640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/3550593132 >> v1:172.21.15.6:6789/0 conn(0x7ff1f015e190 legacy=0x7ff1f006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/3550593132 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00d09b0 con 0x7ff1f00ce710 2024-05-09T06:32:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1eeffd640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:32:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.609+0000 7ff1ef7fe640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24443f0267f8e513 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1ee7fc640 1 -- 172.21.15.1:0/3550593132 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037070 con 0x7ff1f00ce710 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1ee7fc640 1 -- 172.21.15.1:0/3550593132 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e002ae50 con 0x7ff1f00ce710 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1ee7fc640 1 -- 172.21.15.1:0/3550593132 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0032660 con 0x7ff1f00ce710 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00ce710 msgr2=0x7ff1f00b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e002a100 comp rx=0 tx=0).stop 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/3550593132 shutdown_connections 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/3550593132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/3550593132 >> 172.21.15.1:0/3550593132 conn(0x7ff1f00b3640 msgr2=0x7ff1f00b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/3550593132 shutdown_connections 2024-05-09T06:32:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.610+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/3550593132 wait complete. 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 Processor -- start 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 -- start start 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f006f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f015e190 0x7ff1f006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00b2f60 con 0x7ff1f00ce710 2024-05-09T06:32:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f015e620 con 0x7ff1f015e190 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.612+0000 7ff1f4f9e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00b30e0 con 0x7ff1f00ceaf0 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52448/0 (socket says 172.21.15.1:52448) 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/904237244 learned_addr learned my addr 172.21.15.1:0/904237244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1effff640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f015e190 0x7ff1f006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1eeffd640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 msgr2=0x7ff1f006f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f006f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f015e190 msgr2=0x7ff1f006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f015e190 0x7ff1f006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 -- 172.21.15.1:0/904237244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00b6350 con 0x7ff1f00ce710 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1ef7fe640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0042070 con 0x7ff1f00ce710 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e003e460 con 0x7ff1f00ce710 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00b6630 con 0x7ff1f00ce710 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037040 con 0x7ff1f00ce710 2024-05-09T06:32:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.613+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00b6bf0 con 0x7ff1f00ce710 2024-05-09T06:32:09.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.615+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f00b2b60 con 0x7ff1f00ce710 2024-05-09T06:32:09.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.615+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1e0045070 con 0x7ff1f00ce710 2024-05-09T06:32:09.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.615+0000 7ff1d7fff640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 0x7ff1dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.616+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7ff1e00ab670 con 0x7ff1f00ce710 2024-05-09T06:32:09.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.616+0000 7ff1eeffd640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 0x7ff1dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.616+0000 7ff1eeffd640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 0x7ff1dc06e320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009560 tx=0x7ff1e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.620+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1f00b2b60 con 0x7ff1f00ce710 2024-05-09T06:32:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.800+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1f00752b0 con 0x7ff1dc06be60 2024-05-09T06:32:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.806+0000 7ff1d7fff640 1 -- 172.21.15.1:0/904237244 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252072 (secure 0 0 0) 0x7ff1f00752b0 con 0x7ff1dc06be60 2024-05-09T06:32:09.790 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:32:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 msgr2=0x7ff1dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 0x7ff1dc06e320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009560 tx=0x7ff1e4009190 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 msgr2=0x7ff1f00b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00097a0 tx=0x7ff1e00046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 shutdown_connections 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff1dc06be60 0x7ff1dc06e320 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1f015e190 0x7ff1f006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1f00ceaf0 0x7ff1f006f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 --2- 172.21.15.1:0/904237244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f00ce710 0x7ff1f00b4960 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 >> 172.21.15.1:0/904237244 conn(0x7ff1f00b3640 msgr2=0x7ff1f00b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 shutdown_connections 2024-05-09T06:32:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.809+0000 7ff1f4f9e640 1 -- 172.21.15.1:0/904237244 wait complete. 2024-05-09T06:32:09.802 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-05-09T06:32:09.130732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245704,"kb_used_data":8680,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729016,"statfs":{"total":773094113280,"available":772842512384,"internally_reserved":0,"allocated":8888320,"data_stored":4683332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.717381"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430015+0000","last_change":"2024-05-09T06:32:05.448168+0000","last_active":"2024-05-09T06:32:06.430015+0000","last_peered":"2024-05-09T06:32:06.430015+0000","last_clean":"2024-05-09T06:32:06.430015+0000","last_became_active":"2024-05-09T06:32:05.447857+0000","last_became_peered":"2024-05-09T06:32:05.447857+0000","last_unstale":"2024-05-09T06:32:06.430015+0000","last_undegraded":"2024-05-09T06:32:06.430015+0000","last_fullsized":"2024-05-09T06:32:06.430015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:23.611187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428731+0000","last_change":"2024-05-09T06:32:05.433758+0000","last_active":"2024-05-09T06:32:06.428731+0000","last_peered":"2024-05-09T06:32:06.428731+0000","last_clean":"2024-05-09T06:32:06.428731+0000","last_became_active":"2024-05-09T06:32:05.433478+0000","last_became_peered":"2024-05-09T06:32:05.433478+0000","last_unstale":"2024-05-09T06:32:06.428731+0000","last_undegraded":"2024-05-09T06:32:06.428731+0000","last_fullsized":"2024-05-09T06:32:06.428731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:58.313762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428704+0000","last_change":"2024-05-09T06:32:05.443243+0000","last_active":"2024-05-09T06:32:06.428704+0000","last_peered":"2024-05-09T06:32:06.428704+0000","last_clean":"2024-05-09T06:32:06.428704+0000","last_became_active":"2024-05-09T06:32:05.442955+0000","last_became_peered":"2024-05-09T06:32:05.442955+0000","last_unstale":"2024-05-09T06:32:06.428704+0000","last_undegraded":"2024-05-09T06:32:06.428704+0000","last_fullsized":"2024-05-09T06:32:06.428704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:36.502528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430061+0000","last_change":"2024-05-09T06:32:05.449877+0000","last_active":"2024-05-09T06:32:06.430061+0000","last_peered":"2024-05-09T06:32:06.430061+0000","last_clean":"2024-05-09T06:32:06.430061+0000","last_became_active":"2024-05-09T06:32:05.449461+0000","last_became_peered":"2024-05-09T06:32:05.449461+0000","last_unstale":"2024-05-09T06:32:06.430061+0000","last_undegraded":"2024-05-09T06:32:06.430061+0000","last_fullsized":"2024-05-09T06:32:06.430061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:39.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423946+0000","last_change":"2024-05-09T06:32:05.440360+0000","last_active":"2024-05-09T06:32:06.423946+0000","last_peered":"2024-05-09T06:32:06.423946+0000","last_clean":"2024-05-09T06:32:06.423946+0000","last_became_active":"2024-05-09T06:32:05.440079+0000","last_became_peered":"2024-05-09T06:32:05.440079+0000","last_unstale":"2024-05-09T06:32:06.423946+0000","last_undegraded":"2024-05-09T06:32:06.423946+0000","last_fullsized":"2024-05-09T06:32:06.423946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:28.313413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429205+0000","last_change":"2024-05-09T06:32:05.451724+0000","last_active":"2024-05-09T06:32:06.429205+0000","last_peered":"2024-05-09T06:32:06.429205+0000","last_clean":"2024-05-09T06:32:06.429205+0000","last_became_active":"2024-05-09T06:32:05.451447+0000","last_became_peered":"2024-05-09T06:32:05.451447+0000","last_unstale":"2024-05-09T06:32:06.429205+0000","last_undegraded":"2024-05-09T06:32:06.429205+0000","last_fullsized":"2024-05-09T06:32:06.429205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:15.947058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425300+0000","last_change":"2024-05-09T06:32:05.437648+0000","last_active":"2024-05-09T06:32:06.425300+0000","last_peered":"2024-05-09T06:32:06.425300+0000","last_clean":"2024-05-09T06:32:06.425300+0000","last_became_active":"2024-05-09T06:32:05.437379+0000","last_became_peered":"2024-05-09T06:32:05.437379+0000","last_unstale":"2024-05-09T06:32:06.425300+0000","last_undegraded":"2024-05-09T06:32:06.425300+0000","last_fullsized":"2024-05-09T06:32:06.425300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:04.000848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423779+0000","last_change":"2024-05-09T06:32:05.448861+0000","last_active":"2024-05-09T06:32:06.423779+0000","last_peered":"2024-05-09T06:32:06.423779+0000","last_clean":"2024-05-09T06:32:06.423779+0000","last_became_active":"2024-05-09T06:32:05.448550+0000","last_became_peered":"2024-05-09T06:32:05.448550+0000","last_unstale":"2024-05-09T06:32:06.423779+0000","last_undegraded":"2024-05-09T06:32:06.423779+0000","last_fullsized":"2024-05-09T06:32:06.423779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:40.242265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429544+0000","last_change":"2024-05-09T06:32:05.445624+0000","last_active":"2024-05-09T06:32:06.429544+0000","last_peered":"2024-05-09T06:32:06.429544+0000","last_clean":"2024-05-09T06:32:06.429544+0000","last_became_active":"2024-05-09T06:32:05.445342+0000","last_became_peered":"2024-05-09T06:32:05.445342+0000","last_unstale":"2024-05-09T06:32:06.429544+0000","last_undegraded":"2024-05-09T06:32:06.429544+0000","last_fullsized":"2024-05-09T06:32:06.429544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:45.500091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428566+0000","last_change":"2024-05-09T06:32:05.447975+0000","last_active":"2024-05-09T06:32:06.428566+0000","last_peered":"2024-05-09T06:32:06.428566+0000","last_clean":"2024-05-09T06:32:06.428566+0000","last_became_active":"2024-05-09T06:32:05.447456+0000","last_became_peered":"2024-05-09T06:32:05.447456+0000","last_unstale":"2024-05-09T06:32:06.428566+0000","last_undegraded":"2024-05-09T06:32:06.428566+0000","last_fullsized":"2024-05-09T06:32:06.428566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:33.213776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428050+0000","last_change":"2024-05-09T06:32:05.440652+0000","last_active":"2024-05-09T06:32:06.428050+0000","last_peered":"2024-05-09T06:32:06.428050+0000","last_clean":"2024-05-09T06:32:06.428050+0000","last_became_active":"2024-05-09T06:32:05.440396+0000","last_became_peered":"2024-05-09T06:32:05.440396+0000","last_unstale":"2024-05-09T06:32:06.428050+0000","last_undegraded":"2024-05-09T06:32:06.428050+0000","last_fullsized":"2024-05-09T06:32:06.428050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:55:19.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430237+0000","last_change":"2024-05-09T06:32:05.442772+0000","last_active":"2024-05-09T06:32:06.430237+0000","last_peered":"2024-05-09T06:32:06.430237+0000","last_clean":"2024-05-09T06:32:06.430237+0000","last_became_active":"2024-05-09T06:32:05.442240+0000","last_became_peered":"2024-05-09T06:32:05.442240+0000","last_unstale":"2024-05-09T06:32:06.430237+0000","last_undegraded":"2024-05-09T06:32:06.430237+0000","last_fullsized":"2024-05-09T06:32:06.430237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:44.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424080+0000","last_change":"2024-05-09T06:32:05.430314+0000","last_active":"2024-05-09T06:32:06.424080+0000","last_peered":"2024-05-09T06:32:06.424080+0000","last_clean":"2024-05-09T06:32:06.424080+0000","last_became_active":"2024-05-09T06:32:05.430024+0000","last_became_peered":"2024-05-09T06:32:05.430024+0000","last_unstale":"2024-05-09T06:32:06.424080+0000","last_undegraded":"2024-05-09T06:32:06.424080+0000","last_fullsized":"2024-05-09T06:32:06.424080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:31.152509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429260+0000","last_change":"2024-05-09T06:32:05.452453+0000","last_active":"2024-05-09T06:32:06.429260+0000","last_peered":"2024-05-09T06:32:06.429260+0000","last_clean":"2024-05-09T06:32:06.429260+0000","last_became_active":"2024-05-09T06:32:05.452172+0000","last_became_peered":"2024-05-09T06:32:05.452172+0000","last_unstale":"2024-05-09T06:32:06.429260+0000","last_undegraded":"2024-05-09T06:32:06.429260+0000","last_fullsized":"2024-05-09T06:32:06.429260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:58.313762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428234+0000","last_change":"2024-05-09T06:32:05.433432+0000","last_active":"2024-05-09T06:32:06.428234+0000","last_peered":"2024-05-09T06:32:06.428234+0000","last_clean":"2024-05-09T06:32:06.428234+0000","last_became_active":"2024-05-09T06:32:05.433126+0000","last_became_peered":"2024-05-09T06:32:05.433126+0000","last_unstale":"2024-05-09T06:32:06.428234+0000","last_undegraded":"2024-05-09T06:32:06.428234+0000","last_fullsized":"2024-05-09T06:32:06.428234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.617088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429541+0000","last_change":"2024-05-09T06:32:05.452517+0000","last_active":"2024-05-09T06:32:06.429541+0000","last_peered":"2024-05-09T06:32:06.429541+0000","last_clean":"2024-05-09T06:32:06.429541+0000","last_became_active":"2024-05-09T06:32:05.452218+0000","last_became_peered":"2024-05-09T06:32:05.452218+0000","last_unstale":"2024-05-09T06:32:06.429541+0000","last_undegraded":"2024-05-09T06:32:06.429541+0000","last_fullsized":"2024-05-09T06:32:06.429541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:48.323284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424396+0000","last_change":"2024-05-09T06:32:05.450790+0000","last_active":"2024-05-09T06:32:06.424396+0000","last_peered":"2024-05-09T06:32:06.424396+0000","last_clean":"2024-05-09T06:32:06.424396+0000","last_became_active":"2024-05-09T06:32:05.450511+0000","last_became_peered":"2024-05-09T06:32:05.450511+0000","last_unstale":"2024-05-09T06:32:06.424396+0000","last_undegraded":"2024-05-09T06:32:06.424396+0000","last_fullsized":"2024-05-09T06:32:06.424396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:23.532300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428916+0000","last_change":"2024-05-09T06:32:05.447446+0000","last_active":"2024-05-09T06:32:06.428916+0000","last_peered":"2024-05-09T06:32:06.428916+0000","last_clean":"2024-05-09T06:32:06.428916+0000","last_became_active":"2024-05-09T06:32:05.446966+0000","last_became_peered":"2024-05-09T06:32:05.446966+0000","last_unstale":"2024-05-09T06:32:06.428916+0000","last_undegraded":"2024-05-09T06:32:06.428916+0000","last_fullsized":"2024-05-09T06:32:06.428916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:04.000848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428540+0000","last_change":"2024-05-09T06:32:05.435911+0000","last_active":"2024-05-09T06:32:06.428540+0000","last_peered":"2024-05-09T06:32:06.428540+0000","last_clean":"2024-05-09T06:32:06.428540+0000","last_became_active":"2024-05-09T06:32:05.435616+0000","last_became_peered":"2024-05-09T06:32:05.435616+0000","last_unstale":"2024-05-09T06:32:06.428540+0000","last_undegraded":"2024-05-09T06:32:06.428540+0000","last_fullsized":"2024-05-09T06:32:06.428540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:50.352947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424448+0000","last_change":"2024-05-09T06:32:05.436497+0000","last_active":"2024-05-09T06:32:06.424448+0000","last_peered":"2024-05-09T06:32:06.424448+0000","last_clean":"2024-05-09T06:32:06.424448+0000","last_became_active":"2024-05-09T06:32:05.436236+0000","last_became_peered":"2024-05-09T06:32:05.436236+0000","last_unstale":"2024-05-09T06:32:06.424448+0000","last_undegraded":"2024-05-09T06:32:06.424448+0000","last_fullsized":"2024-05-09T06:32:06.424448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:13.879666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429618+0000","last_change":"2024-05-09T06:32:05.450670+0000","last_active":"2024-05-09T06:32:06.429618+0000","last_peered":"2024-05-09T06:32:06.429618+0000","last_clean":"2024-05-09T06:32:06.429618+0000","last_became_active":"2024-05-09T06:32:05.450185+0000","last_became_peered":"2024-05-09T06:32:05.450185+0000","last_unstale":"2024-05-09T06:32:06.429618+0000","last_undegraded":"2024-05-09T06:32:06.429618+0000","last_fullsized":"2024-05-09T06:32:06.429618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:28.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424008+0000","last_change":"2024-05-09T06:32:05.430683+0000","last_active":"2024-05-09T06:32:06.424008+0000","last_peered":"2024-05-09T06:32:06.424008+0000","last_clean":"2024-05-09T06:32:06.424008+0000","last_became_active":"2024-05-09T06:32:05.430396+0000","last_became_peered":"2024-05-09T06:32:05.430396+0000","last_unstale":"2024-05-09T06:32:06.424008+0000","last_undegraded":"2024-05-09T06:32:06.424008+0000","last_fullsized":"2024-05-09T06:32:06.424008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:51.921616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428408+0000","last_change":"2024-05-09T06:32:05.454805+0000","last_active":"2024-05-09T06:32:06.428408+0000","last_peered":"2024-05-09T06:32:06.428408+0000","last_clean":"2024-05-09T06:32:06.428408+0000","last_became_active":"2024-05-09T06:32:05.454400+0000","last_became_peered":"2024-05-09T06:32:05.454400+0000","last_unstale":"2024-05-09T06:32:06.428408+0000","last_undegraded":"2024-05-09T06:32:06.428408+0000","last_fullsized":"2024-05-09T06:32:06.428408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:21.800018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427832+0000","last_change":"2024-05-09T06:32:05.432716+0000","last_active":"2024-05-09T06:32:06.427832+0000","last_peered":"2024-05-09T06:32:06.427832+0000","last_clean":"2024-05-09T06:32:06.427832+0000","last_became_active":"2024-05-09T06:32:05.432408+0000","last_became_peered":"2024-05-09T06:32:05.432408+0000","last_unstale":"2024-05-09T06:32:06.427832+0000","last_undegraded":"2024-05-09T06:32:06.427832+0000","last_fullsized":"2024-05-09T06:32:06.427832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:28.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427798+0000","last_change":"2024-05-09T06:32:05.430193+0000","last_active":"2024-05-09T06:32:06.427798+0000","last_peered":"2024-05-09T06:32:06.427798+0000","last_clean":"2024-05-09T06:32:06.427798+0000","last_became_active":"2024-05-09T06:32:05.429905+0000","last_became_peered":"2024-05-09T06:32:05.429905+0000","last_unstale":"2024-05-09T06:32:06.427798+0000","last_undegraded":"2024-05-09T06:32:06.427798+0000","last_fullsized":"2024-05-09T06:32:06.427798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:33.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428739+0000","last_change":"2024-05-09T06:32:05.447723+0000","last_active":"2024-05-09T06:32:06.428739+0000","last_peered":"2024-05-09T06:32:06.428739+0000","last_clean":"2024-05-09T06:32:06.428739+0000","last_became_active":"2024-05-09T06:32:05.447145+0000","last_became_peered":"2024-05-09T06:32:05.447145+0000","last_unstale":"2024-05-09T06:32:06.428739+0000","last_undegraded":"2024-05-09T06:32:06.428739+0000","last_fullsized":"2024-05-09T06:32:06.428739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:04.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430508+0000","last_change":"2024-05-09T06:32:05.452424+0000","last_active":"2024-05-09T06:32:06.430508+0000","last_peered":"2024-05-09T06:32:06.430508+0000","last_clean":"2024-05-09T06:32:06.430508+0000","last_became_active":"2024-05-09T06:32:05.452125+0000","last_became_peered":"2024-05-09T06:32:05.452125+0000","last_unstale":"2024-05-09T06:32:06.430508+0000","last_undegraded":"2024-05-09T06:32:06.430508+0000","last_fullsized":"2024-05-09T06:32:06.430508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:11.194247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424352+0000","last_change":"2024-05-09T06:32:05.451263+0000","last_active":"2024-05-09T06:32:06.424352+0000","last_peered":"2024-05-09T06:32:06.424352+0000","last_clean":"2024-05-09T06:32:06.424352+0000","last_became_active":"2024-05-09T06:32:05.450978+0000","last_became_peered":"2024-05-09T06:32:05.450978+0000","last_unstale":"2024-05-09T06:32:06.424352+0000","last_undegraded":"2024-05-09T06:32:06.424352+0000","last_fullsized":"2024-05-09T06:32:06.424352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:23.516072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428126+0000","last_change":"2024-05-09T06:32:05.441851+0000","last_active":"2024-05-09T06:32:06.428126+0000","last_peered":"2024-05-09T06:32:06.428126+0000","last_clean":"2024-05-09T06:32:06.428126+0000","last_became_active":"2024-05-09T06:32:05.441559+0000","last_became_peered":"2024-05-09T06:32:05.441559+0000","last_unstale":"2024-05-09T06:32:06.428126+0000","last_undegraded":"2024-05-09T06:32:06.428126+0000","last_fullsized":"2024-05-09T06:32:06.428126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:37.044372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429555+0000","last_change":"2024-05-09T06:32:05.444309+0000","last_active":"2024-05-09T06:32:06.429555+0000","last_peered":"2024-05-09T06:32:06.429555+0000","last_clean":"2024-05-09T06:32:06.429555+0000","last_became_active":"2024-05-09T06:32:05.444022+0000","last_became_peered":"2024-05-09T06:32:05.444022+0000","last_unstale":"2024-05-09T06:32:06.429555+0000","last_undegraded":"2024-05-09T06:32:06.429555+0000","last_fullsized":"2024-05-09T06:32:06.429555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:44.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428061+0000","last_change":"2024-05-09T06:32:05.444428+0000","last_active":"2024-05-09T06:32:06.428061+0000","last_peered":"2024-05-09T06:32:06.428061+0000","last_clean":"2024-05-09T06:32:06.428061+0000","last_became_active":"2024-05-09T06:32:05.444146+0000","last_became_peered":"2024-05-09T06:32:05.444146+0000","last_unstale":"2024-05-09T06:32:06.428061+0000","last_undegraded":"2024-05-09T06:32:06.428061+0000","last_fullsized":"2024-05-09T06:32:06.428061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:30.829218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429709+0000","last_change":"2024-05-09T06:32:05.448116+0000","last_active":"2024-05-09T06:32:06.429709+0000","last_peered":"2024-05-09T06:32:06.429709+0000","last_clean":"2024-05-09T06:32:06.429709+0000","last_became_active":"2024-05-09T06:32:05.447820+0000","last_became_peered":"2024-05-09T06:32:05.447820+0000","last_unstale":"2024-05-09T06:32:06.429709+0000","last_undegraded":"2024-05-09T06:32:06.429709+0000","last_fullsized":"2024-05-09T06:32:06.429709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:21.257194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427845+0000","last_change":"2024-05-09T06:32:05.446752+0000","last_active":"2024-05-09T06:32:06.427845+0000","last_peered":"2024-05-09T06:32:06.427845+0000","last_clean":"2024-05-09T06:32:06.427845+0000","last_became_active":"2024-05-09T06:32:05.446494+0000","last_became_peered":"2024-05-09T06:32:05.446494+0000","last_unstale":"2024-05-09T06:32:06.427845+0000","last_undegraded":"2024-05-09T06:32:06.427845+0000","last_fullsized":"2024-05-09T06:32:06.427845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:35.507566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424407+0000","last_change":"2024-05-09T06:32:04.441881+0000","last_active":"2024-05-09T06:32:06.424407+0000","last_peered":"2024-05-09T06:32:06.424407+0000","last_clean":"2024-05-09T06:32:06.424407+0000","last_became_active":"2024-05-09T06:32:04.441564+0000","last_became_peered":"2024-05-09T06:32:04.441564+0000","last_unstale":"2024-05-09T06:32:06.424407+0000","last_undegraded":"2024-05-09T06:32:06.424407+0000","last_fullsized":"2024-05-09T06:32:06.424407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:01.824978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428351+0000","last_change":"2024-05-09T06:32:05.444310+0000","last_active":"2024-05-09T06:32:06.428351+0000","last_peered":"2024-05-09T06:32:06.428351+0000","last_clean":"2024-05-09T06:32:06.428351+0000","last_became_active":"2024-05-09T06:32:05.443546+0000","last_became_peered":"2024-05-09T06:32:05.443546+0000","last_unstale":"2024-05-09T06:32:06.428351+0000","last_undegraded":"2024-05-09T06:32:06.428351+0000","last_fullsized":"2024-05-09T06:32:06.428351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:08.545595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430385+0000","last_change":"2024-05-09T06:32:04.440065+0000","last_active":"2024-05-09T06:32:06.430385+0000","last_peered":"2024-05-09T06:32:06.430385+0000","last_clean":"2024-05-09T06:32:06.430385+0000","last_became_active":"2024-05-09T06:32:04.439727+0000","last_became_peered":"2024-05-09T06:32:04.439727+0000","last_unstale":"2024-05-09T06:32:06.430385+0000","last_undegraded":"2024-05-09T06:32:06.430385+0000","last_fullsized":"2024-05-09T06:32:06.430385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:02.995144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429505+0000","last_change":"2024-05-09T06:32:05.446051+0000","last_active":"2024-05-09T06:32:06.429505+0000","last_peered":"2024-05-09T06:32:06.429505+0000","last_clean":"2024-05-09T06:32:06.429505+0000","last_became_active":"2024-05-09T06:32:05.445752+0000","last_became_peered":"2024-05-09T06:32:05.445752+0000","last_unstale":"2024-05-09T06:32:06.429505+0000","last_undegraded":"2024-05-09T06:32:06.429505+0000","last_fullsized":"2024-05-09T06:32:06.429505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:43.372747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430477+0000","last_change":"2024-05-09T06:32:04.443214+0000","last_active":"2024-05-09T06:32:06.430477+0000","last_peered":"2024-05-09T06:32:06.430477+0000","last_clean":"2024-05-09T06:32:06.430477+0000","last_became_active":"2024-05-09T06:32:04.442700+0000","last_became_peered":"2024-05-09T06:32:04.442700+0000","last_unstale":"2024-05-09T06:32:06.430477+0000","last_undegraded":"2024-05-09T06:32:06.430477+0000","last_fullsized":"2024-05-09T06:32:06.430477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:35.496823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429632+0000","last_change":"2024-05-09T06:32:05.450473+0000","last_active":"2024-05-09T06:32:06.429632+0000","last_peered":"2024-05-09T06:32:06.429632+0000","last_clean":"2024-05-09T06:32:06.429632+0000","last_became_active":"2024-05-09T06:32:05.449824+0000","last_became_peered":"2024-05-09T06:32:05.449824+0000","last_unstale":"2024-05-09T06:32:06.429632+0000","last_undegraded":"2024-05-09T06:32:06.429632+0000","last_fullsized":"2024-05-09T06:32:06.429632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:20:02.491562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425619+0000","last_change":"2024-05-09T06:32:04.443113+0000","last_active":"2024-05-09T06:32:06.425619+0000","last_peered":"2024-05-09T06:32:06.425619+0000","last_clean":"2024-05-09T06:32:06.425619+0000","last_became_active":"2024-05-09T06:32:04.442646+0000","last_became_peered":"2024-05-09T06:32:04.442646+0000","last_unstale":"2024-05-09T06:32:06.425619+0000","last_undegraded":"2024-05-09T06:32:06.425619+0000","last_fullsized":"2024-05-09T06:32:06.425619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:27.405031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427894+0000","last_change":"2024-05-09T06:32:05.441524+0000","last_active":"2024-05-09T06:32:06.427894+0000","last_peered":"2024-05-09T06:32:06.427894+0000","last_clean":"2024-05-09T06:32:06.427894+0000","last_became_active":"2024-05-09T06:32:05.441224+0000","last_became_peered":"2024-05-09T06:32:05.441224+0000","last_unstale":"2024-05-09T06:32:06.427894+0000","last_undegraded":"2024-05-09T06:32:06.427894+0000","last_fullsized":"2024-05-09T06:32:06.427894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:08.574415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429735+0000","last_change":"2024-05-09T06:32:04.440222+0000","last_active":"2024-05-09T06:32:06.429735+0000","last_peered":"2024-05-09T06:32:06.429735+0000","last_clean":"2024-05-09T06:32:06.429735+0000","last_became_active":"2024-05-09T06:32:04.439927+0000","last_became_peered":"2024-05-09T06:32:04.439927+0000","last_unstale":"2024-05-09T06:32:06.429735+0000","last_undegraded":"2024-05-09T06:32:06.429735+0000","last_fullsized":"2024-05-09T06:32:06.429735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:04.276043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428448+0000","last_change":"2024-05-09T06:32:05.439936+0000","last_active":"2024-05-09T06:32:06.428448+0000","last_peered":"2024-05-09T06:32:06.428448+0000","last_clean":"2024-05-09T06:32:06.428448+0000","last_became_active":"2024-05-09T06:32:05.439623+0000","last_became_peered":"2024-05-09T06:32:05.439623+0000","last_unstale":"2024-05-09T06:32:06.428448+0000","last_undegraded":"2024-05-09T06:32:06.428448+0000","last_fullsized":"2024-05-09T06:32:06.428448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:50.352947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429050+0000","last_change":"2024-05-09T06:32:04.438136+0000","last_active":"2024-05-09T06:32:06.429050+0000","last_peered":"2024-05-09T06:32:06.429050+0000","last_clean":"2024-05-09T06:32:06.429050+0000","last_became_active":"2024-05-09T06:32:04.437793+0000","last_became_peered":"2024-05-09T06:32:04.437793+0000","last_unstale":"2024-05-09T06:32:06.429050+0000","last_undegraded":"2024-05-09T06:32:06.429050+0000","last_fullsized":"2024-05-09T06:32:06.429050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:35.496823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429039+0000","last_change":"2024-05-09T06:32:05.446685+0000","last_active":"2024-05-09T06:32:06.429039+0000","last_peered":"2024-05-09T06:32:06.429039+0000","last_clean":"2024-05-09T06:32:06.429039+0000","last_became_active":"2024-05-09T06:32:05.446260+0000","last_became_peered":"2024-05-09T06:32:05.446260+0000","last_unstale":"2024-05-09T06:32:06.429039+0000","last_undegraded":"2024-05-09T06:32:06.429039+0000","last_fullsized":"2024-05-09T06:32:06.429039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:55.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428330+0000","last_change":"2024-05-09T06:32:04.440897+0000","last_active":"2024-05-09T06:32:06.428330+0000","last_peered":"2024-05-09T06:32:06.428330+0000","last_clean":"2024-05-09T06:32:06.428330+0000","last_became_active":"2024-05-09T06:32:04.440603+0000","last_became_peered":"2024-05-09T06:32:04.440603+0000","last_unstale":"2024-05-09T06:32:06.428330+0000","last_undegraded":"2024-05-09T06:32:06.428330+0000","last_fullsized":"2024-05-09T06:32:06.428330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:03.775465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425727+0000","last_change":"2024-05-09T06:32:05.448449+0000","last_active":"2024-05-09T06:32:06.425727+0000","last_peered":"2024-05-09T06:32:06.425727+0000","last_clean":"2024-05-09T06:32:06.425727+0000","last_became_active":"2024-05-09T06:32:05.442306+0000","last_became_peered":"2024-05-09T06:32:05.442306+0000","last_unstale":"2024-05-09T06:32:06.425727+0000","last_undegraded":"2024-05-09T06:32:06.425727+0000","last_fullsized":"2024-05-09T06:32:06.425727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:33.213776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423758+0000","last_change":"2024-05-09T06:32:04.441253+0000","last_active":"2024-05-09T06:32:06.423758+0000","last_peered":"2024-05-09T06:32:06.423758+0000","last_clean":"2024-05-09T06:32:06.423758+0000","last_became_active":"2024-05-09T06:32:04.440770+0000","last_became_peered":"2024-05-09T06:32:04.440770+0000","last_unstale":"2024-05-09T06:32:06.423758+0000","last_undegraded":"2024-05-09T06:32:06.423758+0000","last_fullsized":"2024-05-09T06:32:06.423758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:54.384977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427977+0000","last_change":"2024-05-09T06:32:05.440356+0000","last_active":"2024-05-09T06:32:06.427977+0000","last_peered":"2024-05-09T06:32:06.427977+0000","last_clean":"2024-05-09T06:32:06.427977+0000","last_became_active":"2024-05-09T06:32:05.440069+0000","last_became_peered":"2024-05-09T06:32:05.440069+0000","last_unstale":"2024-05-09T06:32:06.427977+0000","last_undegraded":"2024-05-09T06:32:06.427977+0000","last_fullsized":"2024-05-09T06:32:06.427977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:04.955309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430490+0000","last_change":"2024-05-09T06:32:04.439944+0000","last_active":"2024-05-09T06:32:06.430490+0000","last_peered":"2024-05-09T06:32:06.430490+0000","last_clean":"2024-05-09T06:32:06.430490+0000","last_became_active":"2024-05-09T06:32:04.439443+0000","last_became_peered":"2024-05-09T06:32:04.439443+0000","last_unstale":"2024-05-09T06:32:06.430490+0000","last_undegraded":"2024-05-09T06:32:06.430490+0000","last_fullsized":"2024-05-09T06:32:06.430490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:34.909003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428867+0000","last_change":"2024-05-09T06:32:05.453459+0000","last_active":"2024-05-09T06:32:06.428867+0000","last_peered":"2024-05-09T06:32:06.428867+0000","last_clean":"2024-05-09T06:32:06.428867+0000","last_became_active":"2024-05-09T06:32:05.453155+0000","last_became_peered":"2024-05-09T06:32:05.453155+0000","last_unstale":"2024-05-09T06:32:06.428867+0000","last_undegraded":"2024-05-09T06:32:06.428867+0000","last_fullsized":"2024-05-09T06:32:06.428867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:39:53.867438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428849+0000","last_change":"2024-05-09T06:32:04.438599+0000","last_active":"2024-05-09T06:32:06.428849+0000","last_peered":"2024-05-09T06:32:06.428849+0000","last_clean":"2024-05-09T06:32:06.428849+0000","last_became_active":"2024-05-09T06:32:04.438156+0000","last_became_peered":"2024-05-09T06:32:04.438156+0000","last_unstale":"2024-05-09T06:32:06.428849+0000","last_undegraded":"2024-05-09T06:32:06.428849+0000","last_fullsized":"2024-05-09T06:32:06.428849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:59.105593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429424+0000","last_change":"2024-05-09T06:32:05.432674+0000","last_active":"2024-05-09T06:32:06.429424+0000","last_peered":"2024-05-09T06:32:06.429424+0000","last_clean":"2024-05-09T06:32:06.429424+0000","last_became_active":"2024-05-09T06:32:05.432340+0000","last_became_peered":"2024-05-09T06:32:05.432340+0000","last_unstale":"2024-05-09T06:32:06.429424+0000","last_undegraded":"2024-05-09T06:32:06.429424+0000","last_fullsized":"2024-05-09T06:32:06.429424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:47.662049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424499+0000","last_change":"2024-05-09T06:32:04.444691+0000","last_active":"2024-05-09T06:32:06.424499+0000","last_peered":"2024-05-09T06:32:06.424499+0000","last_clean":"2024-05-09T06:32:06.424499+0000","last_became_active":"2024-05-09T06:32:04.444398+0000","last_became_peered":"2024-05-09T06:32:04.444398+0000","last_unstale":"2024-05-09T06:32:06.424499+0000","last_undegraded":"2024-05-09T06:32:06.424499+0000","last_fullsized":"2024-05-09T06:32:06.424499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:11.610791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428470+0000","last_change":"2024-05-09T06:32:05.441913+0000","last_active":"2024-05-09T06:32:06.428470+0000","last_peered":"2024-05-09T06:32:06.428470+0000","last_clean":"2024-05-09T06:32:06.428470+0000","last_became_active":"2024-05-09T06:32:05.441635+0000","last_became_peered":"2024-05-09T06:32:05.441635+0000","last_unstale":"2024-05-09T06:32:06.428470+0000","last_undegraded":"2024-05-09T06:32:06.428470+0000","last_fullsized":"2024-05-09T06:32:06.428470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:14.868754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425751+0000","last_change":"2024-05-09T06:32:04.432822+0000","last_active":"2024-05-09T06:32:06.425751+0000","last_peered":"2024-05-09T06:32:06.425751+0000","last_clean":"2024-05-09T06:32:06.425751+0000","last_became_active":"2024-05-09T06:32:04.432497+0000","last_became_peered":"2024-05-09T06:32:04.432497+0000","last_unstale":"2024-05-09T06:32:06.425751+0000","last_undegraded":"2024-05-09T06:32:06.425751+0000","last_fullsized":"2024-05-09T06:32:06.425751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:20:01.485858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430229+0000","last_change":"2024-05-09T06:32:05.442529+0000","last_active":"2024-05-09T06:32:06.430229+0000","last_peered":"2024-05-09T06:32:06.430229+0000","last_clean":"2024-05-09T06:32:06.430229+0000","last_became_active":"2024-05-09T06:32:05.441926+0000","last_became_peered":"2024-05-09T06:32:05.441926+0000","last_unstale":"2024-05-09T06:32:06.430229+0000","last_undegraded":"2024-05-09T06:32:06.430229+0000","last_fullsized":"2024-05-09T06:32:06.430229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:29:48.323284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429626+0000","last_change":"2024-05-09T06:32:04.433926+0000","last_active":"2024-05-09T06:32:06.429626+0000","last_peered":"2024-05-09T06:32:06.429626+0000","last_clean":"2024-05-09T06:32:06.429626+0000","last_became_active":"2024-05-09T06:32:04.433611+0000","last_became_peered":"2024-05-09T06:32:04.433611+0000","last_unstale":"2024-05-09T06:32:06.429626+0000","last_undegraded":"2024-05-09T06:32:06.429626+0000","last_fullsized":"2024-05-09T06:32:06.429626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:29.823513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425562+0000","last_change":"2024-05-09T06:32:05.442937+0000","last_active":"2024-05-09T06:32:06.425562+0000","last_peered":"2024-05-09T06:32:06.425562+0000","last_clean":"2024-05-09T06:32:06.425562+0000","last_became_active":"2024-05-09T06:32:05.442618+0000","last_became_peered":"2024-05-09T06:32:05.442618+0000","last_unstale":"2024-05-09T06:32:06.425562+0000","last_undegraded":"2024-05-09T06:32:06.425562+0000","last_fullsized":"2024-05-09T06:32:06.425562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:44.110160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430263+0000","last_change":"2024-05-09T06:32:04.443631+0000","last_active":"2024-05-09T06:32:06.430263+0000","last_peered":"2024-05-09T06:32:06.430263+0000","last_clean":"2024-05-09T06:32:06.430263+0000","last_became_active":"2024-05-09T06:32:04.442472+0000","last_became_peered":"2024-05-09T06:32:04.442472+0000","last_unstale":"2024-05-09T06:32:06.430263+0000","last_undegraded":"2024-05-09T06:32:06.430263+0000","last_fullsized":"2024-05-09T06:32:06.430263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:13.863049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425671+0000","last_change":"2024-05-09T06:32:05.447261+0000","last_active":"2024-05-09T06:32:06.425671+0000","last_peered":"2024-05-09T06:32:06.425671+0000","last_clean":"2024-05-09T06:32:06.425671+0000","last_became_active":"2024-05-09T06:32:05.446979+0000","last_became_peered":"2024-05-09T06:32:05.446979+0000","last_unstale":"2024-05-09T06:32:06.425671+0000","last_undegraded":"2024-05-09T06:32:06.425671+0000","last_fullsized":"2024-05-09T06:32:06.425671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:43.529899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423840+0000","last_change":"2024-05-09T06:32:04.443235+0000","last_active":"2024-05-09T06:32:06.423840+0000","last_peered":"2024-05-09T06:32:06.423840+0000","last_clean":"2024-05-09T06:32:06.423840+0000","last_became_active":"2024-05-09T06:32:04.442813+0000","last_became_peered":"2024-05-09T06:32:04.442813+0000","last_unstale":"2024-05-09T06:32:06.423840+0000","last_undegraded":"2024-05-09T06:32:06.423840+0000","last_fullsized":"2024-05-09T06:32:06.423840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:26.977785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425720+0000","last_change":"2024-05-09T06:32:05.448712+0000","last_active":"2024-05-09T06:32:06.425720+0000","last_peered":"2024-05-09T06:32:06.425720+0000","last_clean":"2024-05-09T06:32:06.425720+0000","last_became_active":"2024-05-09T06:32:05.442340+0000","last_became_peered":"2024-05-09T06:32:05.442340+0000","last_unstale":"2024-05-09T06:32:06.425720+0000","last_undegraded":"2024-05-09T06:32:06.425720+0000","last_fullsized":"2024-05-09T06:32:06.425720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:30.875878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428075+0000","last_change":"2024-05-09T06:32:04.432252+0000","last_active":"2024-05-09T06:32:06.428075+0000","last_peered":"2024-05-09T06:32:06.428075+0000","last_clean":"2024-05-09T06:32:06.428075+0000","last_became_active":"2024-05-09T06:32:04.431361+0000","last_became_peered":"2024-05-09T06:32:04.431361+0000","last_unstale":"2024-05-09T06:32:06.428075+0000","last_undegraded":"2024-05-09T06:32:06.428075+0000","last_fullsized":"2024-05-09T06:32:06.428075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:39:52.861734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428568+0000","last_change":"2024-05-09T06:32:05.435262+0000","last_active":"2024-05-09T06:32:06.428568+0000","last_peered":"2024-05-09T06:32:06.428568+0000","last_clean":"2024-05-09T06:32:06.428568+0000","last_became_active":"2024-05-09T06:32:05.434975+0000","last_became_peered":"2024-05-09T06:32:05.434975+0000","last_unstale":"2024-05-09T06:32:06.428568+0000","last_undegraded":"2024-05-09T06:32:06.428568+0000","last_fullsized":"2024-05-09T06:32:06.428568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.617088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430503+0000","last_change":"2024-05-09T06:32:04.444811+0000","last_active":"2024-05-09T06:32:06.430503+0000","last_peered":"2024-05-09T06:32:06.430503+0000","last_clean":"2024-05-09T06:32:06.430503+0000","last_became_active":"2024-05-09T06:32:04.444519+0000","last_became_peered":"2024-05-09T06:32:04.444519+0000","last_unstale":"2024-05-09T06:32:06.430503+0000","last_undegraded":"2024-05-09T06:32:06.430503+0000","last_fullsized":"2024-05-09T06:32:06.430503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:29.001891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423833+0000","last_change":"2024-05-09T06:32:05.443411+0000","last_active":"2024-05-09T06:32:06.423833+0000","last_peered":"2024-05-09T06:32:06.423833+0000","last_clean":"2024-05-09T06:32:06.423833+0000","last_became_active":"2024-05-09T06:32:05.442749+0000","last_became_peered":"2024-05-09T06:32:05.442749+0000","last_unstale":"2024-05-09T06:32:06.423833+0000","last_undegraded":"2024-05-09T06:32:06.423833+0000","last_fullsized":"2024-05-09T06:32:06.423833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:41.345800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429155+0000","last_change":"2024-05-09T06:32:04.432260+0000","last_active":"2024-05-09T06:32:06.429155+0000","last_peered":"2024-05-09T06:32:06.429155+0000","last_clean":"2024-05-09T06:32:06.429155+0000","last_became_active":"2024-05-09T06:32:04.431915+0000","last_became_peered":"2024-05-09T06:32:04.431915+0000","last_unstale":"2024-05-09T06:32:06.429155+0000","last_undegraded":"2024-05-09T06:32:06.429155+0000","last_fullsized":"2024-05-09T06:32:06.429155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:52.511397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424171+0000","last_change":"2024-05-09T06:32:05.435866+0000","last_active":"2024-05-09T06:32:06.424171+0000","last_peered":"2024-05-09T06:32:06.424171+0000","last_clean":"2024-05-09T06:32:06.424171+0000","last_became_active":"2024-05-09T06:32:05.434714+0000","last_became_peered":"2024-05-09T06:32:05.434714+0000","last_unstale":"2024-05-09T06:32:06.424171+0000","last_undegraded":"2024-05-09T06:32:06.424171+0000","last_fullsized":"2024-05-09T06:32:06.424171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:54.472525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.424525+0000","last_change":"2024-05-09T06:32:04.440739+0000","last_active":"2024-05-09T06:32:06.424525+0000","last_peered":"2024-05-09T06:32:06.424525+0000","last_clean":"2024-05-09T06:32:06.424525+0000","last_became_active":"2024-05-09T06:32:04.440446+0000","last_became_peered":"2024-05-09T06:32:04.440446+0000","last_unstale":"2024-05-09T06:32:06.424525+0000","last_undegraded":"2024-05-09T06:32:06.424525+0000","last_fullsized":"2024-05-09T06:32:06.424525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:04.021978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428896+0000","last_change":"2024-05-09T06:32:05.435396+0000","last_active":"2024-05-09T06:32:06.428896+0000","last_peered":"2024-05-09T06:32:06.428896+0000","last_clean":"2024-05-09T06:32:06.428896+0000","last_became_active":"2024-05-09T06:32:05.435102+0000","last_became_peered":"2024-05-09T06:32:05.435102+0000","last_unstale":"2024-05-09T06:32:06.428896+0000","last_undegraded":"2024-05-09T06:32:06.428896+0000","last_fullsized":"2024-05-09T06:32:06.428896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:08.545595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428364+0000","last_change":"2024-05-09T06:32:04.441944+0000","last_active":"2024-05-09T06:32:06.428364+0000","last_peered":"2024-05-09T06:32:06.428364+0000","last_clean":"2024-05-09T06:32:06.428364+0000","last_became_active":"2024-05-09T06:32:04.441589+0000","last_became_peered":"2024-05-09T06:32:04.441589+0000","last_unstale":"2024-05-09T06:32:06.428364+0000","last_undegraded":"2024-05-09T06:32:06.428364+0000","last_fullsized":"2024-05-09T06:32:06.428364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:54.435307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428044+0000","last_change":"2024-05-09T06:32:05.449974+0000","last_active":"2024-05-09T06:32:06.428044+0000","last_peered":"2024-05-09T06:32:06.428044+0000","last_clean":"2024-05-09T06:32:06.428044+0000","last_became_active":"2024-05-09T06:32:05.448565+0000","last_became_peered":"2024-05-09T06:32:05.448565+0000","last_unstale":"2024-05-09T06:32:06.428044+0000","last_undegraded":"2024-05-09T06:32:06.428044+0000","last_fullsized":"2024-05-09T06:32:06.428044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:54.655094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429003+0000","last_change":"2024-05-09T06:32:04.434026+0000","last_active":"2024-05-09T06:32:06.429003+0000","last_peered":"2024-05-09T06:32:06.429003+0000","last_clean":"2024-05-09T06:32:06.429003+0000","last_became_active":"2024-05-09T06:32:04.433725+0000","last_became_peered":"2024-05-09T06:32:04.433725+0000","last_unstale":"2024-05-09T06:32:06.429003+0000","last_undegraded":"2024-05-09T06:32:06.429003+0000","last_fullsized":"2024-05-09T06:32:06.429003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:23:11.863059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428866+0000","last_change":"2024-05-09T06:32:05.451861+0000","last_active":"2024-05-09T06:32:06.428866+0000","last_peered":"2024-05-09T06:32:06.428866+0000","last_clean":"2024-05-09T06:32:06.428866+0000","last_became_active":"2024-05-09T06:32:05.451049+0000","last_became_peered":"2024-05-09T06:32:05.451049+0000","last_unstale":"2024-05-09T06:32:06.428866+0000","last_undegraded":"2024-05-09T06:32:06.428866+0000","last_fullsized":"2024-05-09T06:32:06.428866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:47.662049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428654+0000","last_change":"2024-05-09T06:32:04.433135+0000","last_active":"2024-05-09T06:32:06.428654+0000","last_peered":"2024-05-09T06:32:06.428654+0000","last_clean":"2024-05-09T06:32:06.428654+0000","last_became_active":"2024-05-09T06:32:04.432817+0000","last_became_peered":"2024-05-09T06:32:04.432817+0000","last_unstale":"2024-05-09T06:32:06.428654+0000","last_undegraded":"2024-05-09T06:32:06.428654+0000","last_fullsized":"2024-05-09T06:32:06.428654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:07:42.302696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425847+0000","last_change":"2024-05-09T06:32:05.449545+0000","last_active":"2024-05-09T06:32:06.425847+0000","last_peered":"2024-05-09T06:32:06.425847+0000","last_clean":"2024-05-09T06:32:06.425847+0000","last_became_active":"2024-05-09T06:32:05.449279+0000","last_became_peered":"2024-05-09T06:32:05.449279+0000","last_unstale":"2024-05-09T06:32:06.425847+0000","last_undegraded":"2024-05-09T06:32:06.425847+0000","last_fullsized":"2024-05-09T06:32:06.425847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:04.781170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428724+0000","last_change":"2024-05-09T06:32:04.433089+0000","last_active":"2024-05-09T06:32:06.428724+0000","last_peered":"2024-05-09T06:32:06.428724+0000","last_clean":"2024-05-09T06:32:06.428724+0000","last_became_active":"2024-05-09T06:32:04.432781+0000","last_became_peered":"2024-05-09T06:32:04.432781+0000","last_unstale":"2024-05-09T06:32:06.428724+0000","last_undegraded":"2024-05-09T06:32:06.428724+0000","last_fullsized":"2024-05-09T06:32:06.428724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:06.529979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428575+0000","last_change":"2024-05-09T06:32:05.448698+0000","last_active":"2024-05-09T06:32:06.428575+0000","last_peered":"2024-05-09T06:32:06.428575+0000","last_clean":"2024-05-09T06:32:06.428575+0000","last_became_active":"2024-05-09T06:32:05.447889+0000","last_became_peered":"2024-05-09T06:32:05.447889+0000","last_unstale":"2024-05-09T06:32:06.428575+0000","last_undegraded":"2024-05-09T06:32:06.428575+0000","last_fullsized":"2024-05-09T06:32:06.428575+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:16.633682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429490+0000","last_change":"2024-05-09T06:32:04.436489+0000","last_active":"2024-05-09T06:32:06.429490+0000","last_peered":"2024-05-09T06:32:06.429490+0000","last_clean":"2024-05-09T06:32:06.429490+0000","last_became_active":"2024-05-09T06:32:04.436148+0000","last_became_peered":"2024-05-09T06:32:04.436148+0000","last_unstale":"2024-05-09T06:32:06.429490+0000","last_undegraded":"2024-05-09T06:32:06.429490+0000","last_fullsized":"2024-05-09T06:32:06.429490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:13.863049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425238+0000","last_change":"2024-05-09T06:32:05.436501+0000","last_active":"2024-05-09T06:32:06.425238+0000","last_peered":"2024-05-09T06:32:06.425238+0000","last_clean":"2024-05-09T06:32:06.425238+0000","last_became_active":"2024-05-09T06:32:05.436196+0000","last_became_peered":"2024-05-09T06:32:05.436196+0000","last_unstale":"2024-05-09T06:32:06.425238+0000","last_undegraded":"2024-05-09T06:32:06.425238+0000","last_fullsized":"2024-05-09T06:32:06.425238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:21:00.111297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.438281+0000","last_change":"2024-05-09T06:32:04.438167+0000","last_active":"2024-05-09T06:32:06.438281+0000","last_peered":"2024-05-09T06:32:06.438281+0000","last_clean":"2024-05-09T06:32:06.438281+0000","last_became_active":"2024-05-09T06:32:04.437870+0000","last_became_peered":"2024-05-09T06:32:04.437870+0000","last_unstale":"2024-05-09T06:32:06.438281+0000","last_undegraded":"2024-05-09T06:32:06.438281+0000","last_fullsized":"2024-05-09T06:32:06.438281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:06:38.693755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429428+0000","last_change":"2024-05-09T06:32:05.444218+0000","last_active":"2024-05-09T06:32:06.429428+0000","last_peered":"2024-05-09T06:32:06.429428+0000","last_clean":"2024-05-09T06:32:06.429428+0000","last_became_active":"2024-05-09T06:32:05.443930+0000","last_became_peered":"2024-05-09T06:32:05.443930+0000","last_unstale":"2024-05-09T06:32:06.429428+0000","last_undegraded":"2024-05-09T06:32:06.429428+0000","last_fullsized":"2024-05-09T06:32:06.429428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:09.308149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430594+0000","last_change":"2024-05-09T06:32:04.441464+0000","last_active":"2024-05-09T06:32:06.430594+0000","last_peered":"2024-05-09T06:32:06.430594+0000","last_clean":"2024-05-09T06:32:06.430594+0000","last_became_active":"2024-05-09T06:32:04.440040+0000","last_became_peered":"2024-05-09T06:32:04.440040+0000","last_unstale":"2024-05-09T06:32:06.430594+0000","last_undegraded":"2024-05-09T06:32:06.430594+0000","last_fullsized":"2024-05-09T06:32:06.430594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:33:43.104455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428421+0000","last_change":"2024-05-09T06:32:05.445587+0000","last_active":"2024-05-09T06:32:06.428421+0000","last_peered":"2024-05-09T06:32:06.428421+0000","last_clean":"2024-05-09T06:32:06.428421+0000","last_became_active":"2024-05-09T06:32:05.444735+0000","last_became_peered":"2024-05-09T06:32:05.444735+0000","last_unstale":"2024-05-09T06:32:06.428421+0000","last_undegraded":"2024-05-09T06:32:06.428421+0000","last_fullsized":"2024-05-09T06:32:06.428421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:15.947058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430301+0000","last_change":"2024-05-09T06:32:04.440992+0000","last_active":"2024-05-09T06:32:06.430301+0000","last_peered":"2024-05-09T06:32:06.430301+0000","last_clean":"2024-05-09T06:32:06.430301+0000","last_became_active":"2024-05-09T06:32:04.439755+0000","last_became_peered":"2024-05-09T06:32:04.439755+0000","last_unstale":"2024-05-09T06:32:06.430301+0000","last_undegraded":"2024-05-09T06:32:06.430301+0000","last_fullsized":"2024-05-09T06:32:06.430301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:59.105593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425367+0000","last_change":"2024-05-09T06:32:05.443456+0000","last_active":"2024-05-09T06:32:06.425367+0000","last_peered":"2024-05-09T06:32:06.425367+0000","last_clean":"2024-05-09T06:32:06.425367+0000","last_became_active":"2024-05-09T06:32:05.443146+0000","last_became_peered":"2024-05-09T06:32:05.443146+0000","last_unstale":"2024-05-09T06:32:06.425367+0000","last_undegraded":"2024-05-09T06:32:06.425367+0000","last_fullsized":"2024-05-09T06:32:06.425367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:35.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425105+0000","last_change":"2024-05-09T06:32:04.442472+0000","last_active":"2024-05-09T06:32:06.425105+0000","last_peered":"2024-05-09T06:32:06.425105+0000","last_clean":"2024-05-09T06:32:06.425105+0000","last_became_active":"2024-05-09T06:32:04.442184+0000","last_became_peered":"2024-05-09T06:32:04.442184+0000","last_unstale":"2024-05-09T06:32:06.425105+0000","last_undegraded":"2024-05-09T06:32:06.425105+0000","last_fullsized":"2024-05-09T06:32:06.425105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:44.494387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425532+0000","last_change":"2024-05-09T06:32:05.445778+0000","last_active":"2024-05-09T06:32:06.425532+0000","last_peered":"2024-05-09T06:32:06.425532+0000","last_clean":"2024-05-09T06:32:06.425532+0000","last_became_active":"2024-05-09T06:32:05.442727+0000","last_became_peered":"2024-05-09T06:32:05.442727+0000","last_unstale":"2024-05-09T06:32:06.425532+0000","last_undegraded":"2024-05-09T06:32:06.425532+0000","last_fullsized":"2024-05-09T06:32:06.425532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:30.007595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.423809+0000","last_change":"2024-05-09T06:32:04.440551+0000","last_active":"2024-05-09T06:32:06.423809+0000","last_peered":"2024-05-09T06:32:06.423809+0000","last_clean":"2024-05-09T06:32:06.423809+0000","last_became_active":"2024-05-09T06:32:04.439970+0000","last_became_peered":"2024-05-09T06:32:04.439970+0000","last_unstale":"2024-05-09T06:32:06.423809+0000","last_undegraded":"2024-05-09T06:32:06.423809+0000","last_fullsized":"2024-05-09T06:32:06.423809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:37.790237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428950+0000","last_change":"2024-05-09T06:32:05.442577+0000","last_active":"2024-05-09T06:32:06.428950+0000","last_peered":"2024-05-09T06:32:06.428950+0000","last_clean":"2024-05-09T06:32:06.428950+0000","last_became_active":"2024-05-09T06:32:05.442291+0000","last_became_peered":"2024-05-09T06:32:05.442291+0000","last_unstale":"2024-05-09T06:32:06.428950+0000","last_undegraded":"2024-05-09T06:32:06.428950+0000","last_fullsized":"2024-05-09T06:32:06.428950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:35.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.425586+0000","last_change":"2024-05-09T06:32:04.434957+0000","last_active":"2024-05-09T06:32:06.425586+0000","last_peered":"2024-05-09T06:32:06.425586+0000","last_clean":"2024-05-09T06:32:06.425586+0000","last_became_active":"2024-05-09T06:32:04.434661+0000","last_became_peered":"2024-05-09T06:32:04.434661+0000","last_unstale":"2024-05-09T06:32:06.425586+0000","last_undegraded":"2024-05-09T06:32:06.425586+0000","last_fullsized":"2024-05-09T06:32:06.425586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:54.435307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.428767+0000","last_change":"2024-05-09T06:32:05.446985+0000","last_active":"2024-05-09T06:32:06.428767+0000","last_peered":"2024-05-09T06:32:06.428767+0000","last_clean":"2024-05-09T06:32:06.428767+0000","last_became_active":"2024-05-09T06:32:05.446109+0000","last_became_peered":"2024-05-09T06:32:05.446109+0000","last_unstale":"2024-05-09T06:32:06.428767+0000","last_undegraded":"2024-05-09T06:32:06.428767+0000","last_fullsized":"2024-05-09T06:32:06.428767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:21:00.111297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.427822+0000","last_change":"2024-05-09T06:32:04.431721+0000","last_active":"2024-05-09T06:32:06.427822+0000","last_peered":"2024-05-09T06:32:06.427822+0000","last_clean":"2024-05-09T06:32:06.427822+0000","last_became_active":"2024-05-09T06:32:04.431373+0000","last_became_peered":"2024-05-09T06:32:04.431373+0000","last_unstale":"2024-05-09T06:32:06.427822+0000","last_undegraded":"2024-05-09T06:32:06.427822+0000","last_fullsized":"2024-05-09T06:32:06.427822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.430598+0000","last_change":"2024-05-09T06:32:05.445098+0000","last_active":"2024-05-09T06:32:06.430598+0000","last_peered":"2024-05-09T06:32:06.430598+0000","last_clean":"2024-05-09T06:32:06.430598+0000","last_became_active":"2024-05-09T06:32:05.444765+0000","last_became_peered":"2024-05-09T06:32:05.444765+0000","last_unstale":"2024-05-09T06:32:06.430598+0000","last_undegraded":"2024-05-09T06:32:06.430598+0000","last_fullsized":"2024-05-09T06:32:06.430598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:04.413465+0000","last_clean_scrub_stamp":"2024-05-09T06:32:04.413465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:41:38.643747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429435+0000","last_change":"2024-05-09T06:32:04.438318+0000","last_active":"2024-05-09T06:32:06.429435+0000","last_peered":"2024-05-09T06:32:06.429435+0000","last_clean":"2024-05-09T06:32:06.429435+0000","last_became_active":"2024-05-09T06:32:04.438002+0000","last_became_peered":"2024-05-09T06:32:04.438002+0000","last_unstale":"2024-05-09T06:32:06.429435+0000","last_undegraded":"2024-05-09T06:32:06.429435+0000","last_fullsized":"2024-05-09T06:32:06.429435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:03.407760+0000","last_clean_scrub_stamp":"2024-05-09T06:32:03.407760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:32.855503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T06:32:06.429809+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:32:06.429809+0000","last_peered":"2024-05-09T06:32:06.429809+0000","last_clean":"2024-05-09T06:32:06.429809+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:32:06.429809+0000","last_undegraded":"2024-05-09T06:32:06.429809+0000","last_fullsized":"2024-05-09T06:32:06.429809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":1027612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":438834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":437256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":438938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":958464,"data_stored":437804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":437964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":437334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":1027590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:32:09.805 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:32:09.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:32:09.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 Processor -- start 2024-05-09T06:32:09.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 -- start start 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2020158a50 0x7f20200718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 0x7f2020071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202015b420 con 0x7f202006e6d0 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2020158e30 con 0x7f2020158a50 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.972+0000 7f2025563640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2020072480 con 0x7f202015c4a0 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2020158a50 0x7f20200718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f202006e6d0 0x7f20200cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55272/0 (socket says 172.21.15.1:55272) 2024-05-09T06:32:09.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201f7fe640 1 -- 172.21.15.1:0/4208973226 learned_addr learned my addr 172.21.15.1:0/4208973226 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201effd640 1 -- 172.21.15.1:0/4208973226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 msgr2=0x7f2020071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201e7fc640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 0x7f2020071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201effd640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 0x7f2020071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201dffb640 1 -- 172.21.15.1:0/4208973226 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171152701 0 0) 0x7f202015b420 con 0x7f202006e6d0 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201effd640 1 -- 172.21.15.1:0/4208973226 >> v1:172.21.15.1:6789/0 conn(0x7f202006e6d0 legacy=0x7f20200cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201effd640 1 -- 172.21.15.1:0/4208973226 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202015b6e0 con 0x7f2020158a50 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.973+0000 7f201e7fc640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 0x7f2020071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f201effd640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020158a50 0x7f20200718c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2010009d90 tx=0x7f201002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b011c8e1fefec84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f201dffb640 1 -- 172.21.15.1:0/4208973226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010007740 con 0x7f2020158a50 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f201dffb640 1 -- 172.21.15.1:0/4208973226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20100078e0 con 0x7f2020158a50 2024-05-09T06:32:09.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f201dffb640 1 -- 172.21.15.1:0/4208973226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010007cb0 con 0x7f2020158a50 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 -- 172.21.15.1:0/4208973226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020158a50 msgr2=0x7f20200718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020158a50 0x7f20200718c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2010009d90 tx=0x7f201002a660 comp rx=0 tx=0).stop 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 -- 172.21.15.1:0/4208973226 shutdown_connections 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f202015c4a0 0x7f2020071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 --2- 172.21.15.1:0/4208973226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020158a50 0x7f20200718c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 -- 172.21.15.1:0/4208973226 >> 172.21.15.1:0/4208973226 conn(0x7f20200cc4d0 msgr2=0x7f20200cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 -- 172.21.15.1:0/4208973226 shutdown_connections 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.974+0000 7f2025563640 1 -- 172.21.15.1:0/4208973226 wait complete. 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 Processor -- start 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 -- start start 2024-05-09T06:32:09.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 0x7f2020145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2020158a50 0x7f2020156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f202015bf80 con 0x7f202015c4a0 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f202015bca0 con 0x7f202006e6d0 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.976+0000 7f2025563640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2020070ea0 con 0x7f2020158a50 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52462/0 (socket says 172.21.15.1:52462) 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 -- 172.21.15.1:0/446501737 learned_addr learned my addr 172.21.15.1:0/446501737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201e7fc640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2020158a50 0x7f2020156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201effd640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 0x7f2020145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 -- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2020158a50 msgr2=0x7f2020156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2020158a50 0x7f2020156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 -- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 msgr2=0x7f2020145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 0x7f2020145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:09.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 -- 172.21.15.1:0/446501737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020145ca0 con 0x7f202015c4a0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201effd640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 0x7f2020145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f201f7fe640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2018001eb0 tx=0x7f201800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180074e0 con 0x7f202015c4a0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180047e0 con 0x7f202015c4a0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018004bb0 con 0x7f202015c4a0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201f2400 con 0x7f202015c4a0 2024-05-09T06:32:09.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.977+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201f2cc0 con 0x7f202015c4a0 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.978+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200752b0 con 0x7f202015c4a0 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.979+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2018011070 con 0x7f202015c4a0 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.979+0000 7f20177fe640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 0x7f200006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.979+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f201808aae0 con 0x7f202015c4a0 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.979+0000 7f201effd640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 0x7f200006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:09.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.980+0000 7f201effd640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 0x7f200006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2010034000 tx=0x7f201002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:09.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:09.984+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2018059310 con 0x7f202015c4a0 2024-05-09T06:32:10.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.239+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f202006e420 con 0x7f202015c4a0 2024-05-09T06:32:10.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.241+0000 7f20177fe640 1 -- 172.21.15.1:0/446501737 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f2018058c10 con 0x7f202015c4a0 2024-05-09T06:32:10.222 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:32:10.223 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:32:10.223 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:32:10.223 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:32:10.223 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:32:10.223 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 msgr2=0x7f200006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 0x7f200006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2010034000 tx=0x7f201002f000 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 msgr2=0x7f2020156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2018001eb0 tx=0x7f201800c870 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 shutdown_connections 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f200006be30 0x7f200006e2f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202015c4a0 0x7f2020156ca0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2020158a50 0x7f2020156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 --2- 172.21.15.1:0/446501737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f202006e6d0 0x7f2020145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 >> 172.21.15.1:0/446501737 conn(0x7f20200cc4d0 msgr2=0x7f20200cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 shutdown_connections 2024-05-09T06:32:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.243+0000 7f2025563640 1 -- 172.21.15.1:0/446501737 wait complete. 2024-05-09T06:32:10.236 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:10.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.358+0000 7f5f20b34640 1 Processor -- start 2024-05-09T06:32:10.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.358+0000 7f5f20b34640 1 -- start start 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f20b34640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f1c15e050 0x7f5f1c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f20b34640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f1c0ae9b0 0x7f5f1c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f20b34640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f1c0aef90 con 0x7f5f1c06e4f0 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f20b34640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1c0b3b80 con 0x7f5f1c15e050 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f20b34640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1c0b3530 con 0x7f5f1c0ae9b0 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f1c15e050 0x7f5f1c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f1c15e050 0x7f5f1c15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52464/0 (socket says 172.21.15.1:52464) 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f19d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5f1c06e4f0 0x7f5f1c0ae2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55278/0 (socket says 172.21.15.1:55278) 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 -- 172.21.15.1:0/758032372 learned_addr learned my addr 172.21.15.1:0/758032372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1ad9d640 1 --2- 172.21.15.1:0/758032372 >> v2:172.21.15.6:3300/0 conn(0x7f5f1c0ae9b0 0x7f5f1c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 -- 172.21.15.1:0/758032372 >> v2:172.21.15.6:3300/0 conn(0x7f5f1c0ae9b0 msgr2=0x7f5f1c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/758032372 >> v2:172.21.15.6:3300/0 conn(0x7f5f1c0ae9b0 0x7f5f1c06e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.359+0000 7f5f1a59c640 1 -- 172.21.15.1:0/758032372 >> v1:172.21.15.1:6789/0 conn(0x7f5f1c06e4f0 legacy=0x7f5f1c0ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1a59c640 1 -- 172.21.15.1:0/758032372 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c0b3810 con 0x7f5f1c15e050 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1959a640 1 -- 172.21.15.1:0/758032372 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182051653 0 0) 0x7f5f1c0aef90 con 0x7f5f1c06e4f0 2024-05-09T06:32:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/758032372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c15e430 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0cfb30 tx=0x7f5f1402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3470868f3d630d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1959a640 1 -- 172.21.15.1:0/758032372 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14036070 con 0x7f5f1c15e050 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1959a640 1 -- 172.21.15.1:0/758032372 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f14031600 con 0x7f5f1c15e050 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.360+0000 7f5f1959a640 1 -- 172.21.15.1:0/758032372 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140319d0 con 0x7f5f1c15e050 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 -- 172.21.15.1:0/758032372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 msgr2=0x7f5f1c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 --2- 172.21.15.1:0/758032372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c15e430 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0cfb30 tx=0x7f5f1402a730 comp rx=0 tx=0).stop 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 -- 172.21.15.1:0/758032372 shutdown_connections 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 --2- 172.21.15.1:0/758032372 >> v2:172.21.15.6:3300/0 conn(0x7f5f1c0ae9b0 0x7f5f1c06e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 --2- 172.21.15.1:0/758032372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c15e430 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 -- 172.21.15.1:0/758032372 >> 172.21.15.1:0/758032372 conn(0x7f5f1c06d270 msgr2=0x7f5f1c06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 -- 172.21.15.1:0/758032372 shutdown_connections 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.361+0000 7f5f20b34640 1 -- 172.21.15.1:0/758032372 wait complete. 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.362+0000 7f5f20b34640 1 Processor -- start 2024-05-09T06:32:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.362+0000 7f5f20b34640 1 -- start start 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 0x7f5f1c14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 0x7f5f1c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c0b4010 con 0x7f5f1c15e050 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c15eb80 con 0x7f5f1c0ae9b0 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f20b34640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c0b4190 con 0x7f5f1c06e4f0 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52478/0 (socket says 172.21.15.1:52478) 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f1ad9d640 1 -- 172.21.15.1:0/216575396 learned_addr learned my addr 172.21.15.1:0/216575396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f19d9b640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 0x7f5f1c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.363+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 0x7f5f1c14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 -- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 msgr2=0x7f5f1c14f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 0x7f5f1c14f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 -- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 msgr2=0x7f5f1c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 0x7f5f1c1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 -- 172.21.15.1:0/216575396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c1e6760 con 0x7f5f1c15e050 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 0x7f5f1c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f19d9b640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 0x7f5f1c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f1ad9d640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d800 tx=0x7f5f0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00b6c0 con 0x7f5f1c15e050 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0c019070 con 0x7f5f1c15e050 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c154210 con 0x7f5f1c15e050 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00bac0 con 0x7f5f1c15e050 2024-05-09T06:32:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.364+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1547d0 con 0x7f5f1c15e050 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.365+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c0752b0 con 0x7f5f1c15e050 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.366+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f0c00bc60 con 0x7f5f1c15e050 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.366+0000 7f5f12ffd640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 0x7f5efc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.366+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f5f0c08b4a0 con 0x7f5f1c15e050 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.366+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 0x7f5efc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.367+0000 7f5f1a59c640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 0x7f5efc06e320 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5f14033000 tx=0x7f5f14009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:10.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.371+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f0c059c40 con 0x7f5f1c15e050 2024-05-09T06:32:10.392 INFO:tasks.ceph.mds.d.smithi006.stderr:2024-05-09T06:32:10.411+0000 7fc86d991640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:32:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.591+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f1c072a60 con 0x7f5f1c15e050 2024-05-09T06:32:10.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.592+0000 7f5f12ffd640 1 -- 172.21.15.1:0/216575396 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5168 (secure 0 0 0) 0x7f5f0c059540 con 0x7f5f1c15e050 2024-05-09T06:32:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:10.573 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:07.811817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/2073487547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2073487547},{"type":"v1","addr":"172.21.15.6:6834","nonce":2073487547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7125,"qdb_cluster":[7125]},"id":5}]} 2024-05-09T06:32:10.573 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 45 2024-05-09T06:32:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 msgr2=0x7f5efc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 0x7f5efc06e320 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5f14033000 tx=0x7f5f14009000 comp rx=0 tx=0).stop 2024-05-09T06:32:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 msgr2=0x7f5f1c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d800 tx=0x7f5f0c00dcc0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 shutdown_connections 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5efc06be60 0x7f5efc06e320 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f1c15e050 0x7f5f1c155df0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1c0ae9b0 0x7f5f1c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 --2- 172.21.15.1:0/216575396 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1c06e4f0 0x7f5f1c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 >> 172.21.15.1:0/216575396 conn(0x7f5f1c06d270 msgr2=0x7f5f1c06d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 shutdown_connections 2024-05-09T06:32:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:10.594+0000 7f5f20b34640 1 -- 172.21.15.1:0/216575396 wait complete. 2024-05-09T06:32:10.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:32:04.798236+0000', 'modified': '2024-05-09T06:32:07.811817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7125': {'gid': 7125, 'name': 'd', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6834/2073487547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6832', 'nonce': 2073487547}, {'type': 'v1', 'addr': '172.21.15.6:6834', 'nonce': 2073487547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7125, 'qdb_cluster': [7125]} 2024-05-09T06:32:10.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:32:10.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T06:32:11.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 Processor -- start 2024-05-09T06:32:11.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 -- start start 2024-05-09T06:32:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c48161950 0x7f4c480d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 0x7f4c480d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c480d0f80 0x7f4c480cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4c480d4dd0 con 0x7f4c48161950 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4c480b33c0 con 0x7f4c480d4940 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.118+0000 7f4c4dda9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c480b3660 con 0x7f4c480d0f80 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c48161950 0x7f4c480d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4c48161950 0x7f4c480d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52486/0 (socket says 172.21.15.1:52486) 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 -- 172.21.15.1:0/1732376224 learned_addr learned my addr 172.21.15.1:0/1732376224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c47fff640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c480d0f80 0x7f4c480cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/1732376224 >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 0x7f4c480d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 -- 172.21.15.1:0/1732376224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c480d0f80 msgr2=0x7f4c480cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c480d0f80 0x7f4c480cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 -- 172.21.15.1:0/1732376224 >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 msgr2=0x7f4c480d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1732376224 >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 0x7f4c480d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c477fe640 1 -- 172.21.15.1:0/1732376224 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c480b3940 con 0x7f4c48161950 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.119+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/1732376224 >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 0x7f4c480d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c48161950 0x7f4c480d2400 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4c4806e3d0 tx=0x7f4c3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12353a6d8049407b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c467fc640 1 -- 172.21.15.1:0/1732376224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38037070 con 0x7f4c48161950 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c467fc640 1 -- 172.21.15.1:0/1732376224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38032440 con 0x7f4c48161950 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c467fc640 1 -- 172.21.15.1:0/1732376224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38032810 con 0x7f4c48161950 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1732376224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c48161950 msgr2=0x7f4c480d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c48161950 0x7f4c480d2400 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4c4806e3d0 tx=0x7f4c3802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1732376224 shutdown_connections 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c480d0f80 0x7f4c480cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1732376224 >> v2:172.21.15.6:3300/0 conn(0x7f4c480d4940 0x7f4c480d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1732376224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c48161950 0x7f4c480d2400 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1732376224 >> 172.21.15.1:0/1732376224 conn(0x7f4c480d3e20 msgr2=0x7f4c480d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1732376224 shutdown_connections 2024-05-09T06:32:11.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.120+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1732376224 wait complete. 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.122+0000 7f4c4dda9640 1 Processor -- start 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.122+0000 7f4c4dda9640 1 -- start start 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c48161950 0x7f4c4815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c480b8a30 0x7f4c480cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c480d1d80 con 0x7f4c480d0f80 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c480d1c00 con 0x7f4c480b8a30 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c480d1f00 con 0x7f4c48161950 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52492/0 (socket says 172.21.15.1:52492) 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 -- 172.21.15.1:0/1260307712 learned_addr learned my addr 172.21.15.1:0/1260307712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 -- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c48161950 msgr2=0x7f4c4815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c48161950 0x7f4c4815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c47fff640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c480b8a30 0x7f4c480cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c48161950 0x7f4c4815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 -- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c480b8a30 msgr2=0x7f4c480cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c480b8a30 0x7f4c480cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 -- 172.21.15.1:0/1260307712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c480b7990 con 0x7f4c480d0f80 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c477fe640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802a220 tx=0x7f4c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c3ffff640 1 -- 172.21.15.1:0/1260307712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38041020 con 0x7f4c480d0f80 2024-05-09T06:32:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c3ffff640 1 -- 172.21.15.1:0/1260307712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c380306c0 con 0x7f4c480d0f80 2024-05-09T06:32:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c480b7c70 con 0x7f4c480d0f80 2024-05-09T06:32:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.123+0000 7f4c3ffff640 1 -- 172.21.15.1:0/1260307712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38037040 con 0x7f4c480d0f80 2024-05-09T06:32:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.124+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c480b8230 con 0x7f4c480d0f80 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.125+0000 7f4c3ffff640 1 -- 172.21.15.1:0/1260307712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c38044020 con 0x7f4c480d0f80 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.126+0000 7f4c3ffff640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 0x7f4c2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.126+0000 7f4c3ffff640 1 -- 172.21.15.1:0/1260307712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f4c380ab8b0 con 0x7f4c480d0f80 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.126+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 0x7f4c2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.126+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 0x7f4c2806e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006fd0 tx=0x7f4c30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 Processor -- start 2024-05-09T06:32:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 -- start start 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56235e2f34d0 con 0x56235e390350 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56235e29b3e0 con 0x56235e3ba670 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.127+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56235e2ef490 con 0x56235e3d7f10 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52494/0 (socket says 172.21.15.1:52494) 2024-05-09T06:32:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 -- 172.21.15.1:0/2763531308 learned_addr learned my addr 172.21.15.1:0/2763531308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c477fe640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 -- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 msgr2=0x56235e3d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 -- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 msgr2=0x56235e3baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 -- 172.21.15.1:0/2763531308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e0072d0 con 0x56235e390350 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:11.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c47fff640 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009f40 tx=0x7f4c400077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bfb67b7bcca887 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c3d7fa640 1 -- 172.21.15.1:0/2763531308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40013780 con 0x56235e390350 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c3d7fa640 1 -- 172.21.15.1:0/2763531308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40019070 con 0x56235e390350 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.128+0000 7f4c3d7fa640 1 -- 172.21.15.1:0/2763531308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40013b60 con 0x56235e390350 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 -- 172.21.15.1:0/2763531308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 msgr2=0x56235e3d65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009f40 tx=0x7f4c400077c0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 -- 172.21.15.1:0/2763531308 shutdown_connections 2024-05-09T06:32:11.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e390350 0x56235e3d65d0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 --2- 172.21.15.1:0/2763531308 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 -- 172.21.15.1:0/2763531308 >> 172.21.15.1:0/2763531308 conn(0x56235e38cc80 msgr2=0x56235e3d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 -- 172.21.15.1:0/2763531308 shutdown_connections 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.129+0000 7f4c50b96740 1 -- 172.21.15.1:0/2763531308 wait complete. 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.130+0000 7f4c50b96740 1 -- start start 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.130+0000 7f4c50b96740 1 Processor -- start 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.130+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56235e3910f0 con 0x56235e3b9260 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56235e2ae8e0 con 0x56235e3ba670 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56235e38abb0 con 0x56235e3d7f10 2024-05-09T06:32:11.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 10 client.0 _ll_register_callbacks cb 0x56235e27b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52504/0 (socket says 172.21.15.1:52504) 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 -- 172.21.15.1:0/3959465224 learned_addr learned my addr 172.21.15.1:0/3959465224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c477fe640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 -- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 msgr2=0x56235e3cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 -- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 msgr2=0x56235e3e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3e63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 -- 172.21.15.1:0/3959465224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e0d16b0 con 0x56235e3b9260 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c47fff640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4c400094c0 tx=0x7f4c40004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefc302d61909a75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40004450 con 0x56235e3b9260 2024-05-09T06:32:11.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f4c400045f0 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.131+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c40019070 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c40009b40 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 10 client.7263 fetch_fsmap learned FSMap version 45 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x56235e3bb8f0 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40019070 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f4c40013580 con 0x56235e3b9260 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 10 client.7263 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 10 client.7263 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 20 client.7263 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3e91d0 0x56235e3e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 4 client.7263 mds_command: new command op to 7125 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 --> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235e300c00 con 0x56235e3e91d0 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.132+0000 7f4c477fe640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3e91d0 0x56235e3e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c477fe640 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3e91d0 0x56235e3e95b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4bb9882dd483782 server_cookie=6a3e880735b09c3f in_seq=0 out_seq=0 2024-05-09T06:32:11.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c267fc640 10 client.7263 ms_handle_connect on v2:172.21.15.6:6832/2073487547 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c267fc640 1 -- 172.21.15.1:0/3959465224 <== mds.0 v2:172.21.15.6:6832/2073487547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56235e300c00 con 0x56235e3e91d0 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c267fc640 10 client.7263 handle_command_reply: tid=0 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c50b96740 1 client.7263 shutdown 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 msgr2=0x56235e3b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.133+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4c400094c0 tx=0x7f4c40004ab0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 shutdown_connections 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3e91d0 0x56235e3e95b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3b96e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3d7f10 0x56235e3cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c267fc640 0 client.7263 ms_handle_reset on v2:172.21.15.6:6832/2073487547 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 --2- 172.21.15.1:0/3959465224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3ba670 0x56235e3e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 >> 172.21.15.1:0/3959465224 conn(0x56235e390350 msgr2=0x56235e3d9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 shutdown_connections 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 1 -- 172.21.15.1:0/3959465224 wait complete. 2024-05-09T06:32:11.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.134+0000 7f4c50b96740 20 client.7263 trim_cache size 0 max 16384 2024-05-09T06:32:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.152+0000 7f4c50b96740 1 Processor -- start 2024-05-09T06:32:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.152+0000 7f4c50b96740 1 -- start start 2024-05-09T06:32:11.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e390920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481ffac0 con 0x56235e3b9260 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481f4710 con 0x56235e3d7f10 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c481ff470 con 0x56235e3ba670 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52512/0 (socket says 172.21.15.1:52512) 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c477fe640 1 -- 172.21.15.1:0/597275603 learned_addr learned my addr 172.21.15.1:0/597275603 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e390920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.153+0000 7f4c47fff640 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 -- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 msgr2=0x56235e390920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e390920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 -- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 msgr2=0x56235e3d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 -- 172.21.15.1:0/597275603 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e3d7500 con 0x56235e3b9260 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c477fe640 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c38075db0 tx=0x7f4c3803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b31f43f0c8e59e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c3cff9640 1 -- 172.21.15.1:0/597275603 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38096070 con 0x56235e3b9260 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c3cff9640 1 -- 172.21.15.1:0/597275603 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38082530 con 0x56235e3b9260 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c3cff9640 1 -- 172.21.15.1:0/597275603 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38091030 con 0x56235e3b9260 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c50b96740 1 -- 172.21.15.1:0/597275603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 msgr2=0x56235e3e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.154+0000 7f4c50b96740 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c38075db0 tx=0x7f4c3803f040 comp rx=0 tx=0).stop 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 -- 172.21.15.1:0/597275603 shutdown_connections 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e390920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 --2- 172.21.15.1:0/597275603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3e4ad0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 -- 172.21.15.1:0/597275603 >> 172.21.15.1:0/597275603 conn(0x56235e390350 msgr2=0x56235e38b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 -- 172.21.15.1:0/597275603 shutdown_connections 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 -- 172.21.15.1:0/597275603 wait complete. 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 -- start start 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.155+0000 7f4c50b96740 1 Processor -- start 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e3b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56235e3cd790 con 0x56235e3b9260 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56235e3bb6d0 con 0x56235e3d7f10 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56235e3bd2e0 con 0x56235e3ba670 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c50b96740 10 client.0 _ll_register_callbacks cb 0x7f4c40009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52514/0 (socket says 172.21.15.1:52514) 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 -- 172.21.15.1:0/4006288040 learned_addr learned my addr 172.21.15.1:0/4006288040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e3b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c47fff640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 -- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 msgr2=0x56235e3b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e3b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 -- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 msgr2=0x56235e3b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c477fe640 1 -- 172.21.15.1:0/4006288040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e390e90 con 0x56235e3b9260 2024-05-09T06:32:11.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.156+0000 7f4c47fff640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c477fe640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4c380a9ca0 tx=0x7f4c40013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db141a5a9d600b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38096040 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f4c380a9cd0 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c38082530 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c38097250 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 10 client.7269 fetch_fsmap learned FSMap version 45 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x56235e44f8f0 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38082530 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f4c3803a030 con 0x56235e3b9260 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 10 client.7269 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 10 client.7269 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 20 client.7269 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3b48a0 0x56235e3cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 4 client.7269 mds_command: new command op to 7125 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 --> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] -- command(tid 0: {"prefix": "status"}) -- 0x56235e3ceee0 con 0x56235e3b48a0 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.157+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3b48a0 0x56235e3cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c267fc640 10 client.7269 ms_handle_connect on v2:172.21.15.6:6832/2073487547 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c46ffd640 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3b48a0 0x56235e3cae10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da26324b1aca9c95 server_cookie=71798cf080438ba9 in_seq=0 out_seq=0 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c267fc640 1 -- 172.21.15.1:0/4006288040 <== mds.0 v2:172.21.15.6:6832/2073487547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56235e3ceee0 con 0x56235e3b48a0 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c267fc640 10 client.7269 handle_command_reply: tid=0 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 client.7269 shutdown 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 msgr2=0x56235e3bc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4c380a9ca0 tx=0x7f4c40013000 comp rx=0 tx=0).stop 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 shutdown_connections 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:6832/2073487547,v1:172.21.15.6:6834/2073487547] conn(0x56235e3b48a0 0x56235e3cae10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56235e3d7f10 0x56235e3b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56235e3ba670 0x56235e3b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 --2- 172.21.15.1:0/4006288040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56235e3b9260 0x56235e3bc660 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c267fc640 0 client.7269 ms_handle_reset on v2:172.21.15.6:6832/2073487547 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 >> 172.21.15.1:0/4006288040 conn(0x56235e390350 msgr2=0x56235e3cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 shutdown_connections 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 1 -- 172.21.15.1:0/4006288040 wait complete. 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.158+0000 7f4c50b96740 20 client.7269 trim_cache size 0 max 16384 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7125, 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:32:11.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 5.7270617540000002, 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 45, 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 78, 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 11.714168526 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:11.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 msgr2=0x7f4c2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 0x7f4c2806e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006fd0 tx=0x7f4c30008040 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 msgr2=0x7f4c4815d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4c3802a220 tx=0x7f4c380046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 shutdown_connections 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c2806be30 0x7f4c2806e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c480b8a30 0x7f4c480cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c48161950 0x7f4c4815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 --2- 172.21.15.1:0/1260307712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c480d0f80 0x7f4c4815d6e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 >> 172.21.15.1:0/1260307712 conn(0x7f4c480d3e20 msgr2=0x7f4c480b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 shutdown_connections 2024-05-09T06:32:11.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.159+0000 7f4c4dda9640 1 -- 172.21.15.1:0/1260307712 wait complete. 2024-05-09T06:32:11.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:32:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.273+0000 7fc8b3ebc640 1 Processor -- start 2024-05-09T06:32:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 -- start start 2024-05-09T06:32:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15db10 0x7fc8ac0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ac070f40 con 0x7fc8ac15e500 2024-05-09T06:32:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc8ac0cf170 con 0x7fc8ac0715d0 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.274+0000 7fc8b3ebc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac0cf450 con 0x7fc8ac15db10 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52524/0 (socket says 172.21.15.1:52524) 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 -- 172.21.15.1:0/311856395 learned_addr learned my addr 172.21.15.1:0/311856395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1c58640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15db10 0x7fc8ac0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 -- 172.21.15.1:0/311856395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15db10 msgr2=0x7fc8ac0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15db10 0x7fc8ac0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 -- 172.21.15.1:0/311856395 >> v1:172.21.15.6:6789/0 conn(0x7fc8ac15e500 legacy=0x7fc8ac06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.275+0000 7fc8b1457640 1 -- 172.21.15.1:0/311856395 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac070c10 con 0x7fc8ac0715d0 2024-05-09T06:32:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b1457640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009870 tx=0x7fc8a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d38d7f6490e883 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b0c56640 1 -- 172.21.15.1:0/311856395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0037070 con 0x7fc8ac0715d0 2024-05-09T06:32:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b0c56640 1 -- 172.21.15.1:0/311856395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a0032440 con 0x7fc8ac0715d0 2024-05-09T06:32:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b0c56640 1 -- 172.21.15.1:0/311856395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0032810 con 0x7fc8ac0715d0 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/311856395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac0715d0 msgr2=0x7fc8ac15dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009870 tx=0x7fc8a002a270 comp rx=0 tx=0).stop 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/311856395 shutdown_connections 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15dfc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/311856395 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15db10 0x7fc8ac0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/311856395 >> 172.21.15.1:0/311856395 conn(0x7fc8ac0cc4b0 msgr2=0x7fc8ac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/311856395 shutdown_connections 2024-05-09T06:32:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.276+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/311856395 wait complete. 2024-05-09T06:32:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 Processor -- start 2024-05-09T06:32:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 -- start start 2024-05-09T06:32:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15e500 0x7fc8ac0b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac15edc0 con 0x7fc8ac15db10 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac15ec40 con 0x7fc8ac0715d0 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b3ebc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac06ea50 con 0x7fc8ac15e500 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b1457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b1457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52526/0 (socket says 172.21.15.1:52526) 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.278+0000 7fc8b1457640 1 -- 172.21.15.1:0/2719795186 learned_addr learned my addr 172.21.15.1:0/2719795186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1c58640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b2459640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15e500 0x7fc8ac0b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 -- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15e500 msgr2=0x7fc8ac0b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15e500 0x7fc8ac0b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 -- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8ac0715d0 msgr2=0x7fc8ac15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 -- 172.21.15.1:0/2719795186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1517b0 con 0x7fc8ac15db10 2024-05-09T06:32:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b1457640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009870 tx=0x7fc8a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0041070 con 0x7fc8ac15db10 2024-05-09T06:32:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a003c870 con 0x7fc8ac15db10 2024-05-09T06:32:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac151a90 con 0x7fc8ac15db10 2024-05-09T06:32:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0037040 con 0x7fc8ac15db10 2024-05-09T06:32:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.279+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac152050 con 0x7fc8ac15db10 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.280+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac0752b0 con 0x7fc8ac15db10 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.281+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8a0044070 con 0x7fc8ac15db10 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.281+0000 7fc8aa7fc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.281+0000 7fc8b1c58640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.281+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fc8a00abb60 con 0x7fc8ac15db10 2024-05-09T06:32:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.282+0000 7fc8b1c58640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 0x7fc89806e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:11.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.286+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8a0079e50 con 0x7fc8ac15db10 2024-05-09T06:32:11.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.508+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8ac069290 con 0x7fc8ac15db10 2024-05-09T06:32:11.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.508+0000 7fc8aa7fc640 1 -- 172.21.15.1:0/2719795186 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fc8a007e150 con 0x7fc8ac15db10 2024-05-09T06:32:11.489 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 msgr2=0x7fc89806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 0x7fc89806e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2024-05-09T06:32:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 msgr2=0x7fc8ac150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009870 tx=0x7fc8a00046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 shutdown_connections 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8ac15e500 0x7fc8ac0b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac15db10 0x7fc8ac150d40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 --2- 172.21.15.1:0/2719795186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8ac0715d0 0x7fc8ac15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 >> 172.21.15.1:0/2719795186 conn(0x7fc8ac0cc4b0 msgr2=0x7fc8ac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 shutdown_connections 2024-05-09T06:32:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11.510+0000 7fc8b3ebc640 1 -- 172.21.15.1:0/2719795186 wait complete. 2024-05-09T06:32:11.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:32:11.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:32:11.503 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:11.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:11.503 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:32:11.520 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:32:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:32:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:32:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42411sec preferred_lft 42411sec 2024-05-09T06:32:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:32:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.523 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:32:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:32:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:11.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:11.523 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:32:11.584 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:32:11.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:11.587 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:32:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:32:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:32:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:32:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:32:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:32:11.644 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:32:11.644 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:32:11.644 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:32:11.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:32:11.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:32:11.645 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:32:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:32:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:32:11.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:11.774 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:32:11.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:11.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:11.790 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:32:11.856 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:32:11.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:32:11.856 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:11.912 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:11.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:32:11.912 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:11.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:32:12.035 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:32:12.092 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:32:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:32:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:12.095 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:12.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.096 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:32:12.149 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:32:12.149 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:32:12.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:32:12.218 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:32:12.276 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:32:12.277 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:12.278 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:12.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.279 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:32:12.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:32:12.296+0000 7f30bb922480 -1 init, newargv = 0x556144421250 newargc=15 2024-05-09T06:32:12.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[62457]: starting ceph client 2024-05-09T06:32:12.283 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[62457]: starting fuse 2024-05-09T06:32:12.295 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:32:12.295 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:32:12.296 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:32:12.417 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.62457.asok 2024-05-09T06:32:12.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:32:12.428 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62457.asok 2024-05-09T06:32:12.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62457.asok status 2024-05-09T06:32:12.568 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "62457", 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7284, 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7284 2024-05-09T06:32:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3223499307 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:32:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3223499307 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.7284 192.168.0.1:0/3223499307", 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/3223499307", 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 45, 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 78, 2024-05-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:32:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:32:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:32:12.573 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:12.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.577 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:32:12.577 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:12.593 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:32:12.593 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:12.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.593 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:12.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.661 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-09T06:32:12.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.725 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-09T06:32:12.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:12.784 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-09T06:32:12.843 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:32:12.951 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:32:12.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:32:12.960 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:32:12.960 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:32:13.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:13.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.180+0000 7f225d5a5640 1 Processor -- start 2024-05-09T06:32:13.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.180+0000 7f225d5a5640 1 -- start start 2024-05-09T06:32:13.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f225d5a5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2258161950 0x7f22580d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f225d5a5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f22580d0f80 0x7f22580d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f225d5a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22580b33c0 con 0x7f22580d4940 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f225d5a5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f22580d4dd0 con 0x7f22580d0f80 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f225d5a5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f22580b3660 con 0x7f2258161950 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f22580d0f80 0x7f22580d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f22580d0f80 0x7f22580d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52534/0 (socket says 172.21.15.1:52534) 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 -- 172.21.15.1:0/2131132216 learned_addr learned my addr 172.21.15.1:0/2131132216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f2256ffd640 1 --2- 172.21.15.1:0/2131132216 >> v2:172.21.15.6:3300/0 conn(0x7f2258161950 0x7f22580d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 -- 172.21.15.1:0/2131132216 >> v2:172.21.15.6:3300/0 conn(0x7f2258161950 msgr2=0x7f22580d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 --2- 172.21.15.1:0/2131132216 >> v2:172.21.15.6:3300/0 conn(0x7f2258161950 0x7f22580d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.181+0000 7f22577fe640 1 -- 172.21.15.1:0/2131132216 >> v1:172.21.15.6:6789/0 conn(0x7f22580d4940 legacy=0x7f22580cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.182+0000 7f22577fe640 1 -- 172.21.15.1:0/2131132216 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22580b3940 con 0x7f22580d0f80 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.182+0000 7f22577fe640 1 --2- 172.21.15.1:0/2131132216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f22580d2940 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f225806b7f0 tx=0x7f225002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c441346f18b14ee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.182+0000 7f2255ffb640 1 -- 172.21.15.1:0/2131132216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250036070 con 0x7f22580d0f80 2024-05-09T06:32:13.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.182+0000 7f2255ffb640 1 -- 172.21.15.1:0/2131132216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2250031600 con 0x7f22580d0f80 2024-05-09T06:32:13.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.182+0000 7f2255ffb640 1 -- 172.21.15.1:0/2131132216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500319d0 con 0x7f22580d0f80 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 -- 172.21.15.1:0/2131132216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 msgr2=0x7f22580d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 --2- 172.21.15.1:0/2131132216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f22580d2940 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f225806b7f0 tx=0x7f225002a730 comp rx=0 tx=0).stop 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 -- 172.21.15.1:0/2131132216 shutdown_connections 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 --2- 172.21.15.1:0/2131132216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f22580d2940 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 --2- 172.21.15.1:0/2131132216 >> v2:172.21.15.6:3300/0 conn(0x7f2258161950 0x7f22580d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 -- 172.21.15.1:0/2131132216 >> 172.21.15.1:0/2131132216 conn(0x7f22580d3e20 msgr2=0x7f22580d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 -- 172.21.15.1:0/2131132216 shutdown_connections 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.183+0000 7f225d5a5640 1 -- 172.21.15.1:0/2131132216 wait complete. 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.184+0000 7f225d5a5640 1 Processor -- start 2024-05-09T06:32:13.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.184+0000 7f225d5a5640 1 -- start start 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2258161950 0x7f22580cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 0x7f22580b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b52d0 con 0x7f22580d0f80 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b5150 con 0x7f22581f4960 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f225d5a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f22580b5450 con 0x7f2258161950 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f2256ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f2256ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52546/0 (socket says 172.21.15.1:52546) 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.185+0000 7f2256ffd640 1 -- 172.21.15.1:0/332520501 learned_addr learned my addr 172.21.15.1:0/332520501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f22567fc640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2258161950 0x7f22580cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f22577fe640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 0x7f22580b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 -- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2258161950 msgr2=0x7f22580cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2258161950 0x7f22580cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 -- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 msgr2=0x7f22580b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 0x7f22580b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 -- 172.21.15.1:0/332520501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225815c5a0 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f22577fe640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 0x7f22580b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f2256ffd640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f224800d5f0 tx=0x7f224800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480125f0 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2248019070 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225815c880 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480129d0 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.186+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225815c290 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.187+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22580b7f50 con 0x7f22580d0f80 2024-05-09T06:32:13.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.188+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2248012b70 con 0x7f22580d0f80 2024-05-09T06:32:13.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.188+0000 7f223f7fe640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 0x7f224406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.188+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f224808b400 con 0x7f22580d0f80 2024-05-09T06:32:13.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.188+0000 7f22567fc640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 0x7f224406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.189+0000 7f22567fc640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 0x7f224406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f22580b9540 tx=0x7f224c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.193+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2248059c20 con 0x7f22580d0f80 2024-05-09T06:32:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.377+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22580752b0 con 0x7f22580d0f80 2024-05-09T06:32:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.378+0000 7f223f7fe640 1 -- 172.21.15.1:0/332520501 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16813 (secure 0 0 0) 0x7f2248059520 con 0x7f22580d0f80 2024-05-09T06:32:13.360 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:13.360 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:06.420655+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:13.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 msgr2=0x7f224406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 0x7f224406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f22580b9540 tx=0x7f224c009040 comp rx=0 tx=0).stop 2024-05-09T06:32:13.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 msgr2=0x7f225815bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f224800d5f0 tx=0x7f224800dab0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 shutdown_connections 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f224406be30 0x7f224406e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22581f4960 0x7f22580b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2258161950 0x7f22580cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 --2- 172.21.15.1:0/332520501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22580d0f80 0x7f225815bd50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 >> 172.21.15.1:0/332520501 conn(0x7f22580d3e20 msgr2=0x7f225806bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 shutdown_connections 2024-05-09T06:32:13.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.381+0000 7f225d5a5640 1 -- 172.21.15.1:0/332520501 wait complete. 2024-05-09T06:32:13.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:32:13.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:32:13.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.502+0000 7f6cac264640 1 Processor -- start 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 -- start start 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6ca415a4d0 0x7f6ca40cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ca40cdb90 0x7f6ca4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 0x7f6ca415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca415a8b0 con 0x7f6ca40cdb90 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca406dc10 con 0x7f6ca415a4d0 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.503+0000 7f6cac264640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca40cf1b0 con 0x7f6ca4071440 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ca40cdb90 0x7f6ca4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ca40cdb90 0x7f6ca4070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52548/0 (socket says 172.21.15.1:52548) 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 -- 172.21.15.1:0/3564855109 learned_addr learned my addr 172.21.15.1:0/3564855109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6caa000640 1 --2- 172.21.15.1:0/3564855109 >> v2:172.21.15.6:3300/0 conn(0x7f6ca415a4d0 0x7f6ca40cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6caa801640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 0x7f6ca415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 -- 172.21.15.1:0/3564855109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 msgr2=0x7f6ca415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 0x7f6ca415dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 -- 172.21.15.1:0/3564855109 >> v2:172.21.15.6:3300/0 conn(0x7f6ca415a4d0 msgr2=0x7f6ca40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3564855109 >> v2:172.21.15.6:3300/0 conn(0x7f6ca415a4d0 0x7f6ca40cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6ca97ff640 1 -- 172.21.15.1:0/3564855109 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca40cf490 con 0x7f6ca40cdb90 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.504+0000 7f6caa801640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 0x7f6ca415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.505+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca40cdb90 0x7f6ca4070f00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6ca406c810 tx=0x7f6c9802b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1fce2f1db7597d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.505+0000 7f6ca8ffe640 1 -- 172.21.15.1:0/3564855109 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800b040 con 0x7f6ca40cdb90 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.505+0000 7f6ca8ffe640 1 -- 172.21.15.1:0/3564855109 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c98030710 con 0x7f6ca40cdb90 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.505+0000 7f6ca8ffe640 1 -- 172.21.15.1:0/3564855109 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98030ae0 con 0x7f6ca40cdb90 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 -- 172.21.15.1:0/3564855109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca40cdb90 msgr2=0x7f6ca4070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca40cdb90 0x7f6ca4070f00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6ca406c810 tx=0x7f6c9802b5a0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 -- 172.21.15.1:0/3564855109 shutdown_connections 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca4071440 0x7f6ca415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 --2- 172.21.15.1:0/3564855109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca40cdb90 0x7f6ca4070f00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 --2- 172.21.15.1:0/3564855109 >> v2:172.21.15.6:3300/0 conn(0x7f6ca415a4d0 0x7f6ca40cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 -- 172.21.15.1:0/3564855109 >> 172.21.15.1:0/3564855109 conn(0x7f6ca40cc4d0 msgr2=0x7f6ca40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:13.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 -- 172.21.15.1:0/3564855109 shutdown_connections 2024-05-09T06:32:13.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.506+0000 7f6cac264640 1 -- 172.21.15.1:0/3564855109 wait complete. 2024-05-09T06:32:13.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.507+0000 7f6cac264640 1 Processor -- start 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.507+0000 7f6cac264640 1 -- start start 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca40cdb90 0x7f6ca4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ca415a4d0 0x7f6ca4155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40b2be0 con 0x7f6ca4071440 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40b2a60 con 0x7f6ca415a4d0 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca406cd30 con 0x7f6ca40cdb90 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52554/0 (socket says 172.21.15.1:52554) 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 -- 172.21.15.1:0/3432292349 learned_addr learned my addr 172.21.15.1:0/3432292349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa801640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ca415a4d0 0x7f6ca4155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca40cdb90 0x7f6ca4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 -- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca40cdb90 msgr2=0x7f6ca4155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca40cdb90 0x7f6ca4155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 -- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ca415a4d0 msgr2=0x7f6ca4155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ca415a4d0 0x7f6ca4155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 -- 172.21.15.1:0/3432292349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca4150560 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6caa000640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400d620 tx=0x7f6c9400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400b620 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c94019070 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400ba00 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca4151db0 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.508+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca4152370 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.510+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca40752b0 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.510+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c9400bba0 con 0x7f6ca4071440 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.511+0000 7f6ca27fc640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 0x7f6c9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.511+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f6c9408b400 con 0x7f6ca4071440 2024-05-09T06:32:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.511+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 0x7f6c9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.511+0000 7f6ca97ff640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 0x7f6c9006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c98035000 tx=0x7f6c9800c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.515+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6c94059c20 con 0x7f6ca4071440 2024-05-09T06:32:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.697+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ca4069700 con 0x7f6ca4071440 2024-05-09T06:32:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.697+0000 7f6ca27fc640 1 -- 172.21.15.1:0/3432292349 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f6c94059520 con 0x7f6ca4071440 2024-05-09T06:32:13.679 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 msgr2=0x7f6c9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 0x7f6c9006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c98035000 tx=0x7f6c9800c040 comp rx=0 tx=0).stop 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 msgr2=0x7f6ca4151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400d620 tx=0x7f6c9400dae0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 shutdown_connections 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c9006be30 0x7f6c9006e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ca415a4d0 0x7f6ca4155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ca40cdb90 0x7f6ca4155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 --2- 172.21.15.1:0/3432292349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ca4071440 0x7f6ca4151720 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 >> 172.21.15.1:0/3432292349 conn(0x7f6ca40cc4d0 msgr2=0x7f6ca406dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:13.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.699+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 shutdown_connections 2024-05-09T06:32:13.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.700+0000 7f6cac264640 1 -- 172.21.15.1:0/3432292349 wait complete. 2024-05-09T06:32:13.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:32:13.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.817+0000 7fc10803f640 1 Processor -- start 2024-05-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.817+0000 7fc10803f640 1 -- start start 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc10803f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc100071fd0 0x7fc10006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc10803f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10015b420 con 0x7fc10015c4a0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc10803f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10015b6e0 con 0x7fc100158a50 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc10803f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc100158e30 con 0x7fc100071fd0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc1065dc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc10015c4a0 0x7fc10015ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55284/0 (socket says 172.21.15.1:55284) 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc1065dc640 1 -- 172.21.15.1:0/3119594501 learned_addr learned my addr 172.21.15.1:0/3119594501 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc1055da640 1 --2- 172.21.15.1:0/3119594501 >> v2:172.21.15.6:3300/0 conn(0x7fc100071fd0 0x7fc10006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586372793 0 0) 0x7fc10015b420 con 0x7fc10015c4a0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.818+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0e8003520 con 0x7fc10015c4a0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3764361893 0 0) 0x7fc0e8003520 con 0x7fc10015c4a0 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 >> v2:172.21.15.6:3300/0 conn(0x7fc100071fd0 msgr2=0x7fc10006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 --2- 172.21.15.1:0/3119594501 >> v2:172.21.15.6:3300/0 conn(0x7fc100071fd0 0x7fc10006e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 >> v1:172.21.15.6:6789/0 conn(0x7fc100158a50 legacy=0x7fc1000718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10015b9c0 con 0x7fc10015c4a0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fc0f00019f0 con 0x7fc10015c4a0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 >> v1:172.21.15.1:6789/0 conn(0x7fc10015c4a0 legacy=0x7fc10015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80039b0 0x7fc0e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0e8004800 0x7fc0e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 0x7fc0e800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc100158e30 con 0x7fc0e80051c0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8005990 con 0x7fc0e8004800 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e8005b90 con 0x7fc0e80039b0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 0x7fc0e800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc105ddb640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80039b0 0x7fc0e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1055da640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0e8004800 0x7fc0e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 -- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80039b0 msgr2=0x7fc0e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80039b0 0x7fc0e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 -- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0e8004800 msgr2=0x7fc0e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0e8004800 0x7fc0e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.819+0000 7fc1065dc640 1 -- 172.21.15.1:0/3119594501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e8003520 con 0x7fc0e80051c0 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.820+0000 7fc1065dc640 1 -- 172.21.15.1:0/3119594501 reap_dead start 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.820+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 0x7fc0e800dbd0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc10006c2c0 tx=0x7fc0e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc1fd25149dbc7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.820+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000a480 con 0x7fc0e80051c0 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.820+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f002f070 con 0x7fc0e80051c0 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.820+0000 7fc104dd9640 1 -- 172.21.15.1:0/3119594501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f002bc50 con 0x7fc0e80051c0 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 -- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 msgr2=0x7fc0e800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 0x7fc0e800dbd0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc10006c2c0 tx=0x7fc0e8005830 comp rx=0 tx=0).stop 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 -- 172.21.15.1:0/3119594501 shutdown_connections 2024-05-09T06:32:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 --2- 172.21.15.1:0/3119594501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e80051c0 0x7fc0e800dbd0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 -- 172.21.15.1:0/3119594501 >> 172.21.15.1:0/3119594501 conn(0x7fc1000cc4d0 msgr2=0x7fc1000cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 -- 172.21.15.1:0/3119594501 shutdown_connections 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.821+0000 7fc10803f640 1 -- 172.21.15.1:0/3119594501 wait complete. 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.822+0000 7fc10803f640 1 Processor -- start 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.822+0000 7fc10803f640 1 -- start start 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80051c0 0x7fc100155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc100155620 0x7fc10015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc10015bf80 con 0x7fc1000aaea0 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc100072480 con 0x7fc100155620 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc10803f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc100070ea0 con 0x7fc0e80051c0 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52572/0 (socket says 172.21.15.1:52572) 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 -- 172.21.15.1:0/3676232317 learned_addr learned my addr 172.21.15.1:0/3676232317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1055da640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc100155620 0x7fc10015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 -- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80051c0 msgr2=0x7fc100155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80051c0 0x7fc100155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 -- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc100155620 msgr2=0x7fc10015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc100155620 0x7fc10015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 -- 172.21.15.1:0/3676232317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000aee00 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.823+0000 7fc1065dc640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00012e0 tx=0x7fc10015b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.824+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0044070 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.824+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f002bd50 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.824+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000a480 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.824+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1000af0e0 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.824+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1000a99b0 con 0x7fc1000aaea0 2024-05-09T06:32:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.825+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1000752b0 con 0x7fc1000aaea0 2024-05-09T06:32:13.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.825+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0f0047070 con 0x7fc1000aaea0 2024-05-09T06:32:13.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.826+0000 7fc0fe7fc640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 0x7fc0ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:13.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.826+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fc0f00aa9e0 con 0x7fc1000aaea0 2024-05-09T06:32:13.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.826+0000 7fc105ddb640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 0x7fc0ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:13.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.826+0000 7fc105ddb640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 0x7fc0ec06e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8000c00 tx=0x7fc0f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:13.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:13.830+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0f0078c40 con 0x7fc1000aaea0 2024-05-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.016+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1000cc250 con 0x7fc1000aaea0 2024-05-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.017+0000 7fc0fe7fc640 1 -- 172.21.15.1:0/3676232317 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7fc0f007cf40 con 0x7fc1000aaea0 2024-05-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 msgr2=0x7fc0ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 0x7fc0ec06e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8000c00 tx=0x7fc0f8009040 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 msgr2=0x7fc1000ab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00012e0 tx=0x7fc10015b8b0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 shutdown_connections 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc0ec06be60 0x7fc0ec06e320 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1000aaea0 0x7fc1000ab320 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc100155620 0x7fc10015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 --2- 172.21.15.1:0/3676232317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0e80051c0 0x7fc100155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 >> 172.21.15.1:0/3676232317 conn(0x7fc1000cc4d0 msgr2=0x7fc100159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 shutdown_connections 2024-05-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.019+0000 7fc10803f640 1 -- 172.21.15.1:0/3676232317 wait complete. 2024-05-09T06:32:14.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.137+0000 7f23605a2640 1 Processor -- start 2024-05-09T06:32:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.137+0000 7f23605a2640 1 -- start start 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235815e190 0x7f235815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f23580d1c70 0x7f23580d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580b56d0 0x7f23580d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2358161fe0 con 0x7f235815e190 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f23581622c0 con 0x7f23580d1c70 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f23605a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f23581625a0 con 0x7f23580b56d0 2024-05-09T06:32:14.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235815e190 0x7f235815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235815e190 0x7f235815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52580/0 (socket says 172.21.15.1:52580) 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 -- 172.21.15.1:0/2682971219 learned_addr learned my addr 172.21.15.1:0/2682971219 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235db3d640 1 --2- 172.21.15.1:0/2682971219 >> v2:172.21.15.6:3300/0 conn(0x7f23580d1c70 0x7f23580d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235eb3f640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580b56d0 0x7f23580d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 -- 172.21.15.1:0/2682971219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580b56d0 msgr2=0x7f23580d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580b56d0 0x7f23580d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 -- 172.21.15.1:0/2682971219 >> v2:172.21.15.6:3300/0 conn(0x7f23580d1c70 msgr2=0x7f23580d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 --2- 172.21.15.1:0/2682971219 >> v2:172.21.15.6:3300/0 conn(0x7f23580d1c70 0x7f23580d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.138+0000 7f235e33e640 1 -- 172.21.15.1:0/2682971219 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358162880 con 0x7f235815e190 2024-05-09T06:32:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.139+0000 7f235e33e640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f235815f780 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f23580b6c30 tx=0x7f235402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad18342cebb4dac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.139+0000 7f235d33c640 1 -- 172.21.15.1:0/2682971219 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354036070 con 0x7f235815e190 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.139+0000 7f235d33c640 1 -- 172.21.15.1:0/2682971219 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2354031600 con 0x7f235815e190 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.139+0000 7f235d33c640 1 -- 172.21.15.1:0/2682971219 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540319d0 con 0x7f235815e190 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 -- 172.21.15.1:0/2682971219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 msgr2=0x7f235815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f235815f780 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f23580b6c30 tx=0x7f235402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 -- 172.21.15.1:0/2682971219 shutdown_connections 2024-05-09T06:32:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580b56d0 0x7f23580d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 --2- 172.21.15.1:0/2682971219 >> v2:172.21.15.6:3300/0 conn(0x7f23580d1c70 0x7f23580d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 --2- 172.21.15.1:0/2682971219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f235815f780 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 -- 172.21.15.1:0/2682971219 >> 172.21.15.1:0/2682971219 conn(0x7f2358073340 msgr2=0x7f235809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 -- 172.21.15.1:0/2682971219 shutdown_connections 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.140+0000 7f23605a2640 1 -- 172.21.15.1:0/2682971219 wait complete. 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.141+0000 7f23605a2640 1 Processor -- start 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.141+0000 7f23605a2640 1 -- start start 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23580b56d0 0x7f23580cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580d1c70 0x7f23580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b3f60 con 0x7f235815e190 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b3de0 con 0x7f23580b56d0 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f23605a2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f23580b40e0 con 0x7f23580d1c70 2024-05-09T06:32:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52588/0 (socket says 172.21.15.1:52588) 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 -- 172.21.15.1:0/130647565 learned_addr learned my addr 172.21.15.1:0/130647565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235db3d640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580d1c70 0x7f23580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 -- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580d1c70 msgr2=0x7f23580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580d1c70 0x7f23580cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 -- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23580b56d0 msgr2=0x7f23580cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23580b56d0 0x7f23580cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 -- 172.21.15.1:0/130647565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23580c83b0 con 0x7f235815e190 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.142+0000 7f235eb3f640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f235000e7d0 tx=0x7f235000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.143+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350017070 con 0x7f235815e190 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.143+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235000c940 con 0x7f235815e190 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.143+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23580c8690 con 0x7f235815e190 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.143+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000cd30 con 0x7f235815e190 2024-05-09T06:32:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.143+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23580cbe00 con 0x7f235815e190 2024-05-09T06:32:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.144+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23580752b0 con 0x7f235815e190 2024-05-09T06:32:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.144+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f235001a070 con 0x7f235815e190 2024-05-09T06:32:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.145+0000 7f234effd640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 0x7f233c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.145+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f235008b250 con 0x7f235815e190 2024-05-09T06:32:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.145+0000 7f235e33e640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 0x7f233c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.145+0000 7f235e33e640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 0x7f233c06e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2354033000 tx=0x7f235402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.149+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2350059b00 con 0x7f235815e190 2024-05-09T06:32:14.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.333+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f235808e1c0 con 0x7f235815e190 2024-05-09T06:32:14.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.334+0000 7f234effd640 1 -- 172.21.15.1:0/130647565 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f2350059400 con 0x7f235815e190 2024-05-09T06:32:14.316 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 msgr2=0x7f233c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 0x7f233c06e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2354033000 tx=0x7f235402e000 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 msgr2=0x7f23581528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f235000e7d0 tx=0x7f235000ec90 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 shutdown_connections 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f233c06be30 0x7f233c06e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235815e190 0x7f23581528d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23580d1c70 0x7f23580cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 --2- 172.21.15.1:0/130647565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23580b56d0 0x7f23580cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 >> 172.21.15.1:0/130647565 conn(0x7f2358073340 msgr2=0x7f235815c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 shutdown_connections 2024-05-09T06:32:14.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.336+0000 7f23605a2640 1 -- 172.21.15.1:0/130647565 wait complete. 2024-05-09T06:32:14.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:32:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.452+0000 7f9deebed640 1 Processor -- start 2024-05-09T06:32:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.452+0000 7f9deebed640 1 -- start start 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9deebed640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9de815c680 0x7f9de80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9deebed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80b0170 0x7f9de81601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9deebed640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de815cc40 con 0x7f9de81565e0 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9deebed640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9de815fd30 con 0x7f9de815c680 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9deebed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9de815cf20 con 0x7f9de80b0170 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9dec989640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9de81565e0 0x7f9de8074530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55298/0 (socket says 172.21.15.1:55298) 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9dec989640 1 -- 172.21.15.1:0/478217612 learned_addr learned my addr 172.21.15.1:0/478217612 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9de7fff640 1 --2- 172.21.15.1:0/478217612 >> v2:172.21.15.1:3300/0 conn(0x7f9de815c680 0x7f9de80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9ded18a640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80b0170 0x7f9de81601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.453+0000 7f9de77fe640 1 -- 172.21.15.1:0/478217612 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545826461 0 0) 0x7f9de815cc40 con 0x7f9de81565e0 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de77fe640 1 -- 172.21.15.1:0/478217612 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dd0003520 con 0x7f9de81565e0 2024-05-09T06:32:14.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de7fff640 1 -- 172.21.15.1:0/478217612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80b0170 msgr2=0x7f9de81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de7fff640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80b0170 0x7f9de81601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de7fff640 1 -- 172.21.15.1:0/478217612 >> v1:172.21.15.1:6789/0 conn(0x7f9de81565e0 legacy=0x7f9de8074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de7fff640 1 -- 172.21.15.1:0/478217612 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de815d200 con 0x7f9de815c680 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de7fff640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de815c680 0x7f9de80afc30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0097a0 tx=0x7f9ddc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997798af8fb65a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.454+0000 7f9de77fe640 1 -- 172.21.15.1:0/478217612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc037070 con 0x7f9de815c680 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9de77fe640 1 -- 172.21.15.1:0/478217612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc032440 con 0x7f9de815c680 2024-05-09T06:32:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9de77fe640 1 -- 172.21.15.1:0/478217612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc032810 con 0x7f9de815c680 2024-05-09T06:32:14.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 -- 172.21.15.1:0/478217612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de815c680 msgr2=0x7f9de80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de815c680 0x7f9de80afc30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0097a0 tx=0x7f9ddc02a180 comp rx=0 tx=0).stop 2024-05-09T06:32:14.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 -- 172.21.15.1:0/478217612 shutdown_connections 2024-05-09T06:32:14.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de80b0170 0x7f9de81601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 --2- 172.21.15.1:0/478217612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de815c680 0x7f9de80afc30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 -- 172.21.15.1:0/478217612 >> 172.21.15.1:0/478217612 conn(0x7f9de80b1fb0 msgr2=0x7f9de80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 -- 172.21.15.1:0/478217612 shutdown_connections 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.455+0000 7f9deebed640 1 -- 172.21.15.1:0/478217612 wait complete. 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.456+0000 7f9deebed640 1 Processor -- start 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.456+0000 7f9deebed640 1 -- start start 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de81565e0 0x7f9de815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 0x7f9de815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8160700 con 0x7f9de80b0170 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81569c0 con 0x7f9de815c680 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9deebed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9de8160880 con 0x7f9de81565e0 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9dec989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9dec989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52616/0 (socket says 172.21.15.1:52616) 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.457+0000 7f9dec989640 1 -- 172.21.15.1:0/83739132 learned_addr learned my addr 172.21.15.1:0/83739132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 -- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de81565e0 msgr2=0x7f9de815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9de7fff640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de81565e0 0x7f9de815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de81565e0 0x7f9de815b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 -- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 msgr2=0x7f9de815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9ded18a640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 0x7f9de815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 0x7f9de815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 -- 172.21.15.1:0/83739132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de81f5160 con 0x7f9de80b0170 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9ded18a640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 0x7f9de815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9dec989640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c430 tx=0x7f9dd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8017070 con 0x7f9de80b0170 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8005ce0 con 0x7f9de80b0170 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80072f0 con 0x7f9de80b0170 2024-05-09T06:32:14.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de81f5440 con 0x7f9de80b0170 2024-05-09T06:32:14.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.458+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81f5a00 con 0x7f9de80b0170 2024-05-09T06:32:14.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.459+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de80752b0 con 0x7f9de80b0170 2024-05-09T06:32:14.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.460+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dd801a070 con 0x7f9de80b0170 2024-05-09T06:32:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.460+0000 7f9de57fa640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 0x7f9dd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.460+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f9dd808a3f0 con 0x7f9de80b0170 2024-05-09T06:32:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.460+0000 7f9de7fff640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 0x7f9dd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.461+0000 7f9de7fff640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 0x7f9dd406e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc034000 tx=0x7f9ddc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.465+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9dd8058ca0 con 0x7f9de80b0170 2024-05-09T06:32:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.658+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9de8153e50 con 0x7f9de80b0170 2024-05-09T06:32:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.659+0000 7f9de57fa640 1 -- 172.21.15.1:0/83739132 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f9dd80585a0 con 0x7f9de80b0170 2024-05-09T06:32:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.660+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 msgr2=0x7f9dd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 0x7f9dd406e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc034000 tx=0x7f9ddc02e000 comp rx=0 tx=0).stop 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 msgr2=0x7f9de81f1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c430 tx=0x7f9dd800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 shutdown_connections 2024-05-09T06:32:14.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9dd406be60 0x7f9dd406e320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9de815c680 0x7f9de815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9de81565e0 0x7f9de815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 --2- 172.21.15.1:0/83739132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de80b0170 0x7f9de81f1220 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 >> 172.21.15.1:0/83739132 conn(0x7f9de80b1fb0 msgr2=0x7f9de80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 shutdown_connections 2024-05-09T06:32:14.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.661+0000 7f9deebed640 1 -- 172.21.15.1:0/83739132 wait complete. 2024-05-09T06:32:14.653 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:32:14.653 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:32:14.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.832+0000 7fd5071c8640 1 Processor -- start 2024-05-09T06:32:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.832+0000 7fd5071c8640 1 -- start start 2024-05-09T06:32:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd5071c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd500163c70 0x7fd500166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd5071c8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5000ce0e0 con 0x7fd5000cf180 2024-05-09T06:32:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd5071c8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5000ce3c0 con 0x7fd50006e6d0 2024-05-09T06:32:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd5071c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd5000ce6a0 con 0x7fd500163c70 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd4fffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd5000cf180 0x7fd5000cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55304/0 (socket says 172.21.15.1:55304) 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.833+0000 7fd4fffff640 1 -- 172.21.15.1:0/3006902356 learned_addr learned my addr 172.21.15.1:0/3006902356 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 --2- 172.21.15.1:0/3006902356 >> v2:172.21.15.6:3300/0 conn(0x7fd500163c70 0x7fd500166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437983789 0 0) 0x7fd5000ce0e0 con 0x7fd5000cf180 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4e8003520 con 0x7fd5000cf180 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994154312 0 0) 0x7fd4e8003520 con 0x7fd5000cf180 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 >> v2:172.21.15.6:3300/0 conn(0x7fd500163c70 msgr2=0x7fd500166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 --2- 172.21.15.1:0/3006902356 >> v2:172.21.15.6:3300/0 conn(0x7fd500163c70 0x7fd500166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 >> v1:172.21.15.6:6789/0 conn(0x7fd50006e6d0 legacy=0x7fd50006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5000ce980 con 0x7fd5000cf180 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fd4f4001670 con 0x7fd5000cf180 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 >> v1:172.21.15.1:6789/0 conn(0x7fd5000cf180 legacy=0x7fd5000cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5000ce6a0 con 0x7fd4e80051c0 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e800c060 con 0x7fd4e8004800 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e800c260 con 0x7fd4e80039b0 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd4fffff640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd504f64640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 -- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4e80039b0 msgr2=0x7fd4e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd4e80039b0 0x7fd4e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 -- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e8004800 msgr2=0x7fd4e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e8004800 0x7fd4e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.834+0000 7fd505765640 1 -- 172.21.15.1:0/3006902356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e8003520 con 0x7fd4e80051c0 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.835+0000 7fd505765640 1 -- 172.21.15.1:0/3006902356 reap_dead start 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.835+0000 7fd505765640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009ac0 tx=0x7fd4e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e999d9c540ffbf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.835+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0009690 con 0x7fd4e80051c0 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.835+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f0004020 con 0x7fd4e80051c0 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.835+0000 7fd4ff7fe640 1 -- 172.21.15.1:0/3006902356 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00043d0 con 0x7fd4e80051c0 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 -- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 msgr2=0x7fd4e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009ac0 tx=0x7fd4e8005830 comp rx=0 tx=0).stop 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 -- 172.21.15.1:0/3006902356 shutdown_connections 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 --2- 172.21.15.1:0/3006902356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4e80051c0 0x7fd4e800b9c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 -- 172.21.15.1:0/3006902356 >> 172.21.15.1:0/3006902356 conn(0x7fd5000cc4b0 msgr2=0x7fd5000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 -- 172.21.15.1:0/3006902356 shutdown_connections 2024-05-09T06:32:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.836+0000 7fd5071c8640 1 -- 172.21.15.1:0/3006902356 wait complete. 2024-05-09T06:32:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.837+0000 7fd5071c8640 1 Processor -- start 2024-05-09T06:32:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.837+0000 7fd5071c8640 1 -- start start 2024-05-09T06:32:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e80051c0 0x7fd5001557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd50013be80 0x7fd50013c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5001447c0 con 0x7fd5000ad890 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd500070ed0 con 0x7fd4e80051c0 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd5071c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd500144940 con 0x7fd50013be80 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd505765640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd505765640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52628/0 (socket says 172.21.15.1:52628) 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd505765640 1 -- 172.21.15.1:0/960184268 learned_addr learned my addr 172.21.15.1:0/960184268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd4fffff640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd50013be80 0x7fd50013c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.838+0000 7fd504f64640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e80051c0 0x7fd5001557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 -- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd50013be80 msgr2=0x7fd50013c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd50013be80 0x7fd50013c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 -- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e80051c0 msgr2=0x7fd5001557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e80051c0 0x7fd5001557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 -- 172.21.15.1:0/960184268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5000b55c0 con 0x7fd5000ad890 2024-05-09T06:32:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd505765640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002abe0 tx=0x7fd4f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f003b410 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f0040070 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5000ae650 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f003b610 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.839+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5000aee70 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.840+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5000752b0 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.841+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4f003b7b0 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.841+0000 7fd4fd7fa640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 0x7fd4ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.841+0000 7fd504f64640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 0x7fd4ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.841+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fd4f00aac20 con 0x7fd5000ad890 2024-05-09T06:32:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.842+0000 7fd504f64640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 0x7fd4ec06e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd5000aecd0 tx=0x7fd4f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:14.846+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4f0049020 con 0x7fd5000ad890 2024-05-09T06:32:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.028+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5000afc90 con 0x7fd5000ad890 2024-05-09T06:32:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.029+0000 7fd4fd7fa640 1 -- 172.21.15.1:0/960184268 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16813 (secure 0 0 0) 0x7fd4f0078e80 con 0x7fd5000ad890 2024-05-09T06:32:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:15.011 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:06.420655+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 msgr2=0x7fd4ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 0x7fd4ec06e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd5000aecd0 tx=0x7fd4f8009040 comp rx=0 tx=0).stop 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 msgr2=0x7fd5000add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002abe0 tx=0x7fd4f0004730 comp rx=0 tx=0).stop 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 shutdown_connections 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd4ec06be60 0x7fd4ec06e320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5000ad890 0x7fd5000add40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd50013be80 0x7fd50013c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 --2- 172.21.15.1:0/960184268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4e80051c0 0x7fd5001557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.031+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 >> 172.21.15.1:0/960184268 conn(0x7fd5000cc4b0 msgr2=0x7fd5000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.032+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 shutdown_connections 2024-05-09T06:32:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.032+0000 7fd5071c8640 1 -- 172.21.15.1:0/960184268 wait complete. 2024-05-09T06:32:15.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:32:15.024 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:32:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.150+0000 7f510ee8c640 1 Processor -- start 2024-05-09T06:32:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.150+0000 7f510ee8c640 1 -- start start 2024-05-09T06:32:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510ee8c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5108158a50 0x7f51080718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510ee8c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510815b420 con 0x7f510815c4a0 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510ee8c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510815b6e0 con 0x7f510806ede0 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510ee8c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5108158e30 con 0x7f5108158a50 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510cc28640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5108158a50 0x7f51080718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f5107fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f510815c4a0 0x7f510806e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55310/0 (socket says 172.21.15.1:55310) 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f5107fff640 1 -- 172.21.15.1:0/3767414800 learned_addr learned my addr 172.21.15.1:0/3767414800 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510cc28640 1 -- 172.21.15.1:0/3767414800 >> v1:172.21.15.6:6789/0 conn(0x7f510806ede0 legacy=0x7f510815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510cc28640 1 -- 172.21.15.1:0/3767414800 >> v1:172.21.15.1:6789/0 conn(0x7f510815c4a0 legacy=0x7f510806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.151+0000 7f510cc28640 1 -- 172.21.15.1:0/3767414800 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510815b9c0 con 0x7f5108158a50 2024-05-09T06:32:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.152+0000 7f510cc28640 1 --2- 172.21.15.1:0/3767414800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f51080718c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3066f7aa2e804a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.152+0000 7f51077fe640 1 -- 172.21.15.1:0/3767414800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc037070 con 0x7f5108158a50 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.152+0000 7f51077fe640 1 -- 172.21.15.1:0/3767414800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc032440 con 0x7f5108158a50 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.152+0000 7f51077fe640 1 -- 172.21.15.1:0/3767414800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032810 con 0x7f5108158a50 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 -- 172.21.15.1:0/3767414800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 msgr2=0x7f51080718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 --2- 172.21.15.1:0/3767414800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f51080718c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02a230 comp rx=0 tx=0).stop 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 -- 172.21.15.1:0/3767414800 shutdown_connections 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 --2- 172.21.15.1:0/3767414800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f51080718c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 -- 172.21.15.1:0/3767414800 >> 172.21.15.1:0/3767414800 conn(0x7f51080cc4d0 msgr2=0x7f51080cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 -- 172.21.15.1:0/3767414800 shutdown_connections 2024-05-09T06:32:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.153+0000 7f510ee8c640 1 -- 172.21.15.1:0/3767414800 wait complete. 2024-05-09T06:32:15.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.154+0000 7f510ee8c640 1 Processor -- start 2024-05-09T06:32:15.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.154+0000 7f510ee8c640 1 -- start start 2024-05-09T06:32:15.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f510806ede0 0x7f51080b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f510815c4a0 0x7f51080b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f510815bf80 con 0x7f5108158a50 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5108072480 con 0x7f510806ede0 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f510ee8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5108070ea0 con 0x7f510815c4a0 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52646/0 (socket says 172.21.15.1:52646) 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 -- 172.21.15.1:0/564154133 learned_addr learned my addr 172.21.15.1:0/564154133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 -- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f510815c4a0 msgr2=0x7f51080b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f510815c4a0 0x7f51080b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 -- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f510806ede0 msgr2=0x7f51080b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f510806ede0 0x7f51080b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 -- 172.21.15.1:0/564154133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51080b48d0 con 0x7f5108158a50 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.155+0000 7f5107fff640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c450 tx=0x7f50f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.156+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f5108158a50 2024-05-09T06:32:15.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.156+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4005ce0 con 0x7f5108158a50 2024-05-09T06:32:15.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.156+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40072b0 con 0x7f5108158a50 2024-05-09T06:32:15.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.156+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51080b5050 con 0x7f5108158a50 2024-05-09T06:32:15.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.156+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51080b3650 con 0x7f5108158a50 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.157+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51080752b0 con 0x7f5108158a50 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.157+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50f401a070 con 0x7f5108158a50 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.158+0000 7f51057fa640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 0x7f50ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.158+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f50f408a270 con 0x7f5108158a50 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.158+0000 7f510cc28640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 0x7f50ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.158+0000 7f510cc28640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 0x7f50ec06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f50fc034000 tx=0x7f50fc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.162+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50f4058b20 con 0x7f5108158a50 2024-05-09T06:32:15.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.343+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51080cf420 con 0x7f5108158a50 2024-05-09T06:32:15.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.344+0000 7f51057fa640 1 -- 172.21.15.1:0/564154133 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f50f4058420 con 0x7f5108158a50 2024-05-09T06:32:15.325 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 msgr2=0x7f50ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 0x7f50ec06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f50fc034000 tx=0x7f50fc02c000 comp rx=0 tx=0).stop 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 msgr2=0x7f510814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c450 tx=0x7f50f400c910 comp rx=0 tx=0).stop 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 shutdown_connections 2024-05-09T06:32:15.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50ec06be30 0x7f50ec06e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f510815c4a0 0x7f51080b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5108158a50 0x7f510814b4c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 --2- 172.21.15.1:0/564154133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f510806ede0 0x7f51080b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 >> 172.21.15.1:0/564154133 conn(0x7f51080cc4d0 msgr2=0x7f51080adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 shutdown_connections 2024-05-09T06:32:15.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.346+0000 7f510ee8c640 1 -- 172.21.15.1:0/564154133 wait complete. 2024-05-09T06:32:15.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:32:15.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.464+0000 7f58a0fa2640 1 Processor -- start 2024-05-09T06:32:15.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.464+0000 7f58a0fa2640 1 -- start start 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f58a0fa2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f589c15cfb0 0x7f589c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f58a0fa2640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589c06cf40 con 0x7f589c06dd00 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f58a0fa2640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589c06d220 con 0x7f589c15c4c0 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f58a0fa2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f589c156980 con 0x7f589c15cfb0 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f589ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f589c15cfb0 0x7f589c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f5899d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f589c06dd00 0x7f589c15c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55318/0 (socket says 172.21.15.1:55318) 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f5899d9b640 1 -- 172.21.15.1:0/1983269703 learned_addr learned my addr 172.21.15.1:0/1983269703 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f589ad9d640 1 -- 172.21.15.1:0/1983269703 >> v1:172.21.15.6:6789/0 conn(0x7f589c15c4c0 legacy=0x7f589c0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f589ad9d640 1 -- 172.21.15.1:0/1983269703 >> v1:172.21.15.1:6789/0 conn(0x7f589c06dd00 legacy=0x7f589c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f589959a640 1 -- 172.21.15.1:0/1983269703 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939753532 0 0) 0x7f589c06cf40 con 0x7f589c06dd00 2024-05-09T06:32:15.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.465+0000 7f589ad9d640 1 -- 172.21.15.1:0/1983269703 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c06d500 con 0x7f589c15cfb0 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.466+0000 7f589ad9d640 1 --2- 172.21.15.1:0/1983269703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c15cfb0 0x7f589c070c10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5894009d90 tx=0x7f589402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edc6c6c97353af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.466+0000 7f589959a640 1 -- 172.21.15.1:0/1983269703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894007740 con 0x7f589c15cfb0 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.466+0000 7f589959a640 1 -- 172.21.15.1:0/1983269703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58940078e0 con 0x7f589c15cfb0 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.466+0000 7f589959a640 1 -- 172.21.15.1:0/1983269703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894007cb0 con 0x7f589c15cfb0 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1983269703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c15cfb0 msgr2=0x7f589c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1983269703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c15cfb0 0x7f589c070c10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5894009d90 tx=0x7f589402a660 comp rx=0 tx=0).stop 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1983269703 shutdown_connections 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1983269703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c15cfb0 0x7f589c070c10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1983269703 >> 172.21.15.1:0/1983269703 conn(0x7f589c0cc4d0 msgr2=0x7f589c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1983269703 shutdown_connections 2024-05-09T06:32:15.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.467+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1983269703 wait complete. 2024-05-09T06:32:15.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.468+0000 7f58a0fa2640 1 Processor -- start 2024-05-09T06:32:15.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 -- start start 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f589c15c4c0 0x7f589c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f589c15cfb0 0x7f589c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f589c06d870 con 0x7f589c06dd00 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f589c06e180 con 0x7f589c15c4c0 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f58a0fa2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f589c0b2c40 con 0x7f589c15cfb0 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.469+0000 7f589a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52672/0 (socket says 172.21.15.1:52672) 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 -- 172.21.15.1:0/1434834523 learned_addr learned my addr 172.21.15.1:0/1434834523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f5899d9b640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f589c15c4c0 0x7f589c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589ad9d640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f589c15cfb0 0x7f589c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 -- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f589c15cfb0 msgr2=0x7f589c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f589c15cfb0 0x7f589c0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 -- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f589c15c4c0 msgr2=0x7f589c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f589c15c4c0 0x7f589c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 -- 172.21.15.1:0/1434834523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c0aec10 con 0x7f589c06dd00 2024-05-09T06:32:15.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.470+0000 7f589a59c640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c480 tx=0x7f588c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.472+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0052d0 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.472+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c005470 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.472+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c0aeef0 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.472+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c012870 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.472+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c0ae790 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.473+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c0752b0 con 0x7f589c06dd00 2024-05-09T06:32:15.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.473+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f588c012a10 con 0x7f589c06dd00 2024-05-09T06:32:15.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.474+0000 7f588affd640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 0x7f587c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.474+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f588c08aa50 con 0x7f589c06dd00 2024-05-09T06:32:15.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.474+0000 7f5899d9b640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 0x7f587c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.474+0000 7f5899d9b640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 0x7f587c06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5890007f70 tx=0x7f5890007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.478+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f588c058cb0 con 0x7f589c06dd00 2024-05-09T06:32:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.664+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f589c0cc230 con 0x7f589c06dd00 2024-05-09T06:32:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.665+0000 7f588affd640 1 -- 172.21.15.1:0/1434834523 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f588c05cfb0 con 0x7f589c06dd00 2024-05-09T06:32:15.646 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:32:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 msgr2=0x7f587c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 0x7f587c06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5890007f70 tx=0x7f5890007f00 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 msgr2=0x7f589c0ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c480 tx=0x7f588c00c940 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 shutdown_connections 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f587c06be30 0x7f587c06e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f589c15cfb0 0x7f589c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f589c15c4c0 0x7f589c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 --2- 172.21.15.1:0/1434834523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f589c06dd00 0x7f589c0ae250 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 >> 172.21.15.1:0/1434834523 conn(0x7f589c0cc4d0 msgr2=0x7f589c0b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 shutdown_connections 2024-05-09T06:32:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.667+0000 7f58a0fa2640 1 -- 172.21.15.1:0/1434834523 wait complete. 2024-05-09T06:32:15.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:32:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.785+0000 7f032d514640 1 Processor -- start 2024-05-09T06:32:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.785+0000 7f032d514640 1 -- start start 2024-05-09T06:32:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f032d514640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f032815e190 0x7f032815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f032d514640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f03280d45f0 0x7f03280b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f032d514640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0328161fe0 con 0x7f03280d1c70 2024-05-09T06:32:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f032d514640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f032815f580 con 0x7f032815e190 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f032d514640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f03281622c0 con 0x7f03280d45f0 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f032815e190 0x7f032815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f032815e190 0x7f032815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52674/0 (socket says 172.21.15.1:52674) 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 -- 172.21.15.1:0/2272034938 learned_addr learned my addr 172.21.15.1:0/2272034938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f03277fe640 1 --2- 172.21.15.1:0/2272034938 >> v2:172.21.15.6:3300/0 conn(0x7f03280d45f0 0x7f03280b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 -- 172.21.15.1:0/2272034938 >> v2:172.21.15.6:3300/0 conn(0x7f03280d45f0 msgr2=0x7f03280b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 --2- 172.21.15.1:0/2272034938 >> v2:172.21.15.6:3300/0 conn(0x7f03280d45f0 0x7f03280b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 -- 172.21.15.1:0/2272034938 >> v1:172.21.15.6:6789/0 conn(0x7f03280d1c70 legacy=0x7f03280d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.786+0000 7f0326ffd640 1 -- 172.21.15.1:0/2272034938 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281625a0 con 0x7f032815e190 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.787+0000 7f0326ffd640 1 --2- 172.21.15.1:0/2272034938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f032815f780 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f03280ce7f0 tx=0x7f031802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c570ba1941d3f13d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.787+0000 7f0325ffb640 1 -- 172.21.15.1:0/2272034938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318037070 con 0x7f032815e190 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.787+0000 7f0325ffb640 1 -- 172.21.15.1:0/2272034938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318032440 con 0x7f032815e190 2024-05-09T06:32:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.787+0000 7f0325ffb640 1 -- 172.21.15.1:0/2272034938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318032810 con 0x7f032815e190 2024-05-09T06:32:15.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 -- 172.21.15.1:0/2272034938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 msgr2=0x7f032815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 --2- 172.21.15.1:0/2272034938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f032815f780 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f03280ce7f0 tx=0x7f031802a270 comp rx=0 tx=0).stop 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 -- 172.21.15.1:0/2272034938 shutdown_connections 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 --2- 172.21.15.1:0/2272034938 >> v2:172.21.15.6:3300/0 conn(0x7f03280d45f0 0x7f03280b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 --2- 172.21.15.1:0/2272034938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f032815f780 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 -- 172.21.15.1:0/2272034938 >> 172.21.15.1:0/2272034938 conn(0x7f0328073340 msgr2=0x7f032809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 -- 172.21.15.1:0/2272034938 shutdown_connections 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.788+0000 7f032d514640 1 -- 172.21.15.1:0/2272034938 wait complete. 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.789+0000 7f032d514640 1 Processor -- start 2024-05-09T06:32:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.789+0000 7f032d514640 1 -- start start 2024-05-09T06:32:15.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03280d1c70 0x7f03280c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03280d45f0 0x7f0328152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03280b56d0 con 0x7f032815e190 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0328161c60 con 0x7f03280d45f0 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f032d514640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f03280b5850 con 0x7f03280d1c70 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52688/0 (socket says 172.21.15.1:52688) 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 -- 172.21.15.1:0/2215736073 learned_addr learned my addr 172.21.15.1:0/2215736073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f0326ffd640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03280d1c70 0x7f03280c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03267fc640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03280d45f0 0x7f0328152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 -- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03280d1c70 msgr2=0x7f03280c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03280d1c70 0x7f03280c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 -- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03280d45f0 msgr2=0x7f0328152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03280d45f0 0x7f0328152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.790+0000 7f03277fe640 1 -- 172.21.15.1:0/2215736073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280cc850 con 0x7f032815e190 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f03277fe640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0320007e30 tx=0x7f032000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320014620 con 0x7f032815e190 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0320019070 con 0x7f032815e190 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320014a00 con 0x7f032815e190 2024-05-09T06:32:15.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03280b1a70 con 0x7f032815e190 2024-05-09T06:32:15.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.791+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0328153020 con 0x7f032815e190 2024-05-09T06:32:15.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.792+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03280752b0 con 0x7f032815e190 2024-05-09T06:32:15.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.792+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0320014ba0 con 0x7f032815e190 2024-05-09T06:32:15.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.793+0000 7f030f7fe640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:15.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.793+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f032008b710 con 0x7f032815e190 2024-05-09T06:32:15.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.793+0000 7f0326ffd640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:15.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.793+0000 7f0326ffd640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0318034000 tx=0x7f031802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.797+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0320059970 con 0x7f032815e190 2024-05-09T06:32:15.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.980+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03280d43a0 con 0x7f032815e190 2024-05-09T06:32:15.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.980+0000 7f030f7fe640 1 -- 172.21.15.1:0/2215736073 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f032005dc70 con 0x7f032815e190 2024-05-09T06:32:15.962 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 msgr2=0x7f031406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0318034000 tx=0x7f031802e000 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 msgr2=0x7f03280cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0320007e30 tx=0x7f032000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 shutdown_connections 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f032815e190 0x7f03280cbd40 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03280d45f0 0x7f0328152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 --2- 172.21.15.1:0/2215736073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03280d1c70 0x7f03280c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 >> 172.21.15.1:0/2215736073 conn(0x7f0328073340 msgr2=0x7f03280b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 shutdown_connections 2024-05-09T06:32:15.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:15.982+0000 7f032d514640 1 -- 172.21.15.1:0/2215736073 wait complete. 2024-05-09T06:32:15.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:32:16.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.098+0000 7fb484012640 1 Processor -- start 2024-05-09T06:32:16.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.098+0000 7fb484012640 1 -- start start 2024-05-09T06:32:16.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 0x7fb47c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 0x7fb47c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb47c0b4da0 0x7fb47c15f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c0b5230 con 0x7fb47c0b4da0 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c06eb40 con 0x7fb47c0b2a60 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb484012640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c06ee20 con 0x7fb47c1454b0 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb47c0b4da0 0x7fb47c15f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb47c0b4da0 0x7fb47c15f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52696/0 (socket says 172.21.15.1:52696) 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 -- 172.21.15.1:0/3213651598 learned_addr learned my addr 172.21.15.1:0/3213651598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb481dae640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 0x7fb47c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4815ad640 1 --2- 172.21.15.1:0/3213651598 >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 0x7fb47c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 -- 172.21.15.1:0/3213651598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 msgr2=0x7fb47c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 0x7fb47c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 -- 172.21.15.1:0/3213651598 >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 msgr2=0x7fb47c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 --2- 172.21.15.1:0/3213651598 >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 0x7fb47c0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4825af640 1 -- 172.21.15.1:0/3213651598 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c06d8e0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb481dae640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 0x7fb47c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.099+0000 7fb4815ad640 1 --2- 172.21.15.1:0/3213651598 >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 0x7fb47c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.100+0000 7fb4825af640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c15f960 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0731c0 tx=0x7fb47402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8589fbf035fa415 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.100+0000 7fb480dac640 1 -- 172.21.15.1:0/3213651598 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474036070 con 0x7fb47c0b4da0 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.100+0000 7fb480dac640 1 -- 172.21.15.1:0/3213651598 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb474031600 con 0x7fb47c0b4da0 2024-05-09T06:32:16.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.100+0000 7fb480dac640 1 -- 172.21.15.1:0/3213651598 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740319d0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 -- 172.21.15.1:0/3213651598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 msgr2=0x7fb47c15f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:16.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c15f960 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0731c0 tx=0x7fb47402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 -- 172.21.15.1:0/3213651598 shutdown_connections 2024-05-09T06:32:16.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c15f960 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 --2- 172.21.15.1:0/3213651598 >> v2:172.21.15.6:3300/0 conn(0x7fb47c0b2a60 0x7fb47c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 --2- 172.21.15.1:0/3213651598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c1454b0 0x7fb47c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 -- 172.21.15.1:0/3213651598 >> 172.21.15.1:0/3213651598 conn(0x7fb47c0cc4b0 msgr2=0x7fb47c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 -- 172.21.15.1:0/3213651598 shutdown_connections 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.101+0000 7fb484012640 1 -- 172.21.15.1:0/3213651598 wait complete. 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.102+0000 7fb484012640 1 Processor -- start 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.102+0000 7fb484012640 1 -- start start 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c0b2a60 0x7fb47c1e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb47c1454b0 0x7fb47c0aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c06e0e0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c145890 con 0x7fb47c1454b0 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb484012640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c06e260 con 0x7fb47c0b2a60 2024-05-09T06:32:16.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52702/0 (socket says 172.21.15.1:52702) 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 -- 172.21.15.1:0/1852327985 learned_addr learned my addr 172.21.15.1:0/1852327985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb481dae640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c0b2a60 0x7fb47c1e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4825af640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb47c1454b0 0x7fb47c0aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 -- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c0b2a60 msgr2=0x7fb47c1e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c0b2a60 0x7fb47c1e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 -- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb47c1454b0 msgr2=0x7fb47c0aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb47c1454b0 0x7fb47c0aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 -- 172.21.15.1:0/1852327985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c0af2e0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4815ad640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d7d0 tx=0x7fb46c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004030 con 0x7fb47c0b4da0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.103+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb46c0041d0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.104+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c0a9db0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.104+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00b6c0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.104+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c0aa370 con 0x7fb47c0b4da0 2024-05-09T06:32:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.104+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c0aab30 con 0x7fb47c0b4da0 2024-05-09T06:32:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.105+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb46c00b860 con 0x7fb47c0b4da0 2024-05-09T06:32:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.105+0000 7fb4727fc640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.106+0000 7fb481dae640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.106+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fb46c08b210 con 0x7fb47c0b4da0 2024-05-09T06:32:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.106+0000 7fb481dae640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 0x7fb46006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb478006fd0 tx=0x7fb478008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.110+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb47c0aab30 con 0x7fb47c0b4da0 2024-05-09T06:32:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.307+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb47c0752b0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.307+0000 7fb4727fc640 1 -- 172.21.15.1:0/1852327985 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7fb46c059ac0 con 0x7fb47c0b4da0 2024-05-09T06:32:16.289 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 msgr2=0x7fb46006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 0x7fb46006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb478006fd0 tx=0x7fb478008040 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 msgr2=0x7fb47c0acce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d7d0 tx=0x7fb46c00dc90 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 shutdown_connections 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb46006be30 0x7fb46006e2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb47c1454b0 0x7fb47c0aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47c0b4da0 0x7fb47c0acce0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 --2- 172.21.15.1:0/1852327985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb47c0b2a60 0x7fb47c1e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 >> 172.21.15.1:0/1852327985 conn(0x7fb47c0cc4b0 msgr2=0x7fb47c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 shutdown_connections 2024-05-09T06:32:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:16.309+0000 7fb484012640 1 -- 172.21.15.1:0/1852327985 wait complete. 2024-05-09T06:32:16.301 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:32:16.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:16.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:16.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[62457]: fuse finished with error 0 and tester_r 0 2024-05-09T06:32:16.327 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:32:18.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:32:18.188 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:32:22.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:32:22.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:22.430 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:32:22.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.624+0000 7fd77071b640 1 Processor -- start 2024-05-09T06:32:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.624+0000 7fd77071b640 1 -- start start 2024-05-09T06:32:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.625+0000 7fd77071b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7681617a0 0x7fd7680b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.625+0000 7fd77071b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.625+0000 7fd77071b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7680d0870 con 0x7fd7680b2dd0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd77071b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd7680d1480 con 0x7fd7681617a0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd77071b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7680d0b50 con 0x7fd7680d28e0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76e4b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7681617a0 0x7fd7680b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76dcb6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd7680b2dd0 0x7fd7680d21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35424/0 (socket says 172.21.15.1:35424) 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76dcb6640 1 -- 172.21.15.1:0/2142071866 learned_addr learned my addr 172.21.15.1:0/2142071866 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76ecb8640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76e4b7640 1 -- 172.21.15.1:0/2142071866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 msgr2=0x7fd7680ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76e4b7640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76e4b7640 1 -- 172.21.15.1:0/2142071866 >> v1:172.21.15.1:6789/0 conn(0x7fd7680b2dd0 legacy=0x7fd7680d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76e4b7640 1 -- 172.21.15.1:0/2142071866 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7680d0e30 con 0x7fd7681617a0 2024-05-09T06:32:22.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.626+0000 7fd76d4b5640 1 -- 172.21.15.1:0/2142071866 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551501219 0 0) 0x7fd7680d0870 con 0x7fd7680b2dd0 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.627+0000 7fd76e4b7640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd7680b5010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097a0 tx=0x7fd76402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aa1822cd884ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.627+0000 7fd76d4b5640 1 -- 172.21.15.1:0/2142071866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764037070 con 0x7fd7681617a0 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.627+0000 7fd76d4b5640 1 -- 172.21.15.1:0/2142071866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd764032440 con 0x7fd7681617a0 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.627+0000 7fd76d4b5640 1 -- 172.21.15.1:0/2142071866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764032810 con 0x7fd7681617a0 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.627+0000 7fd77071b640 1 -- 172.21.15.1:0/2142071866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 msgr2=0x7fd7680b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd7680b5010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097a0 tx=0x7fd76402a180 comp rx=0 tx=0).stop 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 -- 172.21.15.1:0/2142071866 shutdown_connections 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 --2- 172.21.15.1:0/2142071866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd7680b5010 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 -- 172.21.15.1:0/2142071866 >> 172.21.15.1:0/2142071866 conn(0x7fd76809d180 msgr2=0x7fd7680d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:22.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 -- 172.21.15.1:0/2142071866 shutdown_connections 2024-05-09T06:32:22.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.628+0000 7fd77071b640 1 -- 172.21.15.1:0/2142071866 wait complete. 2024-05-09T06:32:22.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.629+0000 7fd77071b640 1 Processor -- start 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.629+0000 7fd77071b640 1 -- start start 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7680b2dd0 0x7fd7680cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd768161d00 con 0x7fd7681617a0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd768161b80 con 0x7fd7680b2dd0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd77071b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7680b5550 con 0x7fd7680d28e0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55820/0 (socket says 172.21.15.1:55820) 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 -- 172.21.15.1:0/2495778658 learned_addr learned my addr 172.21.15.1:0/2495778658 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76dcb6640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76e4b7640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7680b2dd0 0x7fd7680cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 -- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 msgr2=0x7fd7680b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 -- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7680b2dd0 msgr2=0x7fd7680cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7680b2dd0 0x7fd7680cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.630+0000 7fd76ecb8640 1 -- 172.21.15.1:0/2495778658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7680b7c40 con 0x7fd7681617a0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd76ecb8640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd76000cd40 tx=0x7fd76000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760003b90 con 0x7fd7681617a0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd760003d30 con 0x7fd7681617a0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7680b7f20 con 0x7fd7681617a0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760004240 con 0x7fd7681617a0 2024-05-09T06:32:22.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.631+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7680b84e0 con 0x7fd7681617a0 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.632+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7680752b0 con 0x7fd7681617a0 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.633+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7600043e0 con 0x7fd7681617a0 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.633+0000 7fd75effd640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.633+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fd76008b960 con 0x7fd7681617a0 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.633+0000 7fd76e4b7640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:22.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.634+0000 7fd76e4b7640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 0x7fd74c06e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd764034000 tx=0x7fd76402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.638+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd760059bc0 con 0x7fd7681617a0 2024-05-09T06:32:22.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.863+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd76809d6d0 con 0x7fd7681617a0 2024-05-09T06:32:22.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.864+0000 7fd75effd640 1 -- 172.21.15.1:0/2495778658 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5168 (secure 0 0 0) 0x7fd76005dec0 con 0x7fd7681617a0 2024-05-09T06:32:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:22.845 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:07.811817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6834/2073487547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2073487547},{"type":"v1","addr":"172.21.15.6:6834","nonce":2073487547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7125,"qdb_cluster":[7125]},"id":5}]} 2024-05-09T06:32:22.845 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 45 2024-05-09T06:32:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 msgr2=0x7fd74c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 0x7fd74c06e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd764034000 tx=0x7fd76402e000 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 msgr2=0x7fd76815a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd76000cd40 tx=0x7fd76000a660 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 shutdown_connections 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd74c06be30 0x7fd74c06e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7681617a0 0x7fd76815a1b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7680d28e0 0x7fd7680b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 --2- 172.21.15.1:0/2495778658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7680b2dd0 0x7fd7680cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 >> 172.21.15.1:0/2495778658 conn(0x7fd76809d180 msgr2=0x7fd7680b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 shutdown_connections 2024-05-09T06:32:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:22.866+0000 7fd77071b640 1 -- 172.21.15.1:0/2495778658 wait complete. 2024-05-09T06:32:22.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-09T06:32:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2024-05-09T06:32:23.003 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:32:23.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.127+0000 7fe126864640 1 Processor -- start 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.127+0000 7fe126864640 1 -- start start 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe126864640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1200b2aa0 0x7fe12015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe126864640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe120099930 con 0x7fe1200c6860 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe126864640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe120162930 con 0x7fe120099b30 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe126864640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe1200b1e20 con 0x7fe1200b2aa0 2024-05-09T06:32:23.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe11ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe1200c6860 0x7fe1200c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35432/0 (socket says 172.21.15.1:35432) 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe11ffff640 1 -- 172.21.15.1:0/2057876973 learned_addr learned my addr 172.21.15.1:0/2057876973 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.128+0000 7fe124e01640 1 --2- 172.21.15.1:0/2057876973 >> v2:172.21.15.1:3300/0 conn(0x7fe1200b2aa0 0x7fe12015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3874787374 0 0) 0x7fe110001110 con 0x7fe1200c6860 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe10c003520 con 0x7fe1200c6860 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123422767 0 0) 0x7fe120162930 con 0x7fe120099b30 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe110001110 con 0x7fe120099b30 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe124e01640 1 -- 172.21.15.1:0/2057876973 >> v1:172.21.15.6:6789/0 conn(0x7fe120099b30 legacy=0x7fe12015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe124e01640 1 -- 172.21.15.1:0/2057876973 >> v1:172.21.15.1:6789/0 conn(0x7fe1200c6860 legacy=0x7fe1200c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe124e01640 1 -- 172.21.15.1:0/2057876973 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe120162c10 con 0x7fe1200b2aa0 2024-05-09T06:32:23.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.129+0000 7fe124e01640 1 --2- 172.21.15.1:0/2057876973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe12015e4b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe118009d90 tx=0x7fe11802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9d5e18aaa77417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118007740 con 0x7fe1200b2aa0 2024-05-09T06:32:23.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1180078e0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe11f7fe640 1 -- 172.21.15.1:0/2057876973 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118007c90 con 0x7fe1200b2aa0 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 -- 172.21.15.1:0/2057876973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 msgr2=0x7fe12015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 --2- 172.21.15.1:0/2057876973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe12015e4b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe118009d90 tx=0x7fe11802a660 comp rx=0 tx=0).stop 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 -- 172.21.15.1:0/2057876973 shutdown_connections 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 --2- 172.21.15.1:0/2057876973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe12015e4b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 -- 172.21.15.1:0/2057876973 >> 172.21.15.1:0/2057876973 conn(0x7fe1200b1fb0 msgr2=0x7fe1200b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 -- 172.21.15.1:0/2057876973 shutdown_connections 2024-05-09T06:32:23.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.130+0000 7fe126864640 1 -- 172.21.15.1:0/2057876973 wait complete. 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 Processor -- start 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 -- start start 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe120099b30 0x7fe1200cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1200c6860 0x7fe1201ea070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe12015fa30 con 0x7fe1200b2aa0 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe12015ee90 con 0x7fe1200c6860 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe126864640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe1200cdf50 con 0x7fe120099b30 2024-05-09T06:32:23.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.132+0000 7fe117fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55838/0 (socket says 172.21.15.1:55838) 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 -- 172.21.15.1:0/4149297984 learned_addr learned my addr 172.21.15.1:0/4149297984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe124e01640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1200c6860 0x7fe1201ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe11ffff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe120099b30 0x7fe1200cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 -- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe120099b30 msgr2=0x7fe1200cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe120099b30 0x7fe1200cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 -- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1200c6860 msgr2=0x7fe1201ea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1200c6860 0x7fe1201ea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 -- 172.21.15.1:0/4149297984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1200cd2a0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe117fff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c500 tx=0x7fe10800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080052d0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108005490 con 0x7fe1200b2aa0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108012870 con 0x7fe1200b2aa0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1200ccfc0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.133+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1200cae30 con 0x7fe1200b2aa0 2024-05-09T06:32:23.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.134+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1200752b0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.135+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe108012a10 con 0x7fe1200b2aa0 2024-05-09T06:32:23.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.135+0000 7fe11d7fa640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 0x7fe10006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.135+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fe10808add0 con 0x7fe1200b2aa0 2024-05-09T06:32:23.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.135+0000 7fe11ffff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 0x7fe10006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.136+0000 7fe11ffff640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 0x7fe10006e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe1100013b0 tx=0x7fe110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.140+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe108059560 con 0x7fe1200b2aa0 2024-05-09T06:32:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.360+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe12006ce30 con 0x7fe1200b2aa0 2024-05-09T06:32:23.342 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:32:23.361+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:32:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.362+0000 7fe11d7fa640 1 -- 172.21.15.1:0/4149297984 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fe108058e60 con 0x7fe1200b2aa0 2024-05-09T06:32:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 msgr2=0x7fe10006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 0x7fe10006e2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe1100013b0 tx=0x7fe110009040 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 msgr2=0x7fe1200cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c500 tx=0x7fe10800c9c0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 shutdown_connections 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe10006be30 0x7fe10006e2f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1200c6860 0x7fe1201ea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1200b2aa0 0x7fe1200cc670 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 --2- 172.21.15.1:0/4149297984 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe120099b30 0x7fe1200cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 >> 172.21.15.1:0/4149297984 conn(0x7fe1200b1fb0 msgr2=0x7fe1200c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 shutdown_connections 2024-05-09T06:32:23.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.365+0000 7fe126864640 1 -- 172.21.15.1:0/4149297984 wait complete. 2024-05-09T06:32:23.348 INFO:tasks.ceph.mds.d.smithi006.stderr: -3498> 2024-05-09T06:32:10.411+0000 7fc86d991640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:32:23.358 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-09T06:32:23.379 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:32:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.422+0000 7f09d1f35f40 1 Processor -- start 2024-05-09T06:32:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.422+0000 7f09d1f35f40 1 -- start start 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8712dd0 0x55a3e8711e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8741bc0 0x55a3e87ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3e87d1dd0 con 0x55a3e87ce850 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a3e86bbc80 con 0x55a3e8712dd0 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a3e87ebe90 con 0x55a3e8741bc0 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55858/0 (socket says 172.21.15.1:55858) 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 -- 172.21.15.1:0/251104171 learned_addr learned my addr 172.21.15.1:0/251104171 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 -- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8741bc0 msgr2=0x55a3e87ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8741bc0 0x55a3e87ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 -- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8712dd0 msgr2=0x55a3e8711e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8712dd0 0x55a3e8711e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.423+0000 7f09d0237640 1 -- 172.21.15.1:0/251104171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e8640cb0 con 0x55a3e87ce850 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d0237640 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f09c8025be0 tx=0x7f09c804e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88af902f5c30de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09ce233640 1 -- 172.21.15.1:0/251104171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8065070 con 0x55a3e87ce850 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09ce233640 1 -- 172.21.15.1:0/251104171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c80549f0 con 0x55a3e87ce850 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09ce233640 1 -- 172.21.15.1:0/251104171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8059040 con 0x55a3e87ce850 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 -- 172.21.15.1:0/251104171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 msgr2=0x55a3e87eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f09c8025be0 tx=0x7f09c804e070 comp rx=0 tx=0).stop 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 -- 172.21.15.1:0/251104171 shutdown_connections 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e87eeac0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8741bc0 0x55a3e87ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/251104171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8712dd0 0x55a3e8711e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 -- 172.21.15.1:0/251104171 >> 172.21.15.1:0/251104171 conn(0x55a3e87407c0 msgr2=0x55a3e87eafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 -- 172.21.15.1:0/251104171 shutdown_connections 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.424+0000 7f09d1f35f40 1 -- 172.21.15.1:0/251104171 wait complete. 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.425+0000 7f09d1f35f40 1 Processor -- start 2024-05-09T06:32:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.425+0000 7f09d1f35f40 1 -- start start 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8712dd0 0x55a3e8743960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8741bc0 0x55a3e8758030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e8885660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3e8740d80 con 0x55a3e87ce850 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a3e875d710 con 0x55a3e8741bc0 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a3e875d9f0 con 0x55a3e8712dd0 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e8885660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e8885660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55872/0 (socket says 172.21.15.1:55872) 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 -- 172.21.15.1:0/595639815 learned_addr learned my addr 172.21.15.1:0/595639815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:23.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cf235640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8741bc0 0x55a3e8758030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cfa36640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8712dd0 0x55a3e8743960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 -- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8712dd0 msgr2=0x55a3e8743960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8712dd0 0x55a3e8743960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 -- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8741bc0 msgr2=0x55a3e8758030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8741bc0 0x55a3e8758030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 -- 172.21.15.1:0/595639815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e87e5150 con 0x55a3e87ce850 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cf235640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3e8741bc0 0x55a3e8758030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cfa36640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3e8712dd0 0x55a3e8743960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d0237640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e8885660 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f09c8045e10 tx=0x7f09c8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cd231640 1 -- 172.21.15.1:0/595639815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c806f020 con 0x55a3e87ce850 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cd231640 1 -- 172.21.15.1:0/595639815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c806b470 con 0x55a3e87ce850 2024-05-09T06:32:23.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a3e87422d0 con 0x55a3e87ce850 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.426+0000 7f09cd231640 1 -- 172.21.15.1:0/595639815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8065040 con 0x55a3e87ce850 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.427+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e87e53c0 con 0x55a3e87ce850 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.428+0000 7f09cd231640 1 -- 172.21.15.1:0/595639815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09c8072020 con 0x55a3e87ce850 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.428+0000 7f09cd231640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006a840 0x7f09b006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09cd231640 1 -- 172.21.15.1:0/595639815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f09c80cf040 con 0x55a3e87ce850 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09cfa36640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006a840 0x7f09b006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a3e8758570 0x55a3e87cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) -- 0x55a3e87e5c10 con 0x55a3e8758570 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09cf235640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a3e8758570 0x55a3e87cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09cfa36640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006a840 0x7f09b006cd00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003420 tx=0x7f09c0007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.429+0000 7f09cf235640 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a3e8758570 0x55a3e87cfcd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09cf235640 1 -- 172.21.15.1:0/595639815 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f09b800c3e0 con 0x55a3e8758570 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a3e8758570 msgr2=0x55a3e87cfcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a3e8758570 0x55a3e87cfcd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006a840 msgr2=0x7f09b006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f09b006a840 0x7f09b006cd00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003420 tx=0x7f09c0007040 comp rx=0 tx=0).stop 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 msgr2=0x55a3e8885660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.432+0000 7f09d1f35f40 1 --2- 172.21.15.1:0/595639815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e87ce850 0x55a3e8885660 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f09c8045e10 tx=0x7f09c8002fb0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.433+0000 7f09d0237640 1 -- 172.21.15.1:0/595639815 reap_dead start 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.433+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 shutdown_connections 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.433+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 >> 172.21.15.1:0/595639815 conn(0x55a3e87407c0 msgr2=0x55a3e87eb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.433+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 shutdown_connections 2024-05-09T06:32:23.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.433+0000 7f09d1f35f40 1 -- 172.21.15.1:0/595639815 wait complete. 2024-05-09T06:32:23.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:32:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.542+0000 7f6d7a5db640 1 Processor -- start 2024-05-09T06:32:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.542+0000 7f6d7a5db640 1 -- start start 2024-05-09T06:32:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d7a5db640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d74158a50 0x7f6d740718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d7a5db640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6d7406ede0 0x7f6d74071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d7a5db640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d7415b420 con 0x7f6d7415c4a0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d7a5db640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6d74158e30 con 0x7f6d74158a50 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d7a5db640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6d74072480 con 0x7f6d7406ede0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d73fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d74158a50 0x7f6d740718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d737fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6d7415c4a0 0x7f6d7406e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35452/0 (socket says 172.21.15.1:35452) 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d737fe640 1 -- 172.21.15.1:0/3636347206 learned_addr learned my addr 172.21.15.1:0/3636347206 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d78b78640 1 --2- 172.21.15.1:0/3636347206 >> v2:172.21.15.6:3300/0 conn(0x7f6d7406ede0 0x7f6d74071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d72ffd640 1 -- 172.21.15.1:0/3636347206 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664744698 0 0) 0x7f6d7415b420 con 0x7f6d7415c4a0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d72ffd640 1 -- 172.21.15.1:0/3636347206 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d5c003520 con 0x7f6d7415c4a0 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d73fff640 1 -- 172.21.15.1:0/3636347206 >> v2:172.21.15.6:3300/0 conn(0x7f6d7406ede0 msgr2=0x7f6d74071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.543+0000 7f6d73fff640 1 --2- 172.21.15.1:0/3636347206 >> v2:172.21.15.6:3300/0 conn(0x7f6d7406ede0 0x7f6d74071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d73fff640 1 -- 172.21.15.1:0/3636347206 >> v1:172.21.15.1:6789/0 conn(0x7f6d7415c4a0 legacy=0x7f6d7406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d73fff640 1 -- 172.21.15.1:0/3636347206 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7415b6e0 con 0x7f6d74158a50 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d73fff640 1 --2- 172.21.15.1:0/3636347206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740718c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6d740741b0 tx=0x7f6d6402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7117f908755f1235 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d72ffd640 1 -- 172.21.15.1:0/3636347206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037070 con 0x7f6d74158a50 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d72ffd640 1 -- 172.21.15.1:0/3636347206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d64032440 con 0x7f6d74158a50 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.544+0000 7f6d72ffd640 1 -- 172.21.15.1:0/3636347206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64032810 con 0x7f6d74158a50 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 -- 172.21.15.1:0/3636347206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 msgr2=0x7f6d740718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/3636347206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740718c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6d740741b0 tx=0x7f6d6402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 -- 172.21.15.1:0/3636347206 shutdown_connections 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/3636347206 >> v2:172.21.15.6:3300/0 conn(0x7f6d7406ede0 0x7f6d74071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/3636347206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740718c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 -- 172.21.15.1:0/3636347206 >> 172.21.15.1:0/3636347206 conn(0x7f6d740cc4d0 msgr2=0x7f6d740cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 -- 172.21.15.1:0/3636347206 shutdown_connections 2024-05-09T06:32:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.545+0000 7f6d7a5db640 1 -- 172.21.15.1:0/3636347206 wait complete. 2024-05-09T06:32:23.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.546+0000 7f6d7a5db640 1 Processor -- start 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.546+0000 7f6d7a5db640 1 -- start start 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d7406ede0 0x7f6d74165b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d7415c4a0 0x7f6d741641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7415bf80 con 0x7f6d74158a50 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7415bca0 con 0x7f6d7406ede0 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d7a5db640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d74070ea0 con 0x7f6d7415c4a0 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d737fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d737fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55884/0 (socket says 172.21.15.1:55884) 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d737fe640 1 -- 172.21.15.1:0/1865216316 learned_addr learned my addr 172.21.15.1:0/1865216316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.547+0000 7f6d73fff640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d7406ede0 0x7f6d74165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d78b78640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d7415c4a0 0x7f6d741641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 -- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d7415c4a0 msgr2=0x7f6d741641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d7415c4a0 0x7f6d741641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 -- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d7406ede0 msgr2=0x7f6d74165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d7406ede0 0x7f6d74165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 -- 172.21.15.1:0/1865216316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d74166250 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d737fe640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800c430 tx=0x7f6d6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68017070 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d68005ce0 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d680072d0 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d74166530 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.548+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d74166af0 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.549+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d740752b0 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.550+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d6801a070 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.550+0000 7f6d70ff9640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 0x7f6d6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.550+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f6d6808a310 con 0x7f6d74158a50 2024-05-09T06:32:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.550+0000 7f6d73fff640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 0x7f6d6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.551+0000 7f6d73fff640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 0x7f6d6006e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6d64034000 tx=0x7f6d6402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:23.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.555+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d680584a0 con 0x7f6d74158a50 2024-05-09T06:32:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:23.777+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6d7415f6f0 con 0x7f6d74158a50 2024-05-09T06:32:24.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.363+0000 7f6d70ff9640 1 -- 172.21.15.1:0/1865216316 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7f6d6805c7a0 con 0x7f6d74158a50 2024-05-09T06:32:24.345 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 msgr2=0x7f6d6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 0x7f6d6006e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6d64034000 tx=0x7f6d6402e000 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 msgr2=0x7f6d740b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800c430 tx=0x7f6d6800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 shutdown_connections 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d6006be30 0x7f6d6006e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d7415c4a0 0x7f6d741641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d74158a50 0x7f6d740b41a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 --2- 172.21.15.1:0/1865216316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d7406ede0 0x7f6d74165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 >> 172.21.15.1:0/1865216316 conn(0x7f6d740cc4d0 msgr2=0x7f6d74159120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:24.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 shutdown_connections 2024-05-09T06:32:24.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.366+0000 7f6d7a5db640 1 -- 172.21.15.1:0/1865216316 wait complete. 2024-05-09T06:32:24.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:24.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.489+0000 7f805c293640 1 Processor -- start 2024-05-09T06:32:24.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.489+0000 7f805c293640 1 -- start start 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805c293640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8054161950 0x7f80540d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805c293640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f80540d4940 0x7f80540d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805c293640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80540b33c0 con 0x7f80540d0f80 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805c293640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f80540d4dd0 con 0x7f80540d4940 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805c293640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80540b3660 con 0x7f8054161950 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805a830640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f80540d0f80 0x7f80540cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35464/0 (socket says 172.21.15.1:35464) 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805a830640 1 -- 172.21.15.1:0/567305565 learned_addr learned my addr 172.21.15.1:0/567305565 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805a02f640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8054161950 0x7f80540d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805982e640 1 --2- 172.21.15.1:0/567305565 >> v2:172.21.15.6:3300/0 conn(0x7f80540d4940 0x7f80540d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805902d640 1 -- 172.21.15.1:0/567305565 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030341287 0 0) 0x7f80540b33c0 con 0x7f80540d0f80 2024-05-09T06:32:24.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805902d640 1 -- 172.21.15.1:0/567305565 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8040003520 con 0x7f80540d0f80 2024-05-09T06:32:24.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805982e640 1 -- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8054161950 msgr2=0x7f80540d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:24.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.490+0000 7f805982e640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8054161950 0x7f80540d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.491+0000 7f805982e640 1 -- 172.21.15.1:0/567305565 >> v1:172.21.15.1:6789/0 conn(0x7f80540d0f80 legacy=0x7f80540cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.491+0000 7f805982e640 1 -- 172.21.15.1:0/567305565 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80540b3940 con 0x7f80540d4940 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.491+0000 7f805982e640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540d4940 0x7f80540d2940 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f80540b3590 tx=0x7f804402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6d25afddd84e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.491+0000 7f805902d640 1 -- 172.21.15.1:0/567305565 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044037070 con 0x7f80540d4940 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805902d640 1 -- 172.21.15.1:0/567305565 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8044032440 con 0x7f80540d4940 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805902d640 1 -- 172.21.15.1:0/567305565 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044032810 con 0x7f80540d4940 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 -- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540d4940 msgr2=0x7f80540d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540d4940 0x7f80540d2940 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f80540b3590 tx=0x7f804402a180 comp rx=0 tx=0).stop 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 -- 172.21.15.1:0/567305565 shutdown_connections 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540d4940 0x7f80540d2940 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 --2- 172.21.15.1:0/567305565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8054161950 0x7f80540d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 -- 172.21.15.1:0/567305565 >> 172.21.15.1:0/567305565 conn(0x7f80540d3e20 msgr2=0x7f80540d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:24.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 -- 172.21.15.1:0/567305565 shutdown_connections 2024-05-09T06:32:24.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.492+0000 7f805c293640 1 -- 172.21.15.1:0/567305565 wait complete. 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.493+0000 7f805c293640 1 Processor -- start 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.493+0000 7f805c293640 1 -- start start 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80540d0f80 0x7f80540b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 0x7f805415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80540b52d0 con 0x7f8054161950 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f80540b5150 con 0x7f80540b8450 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805c293640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80540b5450 con 0x7f80540d0f80 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55886/0 (socket says 172.21.15.1:55886) 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 -- 172.21.15.1:0/3535586707 learned_addr learned my addr 172.21.15.1:0/3535586707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805a830640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 0x7f805415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805a02f640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80540d0f80 0x7f80540b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 -- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80540d0f80 msgr2=0x7f80540b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80540d0f80 0x7f80540b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 -- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 msgr2=0x7f805415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 0x7f805415edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.494+0000 7f805982e640 1 -- 172.21.15.1:0/3535586707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80540b8ea0 con 0x7f8054161950 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f805a830640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 0x7f805415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f805982e640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f804402a1c0 tx=0x7f80440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044041030 con 0x7f8054161950 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804403d420 con 0x7f8054161950 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80540b6bb0 con 0x7f8054161950 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044037040 con 0x7f8054161950 2024-05-09T06:32:24.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.495+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80540b7bd0 con 0x7f8054161950 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.496+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80540752b0 con 0x7f8054161950 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.497+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8044044070 con 0x7f8054161950 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.497+0000 7f804affd640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 0x7f803806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.497+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f80440ab6d0 con 0x7f8054161950 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.497+0000 7f805a02f640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 0x7f803806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:24.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.497+0000 7f805a02f640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 0x7f803806e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8050009a30 tx=0x7f8050008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:24.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.501+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8044079eb0 con 0x7f8054161950 2024-05-09T06:32:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.724+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80540d21e0 con 0x7f8054161950 2024-05-09T06:32:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.725+0000 7f804affd640 1 -- 172.21.15.1:0/3535586707 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5162 (secure 0 0 0) 0x7f80440797b0 con 0x7f8054161950 2024-05-09T06:32:24.707 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:24.707 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:24.367334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-09T06:32:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 48 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 msgr2=0x7f803806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 0x7f803806e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8050009a30 tx=0x7f8050008040 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 msgr2=0x7f8054153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f804402a1c0 tx=0x7f80440046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 shutdown_connections 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f803806be30 0x7f803806e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80540b8450 0x7f805415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8054161950 0x7f8054153080 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 --2- 172.21.15.1:0/3535586707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80540d0f80 0x7f80540b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 >> 172.21.15.1:0/3535586707 conn(0x7f80540d3e20 msgr2=0x7f80540d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 shutdown_connections 2024-05-09T06:32:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:24.727+0000 7f805c293640 1 -- 172.21.15.1:0/3535586707 wait complete. 2024-05-09T06:32:25.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:32:04.798236+0000', 'modified': '2024-05-09T06:32:24.367334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7113': {'gid': 7113, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6836/3897834247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 3897834247}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 3897834247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:32:25.122 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7113:up:replay 2024-05-09T06:32:26.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:26.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.258+0000 7f848862c640 1 Processor -- start 2024-05-09T06:32:26.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.258+0000 7f848862c640 1 -- start start 2024-05-09T06:32:26.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f848862c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8480161950 0x7f84800d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f848862c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84800d0f80 0x7f84800d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f848862c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84800b33c0 con 0x7f84800d4940 2024-05-09T06:32:26.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f848862c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f84800d4dd0 con 0x7f84800d0f80 2024-05-09T06:32:26.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f848862c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84800b3660 con 0x7f8480161950 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.259+0000 7f8486bc9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84800d0f80 0x7f84800d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84800d0f80 0x7f84800d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55890/0 (socket says 172.21.15.1:55890) 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 -- 172.21.15.1:0/1546087990 learned_addr learned my addr 172.21.15.1:0/1546087990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f84863c8640 1 --2- 172.21.15.1:0/1546087990 >> v2:172.21.15.6:3300/0 conn(0x7f8480161950 0x7f84800d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 -- 172.21.15.1:0/1546087990 >> v2:172.21.15.6:3300/0 conn(0x7f8480161950 msgr2=0x7f84800d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 --2- 172.21.15.1:0/1546087990 >> v2:172.21.15.6:3300/0 conn(0x7f8480161950 0x7f84800d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 -- 172.21.15.1:0/1546087990 >> v1:172.21.15.6:6789/0 conn(0x7f84800d4940 legacy=0x7f84800cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.260+0000 7f8486bc9640 1 -- 172.21.15.1:0/1546087990 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84800b3940 con 0x7f84800d0f80 2024-05-09T06:32:26.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f8486bc9640 1 --2- 172.21.15.1:0/1546087990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f84800d2940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f848006b7f0 tx=0x7f847802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748782a10705e550 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:26.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f84853c6640 1 -- 172.21.15.1:0/1546087990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478037070 con 0x7f84800d0f80 2024-05-09T06:32:26.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f84853c6640 1 -- 172.21.15.1:0/1546087990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8478032440 con 0x7f84800d0f80 2024-05-09T06:32:26.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f84853c6640 1 -- 172.21.15.1:0/1546087990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478032810 con 0x7f84800d0f80 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 -- 172.21.15.1:0/1546087990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 msgr2=0x7f84800d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 --2- 172.21.15.1:0/1546087990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f84800d2940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f848006b7f0 tx=0x7f847802a270 comp rx=0 tx=0).stop 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 -- 172.21.15.1:0/1546087990 shutdown_connections 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 --2- 172.21.15.1:0/1546087990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f84800d2940 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 --2- 172.21.15.1:0/1546087990 >> v2:172.21.15.6:3300/0 conn(0x7f8480161950 0x7f84800d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 -- 172.21.15.1:0/1546087990 >> 172.21.15.1:0/1546087990 conn(0x7f84800d3e20 msgr2=0x7f84800d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.261+0000 7f848862c640 1 -- 172.21.15.1:0/1546087990 shutdown_connections 2024-05-09T06:32:26.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.262+0000 7f848862c640 1 -- 172.21.15.1:0/1546087990 wait complete. 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.263+0000 7f848862c640 1 Processor -- start 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.263+0000 7f848862c640 1 -- start start 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.263+0000 7f848862c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.263+0000 7f848862c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8480161950 0x7f84801f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 0x7f84800b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84800b52d0 con 0x7f84800d0f80 2024-05-09T06:32:26.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84800b5150 con 0x7f84800cdb00 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f84800b5450 con 0x7f8480161950 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55898/0 (socket says 172.21.15.1:55898) 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 -- 172.21.15.1:0/3347938491 learned_addr learned my addr 172.21.15.1:0/3347938491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8486bc9640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 0x7f84800b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8485bc7640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8480161950 0x7f84801f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 -- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8480161950 msgr2=0x7f84801f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8480161950 0x7f84801f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 -- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 msgr2=0x7f84800b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 0x7f84800b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 -- 172.21.15.1:0/3347938491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84801f5380 con 0x7f84800d0f80 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8486bc9640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 0x7f84800b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f84863c8640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f847c007e70 tx=0x7f847c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c014620 con 0x7f84800d0f80 2024-05-09T06:32:26.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c019070 con 0x7f84800d0f80 2024-05-09T06:32:26.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84800b8910 con 0x7f84800d0f80 2024-05-09T06:32:26.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c014a00 con 0x7f84800d0f80 2024-05-09T06:32:26.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.264+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84800b8ed0 con 0x7f84800d0f80 2024-05-09T06:32:26.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.265+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84800752b0 con 0x7f84800d0f80 2024-05-09T06:32:26.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.266+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f847c014ba0 con 0x7f84800d0f80 2024-05-09T06:32:26.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.266+0000 7f8476ffd640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 0x7f846406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:26.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.266+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f847c08b8f0 con 0x7f84800d0f80 2024-05-09T06:32:26.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.266+0000 7f8485bc7640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 0x7f846406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:26.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.267+0000 7f8485bc7640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 0x7f846406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f848006d880 tx=0x7f8470009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:26.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.271+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84801f5380 con 0x7f84800d0f80 2024-05-09T06:32:26.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.493+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84800b6cc0 con 0x7f84800d0f80 2024-05-09T06:32:26.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.494+0000 7f8476ffd640 1 -- 172.21.15.1:0/3347938491 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5162 (secure 0 0 0) 0x7f847c05a0c0 con 0x7f84800d0f80 2024-05-09T06:32:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:26.475 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:25.373128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-09T06:32:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 50 2024-05-09T06:32:26.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 msgr2=0x7f846406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 0x7f846406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f848006d880 tx=0x7f8470009040 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 msgr2=0x7f8480159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f847c007e70 tx=0x7f847c00e4b0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 shutdown_connections 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f846406be30 0x7f846406e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84800cdb00 0x7f84800b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8480161950 0x7f84801f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 --2- 172.21.15.1:0/3347938491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84800d0f80 0x7f8480159980 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 >> 172.21.15.1:0/3347938491 conn(0x7f84800d3e20 msgr2=0x7f8480160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 shutdown_connections 2024-05-09T06:32:26.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:26.496+0000 7f848862c640 1 -- 172.21.15.1:0/3347938491 wait complete. 2024-05-09T06:32:26.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:32:04.798236+0000', 'modified': '2024-05-09T06:32:25.373128+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7113': {'gid': 7113, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.1:6836/3897834247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 3897834247}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 3897834247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:32:26.891 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7113:up:rejoin 2024-05-09T06:32:27.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:32:28.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.027+0000 7fad3496a640 1 Processor -- start 2024-05-09T06:32:28.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.027+0000 7fad3496a640 1 -- start start 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad3496a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fad300cf180 0x7fad300cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad3496a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad3496a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad300ce0e0 con 0x7fad3006e6d0 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad3496a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad300ce3c0 con 0x7fad300cf180 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad3496a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad300ce6a0 con 0x7fad300cdab0 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad2e59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fad3006e6d0 0x7fad3006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37444/0 (socket says 172.21.15.1:37444) 2024-05-09T06:32:28.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad2e59c640 1 -- 172.21.15.1:0/2214490927 learned_addr learned my addr 172.21.15.1:0/2214490927 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad25d9b640 1 --2- 172.21.15.1:0/2214490927 >> v2:172.21.15.6:3300/0 conn(0x7fad300cf180 0x7fad300cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad2ed9d640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.028+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310970567 0 0) 0x7fad300ce0e0 con 0x7fad3006e6d0 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad1c003520 con 0x7fad3006e6d0 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad25d9b640 1 -- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 msgr2=0x7fad300b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad25d9b640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293338043 0 0) 0x7fad1c003520 con 0x7fad3006e6d0 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad25d9b640 1 -- 172.21.15.1:0/2214490927 >> v1:172.21.15.1:6789/0 conn(0x7fad3006e6d0 legacy=0x7fad3006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad25d9b640 1 -- 172.21.15.1:0/2214490927 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300ce980 con 0x7fad300cf180 2024-05-09T06:32:28.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.029+0000 7fad2ed9d640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:32:28.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad25d9b640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad300cd570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad180097a0 tx=0x7fad1802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5f3d1ffbd1c733 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18037070 con 0x7fad300cf180 2024-05-09T06:32:28.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1802aba0 con 0x7fad300cf180 2024-05-09T06:32:28.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad2dd9b640 1 -- 172.21.15.1:0/2214490927 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20001670 con 0x7fad300cf180 2024-05-09T06:32:28.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 -- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 msgr2=0x7fad300cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad300cd570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fad180097a0 tx=0x7fad1802a110 comp rx=0 tx=0).stop 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 -- 172.21.15.1:0/2214490927 shutdown_connections 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 --2- 172.21.15.1:0/2214490927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad300cd570 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 -- 172.21.15.1:0/2214490927 >> 172.21.15.1:0/2214490927 conn(0x7fad300cc4b0 msgr2=0x7fad300cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 -- 172.21.15.1:0/2214490927 shutdown_connections 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.030+0000 7fad3496a640 1 -- 172.21.15.1:0/2214490927 wait complete. 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.032+0000 7fad3496a640 1 Processor -- start 2024-05-09T06:32:28.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.032+0000 7fad3496a640 1 -- start start 2024-05-09T06:32:28.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad301f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad301447c0 con 0x7fad3006e6d0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad30070ed0 con 0x7fad300cf180 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad3496a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fad30144940 con 0x7fad300cdab0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50180/0 (socket says 172.21.15.1:50180) 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 -- 172.21.15.1:0/176424948 learned_addr learned my addr 172.21.15.1:0/176424948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2ed9d640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad301f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad25d9b640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 -- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 msgr2=0x7fad300accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 -- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 msgr2=0x7fad301f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad301f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.033+0000 7fad2e59c640 1 -- 172.21.15.1:0/176424948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad301f5b30 con 0x7fad3006e6d0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad2e59c640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fad20001eb0 tx=0x7fad2000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20017070 con 0x7fad3006e6d0 2024-05-09T06:32:28.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad20005ce0 con 0x7fad3006e6d0 2024-05-09T06:32:28.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301f1a60 con 0x7fad3006e6d0 2024-05-09T06:32:28.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20007290 con 0x7fad3006e6d0 2024-05-09T06:32:28.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.034+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad300af1c0 con 0x7fad3006e6d0 2024-05-09T06:32:28.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.035+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad300af450 con 0x7fad3006e6d0 2024-05-09T06:32:28.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.035+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad2001a070 con 0x7fad3006e6d0 2024-05-09T06:32:28.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.036+0000 7fad277fe640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 0x7fad1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.036+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fad2008a4b0 con 0x7fad3006e6d0 2024-05-09T06:32:28.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.036+0000 7fad25d9b640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 0x7fad1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.036+0000 7fad25d9b640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 0x7fad1006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad18034000 tx=0x7fad1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.041+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad300af450 con 0x7fad3006e6d0 2024-05-09T06:32:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.265+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad300752b0 con 0x7fad3006e6d0 2024-05-09T06:32:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.265+0000 7fad277fe640 1 -- 172.21.15.1:0/176424948 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5170 (secure 0 0 0) 0x7fad20058d10 con 0x7fad3006e6d0 2024-05-09T06:32:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:28.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:27.372282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":10,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:32:28.247 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 51 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 msgr2=0x7fad1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 0x7fad1006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad18034000 tx=0x7fad1802e000 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 msgr2=0x7fad300ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fad20001eb0 tx=0x7fad2000c9b0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 shutdown_connections 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fad1006be30 0x7fad1006e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad300cf180 0x7fad301f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fad300cdab0 0x7fad300accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 --2- 172.21.15.1:0/176424948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3006e6d0 0x7fad300ac7b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 >> 172.21.15.1:0/176424948 conn(0x7fad300cc4b0 msgr2=0x7fad300b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 shutdown_connections 2024-05-09T06:32:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.267+0000 7fad3496a640 1 -- 172.21.15.1:0/176424948 wait complete. 2024-05-09T06:32:28.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:32:04.798236+0000', 'modified': '2024-05-09T06:32:27.372282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7113': {'gid': 7113, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.1:6836/3897834247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 3897834247}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 3897834247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7113, 'qdb_cluster': [7113]} 2024-05-09T06:32:28.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:32:28.663 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:32:28.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.790+0000 7f12100f5640 1 Processor -- start 2024-05-09T06:32:28.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.791+0000 7f12100f5640 1 -- start start 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.791+0000 7f12100f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1208161950 0x7f12080d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.791+0000 7f12100f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 0x7f12080d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.791+0000 7f12100f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12080d0f80 0x7f12080cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.791+0000 7f12100f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f12080d4dd0 con 0x7f12080d0f80 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f12100f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f12080b33c0 con 0x7f1208161950 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f12100f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f12080b3660 con 0x7f12080d4940 2024-05-09T06:32:28.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12080d0f80 0x7f12080cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12080d0f80 0x7f12080cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50190/0 (socket says 172.21.15.1:50190) 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 -- 172.21.15.1:0/1714553850 learned_addr learned my addr 172.21.15.1:0/1714553850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120d690640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 0x7f12080d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 -- 172.21.15.1:0/1714553850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 msgr2=0x7f12080d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 0x7f12080d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 -- 172.21.15.1:0/1714553850 >> v2:172.21.15.6:3300/0 conn(0x7f1208161950 msgr2=0x7f12080d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 --2- 172.21.15.1:0/1714553850 >> v2:172.21.15.6:3300/0 conn(0x7f1208161950 0x7f12080d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120e692640 1 -- 172.21.15.1:0/1714553850 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12080b3940 con 0x7f12080d0f80 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.792+0000 7f120d690640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 0x7f12080d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f120e692640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080d0f80 0x7f12080cea30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f120806b7f0 tx=0x7f11f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a706cf86c42c88e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f120ce8f640 1 -- 172.21.15.1:0/1714553850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8036070 con 0x7f12080d0f80 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f120ce8f640 1 -- 172.21.15.1:0/1714553850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8031600 con 0x7f12080d0f80 2024-05-09T06:32:28.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f120ce8f640 1 -- 172.21.15.1:0/1714553850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80319d0 con 0x7f12080d0f80 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 -- 172.21.15.1:0/1714553850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080d0f80 msgr2=0x7f12080cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080d0f80 0x7f12080cea30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f120806b7f0 tx=0x7f11f802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 -- 172.21.15.1:0/1714553850 shutdown_connections 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080d0f80 0x7f12080cea30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 --2- 172.21.15.1:0/1714553850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12080d4940 0x7f12080d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.793+0000 7f12100f5640 1 --2- 172.21.15.1:0/1714553850 >> v2:172.21.15.6:3300/0 conn(0x7f1208161950 0x7f12080d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.794+0000 7f12100f5640 1 -- 172.21.15.1:0/1714553850 >> 172.21.15.1:0/1714553850 conn(0x7f12080d3e20 msgr2=0x7f12080d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.794+0000 7f12100f5640 1 -- 172.21.15.1:0/1714553850 shutdown_connections 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.794+0000 7f12100f5640 1 -- 172.21.15.1:0/1714553850 wait complete. 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.795+0000 7f12100f5640 1 Processor -- start 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.795+0000 7f12100f5640 1 -- start start 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.795+0000 7f12100f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12080d0f80 0x7f12080cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1208161950 0x7f12081f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12080d1d80 con 0x7f12080b9520 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f12080d1c00 con 0x7f12080d0f80 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f12080d1f00 con 0x7f1208161950 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50204/0 (socket says 172.21.15.1:50204) 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 -- 172.21.15.1:0/4162523663 learned_addr learned my addr 172.21.15.1:0/4162523663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120de91640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12080d0f80 0x7f12080cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120d690640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1208161950 0x7f12081f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 -- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1208161950 msgr2=0x7f12081f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1208161950 0x7f12081f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 -- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12080d0f80 msgr2=0x7f12080cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12080d0f80 0x7f12080cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 -- 172.21.15.1:0/4162523663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081f4430 con 0x7f12080b9520 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f120e692640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009f40 tx=0x7f11f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12067fc640 1 -- 172.21.15.1:0/4162523663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8036050 con 0x7f12080b9520 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12067fc640 1 -- 172.21.15.1:0/4162523663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8042070 con 0x7f12080b9520 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081f4710 con 0x7f12080b9520 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12067fc640 1 -- 172.21.15.1:0/4162523663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8009510 con 0x7f12080b9520 2024-05-09T06:32:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.796+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081f4d40 con 0x7f12080b9520 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.798+0000 7f12067fc640 1 -- 172.21.15.1:0/4162523663 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11f8007bd0 con 0x7f12080b9520 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.799+0000 7f12067fc640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 0x7f11f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.799+0000 7f12067fc640 1 -- 172.21.15.1:0/4162523663 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f11f80abeb0 con 0x7f12080b9520 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.799+0000 7f120de91640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 0x7f11f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.800+0000 7f120de91640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 0x7f11f406e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f12000098e0 tx=0x7f1200009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.800+0000 7f1212e03740 1 Processor -- start 2024-05-09T06:32:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.800+0000 7f1212e03740 1 -- start start 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee216a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee1ec350 0x55aaee2325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aaee14f4d0 con 0x55aaee233f10 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55aaee0f73e0 con 0x55aaee216670 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aaee14b490 con 0x55aaee1ec350 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50218/0 (socket says 172.21.15.1:50218) 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 -- 172.21.15.1:0/2844667282 learned_addr learned my addr 172.21.15.1:0/2844667282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120de91640 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee216a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120e692640 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee1ec350 0x55aaee2325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 -- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee1ec350 msgr2=0x55aaee2325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee1ec350 0x55aaee2325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 -- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 msgr2=0x55aaee216a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee216a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 -- 172.21.15.1:0/2844667282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaede632d0 con 0x55aaee233f10 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f120d690640 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ac20 tx=0x7f11fc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcfae84f856e3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f11ef7fe640 1 -- 172.21.15.1:0/2844667282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc004020 con 0x55aaee233f10 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f11ef7fe640 1 -- 172.21.15.1:0/2844667282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc0041c0 con 0x55aaee233f10 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.801+0000 7f11ef7fe640 1 -- 172.21.15.1:0/2844667282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc003610 con 0x55aaee233f10 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 -- 172.21.15.1:0/2844667282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 msgr2=0x55aaee234460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ac20 tx=0x7f11fc00f610 comp rx=0 tx=0).stop 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 -- 172.21.15.1:0/2844667282 shutdown_connections 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee1ec350 0x55aaee2325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee234460 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 --2- 172.21.15.1:0/2844667282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee216a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 -- 172.21.15.1:0/2844667282 >> 172.21.15.1:0/2844667282 conn(0x55aaee1e8c80 msgr2=0x55aaee231480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 -- 172.21.15.1:0/2844667282 shutdown_connections 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.802+0000 7f1212e03740 1 -- 172.21.15.1:0/2844667282 wait complete. 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.803+0000 7f1212e03740 1 -- start start 2024-05-09T06:32:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.803+0000 7f1212e03740 1 Processor -- start 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee217b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee245410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aaee1ed0f0 con 0x55aaee233f10 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55aaee10a8e0 con 0x55aaee219d40 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aaee1e6bb0 con 0x55aaee216670 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 10 client.0 _ll_register_callbacks cb 0x55aaee0d79c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50224/0 (socket says 172.21.15.1:50224) 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 -- 172.21.15.1:0/1759217779 learned_addr learned my addr 172.21.15.1:0/1759217779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120e692640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee245410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 -- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 msgr2=0x55aaee217b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee217b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 -- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 msgr2=0x55aaee245410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee245410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 -- 172.21.15.1:0/1759217779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaedf2d6b0 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f120d690640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00a1a0 tx=0x7f11fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6935cf3d729c1471 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f11ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc02a060 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f11fc003820 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.804+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc0041c0 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f11fc030050 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 10 client.7428 fetch_fsmap learned FSMap version 51 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55aaee244ef0 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc0041c0 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f11fc033020 con 0x55aaee233f10 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 10 client.7428 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T06:32:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 10 client.7428 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 20 client.7428 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee2188a0 0x55aaee218c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 4 client.7428 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaee15cc00 con 0x55aaee2188a0 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f120de91640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee2188a0 0x55aaee218c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f11ecff9640 10 client.7428 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.805+0000 7f120de91640 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee2188a0 0x55aaee218c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b300bd371f1032d server_cookie=659620532ff18170 in_seq=0 out_seq=0 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f11ecff9640 1 -- 172.21.15.1:0/1759217779 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaee15cc00 con 0x55aaee2188a0 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f11ecff9640 10 client.7428 handle_command_reply: tid=0 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 client.7428 shutdown 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 msgr2=0x55aaee212e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00a1a0 tx=0x7f11fc004730 comp rx=0 tx=0).stop 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 shutdown_connections 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee245410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee212e30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee2188a0 0x55aaee218c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 --2- 172.21.15.1:0/1759217779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee217b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 >> 172.21.15.1:0/1759217779 conn(0x55aaee1ec350 msgr2=0x55aaee1e6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f11ecff9640 0 client.7428 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 shutdown_connections 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 1 -- 172.21.15.1:0/1759217779 wait complete. 2024-05-09T06:32:28.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.806+0000 7f1212e03740 20 client.7428 trim_cache size 0 max 16384 2024-05-09T06:32:28.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 Processor -- start 2024-05-09T06:32:28.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 -- start start 2024-05-09T06:32:28.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee217330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 0x55aaee1e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ffd20 con 0x55aaee219d40 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f12081f65b0 con 0x55aaee216670 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f12081ff6b0 con 0x55aaee233f10 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.825+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50234/0 (socket says 172.21.15.1:50234) 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 -- 172.21.15.1:0/3362107809 learned_addr learned my addr 172.21.15.1:0/3362107809 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120e692640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 0x55aaee1e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120de91640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee217330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 -- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 msgr2=0x55aaee1e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 0x55aaee1e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 -- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 msgr2=0x55aaee217330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee217330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 -- 172.21.15.1:0/3362107809 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaee2448e0 con 0x55aaee219d40 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120e692640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 0x55aaee1e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120de91640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee217330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f120d690640 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00fb90 tx=0x7f11fc01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370fd1ae82dfa3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f11eeffd640 1 -- 172.21.15.1:0/3362107809 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc039070 con 0x55aaee219d40 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f11eeffd640 1 -- 172.21.15.1:0/3362107809 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc003cd0 con 0x55aaee219d40 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.826+0000 7f11eeffd640 1 -- 172.21.15.1:0/3362107809 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0040c0 con 0x55aaee219d40 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- 172.21.15.1:0/3362107809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 msgr2=0x55aaee219690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00fb90 tx=0x7f11fc01e070 comp rx=0 tx=0).stop 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- 172.21.15.1:0/3362107809 shutdown_connections 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee233f10 0x55aaee1e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee219d40 0x55aaee219690 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 --2- 172.21.15.1:0/3362107809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee216670 0x55aaee217330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- 172.21.15.1:0/3362107809 >> 172.21.15.1:0/3362107809 conn(0x55aaee1ec350 msgr2=0x55aaee1ed690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- 172.21.15.1:0/3362107809 shutdown_connections 2024-05-09T06:32:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- 172.21.15.1:0/3362107809 wait complete. 2024-05-09T06:32:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 -- start start 2024-05-09T06:32:28.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.827+0000 7f1212e03740 1 Processor -- start 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee1e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee2940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aaee241950 con 0x55aaee233f10 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55aaee218160 con 0x55aaee219d40 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aaee234a50 con 0x55aaee216670 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f1212e03740 10 client.0 _ll_register_callbacks cb 0x7f11fc00aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.828+0000 7f120e692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50238/0 (socket says 172.21.15.1:50238) 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 -- 172.21.15.1:0/2687491918 learned_addr learned my addr 172.21.15.1:0/2687491918 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120d690640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee2940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120de91640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee1e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 -- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 msgr2=0x55aaee1e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee1e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 -- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 msgr2=0x55aaee2940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee2940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 -- 172.21.15.1:0/2687491918 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaee1e60b0 con 0x55aaee233f10 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120de91640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee1e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f120e692640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f11f8076490 tx=0x7f11f8079990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd026c7d60548c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f11ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:28.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80a7030 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f11f80a3b60 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc003cd0 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f11f807e330 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f1212e03740 10 client.7434 fetch_fsmap learned FSMap version 51 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55aaee231a30 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.829+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc003cd0 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f11f8079480 con 0x55aaee233f10 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 10 client.7434 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 10 client.7434 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 20 client.7434 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee228e30 0x55aaee2ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 4 client.7434 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "status"}) -- 0x55aaee1e7aa0 con 0x55aaee228e30 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f120de91640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee228e30 0x55aaee2ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f11ecff9640 10 client.7434 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f120de91640 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee228e30 0x55aaee2ab3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc382f5dc8a2974 server_cookie=73dbb886ecf22b1c in_seq=0 out_seq=0 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f11ecff9640 1 -- 172.21.15.1:0/2687491918 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55aaee1e7aa0 con 0x55aaee228e30 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f11ecff9640 10 client.7434 handle_command_reply: tid=0 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 client.7434 shutdown 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 msgr2=0x55aaee294610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f11f8076490 tx=0x7f11f8079990 comp rx=0 tx=0).stop 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 shutdown_connections 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aaee233f10 0x55aaee294610 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aaee219d40 0x55aaee2940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aaee228e30 0x55aaee2ab3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 --2- 172.21.15.1:0/2687491918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aaee216670 0x55aaee1e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f11ecff9640 0 client.7434 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.830+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 >> 172.21.15.1:0/2687491918 conn(0x55aaee1ec350 msgr2=0x55aaee229c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 shutdown_connections 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f1212e03740 1 -- 172.21.15.1:0/2687491918 wait complete. 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f1212e03740 20 client.7434 trim_cache size 0 max 16384 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7113, 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4618159669999997, 2024-05-09T06:32:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 51, 2024-05-09T06:32:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 79, 2024-05-09T06:32:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T06:32:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 29.396127597 2024-05-09T06:32:28.820 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:28.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 msgr2=0x7f11f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 0x7f11f406e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f12000098e0 tx=0x7f1200009060 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 msgr2=0x7f12080cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009f40 tx=0x7f11f8004a40 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 shutdown_connections 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f11f406be30 0x7f11f406e2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12080b9520 0x7f12080cc4e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1208161950 0x7f12081f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 --2- 172.21.15.1:0/4162523663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12080d0f80 0x7f12080cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 >> 172.21.15.1:0/4162523663 conn(0x7f12080d3e20 msgr2=0x7f1208160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 shutdown_connections 2024-05-09T06:32:28.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.831+0000 7f12100f5640 1 -- 172.21.15.1:0/4162523663 wait complete. 2024-05-09T06:32:28.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:32:28.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.945+0000 7f5d642f5640 1 Processor -- start 2024-05-09T06:32:28.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.945+0000 7f5d642f5640 1 -- start start 2024-05-09T06:32:28.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d642f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5d5c164ce0 0x7f5d5c1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d642f5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5c071c20 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d642f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5c06eb40 con 0x7f5d5c1454b0 2024-05-09T06:32:28.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d642f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c06ee20 con 0x7f5d5c164ce0 2024-05-09T06:32:28.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d61890640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5d5c0b2a60 0x7f5d5c0b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37454/0 (socket says 172.21.15.1:37454) 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d61890640 1 -- 172.21.15.1:0/780096674 learned_addr learned my addr 172.21.15.1:0/780096674 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173398153 0 0) 0x7f5d5c071c20 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d62892640 1 --2- 172.21.15.1:0/780096674 >> v2:172.21.15.6:3300/0 conn(0x7f5d5c164ce0 0x7f5d5c1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.946+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d48003520 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1204621841 0 0) 0x7f5d48003520 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 >> v2:172.21.15.6:3300/0 conn(0x7f5d5c164ce0 msgr2=0x7f5d5c1670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 --2- 172.21.15.1:0/780096674 >> v2:172.21.15.6:3300/0 conn(0x7f5d5c164ce0 0x7f5d5c1670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 >> v1:172.21.15.6:6789/0 conn(0x7f5d5c1454b0 legacy=0x7f5d5c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c06d8e0 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f5d4c001670 con 0x7f5d5c0b2a60 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 >> v1:172.21.15.1:6789/0 conn(0x7f5d5c0b2a60 legacy=0x7f5d5c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d480039b0 0x7f5d48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d480051c0 0x7f5d4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c06ee20 con 0x7f5d48004800 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4800c060 con 0x7f5d480051c0 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4800c260 con 0x7f5d480039b0 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d62091640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d480039b0 0x7f5d48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d62892640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d480051c0 0x7f5d4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 -- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d480039b0 msgr2=0x7f5d48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d480039b0 0x7f5d48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 -- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d480051c0 msgr2=0x7f5d4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d480051c0 0x7f5d4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d61890640 1 -- 172.21.15.1:0/780096674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d48003520 con 0x7f5d48004800 2024-05-09T06:32:28.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.947+0000 7f5d62892640 1 -- 172.21.15.1:0/780096674 reap_dead start 2024-05-09T06:32:28.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.948+0000 7f5d61890640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d48004c80 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c240 tx=0x7f5d48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f313ad29d550579d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.948+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007d10 con 0x7f5d48004800 2024-05-09T06:32:28.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.948+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c005ad0 con 0x7f5d48004800 2024-05-09T06:32:28.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.948+0000 7f5d6108f640 1 -- 172.21.15.1:0/780096674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007070 con 0x7f5d48004800 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 -- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 msgr2=0x7f5d48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d48004c80 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c240 tx=0x7f5d48005830 comp rx=0 tx=0).stop 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 -- 172.21.15.1:0/780096674 shutdown_connections 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 --2- 172.21.15.1:0/780096674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d48004c80 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 -- 172.21.15.1:0/780096674 >> 172.21.15.1:0/780096674 conn(0x7f5d5c0cc4b0 msgr2=0x7f5d5c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 -- 172.21.15.1:0/780096674 shutdown_connections 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.949+0000 7f5d642f5640 1 -- 172.21.15.1:0/780096674 wait complete. 2024-05-09T06:32:28.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.950+0000 7f5d642f5640 1 Processor -- start 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.950+0000 7f5d642f5640 1 -- start start 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d5c13c9b0 0x7f5d5c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d5c0a99e0 0x7f5d5c141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c06e0e0 con 0x7f5d48004800 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c145890 con 0x7f5d5c0a99e0 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c06e260 con 0x7f5d5c13c9b0 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50256/0 (socket says 172.21.15.1:50256) 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 -- 172.21.15.1:0/2555147251 learned_addr learned my addr 172.21.15.1:0/2555147251 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 -- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d5c13c9b0 msgr2=0x7f5d5c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d5c13c9b0 0x7f5d5c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 -- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d5c0a99e0 msgr2=0x7f5d5c141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d5c0a99e0 0x7f5d5c141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 -- 172.21.15.1:0/2555147251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c13b630 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d62091640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400f5f0 tx=0x7f5d5400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540052e0 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d54018070 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c13b910 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54003830 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.951+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c13bed0 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.952+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c0752b0 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.953+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d540039d0 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.953+0000 7f5d52ffd640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 0x7f5d4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.953+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f5d5408ad60 con 0x7f5d48004800 2024-05-09T06:32:28.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.953+0000 7f5d61890640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 0x7f5d4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:28.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.954+0000 7f5d61890640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 0x7f5d4006e320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c032000 tx=0x7f5d4c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:28.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:28.958+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d540595c0 con 0x7f5d48004800 2024-05-09T06:32:29.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.180+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d5c0b1210 con 0x7f5d48004800 2024-05-09T06:32:29.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.180+0000 7f5d52ffd640 1 -- 172.21.15.1:0/2555147251 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f5d54058ec0 con 0x7f5d48004800 2024-05-09T06:32:29.162 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:32:29.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 msgr2=0x7f5d4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:29.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 0x7f5d4006e320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c032000 tx=0x7f5d4c009030 comp rx=0 tx=0).stop 2024-05-09T06:32:29.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 msgr2=0x7f5d5c1f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:29.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400f5f0 tx=0x7f5d5400fab0 comp rx=0 tx=0).stop 2024-05-09T06:32:29.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 shutdown_connections 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4006be60 0x7f5d4006e320 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d5c0a99e0 0x7f5d5c141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d5c13c9b0 0x7f5d5c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 --2- 172.21.15.1:0/2555147251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d48004800 0x7f5d5c1f5630 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 >> 172.21.15.1:0/2555147251 conn(0x7f5d5c0cc4b0 msgr2=0x7f5d5c0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:29.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 shutdown_connections 2024-05-09T06:32:29.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29.182+0000 7f5d642f5640 1 -- 172.21.15.1:0/2555147251 wait complete. 2024-05-09T06:32:29.174 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:32:29.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:32:29.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:32:29.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:29.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.176 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:32:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:32:29.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.196 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:32:29.258 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:32:29.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.259 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:32:29.317 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:32:29.317 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:32:29.317 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:32:29.317 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:32:29.317 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:32:29.317 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:32:29.317 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:32:29.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:32:29.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:32:29.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:32:29.318 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:32:29.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:32:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:32:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.449 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:32:29.465 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:32:29.465 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:32:29.465 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:29.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:32:29.532 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:32:29.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:32:29.532 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:29.588 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:32:29.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:32:29.588 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:29.644 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:32:29.710 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:32:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:32:29.769 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:29.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.770 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:32:29.825 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:32:29.825 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:32:29.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:32:29.893 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:32:29.951 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:32:29.952 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:29.953 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:32:29.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:29.954 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:32:29.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:32:29.970+0000 7fdd880e7480 -1 init, newargv = 0x561ba6312290 newargc=15 2024-05-09T06:32:29.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[63818]: starting ceph client 2024-05-09T06:32:29.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[63818]: starting fuse 2024-05-09T06:32:29.971 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:32:29.971 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:32:29.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:32:29.972 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:32:29.973 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:32:30.092 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.63818.asok 2024-05-09T06:32:30.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:32:30.102 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63818.asok 2024-05-09T06:32:30.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:30.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63818.asok status 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:32:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "63818", 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7443, 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7443 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4144407710 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4144407710 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.7443 192.168.0.1:0/4144407710", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/4144407710", 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:32:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 51, 2024-05-09T06:32:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 79, 2024-05-09T06:32:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:32:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:32:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:32:30.245 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:30.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:30.251 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:32:30.251 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:30.269 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:32:30.269 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:30.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:30.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:30.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:30.335 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-09T06:32:30.392 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:32:30.411+0000 7f8b218be640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:32:30.394 INFO:teuthology.orchestra.run.smithi001.stdout:file_undamaged 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> import json 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> import sys 2024-05-09T06:32:30.395 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> try: 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> except OSError as e: 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> sys.exit(e.errno) 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> print(json.dumps( 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> indent=2)) 2024-05-09T06:32:30.396 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:32:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-09T06:32:30.514 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-09T06:32:30.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:30.515 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-09T06:32:30.573 INFO:teuthology.orchestra.run.smithi001.stdout:2 2024-05-09T06:32:30.573 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:32:30.573 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:32:30.629 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.791+0000 7fbbdffa9640 1 Processor -- start 2024-05-09T06:32:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.792+0000 7fbbdffa9640 1 -- start start 2024-05-09T06:32:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.792+0000 7fbbdffa9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbbd815db10 0x7fbbd80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.792+0000 7fbbdffa9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd8070f40 con 0x7fbbd815e6d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.792+0000 7fbbdffa9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd80cf170 con 0x7fbbd80715d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.792+0000 7fbbdffa9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd80cf450 con 0x7fbbd815db10 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbde546640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbbd815e6d0 0x7fbbd806e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37470/0 (socket says 172.21.15.1:37470) 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbde546640 1 -- 172.21.15.1:0/3187876607 learned_addr learned my addr 172.21.15.1:0/3187876607 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbddd45640 1 --2- 172.21.15.1:0/3187876607 >> v2:172.21.15.6:3300/0 conn(0x7fbbd815db10 0x7fbbd80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494586883 0 0) 0x7fbbd8070f40 con 0x7fbbd815e6d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbc4003520 con 0x7fbbd815e6d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936477272 0 0) 0x7fbbd80cf170 con 0x7fbbd80715d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbd8070f40 con 0x7fbbd80715d0 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbddd45640 1 -- 172.21.15.1:0/3187876607 >> v1:172.21.15.6:6789/0 conn(0x7fbbd80715d0 legacy=0x7fbbd815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbddd45640 1 -- 172.21.15.1:0/3187876607 >> v1:172.21.15.1:6789/0 conn(0x7fbbd815e6d0 legacy=0x7fbbd806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.793+0000 7fbbddd45640 1 -- 172.21.15.1:0/3187876607 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd8070c10 con 0x7fbbd815db10 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.794+0000 7fbbddd45640 1 --2- 172.21.15.1:0/3187876607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd815db10 0x7fbbd80b2a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400a940 tx=0x7fbbd402b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ecaf09d9fcb7d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.794+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400b040 con 0x7fbbd815db10 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.794+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd4030510 con 0x7fbbd815db10 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.794+0000 7fbbdcd43640 1 -- 172.21.15.1:0/3187876607 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40308c0 con 0x7fbbd815db10 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3187876607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd815db10 msgr2=0x7fbbd80b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3187876607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd815db10 0x7fbbd80b2a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400a940 tx=0x7fbbd402b5e0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3187876607 shutdown_connections 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3187876607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd815db10 0x7fbbd80b2a60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3187876607 >> 172.21.15.1:0/3187876607 conn(0x7fbbd80cc4b0 msgr2=0x7fbbd80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3187876607 shutdown_connections 2024-05-09T06:32:30.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.795+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3187876607 wait complete. 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.796+0000 7fbbdffa9640 1 Processor -- start 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.796+0000 7fbbdffa9640 1 -- start start 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd80715d0 0x7fbbd80b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbd815e6d0 0x7fbbd8154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd815edc0 con 0x7fbbd815db10 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd815ec40 con 0x7fbbd80715d0 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdffa9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd806ea50 con 0x7fbbd815e6d0 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50266/0 (socket says 172.21.15.1:50266) 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 -- 172.21.15.1:0/3185638133 learned_addr learned my addr 172.21.15.1:0/3185638133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbddd45640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd80715d0 0x7fbbd80b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbde546640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbd815e6d0 0x7fbbd8154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 -- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbd815e6d0 msgr2=0x7fbbd8154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbd815e6d0 0x7fbbd8154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 -- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd80715d0 msgr2=0x7fbbd80b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd80715d0 0x7fbbd80b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 -- 172.21.15.1:0/3185638133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd8153160 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbdd544640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c500 tx=0x7fbbc800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.797+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80052d0 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.798+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc8005490 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.798+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd8153440 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.798+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8012870 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.798+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd8153a00 con 0x7fbbd815db10 2024-05-09T06:32:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.799+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd80752b0 con 0x7fbbd815db10 2024-05-09T06:32:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.799+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbc8012a10 con 0x7fbbd815db10 2024-05-09T06:32:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.799+0000 7fbbce7fc640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.800+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fbbc808ac40 con 0x7fbbd815db10 2024-05-09T06:32:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.800+0000 7fbbddd45640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.800+0000 7fbbddd45640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035000 tx=0x7fbbd400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:30.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.804+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbbc8058d30 con 0x7fbbd815db10 2024-05-09T06:32:30.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.988+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbd8074930 con 0x7fbbd815db10 2024-05-09T06:32:30.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.989+0000 7fbbce7fc640 1 -- 172.21.15.1:0/3185638133 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7fbbc805d030 con 0x7fbbd815db10 2024-05-09T06:32:30.971 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:30.971 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 msgr2=0x7fbbbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035000 tx=0x7fbbd400c040 comp rx=0 tx=0).stop 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 msgr2=0x7fbbd81549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c500 tx=0x7fbbc800c9c0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 shutdown_connections 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbd815e6d0 0x7fbbd8154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbd815db10 0x7fbbd81549a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 --2- 172.21.15.1:0/3185638133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbd80715d0 0x7fbbd80b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:30.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 >> 172.21.15.1:0/3185638133 conn(0x7fbbd80cc4b0 msgr2=0x7fbbd80cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:30.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 shutdown_connections 2024-05-09T06:32:30.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:30.991+0000 7fbbdffa9640 1 -- 172.21.15.1:0/3185638133 wait complete. 2024-05-09T06:32:30.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:32:30.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:32:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.109+0000 7f66705e3640 1 Processor -- start 2024-05-09T06:32:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.109+0000 7f66705e3640 1 -- start start 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f66705e3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 0x7f66680d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f66705e3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f666806ffb0 0x7f66680d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f66705e3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666806c420 con 0x7f66680ae9c0 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f66705e3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f666806f3f0 con 0x7f666806ffb0 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f66705e3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f666806c700 con 0x7f666815e300 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666eb80640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f666806ffb0 0x7f66680d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666eb80640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f666806ffb0 0x7f66680d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50280/0 (socket says 172.21.15.1:50280) 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666eb80640 1 -- 172.21.15.1:0/1162578735 learned_addr learned my addr 172.21.15.1:0/1162578735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666e37f640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 0x7f66680d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666eb80640 1 -- 172.21.15.1:0/1162578735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 msgr2=0x7f66680d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.110+0000 7f666eb80640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 0x7f66680d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666eb80640 1 -- 172.21.15.1:0/1162578735 >> v1:172.21.15.6:6789/0 conn(0x7f66680ae9c0 legacy=0x7f666806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666eb80640 1 -- 172.21.15.1:0/1162578735 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666806c9e0 con 0x7f666806ffb0 2024-05-09T06:32:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666e37f640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 0x7f66680d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666eb80640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680d0e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6660009e50 tx=0x7f666002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4abd0e0a59008e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666d37d640 1 -- 172.21.15.1:0/1162578735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660036070 con 0x7f666806ffb0 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666d37d640 1 -- 172.21.15.1:0/1162578735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6660031600 con 0x7f666806ffb0 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.111+0000 7f666d37d640 1 -- 172.21.15.1:0/1162578735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600319d0 con 0x7f666806ffb0 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 -- 172.21.15.1:0/1162578735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 msgr2=0x7f66680d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680d0e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6660009e50 tx=0x7f666002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 -- 172.21.15.1:0/1162578735 shutdown_connections 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680d0e70 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 --2- 172.21.15.1:0/1162578735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f666815e300 0x7f66680d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 -- 172.21.15.1:0/1162578735 >> 172.21.15.1:0/1162578735 conn(0x7f66680d1c50 msgr2=0x7f66680d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 -- 172.21.15.1:0/1162578735 shutdown_connections 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.112+0000 7f66705e3640 1 -- 172.21.15.1:0/1162578735 wait complete. 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.113+0000 7f66705e3640 1 Processor -- start 2024-05-09T06:32:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.113+0000 7f66705e3640 1 -- start start 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66680ae9c0 0x7f66680c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 0x7f666815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f666806d4a0 con 0x7f666806ffb0 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f666806cfa0 con 0x7f666815e300 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f66705e3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f666806d620 con 0x7f66680ae9c0 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50290/0 (socket says 172.21.15.1:50290) 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 -- 172.21.15.1:0/698169728 learned_addr learned my addr 172.21.15.1:0/698169728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666eb80640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 0x7f666815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666db7e640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66680ae9c0 0x7f66680c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 -- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66680ae9c0 msgr2=0x7f66680c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66680ae9c0 0x7f66680c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 -- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 msgr2=0x7f666815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 0x7f666815d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.114+0000 7f666e37f640 1 -- 172.21.15.1:0/698169728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680c7a00 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f666eb80640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 0x7f666815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f666e37f640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f666400c7f0 tx=0x7f666400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664015070 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6664007820 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66680c7ce0 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664007bd0 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.115+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66680c7640 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.116+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66680b4860 con 0x7f666806ffb0 2024-05-09T06:32:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.116+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6664018070 con 0x7f666806ffb0 2024-05-09T06:32:31.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.117+0000 7f665effd640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 0x7f664c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.117+0000 7f666db7e640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 0x7f664c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.117+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f666408b160 con 0x7f666806ffb0 2024-05-09T06:32:31.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.118+0000 7f666db7e640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 0x7f664c06e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f666815db70 tx=0x7f6658009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.122+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f66680b4860 con 0x7f666806ffb0 2024-05-09T06:32:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.307+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66680752b0 con 0x7f666806ffb0 2024-05-09T06:32:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.307+0000 7f665effd640 1 -- 172.21.15.1:0/698169728 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f6664059930 con 0x7f666806ffb0 2024-05-09T06:32:31.289 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 msgr2=0x7f664c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 0x7f664c06e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f666815db70 tx=0x7f6658009040 comp rx=0 tx=0).stop 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 msgr2=0x7f66680c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f666400c7f0 tx=0x7f666400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 shutdown_connections 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f664c06be30 0x7f664c06e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f666815e300 0x7f666815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66680ae9c0 0x7f66680c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 --2- 172.21.15.1:0/698169728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f666806ffb0 0x7f66680c6120 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 >> 172.21.15.1:0/698169728 conn(0x7f66680d1c50 msgr2=0x7f666815b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 shutdown_connections 2024-05-09T06:32:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.309+0000 7f66705e3640 1 -- 172.21.15.1:0/698169728 wait complete. 2024-05-09T06:32:31.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:32:31.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.427+0000 7f7f0429e640 1 Processor -- start 2024-05-09T06:32:31.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.427+0000 7f7f0429e640 1 -- start start 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0429e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc071610 0x7f7efc0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0429e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7efc0cdb90 0x7f7efc06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0429e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc0cf1b0 con 0x7f7efc15a4d0 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0429e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc15a8b0 con 0x7f7efc0cdb90 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0429e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc06dc10 con 0x7f7efc071610 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0203a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7efc15a4d0 0x7f7efc070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37474/0 (socket says 172.21.15.1:37474) 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0283b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7efc0cdb90 0x7f7efc06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.428+0000 7f7f0203a640 1 -- 172.21.15.1:0/1844543325 learned_addr learned my addr 172.21.15.1:0/1844543325 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:31.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f01839640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc071610 0x7f7efc0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f0283b640 1 -- 172.21.15.1:0/1844543325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc071610 msgr2=0x7f7efc0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f0283b640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc071610 0x7f7efc0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f0283b640 1 -- 172.21.15.1:0/1844543325 >> v1:172.21.15.1:6789/0 conn(0x7f7efc15a4d0 legacy=0x7f7efc070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f01038640 1 -- 172.21.15.1:0/1844543325 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347237836 0 0) 0x7f7efc0cf1b0 con 0x7f7efc15a4d0 2024-05-09T06:32:31.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f0283b640 1 -- 172.21.15.1:0/1844543325 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0cf490 con 0x7f7efc0cdb90 2024-05-09T06:32:31.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f0283b640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc0cdb90 0x7f7efc06e0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad8af79446e5339 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f01038640 1 -- 172.21.15.1:0/1844543325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007740 con 0x7f7efc0cdb90 2024-05-09T06:32:31.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f01038640 1 -- 172.21.15.1:0/1844543325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec0078e0 con 0x7f7efc0cdb90 2024-05-09T06:32:31.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.429+0000 7f7f01038640 1 -- 172.21.15.1:0/1844543325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007cb0 con 0x7f7efc0cdb90 2024-05-09T06:32:31.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 -- 172.21.15.1:0/1844543325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc0cdb90 msgr2=0x7f7efc06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc0cdb90 0x7f7efc06e0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec02a660 comp rx=0 tx=0).stop 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 -- 172.21.15.1:0/1844543325 shutdown_connections 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc071610 0x7f7efc0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 --2- 172.21.15.1:0/1844543325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc0cdb90 0x7f7efc06e0a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 -- 172.21.15.1:0/1844543325 >> 172.21.15.1:0/1844543325 conn(0x7f7efc0cc4d0 msgr2=0x7f7efc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 -- 172.21.15.1:0/1844543325 shutdown_connections 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.430+0000 7f7f0429e640 1 -- 172.21.15.1:0/1844543325 wait complete. 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.431+0000 7f7f0429e640 1 Processor -- start 2024-05-09T06:32:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.431+0000 7f7f0429e640 1 -- start start 2024-05-09T06:32:31.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7efc0cdb90 0x7f7efc14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc15a4d0 0x7f7efc14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc0b2be0 con 0x7f7efc071610 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc0b2a60 con 0x7f7efc0cdb90 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0429e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc06cd30 con 0x7f7efc15a4d0 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50298/0 (socket says 172.21.15.1:50298) 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 -- 172.21.15.1:0/263081614 learned_addr learned my addr 172.21.15.1:0/263081614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0283b640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc15a4d0 0x7f7efc14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f01839640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7efc0cdb90 0x7f7efc14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 -- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc15a4d0 msgr2=0x7f7efc14dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc15a4d0 0x7f7efc14dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 -- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7efc0cdb90 msgr2=0x7f7efc14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.432+0000 7f7f0203a640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7efc0cdb90 0x7f7efc14d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7f0203a640 1 -- 172.21.15.1:0/263081614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0b3fd0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7f0203a640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c430 tx=0x7f7ef400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40074e0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef40047e0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc0b2f30 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004bd0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.433+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc0b34f0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.434+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc0752b0 con 0x7f7efc071610 2024-05-09T06:32:31.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.434+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ef4011070 con 0x7f7efc071610 2024-05-09T06:32:31.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.435+0000 7f7efaffd640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 0x7f7ee806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.435+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f7ef408abf0 con 0x7f7efc071610 2024-05-09T06:32:31.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.435+0000 7f7f01839640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 0x7f7ee806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.436+0000 7f7f01839640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 0x7f7ee806e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0b5010 tx=0x7f7ef0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.440+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ef4059450 con 0x7f7efc071610 2024-05-09T06:32:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.622+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7efc15eb80 con 0x7f7efc071610 2024-05-09T06:32:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.623+0000 7f7efaffd640 1 -- 172.21.15.1:0/263081614 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f7ef4058d50 con 0x7f7efc071610 2024-05-09T06:32:31.604 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 msgr2=0x7f7ee806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 0x7f7ee806e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0b5010 tx=0x7f7ef0008040 comp rx=0 tx=0).stop 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 msgr2=0x7f7efc148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c430 tx=0x7f7ef400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 shutdown_connections 2024-05-09T06:32:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7ee806be30 0x7f7ee806e2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7efc15a4d0 0x7f7efc14dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7efc0cdb90 0x7f7efc14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 --2- 172.21.15.1:0/263081614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7efc071610 0x7f7efc148570 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 >> 172.21.15.1:0/263081614 conn(0x7f7efc0cc4d0 msgr2=0x7f7efc15df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 shutdown_connections 2024-05-09T06:32:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.625+0000 7f7f0429e640 1 -- 172.21.15.1:0/263081614 wait complete. 2024-05-09T06:32:31.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:32:31.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.742+0000 7f10a97f9640 1 Processor -- start 2024-05-09T06:32:31.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.742+0000 7f10a97f9640 1 -- start start 2024-05-09T06:32:31.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a40cd450 0x7f10a415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f10a4074660 0x7f10a4160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 0x7f10a415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f10a415c920 con 0x7f10a4074660 2024-05-09T06:32:31.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f10a415cba0 con 0x7f10a41605d0 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a97f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f10a415ce80 con 0x7f10a40cd450 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f10a4074660 0x7f10a4160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f10a4074660 0x7f10a4160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50310/0 (socket says 172.21.15.1:50310) 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 -- 172.21.15.1:0/995524218 learned_addr learned my addr 172.21.15.1:0/995524218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a2ffd640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a40cd450 0x7f10a415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a37fe640 1 --2- 172.21.15.1:0/995524218 >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 0x7f10a415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 -- 172.21.15.1:0/995524218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a40cd450 msgr2=0x7f10a415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a40cd450 0x7f10a415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 -- 172.21.15.1:0/995524218 >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 msgr2=0x7f10a415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 --2- 172.21.15.1:0/995524218 >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 0x7f10a415d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f109bfff640 1 -- 172.21.15.1:0/995524218 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a415d160 con 0x7f10a4074660 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.743+0000 7f10a37fe640 1 --2- 172.21.15.1:0/995524218 >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 0x7f10a415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.744+0000 7f109bfff640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a4074660 0x7f10a4160090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f10a409e380 tx=0x7f108c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c8f8431b23b8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.744+0000 7f10a27fc640 1 -- 172.21.15.1:0/995524218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c037070 con 0x7f10a4074660 2024-05-09T06:32:31.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.744+0000 7f10a27fc640 1 -- 172.21.15.1:0/995524218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108c032440 con 0x7f10a4074660 2024-05-09T06:32:31.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.744+0000 7f10a27fc640 1 -- 172.21.15.1:0/995524218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c032810 con 0x7f10a4074660 2024-05-09T06:32:31.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 -- 172.21.15.1:0/995524218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a4074660 msgr2=0x7f10a4160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a4074660 0x7f10a4160090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f10a409e380 tx=0x7f108c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 -- 172.21.15.1:0/995524218 shutdown_connections 2024-05-09T06:32:31.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 --2- 172.21.15.1:0/995524218 >> v2:172.21.15.6:3300/0 conn(0x7f10a41605d0 0x7f10a415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a4074660 0x7f10a4160090 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 --2- 172.21.15.1:0/995524218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a40cd450 0x7f10a415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 -- 172.21.15.1:0/995524218 >> 172.21.15.1:0/995524218 conn(0x7f10a40b1da0 msgr2=0x7f10a40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 -- 172.21.15.1:0/995524218 shutdown_connections 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.745+0000 7f10a97f9640 1 -- 172.21.15.1:0/995524218 wait complete. 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.746+0000 7f10a97f9640 1 Processor -- start 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.746+0000 7f10a97f9640 1 -- start start 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a4074660 0x7f10a40992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10a40cd450 0x7f10a40c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40cd9b0 con 0x7f10a41605d0 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40cd830 con 0x7f10a40cd450 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a97f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f10a415dae0 con 0x7f10a4074660 2024-05-09T06:32:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50316/0 (socket says 172.21.15.1:50316) 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 -- 172.21.15.1:0/2492731735 learned_addr learned my addr 172.21.15.1:0/2492731735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a2ffd640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a4074660 0x7f10a40992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f109bfff640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10a40cd450 0x7f10a40c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 -- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a4074660 msgr2=0x7f10a40992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a4074660 0x7f10a40992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 -- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10a40cd450 msgr2=0x7f10a40c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10a40cd450 0x7f10a40c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 -- 172.21.15.1:0/2492731735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a40c7ae0 con 0x7f10a41605d0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f10a37fe640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f109c007df0 tx=0x7f109c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.747+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c01a070 con 0x7f10a41605d0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.748+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c003990 con 0x7f10a41605d0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.748+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a40b3e10 con 0x7f10a41605d0 2024-05-09T06:32:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.748+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c003d80 con 0x7f10a41605d0 2024-05-09T06:32:31.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.748+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a40b43d0 con 0x7f10a41605d0 2024-05-09T06:32:31.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.749+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a40c7cb0 con 0x7f10a41605d0 2024-05-09T06:32:31.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.749+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f109c01d070 con 0x7f10a41605d0 2024-05-09T06:32:31.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.750+0000 7f109b7fe640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 0x7f108406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:31.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.750+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f109c08b2c0 con 0x7f10a41605d0 2024-05-09T06:32:31.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.750+0000 7f10a2ffd640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 0x7f108406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:31.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.750+0000 7f10a2ffd640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 0x7f108406e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1094009a50 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:31.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.754+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f109c010070 con 0x7f10a41605d0 2024-05-09T06:32:31.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.937+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10a40752b0 con 0x7f10a41605d0 2024-05-09T06:32:31.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.937+0000 7f109b7fe640 1 -- 172.21.15.1:0/2492731735 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f109c059a90 con 0x7f10a41605d0 2024-05-09T06:32:31.919 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:32:31.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 msgr2=0x7f108406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 0x7f108406e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1094009a50 tx=0x7f1094008040 comp rx=0 tx=0).stop 2024-05-09T06:32:31.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 msgr2=0x7f10a40c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:31.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f109c007df0 tx=0x7f109c00d4a0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 shutdown_connections 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f108406be30 0x7f108406e2f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10a41605d0 0x7f10a40c5a70 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10a40cd450 0x7f10a40c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 --2- 172.21.15.1:0/2492731735 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10a4074660 0x7f10a40992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 >> 172.21.15.1:0/2492731735 conn(0x7f10a40b1da0 msgr2=0x7f10a415f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 shutdown_connections 2024-05-09T06:32:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:31.940+0000 7f10a97f9640 1 -- 172.21.15.1:0/2492731735 wait complete. 2024-05-09T06:32:31.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:32:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.056+0000 7ff99f96a640 1 Processor -- start 2024-05-09T06:32:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.056+0000 7ff99f96a640 1 -- start start 2024-05-09T06:32:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99f96a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 0x7ff998166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99f96a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980ce0e0 con 0x7ff99806e6d0 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99f96a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980ce3c0 con 0x7ff9980cf180 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99f96a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff9980ce6a0 con 0x7ff998163c70 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99d706640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff99806e6d0 0x7ff99806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37486/0 (socket says 172.21.15.1:37486) 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99d706640 1 -- 172.21.15.1:0/1869391497 learned_addr learned my addr 172.21.15.1:0/1869391497 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.057+0000 7ff99df07640 1 --2- 172.21.15.1:0/1869391497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 0x7ff998166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920580182 0 0) 0x7ff9980ce0e0 con 0x7ff99806e6d0 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff980003520 con 0x7ff99806e6d0 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367443138 0 0) 0x7ff9980ce3c0 con 0x7ff9980cf180 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9980ce0e0 con 0x7ff9980cf180 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff99df07640 1 -- 172.21.15.1:0/1869391497 >> v1:172.21.15.6:6789/0 conn(0x7ff9980cf180 legacy=0x7ff9980cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff99df07640 1 -- 172.21.15.1:0/1869391497 >> v1:172.21.15.1:6789/0 conn(0x7ff99806e6d0 legacy=0x7ff99806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.058+0000 7ff99df07640 1 -- 172.21.15.1:0/1869391497 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980ce980 con 0x7ff998163c70 2024-05-09T06:32:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99df07640 1 --2- 172.21.15.1:0/1869391497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 0x7ff998166060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff988009dc0 tx=0x7ff98802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d606677924dc8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988007740 con 0x7ff998163c70 2024-05-09T06:32:32.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9880078e0 con 0x7ff998163c70 2024-05-09T06:32:32.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff997fff640 1 -- 172.21.15.1:0/1869391497 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988007c90 con 0x7ff998163c70 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 -- 172.21.15.1:0/1869391497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 msgr2=0x7ff998166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1869391497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 0x7ff998166060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff988009dc0 tx=0x7ff98802a6d0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 -- 172.21.15.1:0/1869391497 shutdown_connections 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1869391497 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff998163c70 0x7ff998166060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 -- 172.21.15.1:0/1869391497 >> 172.21.15.1:0/1869391497 conn(0x7ff9980cc4b0 msgr2=0x7ff9980cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 -- 172.21.15.1:0/1869391497 shutdown_connections 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.059+0000 7ff99f96a640 1 -- 172.21.15.1:0/1869391497 wait complete. 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.060+0000 7ff99f96a640 1 Processor -- start 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 -- start start 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9980cf180 0x7ff99815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff998163c70 0x7ff9980b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981447c0 con 0x7ff99806e6d0 2024-05-09T06:32:32.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff998070ed0 con 0x7ff998163c70 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99f96a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff998144940 con 0x7ff9980cf180 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99d706640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99d706640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50322/0 (socket says 172.21.15.1:50322) 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.061+0000 7ff99d706640 1 -- 172.21.15.1:0/1907244334 learned_addr learned my addr 172.21.15.1:0/1907244334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99cf05640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9980cf180 0x7ff99815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99df07640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff998163c70 0x7ff9980b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 -- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9980cf180 msgr2=0x7ff99815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9980cf180 0x7ff99815fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 -- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff998163c70 msgr2=0x7ff9980b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff998163c70 0x7ff9980b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 -- 172.21.15.1:0/1907244334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99815add0 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99d706640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c410 tx=0x7ff99000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990017070 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990005ce0 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9980af0c0 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900072d0 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.062+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff998143870 con 0x7ff99806e6d0 2024-05-09T06:32:32.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.063+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980752b0 con 0x7ff99806e6d0 2024-05-09T06:32:32.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.064+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff99001a070 con 0x7ff99806e6d0 2024-05-09T06:32:32.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.064+0000 7ff995ffb640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 0x7ff98406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.064+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7ff99008a3f0 con 0x7ff99806e6d0 2024-05-09T06:32:32.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.064+0000 7ff99cf05640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 0x7ff98406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.065+0000 7ff99cf05640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 0x7ff98406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.069+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9900584f0 con 0x7ff99806e6d0 2024-05-09T06:32:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.267+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9980728c0 con 0x7ff99806e6d0 2024-05-09T06:32:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.268+0000 7ff995ffb640 1 -- 172.21.15.1:0/1907244334 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) ==== 80+0+120 (secure 0 0 0) 0x7ff99005c7f0 con 0x7ff99806e6d0 2024-05-09T06:32:32.249 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6834/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:32:32.249 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6832/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:32:32.250 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:32:32.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.269+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 msgr2=0x7ff98406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.269+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 0x7ff98406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c009040 comp rx=0 tx=0).stop 2024-05-09T06:32:32.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.269+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 msgr2=0x7ff99815a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.269+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c410 tx=0x7ff99000c8d0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 shutdown_connections 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff98406be30 0x7ff98406e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff998163c70 0x7ff9980b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9980cf180 0x7ff99815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 --2- 172.21.15.1:0/1907244334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff99806e6d0 0x7ff99815a890 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 >> 172.21.15.1:0/1907244334 conn(0x7ff9980cc4b0 msgr2=0x7ff9980cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 shutdown_connections 2024-05-09T06:32:32.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.270+0000 7ff99f96a640 1 -- 172.21.15.1:0/1907244334 wait complete. 2024-05-09T06:32:32.262 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:32:32.262 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:32:32.279 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:32:32.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.442+0000 7f6eda14f640 1 Processor -- start 2024-05-09T06:32:32.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.442+0000 7f6eda14f640 1 -- start start 2024-05-09T06:32:32.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6eda14f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ed406e4f0 0x7f6ed406e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6eda14f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed415eb40 con 0x7f6ed40ae2a0 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6eda14f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed40aef90 con 0x7f6ed415e050 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6eda14f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed40b3b80 con 0x7f6ed406e4f0 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed2ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ed406e4f0 0x7f6ed406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6ed40ae2a0 0x7f6ed40ae6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37492/0 (socket says 172.21.15.1:37492) 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed3fff640 1 -- 172.21.15.1:0/2039051729 learned_addr learned my addr 172.21.15.1:0/2039051729 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed2ffd640 1 -- 172.21.15.1:0/2039051729 >> v1:172.21.15.6:6789/0 conn(0x7f6ed415e050 legacy=0x7f6ed415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed2ffd640 1 -- 172.21.15.1:0/2039051729 >> v1:172.21.15.1:6789/0 conn(0x7f6ed40ae2a0 legacy=0x7f6ed40ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed2ffd640 1 -- 172.21.15.1:0/2039051729 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40b3530 con 0x7f6ed406e4f0 2024-05-09T06:32:32.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.443+0000 7f6ed27fc640 1 -- 172.21.15.1:0/2039051729 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348605621 0 0) 0x7f6ed415eb40 con 0x7f6ed40ae2a0 2024-05-09T06:32:32.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6ed2ffd640 1 --2- 172.21.15.1:0/2039051729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed406e980 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80097a0 tx=0x7f6ec802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44423abb9f343950 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6ed27fc640 1 -- 172.21.15.1:0/2039051729 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8037070 con 0x7f6ed406e4f0 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6ed27fc640 1 -- 172.21.15.1:0/2039051729 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec8032440 con 0x7f6ed406e4f0 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6ed27fc640 1 -- 172.21.15.1:0/2039051729 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8032810 con 0x7f6ed406e4f0 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6eda14f640 1 -- 172.21.15.1:0/2039051729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 msgr2=0x7f6ed406e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6eda14f640 1 --2- 172.21.15.1:0/2039051729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed406e980 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80097a0 tx=0x7f6ec802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6eda14f640 1 -- 172.21.15.1:0/2039051729 shutdown_connections 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.444+0000 7f6eda14f640 1 --2- 172.21.15.1:0/2039051729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed406e980 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.445+0000 7f6eda14f640 1 -- 172.21.15.1:0/2039051729 >> 172.21.15.1:0/2039051729 conn(0x7f6ed406d270 msgr2=0x7f6ed406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.445+0000 7f6eda14f640 1 -- 172.21.15.1:0/2039051729 shutdown_connections 2024-05-09T06:32:32.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.445+0000 7f6eda14f640 1 -- 172.21.15.1:0/2039051729 wait complete. 2024-05-09T06:32:32.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.446+0000 7f6eda14f640 1 Processor -- start 2024-05-09T06:32:32.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.446+0000 7f6eda14f640 1 -- start start 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 0x7f6ed41e6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 0x7f6ed4147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed40b4190 con 0x7f6ed406e4f0 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed40b4010 con 0x7f6ed40ae2a0 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6eda14f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed40b4310 con 0x7f6ed415e050 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50340/0 (socket says 172.21.15.1:50340) 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 -- 172.21.15.1:0/230492339 learned_addr learned my addr 172.21.15.1:0/230492339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed2ffd640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 0x7f6ed41e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed3fff640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 0x7f6ed4147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 -- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 msgr2=0x7f6ed4147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 0x7f6ed4147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 -- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 msgr2=0x7f6ed41e6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 0x7f6ed41e6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 -- 172.21.15.1:0/230492339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40b2050 con 0x7f6ed406e4f0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed2ffd640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 0x7f6ed41e6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed3fff640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 0x7f6ed4147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.447+0000 7f6ed37fe640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b420 tx=0x7f6ec400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.448+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4011830 con 0x7f6ed406e4f0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.448+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4016070 con 0x7f6ed406e4f0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.448+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed40b2330 con 0x7f6ed406e4f0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.448+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4011c10 con 0x7f6ed406e4f0 2024-05-09T06:32:32.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.448+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed40b28f0 con 0x7f6ed406e4f0 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.449+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed40752b0 con 0x7f6ed406e4f0 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.449+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ec401b460 con 0x7f6ed406e4f0 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.450+0000 7f6ebbfff640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.450+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f6ec408aa90 con 0x7f6ed406e4f0 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.450+0000 7f6ed2ffd640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.450+0000 7f6ed2ffd640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 0x7f6ec006e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8034000 tx=0x7f6ec802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.454+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ec4020020 con 0x7f6ed406e4f0 2024-05-09T06:32:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.642+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ed415e4d0 con 0x7f6ed406e4f0 2024-05-09T06:32:32.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.643+0000 7f6ebbfff640 1 -- 172.21.15.1:0/230492339 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7f6ec40592f0 con 0x7f6ed406e4f0 2024-05-09T06:32:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:32:32.625 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 msgr2=0x7f6ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 0x7f6ec006e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8034000 tx=0x7f6ec802e000 comp rx=0 tx=0).stop 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 msgr2=0x7f6ed415b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b420 tx=0x7f6ec400b8e0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 shutdown_connections 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ed415e050 0x7f6ed4147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed40ae2a0 0x7f6ed41e6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 --2- 172.21.15.1:0/230492339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed406e4f0 0x7f6ed415b660 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 >> 172.21.15.1:0/230492339 conn(0x7f6ed406d270 msgr2=0x7f6ed415f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 shutdown_connections 2024-05-09T06:32:32.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.645+0000 7f6eda14f640 1 -- 172.21.15.1:0/230492339 wait complete. 2024-05-09T06:32:32.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:32:32.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:32:32.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.763+0000 7f0e42bfb640 1 Processor -- start 2024-05-09T06:32:32.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.763+0000 7f0e42bfb640 1 -- start start 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e42bfb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 0x7f0e3c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e42bfb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 0x7f0e3c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e42bfb640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c0b1340 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e42bfb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3c0cc260 con 0x7f0e3c0d08b0 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e42bfb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c0d1020 con 0x7f0e3c15e270 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e41198640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0e3c0ce9d0 0x7f0e3c06c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37508/0 (socket says 172.21.15.1:37508) 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e41198640 1 -- 172.21.15.1:0/2882260542 learned_addr learned my addr 172.21.15.1:0/2882260542 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:32.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e3bfff640 1 --2- 172.21.15.1:0/2882260542 >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 0x7f0e3c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.764+0000 7f0e40997640 1 --2- 172.21.15.1:0/2882260542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 0x7f0e3c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579442322 0 0) 0x7f0e34001490 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e24003520 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1826999607 0 0) 0x7f0e24003520 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e40997640 1 -- 172.21.15.1:0/2882260542 >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 msgr2=0x7f0e3c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e40997640 1 --2- 172.21.15.1:0/2882260542 >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 0x7f0e3c0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e40997640 1 -- 172.21.15.1:0/2882260542 >> v1:172.21.15.1:6789/0 conn(0x7f0e3c0ce9d0 legacy=0x7f0e3c06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e40997640 1 -- 172.21.15.1:0/2882260542 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c0cc4b0 con 0x7f0e3c15e270 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.765+0000 7f0e3bfff640 1 --2- 172.21.15.1:0/2882260542 >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 0x7f0e3c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e40997640 1 --2- 172.21.15.1:0/2882260542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 0x7f0e3c0cf180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0097a0 tx=0x7f0e2c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4ff96591c0dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037070 con 0x7f0e3c15e270 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02ac30 con 0x7f0e3c15e270 2024-05-09T06:32:32.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e3b7fe640 1 -- 172.21.15.1:0/2882260542 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34001bc0 con 0x7f0e3c15e270 2024-05-09T06:32:32.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2882260542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 msgr2=0x7f0e3c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2882260542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 0x7f0e3c0cf180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0097a0 tx=0x7f0e2c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2882260542 shutdown_connections 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2882260542 >> v2:172.21.15.6:3300/0 conn(0x7f0e3c0d08b0 0x7f0e3c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2882260542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c15e270 0x7f0e3c0cf180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2882260542 >> 172.21.15.1:0/2882260542 conn(0x7f0e3c0d1c10 msgr2=0x7f0e3c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.766+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2882260542 shutdown_connections 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.767+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2882260542 wait complete. 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.768+0000 7f0e42bfb640 1 Processor -- start 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.768+0000 7f0e42bfb640 1 -- start start 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c0d08b0 0x7f0e3c149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3c15e270 0x7f0e3c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c15e650 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c0cca70 con 0x7f0e3c15e270 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c0d0cf0 con 0x7f0e3c0d08b0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50348/0 (socket says 172.21.15.1:50348) 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 -- 172.21.15.1:0/2622346263 learned_addr learned my addr 172.21.15.1:0/2622346263 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e41198640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3c15e270 0x7f0e3c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e3bfff640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c0d08b0 0x7f0e3c149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 -- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c0d08b0 msgr2=0x7f0e3c149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c0d08b0 0x7f0e3c149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 -- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3c15e270 msgr2=0x7f0e3c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3c15e270 0x7f0e3c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 -- 172.21.15.1:0/2622346263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c14a500 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e40997640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c035040 tx=0x7f0e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c041020 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2c03d620 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c0c7670 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037040 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.769+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c0c7c20 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.770+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c0c96d0 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.771+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e2c044020 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.771+0000 7f0e397fa640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 0x7f0e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.771+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f0e2c0ab4d0 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.772+0000 7f0e3bfff640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 0x7f0e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.772+0000 7f0e3bfff640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 0x7f0e2806e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0af680 tx=0x7f0e30009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.776+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e2c04b030 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.962+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e3c0752b0 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.963+0000 7f0e397fa640 1 -- 172.21.15.1:0/2622346263 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f0e2c0795c0 con 0x7f0e3c0ce9d0 2024-05-09T06:32:32.944 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:32:32.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 msgr2=0x7f0e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 0x7f0e2806e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0af680 tx=0x7f0e30009190 comp rx=0 tx=0).stop 2024-05-09T06:32:32.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 msgr2=0x7f0e3c153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c035040 tx=0x7f0e2c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 shutdown_connections 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0e2806be30 0x7f0e2806e2f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3c15e270 0x7f0e3c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e3c0d08b0 0x7f0e3c149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 --2- 172.21.15.1:0/2622346263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e3c0ce9d0 0x7f0e3c153740 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 >> 172.21.15.1:0/2622346263 conn(0x7f0e3c0d1c10 msgr2=0x7f0e3c0aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 shutdown_connections 2024-05-09T06:32:32.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:32.965+0000 7f0e42bfb640 1 -- 172.21.15.1:0/2622346263 wait complete. 2024-05-09T06:32:32.957 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:32:33.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.083+0000 7fce17b90640 1 Processor -- start 2024-05-09T06:32:33.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.083+0000 7fce17b90640 1 -- start start 2024-05-09T06:32:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce17b90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce100cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce17b90640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fce100cdb90 0x7fce1006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce17b90640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce100cf1b0 con 0x7fce1015a4d0 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce17b90640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fce1015a8b0 con 0x7fce100cdb90 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce17b90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce1006dc10 con 0x7fce10071610 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1612d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fce100cdb90 0x7fce1006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1592c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fce1015a4d0 0x7fce10070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:60346/0 (socket says 172.21.15.1:60346) 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1592c640 1 -- 172.21.15.1:0/3161763827 learned_addr learned my addr 172.21.15.1:0/3161763827 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1512b640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce100cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1612d640 1 -- 172.21.15.1:0/3161763827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 msgr2=0x7fce100cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1612d640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce100cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1612d640 1 -- 172.21.15.1:0/3161763827 >> v1:172.21.15.6:6789/0 conn(0x7fce1015a4d0 legacy=0x7fce10070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1612d640 1 -- 172.21.15.1:0/3161763827 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100cf490 con 0x7fce100cdb90 2024-05-09T06:32:33.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.084+0000 7fce1492a640 1 -- 172.21.15.1:0/3161763827 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491315977 0 0) 0x7fce100cf1b0 con 0x7fce1015a4d0 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.085+0000 7fce1612d640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce100cdb90 0x7fce1006e0a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6a89720524848f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.085+0000 7fce1492a640 1 -- 172.21.15.1:0/3161763827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08007740 con 0x7fce100cdb90 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.085+0000 7fce1492a640 1 -- 172.21.15.1:0/3161763827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce080078e0 con 0x7fce100cdb90 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.085+0000 7fce1492a640 1 -- 172.21.15.1:0/3161763827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08007cb0 con 0x7fce100cdb90 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 -- 172.21.15.1:0/3161763827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce100cdb90 msgr2=0x7fce1006e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce100cdb90 0x7fce1006e0a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0802a660 comp rx=0 tx=0).stop 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 -- 172.21.15.1:0/3161763827 shutdown_connections 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce100cdb90 0x7fce1006e0a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 --2- 172.21.15.1:0/3161763827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce100cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 -- 172.21.15.1:0/3161763827 >> 172.21.15.1:0/3161763827 conn(0x7fce100cc4d0 msgr2=0x7fce100cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 -- 172.21.15.1:0/3161763827 shutdown_connections 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.086+0000 7fce17b90640 1 -- 172.21.15.1:0/3161763827 wait complete. 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.087+0000 7fce17b90640 1 Processor -- start 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.087+0000 7fce17b90640 1 -- start start 2024-05-09T06:32:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce10166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce100cdb90 0x7fce101668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fce100b2be0 con 0x7fce1015a4d0 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fce100b2a60 con 0x7fce100cdb90 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce17b90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce1006cd30 con 0x7fce10071610 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50358/0 (socket says 172.21.15.1:50358) 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 -- 172.21.15.1:0/2292032677 learned_addr learned my addr 172.21.15.1:0/2292032677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1512b640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce100cdb90 0x7fce101668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1592c640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce10166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 -- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 msgr2=0x7fce10166370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce10166370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 -- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce100cdb90 msgr2=0x7fce101668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce100cdb90 0x7fce101668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 -- 172.21.15.1:0/2292032677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1014a180 con 0x7fce1015a4d0 2024-05-09T06:32:33.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1592c640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce10166370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce1612d640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fce08009eb0 tx=0x7fce08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.088+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08040070 con 0x7fce1015a4d0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.089+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce08031bc0 con 0x7fce1015a4d0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.089+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce101493c0 con 0x7fce1015a4d0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.089+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08009450 con 0x7fce1015a4d0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.089+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce10149980 con 0x7fce1015a4d0 2024-05-09T06:32:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.089+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce100752b0 con 0x7fce1015a4d0 2024-05-09T06:32:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.090+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce08039040 con 0x7fce1015a4d0 2024-05-09T06:32:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.090+0000 7fce067fc640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 0x7fcdf406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.090+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fce080ac470 con 0x7fce1015a4d0 2024-05-09T06:32:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.090+0000 7fce1592c640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 0x7fcdf406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.091+0000 7fce1592c640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 0x7fcdf406e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce0c000f60 tx=0x7fce0c007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.095+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce0804a060 con 0x7fce1015a4d0 2024-05-09T06:32:33.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.283+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce10150f20 con 0x7fce1015a4d0 2024-05-09T06:32:33.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.283+0000 7fce067fc640 1 -- 172.21.15.1:0/2292032677 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fce0807acd0 con 0x7fce1015a4d0 2024-05-09T06:32:33.265 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 msgr2=0x7fcdf406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 0x7fcdf406e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce0c000f60 tx=0x7fce0c007ca0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 msgr2=0x7fce1014ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fce08009eb0 tx=0x7fce08004a40 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 shutdown_connections 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdf406be60 0x7fcdf406e320 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce1015a4d0 0x7fce1014ab20 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce100cdb90 0x7fce101668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 --2- 172.21.15.1:0/2292032677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce10071610 0x7fce10166370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 >> 172.21.15.1:0/2292032677 conn(0x7fce100cc4d0 msgr2=0x7fce100b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 shutdown_connections 2024-05-09T06:32:33.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.285+0000 7fce17b90640 1 -- 172.21.15.1:0/2292032677 wait complete. 2024-05-09T06:32:33.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:32:33.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.403+0000 7fe62b909640 1 Processor -- start 2024-05-09T06:32:33.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.403+0000 7fe62b909640 1 -- start start 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe62b909640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe62415c4c0 0x7fe62406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe62b909640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62406cf40 con 0x7fe6240b4de0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe62b909640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62406d220 con 0x7fe62415cfb0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe62b909640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe624156980 con 0x7fe62415c4c0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe628ea4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe6240b4de0 0x7fe62415c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37524/0 (socket says 172.21.15.1:37524) 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe628ea4640 1 -- 172.21.15.1:0/3975722974 learned_addr learned my addr 172.21.15.1:0/3975722974 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3975722974 >> v2:172.21.15.6:3300/0 conn(0x7fe62415c4c0 0x7fe62406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887286000 0 0) 0x7fe62406cf40 con 0x7fe6240b4de0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe610003520 con 0x7fe6240b4de0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1018227740 0 0) 0x7fe610003520 con 0x7fe6240b4de0 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 >> v2:172.21.15.6:3300/0 conn(0x7fe62415c4c0 msgr2=0x7fe62406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 --2- 172.21.15.1:0/3975722974 >> v2:172.21.15.6:3300/0 conn(0x7fe62415c4c0 0x7fe62406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 >> v1:172.21.15.6:6789/0 conn(0x7fe62415cfb0 legacy=0x7fe624070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62406d500 con 0x7fe6240b4de0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fe614001670 con 0x7fe6240b4de0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.404+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 >> v1:172.21.15.1:6789/0 conn(0x7fe6240b4de0 legacy=0x7fe62415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 0x7fe610003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe610004800 0x7fe610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6100051c0 0x7fe61000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe61000c130 con 0x7fe6100039b0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe61000c2e0 con 0x7fe6100051c0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe61000c4e0 con 0x7fe610004800 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 0x7fe610003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe629ea6640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6100051c0 0x7fe61000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe628ea4640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe610004800 0x7fe610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 -- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe610004800 msgr2=0x7fe610004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe610004800 0x7fe610004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 -- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6100051c0 msgr2=0x7fe61000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6100051c0 0x7fe61000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe6296a5640 1 -- 172.21.15.1:0/3975722974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe610003520 con 0x7fe6100039b0 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe628ea4640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe610004800 0x7fe610004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe629ea6640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6100051c0 0x7fe61000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.405+0000 7fe629ea6640 1 -- 172.21.15.1:0/3975722974 reap_dead start 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 0x7fe610003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe620009ac0 tx=0x7fe610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb5901f693f8e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620036070 con 0x7fe6100039b0 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe620031400 con 0x7fe6100039b0 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe61bfff640 1 -- 172.21.15.1:0/3975722974 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620009690 con 0x7fe6100039b0 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 -- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 msgr2=0x7fe610003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 0x7fe610003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe620009ac0 tx=0x7fe610005830 comp rx=0 tx=0).stop 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 -- 172.21.15.1:0/3975722974 shutdown_connections 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 --2- 172.21.15.1:0/3975722974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6100039b0 0x7fe610003dd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 -- 172.21.15.1:0/3975722974 >> 172.21.15.1:0/3975722974 conn(0x7fe6240cc4d0 msgr2=0x7fe6240cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.406+0000 7fe62b909640 1 -- 172.21.15.1:0/3975722974 shutdown_connections 2024-05-09T06:32:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.407+0000 7fe62b909640 1 -- 172.21.15.1:0/3975722974 wait complete. 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 Processor -- start 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 -- start start 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6100039b0 0x7fe62415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 0x7fe6241f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6240b5260 con 0x7fe6240aeb60 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe62406d870 con 0x7fe6241f4b90 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.408+0000 7fe62b909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240b53e0 con 0x7fe6100039b0 2024-05-09T06:32:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50382/0 (socket says 172.21.15.1:50382) 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 -- 172.21.15.1:0/3040422738 learned_addr learned my addr 172.21.15.1:0/3040422738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6100039b0 0x7fe62415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe629ea6640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 0x7fe6241f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 -- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6100039b0 msgr2=0x7fe62415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6100039b0 0x7fe62415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 -- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 msgr2=0x7fe6241f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 0x7fe6241f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 -- 172.21.15.1:0/3040422738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe624141290 con 0x7fe6240aeb60 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe629ea6640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 0x7fe6241f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe628ea4640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e780 tx=0x7fe61400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61400ee30 con 0x7fe6240aeb60 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614018070 con 0x7fe6240aeb60 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6241f58c0 con 0x7fe6240aeb60 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61400ca70 con 0x7fe6240aeb60 2024-05-09T06:32:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.409+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe624141830 con 0x7fe6240aeb60 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.410+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe624141a30 con 0x7fe6240aeb60 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.411+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe61400cc10 con 0x7fe6240aeb60 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.411+0000 7fe619ffb640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 0x7fe60806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.412+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fe61408aa60 con 0x7fe6240aeb60 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.412+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 0x7fe60806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.412+0000 7fe6296a5640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 0x7fe60806e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe620033000 tx=0x7fe620008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.416+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe624141a30 con 0x7fe6240aeb60 2024-05-09T06:32:33.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.602+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe6240752b0 con 0x7fe6240aeb60 2024-05-09T06:32:33.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.603+0000 7fe619ffb640 1 -- 172.21.15.1:0/3040422738 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fe614059230 con 0x7fe6240aeb60 2024-05-09T06:32:33.584 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:32:33.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 msgr2=0x7fe60806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 0x7fe60806e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe620033000 tx=0x7fe620008030 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 msgr2=0x7fe6240aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e780 tx=0x7fe61400ec40 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 shutdown_connections 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe60806be60 0x7fe60806e320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6241f4b90 0x7fe6241f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6240aeb60 0x7fe6240aefe0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 --2- 172.21.15.1:0/3040422738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6100039b0 0x7fe62415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 >> 172.21.15.1:0/3040422738 conn(0x7fe6240cc4d0 msgr2=0x7fe62415ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 shutdown_connections 2024-05-09T06:32:33.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.605+0000 7fe62b909640 1 -- 172.21.15.1:0/3040422738 wait complete. 2024-05-09T06:32:33.597 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:32:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.720+0000 7f8bd75b7640 1 Processor -- start 2024-05-09T06:32:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.720+0000 7f8bd75b7640 1 -- start start 2024-05-09T06:32:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd75b7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8bd00ce4d0 0x7f8bd00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd75b7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8bd015e340 0x7f8bd00cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd75b7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd006d070 con 0x7f8bd00ccb40 2024-05-09T06:32:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd75b7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd00cf330 con 0x7f8bd015e340 2024-05-09T06:32:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd75b7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd006d350 con 0x7f8bd00ce4d0 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd4b52640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8bd015e340 0x7f8bd00cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd5b54640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8bd00ccb40 0x7f8bd00ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37532/0 (socket says 172.21.15.1:37532) 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd5b54640 1 -- 172.21.15.1:0/1430858522 learned_addr learned my addr 172.21.15.1:0/1430858522 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd5353640 1 --2- 172.21.15.1:0/1430858522 >> v2:172.21.15.6:3300/0 conn(0x7f8bd00ce4d0 0x7f8bd00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd4b52640 1 -- 172.21.15.1:0/1430858522 >> v2:172.21.15.6:3300/0 conn(0x7f8bd00ce4d0 msgr2=0x7f8bd00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd4b52640 1 --2- 172.21.15.1:0/1430858522 >> v2:172.21.15.6:3300/0 conn(0x7f8bd00ce4d0 0x7f8bd00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd4b52640 1 -- 172.21.15.1:0/1430858522 >> v1:172.21.15.1:6789/0 conn(0x7f8bd00ccb40 legacy=0x7f8bd00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bd4b52640 1 -- 172.21.15.1:0/1430858522 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd00cc340 con 0x7f8bd015e340 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.721+0000 7f8bc7fff640 1 -- 172.21.15.1:0/1430858522 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190453092 0 0) 0x7f8bd006d070 con 0x7f8bd00ccb40 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.722+0000 7f8bd4b52640 1 --2- 172.21.15.1:0/1430858522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd015e340 0x7f8bd00cedf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072a90 tx=0x7f8bc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845c5f5b0781749e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.722+0000 7f8bc7fff640 1 -- 172.21.15.1:0/1430858522 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0037070 con 0x7f8bd015e340 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.722+0000 7f8bc7fff640 1 -- 172.21.15.1:0/1430858522 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc0032440 con 0x7f8bd015e340 2024-05-09T06:32:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.722+0000 7f8bc7fff640 1 -- 172.21.15.1:0/1430858522 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0032810 con 0x7f8bd015e340 2024-05-09T06:32:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 -- 172.21.15.1:0/1430858522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd015e340 msgr2=0x7f8bd00cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/1430858522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd015e340 0x7f8bd00cedf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072a90 tx=0x7f8bc002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 -- 172.21.15.1:0/1430858522 shutdown_connections 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/1430858522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd015e340 0x7f8bd00cedf0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/1430858522 >> v2:172.21.15.6:3300/0 conn(0x7f8bd00ce4d0 0x7f8bd00ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 -- 172.21.15.1:0/1430858522 >> 172.21.15.1:0/1430858522 conn(0x7f8bd00d1c50 msgr2=0x7f8bd00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 -- 172.21.15.1:0/1430858522 shutdown_connections 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.723+0000 7f8bd75b7640 1 -- 172.21.15.1:0/1430858522 wait complete. 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.724+0000 7f8bd75b7640 1 Processor -- start 2024-05-09T06:32:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.724+0000 7f8bd75b7640 1 -- start start 2024-05-09T06:32:33.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bd00ce4d0 0x7f8bd01f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 0x7f8bd00b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006cd00 con 0x7f8bd00ccb40 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006d630 con 0x7f8bd00ce4d0 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd75b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd00d0ea0 con 0x7f8bd00ca040 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50408/0 (socket says 172.21.15.1:50408) 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 -- 172.21.15.1:0/2571051619 learned_addr learned my addr 172.21.15.1:0/2571051619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd4b52640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bd00ce4d0 0x7f8bd01f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5b54640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 0x7f8bd00b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 -- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 msgr2=0x7f8bd00b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 0x7f8bd00b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 -- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bd00ce4d0 msgr2=0x7f8bd01f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.725+0000 7f8bd5353640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bd00ce4d0 0x7f8bd01f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bd5353640 1 -- 172.21.15.1:0/2571051619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd014dd10 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bd5b54640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 0x7f8bd00b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bd5353640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d830 tx=0x7f8bcc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00b660 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc019070 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd014dff0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00ba20 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.726+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd014e5b0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.727+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd00752b0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.728+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bcc01e460 con 0x7f8bd00ccb40 2024-05-09T06:32:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.728+0000 7f8bc5ffb640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 0x7f8bb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.728+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f8bcc08b610 con 0x7f8bd00ccb40 2024-05-09T06:32:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.728+0000 7f8bd4b52640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 0x7f8bb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.729+0000 7f8bd4b52640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 0x7f8bb406e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0034000 tx=0x7f8bc002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.732+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bcc059de0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.927+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bd00b34f0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.927+0000 7f8bc5ffb640 1 -- 172.21.15.1:0/2571051619 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) ==== 80+0+120 (secure 0 0 0) 0x7f8bcc0596e0 con 0x7f8bd00ccb40 2024-05-09T06:32:33.908 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6834/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:32:33.909 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6832/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:32:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 msgr2=0x7f8bb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 0x7f8bb406e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0034000 tx=0x7f8bc002e000 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 msgr2=0x7f8bd014b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d830 tx=0x7f8bcc00dcf0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 shutdown_connections 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8bb406be60 0x7f8bb406e320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bd00ca040 0x7f8bd00b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8bd00ce4d0 0x7f8bd01f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 --2- 172.21.15.1:0/2571051619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd00ccb40 0x7f8bd014b1a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 >> 172.21.15.1:0/2571051619 conn(0x7f8bd00d1c50 msgr2=0x7f8bd015c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 shutdown_connections 2024-05-09T06:32:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:33.929+0000 7f8bd75b7640 1 -- 172.21.15.1:0/2571051619 wait complete. 2024-05-09T06:32:33.921 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:32:33.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:33.921 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:32:33.945 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[63818]: fuse finished with error 0 and tester_r 0 2024-05-09T06:32:33.947 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:32:38.502 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:32:38.522+0000 7f5fea9bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T06:32:40.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:32:40.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:40.050 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:32:40.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:40.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.251+0000 7f5d5e7f9640 1 Processor -- start 2024-05-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.251+0000 7f5d5e7f9640 1 -- start start 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58143bb0 0x7f5d580cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d580d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5d58144110 0x7f5d5815e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5d580ce8d0 con 0x7f5d58143bb0 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5d580cdd50 con 0x7f5d58144110 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5e7f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d580ce030 con 0x7f5d581446d0 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d57fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58143bb0 0x7f5d580cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d57fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58143bb0 0x7f5d580cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60570/0 (socket says 172.21.15.1:60570) 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d57fff640 1 -- 172.21.15.1:0/1782262591 learned_addr learned my addr 172.21.15.1:0/1782262591 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.252+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/1782262591 >> v2:172.21.15.6:3300/0 conn(0x7f5d58144110 0x7f5d5815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 -- 172.21.15.1:0/1782262591 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 msgr2=0x7f5d580d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d580d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d580d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 -- 172.21.15.1:0/1782262591 >> v2:172.21.15.6:3300/0 conn(0x7f5d58144110 msgr2=0x7f5d5815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1782262591 >> v2:172.21.15.6:3300/0 conn(0x7f5d58144110 0x7f5d5815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 -- 172.21.15.1:0/1782262591 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580ce310 con 0x7f5d58143bb0 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d580cfcb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5d5815e8c0 tx=0x7f5d4802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d45f9d1454c493 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.253+0000 7f5d56ffd640 1 -- 172.21.15.1:0/1782262591 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48037070 con 0x7f5d58143bb0 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d56ffd640 1 -- 172.21.15.1:0/1782262591 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d48032440 con 0x7f5d58143bb0 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d56ffd640 1 -- 172.21.15.1:0/1782262591 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48032810 con 0x7f5d58143bb0 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1782262591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 msgr2=0x7f5d580cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d580cfcb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5d5815e8c0 tx=0x7f5d4802a140 comp rx=0 tx=0).stop 2024-05-09T06:32:40.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1782262591 shutdown_connections 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1782262591 >> v2:172.21.15.6:3300/0 conn(0x7f5d58144110 0x7f5d5815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d580d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1782262591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d580cfcb0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1782262591 >> 172.21.15.1:0/1782262591 conn(0x7f5d5806b3c0 msgr2=0x7f5d5806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1782262591 shutdown_connections 2024-05-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.254+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1782262591 wait complete. 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.255+0000 7f5d5e7f9640 1 Processor -- start 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 -- start start 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58144110 0x7f5d58141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d58158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580cdaf0 con 0x7f5d58143bb0 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580cd970 con 0x7f5d58144110 2024-05-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d5e7f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5815eea0 con 0x7f5d581446d0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.256+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60582/0 (socket says 172.21.15.1:60582) 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 -- 172.21.15.1:0/1616317550 learned_addr learned my addr 172.21.15.1:0/1616317550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58144110 0x7f5d58141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d58158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 -- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 msgr2=0x7f5d58158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d58158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 -- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58144110 msgr2=0x7f5d58141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58144110 0x7f5d58141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 -- 172.21.15.1:0/1616317550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d581f4530 con 0x7f5d58143bb0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5d4802a180 tx=0x7f5d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d54ff9640 1 -- 172.21.15.1:0/1616317550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48037040 con 0x7f5d58143bb0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d54ff9640 1 -- 172.21.15.1:0/1616317550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d48042070 con 0x7f5d58143bb0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d58132d10 con 0x7f5d58143bb0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d54ff9640 1 -- 172.21.15.1:0/1616317550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48030bb0 con 0x7f5d58143bb0 2024-05-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.257+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d58133290 con 0x7f5d58143bb0 2024-05-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.259+0000 7f5d54ff9640 1 -- 172.21.15.1:0/1616317550 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d48047460 con 0x7f5d58143bb0 2024-05-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.260+0000 7f5d54ff9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.260+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.260+0000 7f5d54ff9640 1 -- 172.21.15.1:0/1616317550 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f5d480abc30 con 0x7f5d58143bb0 2024-05-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.260+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c006fd0 tx=0x7f5d4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 Processor -- start 2024-05-09T06:32:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 -- start start 2024-05-09T06:32:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x5622193bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ebec0 0x5622193c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562219324f50 con 0x5622193ec600 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622192cc510 con 0x5622194094e0 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.261+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562219320f10 con 0x5622193ebec0 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60588/0 (socket says 172.21.15.1:60588) 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 -- 172.21.15.1:0/2532987473 learned_addr learned my addr 172.21.15.1:0/2532987473 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d57fff640 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x5622193bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ebec0 0x5622193c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 -- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ebec0 msgr2=0x5622193c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ebec0 0x5622193c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 -- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 msgr2=0x5622193bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x5622193bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 -- 172.21.15.1:0/2532987473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562219104a20 con 0x5622193ec600 2024-05-09T06:32:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d577fe640 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00ebd0 tx=0x7f5d4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1729d38342567 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d3dffb640 1 -- 172.21.15.1:0/2532987473 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01a050 con 0x5622193ec600 2024-05-09T06:32:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d3dffb640 1 -- 172.21.15.1:0/2532987473 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c024070 con 0x5622193ec600 2024-05-09T06:32:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.262+0000 7f5d3dffb640 1 -- 172.21.15.1:0/2532987473 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01f440 con 0x5622193ec600 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- 172.21.15.1:0/2532987473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 msgr2=0x5622193eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00ebd0 tx=0x7f5d4c011000 comp rx=0 tx=0).stop 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- 172.21.15.1:0/2532987473 shutdown_connections 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ebec0 0x5622193c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193eb980 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 --2- 172.21.15.1:0/2532987473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x5622193bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- 172.21.15.1:0/2532987473 >> 172.21.15.1:0/2532987473 conn(0x5622193be370 msgr2=0x562219406980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- 172.21.15.1:0/2532987473 shutdown_connections 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- 172.21.15.1:0/2532987473 wait complete. 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.263+0000 7f5d615e2740 1 -- start start 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.264+0000 7f5d615e2740 1 Processor -- start 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.264+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x56221941a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562219416dd0 0x5622193c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622193c0290 con 0x5622193ec600 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622192dfc20 con 0x5622194094e0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5622193bf680 con 0x562219416dd0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 10 client.0 _ll_register_callbacks cb 0x56221931c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60600/0 (socket says 172.21.15.1:60600) 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 -- 172.21.15.1:0/1136882062 learned_addr learned my addr 172.21.15.1:0/1136882062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x56221941a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562219416dd0 0x5622193c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 -- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562219416dd0 msgr2=0x5622193c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562219416dd0 0x5622193c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 -- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 msgr2=0x56221941a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x56221941a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 -- 172.21.15.1:0/1136882062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562218ff8310 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d57fff640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5d480761f0 tx=0x7f5d480454a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87400ff9e0b3a0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48082970 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f5d48079ff0 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4c024070 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5d4807a1f0 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 10 client.7524 fetch_fsmap learned FSMap version 52 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.265+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5622191651b0 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c024070 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5d4803a030 con 0x5622193ec600 2024-05-09T06:32:40.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 10 client.7524 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 10 client.7524 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 20 client.7524 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5622193edaa0 0x5622193e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 4 client.7524 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562219332680 con 0x5622193edaa0 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5622193edaa0 0x5622193e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d36ffd640 10 client.7524 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.266+0000 7f5d577fe640 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5622193edaa0 0x5622193e9940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946204498c893d7e server_cookie=56d9625258280308 in_seq=0 out_seq=0 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d36ffd640 1 -- 172.21.15.1:0/1136882062 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562219332680 con 0x5622193edaa0 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d36ffd640 10 client.7524 handle_command_reply: tid=0 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 client.7524 shutdown 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 msgr2=0x562219403480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5d480761f0 tx=0x7f5d480454a0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 shutdown_connections 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5622193edaa0 0x5622193e9940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562219416dd0 0x5622193c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622194094e0 0x56221941a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 --2- 172.21.15.1:0/1136882062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x562219403480 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d36ffd640 0 client.7524 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 >> 172.21.15.1:0/1136882062 conn(0x5622193ebec0 msgr2=0x562219406980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 shutdown_connections 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 1 -- 172.21.15.1:0/1136882062 wait complete. 2024-05-09T06:32:40.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.267+0000 7f5d615e2740 20 client.7524 trim_cache size 0 max 16384 2024-05-09T06:32:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.285+0000 7f5d615e2740 1 Processor -- start 2024-05-09T06:32:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.285+0000 7f5d615e2740 1 -- start start 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ec600 0x562219406dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x5622193bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58200c20 con 0x5622194094e0 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580ae650 con 0x562219416dd0 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d580ad9b0 con 0x5622193ec600 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60614/0 (socket says 172.21.15.1:60614) 2024-05-09T06:32:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 -- 172.21.15.1:0/512361817 learned_addr learned my addr 172.21.15.1:0/512361817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 -- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ec600 msgr2=0x562219406dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ec600 0x562219406dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 -- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 msgr2=0x5622193bfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x5622193bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x5622193bfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d577fe640 1 -- 172.21.15.1:0/512361817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562218fd1490 con 0x5622194094e0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.286+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x5622193bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d577fe640 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c008010 tx=0x7f5d4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88514fcd037bef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d3d7fa640 1 -- 172.21.15.1:0/512361817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01a040 con 0x5622194094e0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d3d7fa640 1 -- 172.21.15.1:0/512361817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c024070 con 0x5622194094e0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d3d7fa640 1 -- 172.21.15.1:0/512361817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c005990 con 0x5622194094e0 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 -- 172.21.15.1:0/512361817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 msgr2=0x5622193bf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c008010 tx=0x7f5d4c013000 comp rx=0 tx=0).stop 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 -- 172.21.15.1:0/512361817 shutdown_connections 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x5622193bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622194094e0 0x5622193bf680 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 --2- 172.21.15.1:0/512361817 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622193ec600 0x562219406dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 -- 172.21.15.1:0/512361817 >> 172.21.15.1:0/512361817 conn(0x5622193ebec0 msgr2=0x5622193bbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 -- 172.21.15.1:0/512361817 shutdown_connections 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.287+0000 7f5d615e2740 1 -- 172.21.15.1:0/512361817 wait complete. 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.288+0000 7f5d615e2740 1 -- start start 2024-05-09T06:32:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.288+0000 7f5d615e2740 1 Processor -- start 2024-05-09T06:32:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 0x5622193e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x562219419cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580ae000 con 0x5622193ec600 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5622193ef2e0 con 0x562219416dd0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562219407d20 con 0x5622194094e0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d615e2740 10 client.0 _ll_register_callbacks cb 0x7f5d48079c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60618/0 (socket says 172.21.15.1:60618) 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 -- 172.21.15.1:0/4287831547 learned_addr learned my addr 172.21.15.1:0/4287831547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d577fe640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 0x5622193e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 -- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 msgr2=0x5622193e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x562219419cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 0x5622193e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 -- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 msgr2=0x562219419cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x562219419cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d57fff640 1 -- 172.21.15.1:0/4287831547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562219407890 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d577fe640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 0x5622193e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.289+0000 7f5d5cd96640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x562219419cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d57fff640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f5d48079d20 tx=0x7f5d48009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21353153d19ea1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48096050 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f5d48030d50 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4c024070 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5d48082970 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 10 client.7530 fetch_fsmap learned FSMap version 52 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56221940a980 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c024070 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5d48082bc0 con 0x5622193ec600 2024-05-09T06:32:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 10 client.7530 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 10 client.7530 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 20 client.7530 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56221941a230 0x56221941a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 4 client.7530 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x56221932c9d0 con 0x56221941a230 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.290+0000 7f5d577fe640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56221941a230 0x56221941a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.291+0000 7f5d36ffd640 10 client.7530 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.291+0000 7f5d577fe640 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56221941a230 0x56221941a640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5406e7cfd367d5 server_cookie=285c4970e442ba5d in_seq=0 out_seq=0 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.291+0000 7f5d36ffd640 1 -- 172.21.15.1:0/4287831547 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56221932c9d0 con 0x56221941a230 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.291+0000 7f5d36ffd640 10 client.7530 handle_command_reply: tid=0 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.291+0000 7f5d615e2740 1 client.7530 shutdown 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 msgr2=0x5622193c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f5d48079d20 tx=0x7f5d48009240 comp rx=0 tx=0).stop 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 shutdown_connections 2024-05-09T06:32:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562219416dd0 0x562219419cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56221941a230 0x56221941a640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5622194094e0 0x5622193e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 --2- 172.21.15.1:0/4287831547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622193ec600 0x5622193c0d50 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 >> 172.21.15.1:0/4287831547 conn(0x5622193ebec0 msgr2=0x5622193e9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d36ffd640 0 client.7530 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 shutdown_connections 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 1 -- 172.21.15.1:0/4287831547 wait complete. 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d615e2740 20 client.7530 trim_cache size 0 max 16384 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:32:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "53998ee2-f696-4b72-8a5a-c394ef710b58", 2024-05-09T06:32:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:32:40.278 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 msgr2=0x7f5d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c006fd0 tx=0x7f5d4c008040 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 msgr2=0x7f5d581f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5d4802a180 tx=0x7f5d480046c0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 shutdown_connections 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d581446d0 0x7f5d58158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58144110 0x7f5d58141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.292+0000 7f5d5e7f9640 1 --2- 172.21.15.1:0/1616317550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58143bb0 0x7f5d581f3ff0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.293+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 >> 172.21.15.1:0/1616317550 conn(0x7f5d5806b3c0 msgr2=0x7f5d580af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.293+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 shutdown_connections 2024-05-09T06:32:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.293+0000 7f5d5e7f9640 1 -- 172.21.15.1:0/1616317550 wait complete. 2024-05-09T06:32:40.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:32:40.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-09T06:32:40.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.409+0000 7fba5201c640 1 Processor -- start 2024-05-09T06:32:40.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.409+0000 7fba5201c640 1 -- start start 2024-05-09T06:32:40.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba5201c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fba4c0b9700 0x7fba4c0d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba5201c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0d0ea0 0x7fba4c0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba5201c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c0d2120 con 0x7fba4c0b9ae0 2024-05-09T06:32:40.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba5201c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fba4c0d1e40 con 0x7fba4c0b9700 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba5201c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c0d2400 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba4b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fba4c0b9700 0x7fba4c0d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba4affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fba4c0b9ae0 0x7fba4c0d0790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35994/0 (socket says 172.21.15.1:35994) 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba4b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fba4c0b9700 0x7fba4c0d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60624/0 (socket says 172.21.15.1:60624) 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.410+0000 7fba4affd640 1 -- 172.21.15.1:0/2769877862 learned_addr learned my addr 172.21.15.1:0/2769877862 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4bfff640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0d0ea0 0x7fba4c0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4b7fe640 1 -- 172.21.15.1:0/2769877862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0d0ea0 msgr2=0x7fba4c0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0d0ea0 0x7fba4c0b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4b7fe640 1 -- 172.21.15.1:0/2769877862 >> v1:172.21.15.1:6789/0 conn(0x7fba4c0b9ae0 legacy=0x7fba4c0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4a7fc640 1 -- 172.21.15.1:0/2769877862 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330102480 0 0) 0x7fba4c0d2120 con 0x7fba4c0b9ae0 2024-05-09T06:32:40.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.411+0000 7fba4b7fe640 1 -- 172.21.15.1:0/2769877862 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c0d26e0 con 0x7fba4c0b9700 2024-05-09T06:32:40.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0b9700 0x7fba4c0d49c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097a0 tx=0x7fba3c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd84be7ef5dd816c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba4a7fc640 1 -- 172.21.15.1:0/2769877862 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c037070 con 0x7fba4c0b9700 2024-05-09T06:32:40.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba4a7fc640 1 -- 172.21.15.1:0/2769877862 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3c032440 con 0x7fba4c0b9700 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba4a7fc640 1 -- 172.21.15.1:0/2769877862 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c032810 con 0x7fba4c0b9700 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 -- 172.21.15.1:0/2769877862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0b9700 msgr2=0x7fba4c0d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0b9700 0x7fba4c0d49c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097a0 tx=0x7fba3c02a210 comp rx=0 tx=0).stop 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 -- 172.21.15.1:0/2769877862 shutdown_connections 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0d0ea0 0x7fba4c0b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 --2- 172.21.15.1:0/2769877862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0b9700 0x7fba4c0d49c0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 -- 172.21.15.1:0/2769877862 >> 172.21.15.1:0/2769877862 conn(0x7fba4c0d3ea0 msgr2=0x7fba4c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 -- 172.21.15.1:0/2769877862 shutdown_connections 2024-05-09T06:32:40.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.412+0000 7fba5201c640 1 -- 172.21.15.1:0/2769877862 wait complete. 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 Processor -- start 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 -- start start 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4c0b9700 0x7fba4c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0b9ae0 0x7fba4c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c0d2ee0 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c0cfe70 con 0x7fba4c0b9700 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.414+0000 7fba5201c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c0b4a10 con 0x7fba4c0b9ae0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60636/0 (socket says 172.21.15.1:60636) 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 -- 172.21.15.1:0/623062145 learned_addr learned my addr 172.21.15.1:0/623062145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4affd640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0b9ae0 0x7fba4c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4c0b9700 0x7fba4c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 -- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0b9ae0 msgr2=0x7fba4c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0b9ae0 0x7fba4c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 -- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4c0b9700 msgr2=0x7fba4c0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4c0b9700 0x7fba4c0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 -- 172.21.15.1:0/623062145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c15ad10 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba4bfff640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fba44007d40 tx=0x7fba4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba33fff640 1 -- 172.21.15.1:0/623062145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44014420 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba33fff640 1 -- 172.21.15.1:0/623062145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba44019070 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba33fff640 1 -- 172.21.15.1:0/623062145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44014820 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c15b8f0 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.415+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c15aa30 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.417+0000 7fba33fff640 1 -- 172.21.15.1:0/623062145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba440149c0 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.417+0000 7fba33fff640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.417+0000 7fba33fff640 1 -- 172.21.15.1:0/623062145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fba4408b660 con 0x7fba4c0d0ea0 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.417+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.418+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 0x7fba3806e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fba3c034000 tx=0x7fba3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.418+0000 7fba54e09740 1 Processor -- start 2024-05-09T06:32:40.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.418+0000 7fba54e09740 1 -- start start 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea51fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea81af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5573fe9bbd00 con 0x5573fea82030 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5573fe9634c0 con 0x5573fea82770 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5573fe9b7cc0 con 0x5573fea9f650 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.419+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60640/0 (socket says 172.21.15.1:60640) 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 -- 172.21.15.1:0/3630364262 learned_addr learned my addr 172.21.15.1:0/3630364262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea51fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4affd640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea81af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 -- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 msgr2=0x5573fea51fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea51fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 -- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 msgr2=0x5573fea81af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea81af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 -- 172.21.15.1:0/3630364262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fe7997d0 con 0x5573fea82030 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4affd640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea81af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fba44089a40 tx=0x7fba44059870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d9bc61e85752d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba317fa640 1 -- 172.21.15.1:0/3630364262 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44057910 con 0x5573fea82030 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba317fa640 1 -- 172.21.15.1:0/3630364262 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba44057b70 con 0x5573fea82030 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.420+0000 7fba317fa640 1 -- 172.21.15.1:0/3630364262 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44061f10 con 0x5573fea82030 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- 172.21.15.1:0/3630364262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 msgr2=0x5573fea56e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fba44089a40 tx=0x7fba44059870 comp rx=0 tx=0).stop 2024-05-09T06:32:40.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- 172.21.15.1:0/3630364262 shutdown_connections 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea82030 0x5573fea56e40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea81af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 --2- 172.21.15.1:0/3630364262 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea51fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- 172.21.15.1:0/3630364262 >> 172.21.15.1:0/3630364262 conn(0x5573fea54250 msgr2=0x5573fea9caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- 172.21.15.1:0/3630364262 shutdown_connections 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- 172.21.15.1:0/3630364262 wait complete. 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.421+0000 7fba54e09740 1 -- start start 2024-05-09T06:32:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 Processor -- start 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573fea6c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5573fea56400 con 0x5573fea956c0 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5573fe976b90 con 0x5573fea9f650 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5573fea557f0 con 0x5573fea82770 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba54e09740 10 client.0 _ll_register_callbacks cb 0x5573fe9b2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba4bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60652/0 (socket says 172.21.15.1:60652) 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba4bfff640 1 -- 172.21.15.1:0/3354767778 learned_addr learned my addr 172.21.15.1:0/3354767778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573fea6c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.422+0000 7fba4affd640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 -- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 msgr2=0x5573fea6c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573fea6c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 -- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 msgr2=0x5573fea95180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea95180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 -- 172.21.15.1:0/3354767778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fe6cf3f0 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fba44089a90 tx=0x7fba44007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f633e35ab85b17fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44062110 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fba44058cc0 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba44057b70 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fba4407f050 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 10 client.7545 fetch_fsmap learned FSMap version 52 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5573fe68f0f0 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba44057b70 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fba44078b60 con 0x5573fea956c0 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 10 client.7545 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 10 client.7545 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 20 client.7545 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea7e640 0x5573fea7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 4 client.7545 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.423+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573fe9c9430 con 0x5573fea7e640 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.424+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea7e640 0x5573fea7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.424+0000 7fba2a7fc640 10 client.7545 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.424+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea7e640 0x5573fea7ea20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03b6a2e1885a161 server_cookie=d68ffb223324ac0d in_seq=0 out_seq=0 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba2a7fc640 1 -- 172.21.15.1:0/3354767778 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573fe9c9430 con 0x5573fea7e640 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba2a7fc640 10 client.7545 handle_command_reply: tid=0 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 client.7545 shutdown 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 msgr2=0x5573feaa0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fba44089a90 tx=0x7fba44007040 comp rx=0 tx=0).stop 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 shutdown_connections 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaa0200 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea95180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea7e640 0x5573fea7ea20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 --2- 172.21.15.1:0/3354767778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573fea6c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 >> 172.21.15.1:0/3354767778 conn(0x5573fea82030 msgr2=0x5573fea9caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba2a7fc640 0 client.7545 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 shutdown_connections 2024-05-09T06:32:40.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 1 -- 172.21.15.1:0/3354767778 wait complete. 2024-05-09T06:32:40.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.425+0000 7fba54e09740 20 client.7545 trim_cache size 0 max 16384 2024-05-09T06:32:40.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 Processor -- start 2024-05-09T06:32:40.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 -- start start 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea9b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea7cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c200c00 con 0x5573fea956c0 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c1ffc40 con 0x5573fea82770 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.443+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c15cd10 con 0x5573fea9f650 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60656/0 (socket says 172.21.15.1:60656) 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 -- 172.21.15.1:0/3164418432 learned_addr learned my addr 172.21.15.1:0/3164418432 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4bfff640 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea7cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea9b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 -- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 msgr2=0x5573fea7cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea7cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 -- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 msgr2=0x5573fea9b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea9b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 -- 172.21.15.1:0/3164418432 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fea85cc0 con 0x5573fea956c0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba4affd640 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fba40007510 tx=0x7fba40004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3766f3c8b2830f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba30ff9640 1 -- 172.21.15.1:0/3164418432 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40004ce0 con 0x5573fea956c0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba30ff9640 1 -- 172.21.15.1:0/3164418432 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4001b020 con 0x5573fea956c0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.444+0000 7fba30ff9640 1 -- 172.21.15.1:0/3164418432 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400017b0 con 0x5573fea956c0 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- 172.21.15.1:0/3164418432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 msgr2=0x5573fea7ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fba40007510 tx=0x7fba40004bb0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- 172.21.15.1:0/3164418432 shutdown_connections 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea9f650 0x5573fea7cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573fea7ca70 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 --2- 172.21.15.1:0/3164418432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea82770 0x5573fea9b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- 172.21.15.1:0/3164418432 >> 172.21.15.1:0/3164418432 conn(0x5573fea82030 msgr2=0x5573fea7e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- 172.21.15.1:0/3164418432 shutdown_connections 2024-05-09T06:32:40.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- 172.21.15.1:0/3164418432 wait complete. 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 -- start start 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.445+0000 7fba54e09740 1 Processor -- start 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573feaabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea92480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c1ff670 con 0x5573fea956c0 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5573feb175b0 con 0x5573fea9f650 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5573fea7ec20 con 0x5573fea82770 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba54e09740 10 client.0 _ll_register_callbacks cb 0x7fba440592a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60670/0 (socket says 172.21.15.1:60670) 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba4affd640 1 -- 172.21.15.1:0/1249888291 learned_addr learned my addr 172.21.15.1:0/1249888291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba4bfff640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea92480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.446+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573feaabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 -- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 msgr2=0x5573feaabdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573feaabdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 -- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 msgr2=0x5573fea92480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea92480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 -- 172.21.15.1:0/1249888291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573feb16f10 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba4affd640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba40009010 tx=0x7fba40007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee5f6a2fbb218d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4001a070 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fba40016860 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4001b020 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fba40030050 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 10 client.7551 fetch_fsmap learned FSMap version 52 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5573feb13c10 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4001b020 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fba40033020 con 0x5573fea956c0 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 10 client.7551 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 10 client.7551 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 20 client.7551 populate_metadata read hostname 'smithi001' 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea9cb80 0x5573fea929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 4 client.7551 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.447+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573fea536a0 con 0x5573fea9cb80 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea9cb80 0x5573fea929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba4b7fe640 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea9cb80 0x5573fea929c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e8b15d8808d4e6 server_cookie=9826ce6bd6db56d5 in_seq=0 out_seq=0 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba2a7fc640 10 client.7551 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba2a7fc640 1 -- 172.21.15.1:0/1249888291 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573fea536a0 con 0x5573fea9cb80 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba2a7fc640 10 client.7551 handle_command_reply: tid=0 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 client.7551 shutdown 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 msgr2=0x5573feaac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fba40009010 tx=0x7fba40007d10 comp rx=0 tx=0).stop 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 shutdown_connections 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573fea9f650 0x5573fea92480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573fea956c0 0x5573feaac310 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5573fea9cb80 0x5573fea929c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 --2- 172.21.15.1:0/1249888291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5573fea82770 0x5573feaabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba2a7fc640 0 client.7551 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 >> 172.21.15.1:0/1249888291 conn(0x5573fea82030 msgr2=0x5573fea9b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 shutdown_connections 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 1 -- 172.21.15.1:0/1249888291 wait complete. 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.448+0000 7fba54e09740 20 client.7551 trim_cache size 0 max 16384 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout: "53998ee2-f696-4b72-8a5a-c394ef710b58": { 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/subdir", 2024-05-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "53998ee2-f696-4b72-8a5a-c394ef710b58", 2024-05-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive" 2024-05-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 msgr2=0x7fba3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 0x7fba3806e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fba3c034000 tx=0x7fba3c02e000 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 msgr2=0x7fba4c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fba44007d40 tx=0x7fba4400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 shutdown_connections 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba4c0d0ea0 0x7fba4c157790 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fba4c0b9ae0 0x7fba4c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 --2- 172.21.15.1:0/623062145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4c0b9700 0x7fba4c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 >> 172.21.15.1:0/623062145 conn(0x7fba4c0d3ea0 msgr2=0x7fba4c0b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 shutdown_connections 2024-05-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:32:40.449+0000 7fba5201c640 1 -- 172.21.15.1:0/623062145 wait complete. 2024-05-09T06:32:40.440 INFO:tasks.cephfs.filesystem:scrub status for tag:53998ee2-f696-4b72-8a5a-c394ef710b58 - {'path': '/subdir', 'tag': '53998ee2-f696-4b72-8a5a-c394ef710b58', 'options': 'recursive'} 2024-05-09T06:32:48.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:32:48.244 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:33:10.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:10.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-09T06:33:10.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.580+0000 7f7ffab09640 1 Processor -- start 2024-05-09T06:33:10.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.580+0000 7f7ffab09640 1 -- start start 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ffab09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7ff40d1c70 0x7f7ff415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ffab09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 0x7f7ff40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ffab09640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff4161fe0 con 0x7f7ff415e190 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ffab09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff415f580 con 0x7f7ff40d1c70 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ffab09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff41622c0 con 0x7f7ff40d45f0 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7febfff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7ff40d1c70 0x7f7ff415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 0x7f7ff40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7febfff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7ff40d1c70 0x7f7ff415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:39310/0 (socket says 172.21.15.1:39310) 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7ff88a5640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7ff415e190 0x7f7ff40d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:48928/0 (socket says 172.21.15.1:48928) 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.581+0000 7f7febfff640 1 -- 172.21.15.1:0/2706964146 learned_addr learned my addr 172.21.15.1:0/2706964146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7febfff640 1 -- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 msgr2=0x7f7ff40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7febfff640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 0x7f7ff40b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7febfff640 1 -- 172.21.15.1:0/2706964146 >> v1:172.21.15.6:6789/0 conn(0x7f7ff415e190 legacy=0x7f7ff40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7febfff640 1 -- 172.21.15.1:0/2706964146 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff41625a0 con 0x7f7ff40d1c70 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 0x7f7ff40b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.582+0000 7f7febfff640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d1c70 0x7f7ff415f780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009870 tx=0x7f7fe402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9ce544accb9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7feb7fe640 1 -- 172.21.15.1:0/2706964146 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4037070 con 0x7f7ff40d1c70 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7feb7fe640 1 -- 172.21.15.1:0/2706964146 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4032440 con 0x7f7ff40d1c70 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7feb7fe640 1 -- 172.21.15.1:0/2706964146 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4032810 con 0x7f7ff40d1c70 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 -- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d1c70 msgr2=0x7f7ff415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d1c70 0x7f7ff415f780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009870 tx=0x7f7fe402a250 comp rx=0 tx=0).stop 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 -- 172.21.15.1:0/2706964146 shutdown_connections 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d45f0 0x7f7ff40b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2706964146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d1c70 0x7f7ff415f780 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 -- 172.21.15.1:0/2706964146 >> 172.21.15.1:0/2706964146 conn(0x7f7ff4073340 msgr2=0x7f7ff409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 -- 172.21.15.1:0/2706964146 shutdown_connections 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.583+0000 7f7ffab09640 1 -- 172.21.15.1:0/2706964146 wait complete. 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.584+0000 7f7ffab09640 1 Processor -- start 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 -- start start 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d1c70 0x7f7ff40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 0x7f7ff4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40b56d0 con 0x7f7ff415e190 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4161c60 con 0x7f7ff40d45f0 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.585+0000 7f7ffab09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff40b5850 con 0x7f7ff40d1c70 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49918/0 (socket says 172.21.15.1:49918) 2024-05-09T06:33:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 -- 172.21.15.1:0/2475360913 learned_addr learned my addr 172.21.15.1:0/2475360913 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7febfff640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 0x7f7ff4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d1c70 0x7f7ff40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 -- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d1c70 msgr2=0x7f7ff40cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d1c70 0x7f7ff40cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 -- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 msgr2=0x7f7ff4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 0x7f7ff4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 -- 172.21.15.1:0/2475360913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff4159680 con 0x7f7ff415e190 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7febfff640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 0x7f7ff4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00efb0 tx=0x7f7fec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7fe97fa640 1 -- 172.21.15.1:0/2475360913 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec018020 con 0x7f7ff415e190 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7fe97fa640 1 -- 172.21.15.1:0/2475360913 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec014480 con 0x7f7ff415e190 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff40ccb70 con 0x7f7ff415e190 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.586+0000 7f7fe97fa640 1 -- 172.21.15.1:0/2475360913 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec014850 con 0x7f7ff415e190 2024-05-09T06:33:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.587+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff40cd130 con 0x7f7ff415e190 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.588+0000 7f7fe97fa640 1 -- 172.21.15.1:0/2475360913 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fec005d10 con 0x7f7ff415e190 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.589+0000 7f7fe97fa640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 0x7f7fe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.589+0000 7f7fe97fa640 1 -- 172.21.15.1:0/2475360913 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f7fec08bcd0 con 0x7f7ff415e190 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.589+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 0x7f7fe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.589+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 0x7f7fe006e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000f60 tx=0x7f7ff0000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 Processor -- start 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 -- start start 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x5652409fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 0x565240a01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565240966d00 con 0x565240a2d770 2024-05-09T06:33:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56524090e4c0 con 0x565240a4a650 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.590+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x565240962cc0 con 0x565240a2d030 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49930/0 (socket says 172.21.15.1:49930) 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 -- 172.21.15.1:0/2649703986 learned_addr learned my addr 172.21.15.1:0/2649703986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x5652409fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 0x565240a01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 -- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 msgr2=0x565240a01e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 0x565240a01e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 -- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 msgr2=0x5652409fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x5652409fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 -- 172.21.15.1:0/2649703986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652407447d0 con 0x565240a2d770 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 0x565240a01e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7febfff640 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009990 tx=0x7f7fe402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74edbfe376250d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7fda7fc640 1 -- 172.21.15.1:0/2649703986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4037040 con 0x565240a2d770 2024-05-09T06:33:10.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7fda7fc640 1 -- 172.21.15.1:0/2649703986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4032ba0 con 0x565240a2d770 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.591+0000 7f7fda7fc640 1 -- 172.21.15.1:0/2649703986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe403c440 con 0x565240a2d770 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- 172.21.15.1:0/2649703986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 msgr2=0x565240a2caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009990 tx=0x7f7fe402c000 comp rx=0 tx=0).stop 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- 172.21.15.1:0/2649703986 shutdown_connections 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d030 0x565240a01e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a2caf0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 --2- 172.21.15.1:0/2649703986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x5652409fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- 172.21.15.1:0/2649703986 >> 172.21.15.1:0/2649703986 conn(0x5652409ff250 msgr2=0x565240a47af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- 172.21.15.1:0/2649703986 shutdown_connections 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- 172.21.15.1:0/2649703986 wait complete. 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.592+0000 7f7ffd813740 1 -- start start 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.593+0000 7f7ffd813740 1 Processor -- start 2024-05-09T06:33:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 0x565240a4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 0x565240a4b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565240a01400 con 0x565240a3e7b0 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x565240921b90 con 0x565240a2d770 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x565240a007f0 con 0x565240a4a650 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 10 client.0 _ll_register_callbacks cb 0x56524095dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49932/0 (socket says 172.21.15.1:49932) 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 -- 172.21.15.1:0/3976844337 learned_addr learned my addr 172.21.15.1:0/3976844337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 0x565240a4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7febfff640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 0x565240a4b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 -- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 msgr2=0x565240a4b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 0x565240a4b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 -- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 msgr2=0x565240a4af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 0x565240a4af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 -- 172.21.15.1:0/3976844337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524067a3f0 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7febfff640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 0x565240a4b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 0x565240a4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7fec059b50 tx=0x7f7fec059f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dab860eb96662c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7fd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0592a0 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f7fec059440 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe4032ba0 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7fec078050 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.594+0000 7f7ffd813740 10 client.7560 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56524063a0f0 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4032ba0 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f7fec060350 con 0x565240a3e7b0 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 10 client.7560 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 10 client.7560 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 20 client.7560 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a1d640 0x565240a30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 4 client.7560 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565240974430 con 0x565240a1d640 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a1d640 0x565240a30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a1d640 0x565240a30560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae93d7561e2b7f9 server_cookie=a043d6440a436d3d in_seq=0 out_seq=0 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.595+0000 7f7fd37fe640 10 client.7560 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7fd37fe640 1 -- 172.21.15.1:0/3976844337 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565240974430 con 0x565240a1d640 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7fd37fe640 10 client.7560 handle_command_reply: tid=0 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 client.7560 shutdown 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 msgr2=0x565240a30020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7fec059b50 tx=0x7f7fec059f90 comp rx=0 tx=0).stop 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 shutdown_connections 2024-05-09T06:33:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a3e7b0 0x565240a30020 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a1d640 0x565240a30560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a4a650 0x565240a4b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 --2- 172.21.15.1:0/3976844337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a2d770 0x565240a4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7fd37fe640 0 client.7560 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 >> 172.21.15.1:0/3976844337 conn(0x565240a2d030 msgr2=0x565240a01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 shutdown_connections 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 1 -- 172.21.15.1:0/3976844337 wait complete. 2024-05-09T06:33:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.596+0000 7f7ffd813740 20 client.7560 trim_cache size 0 max 16384 2024-05-09T06:33:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.614+0000 7f7ffd813740 1 Processor -- start 2024-05-09T06:33:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.614+0000 7f7ffd813740 1 -- start start 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d770 0x565240a6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a3e7b0 0x565240a30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4200c20 con 0x565240a4a650 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41ffcb0 con 0x565240a3e7b0 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff40cb5d0 con 0x565240a2d770 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49946/0 (socket says 172.21.15.1:49946) 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 -- 172.21.15.1:0/1049460549 learned_addr learned my addr 172.21.15.1:0/1049460549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7febfff640 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a3e7b0 0x565240a30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d770 0x565240a6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 -- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d770 msgr2=0x565240a6b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d770 0x565240a6b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 -- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a3e7b0 msgr2=0x565240a30450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a3e7b0 0x565240a30450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 -- 172.21.15.1:0/1049460549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565240a2f860 con 0x565240a4a650 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.615+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7fec059de0 tx=0x7f7fec01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9786ce9648b121fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7fd9ffb640 1 -- 172.21.15.1:0/1049460549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0592a0 con 0x565240a4a650 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7fd9ffb640 1 -- 172.21.15.1:0/1049460549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec088070 con 0x565240a4a650 2024-05-09T06:33:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7fd9ffb640 1 -- 172.21.15.1:0/1049460549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec055ba0 con 0x565240a4a650 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 -- 172.21.15.1:0/1049460549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 msgr2=0x565240a4af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7fec059de0 tx=0x7f7fec01f070 comp rx=0 tx=0).stop 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 -- 172.21.15.1:0/1049460549 shutdown_connections 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a3e7b0 0x565240a30450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1049460549 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a2d770 0x565240a6b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 -- 172.21.15.1:0/1049460549 >> 172.21.15.1:0/1049460549 conn(0x565240a2d030 msgr2=0x565240a28060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 -- 172.21.15.1:0/1049460549 shutdown_connections 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.616+0000 7f7ffd813740 1 -- 172.21.15.1:0/1049460549 wait complete. 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.617+0000 7f7ffd813740 1 -- start start 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.617+0000 7f7ffd813740 1 Processor -- start 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a3e7b0 0x565240a3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x565240a2cac0 con 0x565240a2d770 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40cbc20 con 0x565240a4a650 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x565240a02ce0 con 0x565240a3e7b0 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ffd813740 10 client.0 _ll_register_callbacks cb 0x7f7fec009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49952/0 (socket says 172.21.15.1:49952) 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 -- 172.21.15.1:0/1459530715 learned_addr learned my addr 172.21.15.1:0/1459530715 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff90a6640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7febfff640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a3e7b0 0x565240a3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 -- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a3e7b0 msgr2=0x565240a3e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a3e7b0 0x565240a3e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 -- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 msgr2=0x565240a4af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.618+0000 7f7ff88a5640 1 -- 172.21.15.1:0/1459530715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565240a2ed90 con 0x565240a2d770 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ff88a5640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0001a20 tx=0x7f7ff0001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80653b536aa0ee3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00272c0 con 0x565240a2d770 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f7ff0010040 con 0x565240a2d770 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fec088070 con 0x565240a2d770 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7ff0034050 con 0x565240a2d770 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 10 client.7566 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:10.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x565240a56a00 con 0x565240a2d770 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec088070 con 0x565240a2d770 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f7ff0037020 con 0x565240a2d770 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 10 client.7566 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 10 client.7566 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 20 client.7566 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a4b450 0x565240a00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 4 client.7566 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56524096e780 con 0x565240a4b450 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.619+0000 7f7febfff640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a4b450 0x565240a00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7febfff640 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a4b450 0x565240a00a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b180e642479f53 server_cookie=59ca921f9d55fff4 in_seq=0 out_seq=0 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7fd37fe640 10 client.7566 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7fd37fe640 1 -- 172.21.15.1:0/1459530715 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56524096e780 con 0x565240a4b450 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7fd37fe640 10 client.7566 handle_command_reply: tid=0 2024-05-09T06:33:10.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 client.7566 shutdown 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 msgr2=0x565240a3f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0001a20 tx=0x7f7ff0001080 comp rx=0 tx=0).stop 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 shutdown_connections 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565240a4a650 0x565240a4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x565240a4b450 0x565240a00a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7fd37fe640 0 client.7566 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565240a3e7b0 0x565240a3e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 --2- 172.21.15.1:0/1459530715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565240a2d770 0x565240a3f0e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 >> 172.21.15.1:0/1459530715 conn(0x565240a2d030 msgr2=0x565240a28060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 shutdown_connections 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 1 -- 172.21.15.1:0/1459530715 wait complete. 2024-05-09T06:33:10.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.620+0000 7f7ffd813740 20 client.7566 trim_cache size 0 max 16384 2024-05-09T06:33:10.608 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:10.608 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:33:10.608 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:33:10.608 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 msgr2=0x7f7fe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 0x7f7fe006e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000f60 tx=0x7f7ff0000ed0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 msgr2=0x7f7ff40c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00efb0 tx=0x7f7fec00c490 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 shutdown_connections 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7fe006be30 0x7f7fe006e2f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ff415e190 0x7f7ff40c8ec0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ff40d45f0 0x7f7ff4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 --2- 172.21.15.1:0/2475360913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ff40d1c70 0x7f7ff40cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 >> 172.21.15.1:0/2475360913 conn(0x7f7ff4073340 msgr2=0x7f7ff415f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 shutdown_connections 2024-05-09T06:33:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.621+0000 7f7ffab09640 1 -- 172.21.15.1:0/2475360913 wait complete. 2024-05-09T06:33:10.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:33:10.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:10.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.736+0000 7f52b4134640 1 Processor -- start 2024-05-09T06:33:10.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.736+0000 7f52b4134640 1 -- start start 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b4134640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f52ac15e050 0x7f52ac15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b4134640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52ac0aef90 con 0x7f52ac06e4f0 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b4134640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52ac0b3530 con 0x7f52ac0ae9b0 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b4134640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f52ac0b3b80 con 0x7f52ac15e050 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b16cf640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f52ac06e4f0 0x7f52ac0ae2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38722/0 (socket says 172.21.15.1:38722) 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b16cf640 1 -- 172.21.15.1:0/1850581022 learned_addr learned my addr 172.21.15.1:0/1850581022 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/1850581022 >> v2:172.21.15.6:3300/0 conn(0x7f52ac15e050 0x7f52ac15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767178703 0 0) 0x7f52ac0aef90 con 0x7f52ac06e4f0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5298003520 con 0x7f52ac06e4f0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1211467380 0 0) 0x7f52ac0b3530 con 0x7f52ac0ae9b0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52ac0aef90 con 0x7f52ac0ae9b0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 817533049 0 0) 0x7f5298003520 con 0x7f52ac06e4f0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 >> v2:172.21.15.6:3300/0 conn(0x7f52ac15e050 msgr2=0x7f52ac15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 --2- 172.21.15.1:0/1850581022 >> v2:172.21.15.6:3300/0 conn(0x7f52ac15e050 0x7f52ac15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 >> v1:172.21.15.6:6789/0 conn(0x7f52ac0ae9b0 legacy=0x7f52ac0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac0b3810 con 0x7f52ac06e4f0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f529c001670 con 0x7f52ac06e4f0 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.737+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 >> v1:172.21.15.1:6789/0 conn(0x7f52ac06e4f0 legacy=0x7f52ac0ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f5298003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5298004800 0x7f5298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52980051c0 0x7f529800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac0b3b80 con 0x7f5298003b40 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f529800c040 con 0x7f52980051c0 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f529800c240 con 0x7f5298004800 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f5298003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b16cf640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5298004800 0x7f5298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b26d1640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52980051c0 0x7f529800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 -- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5298004800 msgr2=0x7f5298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5298004800 0x7f5298004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 -- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52980051c0 msgr2=0x7f529800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52980051c0 0x7f529800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b1ed0640 1 -- 172.21.15.1:0/1850581022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5298003520 con 0x7f5298003b40 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b26d1640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52980051c0 0x7f529800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.738+0000 7f52b26d1640 1 -- 172.21.15.1:0/1850581022 reap_dead start 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f5298003f40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ac0 tx=0x7f5298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c623f953aeff285e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8036070 con 0x7f5298003b40 2024-05-09T06:33:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8031400 con 0x7f5298003b40 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b0ece640 1 -- 172.21.15.1:0/1850581022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009690 con 0x7f5298003b40 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 -- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 msgr2=0x7f5298003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f5298003f40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ac0 tx=0x7f5298005830 comp rx=0 tx=0).stop 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 -- 172.21.15.1:0/1850581022 shutdown_connections 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 --2- 172.21.15.1:0/1850581022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f5298003f40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 -- 172.21.15.1:0/1850581022 >> 172.21.15.1:0/1850581022 conn(0x7f52ac06d270 msgr2=0x7f52ac06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 -- 172.21.15.1:0/1850581022 shutdown_connections 2024-05-09T06:33:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.739+0000 7f52b4134640 1 -- 172.21.15.1:0/1850581022 wait complete. 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 Processor -- start 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 -- start start 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52ac1511d0 0x7f52ac1f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52ac1f7c30 0x7f52ac1f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac06ed10 con 0x7f5298003b40 2024-05-09T06:33:10.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac06eb90 con 0x7f52ac1511d0 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b4134640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac152590 con 0x7f52ac1f7c30 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b1ed0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b1ed0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49966/0 (socket says 172.21.15.1:49966) 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.741+0000 7f52b1ed0640 1 -- 172.21.15.1:0/2238202038 learned_addr learned my addr 172.21.15.1:0/2238202038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b16cf640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52ac1511d0 0x7f52ac1f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b26d1640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52ac1f7c30 0x7f52ac1f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 -- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52ac1f7c30 msgr2=0x7f52ac1f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52ac1f7c30 0x7f52ac1f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 -- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52ac1511d0 msgr2=0x7f52ac1f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52ac1511d0 0x7f52ac1f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 -- 172.21.15.1:0/2238202038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac151980 con 0x7f5298003b40 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b1ed0640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009be0 tx=0x7f52a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a803f020 con 0x7f5298003b40 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8037040 con 0x7f5298003b40 2024-05-09T06:33:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac151c60 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8036070 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.742+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac152220 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.743+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac0752b0 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.744+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52a8042020 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.744+0000 7f52a27fc640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 0x7f529006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.744+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f52a80aaf90 con 0x7f5298003b40 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.744+0000 7f52b16cf640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 0x7f529006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.745+0000 7f52b16cf640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 0x7f529006e320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52ac14f4c0 tx=0x7f529c001e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.749+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52a8049030 con 0x7f5298003b40 2024-05-09T06:33:10.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.970+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52ac15bf50 con 0x7f5298003b40 2024-05-09T06:33:10.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.970+0000 7f52a27fc640 1 -- 172.21.15.1:0/2238202038 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5170 (secure 0 0 0) 0x7f52a8079110 con 0x7f5298003b40 2024-05-09T06:33:10.951 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:10.951 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:32:38.372870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":13,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:10.952 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 52 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 msgr2=0x7f529006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 0x7f529006e320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52ac14f4c0 tx=0x7f529c001e40 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 msgr2=0x7f52ac150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009be0 tx=0x7f52a8004730 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 shutdown_connections 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f529006be60 0x7f529006e320 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52ac1f7c30 0x7f52ac1f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52ac1511d0 0x7f52ac1f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 --2- 172.21.15.1:0/2238202038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5298003b40 0x7f52ac150c90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 >> 172.21.15.1:0/2238202038 conn(0x7f52ac06d270 msgr2=0x7f52ac154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 shutdown_connections 2024-05-09T06:33:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:10.972+0000 7f52b4134640 1 -- 172.21.15.1:0/2238202038 wait complete. 2024-05-09T06:33:10.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-09T06:33:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.089+0000 7f2759912640 1 Processor -- start 2024-05-09T06:33:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.089+0000 7f2759912640 1 -- start start 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2759912640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f27540d1c70 0x7f275415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2759912640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 0x7f27540b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2759912640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2754161fe0 con 0x7f275415e190 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2759912640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f275415f580 con 0x7f27540d1c70 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2759912640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27541622c0 con 0x7f27540d45f0 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f27540d1c70 0x7f275415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f27540d1c70 0x7f275415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49978/0 (socket says 172.21.15.1:49978) 2024-05-09T06:33:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 -- 172.21.15.1:0/3471919567 learned_addr learned my addr 172.21.15.1:0/3471919567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2752ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f275415e190 0x7f27540d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38728/0 (socket says 172.21.15.1:38728) 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 -- 172.21.15.1:0/3471919567 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 msgr2=0x7f27540b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27537fe640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 0x7f27540b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 0x7f27540b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 -- 172.21.15.1:0/3471919567 >> v1:172.21.15.1:6789/0 conn(0x7f275415e190 legacy=0x7f27540d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f2751ffb640 1 -- 172.21.15.1:0/3471919567 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835289150 0 0) 0x7f2754161fe0 con 0x7f275415e190 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27527fc640 1 -- 172.21.15.1:0/3471919567 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541625a0 con 0x7f27540d1c70 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.090+0000 7f27537fe640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 0x7f27540b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.091+0000 7f27527fc640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27540d1c70 0x7f275415f780 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e74d3e33655fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.091+0000 7f2751ffb640 1 -- 172.21.15.1:0/3471919567 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037070 con 0x7f27540d1c70 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.091+0000 7f2751ffb640 1 -- 172.21.15.1:0/3471919567 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c032440 con 0x7f27540d1c70 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.091+0000 7f2751ffb640 1 -- 172.21.15.1:0/3471919567 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c032810 con 0x7f27540d1c70 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 -- 172.21.15.1:0/3471919567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27540d1c70 msgr2=0x7f275415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27540d1c70 0x7f275415f780 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02a270 comp rx=0 tx=0).stop 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 -- 172.21.15.1:0/3471919567 shutdown_connections 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d45f0 0x7f27540b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 --2- 172.21.15.1:0/3471919567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27540d1c70 0x7f275415f780 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 -- 172.21.15.1:0/3471919567 >> 172.21.15.1:0/3471919567 conn(0x7f2754073340 msgr2=0x7f275409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 -- 172.21.15.1:0/3471919567 shutdown_connections 2024-05-09T06:33:11.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.092+0000 7f2759912640 1 -- 172.21.15.1:0/3471919567 wait complete. 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.093+0000 7f2759912640 1 Processor -- start 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.093+0000 7f2759912640 1 -- start start 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 0x7f27540b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27540d45f0 0x7f27540cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27540b56d0 con 0x7f275415e190 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2754161c60 con 0x7f27540d45f0 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2759912640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27540b5850 con 0x7f27540d1c70 2024-05-09T06:33:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49990/0 (socket says 172.21.15.1:49990) 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 -- 172.21.15.1:0/2080031400 learned_addr learned my addr 172.21.15.1:0/2080031400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27527fc640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27540d45f0 0x7f27540cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 0x7f27540b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 -- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 msgr2=0x7f27540b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 0x7f27540b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 -- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27540d45f0 msgr2=0x7f27540cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27540d45f0 0x7f27540cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f27537fe640 1 -- 172.21.15.1:0/2080031400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27540ccdd0 con 0x7f275415e190 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.094+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 0x7f27540b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:11.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f27537fe640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f274400c7f0 tx=0x7f274400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f274bfff640 1 -- 172.21.15.1:0/2080031400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744015070 con 0x7f275415e190 2024-05-09T06:33:11.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f274bfff640 1 -- 172.21.15.1:0/2080031400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744007820 con 0x7f275415e190 2024-05-09T06:33:11.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f274bfff640 1 -- 172.21.15.1:0/2080031400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744007c10 con 0x7f275415e190 2024-05-09T06:33:11.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27540cd0b0 con 0x7f275415e190 2024-05-09T06:33:11.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.095+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27540cd670 con 0x7f275415e190 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.096+0000 7f274bfff640 1 -- 172.21.15.1:0/2080031400 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2744018070 con 0x7f275415e190 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.097+0000 7f274bfff640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.097+0000 7f274bfff640 1 -- 172.21.15.1:0/2080031400 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f274408b090 con 0x7f275415e190 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.097+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.097+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 0x7f273406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f274c0012e0 tx=0x7f274c005b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.098+0000 7f275c617740 1 Processor -- start 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.098+0000 7f275c617740 1 -- start start 2024-05-09T06:33:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7e0180 0x55889a792b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7c2b60 0x55889a797970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55889a6fba30 con 0x55889a7c32a0 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55889a699590 con 0x55889a7e0180 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55889a6f79f0 con 0x55889a7c2b60 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50000/0 (socket says 172.21.15.1:50000) 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 -- 172.21.15.1:0/2752524195 learned_addr learned my addr 172.21.15.1:0/2752524195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7e0180 0x55889a792b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27537fe640 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7c2b60 0x55889a797970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 -- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7c2b60 msgr2=0x55889a797970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7c2b60 0x55889a797970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 -- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7e0180 msgr2=0x55889a792b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7e0180 0x55889a792b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 -- 172.21.15.1:0/2752524195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889a4db7c0 con 0x55889a7c32a0 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.099+0000 7f27527fc640 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee896a3fc51afbab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f27497fa640 1 -- 172.21.15.1:0/2752524195 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037040 con 0x55889a7c32a0 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f27497fa640 1 -- 172.21.15.1:0/2752524195 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c0327c0 con 0x55889a7c32a0 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f27497fa640 1 -- 172.21.15.1:0/2752524195 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c032bb0 con 0x55889a7c32a0 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 -- 172.21.15.1:0/2752524195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 msgr2=0x55889a7c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f273c009870 tx=0x7f273c02c000 comp rx=0 tx=0).stop 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 -- 172.21.15.1:0/2752524195 shutdown_connections 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7c2b60 0x55889a797970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7c2620 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 --2- 172.21.15.1:0/2752524195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7e0180 0x55889a792b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 -- 172.21.15.1:0/2752524195 >> 172.21.15.1:0/2752524195 conn(0x55889a794d80 msgr2=0x55889a7dd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 -- 172.21.15.1:0/2752524195 shutdown_connections 2024-05-09T06:33:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.100+0000 7f275c617740 1 -- 172.21.15.1:0/2752524195 wait complete. 2024-05-09T06:33:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.101+0000 7f275c617740 1 -- start start 2024-05-09T06:33:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.101+0000 7f275c617740 1 Processor -- start 2024-05-09T06:33:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7e0180 0x55889a7aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7ed4f0 0x55889a7c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55889a796f30 con 0x55889a7c32a0 2024-05-09T06:33:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55889a697060 con 0x55889a7ed4f0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55889a796320 con 0x55889a7e0180 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f275c617740 10 client.0 _ll_register_callbacks cb 0x55889a6bbdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50004/0 (socket says 172.21.15.1:50004) 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 -- 172.21.15.1:0/1506221033 learned_addr learned my addr 172.21.15.1:0/1506221033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f27527fc640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7e0180 0x55889a7aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f27537fe640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7ed4f0 0x55889a7c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 -- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7e0180 msgr2=0x55889a7aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7e0180 0x55889a7aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 -- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7ed4f0 msgr2=0x55889a7c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7ed4f0 0x55889a7c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.102+0000 7f2752ffd640 1 -- 172.21.15.1:0/1506221033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889a53bf50 con 0x55889a7c32a0 2024-05-09T06:33:11.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f2752ffd640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f274c015720 tx=0x7f274c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78de88e2d08ee2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c027060 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273c0327c0 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f274c010040 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f274c0124c0 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 10 client.7587 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55889a3cf0b0 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c0327c0 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f274c012660 con 0x55889a7c32a0 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 10 client.7587 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 10 client.7587 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 20 client.7587 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7c3e60 0x55889a7fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 4 client.7587 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889a709160 con 0x55889a7c3e60 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.103+0000 7f27527fc640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7c3e60 0x55889a7fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f27527fc640 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7c3e60 0x55889a7fbf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b27739fa993c0c3 server_cookie=310b7d480e6ec46b in_seq=0 out_seq=0 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f27327fc640 10 client.7587 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f27327fc640 1 -- 172.21.15.1:0/1506221033 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55889a709160 con 0x55889a7c3e60 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f27327fc640 10 client.7587 handle_command_reply: tid=0 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 client.7587 shutdown 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 msgr2=0x55889a7f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f274c015720 tx=0x7f274c011000 comp rx=0 tx=0).stop 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 shutdown_connections 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7ed4f0 0x55889a7c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7c3e60 0x55889a7fbf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7e0180 0x55889a7aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 --2- 172.21.15.1:0/1506221033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7c32a0 0x55889a7f15a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f27327fc640 0 client.7587 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.104+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 >> 172.21.15.1:0/1506221033 conn(0x55889a7c2b60 msgr2=0x55889a792240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.105+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 shutdown_connections 2024-05-09T06:33:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.105+0000 7f275c617740 1 -- 172.21.15.1:0/1506221033 wait complete. 2024-05-09T06:33:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.105+0000 7f275c617740 20 client.7587 trim_cache size 0 max 16384 2024-05-09T06:33:11.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.118+0000 7f275c617740 1 Processor -- start 2024-05-09T06:33:11.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.118+0000 7f275c617740 1 -- start start 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a796750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ffa50 con 0x55889a7e0180 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f27540cbf70 con 0x55889a7c32a0 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27540c8960 con 0x55889a7ed4f0 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50012/0 (socket says 172.21.15.1:50012) 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 -- 172.21.15.1:0/2049006724 learned_addr learned my addr 172.21.15.1:0/2049006724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 -- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 msgr2=0x55889a7c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 -- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 msgr2=0x55889a796750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a796750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 -- 172.21.15.1:0/2049006724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889a795ec0 con 0x55889a7e0180 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f27527fc640 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f273c009990 tx=0x7f273c034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148ca8daeb113709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f2748ff9640 1 -- 172.21.15.1:0/2049006724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c054070 con 0x55889a7e0180 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f2748ff9640 1 -- 172.21.15.1:0/2049006724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c04ec10 con 0x55889a7e0180 2024-05-09T06:33:11.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.119+0000 7f2748ff9640 1 -- 172.21.15.1:0/2049006724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c037040 con 0x55889a7e0180 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 -- 172.21.15.1:0/2049006724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 msgr2=0x55889a796c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f273c009990 tx=0x7f273c034040 comp rx=0 tx=0).stop 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 -- 172.21.15.1:0/2049006724 shutdown_connections 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a796c90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 --2- 172.21.15.1:0/2049006724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a796750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 -- 172.21.15.1:0/2049006724 >> 172.21.15.1:0/2049006724 conn(0x55889a7c2b60 msgr2=0x55889a7dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 -- 172.21.15.1:0/2049006724 shutdown_connections 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.120+0000 7f275c617740 1 -- 172.21.15.1:0/2049006724 wait complete. 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.121+0000 7f275c617740 1 -- start start 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.121+0000 7f275c617740 1 Processor -- start 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a85a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55889a797330 con 0x55889a7e0180 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55889a7be600 con 0x55889a7c32a0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55889a797950 con 0x55889a7ed4f0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 10 client.0 _ll_register_callbacks cb 0x7f274c001b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50028/0 (socket says 172.21.15.1:50028) 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 -- 172.21.15.1:0/2913273920 learned_addr learned my addr 172.21.15.1:0/2913273920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a85a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27537fe640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 -- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 msgr2=0x55889a7ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 -- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 msgr2=0x55889a85a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a85a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 -- 172.21.15.1:0/2913273920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889a793080 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27527fc640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f273c033000 tx=0x7f273c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d153b51cd7e39897 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27327fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c054070 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f273c004400 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.122+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273c04ec10 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f273c060050 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 10 client.7593 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55889a7972e0 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c04ec10 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f273c063020 con 0x55889a7e0180 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 10 client.7593 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 10 client.7593 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 20 client.7593 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7aeb50 0x55889a7aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 4 client.7593 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55889a7034b0 con 0x55889a7aeb50 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7aeb50 0x55889a7aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f2752ffd640 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7aeb50 0x55889a7aef30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa23251b2a3e545 server_cookie=5f283d7c991a2557 in_seq=0 out_seq=0 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 10 client.7593 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 1 -- 172.21.15.1:0/2913273920 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x55889a7034b0 con 0x55889a7aeb50 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f27327fc640 10 client.7593 handle_command_reply: tid=0 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.123+0000 7f275c617740 1 client.7593 shutdown 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 msgr2=0x55889a7ec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f273c033000 tx=0x7f273c004730 comp rx=0 tx=0).stop 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 shutdown_connections 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55889a7aeb50 0x55889a7aef30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55889a7ed4f0 0x55889a7ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f27327fc640 0 client.7593 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55889a7e0180 0x55889a7ec2c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 --2- 172.21.15.1:0/2913273920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55889a7c32a0 0x55889a85a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 >> 172.21.15.1:0/2913273920 conn(0x55889a7c2b60 msgr2=0x55889a796750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 shutdown_connections 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 1 -- 172.21.15.1:0/2913273920 wait complete. 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f275c617740 20 client.7593 trim_cache size 0 max 16384 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "dentry", 2024-05-09T06:33:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 347048580, 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627776, 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "frag": "*", 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "dname": "file_to_be_damaged", 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_id": "head", 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:11.112 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 msgr2=0x7f273406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 0x7f273406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f274c0012e0 tx=0x7f274c005b90 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 msgr2=0x7f27540b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f274400c7f0 tx=0x7f274400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 shutdown_connections 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f273406be30 0x7f273406e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f275415e190 0x7f27540b7790 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27540d45f0 0x7f27540cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 --2- 172.21.15.1:0/2080031400 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27540d1c70 0x7f27540b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.124+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 >> 172.21.15.1:0/2080031400 conn(0x7f2754073340 msgr2=0x7f275415f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.125+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 shutdown_connections 2024-05-09T06:33:11.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.125+0000 7f2759912640 1 -- 172.21.15.1:0/2080031400 wait complete. 2024-05-09T06:33:11.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:33:11.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.238+0000 7f2efe876640 1 Processor -- start 2024-05-09T06:33:11.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.238+0000 7f2efe876640 1 -- start start 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2efe876640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2ef81454b0 0x7f2ef80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2efe876640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef806eb40 con 0x7f2ef80b2a60 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2efe876640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef806ee20 con 0x7f2ef80b4f70 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2efe876640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef806d8e0 con 0x7f2ef81454b0 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2ef77fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2ef80b2a60 0x7f2ef80b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38736/0 (socket says 172.21.15.1:38736) 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2ef77fe640 1 -- 172.21.15.1:0/2493314980 learned_addr learned my addr 172.21.15.1:0/2493314980 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.239+0000 7f2ef7fff640 1 --2- 172.21.15.1:0/2493314980 >> v2:172.21.15.6:3300/0 conn(0x7f2ef81454b0 0x7f2ef80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692592596 0 0) 0x7f2ef806eb40 con 0x7f2ef80b2a60 2024-05-09T06:33:11.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee0003520 con 0x7f2ef80b2a60 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217817358 0 0) 0x7f2ef806ee20 con 0x7f2ef80b4f70 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ef806eb40 con 0x7f2ef80b4f70 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2493314980 >> v1:172.21.15.6:6789/0 conn(0x7f2ef80b4f70 legacy=0x7f2ef8070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2493314980 >> v1:172.21.15.1:6789/0 conn(0x7f2ef80b2a60 legacy=0x7f2ef80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3565071954 0 0) 0x7f2ee0003520 con 0x7f2ef80b2a60 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.240+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2493314980 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef806dbc0 con 0x7f2ef81454b0 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2ef7fff640 1 --2- 172.21.15.1:0/2493314980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef81454b0 0x7f2ef80b4320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80097a0 tx=0x7f2ee802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57acdebece2a546 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8037070 con 0x7f2ef81454b0 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee802ac50 con 0x7f2ef81454b0 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2ef6ffd640 1 -- 172.21.15.1:0/2493314980 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec001670 con 0x7f2ef81454b0 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 -- 172.21.15.1:0/2493314980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef81454b0 msgr2=0x7f2ef80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 --2- 172.21.15.1:0/2493314980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef81454b0 0x7f2ef80b4320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80097a0 tx=0x7f2ee802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 -- 172.21.15.1:0/2493314980 shutdown_connections 2024-05-09T06:33:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 --2- 172.21.15.1:0/2493314980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef81454b0 0x7f2ef80b4320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 -- 172.21.15.1:0/2493314980 >> 172.21.15.1:0/2493314980 conn(0x7f2ef80cc4b0 msgr2=0x7f2ef80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.241+0000 7f2efe876640 1 -- 172.21.15.1:0/2493314980 shutdown_connections 2024-05-09T06:33:11.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.242+0000 7f2efe876640 1 -- 172.21.15.1:0/2493314980 wait complete. 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.243+0000 7f2efe876640 1 Processor -- start 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.243+0000 7f2efe876640 1 -- start start 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef80b4f70 0x7f2ef813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ef81454b0 0x7f2ef815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef806e260 con 0x7f2ef80b2a60 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef806e0e0 con 0x7f2ef80b4f70 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef8071320 con 0x7f2ef81454b0 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50034/0 (socket says 172.21.15.1:50034) 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2712209348 learned_addr learned my addr 172.21.15.1:0/2712209348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ef81454b0 msgr2=0x7f2ef815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ef81454b0 0x7f2ef815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef80b4f70 msgr2=0x7f2ef813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef80b4f70 0x7f2ef813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 -- 172.21.15.1:0/2712209348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef81e66d0 con 0x7f2ef80b2a60 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef7fff640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098c0 tx=0x7f2ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8041020 con 0x7f2ef80b2a60 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee803d420 con 0x7f2ef80b2a60 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8037040 con 0x7f2ef80b2a60 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.244+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef81e69b0 con 0x7f2ef80b2a60 2024-05-09T06:33:11.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.245+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef81e7250 con 0x7f2ef80b2a60 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.245+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef80752b0 con 0x7f2ef80b2a60 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.246+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ee8044020 con 0x7f2ef80b2a60 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.247+0000 7f2ef4ff9640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 0x7f2ee406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.247+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f2ee80ab3b0 con 0x7f2ef80b2a60 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.247+0000 7f2ef77fe640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 0x7f2ee406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.247+0000 7f2ef77fe640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 0x7f2ee406e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2eec005880 tx=0x7f2eec001080 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.251+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ee804b030 con 0x7f2ef80b2a60 2024-05-09T06:33:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.472+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2ef8072fa0 con 0x7f2ef80b2a60 2024-05-09T06:33:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.472+0000 7f2ef4ff9640 1 -- 172.21.15.1:0/2712209348 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f2ee8079b80 con 0x7f2ef80b2a60 2024-05-09T06:33:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:11.454 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-09T06:33:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.474+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 msgr2=0x7f2ee406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.474+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 0x7f2ee406e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2eec005880 tx=0x7f2eec001080 comp rx=0 tx=0).stop 2024-05-09T06:33:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.474+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 msgr2=0x7f2ef81e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.474+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098c0 tx=0x7f2ee80046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 shutdown_connections 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2ee406be30 0x7f2ee406e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ef81454b0 0x7f2ef815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef80b4f70 0x7f2ef813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 --2- 172.21.15.1:0/2712209348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ef80b2a60 0x7f2ef81e60f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 >> 172.21.15.1:0/2712209348 conn(0x7f2ef80cc4b0 msgr2=0x7f2ef80b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 shutdown_connections 2024-05-09T06:33:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.475+0000 7f2efe876640 1 -- 172.21.15.1:0/2712209348 wait complete. 2024-05-09T06:33:11.467 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:33:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:11.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-09T06:33:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 Processor -- start 2024-05-09T06:33:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 -- start start 2024-05-09T06:33:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdc281628f0 0x7fdc280ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280cb0e0 con 0x7fdc2815e5f0 2024-05-09T06:33:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdc280cca50 con 0x7fdc281628f0 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.594+0000 7fdc2e638640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280b17c0 con 0x7fdc2815e210 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc2cbd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdc281628f0 0x7fdc280ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:42444/0 (socket says 172.21.15.1:42444) 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 -- 172.21.15.1:0/3693865702 learned_addr learned my addr 172.21.15.1:0/3693865702 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc277fe640 1 --1- 172.21.15.1:0/3693865702 >> v1:172.21.15.6:6789/0 conn(0x7fdc2815e5f0 0x7fdc280ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:48960/0 (socket says 172.21.15.1:48960) 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 -- 172.21.15.1:0/3693865702 >> v2:172.21.15.6:3300/0 conn(0x7fdc281628f0 msgr2=0x7fdc280ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 --2- 172.21.15.1:0/3693865702 >> v2:172.21.15.6:3300/0 conn(0x7fdc281628f0 0x7fdc280ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 -- 172.21.15.1:0/3693865702 >> v1:172.21.15.6:6789/0 conn(0x7fdc2815e5f0 legacy=0x7fdc280ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.595+0000 7fdc27fff640 1 -- 172.21.15.1:0/3693865702 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc280b1aa0 con 0x7fdc2815e210 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.596+0000 7fdc27fff640 1 --2- 172.21.15.1:0/3693865702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009870 tx=0x7fdc1802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=526908b811d40851 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.596+0000 7fdc26ffd640 1 -- 172.21.15.1:0/3693865702 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18037070 con 0x7fdc2815e210 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.596+0000 7fdc26ffd640 1 -- 172.21.15.1:0/3693865702 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc18032440 con 0x7fdc2815e210 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.596+0000 7fdc26ffd640 1 -- 172.21.15.1:0/3693865702 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18032810 con 0x7fdc2815e210 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 -- 172.21.15.1:0/3693865702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 msgr2=0x7fdc280b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 --2- 172.21.15.1:0/3693865702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009870 tx=0x7fdc1802a250 comp rx=0 tx=0).stop 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 -- 172.21.15.1:0/3693865702 shutdown_connections 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 --2- 172.21.15.1:0/3693865702 >> v2:172.21.15.6:3300/0 conn(0x7fdc281628f0 0x7fdc280ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 --2- 172.21.15.1:0/3693865702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc280b1170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 -- 172.21.15.1:0/3693865702 >> 172.21.15.1:0/3693865702 conn(0x7fdc28071a50 msgr2=0x7fdc280b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 -- 172.21.15.1:0/3693865702 shutdown_connections 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.597+0000 7fdc2e638640 1 -- 172.21.15.1:0/3693865702 wait complete. 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.598+0000 7fdc2e638640 1 Processor -- start 2024-05-09T06:33:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.598+0000 7fdc2e638640 1 -- start start 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc2815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc28160070 0x7fdc280c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280cd420 con 0x7fdc281628f0 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28162d70 con 0x7fdc28160070 2024-05-09T06:33:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2e638640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280cd5a0 con 0x7fdc2815e210 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50048/0 (socket says 172.21.15.1:50048) 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 -- 172.21.15.1:0/2992909205 learned_addr learned my addr 172.21.15.1:0/2992909205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc2815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc28160070 0x7fdc280c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 -- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 msgr2=0x7fdc2815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc2815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 -- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc28160070 msgr2=0x7fdc280c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc28160070 0x7fdc280c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 -- 172.21.15.1:0/2992909205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc28161140 con 0x7fdc281628f0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c430 tx=0x7fdc1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.599+0000 7fdc24ff9640 1 -- 172.21.15.1:0/2992909205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100074e0 con 0x7fdc281628f0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.600+0000 7fdc24ff9640 1 -- 172.21.15.1:0/2992909205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10007680 con 0x7fdc281628f0 2024-05-09T06:33:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.600+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc280c7580 con 0x7fdc281628f0 2024-05-09T06:33:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.600+0000 7fdc24ff9640 1 -- 172.21.15.1:0/2992909205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004a00 con 0x7fdc281628f0 2024-05-09T06:33:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.600+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc28160b80 con 0x7fdc281628f0 2024-05-09T06:33:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.601+0000 7fdc24ff9640 1 -- 172.21.15.1:0/2992909205 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc1001a020 con 0x7fdc281628f0 2024-05-09T06:33:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.602+0000 7fdc24ff9640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.602+0000 7fdc24ff9640 1 -- 172.21.15.1:0/2992909205 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fdc1008a730 con 0x7fdc281628f0 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.602+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.602+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 0x7fdc0806e320 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdc18034000 tx=0x7fdc1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 Processor -- start 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 -- start start 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa7461ae50 0x55aa745deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa746096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa74542b60 con 0x55aa745de780 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55aa744ea780 con 0x55aa74609240 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aa7453eb20 con 0x55aa7461ae50 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50060/0 (socket says 172.21.15.1:50060) 2024-05-09T06:33:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.603+0000 7fdc27fff640 1 -- 172.21.15.1:0/700963543 learned_addr learned my addr 172.21.15.1:0/700963543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc277fe640 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa7461ae50 0x55aa745deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa746096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 -- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa7461ae50 msgr2=0x55aa745deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa7461ae50 0x55aa745deb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 -- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 msgr2=0x55aa746096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa746096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 -- 172.21.15.1:0/700963543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7431ea50 con 0x55aa745de780 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc27fff640 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009990 tx=0x7fdc1802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52260131319c569 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc1dffb640 1 -- 172.21.15.1:0/700963543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1804e070 con 0x55aa745de780 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc1dffb640 1 -- 172.21.15.1:0/700963543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc18032b20 con 0x55aa745de780 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.604+0000 7fdc1dffb640 1 -- 172.21.15.1:0/700963543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18037040 con 0x55aa745de780 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- 172.21.15.1:0/700963543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 msgr2=0x55aa745e08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009990 tx=0x7fdc1802c000 comp rx=0 tx=0).stop 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- 172.21.15.1:0/700963543 shutdown_connections 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa746096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa7461ae50 0x55aa745deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 --2- 172.21.15.1:0/700963543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa745e08e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- 172.21.15.1:0/700963543 >> 172.21.15.1:0/700963543 conn(0x55aa745a5e70 msgr2=0x55aa74624240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- 172.21.15.1:0/700963543 shutdown_connections 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- 172.21.15.1:0/700963543 wait complete. 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 -- start start 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.605+0000 7fdc31429740 1 Processor -- start 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa745f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74609730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa744fdd10 con 0x55aa745de780 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55aa74606d90 con 0x55aa7461ae50 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aa7460c130 con 0x55aa74609240 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc31429740 10 client.0 _ll_register_callbacks cb 0x55aa744f22c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50064/0 (socket says 172.21.15.1:50064) 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc27fff640 1 -- 172.21.15.1:0/2623913846 learned_addr learned my addr 172.21.15.1:0/2623913846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74609730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.606+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa745f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 -- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 msgr2=0x55aa745f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa745f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 -- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 msgr2=0x55aa74609730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74609730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 -- 172.21.15.1:0/2623913846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa742163e0 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdc180090b0 tx=0x7fdc1802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4075a246d49784 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1804e070 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fdc18055ca0 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc18032b20 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdc1805c050 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc31429740 10 client.7608 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55aa7437f1e0 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc18032b20 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fdc1805f020 con 0x55aa745de780 2024-05-09T06:33:11.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc31429740 10 client.7608 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.607+0000 7fdc31429740 10 client.7608 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc31429740 20 client.7608 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa745f68b0 0x55aa74605880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc31429740 4 client.7608 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa74550290 con 0x55aa745f68b0 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa745f68b0 0x55aa74605880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa745f68b0 0x55aa74605880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b9f6345abf78a5 server_cookie=501d2ecb97630219 in_seq=0 out_seq=0 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc06ffd640 10 client.7608 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2623913846 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa74550290 con 0x55aa745f68b0 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc06ffd640 10 client.7608 handle_command_reply: tid=0 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.608+0000 7fdc31429740 1 client.7608 shutdown 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 msgr2=0x55aa7460bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdc180090b0 tx=0x7fdc1802c070 comp rx=0 tx=0).stop 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 shutdown_connections 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74609730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa745f68b0 0x55aa74605880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa745f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 --2- 172.21.15.1:0/2623913846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa7460bbf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc06ffd640 0 client.7608 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 >> 172.21.15.1:0/2623913846 conn(0x55aa745a5e70 msgr2=0x55aa74606f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 shutdown_connections 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 1 -- 172.21.15.1:0/2623913846 wait complete. 2024-05-09T06:33:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.609+0000 7fdc31429740 20 client.7608 trim_cache size 0 max 16384 2024-05-09T06:33:11.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.626+0000 7fdc31429740 1 Processor -- start 2024-05-09T06:33:11.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 -- start start 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa745de780 0x55aa74605720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa74605c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28200d60 con 0x55aa7461ae50 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281ffd80 con 0x55aa74609240 2024-05-09T06:33:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280c4fd0 con 0x55aa745de780 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc2cbd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc2cbd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50066/0 (socket says 172.21.15.1:50066) 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.627+0000 7fdc2cbd5640 1 -- 172.21.15.1:0/172916280 learned_addr learned my addr 172.21.15.1:0/172916280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc277fe640 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa74605c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 -- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa745de780 msgr2=0x55aa74605720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa745de780 0x55aa74605720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 -- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 msgr2=0x55aa74605c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa74605c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 -- 172.21.15.1:0/172916280 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa741ef4a0 con 0x55aa7461ae50 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdc20003ce0 tx=0x7fdc20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86408cf6d6cd3801 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc1d7fa640 1 -- 172.21.15.1:0/172916280 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20017820 con 0x55aa7461ae50 2024-05-09T06:33:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc1d7fa640 1 -- 172.21.15.1:0/172916280 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200179c0 con 0x55aa7461ae50 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc1d7fa640 1 -- 172.21.15.1:0/172916280 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20017db0 con 0x55aa7461ae50 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 -- 172.21.15.1:0/172916280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 msgr2=0x55aa745dcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdc20003ce0 tx=0x7fdc20009750 comp rx=0 tx=0).stop 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 -- 172.21.15.1:0/172916280 shutdown_connections 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa7461ae50 0x55aa745dcf40 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa74609240 0x55aa74605c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 --2- 172.21.15.1:0/172916280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa745de780 0x55aa74605720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.628+0000 7fdc31429740 1 -- 172.21.15.1:0/172916280 >> 172.21.15.1:0/172916280 conn(0x55aa745a5e70 msgr2=0x55aa74626420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.629+0000 7fdc31429740 1 -- 172.21.15.1:0/172916280 shutdown_connections 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.629+0000 7fdc31429740 1 -- 172.21.15.1:0/172916280 wait complete. 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.629+0000 7fdc31429740 1 -- start start 2024-05-09T06:33:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.629+0000 7fdc31429740 1 Processor -- start 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa746918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74619bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa7453eca0 con 0x55aa745de780 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280c5620 con 0x55aa7461ae50 2024-05-09T06:33:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55aa74609020 con 0x55aa74609240 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc31429740 10 client.0 _ll_register_callbacks cb 0x7fdc1804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50076/0 (socket says 172.21.15.1:50076) 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 -- 172.21.15.1:0/2438151886 learned_addr learned my addr 172.21.15.1:0/2438151886 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74619bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa746918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 -- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 msgr2=0x55aa746918d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa746918d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 -- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 msgr2=0x55aa74619bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74619bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc27fff640 1 -- 172.21.15.1:0/2438151886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa74692160 con 0x55aa745de780 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.630+0000 7fdc2cbd5640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74619bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc27fff640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdc18032c00 tx=0x7fdc180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e9730da15ee758 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18037040 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fdc18061c80 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc200179c0 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdc18054410 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 10 client.7614 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55aa74695e00 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200179c0 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fdc1805f030 con 0x55aa745de780 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 10 client.7614 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 10 client.7614 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 20 client.7614 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa7460b570 0x55aa7461a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 4 client.7614 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-09T06:33:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55aa74628120 con 0x55aa7460b570 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.631+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa7460b570 0x55aa7461a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc06ffd640 10 client.7614 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc277fe640 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa7460b570 0x55aa7461a110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a2b36dcedb38b8 server_cookie=59e1733859256cd in_seq=0 out_seq=0 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc06ffd640 1 -- 172.21.15.1:0/2438151886 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aa74628120 con 0x55aa7460b570 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc06ffd640 10 client.7614 handle_command_reply: tid=0 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 client.7614 shutdown 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 msgr2=0x55aa74691390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdc18032c00 tx=0x7fdc180046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 shutdown_connections 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa7461ae50 0x55aa74619bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55aa7460b570 0x55aa7461a110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55aa74609240 0x55aa746918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc06ffd640 0 client.7614 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 --2- 172.21.15.1:0/2438151886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa745de780 0x55aa74691390 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.632+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 >> 172.21.15.1:0/2438151886 conn(0x55aa745a5e70 msgr2=0x55aa74633b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 shutdown_connections 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc31429740 1 -- 172.21.15.1:0/2438151886 wait complete. 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc31429740 20 client.7614 trim_cache size 0 max 16384 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "20fb9650-dbe8-444f-b29f-158fe9d3264c", 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:33:11.620 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 msgr2=0x7fdc0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 0x7fdc0806e320 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdc18034000 tx=0x7fdc1802e000 comp rx=0 tx=0).stop 2024-05-09T06:33:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 msgr2=0x7fdc2815fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c430 tx=0x7fdc1000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 shutdown_connections 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdc0806be60 0x7fdc0806e320 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc28160070 0x7fdc280c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc281628f0 0x7fdc2815fb30 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 --2- 172.21.15.1:0/2992909205 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdc2815e210 0x7fdc2815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 >> 172.21.15.1:0/2992909205 conn(0x7fdc28071a50 msgr2=0x7fdc2806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 shutdown_connections 2024-05-09T06:33:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.633+0000 7fdc2e638640 1 -- 172.21.15.1:0/2992909205 wait complete. 2024-05-09T06:33:11.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:11.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-09T06:33:11.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.750+0000 7fc5c1b68640 1 Processor -- start 2024-05-09T06:33:11.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.750+0000 7fc5c1b68640 1 -- start start 2024-05-09T06:33:11.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5c1b68640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5bc071fd0 0x7fc5bc06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5c1b68640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc15b420 con 0x7fc5bc158a50 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5c1b68640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc15b6e0 con 0x7fc5bc15c4a0 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5c1b68640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc5bc158e30 con 0x7fc5bc071fd0 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5bb7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc5bc158a50 0x7fc5bc0718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38752/0 (socket says 172.21.15.1:38752) 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.751+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/963661456 learned_addr learned my addr 172.21.15.1:0/963661456 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5baffd640 1 --2- 172.21.15.1:0/963661456 >> v2:172.21.15.6:3300/0 conn(0x7fc5bc071fd0 0x7fc5bc06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937050659 0 0) 0x7fc5bc15b420 con 0x7fc5bc158a50 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a4003520 con 0x7fc5bc158a50 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730612479 0 0) 0x7fc5bc15b6e0 con 0x7fc5bc15c4a0 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5bc15b420 con 0x7fc5bc15c4a0 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3938820557 0 0) 0x7fc5a4003520 con 0x7fc5bc158a50 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5baffd640 1 -- 172.21.15.1:0/963661456 >> v1:172.21.15.6:6789/0 conn(0x7fc5bc15c4a0 legacy=0x7fc5bc15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5baffd640 1 -- 172.21.15.1:0/963661456 >> v1:172.21.15.1:6789/0 conn(0x7fc5bc158a50 legacy=0x7fc5bc0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.752+0000 7fc5baffd640 1 -- 172.21.15.1:0/963661456 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc15b9c0 con 0x7fc5bc071fd0 2024-05-09T06:33:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5baffd640 1 --2- 172.21.15.1:0/963661456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc06e6d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00097a0 tx=0x7fc5b002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b9b15eb571a0f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037070 con 0x7fc5bc071fd0 2024-05-09T06:33:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b002ac50 con 0x7fc5bc071fd0 2024-05-09T06:33:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/963661456 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac001670 con 0x7fc5bc071fd0 2024-05-09T06:33:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 -- 172.21.15.1:0/963661456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc071fd0 msgr2=0x7fc5bc06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/963661456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc06e6d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00097a0 tx=0x7fc5b002a1c0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 -- 172.21.15.1:0/963661456 shutdown_connections 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/963661456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc06e6d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 -- 172.21.15.1:0/963661456 >> 172.21.15.1:0/963661456 conn(0x7fc5bc0cc4d0 msgr2=0x7fc5bc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 -- 172.21.15.1:0/963661456 shutdown_connections 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.753+0000 7fc5c1b68640 1 -- 172.21.15.1:0/963661456 wait complete. 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.755+0000 7fc5c1b68640 1 Processor -- start 2024-05-09T06:33:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.755+0000 7fc5c1b68640 1 -- start start 2024-05-09T06:33:11.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc158a50 0x7fc5bc14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5bc15c4a0 0x7fc5bc06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc15bf80 con 0x7fc5bc071fd0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc072480 con 0x7fc5bc158a50 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc070ea0 con 0x7fc5bc15c4a0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50086/0 (socket says 172.21.15.1:50086) 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1391943471 learned_addr learned my addr 172.21.15.1:0/1391943471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5baffd640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc158a50 0x7fc5bc14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5bc15c4a0 0x7fc5bc06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5bc15c4a0 msgr2=0x7fc5bc06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5bc15c4a0 0x7fc5bc06d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc158a50 msgr2=0x7fc5bc14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc158a50 0x7fc5bc14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1391943471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc15de10 con 0x7fc5bc071fd0 2024-05-09T06:33:11.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c750 tx=0x7fc5ac00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5a3fff640 1 -- 172.21.15.1:0/1391943471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac017070 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5a3fff640 1 -- 172.21.15.1:0/1391943471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac005ad0 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.756+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc156ce0 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.757+0000 7fc5a3fff640 1 -- 172.21.15.1:0/1391943471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac007070 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.757+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc156fa0 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.758+0000 7fc5a3fff640 1 -- 172.21.15.1:0/1391943471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5ac01a070 con 0x7fc5bc071fd0 2024-05-09T06:33:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5a3fff640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 0x7fc5a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5a3fff640 1 -- 172.21.15.1:0/1391943471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fc5ac08a640 con 0x7fc5bc071fd0 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5baffd640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 0x7fc5a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5baffd640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 0x7fc5a806e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0034000 tx=0x7fc5b002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5c4872740 1 Processor -- start 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.759+0000 7fc5c4872740 1 -- start start 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c893baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 0x55c3c8910e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3c8875d00 con 0x55c3c8959650 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c3c881d4c0 con 0x55c3c893c030 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c3c8871cc0 con 0x55c3c893c770 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50090/0 (socket says 172.21.15.1:50090) 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1067936851 learned_addr learned my addr 172.21.15.1:0/1067936851 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 0x55c3c8910e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.760+0000 7fc5baffd640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c893baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 msgr2=0x55c3c893baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c893baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 msgr2=0x55c3c8910e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 0x55c3c8910e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 -- 172.21.15.1:0/1067936851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c86537d0 con 0x55c3c8959650 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 0x55c3c8910e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac054af0 tx=0x7fc5ac058590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23ffc55a4a26b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5a17fa640 1 -- 172.21.15.1:0/1067936851 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac056990 con 0x55c3c8959650 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5a17fa640 1 -- 172.21.15.1:0/1067936851 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac054d00 con 0x55c3c8959650 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.761+0000 7fc5a17fa640 1 -- 172.21.15.1:0/1067936851 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac061000 con 0x55c3c8959650 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- 172.21.15.1:0/1067936851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 msgr2=0x55c3c890bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac054af0 tx=0x7fc5ac058590 comp rx=0 tx=0).stop 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- 172.21.15.1:0/1067936851 shutdown_connections 2024-05-09T06:33:11.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c893c030 0x55c3c8910e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c893baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 --2- 172.21.15.1:0/1067936851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c890bfe0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- 172.21.15.1:0/1067936851 >> 172.21.15.1:0/1067936851 conn(0x55c3c890e250 msgr2=0x55c3c8956af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- 172.21.15.1:0/1067936851 shutdown_connections 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- 172.21.15.1:0/1067936851 wait complete. 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 -- start start 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.762+0000 7fc5c4872740 1 Processor -- start 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c8937500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8910d90 0x55c3c8911210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3c8910400 con 0x55c3c8959650 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c3c8830b90 con 0x55c3c8910d90 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c3c890f7f0 con 0x55c3c893c770 2024-05-09T06:33:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5c4872740 10 client.0 _ll_register_callbacks cb 0x55c3c886cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50102/0 (socket says 172.21.15.1:50102) 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5baffd640 1 -- 172.21.15.1:0/106782250 learned_addr learned my addr 172.21.15.1:0/106782250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8910d90 0x55c3c8911210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.763+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c8937500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 -- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 msgr2=0x55c3c8937500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c8937500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 -- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8910d90 msgr2=0x55c3c8911210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8910d90 0x55c3c8911210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 -- 172.21.15.1:0/106782250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c85893f0 con 0x55c3c8959650 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5baffd640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0035040 tx=0x7fc5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de70903de65ce094 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037040 con 0x55c3c8959650 2024-05-09T06:33:11.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fc5b0032e00 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac054d00 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc5b005b050 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 10 client.7626 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55c3c85490f0 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac054d00 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fc5b0053480 con 0x55c3c8959650 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 10 client.7626 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 10 client.7626 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 20 client.7626 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c89672c0 0x55c3c893cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 4 client.7626 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.764+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c8883430 con 0x55c3c89672c0 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c89672c0 0x55c3c893cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc59a7fc640 10 client.7626 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c89672c0 0x55c3c893cc60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f5b98baafdb47b server_cookie=15033c153aaeb5ec in_seq=0 out_seq=0 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc59a7fc640 1 -- 172.21.15.1:0/106782250 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3c8883430 con 0x55c3c89672c0 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc59a7fc640 10 client.7626 handle_command_reply: tid=0 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.765+0000 7fc5c4872740 1 client.7626 shutdown 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 msgr2=0x55c3c893ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0035040 tx=0x7fc5b0004730 comp rx=0 tx=0).stop 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 shutdown_connections 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8910d90 0x55c3c8911210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c8959650 0x55c3c893ede0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c89672c0 0x55c3c893cc60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 --2- 172.21.15.1:0/106782250 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c893c770 0x55c3c8937500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc59a7fc640 0 client.7626 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 >> 172.21.15.1:0/106782250 conn(0x55c3c893c030 msgr2=0x55c3c8958d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 shutdown_connections 2024-05-09T06:33:11.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 1 -- 172.21.15.1:0/106782250 wait complete. 2024-05-09T06:33:11.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.766+0000 7fc5c4872740 20 client.7626 trim_cache size 0 max 16384 2024-05-09T06:33:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.783+0000 7fc5c4872740 1 Processor -- start 2024-05-09T06:33:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 -- start start 2024-05-09T06:33:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c890cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c894eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc200c20 con 0x55c3c893c770 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1ffc70 con 0x55c3c8959650 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc0ae1a0 con 0x55c3c8910d90 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.784+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50108/0 (socket says 172.21.15.1:50108) 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 -- 172.21.15.1:0/3565585791 learned_addr learned my addr 172.21.15.1:0/3565585791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c894eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c890cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 -- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 msgr2=0x55c3c890cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c890cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 -- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 msgr2=0x55c3c894eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c894eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 -- 172.21.15.1:0/3565585791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c890f390 con 0x55c3c893c770 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5baffd640 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006f00 tx=0x7fc5b004a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9104b6075487658 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5a0ff9640 1 -- 172.21.15.1:0/3565585791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0064070 con 0x55c3c893c770 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5a0ff9640 1 -- 172.21.15.1:0/3565585791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0053980 con 0x55c3c893c770 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.785+0000 7fc5a0ff9640 1 -- 172.21.15.1:0/3565585791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037070 con 0x55c3c893c770 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- 172.21.15.1:0/3565585791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 msgr2=0x55c3c893e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006f00 tx=0x7fc5b004a070 comp rx=0 tx=0).stop 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- 172.21.15.1:0/3565585791 shutdown_connections 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c894eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c893e6c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 --2- 172.21.15.1:0/3565585791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c890cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- 172.21.15.1:0/3565585791 >> 172.21.15.1:0/3565585791 conn(0x55c3c893c030 msgr2=0x55c3c8937310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- 172.21.15.1:0/3565585791 shutdown_connections 2024-05-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- 172.21.15.1:0/3565585791 wait complete. 2024-05-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 -- start start 2024-05-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.786+0000 7fc5c4872740 1 Processor -- start 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c8965d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c8957ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3c8966b60 con 0x55c3c893c770 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1ff830 con 0x55c3c8959650 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c3c894e710 con 0x55c3c8910d90 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5c4872740 10 client.0 _ll_register_callbacks cb 0x7fc5b0040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50120/0 (socket says 172.21.15.1:50120) 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5baffd640 1 -- 172.21.15.1:0/763051503 learned_addr learned my addr 172.21.15.1:0/763051503 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5bbfff640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c8957ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.787+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c8965d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 -- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 msgr2=0x55c3c8965d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c8965d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 -- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 msgr2=0x55c3c8957ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c8957ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 -- 172.21.15.1:0/763051503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c8938230 con 0x55c3c893c770 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5baffd640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0062040 tx=0x7fc5b0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd5db821b5f1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0064050 con 0x55c3c893c770 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fc5b005f430 con 0x55c3c893c770 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b0053980 con 0x55c3c893c770 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc5b0037040 con 0x55c3c893c770 2024-05-09T06:33:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 10 client.7632 fetch_fsmap learned FSMap version 52 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55c3c894eb10 con 0x55c3c893c770 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0053980 con 0x55c3c893c770 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fc5b0068630 con 0x55c3c893c770 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 10 client.7632 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 10 client.7632 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 20 client.7632 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c890cd20 0x55c3c8957ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 4 client.7632 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.788+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3c887d780 con 0x55c3c890cd20 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c890cd20 0x55c3c8957ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc59a7fc640 10 client.7632 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5bb7fe640 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c890cd20 0x55c3c8957ff0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cadc481b5cab93e server_cookie=bfb74ab0b636fc02 in_seq=0 out_seq=0 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc59a7fc640 1 -- 172.21.15.1:0/763051503 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55c3c887d780 con 0x55c3c890cd20 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc59a7fc640 10 client.7632 handle_command_reply: tid=0 2024-05-09T06:33:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 client.7632 shutdown 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 msgr2=0x55c3c8966260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0062040 tx=0x7fc5b0009190 comp rx=0 tx=0).stop 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 shutdown_connections 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c3c8959650 0x55c3c8957ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3c893c770 0x55c3c8966260 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x55c3c890cd20 0x55c3c8957ff0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 --2- 172.21.15.1:0/763051503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c3c8910d90 0x55c3c8965d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 >> 172.21.15.1:0/763051503 conn(0x55c3c893c030 msgr2=0x55c3c893ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc59a7fc640 0 client.7632 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 shutdown_connections 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 1 -- 172.21.15.1:0/763051503 wait complete. 2024-05-09T06:33:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.789+0000 7fc5c4872740 20 client.7632 trim_cache size 0 max 16384 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "20fb9650-dbe8-444f-b29f-158fe9d3264c": { 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/subdir", 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "20fb9650-dbe8-444f-b29f-158fe9d3264c", 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:11.778 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:11.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 msgr2=0x7fc5a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 0x7fc5a806e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0034000 tx=0x7fc5b002e000 comp rx=0 tx=0).stop 2024-05-09T06:33:11.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 msgr2=0x7fc5bc1567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:11.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c750 tx=0x7fc5ac00cc10 comp rx=0 tx=0).stop 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 shutdown_connections 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5a806be30 0x7fc5a806e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5bc15c4a0 0x7fc5bc06d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5bc158a50 0x7fc5bc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 --2- 172.21.15.1:0/1391943471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5bc071fd0 0x7fc5bc1567a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 >> 172.21.15.1:0/1391943471 conn(0x7fc5bc0cc4d0 msgr2=0x7fc5bc06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 shutdown_connections 2024-05-09T06:33:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:11.790+0000 7fc5c1b68640 1 -- 172.21.15.1:0/1391943471 wait complete. 2024-05-09T06:33:11.782 INFO:tasks.cephfs.filesystem:scrub status for tag:20fb9650-dbe8-444f-b29f-158fe9d3264c - {'path': '/subdir', 'tag': '20fb9650-dbe8-444f-b29f-158fe9d3264c', 'options': 'recursive,repair'} 2024-05-09T06:33:18.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:33:18.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:33:41.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:41.783 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-09T06:33:41.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.921+0000 7f849ced2640 1 Processor -- start 2024-05-09T06:33:41.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.921+0000 7f849ced2640 1 -- start start 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f849815e270 0x7f84980cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84980d08b0 0x7f84980ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980ce9d0 0x7f849806c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f84980cee20 con 0x7f84980d08b0 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84980cc260 con 0x7f849815e270 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f849ced2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f84980b1340 con 0x7f84980ce9d0 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.922+0000 7f8495d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84980d08b0 0x7f84980ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84980d08b0 0x7f84980ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42838/0 (socket says 172.21.15.1:42838) 2024-05-09T06:33:41.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 -- 172.21.15.1:0/152832963 learned_addr learned my addr 172.21.15.1:0/152832963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f849659c640 1 --2- 172.21.15.1:0/152832963 >> v2:172.21.15.6:3300/0 conn(0x7f849815e270 0x7f84980cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8496d9d640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980ce9d0 0x7f849806c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 -- 172.21.15.1:0/152832963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980ce9d0 msgr2=0x7f849806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980ce9d0 0x7f849806c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 -- 172.21.15.1:0/152832963 >> v2:172.21.15.6:3300/0 conn(0x7f849815e270 msgr2=0x7f84980cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 --2- 172.21.15.1:0/152832963 >> v2:172.21.15.6:3300/0 conn(0x7f849815e270 0x7f84980cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.923+0000 7f8495d9b640 1 -- 172.21.15.1:0/152832963 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84980d1020 con 0x7f84980d08b0 2024-05-09T06:33:41.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f8495d9b640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980d08b0 0x7f84980ce490 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8498072820 tx=0x7f848002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2b06665eb6aaa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849559a640 1 -- 172.21.15.1:0/152832963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480037070 con 0x7f84980d08b0 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849559a640 1 -- 172.21.15.1:0/152832963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8480032440 con 0x7f84980d08b0 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849559a640 1 -- 172.21.15.1:0/152832963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480032810 con 0x7f84980d08b0 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 -- 172.21.15.1:0/152832963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980d08b0 msgr2=0x7f84980ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980d08b0 0x7f84980ce490 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8498072820 tx=0x7f848002a270 comp rx=0 tx=0).stop 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 -- 172.21.15.1:0/152832963 shutdown_connections 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980ce9d0 0x7f849806c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 --2- 172.21.15.1:0/152832963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980d08b0 0x7f84980ce490 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 --2- 172.21.15.1:0/152832963 >> v2:172.21.15.6:3300/0 conn(0x7f849815e270 0x7f84980cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 -- 172.21.15.1:0/152832963 >> 172.21.15.1:0/152832963 conn(0x7f84980d1c10 msgr2=0x7f84980d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 -- 172.21.15.1:0/152832963 shutdown_connections 2024-05-09T06:33:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.924+0000 7f849ced2640 1 -- 172.21.15.1:0/152832963 wait complete. 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.926+0000 7f849ced2640 1 Processor -- start 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.926+0000 7f849ced2640 1 -- start start 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980d08b0 0x7f84980c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849815e270 0x7f84980c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84980cca70 con 0x7f84980ce9d0 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84980cc790 con 0x7f849815e270 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849ced2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f84980d0d20 con 0x7f84980d08b0 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42850/0 (socket says 172.21.15.1:42850) 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 -- 172.21.15.1:0/3112109025 learned_addr learned my addr 172.21.15.1:0/3112109025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f8495d9b640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980d08b0 0x7f84980c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f8496d9d640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849815e270 0x7f84980c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 -- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980d08b0 msgr2=0x7f84980c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980d08b0 0x7f84980c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 -- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849815e270 msgr2=0x7f84980c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849815e270 0x7f84980c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 -- 172.21.15.1:0/3112109025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84980c6100 con 0x7f84980ce9d0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f849659c640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8488007f20 tx=0x7f848800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.927+0000 7f848effd640 1 -- 172.21.15.1:0/3112109025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488014620 con 0x7f84980ce9d0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.928+0000 7f848effd640 1 -- 172.21.15.1:0/3112109025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8488019070 con 0x7f84980ce9d0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.928+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84980c63e0 con 0x7f84980ce9d0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.928+0000 7f848effd640 1 -- 172.21.15.1:0/3112109025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488014a00 con 0x7f84980ce9d0 2024-05-09T06:33:41.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.928+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84980c69a0 con 0x7f84980ce9d0 2024-05-09T06:33:41.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.930+0000 7f848effd640 1 -- 172.21.15.1:0/3112109025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8488014ba0 con 0x7f84980ce9d0 2024-05-09T06:33:41.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.930+0000 7f848effd640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 0x7f847806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.930+0000 7f848effd640 1 -- 172.21.15.1:0/3112109025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f848808b810 con 0x7f84980ce9d0 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.930+0000 7f8495d9b640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 0x7f847806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.931+0000 7f8495d9b640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 0x7f847806e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8480034000 tx=0x7f848002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.931+0000 7f849fbdd740 1 Processor -- start 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.931+0000 7f849fbdd740 1 -- start start 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067567af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568030 0x56406753ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5640674a1d00 con 0x564067585650 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5640674494c0 con 0x564067568770 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56406749dcc0 con 0x564067568030 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42866/0 (socket says 172.21.15.1:42866) 2024-05-09T06:33:41.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 -- 172.21.15.1:0/1248505882 learned_addr learned my addr 172.21.15.1:0/1248505882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f8496d9d640 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568030 0x56406753ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 -- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568030 msgr2=0x56406753ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067567af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568030 0x56406753ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 -- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 msgr2=0x564067567af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067567af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 -- 172.21.15.1:0/1248505882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56406727f7d0 con 0x564067585650 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.932+0000 7f849659c640 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f848800c040 tx=0x7f8488059900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57c2dc601545495 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f8477fff640 1 -- 172.21.15.1:0/1248505882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488057c00 con 0x564067585650 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f8477fff640 1 -- 172.21.15.1:0/1248505882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8488055f70 con 0x564067585650 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f8477fff640 1 -- 172.21.15.1:0/1248505882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488062250 con 0x564067585650 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 -- 172.21.15.1:0/1248505882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 msgr2=0x564067537fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f848800c040 tx=0x7f8488059900 comp rx=0 tx=0).stop 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 -- 172.21.15.1:0/1248505882 shutdown_connections 2024-05-09T06:33:41.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568030 0x56406753ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067567af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1248505882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067537fe0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 -- 172.21.15.1:0/1248505882 >> 172.21.15.1:0/1248505882 conn(0x56406753a250 msgr2=0x564067582af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 -- 172.21.15.1:0/1248505882 shutdown_connections 2024-05-09T06:33:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.933+0000 7f849fbdd740 1 -- 172.21.15.1:0/1248505882 wait complete. 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.934+0000 7f849fbdd740 1 -- start start 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.934+0000 7f849fbdd740 1 Processor -- start 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067563670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56406756b960 0x564067583e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56406753c400 con 0x564067585650 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56406745cb90 con 0x564067568770 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56406753b7f0 con 0x56406756b960 2024-05-09T06:33:41.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849fbdd740 10 client.0 _ll_register_callbacks cb 0x564067498da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42876/0 (socket says 172.21.15.1:42876) 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f8495d9b640 1 -- 172.21.15.1:0/1449446060 learned_addr learned my addr 172.21.15.1:0/1449446060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f8496d9d640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56406756b960 0x564067583e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.935+0000 7f849659c640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067563670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 -- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56406756b960 msgr2=0x564067583e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56406756b960 0x564067583e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 -- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 msgr2=0x564067563670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067563670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 -- 172.21.15.1:0/1449446060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640671b53f0 con 0x564067585650 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f84800090b0 tx=0x7f8480004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a91bf44e324219 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480037040 con 0x564067585650 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f848003c640 con 0x564067585650 2024-05-09T06:33:41.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8488055f70 con 0x564067585650 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f848003c840 con 0x564067585650 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 10 client.7644 fetch_fsmap learned FSMap version 53 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5640671750f0 con 0x564067585650 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8488055f70 con 0x564067585650 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f848003c9e0 con 0x564067585650 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 10 client.7644 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 10 client.7644 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 20 client.7644 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5640675797b0 0x564067579c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 4 client.7644 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.936+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640674af430 con 0x5640675797b0 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f849659c640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5640675797b0 0x564067579c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f849659c640 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5640675797b0 0x564067579c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb7b57dc4fdc4c2 server_cookie=80cf8261ebb5e45a in_seq=0 out_seq=0 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f84757fa640 10 client.7644 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:41.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f84757fa640 1 -- 172.21.15.1:0/1449446060 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640674af430 con 0x5640675797b0 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f84757fa640 10 client.7644 handle_command_reply: tid=0 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.937+0000 7f849fbdd740 1 client.7644 shutdown 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 msgr2=0x564067593070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f84800090b0 tx=0x7f8480004730 comp rx=0 tx=0).stop 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 shutdown_connections 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x5640675797b0 0x564067579c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56406756b960 0x564067583e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067593070 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1449446060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x564067563670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f84757fa640 0 client.7644 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 >> 172.21.15.1:0/1449446060 conn(0x564067568030 msgr2=0x564067584b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 shutdown_connections 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 1 -- 172.21.15.1:0/1449446060 wait complete. 2024-05-09T06:33:41.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.938+0000 7f849fbdd740 20 client.7644 trim_cache size 0 max 16384 2024-05-09T06:33:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.956+0000 7f849fbdd740 1 Processor -- start 2024-05-09T06:33:41.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.956+0000 7f849fbdd740 1 -- start start 2024-05-09T06:33:41.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x56406753d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067585650 0x564067582ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8498200d30 con 0x56406756b960 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84981ffd70 con 0x564067568770 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f84981609c0 con 0x564067585650 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42878/0 (socket says 172.21.15.1:42878) 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 -- 172.21.15.1:0/1844608823 learned_addr learned my addr 172.21.15.1:0/1844608823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 -- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067585650 msgr2=0x564067582ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067585650 0x564067582ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 -- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 msgr2=0x56406753d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x56406753d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 -- 172.21.15.1:0/1844608823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564067583550 con 0x56406756b960 2024-05-09T06:33:41.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f8495d9b640 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7de5e9430b0c258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f84777fe640 1 -- 172.21.15.1:0/1844608823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848005f070 con 0x56406756b960 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f84777fe640 1 -- 172.21.15.1:0/1844608823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848004e070 con 0x56406756b960 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.957+0000 7f84777fe640 1 -- 172.21.15.1:0/1844608823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480037040 con 0x56406756b960 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 -- 172.21.15.1:0/1844608823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 msgr2=0x564067579b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480045000 comp rx=0 tx=0).stop 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 -- 172.21.15.1:0/1844608823 shutdown_connections 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067585650 0x564067582ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56406756b960 0x564067579b90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 --2- 172.21.15.1:0/1844608823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564067568770 0x56406753d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 -- 172.21.15.1:0/1844608823 >> 172.21.15.1:0/1844608823 conn(0x564067568030 msgr2=0x564067584a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 -- 172.21.15.1:0/1844608823 shutdown_connections 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.958+0000 7f849fbdd740 1 -- 172.21.15.1:0/1844608823 wait complete. 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.959+0000 7f849fbdd740 1 -- start start 2024-05-09T06:33:41.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.959+0000 7f849fbdd740 1 Processor -- start 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568770 0x5640675e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56406756b960 0x5640675911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564067567e80 con 0x564067585650 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8498161010 con 0x56406756b960 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x564067539740 con 0x564067568770 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849fbdd740 10 client.0 _ll_register_callbacks cb 0x7f84800096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42884/0 (socket says 172.21.15.1:42884) 2024-05-09T06:33:41.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 -- 172.21.15.1:0/3254523009 learned_addr learned my addr 172.21.15.1:0/3254523009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f849659c640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568770 0x5640675e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8495d9b640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56406756b960 0x5640675911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 -- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568770 msgr2=0x5640675e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568770 0x5640675e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 -- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56406756b960 msgr2=0x5640675911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56406756b960 0x5640675911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 -- 172.21.15.1:0/3254523009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564067569970 con 0x564067585650 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f8496d9d640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8490007d70 tx=0x7f8490014720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd31c5e728adc99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.960+0000 7f84757fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490004030 con 0x564067585650 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848004e070 con 0x564067585650 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f84900041d0 con 0x564067585650 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8490026050 con 0x564067585650 2024-05-09T06:33:41.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 10 client.7650 fetch_fsmap learned FSMap version 53 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5640675927f0 con 0x564067585650 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848004e070 con 0x564067585650 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8490029020 con 0x564067585650 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 10 client.7650 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 10 client.7650 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 20 client.7650 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56406757a740 0x5640675619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 4 client.7650 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640674a9780 con 0x56406757a740 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849659c640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56406757a740 0x5640675619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f84757fa640 10 client.7650 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.961+0000 7f849659c640 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56406757a740 0x5640675619d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724a7448a30873ef server_cookie=d308dc008927429a in_seq=0 out_seq=0 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f84757fa640 1 -- 172.21.15.1:0/3254523009 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640674a9780 con 0x56406757a740 2024-05-09T06:33:41.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f84757fa640 10 client.7650 handle_command_reply: tid=0 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 client.7650 shutdown 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 msgr2=0x564067591730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8490007d70 tx=0x7f8490014720 comp rx=0 tx=0).stop 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 shutdown_connections 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564067585650 0x564067591730 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56406756b960 0x5640675911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x56406757a740 0x5640675619d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 --2- 172.21.15.1:0/3254523009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564067568770 0x5640675e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 >> 172.21.15.1:0/3254523009 conn(0x564067568030 msgr2=0x564067582ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f84757fa640 0 client.7650 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 shutdown_connections 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 1 -- 172.21.15.1:0/3254523009 wait complete. 2024-05-09T06:33:41.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.962+0000 7f849fbdd740 20 client.7650 trim_cache size 0 max 16384 2024-05-09T06:33:41.950 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:33:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:33:41.950 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:41.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 msgr2=0x7f847806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 0x7f847806e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8480034000 tx=0x7f848002e000 comp rx=0 tx=0).stop 2024-05-09T06:33:41.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 msgr2=0x7f84981f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:41.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8488007f20 tx=0x7f848800e570 comp rx=0 tx=0).stop 2024-05-09T06:33:41.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 shutdown_connections 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f847806be30 0x7f847806e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849815e270 0x7f84980c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f84980d08b0 0x7f84980c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 --2- 172.21.15.1:0/3112109025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84980ce9d0 0x7f84981f4180 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 >> 172.21.15.1:0/3112109025 conn(0x7f84980d1c10 msgr2=0x7f84980aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 shutdown_connections 2024-05-09T06:33:41.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:41.963+0000 7f849ced2640 1 -- 172.21.15.1:0/3112109025 wait complete. 2024-05-09T06:33:41.955 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:33:41.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:42.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.077+0000 7fc2fcd96640 1 Processor -- start 2024-05-09T06:33:42.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.077+0000 7fc2fcd96640 1 -- start start 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2fcd96640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc2f80703d0 0x7fc2f815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2fcd96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2f806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2fcd96640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f815cdc0 con 0x7fc2f8153150 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2fcd96640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f8072d20 con 0x7fc2f80703d0 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2fcd96640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f815d380 con 0x7fc2f806b1e0 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc2f8153150 0x7fc2f8072610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39040/0 (socket says 172.21.15.1:39040) 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f659c640 1 -- 172.21.15.1:0/3425978016 learned_addr learned my addr 172.21.15.1:0/3425978016 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/3425978016 >> v2:172.21.15.6:3300/0 conn(0x7fc2f80703d0 0x7fc2f815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 999276064 0 0) 0x7fc2f815cdc0 con 0x7fc2f8153150 2024-05-09T06:33:42.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f6d9d640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2f806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2e4003520 con 0x7fc2f8153150 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1523467788 0 0) 0x7fc2e4003520 con 0x7fc2f8153150 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 msgr2=0x7fc2f806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2f806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 >> v2:172.21.15.6:3300/0 conn(0x7fc2f80703d0 msgr2=0x7fc2f815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 --2- 172.21.15.1:0/3425978016 >> v2:172.21.15.6:3300/0 conn(0x7fc2f80703d0 0x7fc2f815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f815d0a0 con 0x7fc2f8153150 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fc2e8001670 con 0x7fc2f8153150 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 >> v1:172.21.15.1:6789/0 conn(0x7fc2f8153150 legacy=0x7fc2f8072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.078+0000 7fc2f5d9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e40039b0 0x7fc2e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f5d9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2e4004800 0x7fc2e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f5d9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8072d20 con 0x7fc2e4004800 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f815d380 con 0x7fc2e40039b0 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e400b9c0 con 0x7fc2f806b1e0 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2e4004800 0x7fc2e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f659c640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e40039b0 0x7fc2e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2f6d9d640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 -- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 msgr2=0x7fc2e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2f806b1e0 0x7fc2e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 -- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e40039b0 msgr2=0x7fc2e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e40039b0 0x7fc2e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.079+0000 7fc2edd9b640 1 -- 172.21.15.1:0/3425978016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e4003520 con 0x7fc2e4004800 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2e4004800 0x7fc2e4004c80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009af0 tx=0x7fc2e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b9d2a3f483a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0036070 con 0x7fc2e4004800 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e0031400 con 0x7fc2e4004800 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2f5d9b640 1 -- 172.21.15.1:0/3425978016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0009690 con 0x7fc2e4004800 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 -- 172.21.15.1:0/3425978016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2e4004800 msgr2=0x7fc2e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/3425978016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2e4004800 0x7fc2e4004c80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009af0 tx=0x7fc2e4005830 comp rx=0 tx=0).stop 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2f6d9d640 1 -- 172.21.15.1:0/3425978016 reap_dead start 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 -- 172.21.15.1:0/3425978016 shutdown_connections 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 -- 172.21.15.1:0/3425978016 >> 172.21.15.1:0/3425978016 conn(0x7fc2f80d0b50 msgr2=0x7fc2f80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 -- 172.21.15.1:0/3425978016 shutdown_connections 2024-05-09T06:33:42.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.080+0000 7fc2fcd96640 1 -- 172.21.15.1:0/3425978016 wait complete. 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.082+0000 7fc2fcd96640 1 Processor -- start 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.082+0000 7fc2fcd96640 1 -- start start 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2e4004800 0x7fc2f81f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 0x7fc2f8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f815c9b0 con 0x7fc2f80c6770 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8072180 con 0x7fc2f8158810 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2fcd96640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f815cb30 con 0x7fc2e4004800 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42904/0 (socket says 172.21.15.1:42904) 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 -- 172.21.15.1:0/646654913 learned_addr learned my addr 172.21.15.1:0/646654913 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2f6d9d640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 0x7fc2f8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 -- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2e4004800 msgr2=0x7fc2f81f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2f659c640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2e4004800 0x7fc2f81f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2e4004800 0x7fc2f81f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 -- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 msgr2=0x7fc2f8158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 0x7fc2f8158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 -- 172.21.15.1:0/646654913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f8153b10 con 0x7fc2f80c6770 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2f6d9d640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 0x7fc2f8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2edd9b640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009af0 tx=0x7fc2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.083+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0040020 con 0x7fc2f80c6770 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.084+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e0003830 con 0x7fc2f80c6770 2024-05-09T06:33:42.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.084+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f8153df0 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.084+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0036040 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.084+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f81543b0 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.085+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f806ed00 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.085+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2e0043020 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.085+0000 7fc2ef7fe640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 0x7fc2d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.086+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fc2e00ab2f0 con 0x7fc2f80c6770 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.086+0000 7fc2f659c640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 0x7fc2d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.086+0000 7fc2f659c640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 0x7fc2d806e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8007f70 tx=0x7fc2e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.090+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2f806ed00 con 0x7fc2f80c6770 2024-05-09T06:33:42.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.310+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2f815ccb0 con 0x7fc2f80c6770 2024-05-09T06:33:42.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.310+0000 7fc2ef7fe640 1 -- 172.21.15.1:0/646654913 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5170 (secure 0 0 0) 0x7fc2f815ccb0 con 0x7fc2f80c6770 2024-05-09T06:33:42.292 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:33:18.377403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":23,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:42.292 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 53 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 msgr2=0x7fc2d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 0x7fc2d806e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8007f70 tx=0x7fc2e8009040 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 msgr2=0x7fc2f80c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009af0 tx=0x7fc2e0004730 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 shutdown_connections 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2d806be60 0x7fc2d806e320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f8158810 0x7fc2f8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f80c6770 0x7fc2f80c6bf0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 --2- 172.21.15.1:0/646654913 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc2e4004800 0x7fc2f81f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 >> 172.21.15.1:0/646654913 conn(0x7fc2f80d0b50 msgr2=0x7fc2f815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 shutdown_connections 2024-05-09T06:33:42.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.312+0000 7fc2fcd96640 1 -- 172.21.15.1:0/646654913 wait complete. 2024-05-09T06:33:42.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-09T06:33:42.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.429+0000 7f9a19f6a640 1 Processor -- start 2024-05-09T06:33:42.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.429+0000 7f9a19f6a640 1 -- start start 2024-05-09T06:33:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a19f6a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9a140d1c70 0x7f9a1415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a19f6a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9a140d45f0 0x7f9a140b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a19f6a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a14161fe0 con 0x7f9a1415e190 2024-05-09T06:33:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a19f6a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1415f580 con 0x7f9a140d45f0 2024-05-09T06:33:42.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a19f6a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9a141622c0 con 0x7f9a140d1c70 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9a140d45f0 0x7f9a140b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9a140d45f0 0x7f9a140b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42908/0 (socket says 172.21.15.1:42908) 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 -- 172.21.15.1:0/4129586078 learned_addr learned my addr 172.21.15.1:0/4129586078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 -- 172.21.15.1:0/4129586078 >> v2:172.21.15.6:3300/0 conn(0x7f9a140d1c70 msgr2=0x7f9a1415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/4129586078 >> v2:172.21.15.6:3300/0 conn(0x7f9a140d1c70 0x7f9a1415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4129586078 >> v2:172.21.15.6:3300/0 conn(0x7f9a140d1c70 0x7f9a1415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 -- 172.21.15.1:0/4129586078 >> v1:172.21.15.6:6789/0 conn(0x7f9a1415e190 legacy=0x7f9a140d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.430+0000 7f9a13fff640 1 -- 172.21.15.1:0/4129586078 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a141625a0 con 0x7f9a140d45f0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.431+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4129586078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140d45f0 0x7f9a140b3de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9a140b6c30 tx=0x7f9a0c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e535cff54afc3911 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.431+0000 7f9a127fc640 1 -- 172.21.15.1:0/4129586078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c036070 con 0x7f9a140d45f0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.431+0000 7f9a127fc640 1 -- 172.21.15.1:0/4129586078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c031600 con 0x7f9a140d45f0 2024-05-09T06:33:42.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.431+0000 7f9a127fc640 1 -- 172.21.15.1:0/4129586078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0319d0 con 0x7f9a140d45f0 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 -- 172.21.15.1:0/4129586078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140d45f0 msgr2=0x7f9a140b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/4129586078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140d45f0 0x7f9a140b3de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9a140b6c30 tx=0x7f9a0c02a6f0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 -- 172.21.15.1:0/4129586078 shutdown_connections 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/4129586078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140d45f0 0x7f9a140b3de0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/4129586078 >> v2:172.21.15.6:3300/0 conn(0x7f9a140d1c70 0x7f9a1415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 -- 172.21.15.1:0/4129586078 >> 172.21.15.1:0/4129586078 conn(0x7f9a14073340 msgr2=0x7f9a1409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 -- 172.21.15.1:0/4129586078 shutdown_connections 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.432+0000 7f9a19f6a640 1 -- 172.21.15.1:0/4129586078 wait complete. 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.433+0000 7f9a19f6a640 1 Processor -- start 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.433+0000 7f9a19f6a640 1 -- start start 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 0x7f9a14160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 0x7f9a140b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a140b56d0 con 0x7f9a1415e190 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14161c60 con 0x7f9a140d1c70 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a19f6a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9a140b5850 con 0x7f9a140d45f0 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42912/0 (socket says 172.21.15.1:42912) 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 -- 172.21.15.1:0/945550369 learned_addr learned my addr 172.21.15.1:0/945550369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a137fe640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 0x7f9a14160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 0x7f9a140b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 -- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 msgr2=0x7f9a140b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 0x7f9a140b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 -- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 msgr2=0x7f9a14160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 0x7f9a14160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a13fff640 1 -- 172.21.15.1:0/945550369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a140b91b0 con 0x7f9a1415e190 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a137fe640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 0x7f9a14160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.434+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 0x7f9a140b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f9a13fff640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c02ace0 tx=0x7f9a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f99fbfff640 1 -- 172.21.15.1:0/945550369 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c036050 con 0x7f9a1415e190 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f99fbfff640 1 -- 172.21.15.1:0/945550369 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c042070 con 0x7f9a1415e190 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a14148de0 con 0x7f9a1415e190 2024-05-09T06:33:42.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f99fbfff640 1 -- 172.21.15.1:0/945550369 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c009510 con 0x7f9a1415e190 2024-05-09T06:33:42.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.435+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a140b9750 con 0x7f9a1415e190 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.436+0000 7f99fbfff640 1 -- 172.21.15.1:0/945550369 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a0c007bd0 con 0x7f9a1415e190 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.437+0000 7f99fbfff640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 0x7f9a0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.437+0000 7f99fbfff640 1 -- 172.21.15.1:0/945550369 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a0c0abfe0 con 0x7f9a1415e190 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.437+0000 7f9a137fe640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 0x7f9a0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.438+0000 7f9a137fe640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 0x7f9a0006e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9a141f7390 tx=0x7f9a04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.438+0000 7f9a1cc71740 1 Processor -- start 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.438+0000 7f9a1cc71740 1 -- start start 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9b8bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bbb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ee9af4a30 con 0x560ee9bbbb60 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a92590 con 0x560ee9bbc2a0 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ee9af09f0 con 0x560ee9bd9180 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42926/0 (socket says 172.21.15.1:42926) 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 -- 172.21.15.1:0/4002406900 learned_addr learned my addr 172.21.15.1:0/4002406900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a137fe640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9b8bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bbb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 -- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 msgr2=0x560ee9b8bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9b8bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 -- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 msgr2=0x560ee9bbb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bbb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 -- 172.21.15.1:0/4002406900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee98d47c0 con 0x560ee9bbbb60 2024-05-09T06:33:42.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a137fe640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9b8bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.439+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0aa2d0 tx=0x7f9a0c079fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbf8c92ea503662 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f99f97fa640 1 -- 172.21.15.1:0/4002406900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c092020 con 0x560ee9bbbb60 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f99f97fa640 1 -- 172.21.15.1:0/4002406900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c037040 con 0x560ee9bbbb60 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f99f97fa640 1 -- 172.21.15.1:0/4002406900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c031600 con 0x560ee9bbbb60 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4002406900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 msgr2=0x560ee9b90970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0aa2d0 tx=0x7f9a0c079fb0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4002406900 shutdown_connections 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bbbb60 0x560ee9b90970 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bbb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4002406900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9b8bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4002406900 >> 172.21.15.1:0/4002406900 conn(0x560ee9b8dd80 msgr2=0x560ee9bd6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4002406900 shutdown_connections 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.440+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4002406900 wait complete. 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.441+0000 7f9a1cc71740 1 -- start start 2024-05-09T06:33:42.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.441+0000 7f9a1cc71740 1 Processor -- start 2024-05-09T06:33:42.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bcd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9ba9880 0x560ee9be9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ee9b8ff30 con 0x560ee9bd9180 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ee9a90060 con 0x560ee9bbc2a0 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ee9b8f320 con 0x560ee9ba9880 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a1cc71740 10 client.0 _ll_register_callbacks cb 0x560ee9ab4dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42940/0 (socket says 172.21.15.1:42940) 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 -- 172.21.15.1:0/4071724139 learned_addr learned my addr 172.21.15.1:0/4071724139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a13fff640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9ba9880 0x560ee9be9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 -- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9ba9880 msgr2=0x560ee9be9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a137fe640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bcd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9ba9880 0x560ee9be9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 -- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 msgr2=0x560ee9bcd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bcd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 -- 172.21.15.1:0/4071724139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee9934f50 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e380 tx=0x7f9a0800e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c5229457afbb6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.442+0000 7f99f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08012840 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a080129e0 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c037040 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9a08012be0 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 10 client.7668 fetch_fsmap learned FSMap version 53 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x560ee97c80b0 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c037040 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f9a08025080 con 0x560ee9bd9180 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 10 client.7668 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 10 client.7668 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 20 client.7668 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bb9c30 0x560ee9bab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 4 client.7668 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee9b02160 con 0x560ee9bb9c30 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.443+0000 7f9a137fe640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bb9c30 0x560ee9bab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f99f27fc640 10 client.7668 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a137fe640 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bb9c30 0x560ee9bab860 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80461d4975432138 server_cookie=4dba2b50bbfc0d18 in_seq=0 out_seq=0 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f99f27fc640 1 -- 172.21.15.1:0/4071724139 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ee9b02160 con 0x560ee9bb9c30 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f99f27fc640 10 client.7668 handle_command_reply: tid=0 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 client.7668 shutdown 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 msgr2=0x560ee9ba8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e380 tx=0x7f9a0800e840 comp rx=0 tx=0).stop 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 shutdown_connections 2024-05-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bb9c30 0x560ee9bab860 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9ba9880 0x560ee9be9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9ba8e70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f99f27fc640 0 client.7668 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/4071724139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bcd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 >> 172.21.15.1:0/4071724139 conn(0x560ee9bbbb60 msgr2=0x560ee9bd6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 shutdown_connections 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.444+0000 7f9a1cc71740 1 -- 172.21.15.1:0/4071724139 wait complete. 2024-05-09T06:33:42.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.445+0000 7f9a1cc71740 20 client.7668 trim_cache size 0 max 16384 2024-05-09T06:33:42.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.458+0000 7f9a1cc71740 1 Processor -- start 2024-05-09T06:33:42.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.458+0000 7f9a1cc71740 1 -- start start 2024-05-09T06:33:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bd8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9bd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141ffd90 con 0x560ee9ba9880 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141f7e60 con 0x560ee9bbc2a0 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9a141ff740 con 0x560ee9bd9180 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42952/0 (socket says 172.21.15.1:42952) 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 -- 172.21.15.1:0/3816798740 learned_addr learned my addr 172.21.15.1:0/3816798740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 -- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 msgr2=0x560ee9bd8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9bd8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 -- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 msgr2=0x560ee9bd8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bd8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 -- 172.21.15.1:0/3816798740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee9ba8650 con 0x560ee9ba9880 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400cde0 tx=0x7f9a04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1be58926867734 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f99f8ff9640 1 -- 172.21.15.1:0/3816798740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04024150 con 0x560ee9ba9880 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f99f8ff9640 1 -- 172.21.15.1:0/3816798740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0402a070 con 0x560ee9ba9880 2024-05-09T06:33:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.459+0000 7f99f8ff9640 1 -- 172.21.15.1:0/3816798740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04024530 con 0x560ee9ba9880 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3816798740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 msgr2=0x560ee9babd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400cde0 tx=0x7f9a04012000 comp rx=0 tx=0).stop 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3816798740 shutdown_connections 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bd9180 0x560ee9bd8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9bbc2a0 0x560ee9bd8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3816798740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9ba9880 0x560ee9babd10 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3816798740 >> 172.21.15.1:0/3816798740 conn(0x560ee9bbbb60 msgr2=0x560ee9b86de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3816798740 shutdown_connections 2024-05-09T06:33:42.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.460+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3816798740 wait complete. 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 -- start start 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 Processor -- start 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9ba9880 0x560ee9bd8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bbc2a0 0x560ee9c4fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ee9bb9ab0 con 0x560ee9bd9180 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ee9baac10 con 0x560ee9ba9880 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.461+0000 7f9a1cc71740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ee9bba100 con 0x560ee9bbc2a0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a1cc71740 10 client.0 _ll_register_callbacks cb 0x7f9a0800c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42964/0 (socket says 172.21.15.1:42964) 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 -- 172.21.15.1:0/3170845040 learned_addr learned my addr 172.21.15.1:0/3170845040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a12ffd640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bbc2a0 0x560ee9c4fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9ba9880 0x560ee9bd8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 -- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bbc2a0 msgr2=0x560ee9c4fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bbc2a0 0x560ee9c4fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 -- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9ba9880 msgr2=0x560ee9bd8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9ba9880 0x560ee9bd8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 -- 172.21.15.1:0/3170845040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee9bea6b0 con 0x560ee9bd9180 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a13fff640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c007fd0 tx=0x7f9a0c0aa230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58ca12df148113 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f99f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c092040 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f9a0c07e170 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0402a070 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c031600 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a1cc71740 10 client.7674 fetch_fsmap learned FSMap version 53 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.462+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x560ee9c500b0 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0402a070 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f9a0c079530 con 0x560ee9bd9180 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 10 client.7674 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 10 client.7674 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 20 client.7674 populate_metadata read hostname 'smithi001' 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bcbfb0 0x560ee9bcc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 4 client.7674 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 --> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560ee9b8d1d0 con 0x560ee9bcbfb0 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bcbfb0 0x560ee9bcc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 10 client.7674 ms_handle_connect on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a137fe640 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bcbfb0 0x560ee9bcc390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3726ab74bb23956 server_cookie=1a9ed08bf6a0496d in_seq=0 out_seq=0 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 1 -- 172.21.15.1:0/3170845040 <== mds.0 v2:172.21.15.1:6834/3897834247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ee9b8d1d0 con 0x560ee9bcbfb0 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 10 client.7674 handle_command_reply: tid=0 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 client.7674 shutdown 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 msgr2=0x560ee9bd61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c007fd0 tx=0x7f9a0c0aa230 comp rx=0 tx=0).stop 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 shutdown_connections 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ee9bd9180 0x560ee9bd61e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.1:6834/3897834247,v1:172.21.15.1:6836/3897834247] conn(0x560ee9bcbfb0 0x560ee9bcc390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ee9bbc2a0 0x560ee9c4fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 --2- 172.21.15.1:0/3170845040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ee9ba9880 0x560ee9bd8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f99f27fc640 0 client.7674 ms_handle_reset on v2:172.21.15.1:6834/3897834247 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.463+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 >> 172.21.15.1:0/3170845040 conn(0x560ee9bbbb60 msgr2=0x560ee9ba8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 shutdown_connections 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a1cc71740 1 -- 172.21.15.1:0/3170845040 wait complete. 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a1cc71740 20 client.7674 trim_cache size 0 max 16384 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:42.452 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-05-09T06:33:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 msgr2=0x7f9a0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 0x7f9a0006e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9a141f7390 tx=0x7f9a04008040 comp rx=0 tx=0).stop 2024-05-09T06:33:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 msgr2=0x7f9a141488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c02ace0 tx=0x7f9a0c004a40 comp rx=0 tx=0).stop 2024-05-09T06:33:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 shutdown_connections 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9a0006be30 0x7f9a0006e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a1415e190 0x7f9a141488a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a140d45f0 0x7f9a140b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 --2- 172.21.15.1:0/945550369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a140d1c70 0x7f9a14160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 >> 172.21.15.1:0/945550369 conn(0x7f9a14073340 msgr2=0x7f9a140b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 shutdown_connections 2024-05-09T06:33:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.464+0000 7f9a19f6a640 1 -- 172.21.15.1:0/945550369 wait complete. 2024-05-09T06:33:42.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:33:42.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.577+0000 7fef53efc640 1 Processor -- start 2024-05-09T06:33:42.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 -- start start 2024-05-09T06:33:42.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef4c0d4940 0x7fef4c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4c0b33c0 con 0x7fef4c0d0f80 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c0d4dd0 con 0x7fef4c0d4940 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.578+0000 7fef53efc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0b3660 con 0x7fef4c161950 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51497640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef4c0d4940 0x7fef4c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:56866/0 (socket says 172.21.15.1:56866) 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef52499640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fef4c0d0f80 0x7fef4c0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:56080/0 (socket says 172.21.15.1:56080) 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 -- 172.21.15.1:0/702932280 learned_addr learned my addr 172.21.15.1:0/702932280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 -- 172.21.15.1:0/702932280 >> v2:172.21.15.6:3300/0 conn(0x7fef4c0d4940 msgr2=0x7fef4c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 --2- 172.21.15.1:0/702932280 >> v2:172.21.15.6:3300/0 conn(0x7fef4c0d4940 0x7fef4c0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 -- 172.21.15.1:0/702932280 >> v1:172.21.15.6:6789/0 conn(0x7fef4c0d0f80 legacy=0x7fef4c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.579+0000 7fef51c98640 1 -- 172.21.15.1:0/702932280 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c0b3940 con 0x7fef4c161950 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.580+0000 7fef51c98640 1 --2- 172.21.15.1:0/702932280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0b3590 tx=0x7fef4402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5fbaecd653677c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.580+0000 7fef50c96640 1 -- 172.21.15.1:0/702932280 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44037070 con 0x7fef4c161950 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.580+0000 7fef50c96640 1 -- 172.21.15.1:0/702932280 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef44032440 con 0x7fef4c161950 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.580+0000 7fef50c96640 1 -- 172.21.15.1:0/702932280 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44032810 con 0x7fef4c161950 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 -- 172.21.15.1:0/702932280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 msgr2=0x7fef4c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 --2- 172.21.15.1:0/702932280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0b3590 tx=0x7fef4402a250 comp rx=0 tx=0).stop 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 -- 172.21.15.1:0/702932280 shutdown_connections 2024-05-09T06:33:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 --2- 172.21.15.1:0/702932280 >> v2:172.21.15.6:3300/0 conn(0x7fef4c0d4940 0x7fef4c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 --2- 172.21.15.1:0/702932280 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c161950 0x7fef4c0d2400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 -- 172.21.15.1:0/702932280 >> 172.21.15.1:0/702932280 conn(0x7fef4c0d3e20 msgr2=0x7fef4c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 -- 172.21.15.1:0/702932280 shutdown_connections 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.581+0000 7fef53efc640 1 -- 172.21.15.1:0/702932280 wait complete. 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.582+0000 7fef53efc640 1 Processor -- start 2024-05-09T06:33:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.582+0000 7fef53efc640 1 -- start start 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c0d0f80 0x7fef4c0a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c1f7570 0x7fef4c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0b52d0 con 0x7fef4c161950 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0b5150 con 0x7fef4c0d0f80 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef53efc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0b5450 con 0x7fef4c1f7570 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42970/0 (socket says 172.21.15.1:42970) 2024-05-09T06:33:42.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 -- 172.21.15.1:0/2860909504 learned_addr learned my addr 172.21.15.1:0/2860909504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef52499640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c1f7570 0x7fef4c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51c98640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c0d0f80 0x7fef4c0a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 -- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c1f7570 msgr2=0x7fef4c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c1f7570 0x7fef4c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 -- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c0d0f80 msgr2=0x7fef4c0a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c0d0f80 0x7fef4c0a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 -- 172.21.15.1:0/2860909504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c15cbf0 con 0x7fef4c161950 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef51497640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e9e0 tx=0x7fef4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.583+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef4c161950 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.584+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4000c9d0 con 0x7fef4c161950 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.584+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000cdc0 con 0x7fef4c161950 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.584+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c0b80e0 con 0x7fef4c161950 2024-05-09T06:33:42.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.584+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1f4c10 con 0x7fef4c161950 2024-05-09T06:33:42.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.584+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c0752b0 con 0x7fef4c161950 2024-05-09T06:33:42.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.585+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef4001a070 con 0x7fef4c161950 2024-05-09T06:33:42.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.585+0000 7fef4a7fc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 0x7fef3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.586+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fef4008b0c0 con 0x7fef4c161950 2024-05-09T06:33:42.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.586+0000 7fef51c98640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 0x7fef3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.586+0000 7fef51c98640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 0x7fef3806e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef44034000 tx=0x7fef4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.590+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef40059800 con 0x7fef4c161950 2024-05-09T06:33:42.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.817+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fef4c0d21e0 con 0x7fef4c161950 2024-05-09T06:33:42.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.817+0000 7fef4a7fc640 1 -- 172.21.15.1:0/2860909504 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fef40059100 con 0x7fef4c161950 2024-05-09T06:33:42.799 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:42.799 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 msgr2=0x7fef3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 0x7fef3806e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef44034000 tx=0x7fef4402e000 comp rx=0 tx=0).stop 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 msgr2=0x7fef4c1f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e9e0 tx=0x7fef4000eea0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 shutdown_connections 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3806be30 0x7fef3806e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4c1f7570 0x7fef4c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4c161950 0x7fef4c1f7030 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 --2- 172.21.15.1:0/2860909504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4c0d0f80 0x7fef4c0a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 >> 172.21.15.1:0/2860909504 conn(0x7fef4c0d3e20 msgr2=0x7fef4c06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 shutdown_connections 2024-05-09T06:33:42.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.820+0000 7fef53efc640 1 -- 172.21.15.1:0/2860909504 wait complete. 2024-05-09T06:33:42.812 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:33:42.812 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:33:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.935+0000 7fb3aa5a9640 1 Processor -- start 2024-05-09T06:33:42.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.935+0000 7fb3aa5a9640 1 -- start start 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.935+0000 7fb3aa5a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.935+0000 7fb3aa5a9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3a40cf180 0x7fb3a40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.935+0000 7fb3aa5a9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 0x7fb3a415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3aa5a9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a40ce0e0 con 0x7fb3a40cf180 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3aa5a9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a40ce3c0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3aa5a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a40ce6a0 con 0x7fb3a406e6d0 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3a40cf180 0x7fb3a40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb3a40cf180 0x7fb3a40cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42974/0 (socket says 172.21.15.1:42974) 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 -- 172.21.15.1:0/283337541 learned_addr learned my addr 172.21.15.1:0/283337541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a8b46640 1 --2- 172.21.15.1:0/283337541 >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 0x7fb3a415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a3fff640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 -- 172.21.15.1:0/283337541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 msgr2=0x7fb3a406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a406eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 -- 172.21.15.1:0/283337541 >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 msgr2=0x7fb3a415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/283337541 >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 0x7fb3a415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a37fe640 1 -- 172.21.15.1:0/283337541 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a40ce980 con 0x7fb3a40cf180 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.936+0000 7fb3a8b46640 1 --2- 172.21.15.1:0/283337541 >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 0x7fb3a415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cf180 0x7fb3a40cd570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b16f0 tx=0x7fb39802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5365e5f24020ebbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3a2ffd640 1 -- 172.21.15.1:0/283337541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398037070 con 0x7fb3a40cf180 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3a2ffd640 1 -- 172.21.15.1:0/283337541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb398032440 con 0x7fb3a40cf180 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3a2ffd640 1 -- 172.21.15.1:0/283337541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398032810 con 0x7fb3a40cf180 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/283337541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cf180 msgr2=0x7fb3a40cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.937+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cf180 0x7fb3a40cd570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b16f0 tx=0x7fb39802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/283337541 shutdown_connections 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/283337541 >> v2:172.21.15.6:3300/0 conn(0x7fb3a40cdab0 0x7fb3a415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cf180 0x7fb3a40cd570 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/283337541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a406eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/283337541 >> 172.21.15.1:0/283337541 conn(0x7fb3a40cc4b0 msgr2=0x7fb3a40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/283337541 shutdown_connections 2024-05-09T06:33:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.938+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/283337541 wait complete. 2024-05-09T06:33:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.939+0000 7fb3aa5a9640 1 Processor -- start 2024-05-09T06:33:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.939+0000 7fb3aa5a9640 1 -- start start 2024-05-09T06:33:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3a40cf180 0x7fb3a4151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4070ed0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a406eff0 con 0x7fb3a40cf180 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3aa5a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a41447c0 con 0x7fb3a406e6d0 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42978/0 (socket says 172.21.15.1:42978) 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 -- 172.21.15.1:0/3105353258 learned_addr learned my addr 172.21.15.1:0/3105353258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a8b46640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3a40cf180 0x7fb3a4151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a3fff640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 -- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 msgr2=0x7fb3a415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 -- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3a40cf180 msgr2=0x7fb3a4151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3a40cf180 0x7fb3a4151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 -- 172.21.15.1:0/3105353258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a4151cb0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.940+0000 7fb3a37fe640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb39802a220 tx=0x7fb3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.941+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398041070 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.941+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39803c890 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.941+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a40a9b10 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.941+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398037040 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.941+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a40aa0d0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.942+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a40752b0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.942+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb398044070 con 0x7fb3a40cdab0 2024-05-09T06:33:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.943+0000 7fb3a0ff9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.943+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fb3980abcf0 con 0x7fb3a40cdab0 2024-05-09T06:33:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.943+0000 7fb3a3fff640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:42.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.943+0000 7fb3a3fff640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 0x7fb39006e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb394006fd0 tx=0x7fb394008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:42.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:42.947+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3a4151cb0 con 0x7fb3a40cdab0 2024-05-09T06:33:43.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.169+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb3a40b2290 con 0x7fb3a40cdab0 2024-05-09T06:33:43.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.169+0000 7fb3a0ff9640 1 -- 172.21.15.1:0/3105353258 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7fb398079ef0 con 0x7fb3a40cdab0 2024-05-09T06:33:43.151 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:33:43.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 msgr2=0x7fb39006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:43.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 0x7fb39006e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb394006fd0 tx=0x7fb394008040 comp rx=0 tx=0).stop 2024-05-09T06:33:43.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 msgr2=0x7fb3a40a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:43.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb39802a220 tx=0x7fb3980046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:43.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 shutdown_connections 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3a40cf180 0x7fb3a4151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3a40cdab0 0x7fb3a40a9480 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 --2- 172.21.15.1:0/3105353258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3a406e6d0 0x7fb3a415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 >> 172.21.15.1:0/3105353258 conn(0x7fb3a40cc4b0 msgr2=0x7fb3a4140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 shutdown_connections 2024-05-09T06:33:43.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43.172+0000 7fb3aa5a9640 1 -- 172.21.15.1:0/3105353258 wait complete. 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:33:43.164 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:33:43.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.165 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:33:43.181 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:33:43.181 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:33:43.181 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:33:43.182 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42319sec preferred_lft 42319sec 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:33:43.183 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.184 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:33:43.246 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:33:43.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.248 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:33:43.304 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:33:43.305 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:33:43.305 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:33:43.305 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:33:43.305 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:33:43.305 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:33:43.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:33:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:33:43.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.436 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:33:43.452 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:33:43.452 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:33:43.452 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:33:43.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:33:43.519 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:33:43.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:33:43.519 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:43.575 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:33:43.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:33:43.575 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:43.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:33:43.698 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:33:43.755 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:33:43.755 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:33:43.756 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:33:43.757 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:33:43.758 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:33:43.758 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:33:43.758 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:33:43.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.758 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:33:43.812 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:33:43.812 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:33:43.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:33:43.880 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:33:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:33:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:33:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:33:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:33:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:33:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:33:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:33:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:33:43.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:43.941 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:33:43.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:33:43.958+0000 7f4312c0c480 -1 init, newargv = 0x5618fb451230 newargc=15 2024-05-09T06:33:43.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[65508]: starting ceph client 2024-05-09T06:33:43.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[65508]: starting fuse 2024-05-09T06:33:43.960 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:33:43.960 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:33:43.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:33:43.960 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:33:43.961 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:33:43.962 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:33:44.081 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.65508.asok 2024-05-09T06:33:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:33:44.093 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65508.asok 2024-05-09T06:33:44.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:44.094 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65508.asok status 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "65508", 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:33:44.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7689, 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7689 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2138437487 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:44.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2138437487 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.7689 192.168.0.1:0/2138437487", 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/2138437487", 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 53, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 79, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:33:44.235 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:44.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:44.242 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:33:44.242 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:44.258 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:33:44.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:44.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:44.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:44.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:44.325 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-09T06:33:44.383 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:33:44.383 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-09T06:33:44.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.488+0000 7f132aaf6f40 1 Processor -- start 2024-05-09T06:33:44.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.488+0000 7f132aaf6f40 1 -- start start 2024-05-09T06:33:44.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 0x5607356465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 0x5607356eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607356f0890 con 0x56073565faa0 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5607356f05f0 con 0x5607356d5b60 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5607356f0b30 con 0x560735617f50 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f1327df8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f1327df8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43002/0 (socket says 172.21.15.1:43002) 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.489+0000 7f1327df8640 1 -- 172.21.15.1:0/4157019110 learned_addr learned my addr 172.21.15.1:0/4157019110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f13285f9640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 0x5607356465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 -- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 msgr2=0x5607356465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 0x5607356465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 -- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 msgr2=0x5607356eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:44.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1328dfa640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 0x5607356eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 0x5607356eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 -- 172.21.15.1:0/4157019110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560735558df0 con 0x56073565faa0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f13285f9640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 0x5607356465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1328dfa640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 0x5607356eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1327df8640 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f131c0095d0 tx=0x7f131c009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268f41c97413cb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1326df6640 1 -- 172.21.15.1:0/4157019110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c017070 con 0x56073565faa0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1326df6640 1 -- 172.21.15.1:0/4157019110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131c006620 con 0x56073565faa0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f1326df6640 1 -- 172.21.15.1:0/4157019110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0069d0 con 0x56073565faa0 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 -- 172.21.15.1:0/4157019110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 msgr2=0x5607356ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f131c0095d0 tx=0x7f131c009a90 comp rx=0 tx=0).stop 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 -- 172.21.15.1:0/4157019110 shutdown_connections 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5607356d5b60 0x5607356eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56073565faa0 0x5607356ee260 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/4157019110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560735617f50 0x5607356465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.490+0000 7f132aaf6f40 1 -- 172.21.15.1:0/4157019110 >> 172.21.15.1:0/4157019110 conn(0x560735619590 msgr2=0x5607356ecb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.491+0000 7f132aaf6f40 1 -- 172.21.15.1:0/4157019110 shutdown_connections 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.491+0000 7f132aaf6f40 1 -- 172.21.15.1:0/4157019110 wait complete. 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.491+0000 7f132aaf6f40 1 Processor -- start 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.491+0000 7f132aaf6f40 1 -- start start 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560735617f50 0x5607356d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56073565faa0 0x5607356cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 0x56073565d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607356d21d0 con 0x5607356d5b60 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560735644260 con 0x560735617f50 2024-05-09T06:33:44.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f132aaf6f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560735642790 con 0x56073565faa0 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 0x56073565d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 0x56073565d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43006/0 (socket says 172.21.15.1:43006) 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 -- 172.21.15.1:0/3234094664 learned_addr learned my addr 172.21.15.1:0/3234094664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1327df8640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56073565faa0 0x5607356cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f13285f9640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560735617f50 0x5607356d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 -- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56073565faa0 msgr2=0x5607356cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56073565faa0 0x5607356cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 -- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560735617f50 msgr2=0x5607356d5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560735617f50 0x5607356d5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 -- 172.21.15.1:0/3234094664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560735544cb0 con 0x5607356d5b60 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1327df8640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56073565faa0 0x5607356cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f13285f9640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560735617f50 0x5607356d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.492+0000 7f1328dfa640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 0x56073565d8f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f13200091d0 tx=0x7f1320009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.493+0000 7f1325df4640 1 -- 172.21.15.1:0/3234094664 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320014020 con 0x5607356d5b60 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.493+0000 7f1325df4640 1 -- 172.21.15.1:0/3234094664 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320010840 con 0x5607356d5b60 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.493+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56073565ff20 con 0x5607356d5b60 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.493+0000 7f1325df4640 1 -- 172.21.15.1:0/3234094664 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320010c70 con 0x5607356d5b60 2024-05-09T06:33:44.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.493+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607356d6d00 con 0x5607356d5b60 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.494+0000 7f1325df4640 1 -- 172.21.15.1:0/3234094664 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1320017020 con 0x5607356d5b60 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f1325df4640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f131406a810 0x7f131406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f1325df4640 1 -- 172.21.15.1:0/3234094664 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f1320089c30 con 0x5607356d5b60 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f13285f9640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f131406a810 0x7f131406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56073565de30 0x5607356d8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) -- 0x5607356d88d0 con 0x56073565de30 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f1327df8640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56073565de30 0x5607356d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f13285f9640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f131406a810 0x7f131406ccd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1318057000 tx=0x7f1318052000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.495+0000 7f1327df8640 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56073565de30 0x5607356d8390 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f1327df8640 1 -- 172.21.15.1:0/3234094664 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f131c01a040 con 0x56073565de30 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56073565de30 msgr2=0x5607356d8390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56073565de30 0x5607356d8390 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f131406a810 msgr2=0x7f131406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f131406a810 0x7f131406ccd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1318057000 tx=0x7f1318052000 comp rx=0 tx=0).stop 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 msgr2=0x56073565d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 --2- 172.21.15.1:0/3234094664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607356d5b60 0x56073565d8f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f13200091d0 tx=0x7f1320009690 comp rx=0 tx=0).stop 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f1328dfa640 1 -- 172.21.15.1:0/3234094664 reap_dead start 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 shutdown_connections 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 >> 172.21.15.1:0/3234094664 conn(0x560735619590 msgr2=0x560735615b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 shutdown_connections 2024-05-09T06:33:44.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.497+0000 7f132aaf6f40 1 -- 172.21.15.1:0/3234094664 wait complete. 2024-05-09T06:33:44.484 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:33:44.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:33:44.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:33:44.564 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:33:44.564 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:33:44.564 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:33:44.564 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:44.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:44.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.745+0000 7f4aeafee640 1 Processor -- start 2024-05-09T06:33:44.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.745+0000 7f4aeafee640 1 -- start start 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.745+0000 7f4aeafee640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4ae4161950 0x7f4ae40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.745+0000 7f4aeafee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 0x7f4ae40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4aeafee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae40b33c0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4aeafee640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae40d4dd0 con 0x7f4ae4161950 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4aeafee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae40b3660 con 0x7f4ae40d4940 2024-05-09T06:33:44.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae958b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4ae40d0f80 0x7f4ae40cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39054/0 (socket says 172.21.15.1:39054) 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae958b640 1 -- 172.21.15.1:0/1662929995 learned_addr learned my addr 172.21.15.1:0/1662929995 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/1662929995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 0x7f4ae40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae8d8a640 1 --2- 172.21.15.1:0/1662929995 >> v2:172.21.15.6:3300/0 conn(0x7f4ae4161950 0x7f4ae40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793573052 0 0) 0x7f4ae40b33c0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ad0003520 con 0x7f4ae40d0f80 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae3fff640 1 -- 172.21.15.1:0/1662929995 >> v2:172.21.15.6:3300/0 conn(0x7f4ae4161950 msgr2=0x7f4ae40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/1662929995 >> v2:172.21.15.6:3300/0 conn(0x7f4ae4161950 0x7f4ae40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae3fff640 1 -- 172.21.15.1:0/1662929995 >> v1:172.21.15.1:6789/0 conn(0x7f4ae40d0f80 legacy=0x7f4ae40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2765610040 0 0) 0x7f4ad0003520 con 0x7f4ae40d0f80 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.746+0000 7f4ae3fff640 1 -- 172.21.15.1:0/1662929995 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae40b3940 con 0x7f4ae40d4940 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.747+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/1662929995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 0x7f4ae40d2940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40b3590 tx=0x7f4ad402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69364450aea6a4ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.747+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc001670 con 0x7f4ae40d4940 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.747+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad4007540 con 0x7f4ae40d4940 2024-05-09T06:33:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.747+0000 7f4ae37fe640 1 -- 172.21.15.1:0/1662929995 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007910 con 0x7f4ae40d4940 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 -- 172.21.15.1:0/1662929995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 msgr2=0x7f4ae40d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 --2- 172.21.15.1:0/1662929995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 0x7f4ae40d2940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40b3590 tx=0x7f4ad402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 -- 172.21.15.1:0/1662929995 shutdown_connections 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 --2- 172.21.15.1:0/1662929995 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40d4940 0x7f4ae40d2940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 --2- 172.21.15.1:0/1662929995 >> v2:172.21.15.6:3300/0 conn(0x7f4ae4161950 0x7f4ae40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 -- 172.21.15.1:0/1662929995 >> 172.21.15.1:0/1662929995 conn(0x7f4ae40d3e20 msgr2=0x7f4ae40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 -- 172.21.15.1:0/1662929995 shutdown_connections 2024-05-09T06:33:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.748+0000 7f4aeafee640 1 -- 172.21.15.1:0/1662929995 wait complete. 2024-05-09T06:33:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.749+0000 7f4aeafee640 1 Processor -- start 2024-05-09T06:33:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.749+0000 7f4aeafee640 1 -- start start 2024-05-09T06:33:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 0x7f4ae40b9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40ca8b0 0x7f4ae41f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae40b52d0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae40b5150 con 0x7f4ae4161950 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4aeafee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae40b5450 con 0x7f4ae40ca8b0 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43018/0 (socket says 172.21.15.1:43018) 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 -- 172.21.15.1:0/222708610 learned_addr learned my addr 172.21.15.1:0/222708610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 -- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40ca8b0 msgr2=0x7f4ae41f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40ca8b0 0x7f4ae41f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 -- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 msgr2=0x7f4ae40b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 0x7f4ae40b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 0x7f4ae40b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 -- 172.21.15.1:0/222708610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae409def0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 0x7f4ae40b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.750+0000 7f4ae8d8a640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e870 tx=0x7f4acc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.751+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc017070 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.751+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc00c8e0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.751+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae40b7fc0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.751+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00ccd0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.751+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae409e1d0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.752+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae40752b0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.752+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4acc01a070 con 0x7f4ae40d0f80 2024-05-09T06:33:44.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.753+0000 7f4ae17fa640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 0x7f4ac406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.753+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f4acc08b200 con 0x7f4ae40d0f80 2024-05-09T06:33:44.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.753+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 0x7f4ac406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.753+0000 7f4ae3fff640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 0x7f4ac406e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4034000 tx=0x7f4ad402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.757+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4acc0599e0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.941+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ae40d21e0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.942+0000 7f4ae17fa640 1 -- 172.21.15.1:0/222708610 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7f4acc0592e0 con 0x7f4ae40d0f80 2024-05-09T06:33:44.923 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 msgr2=0x7f4ac406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 0x7f4ac406e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4034000 tx=0x7f4ad402f000 comp rx=0 tx=0).stop 2024-05-09T06:33:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 msgr2=0x7f4ae41f4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e870 tx=0x7f4acc00ed30 comp rx=0 tx=0).stop 2024-05-09T06:33:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 shutdown_connections 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ac406be60 0x7f4ac406e320 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ae40ca8b0 0x7f4ae41f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ae4161950 0x7f4ae40b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 --2- 172.21.15.1:0/222708610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae40d0f80 0x7f4ae41f4f40 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 >> 172.21.15.1:0/222708610 conn(0x7f4ae40d3e20 msgr2=0x7f4ae40cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 shutdown_connections 2024-05-09T06:33:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:44.944+0000 7f4aeafee640 1 -- 172.21.15.1:0/222708610 wait complete. 2024-05-09T06:33:44.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:33:44.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:33:45.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.063+0000 7f35bd142640 1 Processor -- start 2024-05-09T06:33:45.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.063+0000 7f35bd142640 1 -- start start 2024-05-09T06:33:45.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35bd142640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f35b8071fd0 0x7f35b806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35bd142640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35b815b420 con 0x7f35b8158a50 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35bd142640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35b815b6e0 con 0x7f35b815c4a0 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35bd142640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f35b8158e30 con 0x7f35b8071fd0 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b6d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f35b8158a50 0x7f35b80718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39058/0 (socket says 172.21.15.1:39058) 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b6d9d640 1 -- 172.21.15.1:0/3832287816 learned_addr learned my addr 172.21.15.1:0/3832287816 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b659c640 1 --2- 172.21.15.1:0/3832287816 >> v2:172.21.15.6:3300/0 conn(0x7f35b8071fd0 0x7f35b806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216581848 0 0) 0x7f35b815b420 con 0x7f35b8158a50 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35a0003520 con 0x7f35b8158a50 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3776219428 0 0) 0x7f35a0003520 con 0x7f35b8158a50 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 >> v2:172.21.15.6:3300/0 conn(0x7f35b8071fd0 msgr2=0x7f35b806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 --2- 172.21.15.1:0/3832287816 >> v2:172.21.15.6:3300/0 conn(0x7f35b8071fd0 0x7f35b806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 >> v1:172.21.15.6:6789/0 conn(0x7f35b815c4a0 legacy=0x7f35b815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b815b9c0 con 0x7f35b8158a50 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f35a8001670 con 0x7f35b8158a50 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 >> v1:172.21.15.1:6789/0 conn(0x7f35b8158a50 legacy=0x7f35b80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35a00039b0 0x7f35a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35a00051c0 0x7f35a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8158e30 con 0x7f35a0004800 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f35a000c060 con 0x7f35a00051c0 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f35a000c260 con 0x7f35a00039b0 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.064+0000 7f35b659c640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b6d9d640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35a00039b0 0x7f35a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b759e640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35a00051c0 0x7f35a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b659c640 1 -- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35a00039b0 msgr2=0x7f35a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b659c640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35a00039b0 0x7f35a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b659c640 1 -- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35a00051c0 msgr2=0x7f35a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b659c640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35a00051c0 0x7f35a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b759e640 1 -- 172.21.15.1:0/3832287816 reap_dead start 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.065+0000 7f35b659c640 1 -- 172.21.15.1:0/3832287816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a0003520 con 0x7f35a0004800 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35b659c640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35a0004c80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ac0 tx=0x7f35a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ede5411914a990e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac036070 con 0x7f35a0004800 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ac031400 con 0x7f35a0004800 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35b5d9b640 1 -- 172.21.15.1:0/3832287816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac009690 con 0x7f35a0004800 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 -- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 msgr2=0x7f35a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35a0004c80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ac0 tx=0x7f35a0005830 comp rx=0 tx=0).stop 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 -- 172.21.15.1:0/3832287816 shutdown_connections 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 --2- 172.21.15.1:0/3832287816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35a0004c80 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 -- 172.21.15.1:0/3832287816 >> 172.21.15.1:0/3832287816 conn(0x7f35b80cc4d0 msgr2=0x7f35b80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 -- 172.21.15.1:0/3832287816 shutdown_connections 2024-05-09T06:33:45.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.066+0000 7f35bd142640 1 -- 172.21.15.1:0/3832287816 wait complete. 2024-05-09T06:33:45.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.068+0000 7f35bd142640 1 Processor -- start 2024-05-09T06:33:45.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.068+0000 7f35bd142640 1 -- start start 2024-05-09T06:33:45.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35b81535c0 0x7f35b8158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35b8153a40 0x7f35b81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35b815bf80 con 0x7f35a0004800 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8072480 con 0x7f35b8153a40 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35bd142640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f35b8070ea0 con 0x7f35b81535c0 2024-05-09T06:33:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43028/0 (socket says 172.21.15.1:43028) 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 -- 172.21.15.1:0/803815384 learned_addr learned my addr 172.21.15.1:0/803815384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b759e640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35b8153a40 0x7f35b81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b659c640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35b81535c0 0x7f35b8158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 -- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35b81535c0 msgr2=0x7f35b8158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35b81535c0 0x7f35b8158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 -- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35b8153a40 msgr2=0x7f35b81594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35b8153a40 0x7f35b81594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 -- 172.21.15.1:0/803815384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b815d3c0 con 0x7f35a0004800 2024-05-09T06:33:45.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f35b6d9d640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f35a80072c0 tx=0x7f35a80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a800f9c0 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a8018070 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.069+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a800fdc0 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.070+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b815d6a0 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.070+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b815d050 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.071+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b80752b0 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.071+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35a801d460 con 0x7f35a0004800 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.071+0000 7f359f7fe640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 0x7f35a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.072+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f35a808b050 con 0x7f35a0004800 2024-05-09T06:33:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.072+0000 7f35b659c640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 0x7f35a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.072+0000 7f35b659c640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 0x7f35a406e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f35ac033000 tx=0x7f35ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.076+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35a80591d0 con 0x7f35a0004800 2024-05-09T06:33:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.263+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35b80cc250 con 0x7f35a0004800 2024-05-09T06:33:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.263+0000 7f359f7fe640 1 -- 172.21.15.1:0/803815384 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f35a805d4d0 con 0x7f35a0004800 2024-05-09T06:33:45.245 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 msgr2=0x7f35a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 0x7f35a406e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f35ac033000 tx=0x7f35ac008030 comp rx=0 tx=0).stop 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 msgr2=0x7f35b814cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f35a80072c0 tx=0x7f35a80076c0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 shutdown_connections 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f35a406be30 0x7f35a406e2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35b8153a40 0x7f35b81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35b81535c0 0x7f35b8158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 --2- 172.21.15.1:0/803815384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35a0004800 0x7f35b814cd40 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 >> 172.21.15.1:0/803815384 conn(0x7f35b80cc4d0 msgr2=0x7f35b8159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 shutdown_connections 2024-05-09T06:33:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.266+0000 7f35bd142640 1 -- 172.21.15.1:0/803815384 wait complete. 2024-05-09T06:33:45.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:33:45.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.385+0000 7f94a935c640 1 Processor -- start 2024-05-09T06:33:45.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.385+0000 7f94a935c640 1 -- start start 2024-05-09T06:33:45.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a935c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a415e190 0x7f94a415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a935c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94a40d45f0 0x7f94a40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a935c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a4161fe0 con 0x7f94a40d1c70 2024-05-09T06:33:45.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a935c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f94a415f580 con 0x7f94a40d45f0 2024-05-09T06:33:45.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a935c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94a41622c0 con 0x7f94a415e190 2024-05-09T06:33:45.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94a40d45f0 0x7f94a40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f949bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f94a40d1c70 0x7f94a40d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39070/0 (socket says 172.21.15.1:39070) 2024-05-09T06:33:45.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f949bfff640 1 -- 172.21.15.1:0/676992364 learned_addr learned my addr 172.21.15.1:0/676992364 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:45.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a2ffd640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a415e190 0x7f94a415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a37fe640 1 -- 172.21.15.1:0/676992364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a415e190 msgr2=0x7f94a415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a37fe640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a415e190 0x7f94a415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a37fe640 1 -- 172.21.15.1:0/676992364 >> v1:172.21.15.1:6789/0 conn(0x7f94a40d1c70 legacy=0x7f94a40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a37fe640 1 -- 172.21.15.1:0/676992364 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a41625a0 con 0x7f94a40d45f0 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.386+0000 7f94a27fc640 1 -- 172.21.15.1:0/676992364 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839238627 0 0) 0x7f94a4161fe0 con 0x7f94a40d1c70 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.387+0000 7f94a37fe640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40b3de0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f949c009d90 tx=0x7f949c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34e3bcf5d3c4430 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.387+0000 7f94a27fc640 1 -- 172.21.15.1:0/676992364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c007740 con 0x7f94a40d45f0 2024-05-09T06:33:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.387+0000 7f94a27fc640 1 -- 172.21.15.1:0/676992364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c0078e0 con 0x7f94a40d45f0 2024-05-09T06:33:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.387+0000 7f94a27fc640 1 -- 172.21.15.1:0/676992364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c007cb0 con 0x7f94a40d45f0 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 -- 172.21.15.1:0/676992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 msgr2=0x7f94a40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40b3de0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f949c009d90 tx=0x7f949c02a660 comp rx=0 tx=0).stop 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 -- 172.21.15.1:0/676992364 shutdown_connections 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40b3de0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 --2- 172.21.15.1:0/676992364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a415e190 0x7f94a415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 -- 172.21.15.1:0/676992364 >> 172.21.15.1:0/676992364 conn(0x7f94a4073340 msgr2=0x7f94a409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 -- 172.21.15.1:0/676992364 shutdown_connections 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.388+0000 7f94a935c640 1 -- 172.21.15.1:0/676992364 wait complete. 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.389+0000 7f94a935c640 1 Processor -- start 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.389+0000 7f94a935c640 1 -- start start 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a40d1c70 0x7f94a415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a415e190 0x7f94a4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40b56d0 con 0x7f94a40d45f0 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4161c60 con 0x7f94a415e190 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a935c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94a40b5850 con 0x7f94a40d1c70 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43038/0 (socket says 172.21.15.1:43038) 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 -- 172.21.15.1:0/1745016375 learned_addr learned my addr 172.21.15.1:0/1745016375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f94a2ffd640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a40d1c70 0x7f94a415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 -- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a40d1c70 msgr2=0x7f94a415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a40d1c70 0x7f94a415c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 -- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a415e190 msgr2=0x7f94a4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a415e190 0x7f94a4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 -- 172.21.15.1:0/1745016375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a40ca8a0 con 0x7f94a40d45f0 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.390+0000 7f949bfff640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c4b0 tx=0x7f948c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.391+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c018070 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.391+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948c007500 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.391+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a40b7f60 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.391+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004a20 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.391+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a40b9a50 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.392+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a40752b0 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.392+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f948c01b070 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.393+0000 7f949b7fe640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 0x7f948406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.393+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f948c08a3c0 con 0x7f94a40d45f0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.393+0000 7f94a2ffd640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 0x7f948406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.393+0000 7f94a2ffd640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 0x7f948406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f94a40b29b0 tx=0x7f9494008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.397+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f948c058c50 con 0x7f94a40d45f0 2024-05-09T06:33:45.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.579+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94a408e1c0 con 0x7f94a40d45f0 2024-05-09T06:33:45.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.579+0000 7f949b7fe640 1 -- 172.21.15.1:0/1745016375 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f948c058550 con 0x7f94a40d45f0 2024-05-09T06:33:45.561 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 msgr2=0x7f948406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 0x7f948406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f94a40b29b0 tx=0x7f9494008040 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 msgr2=0x7f94a40ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c4b0 tx=0x7f948c00c970 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 shutdown_connections 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f948406be30 0x7f948406e2f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94a415e190 0x7f94a4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a40d45f0 0x7f94a40ca1e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 --2- 172.21.15.1:0/1745016375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94a40d1c70 0x7f94a415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 >> 172.21.15.1:0/1745016375 conn(0x7f94a4073340 msgr2=0x7f94a406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 shutdown_connections 2024-05-09T06:33:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.581+0000 7f94a935c640 1 -- 172.21.15.1:0/1745016375 wait complete. 2024-05-09T06:33:45.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:33:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.699+0000 7fafcb06b640 1 Processor -- start 2024-05-09T06:33:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.700+0000 7fafcb06b640 1 -- start start 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.700+0000 7fafcb06b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fafc40d45f0 0x7fafc415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.700+0000 7fafcb06b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc4161fe0 con 0x7fafc40d1c70 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.700+0000 7fafcb06b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc41622c0 con 0x7fafc415e190 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.700+0000 7fafcb06b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fafc415f580 con 0x7fafc40d45f0 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fafc40d1c70 0x7fafc40b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39072/0 (socket says 172.21.15.1:39072) 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc9608640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fafc40d45f0 0x7fafc415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc3fff640 1 -- 172.21.15.1:0/820756447 learned_addr learned my addr 172.21.15.1:0/820756447 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc9608640 1 -- 172.21.15.1:0/820756447 >> v1:172.21.15.6:6789/0 conn(0x7fafc415e190 legacy=0x7fafc40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc9608640 1 -- 172.21.15.1:0/820756447 >> v1:172.21.15.1:6789/0 conn(0x7fafc40d1c70 legacy=0x7fafc40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.701+0000 7fafc9608640 1 -- 172.21.15.1:0/820756447 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc41625a0 con 0x7fafc40d45f0 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafc9608640 1 --2- 172.21.15.1:0/820756447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc40d45f0 0x7fafc415f780 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fafac0097a0 tx=0x7fafac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4464eabf6c9fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafc37fe640 1 -- 172.21.15.1:0/820756447 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac037070 con 0x7fafc40d45f0 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafc37fe640 1 -- 172.21.15.1:0/820756447 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac032440 con 0x7fafc40d45f0 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafc37fe640 1 -- 172.21.15.1:0/820756447 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac032810 con 0x7fafc40d45f0 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 -- 172.21.15.1:0/820756447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc40d45f0 msgr2=0x7fafc415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 --2- 172.21.15.1:0/820756447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc40d45f0 0x7fafc415f780 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fafac0097a0 tx=0x7fafac02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 -- 172.21.15.1:0/820756447 shutdown_connections 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 --2- 172.21.15.1:0/820756447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc40d45f0 0x7fafc415f780 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 -- 172.21.15.1:0/820756447 >> 172.21.15.1:0/820756447 conn(0x7fafc4073340 msgr2=0x7fafc409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 -- 172.21.15.1:0/820756447 shutdown_connections 2024-05-09T06:33:45.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.702+0000 7fafcb06b640 1 -- 172.21.15.1:0/820756447 wait complete. 2024-05-09T06:33:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.703+0000 7fafcb06b640 1 Processor -- start 2024-05-09T06:33:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 -- start start 2024-05-09T06:33:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc40d1c70 0x7fafc40cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafc40d45f0 0x7fafc40b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafc40b56d0 con 0x7fafc415e190 2024-05-09T06:33:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4161c60 con 0x7fafc40d1c70 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.704+0000 7fafcb06b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fafc40b5850 con 0x7fafc40d45f0 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43054/0 (socket says 172.21.15.1:43054) 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 -- 172.21.15.1:0/1312547 learned_addr learned my addr 172.21.15.1:0/1312547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc8e07640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc40d1c70 0x7fafc40cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc3fff640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafc40d45f0 0x7fafc40b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 -- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafc40d45f0 msgr2=0x7fafc40b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafc40d45f0 0x7fafc40b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 -- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc40d1c70 msgr2=0x7fafc40cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc40d1c70 0x7fafc40cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 -- 172.21.15.1:0/1312547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc40b98d0 con 0x7fafc415e190 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc9608640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fafac0097a0 tx=0x7fafac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac041070 con 0x7fafc415e190 2024-05-09T06:33:45.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac03c870 con 0x7fafc415e190 2024-05-09T06:33:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc40b7dd0 con 0x7fafc415e190 2024-05-09T06:33:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac037040 con 0x7fafc415e190 2024-05-09T06:33:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.705+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc40b8670 con 0x7fafc415e190 2024-05-09T06:33:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.706+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc40752b0 con 0x7fafc415e190 2024-05-09T06:33:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.707+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafac044070 con 0x7fafc415e190 2024-05-09T06:33:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.707+0000 7fafc17fa640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 0x7fafa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.707+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fafac0abb30 con 0x7fafc415e190 2024-05-09T06:33:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.707+0000 7fafc8e07640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 0x7fafa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.708+0000 7fafc8e07640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 0x7fafa406e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:45.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.712+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fafc40b98d0 con 0x7fafc415e190 2024-05-09T06:33:45.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.893+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fafc40b3490 con 0x7fafc415e190 2024-05-09T06:33:45.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.893+0000 7fafc17fa640 1 -- 172.21.15.1:0/1312547 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fafac079e60 con 0x7fafc415e190 2024-05-09T06:33:45.875 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:33:45.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 msgr2=0x7fafa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 0x7fafa406e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc009060 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 msgr2=0x7fafc40b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fafac0097a0 tx=0x7fafac0046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 shutdown_connections 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafa406be30 0x7fafa406e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafc415e190 0x7fafc40b9390 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafc40d45f0 0x7fafc40b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 --2- 172.21.15.1:0/1312547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafc40d1c70 0x7fafc40cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 >> 172.21.15.1:0/1312547 conn(0x7fafc4073340 msgr2=0x7fafc415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 shutdown_connections 2024-05-09T06:33:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:45.895+0000 7fafcb06b640 1 -- 172.21.15.1:0/1312547 wait complete. 2024-05-09T06:33:45.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:33:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.010+0000 7f9684a38640 1 Processor -- start 2024-05-09T06:33:45.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.010+0000 7f9684a38640 1 -- start start 2024-05-09T06:33:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f9684a38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 0x7f96800cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f9684a38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968006c2f0 con 0x7f968015e1d0 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f9684a38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968006f500 con 0x7f968015e5b0 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f9684a38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96800cc3c0 con 0x7f968006ffa0 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f967e59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f968015e1d0 0x7f968006f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39082/0 (socket says 172.21.15.1:39082) 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f967e59c640 1 -- 172.21.15.1:0/2201033461 learned_addr learned my addr 172.21.15.1:0/2201033461 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.011+0000 7f967dd9b640 1 --2- 172.21.15.1:0/2201033461 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 0x7f96800cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192389828 0 0) 0x7f9670001110 con 0x7f968015e1d0 2024-05-09T06:33:46.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9668003520 con 0x7f968015e1d0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022132785 0 0) 0x7f968006f500 con 0x7f968015e5b0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9670001110 con 0x7f968015e5b0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967dd9b640 1 -- 172.21.15.1:0/2201033461 >> v1:172.21.15.6:6789/0 conn(0x7f968015e5b0 legacy=0x7f96800d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967dd9b640 1 -- 172.21.15.1:0/2201033461 >> v1:172.21.15.1:6789/0 conn(0x7f968015e1d0 legacy=0x7f968006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2526064991 0 0) 0x7f9668003520 con 0x7f968015e1d0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.012+0000 7f967dd9b640 1 -- 172.21.15.1:0/2201033461 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800cc610 con 0x7f968006ffa0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f967dd9b640 1 --2- 172.21.15.1:0/2201033461 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 0x7f96800cffb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96740097d0 tx=0x7f967402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b3e67e7cf5c545 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674037070 con 0x7f968006ffa0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967402ab90 con 0x7f968006ffa0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f967d59a640 1 -- 172.21.15.1:0/2201033461 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670001840 con 0x7f968006ffa0 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 -- 172.21.15.1:0/2201033461 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 msgr2=0x7f96800cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 --2- 172.21.15.1:0/2201033461 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 0x7f96800cffb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96740097d0 tx=0x7f967402a100 comp rx=0 tx=0).stop 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 -- 172.21.15.1:0/2201033461 shutdown_connections 2024-05-09T06:33:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 --2- 172.21.15.1:0/2201033461 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968006ffa0 0x7f96800cffb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 -- 172.21.15.1:0/2201033461 >> 172.21.15.1:0/2201033461 conn(0x7f96800d1d70 msgr2=0x7f96800d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 -- 172.21.15.1:0/2201033461 shutdown_connections 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.013+0000 7f9684a38640 1 -- 172.21.15.1:0/2201033461 wait complete. 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 Processor -- start 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 -- start start 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f968006ffa0 0x7f9680161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 0x7f96800c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f968006f1b0 con 0x7f968015e5b0 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96800ccc60 con 0x7f968006ffa0 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.015+0000 7f9684a38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96800ce100 con 0x7f968015e1d0 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43066/0 (socket says 172.21.15.1:43066) 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 -- 172.21.15.1:0/584356241 learned_addr learned my addr 172.21.15.1:0/584356241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967e59c640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f968006ffa0 0x7f9680161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967dd9b640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 0x7f96800c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 -- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 msgr2=0x7f96800c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 0x7f96800c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 -- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f968006ffa0 msgr2=0x7f9680161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f968006ffa0 0x7f9680161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 -- 172.21.15.1:0/584356241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800c9f30 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967dd9b640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 0x7f96800c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f967ed9d640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f967800c6e0 tx=0x7f967800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678003b70 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678003d10 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96800c7e70 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678005310 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.016+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96800c8430 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.017+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96800ca100 con 0x7f968015e5b0 2024-05-09T06:33:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.018+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96780054b0 con 0x7f968015e5b0 2024-05-09T06:33:46.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.018+0000 7f966effd640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 0x7f966006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.019+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f967808b030 con 0x7f968015e5b0 2024-05-09T06:33:46.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.019+0000 7f967e59c640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 0x7f966006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.019+0000 7f967e59c640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 0x7f966006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96700013b0 tx=0x7f9670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.023+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96780596e0 con 0x7f968015e5b0 2024-05-09T06:33:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.216+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96800752b0 con 0x7f968015e5b0 2024-05-09T06:33:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.216+0000 7f966effd640 1 -- 172.21.15.1:0/584356241 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) ==== 80+0+120 (secure 0 0 0) 0x7f9678058fe0 con 0x7f968015e5b0 2024-05-09T06:33:46.198 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6834/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:33:46.198 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6832/2073487547 2024-05-10T06:32:23.361954+0000 2024-05-09T06:33:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 msgr2=0x7f966006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 0x7f966006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96700013b0 tx=0x7f9670009040 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 msgr2=0x7f9680146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f967800c6e0 tx=0x7f967800cba0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 shutdown_connections 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f966006be30 0x7f966006e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f968015e5b0 0x7f9680146650 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f968015e1d0 0x7f96800c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 --2- 172.21.15.1:0/584356241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f968006ffa0 0x7f9680161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.218+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 >> 172.21.15.1:0/584356241 conn(0x7f96800d1d70 msgr2=0x7f968006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:46.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.219+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 shutdown_connections 2024-05-09T06:33:46.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.219+0000 7f9684a38640 1 -- 172.21.15.1:0/584356241 wait complete. 2024-05-09T06:33:46.210 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:33:46.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:46.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:33:46.235 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[65508]: fuse finished with error 0 and tester_r 0 2024-05-09T06:33:46.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:33:46.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:46.237 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:33:46.330 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:33:46.330 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:33:46.330 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:46.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:33:46.350 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:33:46.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:46.351 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:33:46.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:33:46.410 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:33:46.410 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:33:46.411 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:33:46.411 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:46.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:33:46.427 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:33:46.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:46.427 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:33:46.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:33:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:33:46.485 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:33:46.485 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:33:46.485 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:46.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:33:46.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:33:46.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:33:46.505 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:33:46.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:33:46.564 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:33:46.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:46.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.688+0000 7f8798c38640 1 Processor -- start 2024-05-09T06:33:46.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.689+0000 7f8798c38640 1 -- start start 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.689+0000 7f8798c38640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f87940d2b10 0x7f87940b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.689+0000 7f8798c38640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940b33c0 con 0x7f8794161950 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f8798c38640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940b3660 con 0x7f87940d4940 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f8798c38640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f87940d4dd0 con 0x7f87940d2b10 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8794161950 0x7f87940d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39094/0 (socket says 172.21.15.1:39094) 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879259c640 1 -- 172.21.15.1:0/1258217878 learned_addr learned my addr 172.21.15.1:0/1258217878 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932107767 0 0) 0x7f87940b33c0 con 0x7f8794161950 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8780003520 con 0x7f8794161950 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f8792d9d640 1 --2- 172.21.15.1:0/1258217878 >> v2:172.21.15.6:3300/0 conn(0x7f87940d2b10 0x7f87940b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1016357755 0 0) 0x7f8780003520 con 0x7f8794161950 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 >> v2:172.21.15.6:3300/0 conn(0x7f87940d2b10 msgr2=0x7f87940b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:46.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 --2- 172.21.15.1:0/1258217878 >> v2:172.21.15.6:3300/0 conn(0x7f87940d2b10 0x7f87940b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 >> v1:172.21.15.6:6789/0 conn(0x7f87940d4940 legacy=0x7f87940b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940b3940 con 0x7f8794161950 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f878c001670 con 0x7f8794161950 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 >> v1:172.21.15.1:6789/0 conn(0x7f8794161950 legacy=0x7f87940d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87800039b0 0x7f8780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 0x7f8780004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87800051c0 0x7f878000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f87940d4dd0 con 0x7f8780004800 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f878000c060 con 0x7f87800051c0 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f878000c260 con 0x7f87800039b0 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.690+0000 7f8791d9b640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 0x7f8780004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f879259c640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87800039b0 0x7f8780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8792d9d640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87800051c0 0x7f878000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 -- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87800039b0 msgr2=0x7f8780003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:46.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87800039b0 0x7f8780003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 -- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87800051c0 msgr2=0x7f878000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87800051c0 0x7f878000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 -- 172.21.15.1:0/1258217878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8780003520 con 0x7f8780004800 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8792d9d640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87800051c0 0x7f878000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8792d9d640 1 -- 172.21.15.1:0/1258217878 reap_dead start 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.691+0000 7f8791d9b640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 0x7f8780004c80 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f877c001720 tx=0x7f8780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3eeda51dcf30a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c001370 con 0x7f8780004800 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c0052d0 con 0x7f8780004800 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f879159a640 1 -- 172.21.15.1:0/1258217878 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c02ab90 con 0x7f8780004800 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 -- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 msgr2=0x7f8780004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 0x7f8780004c80 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f877c001720 tx=0x7f8780005830 comp rx=0 tx=0).stop 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 -- 172.21.15.1:0/1258217878 shutdown_connections 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 --2- 172.21.15.1:0/1258217878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8780004800 0x7f8780004c80 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 -- 172.21.15.1:0/1258217878 >> 172.21.15.1:0/1258217878 conn(0x7f87940d3e20 msgr2=0x7f87940d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:46.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 -- 172.21.15.1:0/1258217878 shutdown_connections 2024-05-09T06:33:46.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.692+0000 7f8798c38640 1 -- 172.21.15.1:0/1258217878 wait complete. 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 Processor -- start 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 -- start start 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 0x7f8794158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87940b8420 0x7f87940b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f87940d1100 con 0x7f8794158fa0 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f87940d0f80 con 0x7f87940b8420 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8798c38640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f87940d1c00 con 0x7f8780004800 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8791d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.694+0000 7f8791d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43078/0 (socket says 172.21.15.1:43078) 2024-05-09T06:33:46.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 -- 172.21.15.1:0/638432093 learned_addr learned my addr 172.21.15.1:0/638432093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f879259c640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 0x7f8794158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8792d9d640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87940b8420 0x7f87940b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 -- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 msgr2=0x7f8794158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 0x7f8794158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 -- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87940b8420 msgr2=0x7f87940b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87940b8420 0x7f87940b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 -- 172.21.15.1:0/638432093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940b7660 con 0x7f8794158fa0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f879259c640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 0x7f8794158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8791d9b640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f877c000c00 tx=0x7f877c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c001370 con 0x7f8794158fa0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c041070 con 0x7f8794158fa0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940b7940 con 0x7f8794158fa0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c007a50 con 0x7f8794158fa0 2024-05-09T06:33:46.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.695+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8794158c60 con 0x7f8794158fa0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.696+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940752b0 con 0x7f8794158fa0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.697+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f877c0052d0 con 0x7f8794158fa0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.697+0000 7f878affd640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.697+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f877c0ab250 con 0x7f8794158fa0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.697+0000 7f879259c640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.698+0000 7f879259c640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 0x7f877406e320 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f878c007d90 tx=0x7f878c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.702+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f877c007680 con 0x7f8794158fa0 2024-05-09T06:33:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.922+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87941600e0 con 0x7f8794158fa0 2024-05-09T06:33:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.922+0000 7f878affd640 1 -- 172.21.15.1:0/638432093 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5170 (secure 0 0 0) 0x7f877c079990 con 0x7f8794158fa0 2024-05-09T06:33:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:46.903 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:33:18.377403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":23,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:46.904 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 53 2024-05-09T06:33:46.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 msgr2=0x7f877406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 0x7f877406e320 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f878c007d90 tx=0x7f878c009040 comp rx=0 tx=0).stop 2024-05-09T06:33:46.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 msgr2=0x7f87940ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:46.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f877c000c00 tx=0x7f877c001120 comp rx=0 tx=0).stop 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 shutdown_connections 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87940b8420 0x7f87940b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8794158fa0 0x7f87940ccc90 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 --2- 172.21.15.1:0/638432093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8780004800 0x7f8794158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 >> 172.21.15.1:0/638432093 conn(0x7f87940d3e20 msgr2=0x7f87940b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 shutdown_connections 2024-05-09T06:33:46.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:46.924+0000 7f8798c38640 1 -- 172.21.15.1:0/638432093 wait complete. 2024-05-09T06:33:46.916 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:47.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.038+0000 7f9785d63640 1 Processor -- start 2024-05-09T06:33:47.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.038+0000 7f9785d63640 1 -- start start 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.039+0000 7f9785d63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97801454b0 0x7f97800b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.039+0000 7f9785d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.039+0000 7f9785d63640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97800b5230 con 0x7f97800b4da0 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.039+0000 7f9785d63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9780071c20 con 0x7f97801454b0 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.039+0000 7f9785d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f978006eb40 con 0x7f97800b2a60 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977f7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97801454b0 0x7f97800b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:57036/0 (socket says 172.21.15.1:57036) 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 -- 172.21.15.1:0/2992947435 learned_addr learned my addr 172.21.15.1:0/2992947435 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 -- 172.21.15.1:0/2992947435 >> v2:172.21.15.6:3300/0 conn(0x7f97801454b0 msgr2=0x7f97800b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 --2- 172.21.15.1:0/2992947435 >> v2:172.21.15.6:3300/0 conn(0x7f97801454b0 0x7f97800b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 -- 172.21.15.1:0/2992947435 >> v1:172.21.15.6:6789/0 conn(0x7f97800b4da0 legacy=0x7f978015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.040+0000 7f977effd640 1 -- 172.21.15.1:0/2992947435 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978006ee20 con 0x7f97800b2a60 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f977effd640 1 --2- 172.21.15.1:0/2992947435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9774009870 tx=0x7f977402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc9e68263522ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f977e7fc640 1 -- 172.21.15.1:0/2992947435 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774037070 con 0x7f97800b2a60 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f977e7fc640 1 -- 172.21.15.1:0/2992947435 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9774032440 con 0x7f97800b2a60 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f977e7fc640 1 -- 172.21.15.1:0/2992947435 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774032810 con 0x7f97800b2a60 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 -- 172.21.15.1:0/2992947435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 msgr2=0x7f97800b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 --2- 172.21.15.1:0/2992947435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9774009870 tx=0x7f977402a270 comp rx=0 tx=0).stop 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 -- 172.21.15.1:0/2992947435 shutdown_connections 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 --2- 172.21.15.1:0/2992947435 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800b4860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 --2- 172.21.15.1:0/2992947435 >> v2:172.21.15.6:3300/0 conn(0x7f97801454b0 0x7f97800b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 -- 172.21.15.1:0/2992947435 >> 172.21.15.1:0/2992947435 conn(0x7f97800cc4b0 msgr2=0x7f97800cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.041+0000 7f9785d63640 1 -- 172.21.15.1:0/2992947435 shutdown_connections 2024-05-09T06:33:47.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.042+0000 7f9785d63640 1 -- 172.21.15.1:0/2992947435 wait complete. 2024-05-09T06:33:47.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.043+0000 7f9785d63640 1 Processor -- start 2024-05-09T06:33:47.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.043+0000 7f9785d63640 1 -- start start 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97801454b0 0x7f97801e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9780145890 con 0x7f97800b4da0 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f978006dbc0 con 0x7f97801454b0 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f978006e0e0 con 0x7f97800b2a60 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43090/0 (socket says 172.21.15.1:43090) 2024-05-09T06:33:47.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 -- 172.21.15.1:0/3584488017 learned_addr learned my addr 172.21.15.1:0/3584488017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977f7fe640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977ffff640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97801454b0 0x7f97801e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 -- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 msgr2=0x7f97800cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 -- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97801454b0 msgr2=0x7f97801e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97801454b0 0x7f97801e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 -- 172.21.15.1:0/3584488017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97800cf580 con 0x7f97800b4da0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f977effd640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f977402a2b0 tx=0x7f97740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774037040 con 0x7f97800b4da0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9774042070 con 0x7f97800b4da0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978013f890 con 0x7f97800b4da0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.044+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774030880 con 0x7f97800b4da0 2024-05-09T06:33:47.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.045+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978013fe50 con 0x7f97800b4da0 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.046+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97800752b0 con 0x7f97800b4da0 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.046+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9774030a20 con 0x7f97800b4da0 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.047+0000 7f9767fff640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 0x7f976c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.047+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f97740ab760 con 0x7f97800b4da0 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.047+0000 7f977f7fe640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 0x7f976c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.047+0000 7f977f7fe640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 0x7f976c06e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9770009b40 tx=0x7f9770009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.051+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97740798e0 con 0x7f97800b4da0 2024-05-09T06:33:47.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.235+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97800b4b80 con 0x7f97800b4da0 2024-05-09T06:33:47.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.236+0000 7f9767fff640 1 -- 172.21.15.1:0/3584488017 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7f977407dbe0 con 0x7f97800b4da0 2024-05-09T06:33:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:47.217 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 msgr2=0x7f976c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 0x7f976c06e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9770009b40 tx=0x7f9770009190 comp rx=0 tx=0).stop 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 msgr2=0x7f978015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f977402a2b0 tx=0x7f97740046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 shutdown_connections 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f976c06be30 0x7f976c06e2f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97801454b0 0x7f97801e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97800b4da0 0x7f978015b000 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 --2- 172.21.15.1:0/3584488017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97800b2a60 0x7f97800cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 >> 172.21.15.1:0/3584488017 conn(0x7f97800cc4b0 msgr2=0x7f97800b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 shutdown_connections 2024-05-09T06:33:47.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.238+0000 7f9785d63640 1 -- 172.21.15.1:0/3584488017 wait complete. 2024-05-09T06:33:47.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:32:02.450232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:32:03.777916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:33:47.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:33:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.356+0000 7f61dc45d640 1 Processor -- start 2024-05-09T06:33:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.356+0000 7f61dc45d640 1 -- start start 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61dc45d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 0x7f61d415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61dc45d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d4072030 con 0x7f61d40d1640 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61dc45d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d4072310 con 0x7f61d406b8f0 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61dc45d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f61d4072cb0 con 0x7f61d4159b90 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61d99f8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f61d40d1640 0x7f61d406b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39104/0 (socket says 172.21.15.1:39104) 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.357+0000 7f61d99f8640 1 -- 172.21.15.1:0/1822138309 learned_addr learned my addr 172.21.15.1:0/1822138309 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61da1f9640 1 --2- 172.21.15.1:0/1822138309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 0x7f61d415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705413454 0 0) 0x7f61d4072030 con 0x7f61d40d1640 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c0003520 con 0x7f61d40d1640 2024-05-09T06:33:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619989105 0 0) 0x7f61d4072310 con 0x7f61d406b8f0 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61d4072030 con 0x7f61d406b8f0 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359124730 0 0) 0x7f61c0003520 con 0x7f61d40d1640 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61da1f9640 1 -- 172.21.15.1:0/1822138309 >> v1:172.21.15.6:6789/0 conn(0x7f61d406b8f0 legacy=0x7f61d406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61da1f9640 1 -- 172.21.15.1:0/1822138309 >> v1:172.21.15.1:6789/0 conn(0x7f61d40d1640 legacy=0x7f61d406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.358+0000 7f61da1f9640 1 -- 172.21.15.1:0/1822138309 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d40725f0 con 0x7f61d4159b90 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61da1f9640 1 --2- 172.21.15.1:0/1822138309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 0x7f61d415c3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a109e5a81d640cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037070 con 0x7f61d4159b90 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d002ab90 con 0x7f61d4159b90 2024-05-09T06:33:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61d91f7640 1 -- 172.21.15.1:0/1822138309 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4001670 con 0x7f61d4159b90 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61dc45d640 1 -- 172.21.15.1:0/1822138309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 msgr2=0x7f61d415c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.359+0000 7f61dc45d640 1 --2- 172.21.15.1:0/1822138309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 0x7f61d415c3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d002a100 comp rx=0 tx=0).stop 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.360+0000 7f61dc45d640 1 -- 172.21.15.1:0/1822138309 shutdown_connections 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.360+0000 7f61dc45d640 1 --2- 172.21.15.1:0/1822138309 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d4159b90 0x7f61d415c3c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.360+0000 7f61dc45d640 1 -- 172.21.15.1:0/1822138309 >> 172.21.15.1:0/1822138309 conn(0x7f61d40d0b50 msgr2=0x7f61d40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.360+0000 7f61dc45d640 1 -- 172.21.15.1:0/1822138309 shutdown_connections 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.360+0000 7f61dc45d640 1 -- 172.21.15.1:0/1822138309 wait complete. 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.361+0000 7f61dc45d640 1 Processor -- start 2024-05-09T06:33:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.361+0000 7f61dc45d640 1 -- start start 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.361+0000 7f61dc45d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.361+0000 7f61dc45d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d40d1640 0x7f61d40c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.361+0000 7f61dc45d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61d4159b90 0x7f61d415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61dc45d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61d415d580 con 0x7f61d406b8f0 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61dc45d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f61d406c150 con 0x7f61d4159b90 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61dc45d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f61d4071ce0 con 0x7f61d40d1640 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43102/0 (socket says 172.21.15.1:43102) 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 -- 172.21.15.1:0/3697490495 learned_addr learned my addr 172.21.15.1:0/3697490495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61d99f8640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d40d1640 0x7f61d40c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da9fa640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61d4159b90 0x7f61d415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 -- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d40d1640 msgr2=0x7f61d40c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d40d1640 0x7f61d40c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 -- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61d4159b90 msgr2=0x7f61d415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61d4159b90 0x7f61d415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 -- 172.21.15.1:0/3697490495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d41522f0 con 0x7f61d406b8f0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61da1f9640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0041070 con 0x7f61d406b8f0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d003c660 con 0x7f61d406b8f0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d4160560 con 0x7f61d406b8f0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037040 con 0x7f61d406b8f0 2024-05-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.362+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d415f5a0 con 0x7f61d406b8f0 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.363+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d40752b0 con 0x7f61d406b8f0 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.364+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61d0044070 con 0x7f61d406b8f0 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.364+0000 7f61caffd640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 0x7f61b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.364+0000 7f61d99f8640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 0x7f61b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.364+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f61d00ab7f0 con 0x7f61d406b8f0 2024-05-09T06:33:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.365+0000 7f61d99f8640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 0x7f61b806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f61d40cae00 tx=0x7f61c4000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.369+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61d0079a90 con 0x7f61d406b8f0 2024-05-09T06:33:47.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.551+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61d40ce360 con 0x7f61d406b8f0 2024-05-09T06:33:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.551+0000 7f61caffd640 1 -- 172.21.15.1:0/3697490495 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f61d007dd90 con 0x7f61d406b8f0 2024-05-09T06:33:47.533 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 msgr2=0x7f61b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 0x7f61b806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f61d40cae00 tx=0x7f61c4000ed0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 msgr2=0x7f61d40c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d00046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 shutdown_connections 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f61b806be30 0x7f61b806e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61d4159b90 0x7f61d415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f61d40d1640 0x7f61d40c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 --2- 172.21.15.1:0/3697490495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61d406b8f0 0x7f61d40c3b50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 >> 172.21.15.1:0/3697490495 conn(0x7f61d40d0b50 msgr2=0x7f61d40b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.553+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 shutdown_connections 2024-05-09T06:33:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.554+0000 7f61dc45d640 1 -- 172.21.15.1:0/3697490495 wait complete. 2024-05-09T06:33:47.546 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:33:47.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.672+0000 7fb92c2b3640 1 Processor -- start 2024-05-09T06:33:47.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.672+0000 7fb92c2b3640 1 -- start start 2024-05-09T06:33:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92c2b3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9240cf180 0x7fb9240d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92c2b3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9240b1340 con 0x7fb92415e270 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92c2b3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9240d1020 con 0x7fb9240ceba0 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92c2b3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb9240d0df0 con 0x7fb9240cf180 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a04f640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb92415e270 0x7fb9240ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:39110/0 (socket says 172.21.15.1:39110) 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a04f640 1 -- 172.21.15.1:0/1640065455 learned_addr learned my addr 172.21.15.1:0/1640065455 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a850640 1 --2- 172.21.15.1:0/1640065455 >> v2:172.21.15.6:3300/0 conn(0x7fb9240cf180 0x7fb9240d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92904d640 1 -- 172.21.15.1:0/1640065455 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290086663 0 0) 0x7fb920001110 con 0x7fb92415e270 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92904d640 1 -- 172.21.15.1:0/1640065455 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb910003520 con 0x7fb92415e270 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a850640 1 -- 172.21.15.1:0/1640065455 >> v1:172.21.15.6:6789/0 conn(0x7fb9240ceba0 legacy=0x7fb9240ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a850640 1 -- 172.21.15.1:0/1640065455 >> v1:172.21.15.1:6789/0 conn(0x7fb92415e270 legacy=0x7fb9240ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.673+0000 7fb92a850640 1 -- 172.21.15.1:0/1640065455 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9240cc4b0 con 0x7fb9240cf180 2024-05-09T06:33:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.674+0000 7fb92a850640 1 --2- 172.21.15.1:0/1640065455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb9240d08b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009d90 tx=0x7fb91c02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e92f4374988045 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.674+0000 7fb92904d640 1 -- 172.21.15.1:0/1640065455 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c037070 con 0x7fb9240cf180 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.674+0000 7fb92904d640 1 -- 172.21.15.1:0/1640065455 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c007740 con 0x7fb9240cf180 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.674+0000 7fb92904d640 1 -- 172.21.15.1:0/1640065455 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c007af0 con 0x7fb9240cf180 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 -- 172.21.15.1:0/1640065455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 msgr2=0x7fb9240d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/1640065455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb9240d08b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009d90 tx=0x7fb91c02a780 comp rx=0 tx=0).stop 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 -- 172.21.15.1:0/1640065455 shutdown_connections 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/1640065455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb9240d08b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 -- 172.21.15.1:0/1640065455 >> 172.21.15.1:0/1640065455 conn(0x7fb9240d1c10 msgr2=0x7fb9240d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 -- 172.21.15.1:0/1640065455 shutdown_connections 2024-05-09T06:33:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.675+0000 7fb92c2b3640 1 -- 172.21.15.1:0/1640065455 wait complete. 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.676+0000 7fb92c2b3640 1 Processor -- start 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.676+0000 7fb92c2b3640 1 -- start start 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9240ceba0 0x7fb9240c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb92406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240cf590 con 0x7fb92415e270 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240cc260 con 0x7fb9240cf180 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92c2b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240cca70 con 0x7fb9240ceba0 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43118/0 (socket says 172.21.15.1:43118) 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 -- 172.21.15.1:0/2174513685 learned_addr learned my addr 172.21.15.1:0/2174513685 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 -- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9240ceba0 msgr2=0x7fb9240c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9240ceba0 0x7fb9240c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 -- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 msgr2=0x7fb92406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb92406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.677+0000 7fb92a850640 1 -- 172.21.15.1:0/2174513685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb924156d80 con 0x7fb92415e270 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb92a850640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009d90 tx=0x7fb91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c042070 con 0x7fb92415e270 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c003990 con 0x7fb92415e270 2024-05-09T06:33:47.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb924157020 con 0x7fb92415e270 2024-05-09T06:33:47.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c037040 con 0x7fb92415e270 2024-05-09T06:33:47.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.678+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241575e0 con 0x7fb92415e270 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.679+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9240752b0 con 0x7fb92415e270 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.679+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb91c032870 con 0x7fb92415e270 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.680+0000 7fb91affd640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.680+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fb91c0abf20 con 0x7fb92415e270 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.680+0000 7fb92a04f640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.680+0000 7fb92a04f640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb920007fd0 tx=0x7fb920001150 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.684+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb91c04c020 con 0x7fb92415e270 2024-05-09T06:33:47.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.866+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9240d29e0 con 0x7fb92415e270 2024-05-09T06:33:47.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.866+0000 7fb91affd640 1 -- 172.21.15.1:0/2174513685 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fb91c07a780 con 0x7fb92415e270 2024-05-09T06:33:47.848 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 msgr2=0x7fb90806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb920007fd0 tx=0x7fb920001150 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 msgr2=0x7fb92406f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009d90 tx=0x7fb91c004a40 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 shutdown_connections 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb92415e270 0x7fb92406f430 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9240cf180 0x7fb92406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 --2- 172.21.15.1:0/2174513685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9240ceba0 0x7fb9240c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 >> 172.21.15.1:0/2174513685 conn(0x7fb9240d1c10 msgr2=0x7fb92415ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 shutdown_connections 2024-05-09T06:33:47.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.868+0000 7fb92c2b3640 1 -- 172.21.15.1:0/2174513685 wait complete. 2024-05-09T06:33:47.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:33:47.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.987+0000 7f3cadfdc640 1 Processor -- start 2024-05-09T06:33:47.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3cadfdc640 1 -- start start 2024-05-09T06:33:47.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3cadfdc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3cadfdc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca8161fe0 con 0x7f3ca815e190 2024-05-09T06:33:47.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3cadfdc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca81622c0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3cadfdc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca815f580 con 0x7f3ca80d45f0 2024-05-09T06:33:47.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3ca77fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3ca815e190 0x7f3ca80d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44438/0 (socket says 172.21.15.1:44438) 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.988+0000 7f3ca77fe640 1 -- 172.21.15.1:0/3416714565 learned_addr learned my addr 172.21.15.1:0/3416714565 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3416714565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222122694 0 0) 0x7f3ca8161fe0 con 0x7f3ca815e190 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c90003520 con 0x7f3ca815e190 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618801737 0 0) 0x7f3ca81622c0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8161fe0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3416714565 >> v1:172.21.15.6:6789/0 conn(0x7f3ca80d1c70 legacy=0x7f3ca80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3416714565 >> v1:172.21.15.1:6789/0 conn(0x7f3ca815e190 legacy=0x7f3ca80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.989+0000 7f3ca7fff640 1 -- 172.21.15.1:0/3416714565 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca81625a0 con 0x7f3ca80d45f0 2024-05-09T06:33:47.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3416714565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca815f780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009dc0 tx=0x7f3ca002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc064dd7d3ab7a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0007740 con 0x7f3ca80d45f0 2024-05-09T06:33:47.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00078e0 con 0x7f3ca80d45f0 2024-05-09T06:33:47.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3ca67fc640 1 -- 172.21.15.1:0/3416714565 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0007cb0 con 0x7f3ca80d45f0 2024-05-09T06:33:47.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3416714565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 msgr2=0x7f3ca815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3416714565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca815f780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009dc0 tx=0x7f3ca002a6d0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3416714565 shutdown_connections 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3416714565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca815f780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.990+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3416714565 >> 172.21.15.1:0/3416714565 conn(0x7f3ca8073340 msgr2=0x7f3ca809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.991+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3416714565 shutdown_connections 2024-05-09T06:33:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.991+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3416714565 wait complete. 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.992+0000 7f3cadfdc640 1 Processor -- start 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.992+0000 7f3cadfdc640 1 -- start start 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca81f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 0x7f3ca80c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80b56d0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8161c60 con 0x7f3ca815e190 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca80b5850 con 0x7f3ca80d45f0 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54408/0 (socket says 172.21.15.1:54408) 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 -- 172.21.15.1:0/3153519259 learned_addr learned my addr 172.21.15.1:0/3153519259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 0x7f3ca80c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca6ffd640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca81f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 -- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 msgr2=0x7f3ca81f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca81f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 -- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 msgr2=0x7f3ca80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 0x7f3ca80c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 -- 172.21.15.1:0/3153519259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca80cdb00 con 0x7f3ca80d1c70 2024-05-09T06:33:47.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 0x7f3ca80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3ca77fe640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800c440 tx=0x7f3c9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98017070 con 0x7f3ca80d1c70 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c98005ce0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca81f8970 con 0x7f3ca80d1c70 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98007270 con 0x7f3ca80d1c70 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.993+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81f4c80 con 0x7f3ca80d1c70 2024-05-09T06:33:47.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.994+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca80752b0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.995+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c9801a070 con 0x7f3ca80d1c70 2024-05-09T06:33:47.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.995+0000 7f3c8ffff640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 0x7f3c9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:47.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.995+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f3c9808a2f0 con 0x7f3ca80d1c70 2024-05-09T06:33:47.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.995+0000 7f3ca6ffd640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 0x7f3c9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:47.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:47.996+0000 7f3ca6ffd640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 0x7f3c9406e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c007fd0 tx=0x7f3c9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:47.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.000+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ca81f8970 con 0x7f3ca80d1c70 2024-05-09T06:33:48.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.183+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ca808e1c0 con 0x7f3ca80d1c70 2024-05-09T06:33:48.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.184+0000 7f3c8ffff640 1 -- 172.21.15.1:0/3153519259 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f3c980583f0 con 0x7f3ca80d1c70 2024-05-09T06:33:48.165 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:33:48.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 msgr2=0x7f3c9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 0x7f3c9406e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c007fd0 tx=0x7f3c9c009040 comp rx=0 tx=0).stop 2024-05-09T06:33:48.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 msgr2=0x7f3ca81f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800c440 tx=0x7f3c9800c900 comp rx=0 tx=0).stop 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 shutdown_connections 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c9406be30 0x7f3c9406e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca815e190 0x7f3ca80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80d45f0 0x7f3ca81f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 --2- 172.21.15.1:0/3153519259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80d1c70 0x7f3ca81f8430 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 >> 172.21.15.1:0/3153519259 conn(0x7f3ca8073340 msgr2=0x7f3ca806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 shutdown_connections 2024-05-09T06:33:48.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.186+0000 7f3cadfdc640 1 -- 172.21.15.1:0/3153519259 wait complete. 2024-05-09T06:33:48.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:48.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.300+0000 7f33b52b3640 1 Processor -- start 2024-05-09T06:33:48.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 -- start start 2024-05-09T06:33:48.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b0161950 0x7f33b00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33b00d0f80 0x7f33b00d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b00b33c0 con 0x7f33b00d4940 2024-05-09T06:33:48.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f33b00d4dd0 con 0x7f33b00d0f80 2024-05-09T06:33:48.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33b52b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00b3660 con 0x7f33b0161950 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33af7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33b00d0f80 0x7f33b00d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33af7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33b00d0f80 0x7f33b00d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54422/0 (socket says 172.21.15.1:54422) 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33ae7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f33b00d4940 0x7f33b00cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44454/0 (socket says 172.21.15.1:44454) 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.301+0000 7f33af7fe640 1 -- 172.21.15.1:0/2724903468 learned_addr learned my addr 172.21.15.1:0/2724903468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33aeffd640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b0161950 0x7f33b00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33af7fe640 1 -- 172.21.15.1:0/2724903468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b0161950 msgr2=0x7f33b00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33af7fe640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b0161950 0x7f33b00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33af7fe640 1 -- 172.21.15.1:0/2724903468 >> v1:172.21.15.1:6789/0 conn(0x7f33b00d4940 legacy=0x7f33b00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33af7fe640 1 -- 172.21.15.1:0/2724903468 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00b3940 con 0x7f33b00d0f80 2024-05-09T06:33:48.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.302+0000 7f33af7fe640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b00d0f80 0x7f33b00d2940 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f33b00b3590 tx=0x7f33a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6137d70bad30b7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33adffb640 1 -- 172.21.15.1:0/2724903468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0037070 con 0x7f33b00d0f80 2024-05-09T06:33:48.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33adffb640 1 -- 172.21.15.1:0/2724903468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0032440 con 0x7f33b00d0f80 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33adffb640 1 -- 172.21.15.1:0/2724903468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0032810 con 0x7f33b00d0f80 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 -- 172.21.15.1:0/2724903468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b00d0f80 msgr2=0x7f33b00d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b00d0f80 0x7f33b00d2940 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f33b00b3590 tx=0x7f33a002a250 comp rx=0 tx=0).stop 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 -- 172.21.15.1:0/2724903468 shutdown_connections 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b00d0f80 0x7f33b00d2940 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 --2- 172.21.15.1:0/2724903468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b0161950 0x7f33b00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 -- 172.21.15.1:0/2724903468 >> 172.21.15.1:0/2724903468 conn(0x7f33b00d3e20 msgr2=0x7f33b00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 -- 172.21.15.1:0/2724903468 shutdown_connections 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.303+0000 7f33b52b3640 1 -- 172.21.15.1:0/2724903468 wait complete. 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 Processor -- start 2024-05-09T06:33:48.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 -- start start 2024-05-09T06:33:48.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33b00d0f80 0x7f33b015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b015d4c0 0x7f33b00caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00b52d0 con 0x7f33b0161950 2024-05-09T06:33:48.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00b5150 con 0x7f33b00d0f80 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33b52b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00b5450 con 0x7f33b015d4c0 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33ae7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33ae7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54438/0 (socket says 172.21.15.1:54438) 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.305+0000 7f33ae7fc640 1 -- 172.21.15.1:0/3434397924 learned_addr learned my addr 172.21.15.1:0/3434397924 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33af7fe640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b015d4c0 0x7f33b00caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33aeffd640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33b00d0f80 0x7f33b015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 -- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b015d4c0 msgr2=0x7f33b00caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b015d4c0 0x7f33b00caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 -- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33b00d0f80 msgr2=0x7f33b015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33b00d0f80 0x7f33b015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 -- 172.21.15.1:0/3434397924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00b9720 con 0x7f33b0161950 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33ae7fc640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f339800c400 tx=0x7f339800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33980074e0 con 0x7f33b0161950 2024-05-09T06:33:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3398007680 con 0x7f33b0161950 2024-05-09T06:33:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004a00 con 0x7f33b0161950 2024-05-09T06:33:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b00cbad0 con 0x7f33b0161950 2024-05-09T06:33:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.306+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b00cc090 con 0x7f33b0161950 2024-05-09T06:33:48.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.307+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b015d8d0 con 0x7f33b0161950 2024-05-09T06:33:48.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.308+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f339801a020 con 0x7f33b0161950 2024-05-09T06:33:48.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.308+0000 7f33a77fe640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 0x7f339006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.308+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f339808a6f0 con 0x7f33b0161950 2024-05-09T06:33:48.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.308+0000 7f33aeffd640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 0x7f339006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.309+0000 7f33aeffd640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 0x7f339006e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33a8003aa0 tx=0x7f33a8009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.312+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33b015d8d0 con 0x7f33b0161950 2024-05-09T06:33:48.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:33:48.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:33:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.536+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b00752b0 con 0x7f33b0161950 2024-05-09T06:33:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.536+0000 7f33a77fe640 1 -- 172.21.15.1:0/3434397924 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5170 (secure 0 0 0) 0x7f33b00b9720 con 0x7f33b0161950 2024-05-09T06:33:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:48.518 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:33:18.377403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":23,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:48.518 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 53 2024-05-09T06:33:48.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 msgr2=0x7f339006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 0x7f339006e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33a8003aa0 tx=0x7f33a8009830 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 msgr2=0x7f33b015cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f339800c400 tx=0x7f339800c8c0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 shutdown_connections 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f339006be60 0x7f339006e320 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33b015d4c0 0x7f33b00caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33b0161950 0x7f33b015cf80 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 --2- 172.21.15.1:0/3434397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33b00d0f80 0x7f33b015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 >> 172.21.15.1:0/3434397924 conn(0x7f33b00d3e20 msgr2=0x7f33b015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 shutdown_connections 2024-05-09T06:33:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.538+0000 7f33b52b3640 1 -- 172.21.15.1:0/3434397924 wait complete. 2024-05-09T06:33:48.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:48.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.693+0000 7f6dade93640 1 Processor -- start 2024-05-09T06:33:48.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6dade93640 1 -- start start 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6dade93640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6da806e4f0 0x7f6da806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6dade93640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da815eb40 con 0x7f6da815e050 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6dade93640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da80aef90 con 0x7f6da80ae2a0 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6dade93640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6da80b3b80 con 0x7f6da806e4f0 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.694+0000 7f6da77fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6da815e050 0x7f6da815e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44462/0 (socket says 172.21.15.1:44462) 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da77fe640 1 -- 172.21.15.1:0/3508904044 learned_addr learned my addr 172.21.15.1:0/3508904044 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da6ffd640 1 --2- 172.21.15.1:0/3508904044 >> v2:172.21.15.6:3300/0 conn(0x7f6da806e4f0 0x7f6da806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380686971 0 0) 0x7f6da815eb40 con 0x7f6da815e050 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d90003520 con 0x7f6da815e050 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1872037048 0 0) 0x7f6d90003520 con 0x7f6da815e050 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 >> v2:172.21.15.6:3300/0 conn(0x7f6da806e4f0 msgr2=0x7f6da806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 --2- 172.21.15.1:0/3508904044 >> v2:172.21.15.6:3300/0 conn(0x7f6da806e4f0 0x7f6da806e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 >> v1:172.21.15.6:6789/0 conn(0x7f6da80ae2a0 legacy=0x7f6da80ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da80b3530 con 0x7f6da815e050 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6d98001670 con 0x7f6da815e050 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 >> v1:172.21.15.1:6789/0 conn(0x7f6da815e050 legacy=0x7f6da815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 0x7f6d90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d90004800 0x7f6d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900051c0 0x7f6d9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6da80b3b80 con 0x7f6d900039b0 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000c060 con 0x7f6d90004800 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9000c260 con 0x7f6d900051c0 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da6ffd640 1 --2- 172.21.15.1:0/3508904044 >> v2:172.21.15.6:3300/0 conn(0x7f6da806e4f0 0x7f6da806e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da77fe640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 0x7f6d90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da7fff640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900051c0 0x7f6d9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.695+0000 7f6da6ffd640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d90004800 0x7f6d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 -- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900051c0 msgr2=0x7f6d9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d900051c0 0x7f6d9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 -- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d90004800 msgr2=0x7f6d90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d90004800 0x7f6d90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 -- 172.21.15.1:0/3508904044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d90003520 con 0x7f6d900039b0 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da7fff640 1 -- 172.21.15.1:0/3508904044 reap_dead start 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da6ffd640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d90004800 0x7f6d90004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.696+0000 7f6da77fe640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 0x7f6d90003dd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c240 tx=0x7f6d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b09d20c1753dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98007d10 con 0x7f6d900039b0 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d980058d0 con 0x7f6d900039b0 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6da67fc640 1 -- 172.21.15.1:0/3508904044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005d40 con 0x7f6d900039b0 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 -- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 msgr2=0x7f6d90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 0x7f6d90003dd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c240 tx=0x7f6d90005830 comp rx=0 tx=0).stop 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 -- 172.21.15.1:0/3508904044 shutdown_connections 2024-05-09T06:33:48.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 --2- 172.21.15.1:0/3508904044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d900039b0 0x7f6d90003dd0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 -- 172.21.15.1:0/3508904044 >> 172.21.15.1:0/3508904044 conn(0x7f6da806d270 msgr2=0x7f6da806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 -- 172.21.15.1:0/3508904044 shutdown_connections 2024-05-09T06:33:48.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.697+0000 7f6dade93640 1 -- 172.21.15.1:0/3508904044 wait complete. 2024-05-09T06:33:48.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.698+0000 7f6dade93640 1 Processor -- start 2024-05-09T06:33:48.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 -- start start 2024-05-09T06:33:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900039b0 0x7f6da81417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6da81501c0 0x7f6da8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6da806eb00 con 0x7f6da8152320 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6da806e980 con 0x7f6d900039b0 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6dade93640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6da806ec80 con 0x7f6da81501c0 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54452/0 (socket says 172.21.15.1:54452) 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.699+0000 7f6da7fff640 1 -- 172.21.15.1:0/2447273531 learned_addr learned my addr 172.21.15.1:0/2447273531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da77fe640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900039b0 0x7f6da81417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da6ffd640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6da81501c0 0x7f6da8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 -- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6da81501c0 msgr2=0x7f6da8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6da81501c0 0x7f6da8155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 -- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900039b0 msgr2=0x7f6da81417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900039b0 0x7f6da81417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 -- 172.21.15.1:0/2447273531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da81541f0 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6da7fff640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f6da8071560 tx=0x7f6da000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0003970 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da0018070 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81544d0 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0003d70 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.700+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da8154a90 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.701+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da80752b0 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.702+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6da00050d0 con 0x7f6da8152320 2024-05-09T06:33:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.702+0000 7f6d8ffff640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 0x7f6d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.702+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f6da008ad30 con 0x7f6da8152320 2024-05-09T06:33:48.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.702+0000 7f6da77fe640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 0x7f6d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.703+0000 7f6da77fe640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 0x7f6d9406e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6d98033000 tx=0x7f6d9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.706+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6da0059500 con 0x7f6da8152320 2024-05-09T06:33:48.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.892+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6da815e680 con 0x7f6da8152320 2024-05-09T06:33:48.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.893+0000 7f6d8ffff640 1 -- 172.21.15.1:0/2447273531 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7f6da0058e00 con 0x7f6da8152320 2024-05-09T06:33:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:48.875 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 msgr2=0x7f6d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 0x7f6d9406e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6d98033000 tx=0x7f6d9802e000 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 msgr2=0x7f6da81527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f6da8071560 tx=0x7f6da000fbb0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 shutdown_connections 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d9406be30 0x7f6d9406e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6da8152320 0x7f6da81527d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6da81501c0 0x7f6da8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 --2- 172.21.15.1:0/2447273531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d900039b0 0x7f6da81417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 >> 172.21.15.1:0/2447273531 conn(0x7f6da806d270 msgr2=0x7f6da815ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 shutdown_connections 2024-05-09T06:33:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:48.895+0000 7f6dade93640 1 -- 172.21.15.1:0/2447273531 wait complete. 2024-05-09T06:33:48.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:33:48.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:48.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.011+0000 7fd07939e640 1 Processor -- start 2024-05-09T06:33:48.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.011+0000 7fd07939e640 1 -- start start 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd07939e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 0x7fd0740d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd07939e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0740d1a00 0x7fd0740d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd07939e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0740b3180 con 0x7fd0740d0cf0 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd07939e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd0740b3460 con 0x7fd0740d1a00 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd07939e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd0740b3740 con 0x7fd07415e160 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0740d1a00 0x7fd0740d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0740d1a00 0x7fd0740d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54454/0 (socket says 172.21.15.1:54454) 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 -- 172.21.15.1:0/1368449055 learned_addr learned my addr 172.21.15.1:0/1368449055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd073fff640 1 --2- 172.21.15.1:0/1368449055 >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 0x7fd0740d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 -- 172.21.15.1:0/1368449055 >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 msgr2=0x7fd0740d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 --2- 172.21.15.1:0/1368449055 >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 0x7fd0740d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 -- 172.21.15.1:0/1368449055 >> v1:172.21.15.6:6789/0 conn(0x7fd0740d0cf0 legacy=0x7fd0740ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:48.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd06bfff640 1 -- 172.21.15.1:0/1368449055 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0740b3a20 con 0x7fd0740d1a00 2024-05-09T06:33:48.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.012+0000 7fd073fff640 1 --2- 172.21.15.1:0/1368449055 >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 0x7fd0740d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.013+0000 7fd06bfff640 1 --2- 172.21.15.1:0/1368449055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0740d1a00 0x7fd0740d2740 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd07406b4a0 tx=0x7fd05c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f679e98da31ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.013+0000 7fd0737fe640 1 -- 172.21.15.1:0/1368449055 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c037070 con 0x7fd0740d1a00 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.013+0000 7fd0737fe640 1 -- 172.21.15.1:0/1368449055 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c032440 con 0x7fd0740d1a00 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.013+0000 7fd0737fe640 1 -- 172.21.15.1:0/1368449055 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c032810 con 0x7fd0740d1a00 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 -- 172.21.15.1:0/1368449055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0740d1a00 msgr2=0x7fd0740d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 --2- 172.21.15.1:0/1368449055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0740d1a00 0x7fd0740d2740 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd07406b4a0 tx=0x7fd05c02a290 comp rx=0 tx=0).stop 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 -- 172.21.15.1:0/1368449055 shutdown_connections 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 --2- 172.21.15.1:0/1368449055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0740d1a00 0x7fd0740d2740 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 --2- 172.21.15.1:0/1368449055 >> v2:172.21.15.6:3300/0 conn(0x7fd07415e160 0x7fd0740d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 -- 172.21.15.1:0/1368449055 >> 172.21.15.1:0/1368449055 conn(0x7fd0740d3c20 msgr2=0x7fd0740d4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 -- 172.21.15.1:0/1368449055 shutdown_connections 2024-05-09T06:33:48.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.014+0000 7fd07939e640 1 -- 172.21.15.1:0/1368449055 wait complete. 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.015+0000 7fd07939e640 1 Processor -- start 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.015+0000 7fd07939e640 1 -- start start 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0740d0cf0 0x7fd07415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0740d1a00 0x7fd07415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740b4f50 con 0x7fd07415e160 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740d4740 con 0x7fd0740d0cf0 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd07939e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740b50d0 con 0x7fd0740d1a00 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd078b9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd078b9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54462/0 (socket says 172.21.15.1:54462) 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd078b9d640 1 -- 172.21.15.1:0/666398209 learned_addr learned my addr 172.21.15.1:0/666398209 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd073fff640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0740d0cf0 0x7fd07415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.016+0000 7fd06bfff640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0740d1a00 0x7fd07415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 -- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0740d1a00 msgr2=0x7fd07415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0740d1a00 0x7fd07415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 -- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0740d0cf0 msgr2=0x7fd07415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0740d0cf0 0x7fd07415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 -- 172.21.15.1:0/666398209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074160d60 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd078b9d640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b970 tx=0x7fd06c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c019070 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c00cb40 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd074161040 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014440 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.017+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd074161600 con 0x7fd07415e160 2024-05-09T06:33:49.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.018+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0740752b0 con 0x7fd07415e160 2024-05-09T06:33:49.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.018+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd06c01c070 con 0x7fd07415e160 2024-05-09T06:33:49.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.019+0000 7fd0717fa640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 0x7fd05406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.019+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fd06c08ac50 con 0x7fd07415e160 2024-05-09T06:33:49.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.019+0000 7fd073fff640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 0x7fd05406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.019+0000 7fd073fff640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 0x7fd05406e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0740d4710 tx=0x7fd064008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.023+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd06c058d40 con 0x7fd07415e160 2024-05-09T06:33:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.249+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0740d1fe0 con 0x7fd07415e160 2024-05-09T06:33:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.250+0000 7fd0717fa640 1 -- 172.21.15.1:0/666398209 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5170 (secure 0 0 0) 0x7fd06c05d040 con 0x7fd07415e160 2024-05-09T06:33:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:49.231 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:33:18.377403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":23,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 53 2024-05-09T06:33:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 msgr2=0x7fd05406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 0x7fd05406e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0740d4710 tx=0x7fd064008040 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 msgr2=0x7fd0741f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b970 tx=0x7fd06c00be30 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 shutdown_connections 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd05406be60 0x7fd05406e320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07415e160 0x7fd0741f70b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0740d1a00 0x7fd07415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 --2- 172.21.15.1:0/666398209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0740d0cf0 0x7fd07415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 >> 172.21.15.1:0/666398209 conn(0x7fd0740d3c20 msgr2=0x7fd0740c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 shutdown_connections 2024-05-09T06:33:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.252+0000 7fd07939e640 1 -- 172.21.15.1:0/666398209 wait complete. 2024-05-09T06:33:49.245 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:49.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.367+0000 7f06c8977640 1 Processor -- start 2024-05-09T06:33:49.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.367+0000 7f06c8977640 1 -- start start 2024-05-09T06:33:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c8977640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06c406e6d0 0x7f06c406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c8977640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 0x7f06c40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c8977640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c40ce0e0 con 0x7f06c40cdab0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c8977640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f06c40ce3c0 con 0x7f06c406e6d0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c8977640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f06c40ce6a0 con 0x7f06c40cf180 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06c406e6d0 0x7f06c406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06c406e6d0 0x7f06c406eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54464/0 (socket says 172.21.15.1:54464) 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 -- 172.21.15.1:0/2895500330 learned_addr learned my addr 172.21.15.1:0/2895500330 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c1d9b640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 0x7f06c40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 -- 172.21.15.1:0/2895500330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 msgr2=0x7f06c40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 0x7f06c40cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 -- 172.21.15.1:0/2895500330 >> v1:172.21.15.6:6789/0 conn(0x7f06c40cdab0 legacy=0x7f06c415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c259c640 1 -- 172.21.15.1:0/2895500330 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40ce980 con 0x7f06c406e6d0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.368+0000 7f06c1d9b640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 0x7f06c40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.369+0000 7f06c259c640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c406e6d0 0x7f06c406eab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009870 tx=0x7f06bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2c19469fb2274b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.369+0000 7f06c159a640 1 -- 172.21.15.1:0/2895500330 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc037070 con 0x7f06c406e6d0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.369+0000 7f06c159a640 1 -- 172.21.15.1:0/2895500330 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06bc032440 con 0x7f06c406e6d0 2024-05-09T06:33:49.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.369+0000 7f06c159a640 1 -- 172.21.15.1:0/2895500330 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc032810 con 0x7f06c406e6d0 2024-05-09T06:33:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 -- 172.21.15.1:0/2895500330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c406e6d0 msgr2=0x7f06c406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c406e6d0 0x7f06c406eab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009870 tx=0x7f06bc02a210 comp rx=0 tx=0).stop 2024-05-09T06:33:49.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 -- 172.21.15.1:0/2895500330 shutdown_connections 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cf180 0x7f06c40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 --2- 172.21.15.1:0/2895500330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c406e6d0 0x7f06c406eab0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 -- 172.21.15.1:0/2895500330 >> 172.21.15.1:0/2895500330 conn(0x7f06c40cc4b0 msgr2=0x7f06c40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 -- 172.21.15.1:0/2895500330 shutdown_connections 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.370+0000 7f06c8977640 1 -- 172.21.15.1:0/2895500330 wait complete. 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.371+0000 7f06c8977640 1 Processor -- start 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.371+0000 7f06c8977640 1 -- start start 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c406e6d0 0x7f06c4130820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 0x7f06c4167c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4070ed0 con 0x7f06c40cf180 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f06c406eff0 con 0x7f06c406e6d0 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c8977640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f06c41447c0 con 0x7f06c40cdab0 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54476/0 (socket says 172.21.15.1:54476) 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 -- 172.21.15.1:0/2597213569 learned_addr learned my addr 172.21.15.1:0/2597213569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c259c640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c406e6d0 0x7f06c4130820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c1d9b640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 0x7f06c4167c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 -- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 msgr2=0x7f06c4167c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 0x7f06c4167c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 -- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c406e6d0 msgr2=0x7f06c4130820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.372+0000 7f06c2d9d640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c406e6d0 0x7f06c4130820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06c2d9d640 1 -- 172.21.15.1:0/2597213569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c4131f60 con 0x7f06c40cf180 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06c1d9b640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 0x7f06c4167c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06c2d9d640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c7b0 tx=0x7f06b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004b60 con 0x7f06c40cf180 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4004d00 con 0x7f06c40cf180 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c4132240 con 0x7f06c40cf180 2024-05-09T06:33:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400aa90 con 0x7f06c40cf180 2024-05-09T06:33:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.373+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c4132800 con 0x7f06c40cf180 2024-05-09T06:33:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.374+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c40752b0 con 0x7f06c40cf180 2024-05-09T06:33:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.374+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06b400ac30 con 0x7f06c40cf180 2024-05-09T06:33:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.375+0000 7f06baffd640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 0x7f06a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.375+0000 7f06c259c640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 0x7f06a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.375+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f06b408b560 con 0x7f06c40cf180 2024-05-09T06:33:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.375+0000 7f06c259c640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 0x7f06a406e320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06bc034000 tx=0x7f06bc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.379+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f06b4059d30 con 0x7f06c40cf180 2024-05-09T06:33:49.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.604+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c406e4b0 con 0x7f06c40cf180 2024-05-09T06:33:49.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.605+0000 7f06baffd640 1 -- 172.21.15.1:0/2597213569 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5170 (secure 0 0 0) 0x7f06b4059630 con 0x7f06c40cf180 2024-05-09T06:33:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:49.587 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:32:04.798236+0000","modified":"2024-05-09T06:33:18.377403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":23,"addr":"172.21.15.1:6836/3897834247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3897834247},{"type":"v1","addr":"172.21.15.1:6836","nonce":3897834247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113]},"id":5}]} 2024-05-09T06:33:49.587 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 53 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 msgr2=0x7f06a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 0x7f06a406e320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06bc034000 tx=0x7f06bc02e000 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 msgr2=0x7f06c4133000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c7b0 tx=0x7f06b400cc70 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 shutdown_connections 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06a406be60 0x7f06a406e320 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06c40cf180 0x7f06c4133000 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06c40cdab0 0x7f06c4167c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 --2- 172.21.15.1:0/2597213569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c406e6d0 0x7f06c4130820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 >> 172.21.15.1:0/2597213569 conn(0x7f06c40cc4b0 msgr2=0x7f06c40b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 shutdown_connections 2024-05-09T06:33:49.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.607+0000 7f06c8977640 1 -- 172.21.15.1:0/2597213569 wait complete. 2024-05-09T06:33:49.599 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.723+0000 7fef553a9640 1 Processor -- start 2024-05-09T06:33:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.723+0000 7fef553a9640 1 -- start start 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef553a9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fef50099b30 0x7fef500b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef553a9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef50099930 con 0x7fef500c6860 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef553a9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef50162930 con 0x7fef5015e4b0 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef553a9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fef500b1e20 con 0x7fef50099b30 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef47fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fef50099b30 0x7fef500b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef4effd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fef500c6860 0x7fef500c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44474/0 (socket says 172.21.15.1:44474) 2024-05-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef4effd640 1 -- 172.21.15.1:0/2079971254 learned_addr learned my addr 172.21.15.1:0/2079971254 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef47fff640 1 -- 172.21.15.1:0/2079971254 >> v1:172.21.15.6:6789/0 conn(0x7fef5015e4b0 legacy=0x7fef5015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef47fff640 1 -- 172.21.15.1:0/2079971254 >> v1:172.21.15.1:6789/0 conn(0x7fef500c6860 legacy=0x7fef500c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef4e7fc640 1 -- 172.21.15.1:0/2079971254 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620478718 0 0) 0x7fef40001110 con 0x7fef500c6860 2024-05-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.724+0000 7fef47fff640 1 -- 172.21.15.1:0/2079971254 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef50162c10 con 0x7fef50099b30 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.725+0000 7fef47fff640 1 --2- 172.21.15.1:0/2079971254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500b2aa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fef380097a0 tx=0x7fef3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7e9df4476f8e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.725+0000 7fef4e7fc640 1 -- 172.21.15.1:0/2079971254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38037070 con 0x7fef50099b30 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.725+0000 7fef4e7fc640 1 -- 172.21.15.1:0/2079971254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef38032440 con 0x7fef50099b30 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.725+0000 7fef4e7fc640 1 -- 172.21.15.1:0/2079971254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38032810 con 0x7fef50099b30 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 -- 172.21.15.1:0/2079971254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 msgr2=0x7fef500b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 --2- 172.21.15.1:0/2079971254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500b2aa0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fef380097a0 tx=0x7fef3802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 -- 172.21.15.1:0/2079971254 shutdown_connections 2024-05-09T06:33:49.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 --2- 172.21.15.1:0/2079971254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500b2aa0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 -- 172.21.15.1:0/2079971254 >> 172.21.15.1:0/2079971254 conn(0x7fef500b1fb0 msgr2=0x7fef500b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:49.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 -- 172.21.15.1:0/2079971254 shutdown_connections 2024-05-09T06:33:49.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.726+0000 7fef553a9640 1 -- 172.21.15.1:0/2079971254 wait complete. 2024-05-09T06:33:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.727+0000 7fef553a9640 1 Processor -- start 2024-05-09T06:33:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.727+0000 7fef553a9640 1 -- start start 2024-05-09T06:33:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef500c6860 0x7fef501527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef5015e4b0 0x7fef500c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef5015fa30 con 0x7fef50099b30 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef5015ee90 con 0x7fef5015e4b0 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef500cdf50 con 0x7fef500c6860 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54498/0 (socket says 172.21.15.1:54498) 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 -- 172.21.15.1:0/4249921839 learned_addr learned my addr 172.21.15.1:0/4249921839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4f7fe640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef5015e4b0 0x7fef500c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef47fff640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef500c6860 0x7fef501527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 -- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef500c6860 msgr2=0x7fef501527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef500c6860 0x7fef501527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 -- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef5015e4b0 msgr2=0x7fef500c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef5015e4b0 0x7fef500c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 -- 172.21.15.1:0/4249921839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef500c8ba0 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef4effd640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c7b0 tx=0x7fef4000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40018070 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef400072d0 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef500c8e80 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.728+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef400047e0 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.729+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef500c9440 con 0x7fef50099b30 2024-05-09T06:33:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.729+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef500752b0 con 0x7fef50099b30 2024-05-09T06:33:49.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.730+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef4001b070 con 0x7fef50099b30 2024-05-09T06:33:49.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.730+0000 7fef477fe640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 0x7fef3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:49.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.730+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7fef4008a440 con 0x7fef50099b30 2024-05-09T06:33:49.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.730+0000 7fef47fff640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 0x7fef3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:49.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.731+0000 7fef47fff640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 0x7fef3006e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fef38034000 tx=0x7fef3802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.735+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef50099930 con 0x7fef50099b30 2024-05-09T06:33:49.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.917+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef5006ce30 con 0x7fef50099b30 2024-05-09T06:33:49.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.918+0000 7fef477fe640 1 -- 172.21.15.1:0/4249921839 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16940 (secure 0 0 0) 0x7fef40058b70 con 0x7fef50099b30 2024-05-09T06:33:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:49.900 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:32:23.361985+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:32:02.450232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-09T06:32:03.777916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 msgr2=0x7fef3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 0x7fef3006e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fef38034000 tx=0x7fef3802e000 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 msgr2=0x7fef500c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c7b0 tx=0x7fef4000cc70 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 shutdown_connections 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef3006be30 0x7fef3006e2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef5015e4b0 0x7fef500c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef500c6860 0x7fef501527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 --2- 172.21.15.1:0/4249921839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef50099b30 0x7fef500c9c40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 >> 172.21.15.1:0/4249921839 conn(0x7fef500b1fb0 msgr2=0x7fef500d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 shutdown_connections 2024-05-09T06:33:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:49.920+0000 7fef553a9640 1 -- 172.21.15.1:0/4249921839 wait complete. 2024-05-09T06:33:49.913 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:33:50.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.036+0000 7f799014f640 1 Processor -- start 2024-05-09T06:33:50.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.036+0000 7f799014f640 1 -- start start 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f799014f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7988099b30 0x7f79880b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f799014f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f798815e4b0 0x7f798815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f799014f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7988099930 con 0x7f79880c6860 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f799014f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f79880b1e20 con 0x7f798815e4b0 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f799014f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f798815ee90 con 0x7f7988099b30 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f798e6ec640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f798815e4b0 0x7f798815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f798e6ec640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f798815e4b0 0x7f798815f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54502/0 (socket says 172.21.15.1:54502) 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f798e6ec640 1 -- 172.21.15.1:0/1580328474 learned_addr learned my addr 172.21.15.1:0/1580328474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:50.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.037+0000 7f798d6ea640 1 --2- 172.21.15.1:0/1580328474 >> v2:172.21.15.6:3300/0 conn(0x7f7988099b30 0x7f79880b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.038+0000 7f798e6ec640 1 -- 172.21.15.1:0/1580328474 >> v2:172.21.15.6:3300/0 conn(0x7f7988099b30 msgr2=0x7f79880b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:50.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.038+0000 7f798e6ec640 1 --2- 172.21.15.1:0/1580328474 >> v2:172.21.15.6:3300/0 conn(0x7f7988099b30 0x7f79880b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.038+0000 7f798e6ec640 1 -- 172.21.15.1:0/1580328474 >> v1:172.21.15.6:6789/0 conn(0x7f79880c6860 legacy=0x7f79880c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.038+0000 7f798e6ec640 1 -- 172.21.15.1:0/1580328474 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7988162930 con 0x7f798815e4b0 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.038+0000 7f798e6ec640 1 --2- 172.21.15.1:0/1580328474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f798815f320 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f798809e570 tx=0x7f797802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd610a42065bec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f798cee9640 1 -- 172.21.15.1:0/1580328474 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978036070 con 0x7f798815e4b0 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f798cee9640 1 -- 172.21.15.1:0/1580328474 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7978031600 con 0x7f798815e4b0 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f798cee9640 1 -- 172.21.15.1:0/1580328474 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79780319d0 con 0x7f798815e4b0 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 -- 172.21.15.1:0/1580328474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 msgr2=0x7f798815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 --2- 172.21.15.1:0/1580328474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f798815f320 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f798809e570 tx=0x7f797802a730 comp rx=0 tx=0).stop 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 -- 172.21.15.1:0/1580328474 shutdown_connections 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 --2- 172.21.15.1:0/1580328474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f798815f320 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 --2- 172.21.15.1:0/1580328474 >> v2:172.21.15.6:3300/0 conn(0x7f7988099b30 0x7f79880b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 -- 172.21.15.1:0/1580328474 >> 172.21.15.1:0/1580328474 conn(0x7f79880b1fb0 msgr2=0x7f79880b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:50.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 -- 172.21.15.1:0/1580328474 shutdown_connections 2024-05-09T06:33:50.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.039+0000 7f799014f640 1 -- 172.21.15.1:0/1580328474 wait complete. 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 Processor -- start 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 -- start start 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7988099b30 0x7f79880c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79880c6860 0x7f79880cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f798815f860 con 0x7f798815e4b0 2024-05-09T06:33:50.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f798815eb20 con 0x7f7988099b30 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f799014f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f798815f9e0 con 0x7f79880c6860 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f798e6ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f798e6ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54518/0 (socket says 172.21.15.1:54518) 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f798e6ec640 1 -- 172.21.15.1:0/2608552617 learned_addr learned my addr 172.21.15.1:0/2608552617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.041+0000 7f798deeb640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7988099b30 0x7f79880c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798d6ea640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79880c6860 0x7f79880cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 -- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79880c6860 msgr2=0x7f79880cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79880c6860 0x7f79880cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 -- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7988099b30 msgr2=0x7f79880c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7988099b30 0x7f79880c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 -- 172.21.15.1:0/2608552617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79880c7ca0 con 0x7f798815e4b0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f798e6ec640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7978034040 tx=0x7f7978004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978036050 con 0x7f798815e4b0 2024-05-09T06:33:50.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7978042070 con 0x7f798815e4b0 2024-05-09T06:33:50.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79880c7f80 con 0x7f798815e4b0 2024-05-09T06:33:50.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978009510 con 0x7f798815e4b0 2024-05-09T06:33:50.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.042+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79880c8540 con 0x7f798815e4b0 2024-05-09T06:33:50.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.043+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79880752b0 con 0x7f798815e4b0 2024-05-09T06:33:50.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.044+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7978007bf0 con 0x7f798815e4b0 2024-05-09T06:33:50.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.044+0000 7f79867fc640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 0x7f797406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.044+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6187+0+0 (secure 0 0 0) 0x7f79780abb40 con 0x7f798815e4b0 2024-05-09T06:33:50.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.044+0000 7f798deeb640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 0x7f797406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.045+0000 7f798deeb640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 0x7f797406e320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f79880b36c0 tx=0x7f7980009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.048+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7978079e70 con 0x7f798815e4b0 2024-05-09T06:33:50.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.269+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f798806ce30 con 0x7f798815e4b0 2024-05-09T06:33:50.250 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:33:50.269+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:33:50.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.271+0000 7f79867fc640 1 -- 172.21.15.1:0/2608552617 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f797807e170 con 0x7f798815e4b0 2024-05-09T06:33:50.252 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:33:50.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.274+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 msgr2=0x7f797406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.274+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 0x7f797406e320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f79880b36c0 tx=0x7f7980009040 comp rx=0 tx=0).stop 2024-05-09T06:33:50.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.274+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 msgr2=0x7f79880cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.274+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7978034040 tx=0x7f7978004a40 comp rx=0 tx=0).stop 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 shutdown_connections 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f797406be60 0x7f797406e320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798815e4b0 0x7f79880cc780 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79880c6860 0x7f79880cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 --2- 172.21.15.1:0/2608552617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7988099b30 0x7f79880c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 >> 172.21.15.1:0/2608552617 conn(0x7f79880b1fb0 msgr2=0x7f798815ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 shutdown_connections 2024-05-09T06:33:50.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.275+0000 7f799014f640 1 -- 172.21.15.1:0/2608552617 wait complete. 2024-05-09T06:33:50.259 INFO:tasks.ceph.mds.a.smithi001.stderr: -4156> 2024-05-09T06:32:30.411+0000 7f8b218be640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:33:50.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:33:50.289 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:33:50.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.393+0000 7f1010d9d640 1 Processor -- start 2024-05-09T06:33:50.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.393+0000 7f1010d9d640 1 -- start start 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1010d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 0x7f100c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1010d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f100c06ede0 0x7f100c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1010d9d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100c15b420 con 0x7f100c15c4a0 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1010d9d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f100c158e30 con 0x7f100c06ede0 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1010d9d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f100c072480 con 0x7f100c158a50 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f100ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f100c06ede0 0x7f100c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1009d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f100c15c4a0 0x7f100c06e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44494/0 (socket says 172.21.15.1:44494) 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f100ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f100c06ede0 0x7f100c071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54544/0 (socket says 172.21.15.1:54544) 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.394+0000 7f1009d9b640 1 -- 172.21.15.1:0/3033842858 learned_addr learned my addr 172.21.15.1:0/3033842858 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100a59c640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 0x7f100c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100ad9d640 1 -- 172.21.15.1:0/3033842858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 msgr2=0x7f100c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100ad9d640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 0x7f100c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100ad9d640 1 -- 172.21.15.1:0/3033842858 >> v1:172.21.15.1:6789/0 conn(0x7f100c15c4a0 legacy=0x7f100c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100ad9d640 1 -- 172.21.15.1:0/3033842858 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c15b6e0 con 0x7f100c06ede0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100959a640 1 -- 172.21.15.1:0/3033842858 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426866472 0 0) 0x7f100c15b420 con 0x7f100c15c4a0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100a59c640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 0x7f100c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.395+0000 7f100ad9d640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c071e00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f100400ba40 tx=0x7f100402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eea8241508cab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f100959a640 1 -- 172.21.15.1:0/3033842858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004032750 con 0x7f100c06ede0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f100959a640 1 -- 172.21.15.1:0/3033842858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1004030050 con 0x7f100c06ede0 2024-05-09T06:33:50.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f100959a640 1 -- 172.21.15.1:0/3033842858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004032b10 con 0x7f100c06ede0 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 -- 172.21.15.1:0/3033842858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 msgr2=0x7f100c071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c071e00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f100400ba40 tx=0x7f100402c850 comp rx=0 tx=0).stop 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 -- 172.21.15.1:0/3033842858 shutdown_connections 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c071e00 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3033842858 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c158a50 0x7f100c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 -- 172.21.15.1:0/3033842858 >> 172.21.15.1:0/3033842858 conn(0x7f100c0cc4d0 msgr2=0x7f100c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 -- 172.21.15.1:0/3033842858 shutdown_connections 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.396+0000 7f1010d9d640 1 -- 172.21.15.1:0/3033842858 wait complete. 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 Processor -- start 2024-05-09T06:33:50.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 -- start start 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 0x7f100c1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c15c4a0 0x7f100c0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f100c15bf80 con 0x7f100c06ede0 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f100c15bca0 con 0x7f100c158a50 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f1010d9d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f100c070ea0 con 0x7f100c15c4a0 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f100a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f100a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54550/0 (socket says 172.21.15.1:54550) 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.398+0000 7f100a59c640 1 -- 172.21.15.1:0/3263660245 learned_addr learned my addr 172.21.15.1:0/3263660245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1009d9b640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 0x7f100c1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100ad9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c15c4a0 0x7f100c0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 -- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c15c4a0 msgr2=0x7f100c0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c15c4a0 0x7f100c0afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 -- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 msgr2=0x7f100c1483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 0x7f100c1483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 -- 172.21.15.1:0/3263660245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c15c8d0 con 0x7f100c06ede0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1009d9b640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 0x7f100c1483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f100a59c640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00cf60 tx=0x7f0ffc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc01a070 con 0x7f100c06ede0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00d840 con 0x7f100c06ede0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c15cbb0 con 0x7f100c06ede0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00dc10 con 0x7f100c06ede0 2024-05-09T06:33:50.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.399+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c15d170 con 0x7f100c06ede0 2024-05-09T06:33:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.400+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c155280 con 0x7f100c06ede0 2024-05-09T06:33:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.401+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ffc01d070 con 0x7f100c06ede0 2024-05-09T06:33:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.401+0000 7f1002ffd640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 0x7f0fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:50.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.401+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6273+0+0 (secure 0 0 0) 0x7f0ffc08b8d0 con 0x7f100c06ede0 2024-05-09T06:33:50.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.401+0000 7f1009d9b640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 0x7f0fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:50.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.402+0000 7f1009d9b640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 0x7f0fec06e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b09d0 tx=0x7f0ff4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:50.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.406+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f100c155280 con 0x7f100c06ede0 2024-05-09T06:33:50.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:50.625+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f100c0752b0 con 0x7f100c06ede0 2024-05-09T06:33:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.272+0000 7f1002ffd640 1 -- 172.21.15.1:0/3263660245 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7f0ffc0599f0 con 0x7f100c06ede0 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 msgr2=0x7f0fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 0x7f0fec06e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b09d0 tx=0x7f0ff4009040 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 msgr2=0x7f100c15dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00cf60 tx=0x7f0ffc00cf90 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 shutdown_connections 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0fec06be30 0x7f0fec06e2f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f100c15c4a0 0x7f100c0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100c158a50 0x7f100c1483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 --2- 172.21.15.1:0/3263660245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f100c06ede0 0x7f100c15dc50 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.275+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 >> 172.21.15.1:0/3263660245 conn(0x7f100c0cc4d0 msgr2=0x7f100c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:51.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.276+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 shutdown_connections 2024-05-09T06:33:51.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.276+0000 7f1010d9d640 1 -- 172.21.15.1:0/3263660245 wait complete. 2024-05-09T06:33:51.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:33:51.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.447+0000 7fc0af9d5640 1 Processor -- start 2024-05-09T06:33:51.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.447+0000 7fc0af9d5640 1 -- start start 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.448+0000 7fc0af9d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 0x7fc0a80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.448+0000 7fc0af9d5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.448+0000 7fc0af9d5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a80b27d0 con 0x7fc0a806c570 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.448+0000 7fc0af9d5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a815e650 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.448+0000 7fc0af9d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a80b3100 con 0x7fc0a80cfea0 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54564/0 (socket says 172.21.15.1:54564) 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 -- 172.21.15.1:0/2314455274 learned_addr learned my addr 172.21.15.1:0/2314455274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0ad771640 1 --2- 172.21.15.1:0/2314455274 >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 0x7fc0a80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 -- 172.21.15.1:0/2314455274 >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 msgr2=0x7fc0a80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 --2- 172.21.15.1:0/2314455274 >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 0x7fc0a80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 -- 172.21.15.1:0/2314455274 >> v1:172.21.15.6:6789/0 conn(0x7fc0a806c570 legacy=0x7fc0a80d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0adf72640 1 -- 172.21.15.1:0/2314455274 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a80b2ab0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.449+0000 7fc0ad771640 1 --2- 172.21.15.1:0/2314455274 >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 0x7fc0a80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:51.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0adf72640 1 --2- 172.21.15.1:0/2314455274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80d2d10 tx=0x7fc0a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85401863510409d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:51.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc09ffff640 1 -- 172.21.15.1:0/2314455274 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0036070 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc09ffff640 1 -- 172.21.15.1:0/2314455274 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a0031600 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc09ffff640 1 -- 172.21.15.1:0/2314455274 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00319d0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 -- 172.21.15.1:0/2314455274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 msgr2=0x7fc0a80aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/2314455274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80d2d10 tx=0x7fc0a002a730 comp rx=0 tx=0).stop 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 -- 172.21.15.1:0/2314455274 shutdown_connections 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/2314455274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a80aedd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/2314455274 >> v2:172.21.15.6:3300/0 conn(0x7fc0a80cfea0 0x7fc0a80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.450+0000 7fc0af9d5640 1 -- 172.21.15.1:0/2314455274 >> 172.21.15.1:0/2314455274 conn(0x7fc0a80d1c10 msgr2=0x7fc0a80d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.451+0000 7fc0af9d5640 1 -- 172.21.15.1:0/2314455274 shutdown_connections 2024-05-09T06:33:51.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.451+0000 7fc0af9d5640 1 -- 172.21.15.1:0/2314455274 wait complete. 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.452+0000 7fc0af9d5640 1 Processor -- start 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.452+0000 7fc0af9d5640 1 -- start start 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a806c570 0x7fc0a809f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0a80cfea0 0x7fc0a8160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a80d0ed0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a80b2580 con 0x7fc0a806c570 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a80d1050 con 0x7fc0a80cfea0 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0ad771640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a806c570 0x7fc0a809f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54574/0 (socket says 172.21.15.1:54574) 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 -- 172.21.15.1:0/1085667383 learned_addr learned my addr 172.21.15.1:0/1085667383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:51.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0adf72640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0a80cfea0 0x7fc0a8160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 -- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0a80cfea0 msgr2=0x7fc0a8160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0a80cfea0 0x7fc0a8160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 -- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a806c570 msgr2=0x7fc0a809f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a806c570 0x7fc0a809f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 -- 172.21.15.1:0/1085667383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81562a0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0acf70640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c400 tx=0x7fc09800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098017070 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0980052d0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a8156580 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.453+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098003830 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.454+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81604a0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.455+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8075320 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.455+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc09801a070 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.456+0000 7fc09dffb640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 0x7fc08c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.456+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6273+0+0 (secure 0 0 0) 0x7fc09808a3f0 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.456+0000 7fc0ad771640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 0x7fc08c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.457+0000 7fc0ad771640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 0x7fc08c06e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:51.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.460+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc098058510 con 0x7fc0a80ae9a0 2024-05-09T06:33:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:51.648+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0a8156750 con 0x7fc0a80ae9a0 2024-05-09T06:33:52.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.275+0000 7fc09dffb640 1 -- 172.21.15.1:0/1085667383 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) ==== 181+0+0 (secure 0 0 0) 0x7fc09805c810 con 0x7fc0a80ae9a0 2024-05-09T06:33:52.256 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:33:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.278+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 msgr2=0x7fc08c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.278+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 0x7fc08c06e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4008040 comp rx=0 tx=0).stop 2024-05-09T06:33:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.279+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 msgr2=0x7fc0a81f3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.279+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c400 tx=0x7fc09800c8c0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 shutdown_connections 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc08c06be30 0x7fc08c06e2f0 secure :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4008040 comp rx=0 tx=0).stop 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0a80cfea0 0x7fc0a8160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0a80ae9a0 0x7fc0a81f3e40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 --2- 172.21.15.1:0/1085667383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0a806c570 0x7fc0a809f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.280+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 >> 172.21.15.1:0/1085667383 conn(0x7fc0a80d1c10 msgr2=0x7fc0a815d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.281+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 shutdown_connections 2024-05-09T06:33:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.281+0000 7fc0af9d5640 1 -- 172.21.15.1:0/1085667383 wait complete. 2024-05-09T06:33:52.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:33:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.405+0000 7f304224c640 1 Processor -- start 2024-05-09T06:33:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.405+0000 7f304224c640 1 -- start start 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.405+0000 7f304224c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f303c0d2630 0x7f303c15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f304224c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c0d1cc0 con 0x7f303c0b2ed0 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f304224c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c0d4d10 con 0x7f303c0d2a10 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f304224c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f303c0d1f80 con 0x7f303c0d2630 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f303c0d2630 0x7f303c15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f303c0d2630 0x7f303c15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54578/0 (socket says 172.21.15.1:54578) 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 -- 172.21.15.1:0/2695110739 learned_addr learned my addr 172.21.15.1:0/2695110739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 -- 172.21.15.1:0/2695110739 >> v1:172.21.15.6:6789/0 conn(0x7f303c0d2a10 legacy=0x7f303c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 -- 172.21.15.1:0/2695110739 >> v1:172.21.15.1:6789/0 conn(0x7f303c0b2ed0 legacy=0x7f303c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.406+0000 7f303b7fe640 1 -- 172.21.15.1:0/2695110739 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c0b0c00 con 0x7f303c0d2630 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f303b7fe640 1 --2- 172.21.15.1:0/2695110739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c15ddf0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f302c009870 tx=0x7f302c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ae71597470f9fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f303a7fc640 1 -- 172.21.15.1:0/2695110739 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c037070 con 0x7f303c0d2630 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f303a7fc640 1 -- 172.21.15.1:0/2695110739 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302c02ae50 con 0x7f303c0d2630 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f303a7fc640 1 -- 172.21.15.1:0/2695110739 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c032860 con 0x7f303c0d2630 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f304224c640 1 -- 172.21.15.1:0/2695110739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 msgr2=0x7f303c15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f304224c640 1 --2- 172.21.15.1:0/2695110739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c15ddf0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f302c009870 tx=0x7f302c02a300 comp rx=0 tx=0).stop 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f304224c640 1 -- 172.21.15.1:0/2695110739 shutdown_connections 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f304224c640 1 --2- 172.21.15.1:0/2695110739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c15ddf0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.407+0000 7f304224c640 1 -- 172.21.15.1:0/2695110739 >> 172.21.15.1:0/2695110739 conn(0x7f303c0d3ee0 msgr2=0x7f303c0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.408+0000 7f304224c640 1 -- 172.21.15.1:0/2695110739 shutdown_connections 2024-05-09T06:33:52.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.408+0000 7f304224c640 1 -- 172.21.15.1:0/2695110739 wait complete. 2024-05-09T06:33:52.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.409+0000 7f304224c640 1 Processor -- start 2024-05-09T06:33:52.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.409+0000 7f304224c640 1 -- start start 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f303c0b2ed0 0x7f303c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f303c0d2a10 0x7f303c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0cfd30 con 0x7f303c0d2630 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0cfbb0 con 0x7f303c0d2a10 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f303c0cfeb0 con 0x7f303c0b2ed0 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54586/0 (socket says 172.21.15.1:54586) 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 -- 172.21.15.1:0/2204549030 learned_addr learned my addr 172.21.15.1:0/2204549030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303bfff640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f303c0d2a10 0x7f303c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303b7fe640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f303c0b2ed0 0x7f303c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 -- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f303c0b2ed0 msgr2=0x7f303c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f303c0b2ed0 0x7f303c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 -- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f303c0d2a10 msgr2=0x7f303c0b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f303c0d2a10 0x7f303c0b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 -- 172.21.15.1:0/2204549030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c15e020 con 0x7f303c0d2630 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f303affd640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f303000c480 tx=0x7f303000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30300052d0 con 0x7f303c0d2630 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3030005470 con 0x7f303c0d2630 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c15e300 con 0x7f303c0d2630 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030012870 con 0x7f303c0d2630 2024-05-09T06:33:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.410+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c15e8c0 con 0x7f303c0d2630 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.411+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c15eac0 con 0x7f303c0d2630 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.412+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3030012a10 con 0x7f303c0d2630 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.413+0000 7f3023fff640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 0x7f302806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.413+0000 7f303b7fe640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 0x7f302806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.413+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5835+0+0 (secure 0 0 0) 0x7f303008a960 con 0x7f303c0d2630 2024-05-09T06:33:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.413+0000 7f303b7fe640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 0x7f302806e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f302c034000 tx=0x7f302c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:52.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.417+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f303c15eac0 con 0x7f303c0d2630 2024-05-09T06:33:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:52.603+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f303c075320 con 0x7f303c0d2630 2024-05-09T06:33:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.276+0000 7f3023fff640 1 -- 172.21.15.1:0/2204549030 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) ==== 169+0+0 (secure 0 0 0) 0x7f3030058cd0 con 0x7f303c0d2630 2024-05-09T06:33:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:33:53.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 msgr2=0x7f302806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 0x7f302806e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f302c034000 tx=0x7f302c02e000 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 msgr2=0x7f303c1f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f303000c480 tx=0x7f303000c940 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 shutdown_connections 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f302806be30 0x7f302806e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f303c0d2a10 0x7f303c0b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f303c0d2630 0x7f303c1f4bf0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 --2- 172.21.15.1:0/2204549030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f303c0b2ed0 0x7f303c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.281+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 >> 172.21.15.1:0/2204549030 conn(0x7f303c0d3ee0 msgr2=0x7f303c0b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.282+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 shutdown_connections 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.282+0000 7f304224c640 1 -- 172.21.15.1:0/2204549030 wait complete. 2024-05-09T06:33:53.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:33:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.409+0000 7f3c658f5640 1 Processor -- start 2024-05-09T06:33:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.409+0000 7f3c658f5640 1 -- start start 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c658f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3c600d1c70 0x7f3c6015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c658f5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c60161fe0 con 0x7f3c6015e190 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c658f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c601622c0 con 0x7f3c600d45f0 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c658f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6015f580 con 0x7f3c600d1c70 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c5e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3c600d1c70 0x7f3c6015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c5effd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3c6015e190 0x7f3c600d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44522/0 (socket says 172.21.15.1:44522) 2024-05-09T06:33:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.410+0000 7f3c5effd640 1 -- 172.21.15.1:0/3601250990 learned_addr learned my addr 172.21.15.1:0/3601250990 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.411+0000 7f3c5e7fc640 1 -- 172.21.15.1:0/3601250990 >> v1:172.21.15.6:6789/0 conn(0x7f3c600d45f0 legacy=0x7f3c600b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.411+0000 7f3c5e7fc640 1 -- 172.21.15.1:0/3601250990 >> v1:172.21.15.1:6789/0 conn(0x7f3c6015e190 legacy=0x7f3c600d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.411+0000 7f3c5e7fc640 1 -- 172.21.15.1:0/3601250990 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c601625a0 con 0x7f3c600d1c70 2024-05-09T06:33:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.411+0000 7f3c5dffb640 1 -- 172.21.15.1:0/3601250990 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684162463 0 0) 0x7f3c60161fe0 con 0x7f3c6015e190 2024-05-09T06:33:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c5e7fc640 1 --2- 172.21.15.1:0/3601250990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c600d1c70 0x7f3c6015f780 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097a0 tx=0x7f3c4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef4ffcfb1bf991 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c5dffb640 1 -- 172.21.15.1:0/3601250990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48037070 con 0x7f3c600d1c70 2024-05-09T06:33:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c5dffb640 1 -- 172.21.15.1:0/3601250990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c48032440 con 0x7f3c600d1c70 2024-05-09T06:33:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c5dffb640 1 -- 172.21.15.1:0/3601250990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48032810 con 0x7f3c600d1c70 2024-05-09T06:33:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 -- 172.21.15.1:0/3601250990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c600d1c70 msgr2=0x7f3c6015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 --2- 172.21.15.1:0/3601250990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c600d1c70 0x7f3c6015f780 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097a0 tx=0x7f3c4802a230 comp rx=0 tx=0).stop 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 -- 172.21.15.1:0/3601250990 shutdown_connections 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 --2- 172.21.15.1:0/3601250990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c600d1c70 0x7f3c6015f780 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 -- 172.21.15.1:0/3601250990 >> 172.21.15.1:0/3601250990 conn(0x7f3c60073340 msgr2=0x7f3c6009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 -- 172.21.15.1:0/3601250990 shutdown_connections 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.412+0000 7f3c658f5640 1 -- 172.21.15.1:0/3601250990 wait complete. 2024-05-09T06:33:53.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.413+0000 7f3c658f5640 1 Processor -- start 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 -- start start 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c600d1c70 0x7f3c600cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c600d45f0 0x7f3c6014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c600b56d0 con 0x7f3c6015e190 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60161c60 con 0x7f3c600d45f0 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.414+0000 7f3c658f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3c600b5850 con 0x7f3c600d1c70 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54602/0 (socket says 172.21.15.1:54602) 2024-05-09T06:33:53.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 -- 172.21.15.1:0/2727417095 learned_addr learned my addr 172.21.15.1:0/2727417095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5e7fc640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c600d45f0 0x7f3c6014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5effd640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c600d1c70 0x7f3c600cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 -- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c600d1c70 msgr2=0x7f3c600cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c600d1c70 0x7f3c600cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 -- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c600d45f0 msgr2=0x7f3c6014e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c600d45f0 0x7f3c6014e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 -- 172.21.15.1:0/2727417095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c600c7e20 con 0x7f3c6015e190 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c5f7fe640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c3a0 tx=0x7f3c5000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50017070 con 0x7f3c6015e190 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c500052d0 con 0x7f3c6015e190 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c600c8100 con 0x7f3c6015e190 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50003830 con 0x7f3c6015e190 2024-05-09T06:33:53.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.415+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c600c86c0 con 0x7f3c6015e190 2024-05-09T06:33:53.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.416+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c600752b0 con 0x7f3c6015e190 2024-05-09T06:33:53.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.417+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c5001a070 con 0x7f3c6015e190 2024-05-09T06:33:53.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.417+0000 7f3c57fff640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 0x7f3c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.417+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f3c5008a0d0 con 0x7f3c6015e190 2024-05-09T06:33:53.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.418+0000 7f3c5effd640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 0x7f3c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.418+0000 7f3c5effd640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 0x7f3c4006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007f30 tx=0x7f3c58007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.422+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c50058570 con 0x7f3c6015e190 2024-05-09T06:33:53.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.604+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3c6008e1c0 con 0x7f3c6015e190 2024-05-09T06:33:53.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.607+0000 7f3c57fff640 1 -- 172.21.15.1:0/2727417095 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f3c5005c870 con 0x7f3c6015e190 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.609+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 msgr2=0x7f3c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.609+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 0x7f3c4006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007f30 tx=0x7f3c58007e80 comp rx=0 tx=0).stop 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 msgr2=0x7f3c600cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c3a0 tx=0x7f3c5000c860 comp rx=0 tx=0).stop 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 shutdown_connections 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3c4006be30 0x7f3c4006e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c6015e190 0x7f3c600cc760 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c600d45f0 0x7f3c6014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 --2- 172.21.15.1:0/2727417095 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c600d1c70 0x7f3c600cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 >> 172.21.15.1:0/2727417095 conn(0x7f3c60073340 msgr2=0x7f3c6015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:53.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 shutdown_connections 2024-05-09T06:33:53.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.610+0000 7f3c658f5640 1 -- 172.21.15.1:0/2727417095 wait complete. 2024-05-09T06:33:53.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-09T06:33:53.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.725+0000 7f67fe95a640 1 Processor -- start 2024-05-09T06:33:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.725+0000 7f67fe95a640 1 -- start start 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fe95a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f67f81565e0 0x7f67f8074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fe95a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f8074a70 con 0x7f67f81601c0 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fe95a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f815cc40 con 0x7f67f815c680 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fe95a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f67f815fd30 con 0x7f67f81565e0 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fcef7640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f67f81601c0 0x7f67f80b1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44526/0 (socket says 172.21.15.1:44526) 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67fcef7640 1 -- 172.21.15.1:0/42996724 learned_addr learned my addr 172.21.15.1:0/42996724 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67f7fff640 1 --2- 172.21.15.1:0/42996724 >> v2:172.21.15.6:3300/0 conn(0x7f67f81565e0 0x7f67f8074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671819909 0 0) 0x7f67f8074a70 con 0x7f67f81601c0 2024-05-09T06:33:53.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.726+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67e0003520 con 0x7f67f81601c0 2024-05-09T06:33:53.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.727+0000 7f67f7fff640 1 -- 172.21.15.1:0/42996724 >> v1:172.21.15.6:6789/0 conn(0x7f67f815c680 legacy=0x7f67f80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.727+0000 7f67f7fff640 1 -- 172.21.15.1:0/42996724 >> v1:172.21.15.1:6789/0 conn(0x7f67f81601c0 legacy=0x7f67f80b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.727+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1606597476 0 0) 0x7f67e0003520 con 0x7f67f81601c0 2024-05-09T06:33:53.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.727+0000 7f67f7fff640 1 -- 172.21.15.1:0/42996724 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f815cf20 con 0x7f67f81565e0 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.727+0000 7f67f7fff640 1 --2- 172.21.15.1:0/42996724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f8074530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67e80097a0 tx=0x7f67e802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947be8f05406cadf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8037070 con 0x7f67f81565e0 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e802ac50 con 0x7f67f81565e0 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67f6ffd640 1 -- 172.21.15.1:0/42996724 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00019f0 con 0x7f67f81565e0 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 -- 172.21.15.1:0/42996724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 msgr2=0x7f67f8074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 --2- 172.21.15.1:0/42996724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f8074530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67e80097a0 tx=0x7f67e802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 -- 172.21.15.1:0/42996724 shutdown_connections 2024-05-09T06:33:53.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 --2- 172.21.15.1:0/42996724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f8074530 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 -- 172.21.15.1:0/42996724 >> 172.21.15.1:0/42996724 conn(0x7f67f80b1fb0 msgr2=0x7f67f80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:53.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 -- 172.21.15.1:0/42996724 shutdown_connections 2024-05-09T06:33:53.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.728+0000 7f67fe95a640 1 -- 172.21.15.1:0/42996724 wait complete. 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.729+0000 7f67fe95a640 1 Processor -- start 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 -- start start 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67f815c680 0x7f67f814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81569c0 con 0x7f67f81601c0 2024-05-09T06:33:53.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f67f815d4e0 con 0x7f67f81565e0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.730+0000 7f67fe95a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f67f80b0340 con 0x7f67f815c680 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54616/0 (socket says 172.21.15.1:54616) 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 -- 172.21.15.1:0/3679335939 learned_addr learned my addr 172.21.15.1:0/3679335939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67f7fff640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67f77fe640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67f815c680 0x7f67f814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 -- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67f815c680 msgr2=0x7f67f814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67f815c680 0x7f67f814f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 -- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 msgr2=0x7f67f80b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f80b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 -- 172.21.15.1:0/3679335939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f8155290 con 0x7f67f81601c0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fcef7640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f67f000cc90 tx=0x7f67f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0017070 con 0x7f67f81601c0 2024-05-09T06:33:53.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f0012400 con 0x7f67f81601c0 2024-05-09T06:33:53.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f80b4170 con 0x7f67f81601c0 2024-05-09T06:33:53.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00127b0 con 0x7f67f81601c0 2024-05-09T06:33:53.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.731+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f80b4730 con 0x7f67f81601c0 2024-05-09T06:33:53.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.732+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f80752b0 con 0x7f67f81601c0 2024-05-09T06:33:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.733+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67f001a070 con 0x7f67f81601c0 2024-05-09T06:33:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.733+0000 7f67f4ff9640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 0x7f67e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.733+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f67f008ab70 con 0x7f67f81601c0 2024-05-09T06:33:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.733+0000 7f67f7fff640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 0x7f67e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.734+0000 7f67f7fff640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 0x7f67e406e320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67e8034000 tx=0x7f67e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:53.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.738+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67f8155290 con 0x7f67f81601c0 2024-05-09T06:33:53.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:53.991+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f67f80cd7e0 con 0x7f67f81601c0 2024-05-09T06:33:54.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.278+0000 7f67f4ff9640 1 -- 172.21.15.1:0/3679335939 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v323) ==== 166+0+0 (secure 0 0 0) 0x7f67f0058f70 con 0x7f67f81601c0 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 msgr2=0x7f67e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 0x7f67e406e320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67e8034000 tx=0x7f67e802e000 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 msgr2=0x7f67f8158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f67f000cc90 tx=0x7f67f000a430 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 shutdown_connections 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67e406be60 0x7f67e406e320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f81601c0 0x7f67f8158920 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67f815c680 0x7f67f814f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 --2- 172.21.15.1:0/3679335939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f81565e0 0x7f67f80b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 >> 172.21.15.1:0/3679335939 conn(0x7f67f80b1fb0 msgr2=0x7f67f806d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 shutdown_connections 2024-05-09T06:33:54.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.280+0000 7f67fe95a640 1 -- 172.21.15.1:0/3679335939 wait complete. 2024-05-09T06:33:54.273 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-09T06:33:54.273 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-09T06:33:54.273 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:33:54.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:54.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 Processor -- start 2024-05-09T06:33:54.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 -- start start 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ae40d4940 0x7f1ae40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae40b33c0 con 0x7f1ae40d0f80 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae40d4dd0 con 0x7f1ae40d4940 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.401+0000 7f1aea3ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae40b3660 con 0x7f1ae4161950 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ae40d4940 0x7f1ae40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae896b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1ae40d0f80 0x7f1ae40cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44538/0 (socket says 172.21.15.1:44538) 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae896b640 1 -- 172.21.15.1:0/602364910 learned_addr learned my addr 172.21.15.1:0/602364910 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1ae40d4940 0x7f1ae40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54622/0 (socket says 172.21.15.1:54622) 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae3fff640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 -- 172.21.15.1:0/602364910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 msgr2=0x7f1ae40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 -- 172.21.15.1:0/602364910 >> v1:172.21.15.1:6789/0 conn(0x7f1ae40d0f80 legacy=0x7f1ae40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae37fe640 1 -- 172.21.15.1:0/602364910 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae40b3940 con 0x7f1ae40d4940 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae2ffd640 1 -- 172.21.15.1:0/602364910 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985198810 0 0) 0x7f1ae40b33c0 con 0x7f1ae40d0f80 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.402+0000 7f1ae3fff640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1ae37fe640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae40d4940 0x7f1ae40d2940 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40b3590 tx=0x7f1ad802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a94b9f2ea3da19 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1ae2ffd640 1 -- 172.21.15.1:0/602364910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8037070 con 0x7f1ae40d4940 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1ae2ffd640 1 -- 172.21.15.1:0/602364910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad8032440 con 0x7f1ae40d4940 2024-05-09T06:33:54.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1ae2ffd640 1 -- 172.21.15.1:0/602364910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8032810 con 0x7f1ae40d4940 2024-05-09T06:33:54.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 -- 172.21.15.1:0/602364910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae40d4940 msgr2=0x7f1ae40d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae40d4940 0x7f1ae40d2940 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40b3590 tx=0x7f1ad802a180 comp rx=0 tx=0).stop 2024-05-09T06:33:54.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 -- 172.21.15.1:0/602364910 shutdown_connections 2024-05-09T06:33:54.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae40d4940 0x7f1ae40d2940 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/602364910 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.403+0000 7f1aea3ce640 1 -- 172.21.15.1:0/602364910 >> 172.21.15.1:0/602364910 conn(0x7f1ae40d3e20 msgr2=0x7f1ae40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.404+0000 7f1aea3ce640 1 -- 172.21.15.1:0/602364910 shutdown_connections 2024-05-09T06:33:54.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.404+0000 7f1aea3ce640 1 -- 172.21.15.1:0/602364910 wait complete. 2024-05-09T06:33:54.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.405+0000 7f1aea3ce640 1 Processor -- start 2024-05-09T06:33:54.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.405+0000 7f1aea3ce640 1 -- start start 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ae40d0f80 0x7f1ae415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae40b52d0 con 0x7f1ae41f7520 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae40b5150 con 0x7f1ae40d0f80 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1aea3ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae40b5450 con 0x7f1ae4161950 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54632/0 (socket says 172.21.15.1:54632) 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 -- 172.21.15.1:0/2836949770 learned_addr learned my addr 172.21.15.1:0/2836949770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae3fff640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ae40d0f80 0x7f1ae415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae37fe640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 -- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 msgr2=0x7f1ae415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae415b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 -- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ae40d0f80 msgr2=0x7f1ae415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ae40d0f80 0x7f1ae415d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 -- 172.21.15.1:0/2836949770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae40cc0c0 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae896b640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c7b0 tx=0x7f1adc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.406+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc018070 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.407+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc00a5e0 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.407+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae415dbe0 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.407+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00aa10 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.407+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae40cc380 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.408+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae40752b0 con 0x7f1ae41f7520 2024-05-09T06:33:54.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.408+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1adc01b070 con 0x7f1ae41f7520 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.408+0000 7f1ae0ff9640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.409+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f1adc08a310 con 0x7f1ae41f7520 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.409+0000 7f1ae3fff640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.409+0000 7f1ae3fff640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 0x7f1ad006e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400aac0 tx=0x7f1ad4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.413+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1adc0586f0 con 0x7f1ae41f7520 2024-05-09T06:33:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.596+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ae40caae0 con 0x7f1ae41f7520 2024-05-09T06:33:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.597+0000 7f1ae0ff9640 1 -- 172.21.15.1:0/2836949770 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+13288 (secure 0 0 0) 0x7f1adc05c9f0 con 0x7f1ae41f7520 2024-05-09T06:33:54.578 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:54.579 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:33:53.275918+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/3897834247":"2024-05-10T06:33:50.270812+0000","172.21.15.1:6834/3897834247":"2024-05-10T06:33:50.270812+0000","172.21.15.6:6834/2073487547":"2024-05-10T06:32:23.361954+0000","172.21.15.6:6832/2073487547":"2024-05-10T06:32:23.361954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 msgr2=0x7f1ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 0x7f1ad006e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400aac0 tx=0x7f1ad4008040 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 msgr2=0x7f1ae41f79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c7b0 tx=0x7f1adc00cc70 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 shutdown_connections 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ae41f7520 0x7f1ae41f79b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1ae4161950 0x7f1ae415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 --2- 172.21.15.1:0/2836949770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ae40d0f80 0x7f1ae415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.599+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 >> 172.21.15.1:0/2836949770 conn(0x7f1ae40d3e20 msgr2=0x7f1ae406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.600+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 shutdown_connections 2024-05-09T06:33:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.600+0000 7f1aea3ce640 1 -- 172.21.15.1:0/2836949770 wait complete. 2024-05-09T06:33:54.592 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:33:54.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:33:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.725+0000 7f43e2563640 1 Processor -- start 2024-05-09T06:33:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.725+0000 7f43e2563640 1 -- start start 2024-05-09T06:33:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43e2563640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f43dc15cfb0 0x7f43dc070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43e2563640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc06cf40 con 0x7f43dc15c4c0 2024-05-09T06:33:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43e2563640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc06d220 con 0x7f43dc06dd00 2024-05-09T06:33:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43e2563640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f43dc156980 con 0x7f43dc15cfb0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43dbfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f43dc15c4c0 0x7f43dc0b4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44554/0 (socket says 172.21.15.1:44554) 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.726+0000 7f43dbfff640 1 -- 172.21.15.1:0/1296853526 learned_addr learned my addr 172.21.15.1:0/1296853526 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43e0b00640 1 --2- 172.21.15.1:0/1296853526 >> v2:172.21.15.6:3300/0 conn(0x7f43dc15cfb0 0x7f43dc070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037842046 0 0) 0x7f43dc06cf40 con 0x7f43dc15c4c0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43c8003520 con 0x7f43dc15c4c0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4284504613 0 0) 0x7f43c8003520 con 0x7f43dc15c4c0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 >> v2:172.21.15.6:3300/0 conn(0x7f43dc15cfb0 msgr2=0x7f43dc070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 --2- 172.21.15.1:0/1296853526 >> v2:172.21.15.6:3300/0 conn(0x7f43dc15cfb0 0x7f43dc070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 >> v1:172.21.15.6:6789/0 conn(0x7f43dc06dd00 legacy=0x7f43dc15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc06d500 con 0x7f43dc15c4c0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f43cc001670 con 0x7f43dc15c4c0 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 >> v1:172.21.15.1:6789/0 conn(0x7f43dc15c4c0 legacy=0x7f43dc0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43c80039b0 0x7f43c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c80051c0 0x7f43c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43c800c130 con 0x7f43c8004800 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43c800c2e0 con 0x7f43c80051c0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f43c800c4e0 con 0x7f43c80039b0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43e0b00640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c80051c0 0x7f43c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43dbfff640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43c80039b0 0x7f43c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 -- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43c80039b0 msgr2=0x7f43c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43c80039b0 0x7f43c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 -- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c80051c0 msgr2=0x7f43c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43c80051c0 0x7f43c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43d3fff640 1 -- 172.21.15.1:0/1296853526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c8003520 con 0x7f43c8004800 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.727+0000 7f43e0b00640 1 -- 172.21.15.1:0/1296853526 reap_dead start 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.728+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43c8004c80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001750 tx=0x7f43c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23b0b745bf0800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.728+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4001370 con 0x7f43c8004800 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.728+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c40052d0 con 0x7f43c8004800 2024-05-09T06:33:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.728+0000 7f43db7fe640 1 -- 172.21.15.1:0/1296853526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c402abc0 con 0x7f43c8004800 2024-05-09T06:33:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 -- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 msgr2=0x7f43c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43c8004c80 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001750 tx=0x7f43c8005830 comp rx=0 tx=0).stop 2024-05-09T06:33:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 -- 172.21.15.1:0/1296853526 shutdown_connections 2024-05-09T06:33:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 --2- 172.21.15.1:0/1296853526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43c8004c80 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 -- 172.21.15.1:0/1296853526 >> 172.21.15.1:0/1296853526 conn(0x7f43dc0cc4d0 msgr2=0x7f43dc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 -- 172.21.15.1:0/1296853526 shutdown_connections 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.729+0000 7f43e2563640 1 -- 172.21.15.1:0/1296853526 wait complete. 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.730+0000 7f43e2563640 1 Processor -- start 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.730+0000 7f43e2563640 1 -- start start 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43dc141810 0x7f43dc1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43dc141c90 0x7f43dc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc06d870 con 0x7f43c8004800 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc06e180 con 0x7f43dc141c90 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e2563640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc15c8a0 con 0x7f43dc141810 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54654/0 (socket says 172.21.15.1:54654) 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 -- 172.21.15.1:0/1527751192 learned_addr learned my addr 172.21.15.1:0/1527751192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 -- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43dc141810 msgr2=0x7f43dc1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43dc141810 0x7f43dc1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43e0b00640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43dc141c90 0x7f43dc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43dc141810 0x7f43dc1f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 -- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43dc141c90 msgr2=0x7f43dc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43dc141c90 0x7f43dc15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.731+0000 7f43dbfff640 1 -- 172.21.15.1:0/1527751192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc0a9600 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43dbfff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0072c0 tx=0x7f43cc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00f960 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc018070 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00fd80 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc1f7960 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.732+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc1f7f20 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.733+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43dc0752b0 con 0x7f43c8004800 2024-05-09T06:33:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.733+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43cc01d460 con 0x7f43c8004800 2024-05-09T06:33:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.734+0000 7f43d97fa640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 0x7f43bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.734+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f43cc08ad40 con 0x7f43c8004800 2024-05-09T06:33:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.734+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 0x7f43bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.734+0000 7f43d3fff640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 0x7f43bc06e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f43c4032000 tx=0x7f43c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.738+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f43cc059800 con 0x7f43c8004800 2024-05-09T06:33:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.920+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43dc1f80f0 con 0x7f43c8004800 2024-05-09T06:33:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.920+0000 7f43d97fa640 1 -- 172.21.15.1:0/1527751192 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f43cc059100 con 0x7f43c8004800 2024-05-09T06:33:54.901 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:33:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 msgr2=0x7f43bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 0x7f43bc06e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f43c4032000 tx=0x7f43c402e000 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 msgr2=0x7f43dc1f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0072c0 tx=0x7f43cc007780 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 shutdown_connections 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f43bc06be60 0x7f43bc06e320 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f43dc141c90 0x7f43dc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f43dc141810 0x7f43dc1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 --2- 172.21.15.1:0/1527751192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c8004800 0x7f43dc1f7260 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 >> 172.21.15.1:0/1527751192 conn(0x7f43dc0cc4d0 msgr2=0x7f43dc0af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 shutdown_connections 2024-05-09T06:33:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:54.922+0000 7f43e2563640 1 -- 172.21.15.1:0/1527751192 wait complete. 2024-05-09T06:33:54.915 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-09T06:33:55.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.037+0000 7f084727d640 1 Processor -- start 2024-05-09T06:33:55.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.037+0000 7f084727d640 1 -- start start 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f084727d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08400cf180 0x7f08400cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f084727d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08400ce0e0 con 0x7f084006e6d0 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f084727d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08400ce3c0 con 0x7f08400cdab0 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f084727d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f08400cdec0 con 0x7f08400cf180 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f0844818640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08400cf180 0x7f08400cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f0844818640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f08400cf180 0x7f08400cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54664/0 (socket says 172.21.15.1:54664) 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f0844818640 1 -- 172.21.15.1:0/215443391 learned_addr learned my addr 172.21.15.1:0/215443391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.038+0000 7f0845019640 1 --1- 172.21.15.1:0/215443391 >> v1:172.21.15.1:6789/0 conn(0x7f084006e6d0 0x7f084006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44570/0 (socket says 172.21.15.1:44570) 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.039+0000 7f0844818640 1 -- 172.21.15.1:0/215443391 >> v1:172.21.15.6:6789/0 conn(0x7f08400cdab0 legacy=0x7f08400b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.039+0000 7f0844818640 1 -- 172.21.15.1:0/215443391 >> v1:172.21.15.1:6789/0 conn(0x7f084006e6d0 legacy=0x7f084006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.039+0000 7f083ffff640 1 -- 172.21.15.1:0/215443391 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841742628 0 0) 0x7f08400ce0e0 con 0x7f084006e6d0 2024-05-09T06:33:55.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.039+0000 7f0844818640 1 -- 172.21.15.1:0/215443391 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08400ce6a0 con 0x7f08400cf180 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.039+0000 7f0844818640 1 --2- 172.21.15.1:0/215443391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cf180 0x7f08400cd570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0834009870 tx=0x7f083402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765f6b408b78c3cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f083ffff640 1 -- 172.21.15.1:0/215443391 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834037070 con 0x7f08400cf180 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f083ffff640 1 -- 172.21.15.1:0/215443391 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0834032440 con 0x7f08400cf180 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f083ffff640 1 -- 172.21.15.1:0/215443391 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834032810 con 0x7f08400cf180 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 -- 172.21.15.1:0/215443391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cf180 msgr2=0x7f08400cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 --2- 172.21.15.1:0/215443391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cf180 0x7f08400cd570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0834009870 tx=0x7f083402a2a0 comp rx=0 tx=0).stop 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 -- 172.21.15.1:0/215443391 shutdown_connections 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 --2- 172.21.15.1:0/215443391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cf180 0x7f08400cd570 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 -- 172.21.15.1:0/215443391 >> 172.21.15.1:0/215443391 conn(0x7f08400cc4b0 msgr2=0x7f08400cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 -- 172.21.15.1:0/215443391 shutdown_connections 2024-05-09T06:33:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.040+0000 7f084727d640 1 -- 172.21.15.1:0/215443391 wait complete. 2024-05-09T06:33:55.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.041+0000 7f084727d640 1 Processor -- start 2024-05-09T06:33:55.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.041+0000 7f084727d640 1 -- start start 2024-05-09T06:33:55.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f084006e6d0 0x7f08401e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 0x7f084006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0840070ed0 con 0x7f08400cdab0 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f08400cec60 con 0x7f08400cf180 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f084727d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f08401447c0 con 0x7f084006e6d0 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f0844818640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f0844818640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54680/0 (socket says 172.21.15.1:54680) 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.042+0000 7f0844818640 1 -- 172.21.15.1:0/3349402281 learned_addr learned my addr 172.21.15.1:0/3349402281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f084581a640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 0x7f084006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0845019640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f084006e6d0 0x7f08401e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 -- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f084006e6d0 msgr2=0x7f08401e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f084006e6d0 0x7f08401e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 -- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 msgr2=0x7f084006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 0x7f084006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:55.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 -- 172.21.15.1:0/3349402281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08400aad40 con 0x7f08400cdab0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f084581a640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 0x7f084006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f0844818640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f08340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834041070 con 0x7f08400cdab0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083403c870 con 0x7f08400cdab0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08400aafb0 con 0x7f08400cdab0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834037040 con 0x7f08400cdab0 2024-05-09T06:33:55.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.043+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08400ab570 con 0x7f08400cdab0 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.044+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08400752b0 con 0x7f08400cdab0 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.045+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0834044070 con 0x7f08400cdab0 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.045+0000 7f083dffb640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 0x7f082c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.045+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f08340ab850 con 0x7f08400cdab0 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.045+0000 7f0845019640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 0x7f082c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:55.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.046+0000 7f0845019640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 0x7f082c06e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0838000f60 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:55.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.050+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f083407a440 con 0x7f08400cdab0 2024-05-09T06:33:55.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:55.304+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f084006cf50 con 0x7f08400cdab0 2024-05-09T06:33:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.284+0000 7f083dffb640 1 -- 172.21.15.1:0/3349402281 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v325) ==== 170+0+0 (secure 0 0 0) 0x7f0834079d40 con 0x7f08400cdab0 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 msgr2=0x7f082c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 0x7f082c06e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0838000f60 tx=0x7f0838009040 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 msgr2=0x7f084015a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f08340046c0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 shutdown_connections 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f082c06be30 0x7f082c06e2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08400cf180 0x7f084006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08400cdab0 0x7f084015a7e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 --2- 172.21.15.1:0/3349402281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f084006e6d0 0x7f08401e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 >> 172.21.15.1:0/3349402281 conn(0x7f08400cc4b0 msgr2=0x7f08400cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 shutdown_connections 2024-05-09T06:33:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.287+0000 7f084727d640 1 -- 172.21.15.1:0/3349402281 wait complete. 2024-05-09T06:33:56.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:33:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.413+0000 7fd57c190640 1 Processor -- start 2024-05-09T06:33:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.414+0000 7fd57c190640 1 -- start start 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57c190640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd5740718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57c190640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57415b420 con 0x7fd57415c4a0 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57c190640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57415b6e0 con 0x7fd57406ede0 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57c190640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd574158e30 con 0x7fd574158a50 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57972b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd57415c4a0 0x7fd57406e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44576/0 (socket says 172.21.15.1:44576) 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd57972b640 1 -- 172.21.15.1:0/3663847388 learned_addr learned my addr 172.21.15.1:0/3663847388 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd579f2c640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd5740718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270274909 0 0) 0x7fd57415b420 con 0x7fd57415c4a0 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd57415c4a0 2024-05-09T06:33:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1381880886 0 0) 0x7fd55c003520 con 0x7fd57415c4a0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 msgr2=0x7fd5740718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd5740718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 >> v1:172.21.15.6:6789/0 conn(0x7fd57406ede0 legacy=0x7fd57415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57415b9c0 con 0x7fd57415c4a0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fd5680019f0 con 0x7fd57415c4a0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 >> v1:172.21.15.1:6789/0 conn(0x7fd57415c4a0 legacy=0x7fd57406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd55c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd55c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd574158e30 con 0x7fd55c003f60 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c00baa0 con 0x7fd55c004d40 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.415+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c00bca0 con 0x7fd574158a50 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd55c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd579f2c640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd55c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57a72d640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 -- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 msgr2=0x7fd55c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd574158a50 0x7fd55c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 -- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd55c004d40 msgr2=0x7fd55c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 -- 172.21.15.1:0/3663847388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c003520 con 0x7fd55c003f60 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.416+0000 7fd57972b640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd55c004800 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd57415c8a0 tx=0x7fd55c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d1a6926cd8cd82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56800a4a0 con 0x7fd55c003f60 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56802f070 con 0x7fd55c003f60 2024-05-09T06:33:56.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd578f2a640 1 -- 172.21.15.1:0/3663847388 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56802bcf0 con 0x7fd55c003f60 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 -- 172.21.15.1:0/3663847388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 msgr2=0x7fd55c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 --2- 172.21.15.1:0/3663847388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd55c004800 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd57415c8a0 tx=0x7fd55c005830 comp rx=0 tx=0).stop 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57a72d640 1 -- 172.21.15.1:0/3663847388 reap_dead start 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 -- 172.21.15.1:0/3663847388 shutdown_connections 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 -- 172.21.15.1:0/3663847388 >> 172.21.15.1:0/3663847388 conn(0x7fd5740cc4d0 msgr2=0x7fd5740cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 -- 172.21.15.1:0/3663847388 shutdown_connections 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.417+0000 7fd57c190640 1 -- 172.21.15.1:0/3663847388 wait complete. 2024-05-09T06:33:56.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 Processor -- start 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 -- start start 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 0x7fd57414c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5741f4ef0 0x7fd57414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd57415bf80 con 0x7fd55c003f60 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd574072480 con 0x7fd5741f4ef0 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd57c190640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd574070ea0 con 0x7fd5741f4a70 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd579f2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd579f2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54690/0 (socket says 172.21.15.1:54690) 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.419+0000 7fd579f2c640 1 -- 172.21.15.1:0/1608660862 learned_addr learned my addr 172.21.15.1:0/1608660862 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd57a72d640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5741f4ef0 0x7fd57414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd57972b640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 0x7fd57414c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 -- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 msgr2=0x7fd57414c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 0x7fd57414c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 -- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5741f4ef0 msgr2=0x7fd57414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5741f4ef0 0x7fd57414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 -- 172.21.15.1:0/1608660862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741f54c0 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd57972b640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 0x7fd57414c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd579f2c640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0119a0 tx=0x7fd56c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00faf0 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c00fc90 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741f57a0 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c014440 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.420+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5740aefd0 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.421+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5740752b0 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.422+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd56c012040 con 0x7fd55c003f60 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.422+0000 7fd5727fc640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 0x7fd56006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.422+0000 7fd57972b640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 0x7fd56006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.422+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7fd56c08b410 con 0x7fd55c003f60 2024-05-09T06:33:56.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.423+0000 7fd57972b640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 0x7fd56006e320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd568033000 tx=0x7fd568009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.426+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd56c019030 con 0x7fd55c003f60 2024-05-09T06:33:56.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.608+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd5741f5970 con 0x7fd55c003f60 2024-05-09T06:33:56.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.611+0000 7fd5727fc640 1 -- 172.21.15.1:0/1608660862 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fd56c059890 con 0x7fd55c003f60 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 msgr2=0x7fd56006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 0x7fd56006e320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd568033000 tx=0x7fd568009030 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 msgr2=0x7fd5740712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0119a0 tx=0x7fd56c011e60 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 shutdown_connections 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd56006be60 0x7fd56006e320 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5741f4ef0 0x7fd57414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5741f4a70 0x7fd57414c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.613+0000 7fd57c190640 1 --2- 172.21.15.1:0/1608660862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd55c003f60 0x7fd5740712f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.614+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 >> 172.21.15.1:0/1608660862 conn(0x7fd5740cc4d0 msgr2=0x7fd5740b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:56.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.614+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 shutdown_connections 2024-05-09T06:33:56.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.614+0000 7fd57c190640 1 -- 172.21.15.1:0/1608660862 wait complete. 2024-05-09T06:33:56.606 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:33:56.606 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:33:56.652 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:33:56.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:33:56.706 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:33:56.706 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:56.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:33:56.722 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:33:56.722 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:56.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:33:56.741 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:33:56.741 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:56.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:33:56.778 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:33:56.778 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:33:56.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:33:56.799 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:33:56.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:33:56.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.940+0000 7f3d5bfa9640 1 Processor -- start 2024-05-09T06:33:56.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.940+0000 7f3d5bfa9640 1 -- start start 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5bfa9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5bfa9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3d5415e4b0 0x7f3d5415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5bfa9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d54099930 con 0x7f3d540b2aa0 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5bfa9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3d540b1e20 con 0x7f3d5415e4b0 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5bfa9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5415ee90 con 0x7f3d540c6860 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5a546640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3d5415e4b0 0x7f3d5415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d59544640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3d540b2aa0 0x7f3d540c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44592/0 (socket says 172.21.15.1:44592) 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d59544640 1 -- 172.21.15.1:0/442155063 learned_addr learned my addr 172.21.15.1:0/442155063 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d59d45640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5a546640 1 -- 172.21.15.1:0/442155063 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 msgr2=0x7f3d54099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5a546640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5a546640 1 -- 172.21.15.1:0/442155063 >> v1:172.21.15.1:6789/0 conn(0x7f3d540b2aa0 legacy=0x7f3d540c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d58d43640 1 -- 172.21.15.1:0/442155063 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947266689 0 0) 0x7f3d44001110 con 0x7f3d540b2aa0 2024-05-09T06:33:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.941+0000 7f3d5a546640 1 -- 172.21.15.1:0/442155063 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d54162930 con 0x7f3d5415e4b0 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.942+0000 7f3d5a546640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d5415f320 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009d90 tx=0x7f3d4c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedd266cb69a6803 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.942+0000 7f3d58d43640 1 -- 172.21.15.1:0/442155063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c007740 con 0x7f3d5415e4b0 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.942+0000 7f3d58d43640 1 -- 172.21.15.1:0/442155063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0078e0 con 0x7f3d5415e4b0 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.942+0000 7f3d58d43640 1 -- 172.21.15.1:0/442155063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c007cb0 con 0x7f3d5415e4b0 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/442155063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 msgr2=0x7f3d5415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d5415f320 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009d90 tx=0x7f3d4c02a660 comp rx=0 tx=0).stop 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/442155063 shutdown_connections 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d5415f320 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/442155063 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/442155063 >> 172.21.15.1:0/442155063 conn(0x7f3d540b1fb0 msgr2=0x7f3d540b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/442155063 shutdown_connections 2024-05-09T06:33:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.943+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/442155063 wait complete. 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.944+0000 7f3d5bfa9640 1 Processor -- start 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.944+0000 7f3d5bfa9640 1 -- start start 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 0x7f3d540cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5415f860 con 0x7f3d5415e4b0 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5415eb20 con 0x7f3d540b2aa0 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5bfa9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5415f9e0 con 0x7f3d540c6860 2024-05-09T06:33:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54710/0 (socket says 172.21.15.1:54710) 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 -- 172.21.15.1:0/2753561506 learned_addr learned my addr 172.21.15.1:0/2753561506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d59d45640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 0x7f3d540cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d59544640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 -- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 msgr2=0x7f3d54157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 -- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 msgr2=0x7f3d540cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 0x7f3d540cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d5a546640 1 -- 172.21.15.1:0/2753561506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541f4280 con 0x7f3d5415e4b0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.945+0000 7f3d59d45640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 0x7f3d540cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d5a546640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009eb0 tx=0x7f3d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c03f070 con 0x7f3d5415e4b0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c031bc0 con 0x7f3d5415e4b0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c044460 con 0x7f3d5415e4b0 2024-05-09T06:33:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d541f4560 con 0x7f3d5415e4b0 2024-05-09T06:33:56.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.946+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d541f4b20 con 0x7f3d5415e4b0 2024-05-09T06:33:56.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.947+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540752b0 con 0x7f3d5415e4b0 2024-05-09T06:33:56.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.947+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d4c007790 con 0x7f3d5415e4b0 2024-05-09T06:33:56.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.948+0000 7f3d4a7fc640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 0x7f3d3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.948+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f3d4c0ac5b0 con 0x7f3d5415e4b0 2024-05-09T06:33:56.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.948+0000 7f3d59d45640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 0x7f3d3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.948+0000 7f3d59d45640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 0x7f3d3806e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:56.952+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d4c07b110 con 0x7f3d5415e4b0 2024-05-09T06:33:57.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.175+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d5406ce30 con 0x7f3d5415e4b0 2024-05-09T06:33:57.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.175+0000 7f3d4a7fc640 1 -- 172.21.15.1:0/2753561506 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3854 (secure 0 0 0) 0x7f3d4c07aa10 con 0x7f3d5415e4b0 2024-05-09T06:33:57.157 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:57.157 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/369833341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":369833341},{"type":"v1","addr":"172.21.15.1:6837","nonce":369833341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1009049848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1009049848},{"type":"v1","addr":"172.21.15.6:6835","nonce":1009049848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3804175791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3804175791},{"type":"v1","addr":"172.21.15.6:6834","nonce":3804175791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/3022178544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3022178544},{"type":"v1","addr":"172.21.15.1:6836","nonce":3022178544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-05-09T06:33:57.157 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 55 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 msgr2=0x7f3d3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 0x7f3d3806e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 msgr2=0x7f3d54157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009eb0 tx=0x7f3d4c004a40 comp rx=0 tx=0).stop 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 shutdown_connections 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3d3806be30 0x7f3d3806e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d5415e4b0 0x7f3d54157d60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d540c6860 0x7f3d54157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 --2- 172.21.15.1:0/2753561506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d540b2aa0 0x7f3d540cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 >> 172.21.15.1:0/2753561506 conn(0x7f3d540b1fb0 msgr2=0x7f3d5406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 shutdown_connections 2024-05-09T06:33:57.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.178+0000 7f3d5bfa9640 1 -- 172.21.15.1:0/2753561506 wait complete. 2024-05-09T06:33:57.170 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:33:57.170 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:33:57.171 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:33:57.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:33:57.171 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:33:57.171 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:33:57.171 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:33:57.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:33:57.171 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:33:57.171 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:33:57.172 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:33:57.172 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:33:57.172 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:33:57.172 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:33:57.172 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:33:57.172 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:33:57.180 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:33:57.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:33:57.181 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:33:57.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:33:57.184 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:33:57.184 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:33:57.184 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:33:57.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:33:57.223 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:33:57.224 DEBUG:teuthology.parallel:result is None 2024-05-09T06:33:57.224 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:33:57.224 DEBUG:teuthology.parallel:result is None 2024-05-09T06:33:57.227 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:33:57.227 DEBUG:teuthology.parallel:result is None 2024-05-09T06:33:57.227 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:33:57.228 DEBUG:teuthology.parallel:result is None 2024-05-09T06:33:57.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:33:57.303 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:33:57.303 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:33:57.312 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:33:57.312 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:33:57.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.358+0000 7f95001af640 1 Processor -- start 2024-05-09T06:33:57.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.358+0000 7f95001af640 1 -- start start 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94f8161950 0x7f94f80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f80d4940 0x7f94f80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 0x7f94f80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f94f80d4dd0 con 0x7f94f8161950 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f94f80b33c0 con 0x7f94f80d0f80 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f95001af640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94f80b3660 con 0x7f94f80d4940 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94f8161950 0x7f94f80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94f8161950 0x7f94f80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54754/0 (socket says 172.21.15.1:54754) 2024-05-09T06:33:57.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 -- 172.21.15.1:0/950466517 learned_addr learned my addr 172.21.15.1:0/950466517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fd74a640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f80d4940 0x7f94f80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fe74c640 1 --2- 172.21.15.1:0/950466517 >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 0x7f94f80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 -- 172.21.15.1:0/950466517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f80d4940 msgr2=0x7f94f80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f80d4940 0x7f94f80d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 -- 172.21.15.1:0/950466517 >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 msgr2=0x7f94f80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/950466517 >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 0x7f94f80cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fdf4b640 1 -- 172.21.15.1:0/950466517 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f80b3940 con 0x7f94f8161950 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.359+0000 7f94fe74c640 1 --2- 172.21.15.1:0/950466517 >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 0x7f94f80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.360+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f8161950 0x7f94f80d2400 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f94f808c710 tx=0x7f94f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a4b79ec3957fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.360+0000 7f94fcf49640 1 -- 172.21.15.1:0/950466517 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4037070 con 0x7f94f8161950 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.360+0000 7f94fcf49640 1 -- 172.21.15.1:0/950466517 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f4032440 con 0x7f94f8161950 2024-05-09T06:33:57.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.360+0000 7f94fcf49640 1 -- 172.21.15.1:0/950466517 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4032810 con 0x7f94f8161950 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 -- 172.21.15.1:0/950466517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f8161950 msgr2=0x7f94f80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f8161950 0x7f94f80d2400 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f94f808c710 tx=0x7f94f402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 -- 172.21.15.1:0/950466517 shutdown_connections 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 --2- 172.21.15.1:0/950466517 >> v2:172.21.15.6:3300/0 conn(0x7f94f80d0f80 0x7f94f80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f80d4940 0x7f94f80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 --2- 172.21.15.1:0/950466517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f8161950 0x7f94f80d2400 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 -- 172.21.15.1:0/950466517 >> 172.21.15.1:0/950466517 conn(0x7f94f80d3e20 msgr2=0x7f94f80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 -- 172.21.15.1:0/950466517 shutdown_connections 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.361+0000 7f95001af640 1 -- 172.21.15.1:0/950466517 wait complete. 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.362+0000 7f95001af640 1 Processor -- start 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.362+0000 7f95001af640 1 -- start start 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 0x7f94f815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f8161950 0x7f94f80b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94f80d1d80 con 0x7f94f80cc950 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94f80d1c00 con 0x7f94f80d0f80 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f95001af640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94f80d1f00 con 0x7f94f8161950 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54764/0 (socket says 172.21.15.1:54764) 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 -- 172.21.15.1:0/1077430094 learned_addr learned my addr 172.21.15.1:0/1077430094 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 0x7f94f815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fd74a640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f8161950 0x7f94f80b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 -- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f8161950 msgr2=0x7f94f80b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f8161950 0x7f94f80b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 -- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 msgr2=0x7f94f815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 0x7f94f815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 -- 172.21.15.1:0/1077430094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f80ccea0 con 0x7f94f80cc950 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 0x7f94f815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.363+0000 7f94fe74c640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f70 tx=0x7f94f000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.364+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f001a070 con 0x7f94f80cc950 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.364+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f0003bb0 con 0x7f94f80cc950 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.364+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004030 con 0x7f94f80cc950 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.364+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f80cd180 con 0x7f94f80cc950 2024-05-09T06:33:57.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.364+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f80cd740 con 0x7f94f80cc950 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.365+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f80752b0 con 0x7f94f80cc950 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.365+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94f001d070 con 0x7f94f80cc950 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.366+0000 7f94ee7fc640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 0x7f94dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.366+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5421+0+0 (secure 0 0 0) 0x7f94f008b070 con 0x7f94f80cc950 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.366+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 0x7f94dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:57.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.366+0000 7f94fdf4b640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 0x7f94dc06e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94f4034000 tx=0x7f94f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:57.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.370+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94f0010070 con 0x7f94f80cc950 2024-05-09T06:33:57.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:57.565+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f94f80d21e0 con 0x7f94f80cc950 2024-05-09T06:33:58.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.625+0000 7f94ee7fc640 1 -- 172.21.15.1:0/1077430094 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v87) ==== 97+0+0 (secure 0 0 0) 0x7f94f0059aa0 con 0x7f94f80cc950 2024-05-09T06:33:58.606 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.628+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 msgr2=0x7f94dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.628+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 0x7f94dc06e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94f4034000 tx=0x7f94f402e000 comp rx=0 tx=0).stop 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.628+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 msgr2=0x7f94f809daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.628+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f70 tx=0x7f94f000d560 comp rx=0 tx=0).stop 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.628+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 shutdown_connections 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94dc06be60 0x7f94dc06e320 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94f80cc950 0x7f94f809daa0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94f8161950 0x7f94f80b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 --2- 172.21.15.1:0/1077430094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f80d0f80 0x7f94f815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 >> 172.21.15.1:0/1077430094 conn(0x7f94f80d3e20 msgr2=0x7f94f80b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 shutdown_connections 2024-05-09T06:33:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.629+0000 7f95001af640 1 -- 172.21.15.1:0/1077430094 wait complete. 2024-05-09T06:33:58.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:33:58.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.753+0000 7f24d2ba9640 1 Processor -- start 2024-05-09T06:33:58.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.753+0000 7f24d2ba9640 1 -- start start 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d2ba9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 0x7f24cc0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d2ba9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f24cc0d2b10 0x7f24cc0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d2ba9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24cc0b33c0 con 0x7f24cc161950 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d2ba9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f24cc0d4dd0 con 0x7f24cc0d2b10 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d2ba9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f24cc0b3660 con 0x7f24cc0d4940 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f24cc0d2b10 0x7f24cc0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f24cc0d2b10 0x7f24cc0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37968/0 (socket says 172.21.15.1:37968) 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 -- 172.21.15.1:0/186606353 learned_addr learned my addr 172.21.15.1:0/186606353 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:58.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24cbfff640 1 --2- 172.21.15.1:0/186606353 >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 0x7f24cc0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 -- 172.21.15.1:0/186606353 >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 msgr2=0x7f24cc0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 --2- 172.21.15.1:0/186606353 >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 0x7f24cc0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 -- 172.21.15.1:0/186606353 >> v1:172.21.15.6:6789/0 conn(0x7f24cc161950 legacy=0x7f24cc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24d1146640 1 -- 172.21.15.1:0/186606353 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc0b3940 con 0x7f24cc0d2b10 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.754+0000 7f24cbfff640 1 --2- 172.21.15.1:0/186606353 >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 0x7f24cc0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.755+0000 7f24d1146640 1 --2- 172.21.15.1:0/186606353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0b4680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f24cc06e3d0 tx=0x7f24c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1ad385d8027e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.755+0000 7f24cb7fe640 1 -- 172.21.15.1:0/186606353 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4036070 con 0x7f24cc0d2b10 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.755+0000 7f24cb7fe640 1 -- 172.21.15.1:0/186606353 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c4031600 con 0x7f24cc0d2b10 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.755+0000 7f24cb7fe640 1 -- 172.21.15.1:0/186606353 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40319d0 con 0x7f24cc0d2b10 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 -- 172.21.15.1:0/186606353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc0d2b10 msgr2=0x7f24cc0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/186606353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0b4680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f24cc06e3d0 tx=0x7f24c402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 -- 172.21.15.1:0/186606353 shutdown_connections 2024-05-09T06:33:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/186606353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0b4680 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/186606353 >> v2:172.21.15.6:3300/0 conn(0x7f24cc0d4940 0x7f24cc0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 -- 172.21.15.1:0/186606353 >> 172.21.15.1:0/186606353 conn(0x7f24cc0d3e20 msgr2=0x7f24cc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 -- 172.21.15.1:0/186606353 shutdown_connections 2024-05-09T06:33:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.756+0000 7f24d2ba9640 1 -- 172.21.15.1:0/186606353 wait complete. 2024-05-09T06:33:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.757+0000 7f24d2ba9640 1 Processor -- start 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.757+0000 7f24d2ba9640 1 -- start start 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24cc161950 0x7f24cc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc0d1100 con 0x7f24cc152e10 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc0d0f80 con 0x7f24cc0d2b10 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d2ba9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc0d1c00 con 0x7f24cc161950 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37974/0 (socket says 172.21.15.1:37974) 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 -- 172.21.15.1:0/2085194729 learned_addr learned my addr 172.21.15.1:0/2085194729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d0945640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24cbfff640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24cc161950 0x7f24cc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 -- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24cc161950 msgr2=0x7f24cc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24cc161950 0x7f24cc0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 -- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24cc0d2b10 msgr2=0x7f24cc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.758+0000 7f24d1146640 1 -- 172.21.15.1:0/2085194729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc1537e0 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24d1146640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f24c402ace0 tx=0x7f24c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4036050 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c4042070 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4009510 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc0cb270 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.759+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc0cb830 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.760+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc0752b0 con 0x7f24cc152e10 2024-05-09T06:33:58.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.760+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24c4007bd0 con 0x7f24cc152e10 2024-05-09T06:33:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.761+0000 7f24c97fa640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 0x7f24b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.761+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f24c40abc50 con 0x7f24cc152e10 2024-05-09T06:33:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.761+0000 7f24d0945640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 0x7f24b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.762+0000 7f24d0945640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 0x7f24b006e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f24c0006fd0 tx=0x7f24c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:58.766+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24c40b1020 con 0x7f24cc152e10 2024-05-09T06:33:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.019+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f24cc0caab0 con 0x7f24cc152e10 2024-05-09T06:33:59.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.020+0000 7f24c97fa640 1 -- 172.21.15.1:0/2085194729 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f24c407a7c0 con 0x7f24cc152e10 2024-05-09T06:33:59.031 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:59.031 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:33:59.031 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:33:59.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.031 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:33:59.032 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:33:59.034 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:33:59.035 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.037 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.038 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.039 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.040 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.041 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:33:59.042 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:33:59.044 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:33:59.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:33:59.045 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 msgr2=0x7f24b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 0x7f24b006e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f24c0006fd0 tx=0x7f24c0008040 comp rx=0 tx=0).stop 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 msgr2=0x7f24cc1532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f24c402ace0 tx=0x7f24c4004a40 comp rx=0 tx=0).stop 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 shutdown_connections 2024-05-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f24b006be30 0x7f24b006e2f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24cc152e10 0x7f24cc1532a0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24cc161950 0x7f24cc0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 --2- 172.21.15.1:0/2085194729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24cc0d2b10 0x7f24cc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 >> 172.21.15.1:0/2085194729 conn(0x7f24cc0d3e20 msgr2=0x7f24cc0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 shutdown_connections 2024-05-09T06:33:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.022+0000 7f24d2ba9640 1 -- 172.21.15.1:0/2085194729 wait complete. 2024-05-09T06:33:59.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:33:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.170+0000 7f67a7c12640 1 Processor -- start 2024-05-09T06:33:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.170+0000 7f67a7c12640 1 -- start start 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 0x7f67a006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f67a00cf180 0x7f67a00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 0x7f67a015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f67a00ce0e0 con 0x7f67a00cf180 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f67a00ce3c0 con 0x7f67a006e6d0 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a7c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f67a00ce6a0 con 0x7f67a00cdab0 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f67a00cf180 0x7f67a00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f67a00cf180 0x7f67a00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37986/0 (socket says 172.21.15.1:37986) 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 -- 172.21.15.1:0/1472723835 learned_addr learned my addr 172.21.15.1:0/1472723835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a61af640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 0x7f67a015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a59ae640 1 --2- 172.21.15.1:0/1472723835 >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 0x7f67a006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 -- 172.21.15.1:0/1472723835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 msgr2=0x7f67a015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 0x7f67a015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 -- 172.21.15.1:0/1472723835 >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 msgr2=0x7f67a006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 --2- 172.21.15.1:0/1472723835 >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 0x7f67a006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a51ad640 1 -- 172.21.15.1:0/1472723835 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a00ce980 con 0x7f67a00cf180 2024-05-09T06:33:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a61af640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 0x7f67a015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.171+0000 7f67a59ae640 1 --2- 172.21.15.1:0/1472723835 >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 0x7f67a006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.172+0000 7f67a51ad640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a00cf180 0x7f67a00cd570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f67a0074650 tx=0x7f679002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4e3cfe75d53503 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.172+0000 7f67a49ac640 1 -- 172.21.15.1:0/1472723835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790037070 con 0x7f67a00cf180 2024-05-09T06:33:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.172+0000 7f67a49ac640 1 -- 172.21.15.1:0/1472723835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6790032440 con 0x7f67a00cf180 2024-05-09T06:33:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.172+0000 7f67a49ac640 1 -- 172.21.15.1:0/1472723835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790032810 con 0x7f67a00cf180 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 -- 172.21.15.1:0/1472723835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a00cf180 msgr2=0x7f67a00cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a00cf180 0x7f67a00cd570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f67a0074650 tx=0x7f679002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 -- 172.21.15.1:0/1472723835 shutdown_connections 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cdab0 0x7f67a015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 --2- 172.21.15.1:0/1472723835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a00cf180 0x7f67a00cd570 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 --2- 172.21.15.1:0/1472723835 >> v2:172.21.15.6:3300/0 conn(0x7f67a006e6d0 0x7f67a006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 -- 172.21.15.1:0/1472723835 >> 172.21.15.1:0/1472723835 conn(0x7f67a00cc4b0 msgr2=0x7f67a00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 -- 172.21.15.1:0/1472723835 shutdown_connections 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.173+0000 7f67a7c12640 1 -- 172.21.15.1:0/1472723835 wait complete. 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.174+0000 7f67a7c12640 1 Processor -- start 2024-05-09T06:33:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.174+0000 7f67a7c12640 1 -- start start 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 0x7f67a00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cf180 0x7f67a013a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0070ed0 con 0x7f67a006e6d0 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f67a006eff0 con 0x7f67a00cdab0 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a7c12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f67a01447c0 con 0x7f67a00cf180 2024-05-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37988/0 (socket says 172.21.15.1:37988) 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 -- 172.21.15.1:0/2825762115 learned_addr learned my addr 172.21.15.1:0/2825762115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a51ad640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 0x7f67a00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 -- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cf180 msgr2=0x7f67a013a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a61af640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cf180 0x7f67a013a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cf180 0x7f67a013a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 -- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 msgr2=0x7f67a00ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 0x7f67a00ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a59ae640 1 -- 172.21.15.1:0/2825762115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a0137460 con 0x7f67a006e6d0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.175+0000 7f67a51ad640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 0x7f67a00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67a59ae640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c7f0 tx=0x7f679c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c015070 con 0x7f67a006e6d0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f679c007820 con 0x7f67a006e6d0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a0136ea0 con 0x7f67a006e6d0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c007c10 con 0x7f67a006e6d0 2024-05-09T06:33:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.176+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01594d0 con 0x7f67a006e6d0 2024-05-09T06:33:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.177+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a00752b0 con 0x7f67a006e6d0 2024-05-09T06:33:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.177+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f679c018070 con 0x7f67a006e6d0 2024-05-09T06:33:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.178+0000 7f67967fc640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 0x7f678406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.178+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f679c08ade0 con 0x7f67a006e6d0 2024-05-09T06:33:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.178+0000 7f67a51ad640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 0x7f678406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.178+0000 7f67a51ad640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 0x7f678406e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6790034000 tx=0x7f679002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.182+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f679c0598d0 con 0x7f67a006e6d0 2024-05-09T06:33:59.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.367+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67a006e4b0 con 0x7f67a006e6d0 2024-05-09T06:33:59.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.368+0000 7f67967fc640 1 -- 172.21.15.1:0/2825762115 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+13033 (secure 0 0 0) 0x7f679c0591d0 con 0x7f67a006e6d0 2024-05-09T06:33:59.349 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:59.349 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":87,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:33:58.624230+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 msgr2=0x7f678406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 0x7f678406e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6790034000 tx=0x7f679002e000 comp rx=0 tx=0).stop 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 msgr2=0x7f67a0158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c7f0 tx=0x7f679c00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 shutdown_connections 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f678406be30 0x7f678406e2f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f67a00cf180 0x7f67a013a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67a00cdab0 0x7f67a00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 --2- 172.21.15.1:0/2825762115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a006e6d0 0x7f67a0158dc0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 >> 172.21.15.1:0/2825762115 conn(0x7f67a00cc4b0 msgr2=0x7f67a00cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 shutdown_connections 2024-05-09T06:33:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.370+0000 7f67a7c12640 1 -- 172.21.15.1:0/2825762115 wait complete. 2024-05-09T06:33:59.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:33:59.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:33:59.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.489+0000 7fdfa1dd5640 1 Processor -- start 2024-05-09T06:33:59.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.489+0000 7fdfa1dd5640 1 -- start start 2024-05-09T06:33:59.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdfa1dd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdfa1dd5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdfa1dd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c161fe0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdfa1dd5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c15f580 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdfa1dd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c1622c0 con 0x7fdf9c15e190 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38004/0 (socket says 172.21.15.1:38004) 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 -- 172.21.15.1:0/218302091 learned_addr learned my addr 172.21.15.1:0/218302091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 -- 172.21.15.1:0/218302091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 msgr2=0x7fdf9c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9b7fe640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 -- 172.21.15.1:0/218302091 >> v1:172.21.15.6:6789/0 conn(0x7fdf9c0d1c70 legacy=0x7fdf9c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:59.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.490+0000 7fdf9bfff640 1 -- 172.21.15.1:0/218302091 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1625a0 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.491+0000 7fdf9bfff640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009e50 tx=0x7fdf9402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ac0573e886ba26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.491+0000 7fdf9affd640 1 -- 172.21.15.1:0/218302091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94036070 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.491+0000 7fdf9affd640 1 -- 172.21.15.1:0/218302091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf94031600 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.491+0000 7fdf9affd640 1 -- 172.21.15.1:0/218302091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf940319d0 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/218302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d45f0 msgr2=0x7fdf9c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009e50 tx=0x7fdf9402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/218302091 shutdown_connections 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b3de0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/218302091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/218302091 >> 172.21.15.1:0/218302091 conn(0x7fdf9c073340 msgr2=0x7fdf9c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/218302091 shutdown_connections 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.492+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/218302091 wait complete. 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.493+0000 7fdfa1dd5640 1 Processor -- start 2024-05-09T06:33:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.493+0000 7fdfa1dd5640 1 -- start start 2024-05-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c14c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0b56d0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c161c60 con 0x7fdf9c0d45f0 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0b5850 con 0x7fdf9c15e190 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38008/0 (socket says 172.21.15.1:38008) 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 -- 172.21.15.1:0/3751152606 learned_addr learned my addr 172.21.15.1:0/3751152606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf93fff640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9bfff640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c14c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 -- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 msgr2=0x7fdf9c14c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c14c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 -- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf9c0d45f0 msgr2=0x7fdf9c0b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 -- 172.21.15.1:0/3751152606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c14dfa0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf9b7fe640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c820 tx=0x7fdf8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c015070 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004020 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c14e280 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0043f0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.495+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0b99a0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.496+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0752b0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.496+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf8c007830 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.497+0000 7fdf98ff9640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.497+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7fdf8c08aad0 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.497+0000 7fdf93fff640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.497+0000 7fdf93fff640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0b2fa0 tx=0x7fdf84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.501+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf8c059640 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.688+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf9c09d820 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.689+0000 7fdf98ff9640 1 -- 172.21.15.1:0/3751152606 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fdf8c058f40 con 0x7fdf9c0d1c70 2024-05-09T06:33:59.670 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 msgr2=0x7fdf7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0b2fa0 tx=0x7fdf84009040 comp rx=0 tx=0).stop 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 msgr2=0x7fdf9c0b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c820 tx=0x7fdf8c00cce0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 shutdown_connections 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf9c15e190 0x7fdf9c14c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf9c0d45f0 0x7fdf9c0b9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 --2- 172.21.15.1:0/3751152606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf9c0d1c70 0x7fdf9c0b6cc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 >> 172.21.15.1:0/3751152606 conn(0x7fdf9c073340 msgr2=0x7fdf9c06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 shutdown_connections 2024-05-09T06:33:59.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.691+0000 7fdfa1dd5640 1 -- 172.21.15.1:0/3751152606 wait complete. 2024-05-09T06:33:59.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:33:59.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.805+0000 7f6bc7e93640 1 Processor -- start 2024-05-09T06:33:59.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.805+0000 7f6bc7e93640 1 -- start start 2024-05-09T06:33:59.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc7e93640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc7e93640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 0x7f6bc006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc7e93640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc0070f40 con 0x7f6bc015db10 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc7e93640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc00cf170 con 0x7f6bc015e500 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc7e93640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00cf450 con 0x7f6bc00b2a60 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.806+0000 7f6bc542e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc6430640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 0x7f6bc006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:44744/0 (socket says 172.21.15.1:44744) 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 -- 172.21.15.1:0/4199567417 learned_addr learned my addr 172.21.15.1:0/4199567417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc5c2f640 1 --1- 172.21.15.1:0/4199567417 >> v1:172.21.15.6:6789/0 conn(0x7f6bc015db10 0x7f6bc00715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:33986/0 (socket says 172.21.15.1:33986) 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 -- 172.21.15.1:0/4199567417 >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 msgr2=0x7f6bc006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 --2- 172.21.15.1:0/4199567417 >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 0x7f6bc006e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 -- 172.21.15.1:0/4199567417 >> v1:172.21.15.6:6789/0 conn(0x7f6bc015db10 legacy=0x7f6bc00715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc542e640 1 -- 172.21.15.1:0/4199567417 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc0070c10 con 0x7f6bc00b2a60 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.807+0000 7f6bc6430640 1 --2- 172.21.15.1:0/4199567417 >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 0x7f6bc006e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc542e640 1 --2- 172.21.15.1:0/4199567417 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009870 tx=0x7f6bb402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d6f106af7ac403 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc4c2d640 1 -- 172.21.15.1:0/4199567417 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4037070 con 0x7f6bc00b2a60 2024-05-09T06:33:59.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc4c2d640 1 -- 172.21.15.1:0/4199567417 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4032440 con 0x7f6bc00b2a60 2024-05-09T06:33:59.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc4c2d640 1 -- 172.21.15.1:0/4199567417 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4032810 con 0x7f6bc00b2a60 2024-05-09T06:33:59.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc7e93640 1 -- 172.21.15.1:0/4199567417 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 msgr2=0x7f6bc015dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/4199567417 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009870 tx=0x7f6bb402a1c0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc7e93640 1 -- 172.21.15.1:0/4199567417 shutdown_connections 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/4199567417 >> v2:172.21.15.6:3300/0 conn(0x7f6bc015e500 0x7f6bc006e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.808+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/4199567417 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc00b2a60 0x7f6bc015dfc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.809+0000 7f6bc7e93640 1 -- 172.21.15.1:0/4199567417 >> 172.21.15.1:0/4199567417 conn(0x7f6bc00cc4b0 msgr2=0x7f6bc00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.809+0000 7f6bc7e93640 1 -- 172.21.15.1:0/4199567417 shutdown_connections 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.809+0000 7f6bc7e93640 1 -- 172.21.15.1:0/4199567417 wait complete. 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.810+0000 7f6bc7e93640 1 Processor -- start 2024-05-09T06:33:59.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.810+0000 7f6bc7e93640 1 -- start start 2024-05-09T06:33:59.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc00b2a60 0x7f6bc013b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc015db10 0x7f6bc013b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006cd30 con 0x7f6bc015e500 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc015ee50 con 0x7f6bc00b2a60 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc006ceb0 con 0x7f6bc015db10 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38024/0 (socket says 172.21.15.1:38024) 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 -- 172.21.15.1:0/1863948447 learned_addr learned my addr 172.21.15.1:0/1863948447 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc542e640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc015db10 0x7f6bc013b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc5c2f640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc00b2a60 0x7f6bc013b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 -- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc015db10 msgr2=0x7f6bc013b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc015db10 0x7f6bc013b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 -- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc00b2a60 msgr2=0x7f6bc013b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc00b2a60 0x7f6bc013b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 -- 172.21.15.1:0/1863948447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc0151940 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc6430640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009e60 tx=0x7f6bb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0018030 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb000cdf0 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc0139440 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.811+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0003bb0 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.812+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc0139a00 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.813+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc00752b0 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.813+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bb0005d10 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.814+0000 7f6bbe7fc640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.814+0000 7f6bc5c2f640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.814+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f6bb008b440 con 0x7f6bc015e500 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.814+0000 7f6bc5c2f640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 0x7f6bac06e320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bb8007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:33:59.819+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6bb0022070 con 0x7f6bc015e500 2024-05-09T06:33:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.005+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6bc00ade70 con 0x7f6bc015e500 2024-05-09T06:33:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.005+0000 7f6bbe7fc640 1 -- 172.21.15.1:0/1863948447 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) ==== 84+0+65 (secure 0 0 0) 0x7f6bb0010040 con 0x7f6bc015e500 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 msgr2=0x7f6bac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 0x7f6bac06e320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bb8007e80 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 msgr2=0x7f6bc013be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009e60 tx=0x7f6bb000c490 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 shutdown_connections 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc015e500 0x7f6bc013be70 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bc015db10 0x7f6bc013b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 --2- 172.21.15.1:0/1863948447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc00b2a60 0x7f6bc013b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.007+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 >> 172.21.15.1:0/1863948447 conn(0x7f6bc00cc4b0 msgr2=0x7f6bc00ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.008+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 shutdown_connections 2024-05-09T06:33:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.008+0000 7f6bc7e93640 1 -- 172.21.15.1:0/1863948447 wait complete. 2024-05-09T06:34:00.000 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:34:00.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:34:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.126+0000 7fa14aebd640 1 Processor -- start 2024-05-09T06:34:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.126+0000 7fa14aebd640 1 -- start start 2024-05-09T06:34:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa14aebd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa144166d00 0x7fa1441690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa14aebd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1440b27d0 con 0x7fa14406c570 2024-05-09T06:34:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa14aebd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1440b2ab0 con 0x7fa1440cfea0 2024-05-09T06:34:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa14aebd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa14415e650 con 0x7fa144166d00 2024-05-09T06:34:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa143fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa14406c570 0x7fa1440d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57514/0 (socket says 172.21.15.1:57514) 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.127+0000 7fa143fff640 1 -- 172.21.15.1:0/3350357573 learned_addr learned my addr 172.21.15.1:0/3350357573 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa14945a640 1 --2- 172.21.15.1:0/3350357573 >> v2:172.21.15.6:3300/0 conn(0x7fa144166d00 0x7fa1441690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073740341 0 0) 0x7fa1440b27d0 con 0x7fa14406c570 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa130003520 con 0x7fa14406c570 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936063150 0 0) 0x7fa1440b2ab0 con 0x7fa1440cfea0 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1440b27d0 con 0x7fa1440cfea0 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa14945a640 1 -- 172.21.15.1:0/3350357573 >> v1:172.21.15.6:6789/0 conn(0x7fa1440cfea0 legacy=0x7fa1440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa14945a640 1 -- 172.21.15.1:0/3350357573 >> v1:172.21.15.1:6789/0 conn(0x7fa14406c570 legacy=0x7fa1440d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.128+0000 7fa14945a640 1 -- 172.21.15.1:0/3350357573 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1440b2d90 con 0x7fa144166d00 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14945a640 1 --2- 172.21.15.1:0/3350357573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1441690f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097a0 tx=0x7fa13c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6829d25e02be9c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c037070 con 0x7fa144166d00 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c032440 con 0x7fa144166d00 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa1437fe640 1 -- 172.21.15.1:0/3350357573 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c032810 con 0x7fa144166d00 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14aebd640 1 -- 172.21.15.1:0/3350357573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 msgr2=0x7fa1441690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14aebd640 1 --2- 172.21.15.1:0/3350357573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1441690f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097a0 tx=0x7fa13c02a230 comp rx=0 tx=0).stop 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14aebd640 1 -- 172.21.15.1:0/3350357573 shutdown_connections 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14aebd640 1 --2- 172.21.15.1:0/3350357573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1441690f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.129+0000 7fa14aebd640 1 -- 172.21.15.1:0/3350357573 >> 172.21.15.1:0/3350357573 conn(0x7fa1440d1c10 msgr2=0x7fa1440d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.130+0000 7fa14aebd640 1 -- 172.21.15.1:0/3350357573 shutdown_connections 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.130+0000 7fa14aebd640 1 -- 172.21.15.1:0/3350357573 wait complete. 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.131+0000 7fa14aebd640 1 Processor -- start 2024-05-09T06:34:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.131+0000 7fa14aebd640 1 -- start start 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 0x7fa144152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1440afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440b3100 con 0x7fa14406c570 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440b2580 con 0x7fa144166d00 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa144169630 con 0x7fa1440cfea0 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38032/0 (socket says 172.21.15.1:38032) 2024-05-09T06:34:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 -- 172.21.15.1:0/1395242592 learned_addr learned my addr 172.21.15.1:0/1395242592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa143fff640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 0x7fa144152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14945a640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1440afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 -- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 msgr2=0x7fa144152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 0x7fa144152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 -- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 msgr2=0x7fa1440afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1440afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 -- 172.21.15.1:0/1395242592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14414d6f0 con 0x7fa14406c570 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa143fff640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 0x7fa144152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa148c59640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c870 tx=0x7fa13400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134003b70 con 0x7fa14406c570 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa134003d30 con 0x7fa14406c570 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13400aa90 con 0x7fa14406c570 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14415e9e0 con 0x7fa14406c570 2024-05-09T06:34:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.132+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14415efa0 con 0x7fa14406c570 2024-05-09T06:34:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.133+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144075320 con 0x7fa14406c570 2024-05-09T06:34:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.134+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa13400ac30 con 0x7fa14406c570 2024-05-09T06:34:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.134+0000 7fa1417fa640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 0x7fa12406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.134+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7fa13408aca0 con 0x7fa14406c570 2024-05-09T06:34:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.134+0000 7fa143fff640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 0x7fa12406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.135+0000 7fa143fff640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 0x7fa12406e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa1440cf0d0 tx=0x7fa12c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:00.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.139+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa134059820 con 0x7fa14406c570 2024-05-09T06:34:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:00.324+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa1440d1020 con 0x7fa14406c570 2024-05-09T06:34:01.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.301+0000 7fa1417fa640 1 -- 172.21.15.1:0/1395242592 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) ==== 138+0+0 (secure 0 0 0) 0x7fa134059120 con 0x7fa14406c570 2024-05-09T06:34:01.281 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:34:01.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.304+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 msgr2=0x7fa12406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.304+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 0x7fa12406e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa1440cf0d0 tx=0x7fa12c009040 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.304+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 msgr2=0x7fa14414cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.304+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c870 tx=0x7fa13400cd30 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.304+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 shutdown_connections 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa12406be30 0x7fa12406e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa144166d00 0x7fa1440afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1440cfea0 0x7fa144152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 --2- 172.21.15.1:0/1395242592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14406c570 0x7fa14414cca0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 >> 172.21.15.1:0/1395242592 conn(0x7fa1440d1c10 msgr2=0x7fa14406fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 shutdown_connections 2024-05-09T06:34:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.305+0000 7fa14aebd640 1 -- 172.21.15.1:0/1395242592 wait complete. 2024-05-09T06:34:01.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:34:01.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.440+0000 7efcd9b12640 1 Processor -- start 2024-05-09T06:34:01.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.440+0000 7efcd9b12640 1 -- start start 2024-05-09T06:34:01.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cea80 0x7efcd40cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efcd40cd140 0x7efcd406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efcd4070d10 0x7efcd40b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efcd40b3760 con 0x7efcd4070d10 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efcd40cf3a0 con 0x7efcd40cd140 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd9b12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efcd406d670 con 0x7efcd40cea80 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efcd4070d10 0x7efcd40b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efcd4070d10 0x7efcd40b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38042/0 (socket says 172.21.15.1:38042) 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd3fff640 1 -- 172.21.15.1:0/2342220390 learned_addr learned my addr 172.21.15.1:0/2342220390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd37fe640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cea80 0x7efcd40cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.441+0000 7efcd2ffd640 1 --2- 172.21.15.1:0/2342220390 >> v2:172.21.15.6:3300/0 conn(0x7efcd40cd140 0x7efcd406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 -- 172.21.15.1:0/2342220390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cea80 msgr2=0x7efcd40cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cea80 0x7efcd40cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 -- 172.21.15.1:0/2342220390 >> v2:172.21.15.6:3300/0 conn(0x7efcd40cd140 msgr2=0x7efcd406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:01.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 --2- 172.21.15.1:0/2342220390 >> v2:172.21.15.6:3300/0 conn(0x7efcd40cd140 0x7efcd406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 -- 172.21.15.1:0/2342220390 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd406d950 con 0x7efcd4070d10 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd3fff640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd4070d10 0x7efcd40b3bf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efcd406c850 tx=0x7efccc02cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7374fcdc3d37abea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.442+0000 7efcd27fc640 1 -- 172.21.15.1:0/2342220390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc034070 con 0x7efcd4070d10 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd27fc640 1 -- 172.21.15.1:0/2342220390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc02e680 con 0x7efcd4070d10 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd27fc640 1 -- 172.21.15.1:0/2342220390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc02ea50 con 0x7efcd4070d10 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 -- 172.21.15.1:0/2342220390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd4070d10 msgr2=0x7efcd40b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd4070d10 0x7efcd40b3bf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efcd406c850 tx=0x7efccc02cb70 comp rx=0 tx=0).stop 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 -- 172.21.15.1:0/2342220390 shutdown_connections 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd4070d10 0x7efcd40b3bf0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 --2- 172.21.15.1:0/2342220390 >> v2:172.21.15.6:3300/0 conn(0x7efcd40cd140 0x7efcd406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 --2- 172.21.15.1:0/2342220390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cea80 0x7efcd40cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 -- 172.21.15.1:0/2342220390 >> 172.21.15.1:0/2342220390 conn(0x7efcd40b1760 msgr2=0x7efcd40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 -- 172.21.15.1:0/2342220390 shutdown_connections 2024-05-09T06:34:01.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.443+0000 7efcd9b12640 1 -- 172.21.15.1:0/2342220390 wait complete. 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.444+0000 7efcd9b12640 1 Processor -- start 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 -- start start 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcd4070d10 0x7efcd40aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cd140 0x7efcd413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4144a30 con 0x7efcd40cea80 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efcd40b33f0 con 0x7efcd4070d10 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd9b12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efcd4144bb0 con 0x7efcd40cd140 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38056/0 (socket says 172.21.15.1:38056) 2024-05-09T06:34:01.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcd4070d10 0x7efcd40aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.445+0000 7efcd3fff640 1 -- 172.21.15.1:0/1286868893 learned_addr learned my addr 172.21.15.1:0/1286868893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd2ffd640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cd140 0x7efcd413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 -- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cd140 msgr2=0x7efcd413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cd140 0x7efcd413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 -- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcd4070d10 msgr2=0x7efcd40aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcd4070d10 0x7efcd40aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 -- 172.21.15.1:0/1286868893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd40aef70 con 0x7efcd40cea80 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd3fff640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efccc02d160 tx=0x7efccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc034030 con 0x7efcd40cea80 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc02f9a0 con 0x7efcd40cea80 2024-05-09T06:34:01.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc009510 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd40af240 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.446+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41f75b0 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.447+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd40752b0 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.448+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efccc037070 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.448+0000 7efcbbfff640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 0x7efcc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.448+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5619+0+0 (secure 0 0 0) 0x7efccc0ab430 con 0x7efcd40cea80 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.448+0000 7efcd37fe640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 0x7efcc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:01.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.449+0000 7efcd37fe640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 0x7efcc006e320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009aa0 tx=0x7efcc4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:01.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.453+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efccc047030 con 0x7efcd40cea80 2024-05-09T06:34:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:01.636+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efcd40b2430 con 0x7efcd40cea80 2024-05-09T06:34:02.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.303+0000 7efcbbfff640 1 -- 172.21.15.1:0/1286868893 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7efccc03e070 con 0x7efcd40cea80 2024-05-09T06:34:02.284 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 msgr2=0x7efcc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 0x7efcc006e320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009aa0 tx=0x7efcc4009190 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 msgr2=0x7efcd40cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efccc02d160 tx=0x7efccc004a40 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 shutdown_connections 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efcc006be60 0x7efcc006e320 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcd40cea80 0x7efcd40cc480 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efcd40cd140 0x7efcd413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 --2- 172.21.15.1:0/1286868893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcd4070d10 0x7efcd40aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.307+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 >> 172.21.15.1:0/1286868893 conn(0x7efcd40b1760 msgr2=0x7efcd40720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:02.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.308+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 shutdown_connections 2024-05-09T06:34:02.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.308+0000 7efcd9b12640 1 -- 172.21.15.1:0/1286868893 wait complete. 2024-05-09T06:34:02.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:34:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.449+0000 7fe1d111b640 1 Processor -- start 2024-05-09T06:34:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.449+0000 7fe1d111b640 1 -- start start 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1d111b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1d111b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe1cc06b1e0 0x7fe1cc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1d111b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc072050 con 0x7fe1cc0d06f0 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1d111b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe1cc072330 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1d111b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe1cc072610 con 0x7fe1cc06b1e0 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38072/0 (socket says 172.21.15.1:38072) 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 -- 172.21.15.1:0/2117561930 learned_addr learned my addr 172.21.15.1:0/2117561930 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 -- 172.21.15.1:0/2117561930 >> v2:172.21.15.6:3300/0 conn(0x7fe1cc06b1e0 msgr2=0x7fe1cc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1cb59e640 1 --2- 172.21.15.1:0/2117561930 >> v2:172.21.15.6:3300/0 conn(0x7fe1cc06b1e0 0x7fe1cc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 --2- 172.21.15.1:0/2117561930 >> v2:172.21.15.6:3300/0 conn(0x7fe1cc06b1e0 0x7fe1cc0cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.450+0000 7fe1ca59c640 1 -- 172.21.15.1:0/2117561930 >> v1:172.21.15.6:6789/0 conn(0x7fe1cc0d06f0 legacy=0x7fe1cc0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.451+0000 7fe1ca59c640 1 -- 172.21.15.1:0/2117561930 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc0728f0 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.451+0000 7fe1ca59c640 1 --2- 172.21.15.1:0/2117561930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b3780 tx=0x7fe1c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3089f95aa2f044 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.451+0000 7fe1c9d9b640 1 -- 172.21.15.1:0/2117561930 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0037070 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.451+0000 7fe1c9d9b640 1 -- 172.21.15.1:0/2117561930 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c0032440 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.451+0000 7fe1c9d9b640 1 -- 172.21.15.1:0/2117561930 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0032810 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 -- 172.21.15.1:0/2117561930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 msgr2=0x7fe1cc0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 --2- 172.21.15.1:0/2117561930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b3780 tx=0x7fe1c002a230 comp rx=0 tx=0).stop 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 -- 172.21.15.1:0/2117561930 shutdown_connections 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 --2- 172.21.15.1:0/2117561930 >> v2:172.21.15.6:3300/0 conn(0x7fe1cc06b1e0 0x7fe1cc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 --2- 172.21.15.1:0/2117561930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0ce8d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 -- 172.21.15.1:0/2117561930 >> 172.21.15.1:0/2117561930 conn(0x7fe1cc0d0b50 msgr2=0x7fe1cc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:02.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 -- 172.21.15.1:0/2117561930 shutdown_connections 2024-05-09T06:34:02.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.452+0000 7fe1d111b640 1 -- 172.21.15.1:0/2117561930 wait complete. 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 Processor -- start 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 -- start start 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1cc06b1e0 0x7fe1cc1f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0d06f0 0x7fe1cc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0ccf70 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0ccdf0 con 0x7fe1cc0d06f0 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1d111b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0cd0f0 con 0x7fe1cc06b1e0 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1ca59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1ca59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38076/0 (socket says 172.21.15.1:38076) 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.454+0000 7fe1ca59c640 1 -- 172.21.15.1:0/1781408935 learned_addr learned my addr 172.21.15.1:0/1781408935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1cad9d640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1cc06b1e0 0x7fe1cc1f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1cb59e640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0d06f0 0x7fe1cc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 -- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1cc06b1e0 msgr2=0x7fe1cc1f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1cc06b1e0 0x7fe1cc1f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 -- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0d06f0 msgr2=0x7fe1cc0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0d06f0 0x7fe1cc0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 -- 172.21.15.1:0/1781408935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc0a0b60 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1ca59c640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0041020 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c0030970 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0037040 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc0a0880 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.455+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc0b3bc0 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.456+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc0752b0 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.457+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1c0044020 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.458+0000 7fe1b37fe640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 0x7fe1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.458+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6001+0+0 (secure 0 0 0) 0x7fe1c00ab960 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.458+0000 7fe1cad9d640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 0x7fe1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.458+0000 7fe1cad9d640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 0x7fe1b806e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00b450 tx=0x7fe1bc00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:02.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.462+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1c004b030 con 0x7fe1cc0c41e0 2024-05-09T06:34:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:02.685+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe1cc072f30 con 0x7fe1cc0c41e0 2024-05-09T06:34:03.315 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 13 and data pool 14 2024-05-09T06:34:03.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.326+0000 7fe1b37fe640 1 -- 172.21.15.1:0/1781408935 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v57) ==== 174+0+0 (secure 0 0 0) 0x7fe1c0079a70 con 0x7fe1cc0c41e0 2024-05-09T06:34:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.339+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 msgr2=0x7fe1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.339+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 0x7fe1b806e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00b450 tx=0x7fe1bc00d040 comp rx=0 tx=0).stop 2024-05-09T06:34:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.339+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 msgr2=0x7fe1cc0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.339+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c00046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 shutdown_connections 2024-05-09T06:34:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe1b806be30 0x7fe1b806e2f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1cc0d06f0 0x7fe1cc0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1cc0c41e0 0x7fe1cc0b5d90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 --2- 172.21.15.1:0/1781408935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1cc06b1e0 0x7fe1cc1f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 >> 172.21.15.1:0/1781408935 conn(0x7fe1cc0d0b50 msgr2=0x7fe1cc15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 shutdown_connections 2024-05-09T06:34:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.342+0000 7fe1d111b640 1 -- 172.21.15.1:0/1781408935 wait complete. 2024-05-09T06:34:03.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:34:03.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.466+0000 7f3e45d42640 1 Processor -- start 2024-05-09T06:34:03.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 -- start start 2024-05-09T06:34:03.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3e40161950 0x7f3e400d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e400d4940 0x7f3e400d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e400d0f80 0x7f3e400cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3e400d4dd0 con 0x7f3e400d0f80 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3e400b33c0 con 0x7f3e40161950 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e45d42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3e400b3660 con 0x7f3e400d4940 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.467+0000 7f3e3ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e400d0f80 0x7f3e400cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3e400d0f80 0x7f3e400cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38090/0 (socket says 172.21.15.1:38090) 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 -- 172.21.15.1:0/3536574089 learned_addr learned my addr 172.21.15.1:0/3536574089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3effd640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e400d4940 0x7f3e400d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3f7fe640 1 --2- 172.21.15.1:0/3536574089 >> v2:172.21.15.6:3300/0 conn(0x7f3e40161950 0x7f3e400d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 -- 172.21.15.1:0/3536574089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e400d4940 msgr2=0x7f3e400d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e400d4940 0x7f3e400d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 -- 172.21.15.1:0/3536574089 >> v2:172.21.15.6:3300/0 conn(0x7f3e40161950 msgr2=0x7f3e400d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 --2- 172.21.15.1:0/3536574089 >> v2:172.21.15.6:3300/0 conn(0x7f3e40161950 0x7f3e400d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.468+0000 7f3e3ffff640 1 -- 172.21.15.1:0/3536574089 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400b3940 con 0x7f3e400d0f80 2024-05-09T06:34:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e3ffff640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e400cea30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3e4006b7f0 tx=0x7f3e3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad894d0d6c20ed99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/3536574089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38036070 con 0x7f3e400d0f80 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/3536574089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e38031600 con 0x7f3e400d0f80 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e3e7fc640 1 -- 172.21.15.1:0/3536574089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380319d0 con 0x7f3e400d0f80 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 -- 172.21.15.1:0/3536574089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 msgr2=0x7f3e400cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e400cea30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3e4006b7f0 tx=0x7f3e3802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 -- 172.21.15.1:0/3536574089 shutdown_connections 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e400cea30 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3536574089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e400d4940 0x7f3e400d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3536574089 >> v2:172.21.15.6:3300/0 conn(0x7f3e40161950 0x7f3e400d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 -- 172.21.15.1:0/3536574089 >> 172.21.15.1:0/3536574089 conn(0x7f3e400d3e20 msgr2=0x7f3e400d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 -- 172.21.15.1:0/3536574089 shutdown_connections 2024-05-09T06:34:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.469+0000 7f3e45d42640 1 -- 172.21.15.1:0/3536574089 wait complete. 2024-05-09T06:34:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.470+0000 7f3e45d42640 1 Processor -- start 2024-05-09T06:34:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 -- start start 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 0x7f3e4009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e4009dfe0 0x7f3e400cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400d1d80 con 0x7f3e400d0f80 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400d1c00 con 0x7f3e40161950 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e45d42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3e400d1f00 con 0x7f3e4009dfe0 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38106/0 (socket says 172.21.15.1:38106) 2024-05-09T06:34:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.471+0000 7f3e3f7fe640 1 -- 172.21.15.1:0/3690740885 learned_addr learned my addr 172.21.15.1:0/3690740885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3effd640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 0x7f3e4009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3ffff640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e4009dfe0 0x7f3e400cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 -- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e4009dfe0 msgr2=0x7f3e400cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e4009dfe0 0x7f3e400cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 -- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 msgr2=0x7f3e4009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 0x7f3e4009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 -- 172.21.15.1:0/3690740885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400cbd40 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3effd640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 0x7f3e4009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e3f7fe640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e870 tx=0x7f3e3000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3000c940 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e400cc8b0 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000cd30 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.472+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e400cce70 con 0x7f3e400d0f80 2024-05-09T06:34:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.473+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e400752b0 con 0x7f3e400d0f80 2024-05-09T06:34:03.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.474+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e3001a070 con 0x7f3e400d0f80 2024-05-09T06:34:03.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.474+0000 7f3e27fff640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.474+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f3e3008b170 con 0x7f3e400d0f80 2024-05-09T06:34:03.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.474+0000 7f3e3effd640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.475+0000 7f3e3effd640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3e400cfcc0 tx=0x7f3e34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.479+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e30059a20 con 0x7f3e400d0f80 2024-05-09T06:34:03.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.662+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e400d21e0 con 0x7f3e400d0f80 2024-05-09T06:34:03.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.663+0000 7f3e27fff640 1 -- 172.21.15.1:0/3690740885 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16766 (secure 0 0 0) 0x7f3e30059320 con 0x7f3e400d0f80 2024-05-09T06:34:03.643 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:03.644 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:34:02.686458+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 msgr2=0x7f3e2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3e400cfcc0 tx=0x7f3e34008040 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 msgr2=0x7f3e4015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e870 tx=0x7f3e3000ed30 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 shutdown_connections 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3e4009dfe0 0x7f3e400cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e40161950 0x7f3e4009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 --2- 172.21.15.1:0/3690740885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e400d0f80 0x7f3e4015c870 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 >> 172.21.15.1:0/3690740885 conn(0x7f3e400d3e20 msgr2=0x7f3e400b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 shutdown_connections 2024-05-09T06:34:03.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.665+0000 7f3e45d42640 1 -- 172.21.15.1:0/3690740885 wait complete. 2024-05-09T06:34:03.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:34:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.781+0000 7f30938ae640 1 Processor -- start 2024-05-09T06:34:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.781+0000 7f30938ae640 1 -- start start 2024-05-09T06:34:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f30938ae640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f308c15db10 0x7f308c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f30938ae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c0715d0 0x7f308c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f30938ae640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c070f40 con 0x7f308c15e500 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f30938ae640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f308c0cf170 con 0x7f308c15db10 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f30938ae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f308c0cf450 con 0x7f308c0715d0 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f3090e49640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c0715d0 0x7f308c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f308c15db10 0x7f308c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f3091e4b640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f308c15e500 0x7f308c06e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:34014/0 (socket says 172.21.15.1:34014) 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f308c15db10 0x7f308c0b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:35720/0 (socket says 172.21.15.1:35720) 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f3091e4b640 1 -- 172.21.15.1:0/2377864837 learned_addr learned my addr 172.21.15.1:0/2377864837 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 -- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c0715d0 msgr2=0x7f308c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 --2- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c0715d0 0x7f308c15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 -- 172.21.15.1:0/2377864837 >> v1:172.21.15.6:6789/0 conn(0x7f308c15e500 legacy=0x7f308c06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.782+0000 7f309164a640 1 -- 172.21.15.1:0/2377864837 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c070c10 con 0x7f308c15db10 2024-05-09T06:34:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.783+0000 7f309164a640 1 --2- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15db10 0x7f308c0b2a60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30880097a0 tx=0x7f308802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9b6f93ad075edb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.783+0000 7f3083fff640 1 -- 172.21.15.1:0/2377864837 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088037070 con 0x7f308c15db10 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.783+0000 7f3083fff640 1 -- 172.21.15.1:0/2377864837 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3088032440 con 0x7f308c15db10 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.783+0000 7f3083fff640 1 -- 172.21.15.1:0/2377864837 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088032810 con 0x7f308c15db10 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 -- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15db10 msgr2=0x7f308c0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 --2- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15db10 0x7f308c0b2a60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30880097a0 tx=0x7f308802a180 comp rx=0 tx=0).stop 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 -- 172.21.15.1:0/2377864837 shutdown_connections 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 --2- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c0715d0 0x7f308c15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 --2- 172.21.15.1:0/2377864837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15db10 0x7f308c0b2a60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 -- 172.21.15.1:0/2377864837 >> 172.21.15.1:0/2377864837 conn(0x7f308c0cc4b0 msgr2=0x7f308c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 -- 172.21.15.1:0/2377864837 shutdown_connections 2024-05-09T06:34:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.784+0000 7f30938ae640 1 -- 172.21.15.1:0/2377864837 wait complete. 2024-05-09T06:34:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.785+0000 7f30938ae640 1 Processor -- start 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.785+0000 7f30938ae640 1 -- start start 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 0x7f308c0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 0x7f308c0ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15edc0 con 0x7f308c0715d0 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15ec40 con 0x7f308c15e500 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f30938ae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f308c06ea50 con 0x7f308c15db10 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38114/0 (socket says 172.21.15.1:38114) 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 -- 172.21.15.1:0/1640662979 learned_addr learned my addr 172.21.15.1:0/1640662979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f3091e4b640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 0x7f308c0ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f3090e49640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 0x7f308c0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 -- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 msgr2=0x7f308c0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 0x7f308c0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 -- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 msgr2=0x7f308c0ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 0x7f308c0ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f309164a640 1 -- 172.21.15.1:0/1640662979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c14b820 con 0x7f308c0715d0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f3091e4b640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 0x7f308c0ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.786+0000 7f3090e49640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 0x7f308c0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f309164a640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f30880098c0 tx=0x7f30880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088041070 con 0x7f308c0715d0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308803c870 con 0x7f308c0715d0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c0c4c70 con 0x7f308c0715d0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088037040 con 0x7f308c0715d0 2024-05-09T06:34:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.787+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c148f70 con 0x7f308c0715d0 2024-05-09T06:34:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.788+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c0752b0 con 0x7f308c0715d0 2024-05-09T06:34:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.789+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3088044070 con 0x7f308c0715d0 2024-05-09T06:34:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.789+0000 7f3081ffb640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.789+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f30880abc10 con 0x7f308c0715d0 2024-05-09T06:34:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.789+0000 7f3090e49640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.790+0000 7f3090e49640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 0x7f307006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f308c06e3b0 tx=0x7f307c00b3a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:03.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.793+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f308c14b820 con 0x7f308c0715d0 2024-05-09T06:34:03.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.981+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f308c06d870 con 0x7f308c0715d0 2024-05-09T06:34:03.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.982+0000 7f3081ffb640 1 -- 172.21.15.1:0/1640662979 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16766 (secure 0 0 0) 0x7f308807a550 con 0x7f308c0715d0 2024-05-09T06:34:03.963 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:03.963 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:34:02.686458+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.984+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 msgr2=0x7f307006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 0x7f307006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f308c06e3b0 tx=0x7f307c00b3a0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 msgr2=0x7f308c14b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f30880098c0 tx=0x7f30880046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 shutdown_connections 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f308c15e500 0x7f308c0ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f308c15db10 0x7f308c0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 --2- 172.21.15.1:0/1640662979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f308c0715d0 0x7f308c14b2e0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:03.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 >> 172.21.15.1:0/1640662979 conn(0x7f308c0cc4b0 msgr2=0x7f308c15a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:03.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 shutdown_connections 2024-05-09T06:34:03.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:03.985+0000 7f30938ae640 1 -- 172.21.15.1:0/1640662979 wait complete. 2024-05-09T06:34:03.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:34:04.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.103+0000 7f49dd7c1640 1 Processor -- start 2024-05-09T06:34:04.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.103+0000 7f49dd7c1640 1 -- start start 2024-05-09T06:34:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49dd7c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 0x7f49d80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49dd7c1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80b1340 con 0x7f49d80ceba0 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49dd7c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80d1020 con 0x7f49d815e270 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49dd7c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80d0df0 con 0x7f49d80cf180 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49cffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f49d80ceba0 0x7f49d80ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57528/0 (socket says 172.21.15.1:57528) 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49cffff640 1 -- 172.21.15.1:0/1304793451 learned_addr learned my addr 172.21.15.1:0/1304793451 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49d77fe640 1 --2- 172.21.15.1:0/1304793451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 0x7f49d80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858092050 0 0) 0x7f49c0001110 con 0x7f49d80ceba0 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49c4003520 con 0x7f49d80ceba0 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76895958 0 0) 0x7f49d80d1020 con 0x7f49d815e270 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.104+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49c0001110 con 0x7f49d815e270 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.105+0000 7f49d77fe640 1 -- 172.21.15.1:0/1304793451 >> v1:172.21.15.6:6789/0 conn(0x7f49d815e270 legacy=0x7f49d80ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.105+0000 7f49d77fe640 1 -- 172.21.15.1:0/1304793451 >> v1:172.21.15.1:6789/0 conn(0x7f49d80ceba0 legacy=0x7f49d80ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.105+0000 7f49d77fe640 1 -- 172.21.15.1:0/1304793451 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80cc4b0 con 0x7f49d80cf180 2024-05-09T06:34:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.105+0000 7f49d77fe640 1 --2- 172.21.15.1:0/1304793451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 0x7f49d80d08b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009dc0 tx=0x7f49d002a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20009e5d9160033 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0007740 con 0x7f49d80cf180 2024-05-09T06:34:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d00078e0 con 0x7f49d80cf180 2024-05-09T06:34:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49d67fc640 1 -- 172.21.15.1:0/1304793451 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0007cb0 con 0x7f49d80cf180 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 -- 172.21.15.1:0/1304793451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 msgr2=0x7f49d80d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/1304793451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 0x7f49d80d08b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009dc0 tx=0x7f49d002a760 comp rx=0 tx=0).stop 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 -- 172.21.15.1:0/1304793451 shutdown_connections 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/1304793451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80cf180 0x7f49d80d08b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 -- 172.21.15.1:0/1304793451 >> 172.21.15.1:0/1304793451 conn(0x7f49d80d1c10 msgr2=0x7f49d80d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 -- 172.21.15.1:0/1304793451 shutdown_connections 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.106+0000 7f49dd7c1640 1 -- 172.21.15.1:0/1304793451 wait complete. 2024-05-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.107+0000 7f49dd7c1640 1 Processor -- start 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.107+0000 7f49dd7c1640 1 -- start start 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80ceba0 0x7f49d80cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 0x7f49d80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80cca70 con 0x7f49d80cf180 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80cc260 con 0x7f49d815e270 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49dd7c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f49d815e650 con 0x7f49d80ceba0 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38116/0 (socket says 172.21.15.1:38116) 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49cffff640 1 -- 172.21.15.1:0/826289599 learned_addr learned my addr 172.21.15.1:0/826289599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49d77fe640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 0x7f49d80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.108+0000 7f49d6ffd640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80ceba0 0x7f49d80cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:04.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 -- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80ceba0 msgr2=0x7f49d80cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80ceba0 0x7f49d80cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 -- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 msgr2=0x7f49d80c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 0x7f49d80c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 -- 172.21.15.1:0/826289599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81f4bd0 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49d77fe640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 0x7f49d80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cffff640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c760 tx=0x7f49c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0005ab0 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d80cadc0 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0007070 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.109+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d80cb380 con 0x7f49d80cf180 2024-05-09T06:34:04.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.110+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80752b0 con 0x7f49d80cf180 2024-05-09T06:34:04.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.111+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49c001a070 con 0x7f49d80cf180 2024-05-09T06:34:04.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.111+0000 7f49cf7fe640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 0x7f49b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:04.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.111+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f49c008a540 con 0x7f49d80cf180 2024-05-09T06:34:04.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.111+0000 7f49d6ffd640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 0x7f49b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:04.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.112+0000 7f49d6ffd640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 0x7f49b806e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49d80c8800 tx=0x7f49c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:04.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.115+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49d80b1340 con 0x7f49d80cf180 2024-05-09T06:34:04.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:04.339+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f49d80d29e0 con 0x7f49d80cf180 2024-05-09T06:34:05.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.336+0000 7f49cf7fe640 1 -- 172.21.15.1:0/826289599 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f49c0058d60 con 0x7f49d80cf180 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 msgr2=0x7f49b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 0x7f49b806e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49d80c8800 tx=0x7f49c8009040 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 msgr2=0x7f49d80c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c760 tx=0x7f49c000cc20 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 shutdown_connections 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f49b806be30 0x7f49b806e2f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49d815e270 0x7f49d80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d80cf180 0x7f49d80c7aa0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 --2- 172.21.15.1:0/826289599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49d80ceba0 0x7f49d80cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 >> 172.21.15.1:0/826289599 conn(0x7f49d80d1c10 msgr2=0x7f49d80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 shutdown_connections 2024-05-09T06:34:05.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.340+0000 7f49dd7c1640 1 -- 172.21.15.1:0/826289599 wait complete. 2024-05-09T06:34:05.334 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:34:05.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:34:05.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.467+0000 7f9bc839d640 1 Processor -- start 2024-05-09T06:34:05.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.467+0000 7f9bc839d640 1 -- start start 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc839d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc00cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc839d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00d0390 con 0x7f9bc00aee30 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc839d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00d0670 con 0x7f9bc00ccbf0 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc839d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc00d2ba0 con 0x7f9bc015c000 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc693a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9bc00aee30 0x7f9bc00af280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57544/0 (socket says 172.21.15.1:57544) 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc693a640 1 -- 172.21.15.1:0/540329121 learned_addr learned my addr 172.21.15.1:0/540329121 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc6139640 1 --2- 172.21.15.1:0/540329121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc00cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207999649 0 0) 0x7f9bc00d0390 con 0x7f9bc00aee30 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.468+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bac003520 con 0x7f9bc00aee30 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.469+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 541855507 0 0) 0x7f9bac003520 con 0x7f9bc00aee30 2024-05-09T06:34:05.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.469+0000 7f9bc6139640 1 -- 172.21.15.1:0/540329121 >> v1:172.21.15.6:6789/0 conn(0x7f9bc00ccbf0 legacy=0x7f9bc00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.469+0000 7f9bc6139640 1 -- 172.21.15.1:0/540329121 >> v1:172.21.15.1:6789/0 conn(0x7f9bc00aee30 legacy=0x7f9bc00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.469+0000 7f9bc6139640 1 -- 172.21.15.1:0/540329121 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00d0950 con 0x7f9bc015c000 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.469+0000 7f9bc6139640 1 --2- 172.21.15.1:0/540329121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc00cc6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0097d0 tx=0x7f9bbc02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54a7a6fa431cd97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc037070 con 0x7f9bc015c000 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc02ab40 con 0x7f9bc015c000 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc5137640 1 -- 172.21.15.1:0/540329121 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80019f0 con 0x7f9bc015c000 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 -- 172.21.15.1:0/540329121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 msgr2=0x7f9bc00cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 --2- 172.21.15.1:0/540329121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc00cc6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0097d0 tx=0x7f9bbc02a0b0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 -- 172.21.15.1:0/540329121 shutdown_connections 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 --2- 172.21.15.1:0/540329121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc00cc6b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 -- 172.21.15.1:0/540329121 >> 172.21.15.1:0/540329121 conn(0x7f9bc00d1c10 msgr2=0x7f9bc00d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 -- 172.21.15.1:0/540329121 shutdown_connections 2024-05-09T06:34:05.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.470+0000 7f9bc839d640 1 -- 172.21.15.1:0/540329121 wait complete. 2024-05-09T06:34:05.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.471+0000 7f9bc839d640 1 Processor -- start 2024-05-09T06:34:05.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.471+0000 7f9bc839d640 1 -- start start 2024-05-09T06:34:05.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bc00ccbf0 0x7f9bc015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc009c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00b0de0 con 0x7f9bc00aee30 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00ae9a0 con 0x7f9bc00ccbf0 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc839d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc00d2730 con 0x7f9bc015c000 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc6139640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc6139640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38132/0 (socket says 172.21.15.1:38132) 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.472+0000 7f9bc6139640 1 -- 172.21.15.1:0/494961189 learned_addr learned my addr 172.21.15.1:0/494961189 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc693a640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc009c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc5938640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bc00ccbf0 0x7f9bc015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 -- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 msgr2=0x7f9bc009c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc009c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 -- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bc00ccbf0 msgr2=0x7f9bc015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bc00ccbf0 0x7f9bc015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 -- 172.21.15.1:0/494961189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc009e0c0 con 0x7f9bc00aee30 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc6139640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0097d0 tx=0x7f9bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc042070 con 0x7f9bc00aee30 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc03d440 con 0x7f9bc00aee30 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc037040 con 0x7f9bc00aee30 2024-05-09T06:34:05.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc009e330 con 0x7f9bc00aee30 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.473+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc009e8f0 con 0x7f9bc00aee30 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.474+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc00752b0 con 0x7f9bc00aee30 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.474+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bbc045070 con 0x7f9bc00aee30 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.475+0000 7f9bb6ffd640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.475+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9bbc0ab6d0 con 0x7f9bc00aee30 2024-05-09T06:34:05.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.475+0000 7f9bc5938640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.475+0000 7f9bc5938640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 0x7f9ba406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bc01f7080 tx=0x7f9bb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:05.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.479+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9bbc079ee0 con 0x7f9bc00aee30 2024-05-09T06:34:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:05.703+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9bc00b0520 con 0x7f9bc00aee30 2024-05-09T06:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.342+0000 7f9bb6ffd640 1 -- 172.21.15.1:0/494961189 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f9bbc0797e0 con 0x7f9bc00aee30 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 msgr2=0x7f9ba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 0x7f9ba406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bc01f7080 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 msgr2=0x7f9bc0156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0097d0 tx=0x7f9bbc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 shutdown_connections 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9bc015c000 0x7f9bc009c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9bc00ccbf0 0x7f9bc015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 --2- 172.21.15.1:0/494961189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc00aee30 0x7f9bc0156280 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 >> 172.21.15.1:0/494961189 conn(0x7f9bc00d1c10 msgr2=0x7f9bc015b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 shutdown_connections 2024-05-09T06:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.345+0000 7f9bc839d640 1 -- 172.21.15.1:0/494961189 wait complete. 2024-05-09T06:34:06.339 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:34:06.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:06.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.469+0000 7fbe6741c640 1 Processor -- start 2024-05-09T06:34:06.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.469+0000 7fbe6741c640 1 -- start start 2024-05-09T06:34:06.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe6741c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbe600d4940 0x7fbe600b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe6741c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600d2b10 0x7fbe600b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe6741c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe600b33c0 con 0x7fbe60161950 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe6741c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbe600d4dd0 con 0x7fbe600d4940 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe6741c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600b3660 con 0x7fbe600d2b10 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe651b8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbe60161950 0x7fbe600d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57560/0 (socket says 172.21.15.1:57560) 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.470+0000 7fbe651b8640 1 -- 172.21.15.1:0/1639479841 learned_addr learned my addr 172.21.15.1:0/1639479841 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600d2b10 0x7fbe600b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe649b7640 1 --2- 172.21.15.1:0/1639479841 >> v2:172.21.15.6:3300/0 conn(0x7fbe600d4940 0x7fbe600b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811652649 0 0) 0x7fbe600b33c0 con 0x7fbe60161950 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe60161950 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 525438349 0 0) 0x7fbe48003520 con 0x7fbe60161950 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600d2b10 msgr2=0x7fbe600b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600d2b10 0x7fbe600b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 >> v2:172.21.15.6:3300/0 conn(0x7fbe600d4940 msgr2=0x7fbe600b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 --2- 172.21.15.1:0/1639479841 >> v2:172.21.15.6:3300/0 conn(0x7fbe600d4940 0x7fbe600b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe600b3940 con 0x7fbe60161950 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fbe58001690 con 0x7fbe60161950 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 >> v1:172.21.15.1:6789/0 conn(0x7fbe60161950 legacy=0x7fbe600d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe480039b0 0x7fbe48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe480051c0 0x7fbe4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600d4dd0 con 0x7fbe480051c0 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600b3660 con 0x7fbe480039b0 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4800bf00 con 0x7fbe48004800 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe649b7640 1 --2- 172.21.15.1:0/1639479841 >> v2:172.21.15.6:3300/0 conn(0x7fbe600d4940 0x7fbe600b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600d2b10 0x7fbe600b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:06.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe480051c0 0x7fbe4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe651b8640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe480039b0 0x7fbe48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.471+0000 7fbe649b7640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 -- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe48004800 msgr2=0x7fbe48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 -- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe480039b0 msgr2=0x7fbe48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe480039b0 0x7fbe48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 -- 172.21.15.1:0/1639479841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe48003520 con 0x7fbe480051c0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe659b9640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe480051c0 0x7fbe4800b9c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe6008e560 tx=0x7fbe48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaacebe7d2d9859 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50036070 con 0x7fbe480051c0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe50004030 con 0x7fbe480051c0 2024-05-09T06:34:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.472+0000 7fbe5ffff640 1 -- 172.21.15.1:0/1639479841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe500097f0 con 0x7fbe480051c0 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 -- 172.21.15.1:0/1639479841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe480051c0 msgr2=0x7fbe4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 --2- 172.21.15.1:0/1639479841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe480051c0 0x7fbe4800b9c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe6008e560 tx=0x7fbe48005830 comp rx=0 tx=0).stop 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe659b9640 1 -- 172.21.15.1:0/1639479841 reap_dead start 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 -- 172.21.15.1:0/1639479841 shutdown_connections 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 -- 172.21.15.1:0/1639479841 >> 172.21.15.1:0/1639479841 conn(0x7fbe600d3e20 msgr2=0x7fbe600d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:06.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 -- 172.21.15.1:0/1639479841 shutdown_connections 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.473+0000 7fbe6741c640 1 -- 172.21.15.1:0/1639479841 wait complete. 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.474+0000 7fbe6741c640 1 Processor -- start 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.474+0000 7fbe6741c640 1 -- start start 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe600b1c50 0x7fbe601f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600b99d0 0x7fbe600a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600d1100 con 0x7fbe600b2030 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600d0f80 con 0x7fbe600b1c50 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe6741c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600d1c00 con 0x7fbe600b99d0 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38148/0 (socket says 172.21.15.1:38148) 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 -- 172.21.15.1:0/2629058750 learned_addr learned my addr 172.21.15.1:0/2629058750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 -- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600b99d0 msgr2=0x7fbe600a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe651b8640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe600b1c50 0x7fbe601f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600b99d0 0x7fbe600a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 -- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe600b1c50 msgr2=0x7fbe601f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe600b1c50 0x7fbe601f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.475+0000 7fbe649b7640 1 -- 172.21.15.1:0/2629058750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe601f43b0 con 0x7fbe600b2030 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe649b7640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400f430 tx=0x7fbe5400f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54003b90 con 0x7fbe600b2030 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe54003d50 con 0x7fbe600b2030 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe601f4690 con 0x7fbe600b2030 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54004270 con 0x7fbe600b2030 2024-05-09T06:34:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.476+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601f4f50 con 0x7fbe600b2030 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.477+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe600752b0 con 0x7fbe600b2030 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.478+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe54004410 con 0x7fbe600b2030 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.478+0000 7fbe5dffb640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 0x7fbe4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.478+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fbe5408c0c0 con 0x7fbe600b2030 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.478+0000 7fbe651b8640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 0x7fbe4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.479+0000 7fbe651b8640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 0x7fbe4c06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe58004840 tx=0x7fbe58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.482+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe54019070 con 0x7fbe600b2030 2024-05-09T06:34:06.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.703+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe600d21e0 con 0x7fbe600b2030 2024-05-09T06:34:06.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.703+0000 7fbe5dffb640 1 -- 172.21.15.1:0/2629058750 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5168 (secure 0 0 0) 0x7fbe54010040 con 0x7fbe600b2030 2024-05-09T06:34:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:06.684 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7899,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:05.704044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7911},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7911":{"gid":7911,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/1974102470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1974102470},{"type":"v1","addr":"172.21.15.6:6835","nonce":1974102470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7911,"qdb_cluster":[7911]},"id":6}]} 2024-05-09T06:34:06.684 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 61 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 msgr2=0x7fbe4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 0x7fbe4c06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe58004840 tx=0x7fbe58009040 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 msgr2=0x7fbe600a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400f430 tx=0x7fbe5400f8f0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 shutdown_connections 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe4c06be30 0x7fbe4c06e2f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe600b99d0 0x7fbe600a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe600b2030 0x7fbe600a0f10 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 --2- 172.21.15.1:0/2629058750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe600b1c50 0x7fbe601f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 >> 172.21.15.1:0/2629058750 conn(0x7fbe600d3e20 msgr2=0x7fbe600b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 shutdown_connections 2024-05-09T06:34:06.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.705+0000 7fbe6741c640 1 -- 172.21.15.1:0/2629058750 wait complete. 2024-05-09T06:34:06.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:34:06.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 Processor -- start 2024-05-09T06:34:06.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 -- start start 2024-05-09T06:34:06.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd7a00d0f80 0x7fd7a00d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a00b33c0 con 0x7fd7a00d4940 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a00d4dd0 con 0x7fd7a00d0f80 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.821+0000 7fd7a5d16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00b3660 con 0x7fd7a0161950 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:44936/0 (socket says 172.21.15.1:44936) 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 -- 172.21.15.1:0/1654887657 learned_addr learned my addr 172.21.15.1:0/1654887657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79ffff640 1 --2- 172.21.15.1:0/1654887657 >> v2:172.21.15.6:3300/0 conn(0x7fd7a00d0f80 0x7fd7a00d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 -- 172.21.15.1:0/1654887657 >> v2:172.21.15.6:3300/0 conn(0x7fd7a00d0f80 msgr2=0x7fd7a00d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 --2- 172.21.15.1:0/1654887657 >> v2:172.21.15.6:3300/0 conn(0x7fd7a00d0f80 0x7fd7a00d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 -- 172.21.15.1:0/1654887657 >> v1:172.21.15.6:6789/0 conn(0x7fd7a00d4940 legacy=0x7fd7a00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.822+0000 7fd79f7fe640 1 -- 172.21.15.1:0/1654887657 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a00b3940 con 0x7fd7a0161950 2024-05-09T06:34:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd79f7fe640 1 --2- 172.21.15.1:0/1654887657 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00b3590 tx=0x7fd79002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6e1c6bc660df9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd79e7fc640 1 -- 172.21.15.1:0/1654887657 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790036070 con 0x7fd7a0161950 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd79e7fc640 1 -- 172.21.15.1:0/1654887657 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790031600 con 0x7fd7a0161950 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd79e7fc640 1 -- 172.21.15.1:0/1654887657 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900319d0 con 0x7fd7a0161950 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1654887657 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 msgr2=0x7fd7a00d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.823+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1654887657 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00b3590 tx=0x7fd79002a710 comp rx=0 tx=0).stop 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1654887657 shutdown_connections 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1654887657 >> v2:172.21.15.6:3300/0 conn(0x7fd7a00d0f80 0x7fd7a00d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1654887657 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a0161950 0x7fd7a00d2400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1654887657 >> 172.21.15.1:0/1654887657 conn(0x7fd7a00d3e20 msgr2=0x7fd7a00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1654887657 shutdown_connections 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.824+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1654887657 wait complete. 2024-05-09T06:34:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.825+0000 7fd7a5d16640 1 Processor -- start 2024-05-09T06:34:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.825+0000 7fd7a5d16640 1 -- start start 2024-05-09T06:34:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a00d0f80 0x7fd7a0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a0153e30 0x7fd7a0151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00b52d0 con 0x7fd7a0161950 2024-05-09T06:34:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00b5150 con 0x7fd7a0153e30 2024-05-09T06:34:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd7a5d16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00b5450 con 0x7fd7a00d0f80 2024-05-09T06:34:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38158/0 (socket says 172.21.15.1:38158) 2024-05-09T06:34:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 -- 172.21.15.1:0/1568600880 learned_addr learned my addr 172.21.15.1:0/1568600880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79ffff640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a0153e30 0x7fd7a0151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79f7fe640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a00d0f80 0x7fd7a0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 -- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a00d0f80 msgr2=0x7fd7a0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a00d0f80 0x7fd7a0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 -- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a0153e30 msgr2=0x7fd7a0151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a0153e30 0x7fd7a0151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 -- 172.21.15.1:0/1568600880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0152850 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd79effd640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd79400c400 tx=0x7fd79400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.826+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940052d0 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.827+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794005490 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.827+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a0152b30 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.827+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794007260 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.827+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01530f0 con 0x7fd7a0161950 2024-05-09T06:34:06.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.828+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a00752b0 con 0x7fd7a0161950 2024-05-09T06:34:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.828+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd794007400 con 0x7fd7a0161950 2024-05-09T06:34:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.829+0000 7fd787fff640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 0x7fd78c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.829+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fd79408a480 con 0x7fd7a0161950 2024-05-09T06:34:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.829+0000 7fd79f7fe640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 0x7fd78c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.829+0000 7fd79f7fe640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 0x7fd78c06e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd790033000 tx=0x7fd790009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:06.833+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd794058650 con 0x7fd7a0161950 2024-05-09T06:34:06.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.018+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7a00d21e0 con 0x7fd7a0161950 2024-05-09T06:34:06.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.019+0000 7fd787fff640 1 -- 172.21.15.1:0/1568600880 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16753 (secure 0 0 0) 0x7fd79405c950 con 0x7fd7a0161950 2024-05-09T06:34:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:07.000 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:34:04.325165+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:07.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 msgr2=0x7fd78c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 0x7fd78c06e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd790033000 tx=0x7fd790009000 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 msgr2=0x7fd7a01538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd79400c400 tx=0x7fd79400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 shutdown_connections 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd78c06be30 0x7fd78c06e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a0153e30 0x7fd7a0151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7a0161950 0x7fd7a01538f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 --2- 172.21.15.1:0/1568600880 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7a00d0f80 0x7fd7a0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 >> 172.21.15.1:0/1568600880 conn(0x7fd7a00d3e20 msgr2=0x7fd7a006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 shutdown_connections 2024-05-09T06:34:07.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.021+0000 7fd7a5d16640 1 -- 172.21.15.1:0/1568600880 wait complete. 2024-05-09T06:34:07.015 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:34:07.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:34:07.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.139+0000 7fb92bd63640 1 Processor -- start 2024-05-09T06:34:07.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.139+0000 7fb92bd63640 1 -- start start 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92bd63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9240b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92bd63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9240b4f70 0x7fb924070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92bd63640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92406eb40 con 0x7fb9240b2a60 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92bd63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb92406ee20 con 0x7fb9240b4f70 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92bd63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb92406d8e0 con 0x7fb9241454b0 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb9292fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb9240b2a60 0x7fb9240b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57566/0 (socket says 172.21.15.1:57566) 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb9292fe640 1 -- 172.21.15.1:0/412422645 learned_addr learned my addr 172.21.15.1:0/412422645 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb929aff640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9240b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382256506 0 0) 0x7fb92406eb40 con 0x7fb9240b2a60 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb910003520 con 0x7fb9240b2a60 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.140+0000 7fb92a300640 1 --2- 172.21.15.1:0/412422645 >> v2:172.21.15.6:3300/0 conn(0x7fb9240b4f70 0x7fb924070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3721243286 0 0) 0x7fb910003520 con 0x7fb9240b2a60 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 msgr2=0x7fb9240b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9240b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 >> v2:172.21.15.6:3300/0 conn(0x7fb9240b4f70 msgr2=0x7fb924070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 --2- 172.21.15.1:0/412422645 >> v2:172.21.15.6:3300/0 conn(0x7fb9240b4f70 0x7fb924070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92406dbc0 con 0x7fb9240b2a60 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb914001670 con 0x7fb9240b2a60 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 >> v1:172.21.15.1:6789/0 conn(0x7fb9240b2a60 legacy=0x7fb9240b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb910003f30 0x7fb910004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406ee20 con 0x7fb910004d40 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406d8e0 con 0x7fb910003f30 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb91000b9c0 con 0x7fb9241454b0 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb9292fe640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb910003f30 0x7fb910004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb929aff640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 -- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 msgr2=0x7fb9100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb9241454b0 0x7fb9100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 -- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb910003f30 msgr2=0x7fb910004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb910003f30 0x7fb910004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.141+0000 7fb92a300640 1 -- 172.21.15.1:0/412422645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb910003520 con 0x7fb910004d40 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb92a300640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009f90 tx=0x7fb910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feb693a90e6b2fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c009c10 con 0x7fb910004d40 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c036070 con 0x7fb910004d40 2024-05-09T06:34:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb928afd640 1 -- 172.21.15.1:0/412422645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c031630 con 0x7fb910004d40 2024-05-09T06:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb92bd63640 1 -- 172.21.15.1:0/412422645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb910004d40 msgr2=0x7fb9100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.142+0000 7fb92bd63640 1 --2- 172.21.15.1:0/412422645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009f90 tx=0x7fb910005830 comp rx=0 tx=0).stop 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.143+0000 7fb92a300640 1 -- 172.21.15.1:0/412422645 reap_dead start 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.143+0000 7fb92bd63640 1 -- 172.21.15.1:0/412422645 shutdown_connections 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.143+0000 7fb92bd63640 1 -- 172.21.15.1:0/412422645 >> 172.21.15.1:0/412422645 conn(0x7fb9240cc4b0 msgr2=0x7fb9240cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.143+0000 7fb92bd63640 1 -- 172.21.15.1:0/412422645 shutdown_connections 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.143+0000 7fb92bd63640 1 -- 172.21.15.1:0/412422645 wait complete. 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.144+0000 7fb92bd63640 1 Processor -- start 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.144+0000 7fb92bd63640 1 -- start start 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 0x7fb9241e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 0x7fb92413dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406e260 con 0x7fb9241e10f0 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406e0e0 con 0x7fb92413d8a0 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240b4320 con 0x7fb910004d40 2024-05-09T06:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38188/0 (socket says 172.21.15.1:38188) 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 -- 172.21.15.1:0/524188444 learned_addr learned my addr 172.21.15.1:0/524188444 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb929aff640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 0x7fb9241e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92a300640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 0x7fb92413dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 -- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 msgr2=0x7fb9241e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 0x7fb9241e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 -- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 msgr2=0x7fb92413dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 0x7fb92413dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 -- 172.21.15.1:0/524188444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92415bcb0 con 0x7fb9241e10f0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92a300640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 0x7fb92413dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb929aff640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 0x7fb9241e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb9292fe640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb914001a70 tx=0x7fb9140077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400f9c0 con 0x7fb9241e10f0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb914018070 con 0x7fb9241e10f0 2024-05-09T06:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241e1c60 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.145+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400fda0 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.146+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241e2220 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.146+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9240752b0 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.147+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb91401d460 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.148+0000 7fb91a7fc640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.148+0000 7fb929aff640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.148+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb91408b120 con 0x7fb9241e10f0 2024-05-09T06:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.148+0000 7fb929aff640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb920009ab0 tx=0x7fb920008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.152+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9140599d0 con 0x7fb9241e10f0 2024-05-09T06:34:07.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.333+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb924068750 con 0x7fb90806be30 2024-05-09T06:34:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.339+0000 7fb91a7fc640 1 -- 172.21.15.1:0/524188444 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7fb924068750 con 0x7fb90806be30 2024-05-09T06:34:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:07.324 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:34:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 msgr2=0x7fb90806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb920009ab0 tx=0x7fb920008040 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 msgr2=0x7fb9241e1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb914001a70 tx=0x7fb9140077a0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 shutdown_connections 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb90806be30 0x7fb90806e2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb92413d8a0 0x7fb92413dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9241e10f0 0x7fb9241e1570 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 --2- 172.21.15.1:0/524188444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb910004d40 0x7fb9241e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 >> 172.21.15.1:0/524188444 conn(0x7fb9240cc4b0 msgr2=0x7fb924071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 shutdown_connections 2024-05-09T06:34:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.343+0000 7fb92bd63640 1 -- 172.21.15.1:0/524188444 wait complete. 2024-05-09T06:34:07.335 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-09T06:34:07.158732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259168,"kb_used_data":9712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715552,"statfs":{"total":773094113280,"available":772828725248,"internally_reserved":0,"allocated":9945088,"data_stored":5197218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856752"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314422+0000","last_change":"2024-05-09T06:34:02.314422+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314422+0000","last_undegraded":"2024-05-09T06:34:02.314422+0000","last_fullsized":"2024-05-09T06:34:02.314422+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320632+0000","last_change":"2024-05-09T06:34:02.320632+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320632+0000","last_undegraded":"2024-05-09T06:34:02.320632+0000","last_fullsized":"2024-05-09T06:34:02.320632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313945+0000","last_change":"2024-05-09T06:34:02.313945+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313945+0000","last_undegraded":"2024-05-09T06:34:02.313945+0000","last_fullsized":"2024-05-09T06:34:02.313945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318875+0000","last_change":"2024-05-09T06:34:02.318875+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318875+0000","last_undegraded":"2024-05-09T06:34:02.318875+0000","last_fullsized":"2024-05-09T06:34:02.318875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308592+0000","last_change":"2024-05-09T06:34:02.308592+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308592+0000","last_undegraded":"2024-05-09T06:34:02.308592+0000","last_fullsized":"2024-05-09T06:34:02.308592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316982+0000","last_change":"2024-05-09T06:34:02.316982+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316982+0000","last_undegraded":"2024-05-09T06:34:02.316982+0000","last_fullsized":"2024-05-09T06:34:02.316982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313987+0000","last_change":"2024-05-09T06:34:02.313987+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313987+0000","last_undegraded":"2024-05-09T06:34:02.313987+0000","last_fullsized":"2024-05-09T06:34:02.313987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320266+0000","last_change":"2024-05-09T06:34:02.320266+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320266+0000","last_undegraded":"2024-05-09T06:34:02.320266+0000","last_fullsized":"2024-05-09T06:34:02.320266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307924+0000","last_change":"2024-05-09T06:34:02.307924+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307924+0000","last_undegraded":"2024-05-09T06:34:02.307924+0000","last_fullsized":"2024-05-09T06:34:02.307924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315657+0000","last_change":"2024-05-09T06:34:02.315657+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315657+0000","last_undegraded":"2024-05-09T06:34:02.315657+0000","last_fullsized":"2024-05-09T06:34:02.315657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308933+0000","last_change":"2024-05-09T06:34:02.308933+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308933+0000","last_undegraded":"2024-05-09T06:34:02.308933+0000","last_fullsized":"2024-05-09T06:34:02.308933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312738+0000","last_change":"2024-05-09T06:34:02.312738+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312738+0000","last_undegraded":"2024-05-09T06:34:02.312738+0000","last_fullsized":"2024-05-09T06:34:02.312738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321819+0000","last_change":"2024-05-09T06:34:02.321819+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321819+0000","last_undegraded":"2024-05-09T06:34:02.321819+0000","last_fullsized":"2024-05-09T06:34:02.321819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310310+0000","last_change":"2024-05-09T06:34:02.310310+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310310+0000","last_undegraded":"2024-05-09T06:34:02.310310+0000","last_fullsized":"2024-05-09T06:34:02.310310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314242+0000","last_change":"2024-05-09T06:34:02.314242+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314242+0000","last_undegraded":"2024-05-09T06:34:02.314242+0000","last_fullsized":"2024-05-09T06:34:02.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316880+0000","last_change":"2024-05-09T06:34:02.316880+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316880+0000","last_undegraded":"2024-05-09T06:34:02.316880+0000","last_fullsized":"2024-05-09T06:34:02.316880+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316455+0000","last_change":"2024-05-09T06:34:02.316455+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316455+0000","last_undegraded":"2024-05-09T06:34:02.316455+0000","last_fullsized":"2024-05-09T06:34:02.316455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313420+0000","last_change":"2024-05-09T06:34:02.313420+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313420+0000","last_undegraded":"2024-05-09T06:34:02.313420+0000","last_fullsized":"2024-05-09T06:34:02.313420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308424+0000","last_change":"2024-05-09T06:34:02.308424+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308424+0000","last_undegraded":"2024-05-09T06:34:02.308424+0000","last_fullsized":"2024-05-09T06:34:02.308424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315522+0000","last_change":"2024-05-09T06:34:02.315522+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315522+0000","last_undegraded":"2024-05-09T06:34:02.315522+0000","last_fullsized":"2024-05-09T06:34:02.315522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309601+0000","last_change":"2024-05-09T06:34:02.309601+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309601+0000","last_undegraded":"2024-05-09T06:34:02.309601+0000","last_fullsized":"2024-05-09T06:34:02.309601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309739+0000","last_change":"2024-05-09T06:34:02.309739+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309739+0000","last_undegraded":"2024-05-09T06:34:02.309739+0000","last_fullsized":"2024-05-09T06:34:02.309739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312820+0000","last_change":"2024-05-09T06:34:02.312820+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312820+0000","last_undegraded":"2024-05-09T06:34:02.312820+0000","last_fullsized":"2024-05-09T06:34:02.312820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313125+0000","last_change":"2024-05-09T06:34:02.313125+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313125+0000","last_undegraded":"2024-05-09T06:34:02.313125+0000","last_fullsized":"2024-05-09T06:34:02.313125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321499+0000","last_change":"2024-05-09T06:34:02.321499+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321499+0000","last_undegraded":"2024-05-09T06:34:02.321499+0000","last_fullsized":"2024-05-09T06:34:02.321499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310029+0000","last_change":"2024-05-09T06:34:02.310029+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310029+0000","last_undegraded":"2024-05-09T06:34:02.310029+0000","last_fullsized":"2024-05-09T06:34:02.310029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312353+0000","last_change":"2024-05-09T06:34:02.312353+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312353+0000","last_undegraded":"2024-05-09T06:34:02.312353+0000","last_fullsized":"2024-05-09T06:34:02.312353+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315046+0000","last_change":"2024-05-09T06:34:02.315046+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315046+0000","last_undegraded":"2024-05-09T06:34:02.315046+0000","last_fullsized":"2024-05-09T06:34:02.315046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314059+0000","last_change":"2024-05-09T06:34:02.314059+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314059+0000","last_undegraded":"2024-05-09T06:34:02.314059+0000","last_fullsized":"2024-05-09T06:34:02.314059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309025+0000","last_change":"2024-05-09T06:34:02.309025+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309025+0000","last_undegraded":"2024-05-09T06:34:02.309025+0000","last_fullsized":"2024-05-09T06:34:02.309025+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320197+0000","last_change":"2024-05-09T06:34:02.320197+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320197+0000","last_undegraded":"2024-05-09T06:34:02.320197+0000","last_fullsized":"2024-05-09T06:34:02.320197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315167+0000","last_change":"2024-05-09T06:34:02.315167+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315167+0000","last_undegraded":"2024-05-09T06:34:02.315167+0000","last_fullsized":"2024-05-09T06:34:02.315167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315036+0000","last_change":"2024-05-09T06:34:02.315036+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315036+0000","last_undegraded":"2024-05-09T06:34:02.315036+0000","last_fullsized":"2024-05-09T06:34:02.315036+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.321306+0000","last_change":"2024-05-09T06:34:02.315932+0000","last_active":"2024-05-09T06:34:02.321306+0000","last_peered":"2024-05-09T06:34:02.321306+0000","last_clean":"2024-05-09T06:34:02.321306+0000","last_became_active":"2024-05-09T06:34:02.315556+0000","last_became_peered":"2024-05-09T06:34:02.315556+0000","last_unstale":"2024-05-09T06:34:02.321306+0000","last_undegraded":"2024-05-09T06:34:02.321306+0000","last_fullsized":"2024-05-09T06:34:02.321306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:50.355406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315873+0000","last_change":"2024-05-09T06:34:02.315873+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315873+0000","last_undegraded":"2024-05-09T06:34:02.315873+0000","last_fullsized":"2024-05-09T06:34:02.315873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325355+0000","last_change":"2024-05-09T06:34:02.317231+0000","last_active":"2024-05-09T06:34:02.325355+0000","last_peered":"2024-05-09T06:34:02.325355+0000","last_clean":"2024-05-09T06:34:02.325355+0000","last_became_active":"2024-05-09T06:34:02.316853+0000","last_became_peered":"2024-05-09T06:34:02.316853+0000","last_unstale":"2024-05-09T06:34:02.325355+0000","last_undegraded":"2024-05-09T06:34:02.325355+0000","last_fullsized":"2024-05-09T06:34:02.325355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:47.234272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314308+0000","last_change":"2024-05-09T06:34:02.314308+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314308+0000","last_undegraded":"2024-05-09T06:34:02.314308+0000","last_fullsized":"2024-05-09T06:34:02.314308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325501+0000","last_change":"2024-05-09T06:34:02.322212+0000","last_active":"2024-05-09T06:34:02.325501+0000","last_peered":"2024-05-09T06:34:02.325501+0000","last_clean":"2024-05-09T06:34:02.325501+0000","last_became_active":"2024-05-09T06:34:02.321940+0000","last_became_peered":"2024-05-09T06:34:02.321940+0000","last_unstale":"2024-05-09T06:34:02.325501+0000","last_undegraded":"2024-05-09T06:34:02.325501+0000","last_fullsized":"2024-05-09T06:34:02.325501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314090+0000","last_change":"2024-05-09T06:34:02.314090+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314090+0000","last_undegraded":"2024-05-09T06:34:02.314090+0000","last_fullsized":"2024-05-09T06:34:02.314090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.334420+0000","last_change":"2024-05-09T06:34:02.328490+0000","last_active":"2024-05-09T06:34:02.334420+0000","last_peered":"2024-05-09T06:34:02.334420+0000","last_clean":"2024-05-09T06:34:02.334420+0000","last_became_active":"2024-05-09T06:34:02.328118+0000","last_became_peered":"2024-05-09T06:34:02.328118+0000","last_unstale":"2024-05-09T06:34:02.334420+0000","last_undegraded":"2024-05-09T06:34:02.334420+0000","last_fullsized":"2024-05-09T06:34:02.334420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319876+0000","last_change":"2024-05-09T06:34:02.319876+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319876+0000","last_undegraded":"2024-05-09T06:34:02.319876+0000","last_fullsized":"2024-05-09T06:34:02.319876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325837+0000","last_change":"2024-05-09T06:34:02.320157+0000","last_active":"2024-05-09T06:34:02.325837+0000","last_peered":"2024-05-09T06:34:02.325837+0000","last_clean":"2024-05-09T06:34:02.325837+0000","last_became_active":"2024-05-09T06:34:02.319814+0000","last_became_peered":"2024-05-09T06:34:02.319814+0000","last_unstale":"2024-05-09T06:34:02.325837+0000","last_undegraded":"2024-05-09T06:34:02.325837+0000","last_fullsized":"2024-05-09T06:34:02.325837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:05.426920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312032+0000","last_change":"2024-05-09T06:34:02.312032+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312032+0000","last_undegraded":"2024-05-09T06:34:02.312032+0000","last_fullsized":"2024-05-09T06:34:02.312032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330331+0000","last_change":"2024-05-09T06:34:02.329390+0000","last_active":"2024-05-09T06:34:02.330331+0000","last_peered":"2024-05-09T06:34:02.330331+0000","last_clean":"2024-05-09T06:34:02.330331+0000","last_became_active":"2024-05-09T06:34:02.329108+0000","last_became_peered":"2024-05-09T06:34:02.329108+0000","last_unstale":"2024-05-09T06:34:02.330331+0000","last_undegraded":"2024-05-09T06:34:02.330331+0000","last_fullsized":"2024-05-09T06:34:02.330331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:08.397934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315492+0000","last_change":"2024-05-09T06:34:02.315492+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315492+0000","last_undegraded":"2024-05-09T06:34:02.315492+0000","last_fullsized":"2024-05-09T06:34:02.315492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326273+0000","last_change":"2024-05-09T06:34:02.324159+0000","last_active":"2024-05-09T06:34:02.326273+0000","last_peered":"2024-05-09T06:34:02.326273+0000","last_clean":"2024-05-09T06:34:02.326273+0000","last_became_active":"2024-05-09T06:34:02.323875+0000","last_became_peered":"2024-05-09T06:34:02.323875+0000","last_unstale":"2024-05-09T06:34:02.326273+0000","last_undegraded":"2024-05-09T06:34:02.326273+0000","last_fullsized":"2024-05-09T06:34:02.326273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316074+0000","last_change":"2024-05-09T06:34:02.316074+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316074+0000","last_undegraded":"2024-05-09T06:34:02.316074+0000","last_fullsized":"2024-05-09T06:34:02.316074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326331+0000","last_change":"2024-05-09T06:34:02.324681+0000","last_active":"2024-05-09T06:34:02.326331+0000","last_peered":"2024-05-09T06:34:02.326331+0000","last_clean":"2024-05-09T06:34:02.326331+0000","last_became_active":"2024-05-09T06:34:02.324383+0000","last_became_peered":"2024-05-09T06:34:02.324383+0000","last_unstale":"2024-05-09T06:34:02.326331+0000","last_undegraded":"2024-05-09T06:34:02.326331+0000","last_fullsized":"2024-05-09T06:34:02.326331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307911+0000","last_change":"2024-05-09T06:34:02.307911+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307911+0000","last_undegraded":"2024-05-09T06:34:02.307911+0000","last_fullsized":"2024-05-09T06:34:02.307911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328828+0000","last_change":"2024-05-09T06:34:02.322716+0000","last_active":"2024-05-09T06:34:02.328828+0000","last_peered":"2024-05-09T06:34:02.328828+0000","last_clean":"2024-05-09T06:34:02.328828+0000","last_became_active":"2024-05-09T06:34:02.322190+0000","last_became_peered":"2024-05-09T06:34:02.322190+0000","last_unstale":"2024-05-09T06:34:02.328828+0000","last_undegraded":"2024-05-09T06:34:02.328828+0000","last_fullsized":"2024-05-09T06:34:02.328828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315128+0000","last_change":"2024-05-09T06:34:02.315128+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315128+0000","last_undegraded":"2024-05-09T06:34:02.315128+0000","last_fullsized":"2024-05-09T06:34:02.315128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329063+0000","last_change":"2024-05-09T06:34:02.328152+0000","last_active":"2024-05-09T06:34:02.329063+0000","last_peered":"2024-05-09T06:34:02.329063+0000","last_clean":"2024-05-09T06:34:02.329063+0000","last_became_active":"2024-05-09T06:34:02.327851+0000","last_became_peered":"2024-05-09T06:34:02.327851+0000","last_unstale":"2024-05-09T06:34:02.329063+0000","last_undegraded":"2024-05-09T06:34:02.329063+0000","last_fullsized":"2024-05-09T06:34:02.329063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312609+0000","last_change":"2024-05-09T06:34:02.312609+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312609+0000","last_undegraded":"2024-05-09T06:34:02.312609+0000","last_fullsized":"2024-05-09T06:34:02.312609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325434+0000","last_change":"2024-05-09T06:34:02.318316+0000","last_active":"2024-05-09T06:34:02.325434+0000","last_peered":"2024-05-09T06:34:02.325434+0000","last_clean":"2024-05-09T06:34:02.325434+0000","last_became_active":"2024-05-09T06:34:02.318030+0000","last_became_peered":"2024-05-09T06:34:02.318030+0000","last_unstale":"2024-05-09T06:34:02.325434+0000","last_undegraded":"2024-05-09T06:34:02.325434+0000","last_fullsized":"2024-05-09T06:34:02.325434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:15.498412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318092+0000","last_change":"2024-05-09T06:34:02.318092+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318092+0000","last_undegraded":"2024-05-09T06:34:02.318092+0000","last_fullsized":"2024-05-09T06:34:02.318092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323563+0000","last_change":"2024-05-09T06:34:02.318480+0000","last_active":"2024-05-09T06:34:02.323563+0000","last_peered":"2024-05-09T06:34:02.323563+0000","last_clean":"2024-05-09T06:34:02.323563+0000","last_became_active":"2024-05-09T06:34:02.318175+0000","last_became_peered":"2024-05-09T06:34:02.318175+0000","last_unstale":"2024-05-09T06:34:02.323563+0000","last_undegraded":"2024-05-09T06:34:02.323563+0000","last_fullsized":"2024-05-09T06:34:02.323563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316864+0000","last_change":"2024-05-09T06:34:02.316864+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316864+0000","last_undegraded":"2024-05-09T06:34:02.316864+0000","last_fullsized":"2024-05-09T06:34:02.316864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329491+0000","last_change":"2024-05-09T06:34:02.325683+0000","last_active":"2024-05-09T06:34:02.329491+0000","last_peered":"2024-05-09T06:34:02.329491+0000","last_clean":"2024-05-09T06:34:02.329491+0000","last_became_active":"2024-05-09T06:34:02.325346+0000","last_became_peered":"2024-05-09T06:34:02.325346+0000","last_unstale":"2024-05-09T06:34:02.329491+0000","last_undegraded":"2024-05-09T06:34:02.329491+0000","last_fullsized":"2024-05-09T06:34:02.329491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307720+0000","last_change":"2024-05-09T06:34:02.307720+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307720+0000","last_undegraded":"2024-05-09T06:34:02.307720+0000","last_fullsized":"2024-05-09T06:34:02.307720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323742+0000","last_change":"2024-05-09T06:34:02.318607+0000","last_active":"2024-05-09T06:34:02.323742+0000","last_peered":"2024-05-09T06:34:02.323742+0000","last_clean":"2024-05-09T06:34:02.323742+0000","last_became_active":"2024-05-09T06:34:02.318297+0000","last_became_peered":"2024-05-09T06:34:02.318297+0000","last_unstale":"2024-05-09T06:34:02.323742+0000","last_undegraded":"2024-05-09T06:34:02.323742+0000","last_fullsized":"2024-05-09T06:34:02.323742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:25.837322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311415+0000","last_change":"2024-05-09T06:34:02.311415+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311415+0000","last_undegraded":"2024-05-09T06:34:02.311415+0000","last_fullsized":"2024-05-09T06:34:02.311415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330020+0000","last_change":"2024-05-09T06:34:02.326169+0000","last_active":"2024-05-09T06:34:02.330020+0000","last_peered":"2024-05-09T06:34:02.330020+0000","last_clean":"2024-05-09T06:34:02.330020+0000","last_became_active":"2024-05-09T06:34:02.325706+0000","last_became_peered":"2024-05-09T06:34:02.325706+0000","last_unstale":"2024-05-09T06:34:02.330020+0000","last_undegraded":"2024-05-09T06:34:02.330020+0000","last_fullsized":"2024-05-09T06:34:02.330020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:28.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312364+0000","last_change":"2024-05-09T06:34:02.312364+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312364+0000","last_undegraded":"2024-05-09T06:34:02.312364+0000","last_fullsized":"2024-05-09T06:34:02.312364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330677+0000","last_change":"2024-05-09T06:34:02.330110+0000","last_active":"2024-05-09T06:34:02.330677+0000","last_peered":"2024-05-09T06:34:02.330677+0000","last_clean":"2024-05-09T06:34:02.330677+0000","last_became_active":"2024-05-09T06:34:02.329847+0000","last_became_peered":"2024-05-09T06:34:02.329847+0000","last_unstale":"2024-05-09T06:34:02.330677+0000","last_undegraded":"2024-05-09T06:34:02.330677+0000","last_fullsized":"2024-05-09T06:34:02.330677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:16.125143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314602+0000","last_change":"2024-05-09T06:34:02.314602+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314602+0000","last_undegraded":"2024-05-09T06:34:02.314602+0000","last_fullsized":"2024-05-09T06:34:02.314602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327482+0000","last_change":"2024-05-09T06:34:02.322056+0000","last_active":"2024-05-09T06:34:02.327482+0000","last_peered":"2024-05-09T06:34:02.327482+0000","last_clean":"2024-05-09T06:34:02.327482+0000","last_became_active":"2024-05-09T06:34:02.321765+0000","last_became_peered":"2024-05-09T06:34:02.321765+0000","last_unstale":"2024-05-09T06:34:02.327482+0000","last_undegraded":"2024-05-09T06:34:02.327482+0000","last_fullsized":"2024-05-09T06:34:02.327482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:12.987046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312055+0000","last_change":"2024-05-09T06:34:02.312055+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312055+0000","last_undegraded":"2024-05-09T06:34:02.312055+0000","last_fullsized":"2024-05-09T06:34:02.312055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331487+0000","last_change":"2024-05-09T06:34:02.327732+0000","last_active":"2024-05-09T06:34:02.331487+0000","last_peered":"2024-05-09T06:34:02.331487+0000","last_clean":"2024-05-09T06:34:02.331487+0000","last_became_active":"2024-05-09T06:34:02.327311+0000","last_became_peered":"2024-05-09T06:34:02.327311+0000","last_unstale":"2024-05-09T06:34:02.331487+0000","last_undegraded":"2024-05-09T06:34:02.331487+0000","last_fullsized":"2024-05-09T06:34:02.331487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308418+0000","last_change":"2024-05-09T06:34:02.308418+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308418+0000","last_undegraded":"2024-05-09T06:34:02.308418+0000","last_fullsized":"2024-05-09T06:34:02.308418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331858+0000","last_change":"2024-05-09T06:34:02.329189+0000","last_active":"2024-05-09T06:34:02.331858+0000","last_peered":"2024-05-09T06:34:02.331858+0000","last_clean":"2024-05-09T06:34:02.331858+0000","last_became_active":"2024-05-09T06:34:02.328869+0000","last_became_peered":"2024-05-09T06:34:02.328869+0000","last_unstale":"2024-05-09T06:34:02.331858+0000","last_undegraded":"2024-05-09T06:34:02.331858+0000","last_fullsized":"2024-05-09T06:34:02.331858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:55.195087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313072+0000","last_change":"2024-05-09T06:34:02.313072+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313072+0000","last_undegraded":"2024-05-09T06:34:02.313072+0000","last_fullsized":"2024-05-09T06:34:02.313072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329582+0000","last_change":"2024-05-09T06:34:02.327306+0000","last_active":"2024-05-09T06:34:02.329582+0000","last_peered":"2024-05-09T06:34:02.329582+0000","last_clean":"2024-05-09T06:34:02.329582+0000","last_became_active":"2024-05-09T06:34:02.326990+0000","last_became_peered":"2024-05-09T06:34:02.326990+0000","last_unstale":"2024-05-09T06:34:02.329582+0000","last_undegraded":"2024-05-09T06:34:02.329582+0000","last_fullsized":"2024-05-09T06:34:02.329582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319661+0000","last_change":"2024-05-09T06:34:02.319661+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319661+0000","last_undegraded":"2024-05-09T06:34:02.319661+0000","last_fullsized":"2024-05-09T06:34:02.319661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329784+0000","last_change":"2024-05-09T06:34:02.328106+0000","last_active":"2024-05-09T06:34:02.329784+0000","last_peered":"2024-05-09T06:34:02.329784+0000","last_clean":"2024-05-09T06:34:02.329784+0000","last_became_active":"2024-05-09T06:34:02.327820+0000","last_became_peered":"2024-05-09T06:34:02.327820+0000","last_unstale":"2024-05-09T06:34:02.329784+0000","last_undegraded":"2024-05-09T06:34:02.329784+0000","last_fullsized":"2024-05-09T06:34:02.329784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:33.383852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318556+0000","last_change":"2024-05-09T06:34:02.318556+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318556+0000","last_undegraded":"2024-05-09T06:34:02.318556+0000","last_fullsized":"2024-05-09T06:34:02.318556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329527+0000","last_change":"2024-05-09T06:34:02.327204+0000","last_active":"2024-05-09T06:34:02.329527+0000","last_peered":"2024-05-09T06:34:02.329527+0000","last_clean":"2024-05-09T06:34:02.329527+0000","last_became_active":"2024-05-09T06:34:02.326876+0000","last_became_peered":"2024-05-09T06:34:02.326876+0000","last_unstale":"2024-05-09T06:34:02.329527+0000","last_undegraded":"2024-05-09T06:34:02.329527+0000","last_fullsized":"2024-05-09T06:34:02.329527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314614+0000","last_change":"2024-05-09T06:34:02.314614+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314614+0000","last_undegraded":"2024-05-09T06:34:02.314614+0000","last_fullsized":"2024-05-09T06:34:02.314614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328608+0000","last_change":"2024-05-09T06:34:02.327053+0000","last_active":"2024-05-09T06:34:02.328608+0000","last_peered":"2024-05-09T06:34:02.328608+0000","last_clean":"2024-05-09T06:34:02.328608+0000","last_became_active":"2024-05-09T06:34:02.326751+0000","last_became_peered":"2024-05-09T06:34:02.326751+0000","last_unstale":"2024-05-09T06:34:02.328608+0000","last_undegraded":"2024-05-09T06:34:02.328608+0000","last_fullsized":"2024-05-09T06:34:02.328608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:11.750079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313077+0000","last_change":"2024-05-09T06:34:02.313077+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313077+0000","last_undegraded":"2024-05-09T06:34:02.313077+0000","last_fullsized":"2024-05-09T06:34:02.313077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331465+0000","last_change":"2024-05-09T06:34:02.326680+0000","last_active":"2024-05-09T06:34:02.331465+0000","last_peered":"2024-05-09T06:34:02.331465+0000","last_clean":"2024-05-09T06:34:02.331465+0000","last_became_active":"2024-05-09T06:34:02.326353+0000","last_became_peered":"2024-05-09T06:34:02.326353+0000","last_unstale":"2024-05-09T06:34:02.331465+0000","last_undegraded":"2024-05-09T06:34:02.331465+0000","last_fullsized":"2024-05-09T06:34:02.331465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311571+0000","last_change":"2024-05-09T06:34:02.311571+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311571+0000","last_undegraded":"2024-05-09T06:34:02.311571+0000","last_fullsized":"2024-05-09T06:34:02.311571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329518+0000","last_change":"2024-05-09T06:34:02.327790+0000","last_active":"2024-05-09T06:34:02.329518+0000","last_peered":"2024-05-09T06:34:02.329518+0000","last_clean":"2024-05-09T06:34:02.329518+0000","last_became_active":"2024-05-09T06:34:02.327409+0000","last_became_peered":"2024-05-09T06:34:02.327409+0000","last_unstale":"2024-05-09T06:34:02.329518+0000","last_undegraded":"2024-05-09T06:34:02.329518+0000","last_fullsized":"2024-05-09T06:34:02.329518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.828382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311765+0000","last_change":"2024-05-09T06:34:02.311765+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311765+0000","last_undegraded":"2024-05-09T06:34:02.311765+0000","last_fullsized":"2024-05-09T06:34:02.311765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.317202+0000","last_change":"2024-05-09T06:34:02.314084+0000","last_active":"2024-05-09T06:34:02.317202+0000","last_peered":"2024-05-09T06:34:02.317202+0000","last_clean":"2024-05-09T06:34:02.317202+0000","last_became_active":"2024-05-09T06:34:02.313093+0000","last_became_peered":"2024-05-09T06:34:02.313093+0000","last_unstale":"2024-05-09T06:34:02.317202+0000","last_undegraded":"2024-05-09T06:34:02.317202+0000","last_fullsized":"2024-05-09T06:34:02.317202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:33.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321022+0000","last_change":"2024-05-09T06:34:02.321022+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321022+0000","last_undegraded":"2024-05-09T06:34:02.321022+0000","last_fullsized":"2024-05-09T06:34:02.321022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325577+0000","last_change":"2024-05-09T06:34:02.319167+0000","last_active":"2024-05-09T06:34:02.325577+0000","last_peered":"2024-05-09T06:34:02.325577+0000","last_clean":"2024-05-09T06:34:02.325577+0000","last_became_active":"2024-05-09T06:34:02.318846+0000","last_became_peered":"2024-05-09T06:34:02.318846+0000","last_unstale":"2024-05-09T06:34:02.325577+0000","last_undegraded":"2024-05-09T06:34:02.325577+0000","last_fullsized":"2024-05-09T06:34:02.325577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:19.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311082+0000","last_change":"2024-05-09T06:34:02.311082+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311082+0000","last_undegraded":"2024-05-09T06:34:02.311082+0000","last_fullsized":"2024-05-09T06:34:02.311082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.324239+0000","last_change":"2024-05-09T06:34:02.317990+0000","last_active":"2024-05-09T06:34:02.324239+0000","last_peered":"2024-05-09T06:34:02.324239+0000","last_clean":"2024-05-09T06:34:02.324239+0000","last_became_active":"2024-05-09T06:34:02.317688+0000","last_became_peered":"2024-05-09T06:34:02.317688+0000","last_unstale":"2024-05-09T06:34:02.324239+0000","last_undegraded":"2024-05-09T06:34:02.324239+0000","last_fullsized":"2024-05-09T06:34:02.324239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312002+0000","last_change":"2024-05-09T06:34:02.312002+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312002+0000","last_undegraded":"2024-05-09T06:34:02.312002+0000","last_fullsized":"2024-05-09T06:34:02.312002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329509+0000","last_change":"2024-05-09T06:34:02.326540+0000","last_active":"2024-05-09T06:34:02.329509+0000","last_peered":"2024-05-09T06:34:02.329509+0000","last_clean":"2024-05-09T06:34:02.329509+0000","last_became_active":"2024-05-09T06:34:02.326253+0000","last_became_peered":"2024-05-09T06:34:02.326253+0000","last_unstale":"2024-05-09T06:34:02.329509+0000","last_undegraded":"2024-05-09T06:34:02.329509+0000","last_fullsized":"2024-05-09T06:34:02.329509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312075+0000","last_change":"2024-05-09T06:34:02.312075+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312075+0000","last_undegraded":"2024-05-09T06:34:02.312075+0000","last_fullsized":"2024-05-09T06:34:02.312075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327577+0000","last_change":"2024-05-09T06:34:02.323636+0000","last_active":"2024-05-09T06:34:02.327577+0000","last_peered":"2024-05-09T06:34:02.327577+0000","last_clean":"2024-05-09T06:34:02.327577+0000","last_became_active":"2024-05-09T06:34:02.323275+0000","last_became_peered":"2024-05-09T06:34:02.323275+0000","last_unstale":"2024-05-09T06:34:02.327577+0000","last_undegraded":"2024-05-09T06:34:02.327577+0000","last_fullsized":"2024-05-09T06:34:02.327577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310800+0000","last_change":"2024-05-09T06:34:02.310800+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310800+0000","last_undegraded":"2024-05-09T06:34:02.310800+0000","last_fullsized":"2024-05-09T06:34:02.310800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331509+0000","last_change":"2024-05-09T06:34:02.328232+0000","last_active":"2024-05-09T06:34:02.331509+0000","last_peered":"2024-05-09T06:34:02.331509+0000","last_clean":"2024-05-09T06:34:02.331509+0000","last_became_active":"2024-05-09T06:34:02.327886+0000","last_became_peered":"2024-05-09T06:34:02.327886+0000","last_unstale":"2024-05-09T06:34:02.331509+0000","last_undegraded":"2024-05-09T06:34:02.331509+0000","last_fullsized":"2024-05-09T06:34:02.331509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315380+0000","last_change":"2024-05-09T06:34:02.315380+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315380+0000","last_undegraded":"2024-05-09T06:34:02.315380+0000","last_fullsized":"2024-05-09T06:34:02.315380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331857+0000","last_change":"2024-05-09T06:34:02.331212+0000","last_active":"2024-05-09T06:34:02.331857+0000","last_peered":"2024-05-09T06:34:02.331857+0000","last_clean":"2024-05-09T06:34:02.331857+0000","last_became_active":"2024-05-09T06:34:02.330939+0000","last_became_peered":"2024-05-09T06:34:02.330939+0000","last_unstale":"2024-05-09T06:34:02.331857+0000","last_undegraded":"2024-05-09T06:34:02.331857+0000","last_fullsized":"2024-05-09T06:34:02.331857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:30:48.018600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.316064+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:34:02.316064+0000","last_peered":"2024-05-09T06:34:02.316064+0000","last_clean":"2024-05-09T06:34:02.316064+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:34:02.316064+0000","last_undegraded":"2024-05-09T06:34:02.316064+0000","last_fullsized":"2024-05-09T06:34:02.316064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1087428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":497060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1094095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:07.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:34:07.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.498+0000 7fa8a87ee640 1 Processor -- start 2024-05-09T06:34:07.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.498+0000 7fa8a87ee640 1 -- start start 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.499+0000 7fa8a87ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8a006e6d0 0x7fa8a006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.499+0000 7fa8a87ee640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 0x7fa8a00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.499+0000 7fa8a87ee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8a00ce0e0 con 0x7fa8a00cf180 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a87ee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a00ce3c0 con 0x7fa8a006e6d0 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a87ee640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a00ce6a0 con 0x7fa8a00b3300 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8a006e6d0 0x7fa8a006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa8a006e6d0 0x7fa8a006eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38198/0 (socket says 172.21.15.1:38198) 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a5d89640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa8a00cf180 0x7fa8a00cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57574/0 (socket says 172.21.15.1:57574) 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 -- 172.21.15.1:0/1556819827 learned_addr learned my addr 172.21.15.1:0/1556819827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a6d8b640 1 --2- 172.21.15.1:0/1556819827 >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 0x7fa8a00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 -- 172.21.15.1:0/1556819827 >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 msgr2=0x7fa8a00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 --2- 172.21.15.1:0/1556819827 >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 0x7fa8a00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 -- 172.21.15.1:0/1556819827 >> v1:172.21.15.1:6789/0 conn(0x7fa8a00cf180 legacy=0x7fa8a00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a658a640 1 -- 172.21.15.1:0/1556819827 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a00ce980 con 0x7fa8a006e6d0 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a5588640 1 -- 172.21.15.1:0/1556819827 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657114392 0 0) 0x7fa8a00ce0e0 con 0x7fa8a00cf180 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.500+0000 7fa8a6d8b640 1 --2- 172.21.15.1:0/1556819827 >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 0x7fa8a00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a658a640 1 --2- 172.21.15.1:0/1556819827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a006e6d0 0x7fa8a006eab0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00b16f0 tx=0x7fa89802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4eb06a7f05ba183 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a5588640 1 -- 172.21.15.1:0/1556819827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898037070 con 0x7fa8a006e6d0 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a5588640 1 -- 172.21.15.1:0/1556819827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa898032440 con 0x7fa8a006e6d0 2024-05-09T06:34:07.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a5588640 1 -- 172.21.15.1:0/1556819827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898032810 con 0x7fa8a006e6d0 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a87ee640 1 -- 172.21.15.1:0/1556819827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a006e6d0 msgr2=0x7fa8a006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/1556819827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a006e6d0 0x7fa8a006eab0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00b16f0 tx=0x7fa89802a270 comp rx=0 tx=0).stop 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a87ee640 1 -- 172.21.15.1:0/1556819827 shutdown_connections 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/1556819827 >> v2:172.21.15.6:3300/0 conn(0x7fa8a00b3300 0x7fa8a00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.501+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/1556819827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a006e6d0 0x7fa8a006eab0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.502+0000 7fa8a87ee640 1 -- 172.21.15.1:0/1556819827 >> 172.21.15.1:0/1556819827 conn(0x7fa8a00cc4b0 msgr2=0x7fa8a00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.502+0000 7fa8a87ee640 1 -- 172.21.15.1:0/1556819827 shutdown_connections 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.502+0000 7fa8a87ee640 1 -- 172.21.15.1:0/1556819827 wait complete. 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 Processor -- start 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 -- start start 2024-05-09T06:34:07.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8a006e6d0 0x7fa8a0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 0x7fa8a01427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0070ed0 con 0x7fa8a00b3300 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006eff0 con 0x7fa8a00cf180 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.503+0000 7fa8a87ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a01447c0 con 0x7fa8a006e6d0 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38206/0 (socket says 172.21.15.1:38206) 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 -- 172.21.15.1:0/3678929239 learned_addr learned my addr 172.21.15.1:0/3678929239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a6d8b640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 0x7fa8a01427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a658a640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8a006e6d0 0x7fa8a0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 -- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8a006e6d0 msgr2=0x7fa8a0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8a006e6d0 0x7fa8a0159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 -- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 msgr2=0x7fa8a01427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 0x7fa8a01427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 -- 172.21.15.1:0/3678929239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a015ca50 con 0x7fa8a00b3300 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a6d8b640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 0x7fa8a01427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a5d89640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa89400c4b0 tx=0x7fa89400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894018070 con 0x7fa8a00b3300 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa894007500 con 0x7fa8a00b3300 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894004a20 con 0x7fa8a00b3300 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a015cd30 con 0x7fa8a00b3300 2024-05-09T06:34:07.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.504+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a015d2f0 con 0x7fa8a00b3300 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.505+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a00752b0 con 0x7fa8a00b3300 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.506+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa89401b070 con 0x7fa8a00b3300 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.506+0000 7fa89effd640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 0x7fa88c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.507+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fa89408a2e0 con 0x7fa8a00b3300 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.507+0000 7fa8a658a640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 0x7fa88c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.507+0000 7fa8a658a640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 0x7fa88c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa898034000 tx=0x7fa89802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.511+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa894058c50 con 0x7fa8a00b3300 2024-05-09T06:34:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.691+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8a006cf50 con 0x7fa88c06be30 2024-05-09T06:34:07.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.697+0000 7fa89effd640 1 -- 172.21.15.1:0/3678929239 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7fa8a006cf50 con 0x7fa88c06be30 2024-05-09T06:34:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:07.681 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 msgr2=0x7fa88c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 0x7fa88c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa898034000 tx=0x7fa89802e000 comp rx=0 tx=0).stop 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 msgr2=0x7fa8a0142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa89400c4b0 tx=0x7fa89400c970 comp rx=0 tx=0).stop 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 shutdown_connections 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa88c06be30 0x7fa88c06e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8a00cf180 0x7fa8a01427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8a00b3300 0x7fa8a0142280 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 --2- 172.21.15.1:0/3678929239 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8a006e6d0 0x7fa8a0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 >> 172.21.15.1:0/3678929239 conn(0x7fa8a00cc4b0 msgr2=0x7fa8a00b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 shutdown_connections 2024-05-09T06:34:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.701+0000 7fa8a87ee640 1 -- 172.21.15.1:0/3678929239 wait complete. 2024-05-09T06:34:07.692 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-09T06:34:07.158732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259168,"kb_used_data":9712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715552,"statfs":{"total":773094113280,"available":772828725248,"internally_reserved":0,"allocated":9945088,"data_stored":5197218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856752"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314422+0000","last_change":"2024-05-09T06:34:02.314422+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314422+0000","last_undegraded":"2024-05-09T06:34:02.314422+0000","last_fullsized":"2024-05-09T06:34:02.314422+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320632+0000","last_change":"2024-05-09T06:34:02.320632+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320632+0000","last_undegraded":"2024-05-09T06:34:02.320632+0000","last_fullsized":"2024-05-09T06:34:02.320632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313945+0000","last_change":"2024-05-09T06:34:02.313945+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313945+0000","last_undegraded":"2024-05-09T06:34:02.313945+0000","last_fullsized":"2024-05-09T06:34:02.313945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318875+0000","last_change":"2024-05-09T06:34:02.318875+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318875+0000","last_undegraded":"2024-05-09T06:34:02.318875+0000","last_fullsized":"2024-05-09T06:34:02.318875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308592+0000","last_change":"2024-05-09T06:34:02.308592+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308592+0000","last_undegraded":"2024-05-09T06:34:02.308592+0000","last_fullsized":"2024-05-09T06:34:02.308592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316982+0000","last_change":"2024-05-09T06:34:02.316982+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316982+0000","last_undegraded":"2024-05-09T06:34:02.316982+0000","last_fullsized":"2024-05-09T06:34:02.316982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313987+0000","last_change":"2024-05-09T06:34:02.313987+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313987+0000","last_undegraded":"2024-05-09T06:34:02.313987+0000","last_fullsized":"2024-05-09T06:34:02.313987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320266+0000","last_change":"2024-05-09T06:34:02.320266+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320266+0000","last_undegraded":"2024-05-09T06:34:02.320266+0000","last_fullsized":"2024-05-09T06:34:02.320266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307924+0000","last_change":"2024-05-09T06:34:02.307924+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307924+0000","last_undegraded":"2024-05-09T06:34:02.307924+0000","last_fullsized":"2024-05-09T06:34:02.307924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315657+0000","last_change":"2024-05-09T06:34:02.315657+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315657+0000","last_undegraded":"2024-05-09T06:34:02.315657+0000","last_fullsized":"2024-05-09T06:34:02.315657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308933+0000","last_change":"2024-05-09T06:34:02.308933+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308933+0000","last_undegraded":"2024-05-09T06:34:02.308933+0000","last_fullsized":"2024-05-09T06:34:02.308933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312738+0000","last_change":"2024-05-09T06:34:02.312738+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312738+0000","last_undegraded":"2024-05-09T06:34:02.312738+0000","last_fullsized":"2024-05-09T06:34:02.312738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321819+0000","last_change":"2024-05-09T06:34:02.321819+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321819+0000","last_undegraded":"2024-05-09T06:34:02.321819+0000","last_fullsized":"2024-05-09T06:34:02.321819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310310+0000","last_change":"2024-05-09T06:34:02.310310+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310310+0000","last_undegraded":"2024-05-09T06:34:02.310310+0000","last_fullsized":"2024-05-09T06:34:02.310310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314242+0000","last_change":"2024-05-09T06:34:02.314242+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314242+0000","last_undegraded":"2024-05-09T06:34:02.314242+0000","last_fullsized":"2024-05-09T06:34:02.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316880+0000","last_change":"2024-05-09T06:34:02.316880+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316880+0000","last_undegraded":"2024-05-09T06:34:02.316880+0000","last_fullsized":"2024-05-09T06:34:02.316880+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316455+0000","last_change":"2024-05-09T06:34:02.316455+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316455+0000","last_undegraded":"2024-05-09T06:34:02.316455+0000","last_fullsized":"2024-05-09T06:34:02.316455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313420+0000","last_change":"2024-05-09T06:34:02.313420+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313420+0000","last_undegraded":"2024-05-09T06:34:02.313420+0000","last_fullsized":"2024-05-09T06:34:02.313420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308424+0000","last_change":"2024-05-09T06:34:02.308424+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308424+0000","last_undegraded":"2024-05-09T06:34:02.308424+0000","last_fullsized":"2024-05-09T06:34:02.308424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315522+0000","last_change":"2024-05-09T06:34:02.315522+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315522+0000","last_undegraded":"2024-05-09T06:34:02.315522+0000","last_fullsized":"2024-05-09T06:34:02.315522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309601+0000","last_change":"2024-05-09T06:34:02.309601+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309601+0000","last_undegraded":"2024-05-09T06:34:02.309601+0000","last_fullsized":"2024-05-09T06:34:02.309601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309739+0000","last_change":"2024-05-09T06:34:02.309739+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309739+0000","last_undegraded":"2024-05-09T06:34:02.309739+0000","last_fullsized":"2024-05-09T06:34:02.309739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312820+0000","last_change":"2024-05-09T06:34:02.312820+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312820+0000","last_undegraded":"2024-05-09T06:34:02.312820+0000","last_fullsized":"2024-05-09T06:34:02.312820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313125+0000","last_change":"2024-05-09T06:34:02.313125+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313125+0000","last_undegraded":"2024-05-09T06:34:02.313125+0000","last_fullsized":"2024-05-09T06:34:02.313125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321499+0000","last_change":"2024-05-09T06:34:02.321499+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321499+0000","last_undegraded":"2024-05-09T06:34:02.321499+0000","last_fullsized":"2024-05-09T06:34:02.321499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310029+0000","last_change":"2024-05-09T06:34:02.310029+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310029+0000","last_undegraded":"2024-05-09T06:34:02.310029+0000","last_fullsized":"2024-05-09T06:34:02.310029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312353+0000","last_change":"2024-05-09T06:34:02.312353+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312353+0000","last_undegraded":"2024-05-09T06:34:02.312353+0000","last_fullsized":"2024-05-09T06:34:02.312353+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315046+0000","last_change":"2024-05-09T06:34:02.315046+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315046+0000","last_undegraded":"2024-05-09T06:34:02.315046+0000","last_fullsized":"2024-05-09T06:34:02.315046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314059+0000","last_change":"2024-05-09T06:34:02.314059+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314059+0000","last_undegraded":"2024-05-09T06:34:02.314059+0000","last_fullsized":"2024-05-09T06:34:02.314059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309025+0000","last_change":"2024-05-09T06:34:02.309025+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309025+0000","last_undegraded":"2024-05-09T06:34:02.309025+0000","last_fullsized":"2024-05-09T06:34:02.309025+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320197+0000","last_change":"2024-05-09T06:34:02.320197+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320197+0000","last_undegraded":"2024-05-09T06:34:02.320197+0000","last_fullsized":"2024-05-09T06:34:02.320197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315167+0000","last_change":"2024-05-09T06:34:02.315167+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315167+0000","last_undegraded":"2024-05-09T06:34:02.315167+0000","last_fullsized":"2024-05-09T06:34:02.315167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315036+0000","last_change":"2024-05-09T06:34:02.315036+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315036+0000","last_undegraded":"2024-05-09T06:34:02.315036+0000","last_fullsized":"2024-05-09T06:34:02.315036+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.321306+0000","last_change":"2024-05-09T06:34:02.315932+0000","last_active":"2024-05-09T06:34:02.321306+0000","last_peered":"2024-05-09T06:34:02.321306+0000","last_clean":"2024-05-09T06:34:02.321306+0000","last_became_active":"2024-05-09T06:34:02.315556+0000","last_became_peered":"2024-05-09T06:34:02.315556+0000","last_unstale":"2024-05-09T06:34:02.321306+0000","last_undegraded":"2024-05-09T06:34:02.321306+0000","last_fullsized":"2024-05-09T06:34:02.321306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:50.355406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315873+0000","last_change":"2024-05-09T06:34:02.315873+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315873+0000","last_undegraded":"2024-05-09T06:34:02.315873+0000","last_fullsized":"2024-05-09T06:34:02.315873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325355+0000","last_change":"2024-05-09T06:34:02.317231+0000","last_active":"2024-05-09T06:34:02.325355+0000","last_peered":"2024-05-09T06:34:02.325355+0000","last_clean":"2024-05-09T06:34:02.325355+0000","last_became_active":"2024-05-09T06:34:02.316853+0000","last_became_peered":"2024-05-09T06:34:02.316853+0000","last_unstale":"2024-05-09T06:34:02.325355+0000","last_undegraded":"2024-05-09T06:34:02.325355+0000","last_fullsized":"2024-05-09T06:34:02.325355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:47.234272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314308+0000","last_change":"2024-05-09T06:34:02.314308+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314308+0000","last_undegraded":"2024-05-09T06:34:02.314308+0000","last_fullsized":"2024-05-09T06:34:02.314308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325501+0000","last_change":"2024-05-09T06:34:02.322212+0000","last_active":"2024-05-09T06:34:02.325501+0000","last_peered":"2024-05-09T06:34:02.325501+0000","last_clean":"2024-05-09T06:34:02.325501+0000","last_became_active":"2024-05-09T06:34:02.321940+0000","last_became_peered":"2024-05-09T06:34:02.321940+0000","last_unstale":"2024-05-09T06:34:02.325501+0000","last_undegraded":"2024-05-09T06:34:02.325501+0000","last_fullsized":"2024-05-09T06:34:02.325501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314090+0000","last_change":"2024-05-09T06:34:02.314090+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314090+0000","last_undegraded":"2024-05-09T06:34:02.314090+0000","last_fullsized":"2024-05-09T06:34:02.314090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.334420+0000","last_change":"2024-05-09T06:34:02.328490+0000","last_active":"2024-05-09T06:34:02.334420+0000","last_peered":"2024-05-09T06:34:02.334420+0000","last_clean":"2024-05-09T06:34:02.334420+0000","last_became_active":"2024-05-09T06:34:02.328118+0000","last_became_peered":"2024-05-09T06:34:02.328118+0000","last_unstale":"2024-05-09T06:34:02.334420+0000","last_undegraded":"2024-05-09T06:34:02.334420+0000","last_fullsized":"2024-05-09T06:34:02.334420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319876+0000","last_change":"2024-05-09T06:34:02.319876+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319876+0000","last_undegraded":"2024-05-09T06:34:02.319876+0000","last_fullsized":"2024-05-09T06:34:02.319876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325837+0000","last_change":"2024-05-09T06:34:02.320157+0000","last_active":"2024-05-09T06:34:02.325837+0000","last_peered":"2024-05-09T06:34:02.325837+0000","last_clean":"2024-05-09T06:34:02.325837+0000","last_became_active":"2024-05-09T06:34:02.319814+0000","last_became_peered":"2024-05-09T06:34:02.319814+0000","last_unstale":"2024-05-09T06:34:02.325837+0000","last_undegraded":"2024-05-09T06:34:02.325837+0000","last_fullsized":"2024-05-09T06:34:02.325837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:05.426920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312032+0000","last_change":"2024-05-09T06:34:02.312032+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312032+0000","last_undegraded":"2024-05-09T06:34:02.312032+0000","last_fullsized":"2024-05-09T06:34:02.312032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330331+0000","last_change":"2024-05-09T06:34:02.329390+0000","last_active":"2024-05-09T06:34:02.330331+0000","last_peered":"2024-05-09T06:34:02.330331+0000","last_clean":"2024-05-09T06:34:02.330331+0000","last_became_active":"2024-05-09T06:34:02.329108+0000","last_became_peered":"2024-05-09T06:34:02.329108+0000","last_unstale":"2024-05-09T06:34:02.330331+0000","last_undegraded":"2024-05-09T06:34:02.330331+0000","last_fullsized":"2024-05-09T06:34:02.330331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:08.397934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315492+0000","last_change":"2024-05-09T06:34:02.315492+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315492+0000","last_undegraded":"2024-05-09T06:34:02.315492+0000","last_fullsized":"2024-05-09T06:34:02.315492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326273+0000","last_change":"2024-05-09T06:34:02.324159+0000","last_active":"2024-05-09T06:34:02.326273+0000","last_peered":"2024-05-09T06:34:02.326273+0000","last_clean":"2024-05-09T06:34:02.326273+0000","last_became_active":"2024-05-09T06:34:02.323875+0000","last_became_peered":"2024-05-09T06:34:02.323875+0000","last_unstale":"2024-05-09T06:34:02.326273+0000","last_undegraded":"2024-05-09T06:34:02.326273+0000","last_fullsized":"2024-05-09T06:34:02.326273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316074+0000","last_change":"2024-05-09T06:34:02.316074+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316074+0000","last_undegraded":"2024-05-09T06:34:02.316074+0000","last_fullsized":"2024-05-09T06:34:02.316074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326331+0000","last_change":"2024-05-09T06:34:02.324681+0000","last_active":"2024-05-09T06:34:02.326331+0000","last_peered":"2024-05-09T06:34:02.326331+0000","last_clean":"2024-05-09T06:34:02.326331+0000","last_became_active":"2024-05-09T06:34:02.324383+0000","last_became_peered":"2024-05-09T06:34:02.324383+0000","last_unstale":"2024-05-09T06:34:02.326331+0000","last_undegraded":"2024-05-09T06:34:02.326331+0000","last_fullsized":"2024-05-09T06:34:02.326331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307911+0000","last_change":"2024-05-09T06:34:02.307911+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307911+0000","last_undegraded":"2024-05-09T06:34:02.307911+0000","last_fullsized":"2024-05-09T06:34:02.307911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328828+0000","last_change":"2024-05-09T06:34:02.322716+0000","last_active":"2024-05-09T06:34:02.328828+0000","last_peered":"2024-05-09T06:34:02.328828+0000","last_clean":"2024-05-09T06:34:02.328828+0000","last_became_active":"2024-05-09T06:34:02.322190+0000","last_became_peered":"2024-05-09T06:34:02.322190+0000","last_unstale":"2024-05-09T06:34:02.328828+0000","last_undegraded":"2024-05-09T06:34:02.328828+0000","last_fullsized":"2024-05-09T06:34:02.328828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315128+0000","last_change":"2024-05-09T06:34:02.315128+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315128+0000","last_undegraded":"2024-05-09T06:34:02.315128+0000","last_fullsized":"2024-05-09T06:34:02.315128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329063+0000","last_change":"2024-05-09T06:34:02.328152+0000","last_active":"2024-05-09T06:34:02.329063+0000","last_peered":"2024-05-09T06:34:02.329063+0000","last_clean":"2024-05-09T06:34:02.329063+0000","last_became_active":"2024-05-09T06:34:02.327851+0000","last_became_peered":"2024-05-09T06:34:02.327851+0000","last_unstale":"2024-05-09T06:34:02.329063+0000","last_undegraded":"2024-05-09T06:34:02.329063+0000","last_fullsized":"2024-05-09T06:34:02.329063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312609+0000","last_change":"2024-05-09T06:34:02.312609+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312609+0000","last_undegraded":"2024-05-09T06:34:02.312609+0000","last_fullsized":"2024-05-09T06:34:02.312609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325434+0000","last_change":"2024-05-09T06:34:02.318316+0000","last_active":"2024-05-09T06:34:02.325434+0000","last_peered":"2024-05-09T06:34:02.325434+0000","last_clean":"2024-05-09T06:34:02.325434+0000","last_became_active":"2024-05-09T06:34:02.318030+0000","last_became_peered":"2024-05-09T06:34:02.318030+0000","last_unstale":"2024-05-09T06:34:02.325434+0000","last_undegraded":"2024-05-09T06:34:02.325434+0000","last_fullsized":"2024-05-09T06:34:02.325434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:15.498412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318092+0000","last_change":"2024-05-09T06:34:02.318092+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318092+0000","last_undegraded":"2024-05-09T06:34:02.318092+0000","last_fullsized":"2024-05-09T06:34:02.318092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323563+0000","last_change":"2024-05-09T06:34:02.318480+0000","last_active":"2024-05-09T06:34:02.323563+0000","last_peered":"2024-05-09T06:34:02.323563+0000","last_clean":"2024-05-09T06:34:02.323563+0000","last_became_active":"2024-05-09T06:34:02.318175+0000","last_became_peered":"2024-05-09T06:34:02.318175+0000","last_unstale":"2024-05-09T06:34:02.323563+0000","last_undegraded":"2024-05-09T06:34:02.323563+0000","last_fullsized":"2024-05-09T06:34:02.323563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316864+0000","last_change":"2024-05-09T06:34:02.316864+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316864+0000","last_undegraded":"2024-05-09T06:34:02.316864+0000","last_fullsized":"2024-05-09T06:34:02.316864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329491+0000","last_change":"2024-05-09T06:34:02.325683+0000","last_active":"2024-05-09T06:34:02.329491+0000","last_peered":"2024-05-09T06:34:02.329491+0000","last_clean":"2024-05-09T06:34:02.329491+0000","last_became_active":"2024-05-09T06:34:02.325346+0000","last_became_peered":"2024-05-09T06:34:02.325346+0000","last_unstale":"2024-05-09T06:34:02.329491+0000","last_undegraded":"2024-05-09T06:34:02.329491+0000","last_fullsized":"2024-05-09T06:34:02.329491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307720+0000","last_change":"2024-05-09T06:34:02.307720+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307720+0000","last_undegraded":"2024-05-09T06:34:02.307720+0000","last_fullsized":"2024-05-09T06:34:02.307720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323742+0000","last_change":"2024-05-09T06:34:02.318607+0000","last_active":"2024-05-09T06:34:02.323742+0000","last_peered":"2024-05-09T06:34:02.323742+0000","last_clean":"2024-05-09T06:34:02.323742+0000","last_became_active":"2024-05-09T06:34:02.318297+0000","last_became_peered":"2024-05-09T06:34:02.318297+0000","last_unstale":"2024-05-09T06:34:02.323742+0000","last_undegraded":"2024-05-09T06:34:02.323742+0000","last_fullsized":"2024-05-09T06:34:02.323742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:25.837322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311415+0000","last_change":"2024-05-09T06:34:02.311415+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311415+0000","last_undegraded":"2024-05-09T06:34:02.311415+0000","last_fullsized":"2024-05-09T06:34:02.311415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330020+0000","last_change":"2024-05-09T06:34:02.326169+0000","last_active":"2024-05-09T06:34:02.330020+0000","last_peered":"2024-05-09T06:34:02.330020+0000","last_clean":"2024-05-09T06:34:02.330020+0000","last_became_active":"2024-05-09T06:34:02.325706+0000","last_became_peered":"2024-05-09T06:34:02.325706+0000","last_unstale":"2024-05-09T06:34:02.330020+0000","last_undegraded":"2024-05-09T06:34:02.330020+0000","last_fullsized":"2024-05-09T06:34:02.330020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:28.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312364+0000","last_change":"2024-05-09T06:34:02.312364+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312364+0000","last_undegraded":"2024-05-09T06:34:02.312364+0000","last_fullsized":"2024-05-09T06:34:02.312364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330677+0000","last_change":"2024-05-09T06:34:02.330110+0000","last_active":"2024-05-09T06:34:02.330677+0000","last_peered":"2024-05-09T06:34:02.330677+0000","last_clean":"2024-05-09T06:34:02.330677+0000","last_became_active":"2024-05-09T06:34:02.329847+0000","last_became_peered":"2024-05-09T06:34:02.329847+0000","last_unstale":"2024-05-09T06:34:02.330677+0000","last_undegraded":"2024-05-09T06:34:02.330677+0000","last_fullsized":"2024-05-09T06:34:02.330677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:16.125143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314602+0000","last_change":"2024-05-09T06:34:02.314602+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314602+0000","last_undegraded":"2024-05-09T06:34:02.314602+0000","last_fullsized":"2024-05-09T06:34:02.314602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327482+0000","last_change":"2024-05-09T06:34:02.322056+0000","last_active":"2024-05-09T06:34:02.327482+0000","last_peered":"2024-05-09T06:34:02.327482+0000","last_clean":"2024-05-09T06:34:02.327482+0000","last_became_active":"2024-05-09T06:34:02.321765+0000","last_became_peered":"2024-05-09T06:34:02.321765+0000","last_unstale":"2024-05-09T06:34:02.327482+0000","last_undegraded":"2024-05-09T06:34:02.327482+0000","last_fullsized":"2024-05-09T06:34:02.327482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:12.987046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312055+0000","last_change":"2024-05-09T06:34:02.312055+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312055+0000","last_undegraded":"2024-05-09T06:34:02.312055+0000","last_fullsized":"2024-05-09T06:34:02.312055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331487+0000","last_change":"2024-05-09T06:34:02.327732+0000","last_active":"2024-05-09T06:34:02.331487+0000","last_peered":"2024-05-09T06:34:02.331487+0000","last_clean":"2024-05-09T06:34:02.331487+0000","last_became_active":"2024-05-09T06:34:02.327311+0000","last_became_peered":"2024-05-09T06:34:02.327311+0000","last_unstale":"2024-05-09T06:34:02.331487+0000","last_undegraded":"2024-05-09T06:34:02.331487+0000","last_fullsized":"2024-05-09T06:34:02.331487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308418+0000","last_change":"2024-05-09T06:34:02.308418+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308418+0000","last_undegraded":"2024-05-09T06:34:02.308418+0000","last_fullsized":"2024-05-09T06:34:02.308418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331858+0000","last_change":"2024-05-09T06:34:02.329189+0000","last_active":"2024-05-09T06:34:02.331858+0000","last_peered":"2024-05-09T06:34:02.331858+0000","last_clean":"2024-05-09T06:34:02.331858+0000","last_became_active":"2024-05-09T06:34:02.328869+0000","last_became_peered":"2024-05-09T06:34:02.328869+0000","last_unstale":"2024-05-09T06:34:02.331858+0000","last_undegraded":"2024-05-09T06:34:02.331858+0000","last_fullsized":"2024-05-09T06:34:02.331858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:55.195087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313072+0000","last_change":"2024-05-09T06:34:02.313072+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313072+0000","last_undegraded":"2024-05-09T06:34:02.313072+0000","last_fullsized":"2024-05-09T06:34:02.313072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329582+0000","last_change":"2024-05-09T06:34:02.327306+0000","last_active":"2024-05-09T06:34:02.329582+0000","last_peered":"2024-05-09T06:34:02.329582+0000","last_clean":"2024-05-09T06:34:02.329582+0000","last_became_active":"2024-05-09T06:34:02.326990+0000","last_became_peered":"2024-05-09T06:34:02.326990+0000","last_unstale":"2024-05-09T06:34:02.329582+0000","last_undegraded":"2024-05-09T06:34:02.329582+0000","last_fullsized":"2024-05-09T06:34:02.329582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319661+0000","last_change":"2024-05-09T06:34:02.319661+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319661+0000","last_undegraded":"2024-05-09T06:34:02.319661+0000","last_fullsized":"2024-05-09T06:34:02.319661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329784+0000","last_change":"2024-05-09T06:34:02.328106+0000","last_active":"2024-05-09T06:34:02.329784+0000","last_peered":"2024-05-09T06:34:02.329784+0000","last_clean":"2024-05-09T06:34:02.329784+0000","last_became_active":"2024-05-09T06:34:02.327820+0000","last_became_peered":"2024-05-09T06:34:02.327820+0000","last_unstale":"2024-05-09T06:34:02.329784+0000","last_undegraded":"2024-05-09T06:34:02.329784+0000","last_fullsized":"2024-05-09T06:34:02.329784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:33.383852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318556+0000","last_change":"2024-05-09T06:34:02.318556+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318556+0000","last_undegraded":"2024-05-09T06:34:02.318556+0000","last_fullsized":"2024-05-09T06:34:02.318556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329527+0000","last_change":"2024-05-09T06:34:02.327204+0000","last_active":"2024-05-09T06:34:02.329527+0000","last_peered":"2024-05-09T06:34:02.329527+0000","last_clean":"2024-05-09T06:34:02.329527+0000","last_became_active":"2024-05-09T06:34:02.326876+0000","last_became_peered":"2024-05-09T06:34:02.326876+0000","last_unstale":"2024-05-09T06:34:02.329527+0000","last_undegraded":"2024-05-09T06:34:02.329527+0000","last_fullsized":"2024-05-09T06:34:02.329527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314614+0000","last_change":"2024-05-09T06:34:02.314614+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314614+0000","last_undegraded":"2024-05-09T06:34:02.314614+0000","last_fullsized":"2024-05-09T06:34:02.314614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328608+0000","last_change":"2024-05-09T06:34:02.327053+0000","last_active":"2024-05-09T06:34:02.328608+0000","last_peered":"2024-05-09T06:34:02.328608+0000","last_clean":"2024-05-09T06:34:02.328608+0000","last_became_active":"2024-05-09T06:34:02.326751+0000","last_became_peered":"2024-05-09T06:34:02.326751+0000","last_unstale":"2024-05-09T06:34:02.328608+0000","last_undegraded":"2024-05-09T06:34:02.328608+0000","last_fullsized":"2024-05-09T06:34:02.328608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:11.750079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313077+0000","last_change":"2024-05-09T06:34:02.313077+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313077+0000","last_undegraded":"2024-05-09T06:34:02.313077+0000","last_fullsized":"2024-05-09T06:34:02.313077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331465+0000","last_change":"2024-05-09T06:34:02.326680+0000","last_active":"2024-05-09T06:34:02.331465+0000","last_peered":"2024-05-09T06:34:02.331465+0000","last_clean":"2024-05-09T06:34:02.331465+0000","last_became_active":"2024-05-09T06:34:02.326353+0000","last_became_peered":"2024-05-09T06:34:02.326353+0000","last_unstale":"2024-05-09T06:34:02.331465+0000","last_undegraded":"2024-05-09T06:34:02.331465+0000","last_fullsized":"2024-05-09T06:34:02.331465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311571+0000","last_change":"2024-05-09T06:34:02.311571+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311571+0000","last_undegraded":"2024-05-09T06:34:02.311571+0000","last_fullsized":"2024-05-09T06:34:02.311571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329518+0000","last_change":"2024-05-09T06:34:02.327790+0000","last_active":"2024-05-09T06:34:02.329518+0000","last_peered":"2024-05-09T06:34:02.329518+0000","last_clean":"2024-05-09T06:34:02.329518+0000","last_became_active":"2024-05-09T06:34:02.327409+0000","last_became_peered":"2024-05-09T06:34:02.327409+0000","last_unstale":"2024-05-09T06:34:02.329518+0000","last_undegraded":"2024-05-09T06:34:02.329518+0000","last_fullsized":"2024-05-09T06:34:02.329518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.828382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311765+0000","last_change":"2024-05-09T06:34:02.311765+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311765+0000","last_undegraded":"2024-05-09T06:34:02.311765+0000","last_fullsized":"2024-05-09T06:34:02.311765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.317202+0000","last_change":"2024-05-09T06:34:02.314084+0000","last_active":"2024-05-09T06:34:02.317202+0000","last_peered":"2024-05-09T06:34:02.317202+0000","last_clean":"2024-05-09T06:34:02.317202+0000","last_became_active":"2024-05-09T06:34:02.313093+0000","last_became_peered":"2024-05-09T06:34:02.313093+0000","last_unstale":"2024-05-09T06:34:02.317202+0000","last_undegraded":"2024-05-09T06:34:02.317202+0000","last_fullsized":"2024-05-09T06:34:02.317202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:33.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321022+0000","last_change":"2024-05-09T06:34:02.321022+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321022+0000","last_undegraded":"2024-05-09T06:34:02.321022+0000","last_fullsized":"2024-05-09T06:34:02.321022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325577+0000","last_change":"2024-05-09T06:34:02.319167+0000","last_active":"2024-05-09T06:34:02.325577+0000","last_peered":"2024-05-09T06:34:02.325577+0000","last_clean":"2024-05-09T06:34:02.325577+0000","last_became_active":"2024-05-09T06:34:02.318846+0000","last_became_peered":"2024-05-09T06:34:02.318846+0000","last_unstale":"2024-05-09T06:34:02.325577+0000","last_undegraded":"2024-05-09T06:34:02.325577+0000","last_fullsized":"2024-05-09T06:34:02.325577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:19.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311082+0000","last_change":"2024-05-09T06:34:02.311082+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311082+0000","last_undegraded":"2024-05-09T06:34:02.311082+0000","last_fullsized":"2024-05-09T06:34:02.311082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.324239+0000","last_change":"2024-05-09T06:34:02.317990+0000","last_active":"2024-05-09T06:34:02.324239+0000","last_peered":"2024-05-09T06:34:02.324239+0000","last_clean":"2024-05-09T06:34:02.324239+0000","last_became_active":"2024-05-09T06:34:02.317688+0000","last_became_peered":"2024-05-09T06:34:02.317688+0000","last_unstale":"2024-05-09T06:34:02.324239+0000","last_undegraded":"2024-05-09T06:34:02.324239+0000","last_fullsized":"2024-05-09T06:34:02.324239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312002+0000","last_change":"2024-05-09T06:34:02.312002+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312002+0000","last_undegraded":"2024-05-09T06:34:02.312002+0000","last_fullsized":"2024-05-09T06:34:02.312002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329509+0000","last_change":"2024-05-09T06:34:02.326540+0000","last_active":"2024-05-09T06:34:02.329509+0000","last_peered":"2024-05-09T06:34:02.329509+0000","last_clean":"2024-05-09T06:34:02.329509+0000","last_became_active":"2024-05-09T06:34:02.326253+0000","last_became_peered":"2024-05-09T06:34:02.326253+0000","last_unstale":"2024-05-09T06:34:02.329509+0000","last_undegraded":"2024-05-09T06:34:02.329509+0000","last_fullsized":"2024-05-09T06:34:02.329509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312075+0000","last_change":"2024-05-09T06:34:02.312075+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312075+0000","last_undegraded":"2024-05-09T06:34:02.312075+0000","last_fullsized":"2024-05-09T06:34:02.312075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327577+0000","last_change":"2024-05-09T06:34:02.323636+0000","last_active":"2024-05-09T06:34:02.327577+0000","last_peered":"2024-05-09T06:34:02.327577+0000","last_clean":"2024-05-09T06:34:02.327577+0000","last_became_active":"2024-05-09T06:34:02.323275+0000","last_became_peered":"2024-05-09T06:34:02.323275+0000","last_unstale":"2024-05-09T06:34:02.327577+0000","last_undegraded":"2024-05-09T06:34:02.327577+0000","last_fullsized":"2024-05-09T06:34:02.327577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310800+0000","last_change":"2024-05-09T06:34:02.310800+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310800+0000","last_undegraded":"2024-05-09T06:34:02.310800+0000","last_fullsized":"2024-05-09T06:34:02.310800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331509+0000","last_change":"2024-05-09T06:34:02.328232+0000","last_active":"2024-05-09T06:34:02.331509+0000","last_peered":"2024-05-09T06:34:02.331509+0000","last_clean":"2024-05-09T06:34:02.331509+0000","last_became_active":"2024-05-09T06:34:02.327886+0000","last_became_peered":"2024-05-09T06:34:02.327886+0000","last_unstale":"2024-05-09T06:34:02.331509+0000","last_undegraded":"2024-05-09T06:34:02.331509+0000","last_fullsized":"2024-05-09T06:34:02.331509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315380+0000","last_change":"2024-05-09T06:34:02.315380+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315380+0000","last_undegraded":"2024-05-09T06:34:02.315380+0000","last_fullsized":"2024-05-09T06:34:02.315380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331857+0000","last_change":"2024-05-09T06:34:02.331212+0000","last_active":"2024-05-09T06:34:02.331857+0000","last_peered":"2024-05-09T06:34:02.331857+0000","last_clean":"2024-05-09T06:34:02.331857+0000","last_became_active":"2024-05-09T06:34:02.330939+0000","last_became_peered":"2024-05-09T06:34:02.330939+0000","last_unstale":"2024-05-09T06:34:02.331857+0000","last_undegraded":"2024-05-09T06:34:02.331857+0000","last_fullsized":"2024-05-09T06:34:02.331857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:30:48.018600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.316064+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:34:02.316064+0000","last_peered":"2024-05-09T06:34:02.316064+0000","last_clean":"2024-05-09T06:34:02.316064+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:34:02.316064+0000","last_undegraded":"2024-05-09T06:34:02.316064+0000","last_fullsized":"2024-05-09T06:34:02.316064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1087428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":497060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1094095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:07.695 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:34:07.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.855+0000 7fc9ea709640 1 Processor -- start 2024-05-09T06:34:07.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.855+0000 7fc9ea709640 1 -- start start 2024-05-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9ea709640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40c6860 0x7fc9e4099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9ea709640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9ea709640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e4099930 con 0x7fc9e40c6c40 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9ea709640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e40b1e20 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9ea709640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e415ee90 con 0x7fc9e40c6860 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9dbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9dbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42500/0 (socket says 172.21.15.1:42500) 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1888498281 learned_addr learned my addr 172.21.15.1:0/1888498281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.856+0000 7fc9e3fff640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40c6860 0x7fc9e4099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.857+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1888498281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40c6860 msgr2=0x7fc9e4099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.857+0000 7fc9dbfff640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40c6860 0x7fc9e4099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.857+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1888498281 >> v1:172.21.15.6:6789/0 conn(0x7fc9e40c6c40 legacy=0x7fc9e415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.857+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1888498281 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e4162930 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.857+0000 7fc9dbfff640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009870 tx=0x7fc9cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b61e7ae83dca846 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9e37fe640 1 -- 172.21.15.1:0/1888498281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc037070 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9e37fe640 1 -- 172.21.15.1:0/1888498281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9cc032440 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9e37fe640 1 -- 172.21.15.1:0/1888498281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc032810 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 -- 172.21.15.1:0/1888498281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40b2aa0 msgr2=0x7fc9e415e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009870 tx=0x7fc9cc02a1c0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 -- 172.21.15.1:0/1888498281 shutdown_connections 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40b2aa0 0x7fc9e415e4b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1888498281 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40c6860 0x7fc9e4099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 -- 172.21.15.1:0/1888498281 >> 172.21.15.1:0/1888498281 conn(0x7fc9e40b1fb0 msgr2=0x7fc9e40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 -- 172.21.15.1:0/1888498281 shutdown_connections 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.858+0000 7fc9ea709640 1 -- 172.21.15.1:0/1888498281 wait complete. 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.859+0000 7fc9ea709640 1 Processor -- start 2024-05-09T06:34:07.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.859+0000 7fc9ea709640 1 -- start start 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40b2aa0 0x7fc9e41504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9e40c6c40 0x7fc9e40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e415eb20 con 0x7fc9e40c6860 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e415fa30 con 0x7fc9e40c6c40 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9ea709640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40c70d0 con 0x7fc9e40b2aa0 2024-05-09T06:34:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42504/0 (socket says 172.21.15.1:42504) 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1220974920 learned_addr learned my addr 172.21.15.1:0/1220974920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9e3fff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40b2aa0 0x7fc9e41504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.860+0000 7fc9e8ca6640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9e40c6c40 0x7fc9e40cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40b2aa0 msgr2=0x7fc9e41504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40b2aa0 0x7fc9e41504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9e40c6c40 msgr2=0x7fc9e40cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9e40c6c40 0x7fc9e40cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 -- 172.21.15.1:0/1220974920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e4150a30 con 0x7fc9e40c6860 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9dbfff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc037040 con 0x7fc9e40c6860 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9cc042070 con 0x7fc9e40c6860 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e40c9520 con 0x7fc9e40c6860 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc030880 con 0x7fc9e40c6860 2024-05-09T06:34:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.861+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e40c9ae0 con 0x7fc9e40c6860 2024-05-09T06:34:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.862+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e40752b0 con 0x7fc9e40c6860 2024-05-09T06:34:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.862+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9cc02ad10 con 0x7fc9e40c6860 2024-05-09T06:34:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.863+0000 7fc9e17fa640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.863+0000 7fc9e3fff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.863+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc9cc0ab5e0 con 0x7fc9e40c6860 2024-05-09T06:34:07.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.864+0000 7fc9e3fff640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 0x7fc9c406e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009a30 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:07.868+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc9cc079d70 con 0x7fc9e40c6860 2024-05-09T06:34:08.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.046+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9e40b18f0 con 0x7fc9c406be30 2024-05-09T06:34:08.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.052+0000 7fc9e17fa640 1 -- 172.21.15.1:0/1220974920 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7fc9e40b18f0 con 0x7fc9c406be30 2024-05-09T06:34:08.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:08.036 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:34:08.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 msgr2=0x7fc9c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 0x7fc9c406e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009a30 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 msgr2=0x7fc9e40cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 shutdown_connections 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9e40c6c40 0x7fc9e40cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e40c6860 0x7fc9e40cc4a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 --2- 172.21.15.1:0/1220974920 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9e40b2aa0 0x7fc9e41504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 >> 172.21.15.1:0/1220974920 conn(0x7fc9e40b1fb0 msgr2=0x7fc9e40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.055+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 shutdown_connections 2024-05-09T06:34:08.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.056+0000 7fc9ea709640 1 -- 172.21.15.1:0/1220974920 wait complete. 2024-05-09T06:34:08.046 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-09T06:34:07.158732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259168,"kb_used_data":9712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715552,"statfs":{"total":773094113280,"available":772828725248,"internally_reserved":0,"allocated":9945088,"data_stored":5197218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856752"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314422+0000","last_change":"2024-05-09T06:34:02.314422+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314422+0000","last_undegraded":"2024-05-09T06:34:02.314422+0000","last_fullsized":"2024-05-09T06:34:02.314422+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320632+0000","last_change":"2024-05-09T06:34:02.320632+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320632+0000","last_undegraded":"2024-05-09T06:34:02.320632+0000","last_fullsized":"2024-05-09T06:34:02.320632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313945+0000","last_change":"2024-05-09T06:34:02.313945+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313945+0000","last_undegraded":"2024-05-09T06:34:02.313945+0000","last_fullsized":"2024-05-09T06:34:02.313945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318875+0000","last_change":"2024-05-09T06:34:02.318875+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318875+0000","last_undegraded":"2024-05-09T06:34:02.318875+0000","last_fullsized":"2024-05-09T06:34:02.318875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308592+0000","last_change":"2024-05-09T06:34:02.308592+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308592+0000","last_undegraded":"2024-05-09T06:34:02.308592+0000","last_fullsized":"2024-05-09T06:34:02.308592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316982+0000","last_change":"2024-05-09T06:34:02.316982+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316982+0000","last_undegraded":"2024-05-09T06:34:02.316982+0000","last_fullsized":"2024-05-09T06:34:02.316982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313987+0000","last_change":"2024-05-09T06:34:02.313987+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313987+0000","last_undegraded":"2024-05-09T06:34:02.313987+0000","last_fullsized":"2024-05-09T06:34:02.313987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320266+0000","last_change":"2024-05-09T06:34:02.320266+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320266+0000","last_undegraded":"2024-05-09T06:34:02.320266+0000","last_fullsized":"2024-05-09T06:34:02.320266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307924+0000","last_change":"2024-05-09T06:34:02.307924+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307924+0000","last_undegraded":"2024-05-09T06:34:02.307924+0000","last_fullsized":"2024-05-09T06:34:02.307924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315657+0000","last_change":"2024-05-09T06:34:02.315657+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315657+0000","last_undegraded":"2024-05-09T06:34:02.315657+0000","last_fullsized":"2024-05-09T06:34:02.315657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308933+0000","last_change":"2024-05-09T06:34:02.308933+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308933+0000","last_undegraded":"2024-05-09T06:34:02.308933+0000","last_fullsized":"2024-05-09T06:34:02.308933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312738+0000","last_change":"2024-05-09T06:34:02.312738+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312738+0000","last_undegraded":"2024-05-09T06:34:02.312738+0000","last_fullsized":"2024-05-09T06:34:02.312738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321819+0000","last_change":"2024-05-09T06:34:02.321819+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321819+0000","last_undegraded":"2024-05-09T06:34:02.321819+0000","last_fullsized":"2024-05-09T06:34:02.321819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310310+0000","last_change":"2024-05-09T06:34:02.310310+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310310+0000","last_undegraded":"2024-05-09T06:34:02.310310+0000","last_fullsized":"2024-05-09T06:34:02.310310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314242+0000","last_change":"2024-05-09T06:34:02.314242+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314242+0000","last_undegraded":"2024-05-09T06:34:02.314242+0000","last_fullsized":"2024-05-09T06:34:02.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316880+0000","last_change":"2024-05-09T06:34:02.316880+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316880+0000","last_undegraded":"2024-05-09T06:34:02.316880+0000","last_fullsized":"2024-05-09T06:34:02.316880+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316455+0000","last_change":"2024-05-09T06:34:02.316455+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316455+0000","last_undegraded":"2024-05-09T06:34:02.316455+0000","last_fullsized":"2024-05-09T06:34:02.316455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313420+0000","last_change":"2024-05-09T06:34:02.313420+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313420+0000","last_undegraded":"2024-05-09T06:34:02.313420+0000","last_fullsized":"2024-05-09T06:34:02.313420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308424+0000","last_change":"2024-05-09T06:34:02.308424+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308424+0000","last_undegraded":"2024-05-09T06:34:02.308424+0000","last_fullsized":"2024-05-09T06:34:02.308424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315522+0000","last_change":"2024-05-09T06:34:02.315522+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315522+0000","last_undegraded":"2024-05-09T06:34:02.315522+0000","last_fullsized":"2024-05-09T06:34:02.315522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309601+0000","last_change":"2024-05-09T06:34:02.309601+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309601+0000","last_undegraded":"2024-05-09T06:34:02.309601+0000","last_fullsized":"2024-05-09T06:34:02.309601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309739+0000","last_change":"2024-05-09T06:34:02.309739+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309739+0000","last_undegraded":"2024-05-09T06:34:02.309739+0000","last_fullsized":"2024-05-09T06:34:02.309739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312820+0000","last_change":"2024-05-09T06:34:02.312820+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312820+0000","last_undegraded":"2024-05-09T06:34:02.312820+0000","last_fullsized":"2024-05-09T06:34:02.312820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313125+0000","last_change":"2024-05-09T06:34:02.313125+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313125+0000","last_undegraded":"2024-05-09T06:34:02.313125+0000","last_fullsized":"2024-05-09T06:34:02.313125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321499+0000","last_change":"2024-05-09T06:34:02.321499+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321499+0000","last_undegraded":"2024-05-09T06:34:02.321499+0000","last_fullsized":"2024-05-09T06:34:02.321499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310029+0000","last_change":"2024-05-09T06:34:02.310029+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310029+0000","last_undegraded":"2024-05-09T06:34:02.310029+0000","last_fullsized":"2024-05-09T06:34:02.310029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312353+0000","last_change":"2024-05-09T06:34:02.312353+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312353+0000","last_undegraded":"2024-05-09T06:34:02.312353+0000","last_fullsized":"2024-05-09T06:34:02.312353+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315046+0000","last_change":"2024-05-09T06:34:02.315046+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315046+0000","last_undegraded":"2024-05-09T06:34:02.315046+0000","last_fullsized":"2024-05-09T06:34:02.315046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314059+0000","last_change":"2024-05-09T06:34:02.314059+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314059+0000","last_undegraded":"2024-05-09T06:34:02.314059+0000","last_fullsized":"2024-05-09T06:34:02.314059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309025+0000","last_change":"2024-05-09T06:34:02.309025+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309025+0000","last_undegraded":"2024-05-09T06:34:02.309025+0000","last_fullsized":"2024-05-09T06:34:02.309025+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320197+0000","last_change":"2024-05-09T06:34:02.320197+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320197+0000","last_undegraded":"2024-05-09T06:34:02.320197+0000","last_fullsized":"2024-05-09T06:34:02.320197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315167+0000","last_change":"2024-05-09T06:34:02.315167+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315167+0000","last_undegraded":"2024-05-09T06:34:02.315167+0000","last_fullsized":"2024-05-09T06:34:02.315167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315036+0000","last_change":"2024-05-09T06:34:02.315036+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315036+0000","last_undegraded":"2024-05-09T06:34:02.315036+0000","last_fullsized":"2024-05-09T06:34:02.315036+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.321306+0000","last_change":"2024-05-09T06:34:02.315932+0000","last_active":"2024-05-09T06:34:02.321306+0000","last_peered":"2024-05-09T06:34:02.321306+0000","last_clean":"2024-05-09T06:34:02.321306+0000","last_became_active":"2024-05-09T06:34:02.315556+0000","last_became_peered":"2024-05-09T06:34:02.315556+0000","last_unstale":"2024-05-09T06:34:02.321306+0000","last_undegraded":"2024-05-09T06:34:02.321306+0000","last_fullsized":"2024-05-09T06:34:02.321306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:50.355406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315873+0000","last_change":"2024-05-09T06:34:02.315873+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315873+0000","last_undegraded":"2024-05-09T06:34:02.315873+0000","last_fullsized":"2024-05-09T06:34:02.315873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325355+0000","last_change":"2024-05-09T06:34:02.317231+0000","last_active":"2024-05-09T06:34:02.325355+0000","last_peered":"2024-05-09T06:34:02.325355+0000","last_clean":"2024-05-09T06:34:02.325355+0000","last_became_active":"2024-05-09T06:34:02.316853+0000","last_became_peered":"2024-05-09T06:34:02.316853+0000","last_unstale":"2024-05-09T06:34:02.325355+0000","last_undegraded":"2024-05-09T06:34:02.325355+0000","last_fullsized":"2024-05-09T06:34:02.325355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:47.234272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314308+0000","last_change":"2024-05-09T06:34:02.314308+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314308+0000","last_undegraded":"2024-05-09T06:34:02.314308+0000","last_fullsized":"2024-05-09T06:34:02.314308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325501+0000","last_change":"2024-05-09T06:34:02.322212+0000","last_active":"2024-05-09T06:34:02.325501+0000","last_peered":"2024-05-09T06:34:02.325501+0000","last_clean":"2024-05-09T06:34:02.325501+0000","last_became_active":"2024-05-09T06:34:02.321940+0000","last_became_peered":"2024-05-09T06:34:02.321940+0000","last_unstale":"2024-05-09T06:34:02.325501+0000","last_undegraded":"2024-05-09T06:34:02.325501+0000","last_fullsized":"2024-05-09T06:34:02.325501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314090+0000","last_change":"2024-05-09T06:34:02.314090+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314090+0000","last_undegraded":"2024-05-09T06:34:02.314090+0000","last_fullsized":"2024-05-09T06:34:02.314090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.334420+0000","last_change":"2024-05-09T06:34:02.328490+0000","last_active":"2024-05-09T06:34:02.334420+0000","last_peered":"2024-05-09T06:34:02.334420+0000","last_clean":"2024-05-09T06:34:02.334420+0000","last_became_active":"2024-05-09T06:34:02.328118+0000","last_became_peered":"2024-05-09T06:34:02.328118+0000","last_unstale":"2024-05-09T06:34:02.334420+0000","last_undegraded":"2024-05-09T06:34:02.334420+0000","last_fullsized":"2024-05-09T06:34:02.334420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319876+0000","last_change":"2024-05-09T06:34:02.319876+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319876+0000","last_undegraded":"2024-05-09T06:34:02.319876+0000","last_fullsized":"2024-05-09T06:34:02.319876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325837+0000","last_change":"2024-05-09T06:34:02.320157+0000","last_active":"2024-05-09T06:34:02.325837+0000","last_peered":"2024-05-09T06:34:02.325837+0000","last_clean":"2024-05-09T06:34:02.325837+0000","last_became_active":"2024-05-09T06:34:02.319814+0000","last_became_peered":"2024-05-09T06:34:02.319814+0000","last_unstale":"2024-05-09T06:34:02.325837+0000","last_undegraded":"2024-05-09T06:34:02.325837+0000","last_fullsized":"2024-05-09T06:34:02.325837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:05.426920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312032+0000","last_change":"2024-05-09T06:34:02.312032+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312032+0000","last_undegraded":"2024-05-09T06:34:02.312032+0000","last_fullsized":"2024-05-09T06:34:02.312032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330331+0000","last_change":"2024-05-09T06:34:02.329390+0000","last_active":"2024-05-09T06:34:02.330331+0000","last_peered":"2024-05-09T06:34:02.330331+0000","last_clean":"2024-05-09T06:34:02.330331+0000","last_became_active":"2024-05-09T06:34:02.329108+0000","last_became_peered":"2024-05-09T06:34:02.329108+0000","last_unstale":"2024-05-09T06:34:02.330331+0000","last_undegraded":"2024-05-09T06:34:02.330331+0000","last_fullsized":"2024-05-09T06:34:02.330331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:08.397934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315492+0000","last_change":"2024-05-09T06:34:02.315492+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315492+0000","last_undegraded":"2024-05-09T06:34:02.315492+0000","last_fullsized":"2024-05-09T06:34:02.315492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326273+0000","last_change":"2024-05-09T06:34:02.324159+0000","last_active":"2024-05-09T06:34:02.326273+0000","last_peered":"2024-05-09T06:34:02.326273+0000","last_clean":"2024-05-09T06:34:02.326273+0000","last_became_active":"2024-05-09T06:34:02.323875+0000","last_became_peered":"2024-05-09T06:34:02.323875+0000","last_unstale":"2024-05-09T06:34:02.326273+0000","last_undegraded":"2024-05-09T06:34:02.326273+0000","last_fullsized":"2024-05-09T06:34:02.326273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316074+0000","last_change":"2024-05-09T06:34:02.316074+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316074+0000","last_undegraded":"2024-05-09T06:34:02.316074+0000","last_fullsized":"2024-05-09T06:34:02.316074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326331+0000","last_change":"2024-05-09T06:34:02.324681+0000","last_active":"2024-05-09T06:34:02.326331+0000","last_peered":"2024-05-09T06:34:02.326331+0000","last_clean":"2024-05-09T06:34:02.326331+0000","last_became_active":"2024-05-09T06:34:02.324383+0000","last_became_peered":"2024-05-09T06:34:02.324383+0000","last_unstale":"2024-05-09T06:34:02.326331+0000","last_undegraded":"2024-05-09T06:34:02.326331+0000","last_fullsized":"2024-05-09T06:34:02.326331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307911+0000","last_change":"2024-05-09T06:34:02.307911+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307911+0000","last_undegraded":"2024-05-09T06:34:02.307911+0000","last_fullsized":"2024-05-09T06:34:02.307911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328828+0000","last_change":"2024-05-09T06:34:02.322716+0000","last_active":"2024-05-09T06:34:02.328828+0000","last_peered":"2024-05-09T06:34:02.328828+0000","last_clean":"2024-05-09T06:34:02.328828+0000","last_became_active":"2024-05-09T06:34:02.322190+0000","last_became_peered":"2024-05-09T06:34:02.322190+0000","last_unstale":"2024-05-09T06:34:02.328828+0000","last_undegraded":"2024-05-09T06:34:02.328828+0000","last_fullsized":"2024-05-09T06:34:02.328828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315128+0000","last_change":"2024-05-09T06:34:02.315128+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315128+0000","last_undegraded":"2024-05-09T06:34:02.315128+0000","last_fullsized":"2024-05-09T06:34:02.315128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329063+0000","last_change":"2024-05-09T06:34:02.328152+0000","last_active":"2024-05-09T06:34:02.329063+0000","last_peered":"2024-05-09T06:34:02.329063+0000","last_clean":"2024-05-09T06:34:02.329063+0000","last_became_active":"2024-05-09T06:34:02.327851+0000","last_became_peered":"2024-05-09T06:34:02.327851+0000","last_unstale":"2024-05-09T06:34:02.329063+0000","last_undegraded":"2024-05-09T06:34:02.329063+0000","last_fullsized":"2024-05-09T06:34:02.329063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312609+0000","last_change":"2024-05-09T06:34:02.312609+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312609+0000","last_undegraded":"2024-05-09T06:34:02.312609+0000","last_fullsized":"2024-05-09T06:34:02.312609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325434+0000","last_change":"2024-05-09T06:34:02.318316+0000","last_active":"2024-05-09T06:34:02.325434+0000","last_peered":"2024-05-09T06:34:02.325434+0000","last_clean":"2024-05-09T06:34:02.325434+0000","last_became_active":"2024-05-09T06:34:02.318030+0000","last_became_peered":"2024-05-09T06:34:02.318030+0000","last_unstale":"2024-05-09T06:34:02.325434+0000","last_undegraded":"2024-05-09T06:34:02.325434+0000","last_fullsized":"2024-05-09T06:34:02.325434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:15.498412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318092+0000","last_change":"2024-05-09T06:34:02.318092+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318092+0000","last_undegraded":"2024-05-09T06:34:02.318092+0000","last_fullsized":"2024-05-09T06:34:02.318092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323563+0000","last_change":"2024-05-09T06:34:02.318480+0000","last_active":"2024-05-09T06:34:02.323563+0000","last_peered":"2024-05-09T06:34:02.323563+0000","last_clean":"2024-05-09T06:34:02.323563+0000","last_became_active":"2024-05-09T06:34:02.318175+0000","last_became_peered":"2024-05-09T06:34:02.318175+0000","last_unstale":"2024-05-09T06:34:02.323563+0000","last_undegraded":"2024-05-09T06:34:02.323563+0000","last_fullsized":"2024-05-09T06:34:02.323563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316864+0000","last_change":"2024-05-09T06:34:02.316864+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316864+0000","last_undegraded":"2024-05-09T06:34:02.316864+0000","last_fullsized":"2024-05-09T06:34:02.316864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329491+0000","last_change":"2024-05-09T06:34:02.325683+0000","last_active":"2024-05-09T06:34:02.329491+0000","last_peered":"2024-05-09T06:34:02.329491+0000","last_clean":"2024-05-09T06:34:02.329491+0000","last_became_active":"2024-05-09T06:34:02.325346+0000","last_became_peered":"2024-05-09T06:34:02.325346+0000","last_unstale":"2024-05-09T06:34:02.329491+0000","last_undegraded":"2024-05-09T06:34:02.329491+0000","last_fullsized":"2024-05-09T06:34:02.329491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307720+0000","last_change":"2024-05-09T06:34:02.307720+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307720+0000","last_undegraded":"2024-05-09T06:34:02.307720+0000","last_fullsized":"2024-05-09T06:34:02.307720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323742+0000","last_change":"2024-05-09T06:34:02.318607+0000","last_active":"2024-05-09T06:34:02.323742+0000","last_peered":"2024-05-09T06:34:02.323742+0000","last_clean":"2024-05-09T06:34:02.323742+0000","last_became_active":"2024-05-09T06:34:02.318297+0000","last_became_peered":"2024-05-09T06:34:02.318297+0000","last_unstale":"2024-05-09T06:34:02.323742+0000","last_undegraded":"2024-05-09T06:34:02.323742+0000","last_fullsized":"2024-05-09T06:34:02.323742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:25.837322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311415+0000","last_change":"2024-05-09T06:34:02.311415+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311415+0000","last_undegraded":"2024-05-09T06:34:02.311415+0000","last_fullsized":"2024-05-09T06:34:02.311415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330020+0000","last_change":"2024-05-09T06:34:02.326169+0000","last_active":"2024-05-09T06:34:02.330020+0000","last_peered":"2024-05-09T06:34:02.330020+0000","last_clean":"2024-05-09T06:34:02.330020+0000","last_became_active":"2024-05-09T06:34:02.325706+0000","last_became_peered":"2024-05-09T06:34:02.325706+0000","last_unstale":"2024-05-09T06:34:02.330020+0000","last_undegraded":"2024-05-09T06:34:02.330020+0000","last_fullsized":"2024-05-09T06:34:02.330020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:28.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312364+0000","last_change":"2024-05-09T06:34:02.312364+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312364+0000","last_undegraded":"2024-05-09T06:34:02.312364+0000","last_fullsized":"2024-05-09T06:34:02.312364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330677+0000","last_change":"2024-05-09T06:34:02.330110+0000","last_active":"2024-05-09T06:34:02.330677+0000","last_peered":"2024-05-09T06:34:02.330677+0000","last_clean":"2024-05-09T06:34:02.330677+0000","last_became_active":"2024-05-09T06:34:02.329847+0000","last_became_peered":"2024-05-09T06:34:02.329847+0000","last_unstale":"2024-05-09T06:34:02.330677+0000","last_undegraded":"2024-05-09T06:34:02.330677+0000","last_fullsized":"2024-05-09T06:34:02.330677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:16.125143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314602+0000","last_change":"2024-05-09T06:34:02.314602+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314602+0000","last_undegraded":"2024-05-09T06:34:02.314602+0000","last_fullsized":"2024-05-09T06:34:02.314602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327482+0000","last_change":"2024-05-09T06:34:02.322056+0000","last_active":"2024-05-09T06:34:02.327482+0000","last_peered":"2024-05-09T06:34:02.327482+0000","last_clean":"2024-05-09T06:34:02.327482+0000","last_became_active":"2024-05-09T06:34:02.321765+0000","last_became_peered":"2024-05-09T06:34:02.321765+0000","last_unstale":"2024-05-09T06:34:02.327482+0000","last_undegraded":"2024-05-09T06:34:02.327482+0000","last_fullsized":"2024-05-09T06:34:02.327482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:12.987046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312055+0000","last_change":"2024-05-09T06:34:02.312055+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312055+0000","last_undegraded":"2024-05-09T06:34:02.312055+0000","last_fullsized":"2024-05-09T06:34:02.312055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331487+0000","last_change":"2024-05-09T06:34:02.327732+0000","last_active":"2024-05-09T06:34:02.331487+0000","last_peered":"2024-05-09T06:34:02.331487+0000","last_clean":"2024-05-09T06:34:02.331487+0000","last_became_active":"2024-05-09T06:34:02.327311+0000","last_became_peered":"2024-05-09T06:34:02.327311+0000","last_unstale":"2024-05-09T06:34:02.331487+0000","last_undegraded":"2024-05-09T06:34:02.331487+0000","last_fullsized":"2024-05-09T06:34:02.331487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308418+0000","last_change":"2024-05-09T06:34:02.308418+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308418+0000","last_undegraded":"2024-05-09T06:34:02.308418+0000","last_fullsized":"2024-05-09T06:34:02.308418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331858+0000","last_change":"2024-05-09T06:34:02.329189+0000","last_active":"2024-05-09T06:34:02.331858+0000","last_peered":"2024-05-09T06:34:02.331858+0000","last_clean":"2024-05-09T06:34:02.331858+0000","last_became_active":"2024-05-09T06:34:02.328869+0000","last_became_peered":"2024-05-09T06:34:02.328869+0000","last_unstale":"2024-05-09T06:34:02.331858+0000","last_undegraded":"2024-05-09T06:34:02.331858+0000","last_fullsized":"2024-05-09T06:34:02.331858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:55.195087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313072+0000","last_change":"2024-05-09T06:34:02.313072+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313072+0000","last_undegraded":"2024-05-09T06:34:02.313072+0000","last_fullsized":"2024-05-09T06:34:02.313072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329582+0000","last_change":"2024-05-09T06:34:02.327306+0000","last_active":"2024-05-09T06:34:02.329582+0000","last_peered":"2024-05-09T06:34:02.329582+0000","last_clean":"2024-05-09T06:34:02.329582+0000","last_became_active":"2024-05-09T06:34:02.326990+0000","last_became_peered":"2024-05-09T06:34:02.326990+0000","last_unstale":"2024-05-09T06:34:02.329582+0000","last_undegraded":"2024-05-09T06:34:02.329582+0000","last_fullsized":"2024-05-09T06:34:02.329582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319661+0000","last_change":"2024-05-09T06:34:02.319661+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319661+0000","last_undegraded":"2024-05-09T06:34:02.319661+0000","last_fullsized":"2024-05-09T06:34:02.319661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329784+0000","last_change":"2024-05-09T06:34:02.328106+0000","last_active":"2024-05-09T06:34:02.329784+0000","last_peered":"2024-05-09T06:34:02.329784+0000","last_clean":"2024-05-09T06:34:02.329784+0000","last_became_active":"2024-05-09T06:34:02.327820+0000","last_became_peered":"2024-05-09T06:34:02.327820+0000","last_unstale":"2024-05-09T06:34:02.329784+0000","last_undegraded":"2024-05-09T06:34:02.329784+0000","last_fullsized":"2024-05-09T06:34:02.329784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:33.383852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318556+0000","last_change":"2024-05-09T06:34:02.318556+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318556+0000","last_undegraded":"2024-05-09T06:34:02.318556+0000","last_fullsized":"2024-05-09T06:34:02.318556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329527+0000","last_change":"2024-05-09T06:34:02.327204+0000","last_active":"2024-05-09T06:34:02.329527+0000","last_peered":"2024-05-09T06:34:02.329527+0000","last_clean":"2024-05-09T06:34:02.329527+0000","last_became_active":"2024-05-09T06:34:02.326876+0000","last_became_peered":"2024-05-09T06:34:02.326876+0000","last_unstale":"2024-05-09T06:34:02.329527+0000","last_undegraded":"2024-05-09T06:34:02.329527+0000","last_fullsized":"2024-05-09T06:34:02.329527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314614+0000","last_change":"2024-05-09T06:34:02.314614+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314614+0000","last_undegraded":"2024-05-09T06:34:02.314614+0000","last_fullsized":"2024-05-09T06:34:02.314614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328608+0000","last_change":"2024-05-09T06:34:02.327053+0000","last_active":"2024-05-09T06:34:02.328608+0000","last_peered":"2024-05-09T06:34:02.328608+0000","last_clean":"2024-05-09T06:34:02.328608+0000","last_became_active":"2024-05-09T06:34:02.326751+0000","last_became_peered":"2024-05-09T06:34:02.326751+0000","last_unstale":"2024-05-09T06:34:02.328608+0000","last_undegraded":"2024-05-09T06:34:02.328608+0000","last_fullsized":"2024-05-09T06:34:02.328608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:11.750079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313077+0000","last_change":"2024-05-09T06:34:02.313077+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313077+0000","last_undegraded":"2024-05-09T06:34:02.313077+0000","last_fullsized":"2024-05-09T06:34:02.313077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331465+0000","last_change":"2024-05-09T06:34:02.326680+0000","last_active":"2024-05-09T06:34:02.331465+0000","last_peered":"2024-05-09T06:34:02.331465+0000","last_clean":"2024-05-09T06:34:02.331465+0000","last_became_active":"2024-05-09T06:34:02.326353+0000","last_became_peered":"2024-05-09T06:34:02.326353+0000","last_unstale":"2024-05-09T06:34:02.331465+0000","last_undegraded":"2024-05-09T06:34:02.331465+0000","last_fullsized":"2024-05-09T06:34:02.331465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311571+0000","last_change":"2024-05-09T06:34:02.311571+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311571+0000","last_undegraded":"2024-05-09T06:34:02.311571+0000","last_fullsized":"2024-05-09T06:34:02.311571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329518+0000","last_change":"2024-05-09T06:34:02.327790+0000","last_active":"2024-05-09T06:34:02.329518+0000","last_peered":"2024-05-09T06:34:02.329518+0000","last_clean":"2024-05-09T06:34:02.329518+0000","last_became_active":"2024-05-09T06:34:02.327409+0000","last_became_peered":"2024-05-09T06:34:02.327409+0000","last_unstale":"2024-05-09T06:34:02.329518+0000","last_undegraded":"2024-05-09T06:34:02.329518+0000","last_fullsized":"2024-05-09T06:34:02.329518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.828382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311765+0000","last_change":"2024-05-09T06:34:02.311765+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311765+0000","last_undegraded":"2024-05-09T06:34:02.311765+0000","last_fullsized":"2024-05-09T06:34:02.311765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.317202+0000","last_change":"2024-05-09T06:34:02.314084+0000","last_active":"2024-05-09T06:34:02.317202+0000","last_peered":"2024-05-09T06:34:02.317202+0000","last_clean":"2024-05-09T06:34:02.317202+0000","last_became_active":"2024-05-09T06:34:02.313093+0000","last_became_peered":"2024-05-09T06:34:02.313093+0000","last_unstale":"2024-05-09T06:34:02.317202+0000","last_undegraded":"2024-05-09T06:34:02.317202+0000","last_fullsized":"2024-05-09T06:34:02.317202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:33.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321022+0000","last_change":"2024-05-09T06:34:02.321022+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321022+0000","last_undegraded":"2024-05-09T06:34:02.321022+0000","last_fullsized":"2024-05-09T06:34:02.321022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325577+0000","last_change":"2024-05-09T06:34:02.319167+0000","last_active":"2024-05-09T06:34:02.325577+0000","last_peered":"2024-05-09T06:34:02.325577+0000","last_clean":"2024-05-09T06:34:02.325577+0000","last_became_active":"2024-05-09T06:34:02.318846+0000","last_became_peered":"2024-05-09T06:34:02.318846+0000","last_unstale":"2024-05-09T06:34:02.325577+0000","last_undegraded":"2024-05-09T06:34:02.325577+0000","last_fullsized":"2024-05-09T06:34:02.325577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:19.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311082+0000","last_change":"2024-05-09T06:34:02.311082+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311082+0000","last_undegraded":"2024-05-09T06:34:02.311082+0000","last_fullsized":"2024-05-09T06:34:02.311082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.324239+0000","last_change":"2024-05-09T06:34:02.317990+0000","last_active":"2024-05-09T06:34:02.324239+0000","last_peered":"2024-05-09T06:34:02.324239+0000","last_clean":"2024-05-09T06:34:02.324239+0000","last_became_active":"2024-05-09T06:34:02.317688+0000","last_became_peered":"2024-05-09T06:34:02.317688+0000","last_unstale":"2024-05-09T06:34:02.324239+0000","last_undegraded":"2024-05-09T06:34:02.324239+0000","last_fullsized":"2024-05-09T06:34:02.324239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312002+0000","last_change":"2024-05-09T06:34:02.312002+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312002+0000","last_undegraded":"2024-05-09T06:34:02.312002+0000","last_fullsized":"2024-05-09T06:34:02.312002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329509+0000","last_change":"2024-05-09T06:34:02.326540+0000","last_active":"2024-05-09T06:34:02.329509+0000","last_peered":"2024-05-09T06:34:02.329509+0000","last_clean":"2024-05-09T06:34:02.329509+0000","last_became_active":"2024-05-09T06:34:02.326253+0000","last_became_peered":"2024-05-09T06:34:02.326253+0000","last_unstale":"2024-05-09T06:34:02.329509+0000","last_undegraded":"2024-05-09T06:34:02.329509+0000","last_fullsized":"2024-05-09T06:34:02.329509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312075+0000","last_change":"2024-05-09T06:34:02.312075+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312075+0000","last_undegraded":"2024-05-09T06:34:02.312075+0000","last_fullsized":"2024-05-09T06:34:02.312075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327577+0000","last_change":"2024-05-09T06:34:02.323636+0000","last_active":"2024-05-09T06:34:02.327577+0000","last_peered":"2024-05-09T06:34:02.327577+0000","last_clean":"2024-05-09T06:34:02.327577+0000","last_became_active":"2024-05-09T06:34:02.323275+0000","last_became_peered":"2024-05-09T06:34:02.323275+0000","last_unstale":"2024-05-09T06:34:02.327577+0000","last_undegraded":"2024-05-09T06:34:02.327577+0000","last_fullsized":"2024-05-09T06:34:02.327577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310800+0000","last_change":"2024-05-09T06:34:02.310800+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310800+0000","last_undegraded":"2024-05-09T06:34:02.310800+0000","last_fullsized":"2024-05-09T06:34:02.310800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331509+0000","last_change":"2024-05-09T06:34:02.328232+0000","last_active":"2024-05-09T06:34:02.331509+0000","last_peered":"2024-05-09T06:34:02.331509+0000","last_clean":"2024-05-09T06:34:02.331509+0000","last_became_active":"2024-05-09T06:34:02.327886+0000","last_became_peered":"2024-05-09T06:34:02.327886+0000","last_unstale":"2024-05-09T06:34:02.331509+0000","last_undegraded":"2024-05-09T06:34:02.331509+0000","last_fullsized":"2024-05-09T06:34:02.331509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315380+0000","last_change":"2024-05-09T06:34:02.315380+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315380+0000","last_undegraded":"2024-05-09T06:34:02.315380+0000","last_fullsized":"2024-05-09T06:34:02.315380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331857+0000","last_change":"2024-05-09T06:34:02.331212+0000","last_active":"2024-05-09T06:34:02.331857+0000","last_peered":"2024-05-09T06:34:02.331857+0000","last_clean":"2024-05-09T06:34:02.331857+0000","last_became_active":"2024-05-09T06:34:02.330939+0000","last_became_peered":"2024-05-09T06:34:02.330939+0000","last_unstale":"2024-05-09T06:34:02.331857+0000","last_undegraded":"2024-05-09T06:34:02.331857+0000","last_fullsized":"2024-05-09T06:34:02.331857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:30:48.018600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.316064+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:34:02.316064+0000","last_peered":"2024-05-09T06:34:02.316064+0000","last_clean":"2024-05-09T06:34:02.316064+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:34:02.316064+0000","last_undegraded":"2024-05-09T06:34:02.316064+0000","last_fullsized":"2024-05-09T06:34:02.316064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1087428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":497060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1094095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:08.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-05-09T06:34:08.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314422+0000', 'last_change': '2024-05-09T06:34:02.314422+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314422+0000', 'last_undegraded': '2024-05-09T06:34:02.314422+0000', 'last_fullsized': '2024-05-09T06:34:02.314422+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-05-09T06:34:08.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.320632+0000', 'last_change': '2024-05-09T06:34:02.320632+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.320632+0000', 'last_undegraded': '2024-05-09T06:34:02.320632+0000', 'last_fullsized': '2024-05-09T06:34:02.320632+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313945+0000', 'last_change': '2024-05-09T06:34:02.313945+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313945+0000', 'last_undegraded': '2024-05-09T06:34:02.313945+0000', 'last_fullsized': '2024-05-09T06:34:02.313945+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.318875+0000', 'last_change': '2024-05-09T06:34:02.318875+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.318875+0000', 'last_undegraded': '2024-05-09T06:34:02.318875+0000', 'last_fullsized': '2024-05-09T06:34:02.318875+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.308592+0000', 'last_change': '2024-05-09T06:34:02.308592+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.308592+0000', 'last_undegraded': '2024-05-09T06:34:02.308592+0000', 'last_fullsized': '2024-05-09T06:34:02.308592+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.316982+0000', 'last_change': '2024-05-09T06:34:02.316982+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.316982+0000', 'last_undegraded': '2024-05-09T06:34:02.316982+0000', 'last_fullsized': '2024-05-09T06:34:02.316982+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313987+0000', 'last_change': '2024-05-09T06:34:02.313987+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313987+0000', 'last_undegraded': '2024-05-09T06:34:02.313987+0000', 'last_fullsized': '2024-05-09T06:34:02.313987+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:34:08.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.320266+0000', 'last_change': '2024-05-09T06:34:02.320266+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.320266+0000', 'last_undegraded': '2024-05-09T06:34:02.320266+0000', 'last_fullsized': '2024-05-09T06:34:02.320266+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.307924+0000', 'last_change': '2024-05-09T06:34:02.307924+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.307924+0000', 'last_undegraded': '2024-05-09T06:34:02.307924+0000', 'last_fullsized': '2024-05-09T06:34:02.307924+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315657+0000', 'last_change': '2024-05-09T06:34:02.315657+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315657+0000', 'last_undegraded': '2024-05-09T06:34:02.315657+0000', 'last_fullsized': '2024-05-09T06:34:02.315657+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.308933+0000', 'last_change': '2024-05-09T06:34:02.308933+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.308933+0000', 'last_undegraded': '2024-05-09T06:34:02.308933+0000', 'last_fullsized': '2024-05-09T06:34:02.308933+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312738+0000', 'last_change': '2024-05-09T06:34:02.312738+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312738+0000', 'last_undegraded': '2024-05-09T06:34:02.312738+0000', 'last_fullsized': '2024-05-09T06:34:02.312738+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-05-09T06:34:08.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.321819+0000', 'last_change': '2024-05-09T06:34:02.321819+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.321819+0000', 'last_undegraded': '2024-05-09T06:34:02.321819+0000', 'last_fullsized': '2024-05-09T06:34:02.321819+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.310310+0000', 'last_change': '2024-05-09T06:34:02.310310+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.310310+0000', 'last_undegraded': '2024-05-09T06:34:02.310310+0000', 'last_fullsized': '2024-05-09T06:34:02.310310+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314242+0000', 'last_change': '2024-05-09T06:34:02.314242+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314242+0000', 'last_undegraded': '2024-05-09T06:34:02.314242+0000', 'last_fullsized': '2024-05-09T06:34:02.314242+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.316880+0000', 'last_change': '2024-05-09T06:34:02.316880+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.316880+0000', 'last_undegraded': '2024-05-09T06:34:02.316880+0000', 'last_fullsized': '2024-05-09T06:34:02.316880+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.316455+0000', 'last_change': '2024-05-09T06:34:02.316455+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.316455+0000', 'last_undegraded': '2024-05-09T06:34:02.316455+0000', 'last_fullsized': '2024-05-09T06:34:02.316455+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313420+0000', 'last_change': '2024-05-09T06:34:02.313420+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313420+0000', 'last_undegraded': '2024-05-09T06:34:02.313420+0000', 'last_fullsized': '2024-05-09T06:34:02.313420+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-05-09T06:34:08.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.308424+0000', 'last_change': '2024-05-09T06:34:02.308424+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.308424+0000', 'last_undegraded': '2024-05-09T06:34:02.308424+0000', 'last_fullsized': '2024-05-09T06:34:02.308424+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315522+0000', 'last_change': '2024-05-09T06:34:02.315522+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315522+0000', 'last_undegraded': '2024-05-09T06:34:02.315522+0000', 'last_fullsized': '2024-05-09T06:34:02.315522+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.309601+0000', 'last_change': '2024-05-09T06:34:02.309601+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.309601+0000', 'last_undegraded': '2024-05-09T06:34:02.309601+0000', 'last_fullsized': '2024-05-09T06:34:02.309601+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.309739+0000', 'last_change': '2024-05-09T06:34:02.309739+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.309739+0000', 'last_undegraded': '2024-05-09T06:34:02.309739+0000', 'last_fullsized': '2024-05-09T06:34:02.309739+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312820+0000', 'last_change': '2024-05-09T06:34:02.312820+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312820+0000', 'last_undegraded': '2024-05-09T06:34:02.312820+0000', 'last_fullsized': '2024-05-09T06:34:02.312820+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313125+0000', 'last_change': '2024-05-09T06:34:02.313125+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313125+0000', 'last_undegraded': '2024-05-09T06:34:02.313125+0000', 'last_fullsized': '2024-05-09T06:34:02.313125+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.321499+0000', 'last_change': '2024-05-09T06:34:02.321499+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.321499+0000', 'last_undegraded': '2024-05-09T06:34:02.321499+0000', 'last_fullsized': '2024-05-09T06:34:02.321499+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.310029+0000', 'last_change': '2024-05-09T06:34:02.310029+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.310029+0000', 'last_undegraded': '2024-05-09T06:34:02.310029+0000', 'last_fullsized': '2024-05-09T06:34:02.310029+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312353+0000', 'last_change': '2024-05-09T06:34:02.312353+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312353+0000', 'last_undegraded': '2024-05-09T06:34:02.312353+0000', 'last_fullsized': '2024-05-09T06:34:02.312353+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315046+0000', 'last_change': '2024-05-09T06:34:02.315046+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315046+0000', 'last_undegraded': '2024-05-09T06:34:02.315046+0000', 'last_fullsized': '2024-05-09T06:34:02.315046+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314059+0000', 'last_change': '2024-05-09T06:34:02.314059+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314059+0000', 'last_undegraded': '2024-05-09T06:34:02.314059+0000', 'last_fullsized': '2024-05-09T06:34:02.314059+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-05-09T06:34:08.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.309025+0000', 'last_change': '2024-05-09T06:34:02.309025+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.309025+0000', 'last_undegraded': '2024-05-09T06:34:02.309025+0000', 'last_fullsized': '2024-05-09T06:34:02.309025+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.320197+0000', 'last_change': '2024-05-09T06:34:02.320197+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.320197+0000', 'last_undegraded': '2024-05-09T06:34:02.320197+0000', 'last_fullsized': '2024-05-09T06:34:02.320197+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315167+0000', 'last_change': '2024-05-09T06:34:02.315167+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315167+0000', 'last_undegraded': '2024-05-09T06:34:02.315167+0000', 'last_fullsized': '2024-05-09T06:34:02.315167+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315036+0000', 'last_change': '2024-05-09T06:34:02.315036+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315036+0000', 'last_undegraded': '2024-05-09T06:34:02.315036+0000', 'last_fullsized': '2024-05-09T06:34:02.315036+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315873+0000', 'last_change': '2024-05-09T06:34:02.315873+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315873+0000', 'last_undegraded': '2024-05-09T06:34:02.315873+0000', 'last_fullsized': '2024-05-09T06:34:02.315873+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314308+0000', 'last_change': '2024-05-09T06:34:02.314308+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314308+0000', 'last_undegraded': '2024-05-09T06:34:02.314308+0000', 'last_fullsized': '2024-05-09T06:34:02.314308+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-05-09T06:34:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314090+0000', 'last_change': '2024-05-09T06:34:02.314090+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314090+0000', 'last_undegraded': '2024-05-09T06:34:02.314090+0000', 'last_fullsized': '2024-05-09T06:34:02.314090+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.319876+0000', 'last_change': '2024-05-09T06:34:02.319876+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.319876+0000', 'last_undegraded': '2024-05-09T06:34:02.319876+0000', 'last_fullsized': '2024-05-09T06:34:02.319876+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312032+0000', 'last_change': '2024-05-09T06:34:02.312032+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312032+0000', 'last_undegraded': '2024-05-09T06:34:02.312032+0000', 'last_fullsized': '2024-05-09T06:34:02.312032+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315492+0000', 'last_change': '2024-05-09T06:34:02.315492+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315492+0000', 'last_undegraded': '2024-05-09T06:34:02.315492+0000', 'last_fullsized': '2024-05-09T06:34:02.315492+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.316074+0000', 'last_change': '2024-05-09T06:34:02.316074+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.316074+0000', 'last_undegraded': '2024-05-09T06:34:02.316074+0000', 'last_fullsized': '2024-05-09T06:34:02.316074+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.307911+0000', 'last_change': '2024-05-09T06:34:02.307911+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.307911+0000', 'last_undegraded': '2024-05-09T06:34:02.307911+0000', 'last_fullsized': '2024-05-09T06:34:02.307911+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-05-09T06:34:08.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315128+0000', 'last_change': '2024-05-09T06:34:02.315128+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315128+0000', 'last_undegraded': '2024-05-09T06:34:02.315128+0000', 'last_fullsized': '2024-05-09T06:34:02.315128+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312609+0000', 'last_change': '2024-05-09T06:34:02.312609+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312609+0000', 'last_undegraded': '2024-05-09T06:34:02.312609+0000', 'last_fullsized': '2024-05-09T06:34:02.312609+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.318092+0000', 'last_change': '2024-05-09T06:34:02.318092+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.318092+0000', 'last_undegraded': '2024-05-09T06:34:02.318092+0000', 'last_fullsized': '2024-05-09T06:34:02.318092+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.316864+0000', 'last_change': '2024-05-09T06:34:02.316864+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.316864+0000', 'last_undegraded': '2024-05-09T06:34:02.316864+0000', 'last_fullsized': '2024-05-09T06:34:02.316864+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.307720+0000', 'last_change': '2024-05-09T06:34:02.307720+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.307720+0000', 'last_undegraded': '2024-05-09T06:34:02.307720+0000', 'last_fullsized': '2024-05-09T06:34:02.307720+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-05-09T06:34:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.311415+0000', 'last_change': '2024-05-09T06:34:02.311415+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.311415+0000', 'last_undegraded': '2024-05-09T06:34:02.311415+0000', 'last_fullsized': '2024-05-09T06:34:02.311415+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312364+0000', 'last_change': '2024-05-09T06:34:02.312364+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312364+0000', 'last_undegraded': '2024-05-09T06:34:02.312364+0000', 'last_fullsized': '2024-05-09T06:34:02.312364+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314602+0000', 'last_change': '2024-05-09T06:34:02.314602+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314602+0000', 'last_undegraded': '2024-05-09T06:34:02.314602+0000', 'last_fullsized': '2024-05-09T06:34:02.314602+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312055+0000', 'last_change': '2024-05-09T06:34:02.312055+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312055+0000', 'last_undegraded': '2024-05-09T06:34:02.312055+0000', 'last_fullsized': '2024-05-09T06:34:02.312055+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.308418+0000', 'last_change': '2024-05-09T06:34:02.308418+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.308418+0000', 'last_undegraded': '2024-05-09T06:34:02.308418+0000', 'last_fullsized': '2024-05-09T06:34:02.308418+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313072+0000', 'last_change': '2024-05-09T06:34:02.313072+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313072+0000', 'last_undegraded': '2024-05-09T06:34:02.313072+0000', 'last_fullsized': '2024-05-09T06:34:02.313072+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-05-09T06:34:08.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.319661+0000', 'last_change': '2024-05-09T06:34:02.319661+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.319661+0000', 'last_undegraded': '2024-05-09T06:34:02.319661+0000', 'last_fullsized': '2024-05-09T06:34:02.319661+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.318556+0000', 'last_change': '2024-05-09T06:34:02.318556+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.318556+0000', 'last_undegraded': '2024-05-09T06:34:02.318556+0000', 'last_fullsized': '2024-05-09T06:34:02.318556+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.314614+0000', 'last_change': '2024-05-09T06:34:02.314614+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.314614+0000', 'last_undegraded': '2024-05-09T06:34:02.314614+0000', 'last_fullsized': '2024-05-09T06:34:02.314614+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.313077+0000', 'last_change': '2024-05-09T06:34:02.313077+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.313077+0000', 'last_undegraded': '2024-05-09T06:34:02.313077+0000', 'last_fullsized': '2024-05-09T06:34:02.313077+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.311571+0000', 'last_change': '2024-05-09T06:34:02.311571+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.311571+0000', 'last_undegraded': '2024-05-09T06:34:02.311571+0000', 'last_fullsized': '2024-05-09T06:34:02.311571+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.311765+0000', 'last_change': '2024-05-09T06:34:02.311765+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.311765+0000', 'last_undegraded': '2024-05-09T06:34:02.311765+0000', 'last_fullsized': '2024-05-09T06:34:02.311765+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-05-09T06:34:08.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.321022+0000', 'last_change': '2024-05-09T06:34:02.321022+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.321022+0000', 'last_undegraded': '2024-05-09T06:34:02.321022+0000', 'last_fullsized': '2024-05-09T06:34:02.321022+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.311082+0000', 'last_change': '2024-05-09T06:34:02.311082+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.311082+0000', 'last_undegraded': '2024-05-09T06:34:02.311082+0000', 'last_fullsized': '2024-05-09T06:34:02.311082+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312002+0000', 'last_change': '2024-05-09T06:34:02.312002+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312002+0000', 'last_undegraded': '2024-05-09T06:34:02.312002+0000', 'last_fullsized': '2024-05-09T06:34:02.312002+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.312075+0000', 'last_change': '2024-05-09T06:34:02.312075+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.312075+0000', 'last_undegraded': '2024-05-09T06:34:02.312075+0000', 'last_fullsized': '2024-05-09T06:34:02.312075+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.310800+0000', 'last_change': '2024-05-09T06:34:02.310800+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.310800+0000', 'last_undegraded': '2024-05-09T06:34:02.310800+0000', 'last_fullsized': '2024-05-09T06:34:02.310800+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-05-09T06:34:08.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-05-09T06:34:02.315380+0000', 'last_change': '2024-05-09T06:34:02.315380+0000', 'last_active': '2024-05-09T06:34:02.301426+0000', 'last_peered': '2024-05-09T06:34:02.301426+0000', 'last_clean': '2024-05-09T06:34:02.301426+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T06:34:02.315380+0000', 'last_undegraded': '2024-05-09T06:34:02.315380+0000', 'last_fullsized': '2024-05-09T06:34:02.315380+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'last_clean_scrub_stamp': '2024-05-09T06:34:02.301426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T06:34:08.062 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-09T06:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.210+0000 7f3b8a84d640 1 Processor -- start 2024-05-09T06:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.210+0000 7f3b8a84d640 1 -- start start 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b8a84d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b8a84d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 0x7f3b8415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b8a84d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b840d04a0 con 0x7f3b840cbb60 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b8a84d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8415f690 con 0x7f3b8415ece0 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b8a84d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b840cbfe0 con 0x7f3b840b0080 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b88dea640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 0x7f3b8415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:55030/0 (socket says 172.21.15.1:55030) 2024-05-09T06:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.211+0000 7f3b83fff640 1 -- 172.21.15.1:0/4157701282 learned_addr learned my addr 172.21.15.1:0/4157701282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b82ffd640 1 -- 172.21.15.1:0/4157701282 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359810284 0 0) 0x7f3b840d04a0 con 0x7f3b840cbb60 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b83fff640 1 -- 172.21.15.1:0/4157701282 >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 msgr2=0x7f3b8415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b83fff640 1 --2- 172.21.15.1:0/4157701282 >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 0x7f3b8415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b83fff640 1 -- 172.21.15.1:0/4157701282 >> v1:172.21.15.6:6789/0 conn(0x7f3b840cbb60 legacy=0x7f3b8415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b83fff640 1 -- 172.21.15.1:0/4157701282 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b840722c0 con 0x7f3b840b0080 2024-05-09T06:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b88dea640 1 --2- 172.21.15.1:0/4157701282 >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 0x7f3b8415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:08.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.212+0000 7f3b83fff640 1 --2- 172.21.15.1:0/4157701282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd07f0d4348f447 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b82ffd640 1 -- 172.21.15.1:0/4157701282 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74037070 con 0x7f3b840b0080 2024-05-09T06:34:08.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b82ffd640 1 -- 172.21.15.1:0/4157701282 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b74032440 con 0x7f3b840b0080 2024-05-09T06:34:08.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b82ffd640 1 -- 172.21.15.1:0/4157701282 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74032810 con 0x7f3b840b0080 2024-05-09T06:34:08.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 -- 172.21.15.1:0/4157701282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 msgr2=0x7f3b84071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/4157701282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7402a270 comp rx=0 tx=0).stop 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 -- 172.21.15.1:0/4157701282 shutdown_connections 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/4157701282 >> v2:172.21.15.6:3300/0 conn(0x7f3b8415ece0 0x7f3b8415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/4157701282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840b0080 0x7f3b84071ce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 -- 172.21.15.1:0/4157701282 >> 172.21.15.1:0/4157701282 conn(0x7f3b840d0b50 msgr2=0x7f3b840d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 -- 172.21.15.1:0/4157701282 shutdown_connections 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.213+0000 7f3b8a84d640 1 -- 172.21.15.1:0/4157701282 wait complete. 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.214+0000 7f3b8a84d640 1 Processor -- start 2024-05-09T06:34:08.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 -- start start 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b840b0080 0x7f3b840c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840cbb60 0x7f3b84157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84072b60 con 0x7f3b8415ece0 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84072880 con 0x7f3b840b0080 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b8a84d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8415e7c0 con 0x7f3b840cbb60 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b88dea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b88dea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42516/0 (socket says 172.21.15.1:42516) 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b88dea640 1 -- 172.21.15.1:0/3766453145 learned_addr learned my addr 172.21.15.1:0/3766453145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b83fff640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b840b0080 0x7f3b840c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.215+0000 7f3b837fe640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840cbb60 0x7f3b84157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 -- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840cbb60 msgr2=0x7f3b84157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840cbb60 0x7f3b84157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 -- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b840b0080 msgr2=0x7f3b840c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b840b0080 0x7f3b840c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 -- 172.21.15.1:0/3766453145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b840c86d0 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b88dea640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00efc0 tx=0x7f3b7c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c018030 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b840c8940 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00cde0 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c003bb0 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.216+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b840ca610 con 0x7f3b8415ece0 2024-05-09T06:34:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.217+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b840ca810 con 0x7f3b8415ece0 2024-05-09T06:34:08.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.218+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b7c005d10 con 0x7f3b8415ece0 2024-05-09T06:34:08.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.218+0000 7f3b80ff9640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 0x7f3b7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.218+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3b7c08b8f0 con 0x7f3b8415ece0 2024-05-09T06:34:08.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.219+0000 7f3b83fff640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 0x7f3b7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.219+0000 7f3b83fff640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 0x7f3b7006e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.223+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b840ca810 con 0x7f3b8415ece0 2024-05-09T06:34:08.279 INFO:tasks.ceph.mds.d.smithi006.stderr:2024-05-09T06:34:08.299+0000 7fb75e56f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:34:08.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.478+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3b840752b0 con 0x7f3b8415ece0 2024-05-09T06:34:08.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.478+0000 7f3b80ff9640 1 -- 172.21.15.1:0/3766453145 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f3b7c022070 con 0x7f3b8415ece0 2024-05-09T06:34:08.458 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:08.458 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":431,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1715236027,"num_in_osds":8,"osd_in_since":1715236019,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265388032,"bytes_avail":772828725248,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":61,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7911}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-09T06:33:51.155862+0000","services":{}},"progress_events":{}} 2024-05-09T06:34:08.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 msgr2=0x7f3b7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 0x7f3b7006e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7402e000 comp rx=0 tx=0).stop 2024-05-09T06:34:08.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 msgr2=0x7f3b840b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00efc0 tx=0x7f3b7c009f90 comp rx=0 tx=0).stop 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 shutdown_connections 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b7006be30 0x7f3b7006e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b8415ece0 0x7f3b840b4be0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b840cbb60 0x7f3b84157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 --2- 172.21.15.1:0/3766453145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b840b0080 0x7f3b840c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 >> 172.21.15.1:0/3766453145 conn(0x7f3b840d0b50 msgr2=0x7f3b8415c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 shutdown_connections 2024-05-09T06:34:08.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.480+0000 7f3b8a84d640 1 -- 172.21.15.1:0/3766453145 wait complete. 2024-05-09T06:34:08.471 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-09T06:34:08.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:34:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.594+0000 7fadeffd5640 1 Processor -- start 2024-05-09T06:34:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.594+0000 7fadeffd5640 1 -- start start 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadeffd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 0x7fade8074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadeffd5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fade815c680 0x7fade80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadeffd5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fade8074a70 con 0x7fade80b0170 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadeffd5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fade815fd30 con 0x7fade815c680 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadeffd5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fade815cc40 con 0x7fade81565e0 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7faded570640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fade815c680 0x7fade80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadee572640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fade80b0170 0x7fade81601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59226/0 (socket says 172.21.15.1:59226) 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7fadee572640 1 -- 172.21.15.1:0/3608146515 learned_addr learned my addr 172.21.15.1:0/3608146515 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.595+0000 7faded570640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fade815c680 0x7fade80afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42526/0 (socket says 172.21.15.1:42526) 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7fadedd71640 1 --2- 172.21.15.1:0/3608146515 >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 0x7fade8074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7faded570640 1 -- 172.21.15.1:0/3608146515 >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 msgr2=0x7fade8074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7faded570640 1 --2- 172.21.15.1:0/3608146515 >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 0x7fade8074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7faded570640 1 -- 172.21.15.1:0/3608146515 >> v1:172.21.15.1:6789/0 conn(0x7fade80b0170 legacy=0x7fade81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7fadecd6f640 1 -- 172.21.15.1:0/3608146515 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216811267 0 0) 0x7fade8074a70 con 0x7fade80b0170 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7faded570640 1 -- 172.21.15.1:0/3608146515 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade815cf20 con 0x7fade815c680 2024-05-09T06:34:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7fadedd71640 1 --2- 172.21.15.1:0/3608146515 >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 0x7fade8074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.596+0000 7faded570640 1 --2- 172.21.15.1:0/3608146515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade815c680 0x7fade80afc30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fade80d13d0 tx=0x7faddc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d7460819141435 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadecd6f640 1 -- 172.21.15.1:0/3608146515 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc037070 con 0x7fade815c680 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadecd6f640 1 -- 172.21.15.1:0/3608146515 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faddc032440 con 0x7fade815c680 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadecd6f640 1 -- 172.21.15.1:0/3608146515 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc032810 con 0x7fade815c680 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 -- 172.21.15.1:0/3608146515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade815c680 msgr2=0x7fade80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 --2- 172.21.15.1:0/3608146515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade815c680 0x7fade80afc30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fade80d13d0 tx=0x7faddc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 -- 172.21.15.1:0/3608146515 shutdown_connections 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 --2- 172.21.15.1:0/3608146515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade815c680 0x7fade80afc30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 --2- 172.21.15.1:0/3608146515 >> v2:172.21.15.6:3300/0 conn(0x7fade81565e0 0x7fade8074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 -- 172.21.15.1:0/3608146515 >> 172.21.15.1:0/3608146515 conn(0x7fade80b1fb0 msgr2=0x7fade80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 -- 172.21.15.1:0/3608146515 shutdown_connections 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.597+0000 7fadeffd5640 1 -- 172.21.15.1:0/3608146515 wait complete. 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.598+0000 7fadeffd5640 1 Processor -- start 2024-05-09T06:34:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 -- start start 2024-05-09T06:34:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 0x7fade814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fade815c680 0x7fade81f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fade81569c0 con 0x7fade81565e0 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fade815d4e0 con 0x7fade80b0170 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7fadeffd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fade81608d0 con 0x7fade815c680 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7faded570640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7faded570640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42536/0 (socket says 172.21.15.1:42536) 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.599+0000 7faded570640 1 -- 172.21.15.1:0/1512309610 learned_addr learned my addr 172.21.15.1:0/1512309610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fadedd71640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 0x7fade814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 -- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fade815c680 msgr2=0x7fade81f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fade815c680 0x7fade81f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 -- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 msgr2=0x7fade814d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 0x7fade814d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 -- 172.21.15.1:0/1512309610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade81f53c0 con 0x7fade81565e0 2024-05-09T06:34:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fadedd71640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 0x7fade814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7faded570640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faddc009890 tx=0x7faddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc037040 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faddc004230 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc032e20 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade80b3810 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.600+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade80b3dd0 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.601+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade80b3fa0 con 0x7fade81565e0 2024-05-09T06:34:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.601+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faddc030770 con 0x7fade81565e0 2024-05-09T06:34:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.602+0000 7fade67fc640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 0x7fadd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.602+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7faddc0ac2b0 con 0x7fade81565e0 2024-05-09T06:34:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.602+0000 7fadedd71640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 0x7fadd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.602+0000 7fadedd71640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 0x7fadd406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.606+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faddc07ac80 con 0x7fade81565e0 2024-05-09T06:34:08.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.785+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fade80b5fb0 con 0x7fadd406be30 2024-05-09T06:34:08.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.791+0000 7fade67fc640 1 -- 172.21.15.1:0/1512309610 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7fade80b5fb0 con 0x7fadd406be30 2024-05-09T06:34:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:08.775 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:34:08.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 msgr2=0x7fadd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 0x7fadd406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0008040 comp rx=0 tx=0).stop 2024-05-09T06:34:08.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 msgr2=0x7fade80b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faddc009890 tx=0x7faddc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 shutdown_connections 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fadd406be30 0x7fadd406e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fade815c680 0x7fade81f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fade81565e0 0x7fade80b30a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 --2- 172.21.15.1:0/1512309610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fade80b0170 0x7fade814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 >> 172.21.15.1:0/1512309610 conn(0x7fade80b1fb0 msgr2=0x7fade8069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 shutdown_connections 2024-05-09T06:34:08.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:08.794+0000 7fadeffd5640 1 -- 172.21.15.1:0/1512309610 wait complete. 2024-05-09T06:34:08.785 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-09T06:34:07.158732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259168,"kb_used_data":9712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715552,"statfs":{"total":773094113280,"available":772828725248,"internally_reserved":0,"allocated":9945088,"data_stored":5197218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856752"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314422+0000","last_change":"2024-05-09T06:34:02.314422+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314422+0000","last_undegraded":"2024-05-09T06:34:02.314422+0000","last_fullsized":"2024-05-09T06:34:02.314422+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320632+0000","last_change":"2024-05-09T06:34:02.320632+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320632+0000","last_undegraded":"2024-05-09T06:34:02.320632+0000","last_fullsized":"2024-05-09T06:34:02.320632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313945+0000","last_change":"2024-05-09T06:34:02.313945+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313945+0000","last_undegraded":"2024-05-09T06:34:02.313945+0000","last_fullsized":"2024-05-09T06:34:02.313945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318875+0000","last_change":"2024-05-09T06:34:02.318875+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318875+0000","last_undegraded":"2024-05-09T06:34:02.318875+0000","last_fullsized":"2024-05-09T06:34:02.318875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308592+0000","last_change":"2024-05-09T06:34:02.308592+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308592+0000","last_undegraded":"2024-05-09T06:34:02.308592+0000","last_fullsized":"2024-05-09T06:34:02.308592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316982+0000","last_change":"2024-05-09T06:34:02.316982+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316982+0000","last_undegraded":"2024-05-09T06:34:02.316982+0000","last_fullsized":"2024-05-09T06:34:02.316982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313987+0000","last_change":"2024-05-09T06:34:02.313987+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313987+0000","last_undegraded":"2024-05-09T06:34:02.313987+0000","last_fullsized":"2024-05-09T06:34:02.313987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320266+0000","last_change":"2024-05-09T06:34:02.320266+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320266+0000","last_undegraded":"2024-05-09T06:34:02.320266+0000","last_fullsized":"2024-05-09T06:34:02.320266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307924+0000","last_change":"2024-05-09T06:34:02.307924+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307924+0000","last_undegraded":"2024-05-09T06:34:02.307924+0000","last_fullsized":"2024-05-09T06:34:02.307924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315657+0000","last_change":"2024-05-09T06:34:02.315657+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315657+0000","last_undegraded":"2024-05-09T06:34:02.315657+0000","last_fullsized":"2024-05-09T06:34:02.315657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308933+0000","last_change":"2024-05-09T06:34:02.308933+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308933+0000","last_undegraded":"2024-05-09T06:34:02.308933+0000","last_fullsized":"2024-05-09T06:34:02.308933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312738+0000","last_change":"2024-05-09T06:34:02.312738+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312738+0000","last_undegraded":"2024-05-09T06:34:02.312738+0000","last_fullsized":"2024-05-09T06:34:02.312738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321819+0000","last_change":"2024-05-09T06:34:02.321819+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321819+0000","last_undegraded":"2024-05-09T06:34:02.321819+0000","last_fullsized":"2024-05-09T06:34:02.321819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310310+0000","last_change":"2024-05-09T06:34:02.310310+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310310+0000","last_undegraded":"2024-05-09T06:34:02.310310+0000","last_fullsized":"2024-05-09T06:34:02.310310+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314242+0000","last_change":"2024-05-09T06:34:02.314242+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314242+0000","last_undegraded":"2024-05-09T06:34:02.314242+0000","last_fullsized":"2024-05-09T06:34:02.314242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316880+0000","last_change":"2024-05-09T06:34:02.316880+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316880+0000","last_undegraded":"2024-05-09T06:34:02.316880+0000","last_fullsized":"2024-05-09T06:34:02.316880+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316455+0000","last_change":"2024-05-09T06:34:02.316455+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316455+0000","last_undegraded":"2024-05-09T06:34:02.316455+0000","last_fullsized":"2024-05-09T06:34:02.316455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313420+0000","last_change":"2024-05-09T06:34:02.313420+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313420+0000","last_undegraded":"2024-05-09T06:34:02.313420+0000","last_fullsized":"2024-05-09T06:34:02.313420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308424+0000","last_change":"2024-05-09T06:34:02.308424+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308424+0000","last_undegraded":"2024-05-09T06:34:02.308424+0000","last_fullsized":"2024-05-09T06:34:02.308424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315522+0000","last_change":"2024-05-09T06:34:02.315522+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315522+0000","last_undegraded":"2024-05-09T06:34:02.315522+0000","last_fullsized":"2024-05-09T06:34:02.315522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309601+0000","last_change":"2024-05-09T06:34:02.309601+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309601+0000","last_undegraded":"2024-05-09T06:34:02.309601+0000","last_fullsized":"2024-05-09T06:34:02.309601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309739+0000","last_change":"2024-05-09T06:34:02.309739+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309739+0000","last_undegraded":"2024-05-09T06:34:02.309739+0000","last_fullsized":"2024-05-09T06:34:02.309739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312820+0000","last_change":"2024-05-09T06:34:02.312820+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312820+0000","last_undegraded":"2024-05-09T06:34:02.312820+0000","last_fullsized":"2024-05-09T06:34:02.312820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313125+0000","last_change":"2024-05-09T06:34:02.313125+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313125+0000","last_undegraded":"2024-05-09T06:34:02.313125+0000","last_fullsized":"2024-05-09T06:34:02.313125+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321499+0000","last_change":"2024-05-09T06:34:02.321499+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321499+0000","last_undegraded":"2024-05-09T06:34:02.321499+0000","last_fullsized":"2024-05-09T06:34:02.321499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310029+0000","last_change":"2024-05-09T06:34:02.310029+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310029+0000","last_undegraded":"2024-05-09T06:34:02.310029+0000","last_fullsized":"2024-05-09T06:34:02.310029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312353+0000","last_change":"2024-05-09T06:34:02.312353+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312353+0000","last_undegraded":"2024-05-09T06:34:02.312353+0000","last_fullsized":"2024-05-09T06:34:02.312353+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315046+0000","last_change":"2024-05-09T06:34:02.315046+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315046+0000","last_undegraded":"2024-05-09T06:34:02.315046+0000","last_fullsized":"2024-05-09T06:34:02.315046+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314059+0000","last_change":"2024-05-09T06:34:02.314059+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314059+0000","last_undegraded":"2024-05-09T06:34:02.314059+0000","last_fullsized":"2024-05-09T06:34:02.314059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.309025+0000","last_change":"2024-05-09T06:34:02.309025+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.309025+0000","last_undegraded":"2024-05-09T06:34:02.309025+0000","last_fullsized":"2024-05-09T06:34:02.309025+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.320197+0000","last_change":"2024-05-09T06:34:02.320197+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.320197+0000","last_undegraded":"2024-05-09T06:34:02.320197+0000","last_fullsized":"2024-05-09T06:34:02.320197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315167+0000","last_change":"2024-05-09T06:34:02.315167+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315167+0000","last_undegraded":"2024-05-09T06:34:02.315167+0000","last_fullsized":"2024-05-09T06:34:02.315167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315036+0000","last_change":"2024-05-09T06:34:02.315036+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315036+0000","last_undegraded":"2024-05-09T06:34:02.315036+0000","last_fullsized":"2024-05-09T06:34:02.315036+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.321306+0000","last_change":"2024-05-09T06:34:02.315932+0000","last_active":"2024-05-09T06:34:02.321306+0000","last_peered":"2024-05-09T06:34:02.321306+0000","last_clean":"2024-05-09T06:34:02.321306+0000","last_became_active":"2024-05-09T06:34:02.315556+0000","last_became_peered":"2024-05-09T06:34:02.315556+0000","last_unstale":"2024-05-09T06:34:02.321306+0000","last_undegraded":"2024-05-09T06:34:02.321306+0000","last_fullsized":"2024-05-09T06:34:02.321306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:50.355406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315873+0000","last_change":"2024-05-09T06:34:02.315873+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315873+0000","last_undegraded":"2024-05-09T06:34:02.315873+0000","last_fullsized":"2024-05-09T06:34:02.315873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325355+0000","last_change":"2024-05-09T06:34:02.317231+0000","last_active":"2024-05-09T06:34:02.325355+0000","last_peered":"2024-05-09T06:34:02.325355+0000","last_clean":"2024-05-09T06:34:02.325355+0000","last_became_active":"2024-05-09T06:34:02.316853+0000","last_became_peered":"2024-05-09T06:34:02.316853+0000","last_unstale":"2024-05-09T06:34:02.325355+0000","last_undegraded":"2024-05-09T06:34:02.325355+0000","last_fullsized":"2024-05-09T06:34:02.325355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:47.234272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314308+0000","last_change":"2024-05-09T06:34:02.314308+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314308+0000","last_undegraded":"2024-05-09T06:34:02.314308+0000","last_fullsized":"2024-05-09T06:34:02.314308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325501+0000","last_change":"2024-05-09T06:34:02.322212+0000","last_active":"2024-05-09T06:34:02.325501+0000","last_peered":"2024-05-09T06:34:02.325501+0000","last_clean":"2024-05-09T06:34:02.325501+0000","last_became_active":"2024-05-09T06:34:02.321940+0000","last_became_peered":"2024-05-09T06:34:02.321940+0000","last_unstale":"2024-05-09T06:34:02.325501+0000","last_undegraded":"2024-05-09T06:34:02.325501+0000","last_fullsized":"2024-05-09T06:34:02.325501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314090+0000","last_change":"2024-05-09T06:34:02.314090+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314090+0000","last_undegraded":"2024-05-09T06:34:02.314090+0000","last_fullsized":"2024-05-09T06:34:02.314090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.334420+0000","last_change":"2024-05-09T06:34:02.328490+0000","last_active":"2024-05-09T06:34:02.334420+0000","last_peered":"2024-05-09T06:34:02.334420+0000","last_clean":"2024-05-09T06:34:02.334420+0000","last_became_active":"2024-05-09T06:34:02.328118+0000","last_became_peered":"2024-05-09T06:34:02.328118+0000","last_unstale":"2024-05-09T06:34:02.334420+0000","last_undegraded":"2024-05-09T06:34:02.334420+0000","last_fullsized":"2024-05-09T06:34:02.334420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319876+0000","last_change":"2024-05-09T06:34:02.319876+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319876+0000","last_undegraded":"2024-05-09T06:34:02.319876+0000","last_fullsized":"2024-05-09T06:34:02.319876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325837+0000","last_change":"2024-05-09T06:34:02.320157+0000","last_active":"2024-05-09T06:34:02.325837+0000","last_peered":"2024-05-09T06:34:02.325837+0000","last_clean":"2024-05-09T06:34:02.325837+0000","last_became_active":"2024-05-09T06:34:02.319814+0000","last_became_peered":"2024-05-09T06:34:02.319814+0000","last_unstale":"2024-05-09T06:34:02.325837+0000","last_undegraded":"2024-05-09T06:34:02.325837+0000","last_fullsized":"2024-05-09T06:34:02.325837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:05.426920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312032+0000","last_change":"2024-05-09T06:34:02.312032+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312032+0000","last_undegraded":"2024-05-09T06:34:02.312032+0000","last_fullsized":"2024-05-09T06:34:02.312032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330331+0000","last_change":"2024-05-09T06:34:02.329390+0000","last_active":"2024-05-09T06:34:02.330331+0000","last_peered":"2024-05-09T06:34:02.330331+0000","last_clean":"2024-05-09T06:34:02.330331+0000","last_became_active":"2024-05-09T06:34:02.329108+0000","last_became_peered":"2024-05-09T06:34:02.329108+0000","last_unstale":"2024-05-09T06:34:02.330331+0000","last_undegraded":"2024-05-09T06:34:02.330331+0000","last_fullsized":"2024-05-09T06:34:02.330331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:08.397934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315492+0000","last_change":"2024-05-09T06:34:02.315492+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315492+0000","last_undegraded":"2024-05-09T06:34:02.315492+0000","last_fullsized":"2024-05-09T06:34:02.315492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326273+0000","last_change":"2024-05-09T06:34:02.324159+0000","last_active":"2024-05-09T06:34:02.326273+0000","last_peered":"2024-05-09T06:34:02.326273+0000","last_clean":"2024-05-09T06:34:02.326273+0000","last_became_active":"2024-05-09T06:34:02.323875+0000","last_became_peered":"2024-05-09T06:34:02.323875+0000","last_unstale":"2024-05-09T06:34:02.326273+0000","last_undegraded":"2024-05-09T06:34:02.326273+0000","last_fullsized":"2024-05-09T06:34:02.326273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316074+0000","last_change":"2024-05-09T06:34:02.316074+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316074+0000","last_undegraded":"2024-05-09T06:34:02.316074+0000","last_fullsized":"2024-05-09T06:34:02.316074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.326331+0000","last_change":"2024-05-09T06:34:02.324681+0000","last_active":"2024-05-09T06:34:02.326331+0000","last_peered":"2024-05-09T06:34:02.326331+0000","last_clean":"2024-05-09T06:34:02.326331+0000","last_became_active":"2024-05-09T06:34:02.324383+0000","last_became_peered":"2024-05-09T06:34:02.324383+0000","last_unstale":"2024-05-09T06:34:02.326331+0000","last_undegraded":"2024-05-09T06:34:02.326331+0000","last_fullsized":"2024-05-09T06:34:02.326331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307911+0000","last_change":"2024-05-09T06:34:02.307911+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307911+0000","last_undegraded":"2024-05-09T06:34:02.307911+0000","last_fullsized":"2024-05-09T06:34:02.307911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328828+0000","last_change":"2024-05-09T06:34:02.322716+0000","last_active":"2024-05-09T06:34:02.328828+0000","last_peered":"2024-05-09T06:34:02.328828+0000","last_clean":"2024-05-09T06:34:02.328828+0000","last_became_active":"2024-05-09T06:34:02.322190+0000","last_became_peered":"2024-05-09T06:34:02.322190+0000","last_unstale":"2024-05-09T06:34:02.328828+0000","last_undegraded":"2024-05-09T06:34:02.328828+0000","last_fullsized":"2024-05-09T06:34:02.328828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315128+0000","last_change":"2024-05-09T06:34:02.315128+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315128+0000","last_undegraded":"2024-05-09T06:34:02.315128+0000","last_fullsized":"2024-05-09T06:34:02.315128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329063+0000","last_change":"2024-05-09T06:34:02.328152+0000","last_active":"2024-05-09T06:34:02.329063+0000","last_peered":"2024-05-09T06:34:02.329063+0000","last_clean":"2024-05-09T06:34:02.329063+0000","last_became_active":"2024-05-09T06:34:02.327851+0000","last_became_peered":"2024-05-09T06:34:02.327851+0000","last_unstale":"2024-05-09T06:34:02.329063+0000","last_undegraded":"2024-05-09T06:34:02.329063+0000","last_fullsized":"2024-05-09T06:34:02.329063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312609+0000","last_change":"2024-05-09T06:34:02.312609+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312609+0000","last_undegraded":"2024-05-09T06:34:02.312609+0000","last_fullsized":"2024-05-09T06:34:02.312609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325434+0000","last_change":"2024-05-09T06:34:02.318316+0000","last_active":"2024-05-09T06:34:02.325434+0000","last_peered":"2024-05-09T06:34:02.325434+0000","last_clean":"2024-05-09T06:34:02.325434+0000","last_became_active":"2024-05-09T06:34:02.318030+0000","last_became_peered":"2024-05-09T06:34:02.318030+0000","last_unstale":"2024-05-09T06:34:02.325434+0000","last_undegraded":"2024-05-09T06:34:02.325434+0000","last_fullsized":"2024-05-09T06:34:02.325434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:15.498412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318092+0000","last_change":"2024-05-09T06:34:02.318092+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318092+0000","last_undegraded":"2024-05-09T06:34:02.318092+0000","last_fullsized":"2024-05-09T06:34:02.318092+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323563+0000","last_change":"2024-05-09T06:34:02.318480+0000","last_active":"2024-05-09T06:34:02.323563+0000","last_peered":"2024-05-09T06:34:02.323563+0000","last_clean":"2024-05-09T06:34:02.323563+0000","last_became_active":"2024-05-09T06:34:02.318175+0000","last_became_peered":"2024-05-09T06:34:02.318175+0000","last_unstale":"2024-05-09T06:34:02.323563+0000","last_undegraded":"2024-05-09T06:34:02.323563+0000","last_fullsized":"2024-05-09T06:34:02.323563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:49.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.316864+0000","last_change":"2024-05-09T06:34:02.316864+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.316864+0000","last_undegraded":"2024-05-09T06:34:02.316864+0000","last_fullsized":"2024-05-09T06:34:02.316864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329491+0000","last_change":"2024-05-09T06:34:02.325683+0000","last_active":"2024-05-09T06:34:02.329491+0000","last_peered":"2024-05-09T06:34:02.329491+0000","last_clean":"2024-05-09T06:34:02.329491+0000","last_became_active":"2024-05-09T06:34:02.325346+0000","last_became_peered":"2024-05-09T06:34:02.325346+0000","last_unstale":"2024-05-09T06:34:02.329491+0000","last_undegraded":"2024-05-09T06:34:02.329491+0000","last_fullsized":"2024-05-09T06:34:02.329491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.307720+0000","last_change":"2024-05-09T06:34:02.307720+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.307720+0000","last_undegraded":"2024-05-09T06:34:02.307720+0000","last_fullsized":"2024-05-09T06:34:02.307720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.323742+0000","last_change":"2024-05-09T06:34:02.318607+0000","last_active":"2024-05-09T06:34:02.323742+0000","last_peered":"2024-05-09T06:34:02.323742+0000","last_clean":"2024-05-09T06:34:02.323742+0000","last_became_active":"2024-05-09T06:34:02.318297+0000","last_became_peered":"2024-05-09T06:34:02.318297+0000","last_unstale":"2024-05-09T06:34:02.323742+0000","last_undegraded":"2024-05-09T06:34:02.323742+0000","last_fullsized":"2024-05-09T06:34:02.323742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:25.837322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311415+0000","last_change":"2024-05-09T06:34:02.311415+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311415+0000","last_undegraded":"2024-05-09T06:34:02.311415+0000","last_fullsized":"2024-05-09T06:34:02.311415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330020+0000","last_change":"2024-05-09T06:34:02.326169+0000","last_active":"2024-05-09T06:34:02.330020+0000","last_peered":"2024-05-09T06:34:02.330020+0000","last_clean":"2024-05-09T06:34:02.330020+0000","last_became_active":"2024-05-09T06:34:02.325706+0000","last_became_peered":"2024-05-09T06:34:02.325706+0000","last_unstale":"2024-05-09T06:34:02.330020+0000","last_undegraded":"2024-05-09T06:34:02.330020+0000","last_fullsized":"2024-05-09T06:34:02.330020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:28.820898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312364+0000","last_change":"2024-05-09T06:34:02.312364+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312364+0000","last_undegraded":"2024-05-09T06:34:02.312364+0000","last_fullsized":"2024-05-09T06:34:02.312364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.330677+0000","last_change":"2024-05-09T06:34:02.330110+0000","last_active":"2024-05-09T06:34:02.330677+0000","last_peered":"2024-05-09T06:34:02.330677+0000","last_clean":"2024-05-09T06:34:02.330677+0000","last_became_active":"2024-05-09T06:34:02.329847+0000","last_became_peered":"2024-05-09T06:34:02.329847+0000","last_unstale":"2024-05-09T06:34:02.330677+0000","last_undegraded":"2024-05-09T06:34:02.330677+0000","last_fullsized":"2024-05-09T06:34:02.330677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:16.125143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314602+0000","last_change":"2024-05-09T06:34:02.314602+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314602+0000","last_undegraded":"2024-05-09T06:34:02.314602+0000","last_fullsized":"2024-05-09T06:34:02.314602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327482+0000","last_change":"2024-05-09T06:34:02.322056+0000","last_active":"2024-05-09T06:34:02.327482+0000","last_peered":"2024-05-09T06:34:02.327482+0000","last_clean":"2024-05-09T06:34:02.327482+0000","last_became_active":"2024-05-09T06:34:02.321765+0000","last_became_peered":"2024-05-09T06:34:02.321765+0000","last_unstale":"2024-05-09T06:34:02.327482+0000","last_undegraded":"2024-05-09T06:34:02.327482+0000","last_fullsized":"2024-05-09T06:34:02.327482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:12.987046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312055+0000","last_change":"2024-05-09T06:34:02.312055+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312055+0000","last_undegraded":"2024-05-09T06:34:02.312055+0000","last_fullsized":"2024-05-09T06:34:02.312055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331487+0000","last_change":"2024-05-09T06:34:02.327732+0000","last_active":"2024-05-09T06:34:02.331487+0000","last_peered":"2024-05-09T06:34:02.331487+0000","last_clean":"2024-05-09T06:34:02.331487+0000","last_became_active":"2024-05-09T06:34:02.327311+0000","last_became_peered":"2024-05-09T06:34:02.327311+0000","last_unstale":"2024-05-09T06:34:02.331487+0000","last_undegraded":"2024-05-09T06:34:02.331487+0000","last_fullsized":"2024-05-09T06:34:02.331487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.308418+0000","last_change":"2024-05-09T06:34:02.308418+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.308418+0000","last_undegraded":"2024-05-09T06:34:02.308418+0000","last_fullsized":"2024-05-09T06:34:02.308418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331858+0000","last_change":"2024-05-09T06:34:02.329189+0000","last_active":"2024-05-09T06:34:02.331858+0000","last_peered":"2024-05-09T06:34:02.331858+0000","last_clean":"2024-05-09T06:34:02.331858+0000","last_became_active":"2024-05-09T06:34:02.328869+0000","last_became_peered":"2024-05-09T06:34:02.328869+0000","last_unstale":"2024-05-09T06:34:02.331858+0000","last_undegraded":"2024-05-09T06:34:02.331858+0000","last_fullsized":"2024-05-09T06:34:02.331858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:55.195087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313072+0000","last_change":"2024-05-09T06:34:02.313072+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313072+0000","last_undegraded":"2024-05-09T06:34:02.313072+0000","last_fullsized":"2024-05-09T06:34:02.313072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329582+0000","last_change":"2024-05-09T06:34:02.327306+0000","last_active":"2024-05-09T06:34:02.329582+0000","last_peered":"2024-05-09T06:34:02.329582+0000","last_clean":"2024-05-09T06:34:02.329582+0000","last_became_active":"2024-05-09T06:34:02.326990+0000","last_became_peered":"2024-05-09T06:34:02.326990+0000","last_unstale":"2024-05-09T06:34:02.329582+0000","last_undegraded":"2024-05-09T06:34:02.329582+0000","last_fullsized":"2024-05-09T06:34:02.329582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.319661+0000","last_change":"2024-05-09T06:34:02.319661+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.319661+0000","last_undegraded":"2024-05-09T06:34:02.319661+0000","last_fullsized":"2024-05-09T06:34:02.319661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329784+0000","last_change":"2024-05-09T06:34:02.328106+0000","last_active":"2024-05-09T06:34:02.329784+0000","last_peered":"2024-05-09T06:34:02.329784+0000","last_clean":"2024-05-09T06:34:02.329784+0000","last_became_active":"2024-05-09T06:34:02.327820+0000","last_became_peered":"2024-05-09T06:34:02.327820+0000","last_unstale":"2024-05-09T06:34:02.329784+0000","last_undegraded":"2024-05-09T06:34:02.329784+0000","last_fullsized":"2024-05-09T06:34:02.329784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:33.383852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.318556+0000","last_change":"2024-05-09T06:34:02.318556+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.318556+0000","last_undegraded":"2024-05-09T06:34:02.318556+0000","last_fullsized":"2024-05-09T06:34:02.318556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329527+0000","last_change":"2024-05-09T06:34:02.327204+0000","last_active":"2024-05-09T06:34:02.329527+0000","last_peered":"2024-05-09T06:34:02.329527+0000","last_clean":"2024-05-09T06:34:02.329527+0000","last_became_active":"2024-05-09T06:34:02.326876+0000","last_became_peered":"2024-05-09T06:34:02.326876+0000","last_unstale":"2024-05-09T06:34:02.329527+0000","last_undegraded":"2024-05-09T06:34:02.329527+0000","last_fullsized":"2024-05-09T06:34:02.329527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.314614+0000","last_change":"2024-05-09T06:34:02.314614+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.314614+0000","last_undegraded":"2024-05-09T06:34:02.314614+0000","last_fullsized":"2024-05-09T06:34:02.314614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.328608+0000","last_change":"2024-05-09T06:34:02.327053+0000","last_active":"2024-05-09T06:34:02.328608+0000","last_peered":"2024-05-09T06:34:02.328608+0000","last_clean":"2024-05-09T06:34:02.328608+0000","last_became_active":"2024-05-09T06:34:02.326751+0000","last_became_peered":"2024-05-09T06:34:02.326751+0000","last_unstale":"2024-05-09T06:34:02.328608+0000","last_undegraded":"2024-05-09T06:34:02.328608+0000","last_fullsized":"2024-05-09T06:34:02.328608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:11.750079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.313077+0000","last_change":"2024-05-09T06:34:02.313077+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.313077+0000","last_undegraded":"2024-05-09T06:34:02.313077+0000","last_fullsized":"2024-05-09T06:34:02.313077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331465+0000","last_change":"2024-05-09T06:34:02.326680+0000","last_active":"2024-05-09T06:34:02.331465+0000","last_peered":"2024-05-09T06:34:02.331465+0000","last_clean":"2024-05-09T06:34:02.331465+0000","last_became_active":"2024-05-09T06:34:02.326353+0000","last_became_peered":"2024-05-09T06:34:02.326353+0000","last_unstale":"2024-05-09T06:34:02.331465+0000","last_undegraded":"2024-05-09T06:34:02.331465+0000","last_fullsized":"2024-05-09T06:34:02.331465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311571+0000","last_change":"2024-05-09T06:34:02.311571+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311571+0000","last_undegraded":"2024-05-09T06:34:02.311571+0000","last_fullsized":"2024-05-09T06:34:02.311571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329518+0000","last_change":"2024-05-09T06:34:02.327790+0000","last_active":"2024-05-09T06:34:02.329518+0000","last_peered":"2024-05-09T06:34:02.329518+0000","last_clean":"2024-05-09T06:34:02.329518+0000","last_became_active":"2024-05-09T06:34:02.327409+0000","last_became_peered":"2024-05-09T06:34:02.327409+0000","last_unstale":"2024-05-09T06:34:02.329518+0000","last_undegraded":"2024-05-09T06:34:02.329518+0000","last_fullsized":"2024-05-09T06:34:02.329518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.828382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311765+0000","last_change":"2024-05-09T06:34:02.311765+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311765+0000","last_undegraded":"2024-05-09T06:34:02.311765+0000","last_fullsized":"2024-05-09T06:34:02.311765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.317202+0000","last_change":"2024-05-09T06:34:02.314084+0000","last_active":"2024-05-09T06:34:02.317202+0000","last_peered":"2024-05-09T06:34:02.317202+0000","last_clean":"2024-05-09T06:34:02.317202+0000","last_became_active":"2024-05-09T06:34:02.313093+0000","last_became_peered":"2024-05-09T06:34:02.313093+0000","last_unstale":"2024-05-09T06:34:02.317202+0000","last_undegraded":"2024-05-09T06:34:02.317202+0000","last_fullsized":"2024-05-09T06:34:02.317202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:33.925697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.321022+0000","last_change":"2024-05-09T06:34:02.321022+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.321022+0000","last_undegraded":"2024-05-09T06:34:02.321022+0000","last_fullsized":"2024-05-09T06:34:02.321022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.325577+0000","last_change":"2024-05-09T06:34:02.319167+0000","last_active":"2024-05-09T06:34:02.325577+0000","last_peered":"2024-05-09T06:34:02.325577+0000","last_clean":"2024-05-09T06:34:02.325577+0000","last_became_active":"2024-05-09T06:34:02.318846+0000","last_became_peered":"2024-05-09T06:34:02.318846+0000","last_unstale":"2024-05-09T06:34:02.325577+0000","last_undegraded":"2024-05-09T06:34:02.325577+0000","last_fullsized":"2024-05-09T06:34:02.325577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:19.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.311082+0000","last_change":"2024-05-09T06:34:02.311082+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.311082+0000","last_undegraded":"2024-05-09T06:34:02.311082+0000","last_fullsized":"2024-05-09T06:34:02.311082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.324239+0000","last_change":"2024-05-09T06:34:02.317990+0000","last_active":"2024-05-09T06:34:02.324239+0000","last_peered":"2024-05-09T06:34:02.324239+0000","last_clean":"2024-05-09T06:34:02.324239+0000","last_became_active":"2024-05-09T06:34:02.317688+0000","last_became_peered":"2024-05-09T06:34:02.317688+0000","last_unstale":"2024-05-09T06:34:02.324239+0000","last_undegraded":"2024-05-09T06:34:02.324239+0000","last_fullsized":"2024-05-09T06:34:02.324239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:25.499439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312002+0000","last_change":"2024-05-09T06:34:02.312002+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312002+0000","last_undegraded":"2024-05-09T06:34:02.312002+0000","last_fullsized":"2024-05-09T06:34:02.312002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.329509+0000","last_change":"2024-05-09T06:34:02.326540+0000","last_active":"2024-05-09T06:34:02.329509+0000","last_peered":"2024-05-09T06:34:02.329509+0000","last_clean":"2024-05-09T06:34:02.329509+0000","last_became_active":"2024-05-09T06:34:02.326253+0000","last_became_peered":"2024-05-09T06:34:02.326253+0000","last_unstale":"2024-05-09T06:34:02.329509+0000","last_undegraded":"2024-05-09T06:34:02.329509+0000","last_fullsized":"2024-05-09T06:34:02.329509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:29.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.312075+0000","last_change":"2024-05-09T06:34:02.312075+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.312075+0000","last_undegraded":"2024-05-09T06:34:02.312075+0000","last_fullsized":"2024-05-09T06:34:02.312075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.327577+0000","last_change":"2024-05-09T06:34:02.323636+0000","last_active":"2024-05-09T06:34:02.327577+0000","last_peered":"2024-05-09T06:34:02.327577+0000","last_clean":"2024-05-09T06:34:02.327577+0000","last_became_active":"2024-05-09T06:34:02.323275+0000","last_became_peered":"2024-05-09T06:34:02.323275+0000","last_unstale":"2024-05-09T06:34:02.327577+0000","last_undegraded":"2024-05-09T06:34:02.327577+0000","last_fullsized":"2024-05-09T06:34:02.327577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.310800+0000","last_change":"2024-05-09T06:34:02.310800+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.310800+0000","last_undegraded":"2024-05-09T06:34:02.310800+0000","last_fullsized":"2024-05-09T06:34:02.310800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331509+0000","last_change":"2024-05-09T06:34:02.328232+0000","last_active":"2024-05-09T06:34:02.331509+0000","last_peered":"2024-05-09T06:34:02.331509+0000","last_clean":"2024-05-09T06:34:02.331509+0000","last_became_active":"2024-05-09T06:34:02.327886+0000","last_became_peered":"2024-05-09T06:34:02.327886+0000","last_unstale":"2024-05-09T06:34:02.331509+0000","last_undegraded":"2024-05-09T06:34:02.331509+0000","last_fullsized":"2024-05-09T06:34:02.331509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-05-09T06:34:02.315380+0000","last_change":"2024-05-09T06:34:02.315380+0000","last_active":"2024-05-09T06:34:02.301426+0000","last_peered":"2024-05-09T06:34:02.301426+0000","last_clean":"2024-05-09T06:34:02.301426+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T06:34:02.315380+0000","last_undegraded":"2024-05-09T06:34:02.315380+0000","last_fullsized":"2024-05-09T06:34:02.315380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.331857+0000","last_change":"2024-05-09T06:34:02.331212+0000","last_active":"2024-05-09T06:34:02.331857+0000","last_peered":"2024-05-09T06:34:02.331857+0000","last_clean":"2024-05-09T06:34:02.331857+0000","last_became_active":"2024-05-09T06:34:02.330939+0000","last_became_peered":"2024-05-09T06:34:02.330939+0000","last_unstale":"2024-05-09T06:34:02.331857+0000","last_undegraded":"2024-05-09T06:34:02.331857+0000","last_fullsized":"2024-05-09T06:34:02.331857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:30:48.018600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T06:34:02.316064+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:34:02.316064+0000","last_peered":"2024-05-09T06:34:02.316064+0000","last_clean":"2024-05-09T06:34:02.316064+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:34:02.316064+0000","last_undegraded":"2024-05-09T06:34:02.316064+0000","last_fullsized":"2024-05-09T06:34:02.316064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1087428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":497060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":503727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1094095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:11.790 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:34:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.925+0000 7fcdde5a9640 1 Processor -- start 2024-05-09T06:34:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.925+0000 7fcdde5a9640 1 -- start start 2024-05-09T06:34:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcdde5a9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcdd80b2dd0 0x7fcdd80d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcdde5a9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd80d2b10 con 0x7fcdd80d2710 2024-05-09T06:34:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcdde5a9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd80d0870 con 0x7fcdd81617a0 2024-05-09T06:34:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcdde5a9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd80d1480 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcddcb46640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcdd80d2710 0x7fcdd80ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59240/0 (socket says 172.21.15.1:59240) 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.926+0000 7fcddcb46640 1 -- 172.21.15.1:0/4112053120 learned_addr learned my addr 172.21.15.1:0/4112053120 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd77fe640 1 --2- 172.21.15.1:0/4112053120 >> v2:172.21.15.6:3300/0 conn(0x7fcdd80b2dd0 0x7fcdd80d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd6ffd640 1 -- 172.21.15.1:0/4112053120 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759154484 0 0) 0x7fcdd80d2b10 con 0x7fcdd80d2710 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd6ffd640 1 -- 172.21.15.1:0/4112053120 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdc0003520 con 0x7fcdd80d2710 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd77fe640 1 -- 172.21.15.1:0/4112053120 >> v1:172.21.15.6:6789/0 conn(0x7fcdd81617a0 legacy=0x7fcdd80b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd77fe640 1 -- 172.21.15.1:0/4112053120 >> v1:172.21.15.1:6789/0 conn(0x7fcdd80d2710 legacy=0x7fcdd80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.927+0000 7fcdd77fe640 1 -- 172.21.15.1:0/4112053120 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd80d0b50 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.928+0000 7fcdd77fe640 1 --2- 172.21.15.1:0/4112053120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80b2dd0 0x7fcdd80d21d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0097a0 tx=0x7fcdcc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654a3a624356b678 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:11.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.928+0000 7fcdd6ffd640 1 -- 172.21.15.1:0/4112053120 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc037070 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.928+0000 7fcdd6ffd640 1 -- 172.21.15.1:0/4112053120 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdcc032440 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.928+0000 7fcdd6ffd640 1 -- 172.21.15.1:0/4112053120 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0327f0 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 -- 172.21.15.1:0/4112053120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80b2dd0 msgr2=0x7fcdd80d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/4112053120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80b2dd0 0x7fcdd80d21d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0097a0 tx=0x7fcdcc02a230 comp rx=0 tx=0).stop 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 -- 172.21.15.1:0/4112053120 shutdown_connections 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/4112053120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80b2dd0 0x7fcdd80d21d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 -- 172.21.15.1:0/4112053120 >> 172.21.15.1:0/4112053120 conn(0x7fcdd809d180 msgr2=0x7fcdd80d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 -- 172.21.15.1:0/4112053120 shutdown_connections 2024-05-09T06:34:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.929+0000 7fcdde5a9640 1 -- 172.21.15.1:0/4112053120 wait complete. 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.930+0000 7fcdde5a9640 1 Processor -- start 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.930+0000 7fcdde5a9640 1 -- start start 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdd80b2dd0 0x7fcdd8150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 0x7fcdd81537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd80d1110 con 0x7fcdd81617a0 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd80b5720 con 0x7fcdd80d2710 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdde5a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd8161b80 con 0x7fcdd80b2dd0 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42542/0 (socket says 172.21.15.1:42542) 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 -- 172.21.15.1:0/3335960731 learned_addr learned my addr 172.21.15.1:0/3335960731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdd77fe640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 0x7fcdd81537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdd7fff640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdd80b2dd0 0x7fcdd8150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 -- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdd80b2dd0 msgr2=0x7fcdd8150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdd80b2dd0 0x7fcdd8150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 -- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 msgr2=0x7fcdd81537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 0x7fcdd81537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcddcb46640 1 -- 172.21.15.1:0/3335960731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd8150b60 con 0x7fcdd81617a0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.931+0000 7fcdd77fe640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 0x7fcdd81537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcddcb46640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000cb70 tx=0x7fcdd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0017070 con 0x7fcdd81617a0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd000ad40 con 0x7fcdd81617a0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd80b6fd0 con 0x7fcdd81617a0 2024-05-09T06:34:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0012620 con 0x7fcdd81617a0 2024-05-09T06:34:11.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.932+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd80b7590 con 0x7fcdd81617a0 2024-05-09T06:34:11.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.933+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd80752b0 con 0x7fcdd81617a0 2024-05-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.933+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdd001a070 con 0x7fcdd81617a0 2024-05-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.934+0000 7fcdd4ff9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 0x7fcdc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.934+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fcdd008aee0 con 0x7fcdd81617a0 2024-05-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.934+0000 7fcdd7fff640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 0x7fcdc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.934+0000 7fcdd7fff640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 0x7fcdc406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8001500 tx=0x7fcdc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:11.939+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcdd0059680 con 0x7fcdd81617a0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.119+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdd809d6d0 con 0x7fcdc406be30 2024-05-09T06:34:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.125+0000 7fcdd4ff9640 1 -- 172.21.15.1:0/3335960731 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252075 (secure 0 0 0) 0x7fcdd809d6d0 con 0x7fcdc406be30 2024-05-09T06:34:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:34:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 msgr2=0x7fcdc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 0x7fcdc406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8001500 tx=0x7fcdc8009040 comp rx=0 tx=0).stop 2024-05-09T06:34:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 msgr2=0x7fcdd80cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000cb70 tx=0x7fcdd000a430 comp rx=0 tx=0).stop 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 shutdown_connections 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcdc406be30 0x7fcdc406e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdd81617a0 0x7fcdd80cd050 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdd80d2710 0x7fcdd81537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 --2- 172.21.15.1:0/3335960731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdd80b2dd0 0x7fcdd8150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 >> 172.21.15.1:0/3335960731 conn(0x7fcdd809d180 msgr2=0x7fcdd806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 shutdown_connections 2024-05-09T06:34:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.128+0000 7fcdde5a9640 1 -- 172.21.15.1:0/3335960731 wait complete. 2024-05-09T06:34:12.120 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-05-09T06:34:11.159807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259440,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715280,"statfs":{"total":773094113280,"available":772828446720,"internally_reserved":0,"allocated":10231808,"data_stored":5326244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001858"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.455059+0000","last_change":"2024-05-09T06:34:03.329217+0000","last_active":"2024-05-09T06:34:04.455059+0000","last_peered":"2024-05-09T06:34:04.455059+0000","last_clean":"2024-05-09T06:34:04.455059+0000","last_became_active":"2024-05-09T06:34:03.328941+0000","last_became_peered":"2024-05-09T06:34:03.328941+0000","last_unstale":"2024-05-09T06:34:04.455059+0000","last_undegraded":"2024-05-09T06:34:04.455059+0000","last_fullsized":"2024-05-09T06:34:04.455059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:55:33.897189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337315+0000","last_change":"2024-05-09T06:34:03.335858+0000","last_active":"2024-05-09T06:34:05.337315+0000","last_peered":"2024-05-09T06:34:05.337315+0000","last_clean":"2024-05-09T06:34:05.337315+0000","last_became_active":"2024-05-09T06:34:03.335542+0000","last_became_peered":"2024-05-09T06:34:03.335542+0000","last_unstale":"2024-05-09T06:34:05.337315+0000","last_undegraded":"2024-05-09T06:34:05.337315+0000","last_fullsized":"2024-05-09T06:34:05.337315+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:28.857355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.454952+0000","last_change":"2024-05-09T06:34:03.333303+0000","last_active":"2024-05-09T06:34:04.454952+0000","last_peered":"2024-05-09T06:34:04.454952+0000","last_clean":"2024-05-09T06:34:04.454952+0000","last_became_active":"2024-05-09T06:34:03.333005+0000","last_became_peered":"2024-05-09T06:34:03.333005+0000","last_unstale":"2024-05-09T06:34:04.454952+0000","last_undegraded":"2024-05-09T06:34:04.454952+0000","last_fullsized":"2024-05-09T06:34:04.454952+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:13.862254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150577+0000","last_change":"2024-05-09T06:34:03.345256+0000","last_active":"2024-05-09T06:34:05.150577+0000","last_peered":"2024-05-09T06:34:05.150577+0000","last_clean":"2024-05-09T06:34:05.150577+0000","last_became_active":"2024-05-09T06:34:03.344969+0000","last_became_peered":"2024-05-09T06:34:03.344969+0000","last_unstale":"2024-05-09T06:34:05.150577+0000","last_undegraded":"2024-05-09T06:34:05.150577+0000","last_fullsized":"2024-05-09T06:34:05.150577+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:50:42.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333915+0000","last_change":"2024-05-09T06:34:03.337190+0000","last_active":"2024-05-09T06:34:05.333915+0000","last_peered":"2024-05-09T06:34:05.333915+0000","last_clean":"2024-05-09T06:34:05.333915+0000","last_became_active":"2024-05-09T06:34:03.336861+0000","last_became_peered":"2024-05-09T06:34:03.336861+0000","last_unstale":"2024-05-09T06:34:05.333915+0000","last_undegraded":"2024-05-09T06:34:05.333915+0000","last_fullsized":"2024-05-09T06:34:05.333915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:22.878186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911645+0000","last_change":"2024-05-09T06:34:03.336013+0000","last_active":"2024-05-09T06:34:04.911645+0000","last_peered":"2024-05-09T06:34:04.911645+0000","last_clean":"2024-05-09T06:34:04.911645+0000","last_became_active":"2024-05-09T06:34:03.335772+0000","last_became_peered":"2024-05-09T06:34:03.335772+0000","last_unstale":"2024-05-09T06:34:04.911645+0000","last_undegraded":"2024-05-09T06:34:04.911645+0000","last_fullsized":"2024-05-09T06:34:04.911645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:51:51.232404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012730+0000","last_change":"2024-05-09T06:34:03.330841+0000","last_active":"2024-05-09T06:34:06.012730+0000","last_peered":"2024-05-09T06:34:06.012730+0000","last_clean":"2024-05-09T06:34:06.012730+0000","last_became_active":"2024-05-09T06:34:03.330558+0000","last_became_peered":"2024-05-09T06:34:03.330558+0000","last_unstale":"2024-05-09T06:34:06.012730+0000","last_undegraded":"2024-05-09T06:34:06.012730+0000","last_fullsized":"2024-05-09T06:34:06.012730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:28.857355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150029+0000","last_change":"2024-05-09T06:34:03.341711+0000","last_active":"2024-05-09T06:34:05.150029+0000","last_peered":"2024-05-09T06:34:05.150029+0000","last_clean":"2024-05-09T06:34:05.150029+0000","last_became_active":"2024-05-09T06:34:03.341405+0000","last_became_peered":"2024-05-09T06:34:03.341405+0000","last_unstale":"2024-05-09T06:34:05.150029+0000","last_undegraded":"2024-05-09T06:34:05.150029+0000","last_fullsized":"2024-05-09T06:34:05.150029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:55.509093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334408+0000","last_change":"2024-05-09T06:34:03.338599+0000","last_active":"2024-05-09T06:34:05.334408+0000","last_peered":"2024-05-09T06:34:05.334408+0000","last_clean":"2024-05-09T06:34:05.334408+0000","last_became_active":"2024-05-09T06:34:03.338287+0000","last_became_peered":"2024-05-09T06:34:03.338287+0000","last_unstale":"2024-05-09T06:34:05.334408+0000","last_undegraded":"2024-05-09T06:34:05.334408+0000","last_fullsized":"2024-05-09T06:34:05.334408+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:06.156746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328512+0000","last_change":"2024-05-09T06:34:03.338870+0000","last_active":"2024-05-09T06:34:04.328512+0000","last_peered":"2024-05-09T06:34:04.328512+0000","last_clean":"2024-05-09T06:34:04.328512+0000","last_became_active":"2024-05-09T06:34:03.338596+0000","last_became_peered":"2024-05-09T06:34:03.338596+0000","last_unstale":"2024-05-09T06:34:04.328512+0000","last_undegraded":"2024-05-09T06:34:04.328512+0000","last_fullsized":"2024-05-09T06:34:04.328512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:21.499148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334041+0000","last_change":"2024-05-09T06:34:03.336656+0000","last_active":"2024-05-09T06:34:05.334041+0000","last_peered":"2024-05-09T06:34:05.334041+0000","last_clean":"2024-05-09T06:34:05.334041+0000","last_became_active":"2024-05-09T06:34:03.332718+0000","last_became_peered":"2024-05-09T06:34:03.332718+0000","last_unstale":"2024-05-09T06:34:05.334041+0000","last_undegraded":"2024-05-09T06:34:05.334041+0000","last_fullsized":"2024-05-09T06:34:05.334041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:55.509093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328444+0000","last_change":"2024-05-09T06:34:03.328343+0000","last_active":"2024-05-09T06:34:04.328444+0000","last_peered":"2024-05-09T06:34:04.328444+0000","last_clean":"2024-05-09T06:34:04.328444+0000","last_became_active":"2024-05-09T06:34:03.328060+0000","last_became_peered":"2024-05-09T06:34:03.328060+0000","last_unstale":"2024-05-09T06:34:04.328444+0000","last_undegraded":"2024-05-09T06:34:04.328444+0000","last_fullsized":"2024-05-09T06:34:04.328444+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:28:19.145155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150307+0000","last_change":"2024-05-09T06:34:03.341810+0000","last_active":"2024-05-09T06:34:05.150307+0000","last_peered":"2024-05-09T06:34:05.150307+0000","last_clean":"2024-05-09T06:34:05.150307+0000","last_became_active":"2024-05-09T06:34:03.341484+0000","last_became_peered":"2024-05-09T06:34:03.341484+0000","last_unstale":"2024-05-09T06:34:05.150307+0000","last_undegraded":"2024-05-09T06:34:05.150307+0000","last_fullsized":"2024-05-09T06:34:05.150307+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.256448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333629+0000","last_change":"2024-05-09T06:34:03.338080+0000","last_active":"2024-05-09T06:34:05.333629+0000","last_peered":"2024-05-09T06:34:05.333629+0000","last_clean":"2024-05-09T06:34:05.333629+0000","last_became_active":"2024-05-09T06:34:03.337787+0000","last_became_peered":"2024-05-09T06:34:03.337787+0000","last_unstale":"2024-05-09T06:34:05.333629+0000","last_undegraded":"2024-05-09T06:34:05.333629+0000","last_fullsized":"2024-05-09T06:34:05.333629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:47.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150173+0000","last_change":"2024-05-09T06:34:03.342030+0000","last_active":"2024-05-09T06:34:05.150173+0000","last_peered":"2024-05-09T06:34:05.150173+0000","last_clean":"2024-05-09T06:34:05.150173+0000","last_became_active":"2024-05-09T06:34:03.341714+0000","last_became_peered":"2024-05-09T06:34:03.341714+0000","last_unstale":"2024-05-09T06:34:05.150173+0000","last_undegraded":"2024-05-09T06:34:05.150173+0000","last_fullsized":"2024-05-09T06:34:05.150173+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:33.616196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911517+0000","last_change":"2024-05-09T06:34:03.327551+0000","last_active":"2024-05-09T06:34:04.911517+0000","last_peered":"2024-05-09T06:34:04.911517+0000","last_clean":"2024-05-09T06:34:04.911517+0000","last_became_active":"2024-05-09T06:34:03.327289+0000","last_became_peered":"2024-05-09T06:34:03.327289+0000","last_unstale":"2024-05-09T06:34:04.911517+0000","last_undegraded":"2024-05-09T06:34:04.911517+0000","last_fullsized":"2024-05-09T06:34:04.911517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:41.277383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.912016+0000","last_change":"2024-05-09T06:34:03.328275+0000","last_active":"2024-05-09T06:34:04.912016+0000","last_peered":"2024-05-09T06:34:04.912016+0000","last_clean":"2024-05-09T06:34:04.912016+0000","last_became_active":"2024-05-09T06:34:03.327939+0000","last_became_peered":"2024-05-09T06:34:03.327939+0000","last_unstale":"2024-05-09T06:34:04.912016+0000","last_undegraded":"2024-05-09T06:34:04.912016+0000","last_fullsized":"2024-05-09T06:34:04.912016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:46.186164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.454976+0000","last_change":"2024-05-09T06:34:03.338851+0000","last_active":"2024-05-09T06:34:04.454976+0000","last_peered":"2024-05-09T06:34:04.454976+0000","last_clean":"2024-05-09T06:34:04.454976+0000","last_became_active":"2024-05-09T06:34:03.338360+0000","last_became_peered":"2024-05-09T06:34:03.338360+0000","last_unstale":"2024-05-09T06:34:04.454976+0000","last_undegraded":"2024-05-09T06:34:04.454976+0000","last_fullsized":"2024-05-09T06:34:04.454976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:30.228253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334254+0000","last_change":"2024-05-09T06:34:03.340881+0000","last_active":"2024-05-09T06:34:05.334254+0000","last_peered":"2024-05-09T06:34:05.334254+0000","last_clean":"2024-05-09T06:34:05.334254+0000","last_became_active":"2024-05-09T06:34:03.340047+0000","last_became_peered":"2024-05-09T06:34:03.340047+0000","last_unstale":"2024-05-09T06:34:05.334254+0000","last_undegraded":"2024-05-09T06:34:05.334254+0000","last_fullsized":"2024-05-09T06:34:05.334254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:59.641223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150674+0000","last_change":"2024-05-09T06:34:03.343286+0000","last_active":"2024-05-09T06:34:05.150674+0000","last_peered":"2024-05-09T06:34:05.150674+0000","last_clean":"2024-05-09T06:34:05.150674+0000","last_became_active":"2024-05-09T06:34:03.342942+0000","last_became_peered":"2024-05-09T06:34:03.342942+0000","last_unstale":"2024-05-09T06:34:05.150674+0000","last_undegraded":"2024-05-09T06:34:05.150674+0000","last_fullsized":"2024-05-09T06:34:05.150674+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:49:28.857355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333858+0000","last_change":"2024-05-09T06:34:03.332075+0000","last_active":"2024-05-09T06:34:05.333858+0000","last_peered":"2024-05-09T06:34:05.333858+0000","last_clean":"2024-05-09T06:34:05.333858+0000","last_became_active":"2024-05-09T06:34:03.330923+0000","last_became_peered":"2024-05-09T06:34:03.330923+0000","last_unstale":"2024-05-09T06:34:05.333858+0000","last_undegraded":"2024-05-09T06:34:05.333858+0000","last_fullsized":"2024-05-09T06:34:05.333858+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:43.436586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333667+0000","last_change":"2024-05-09T06:34:03.336041+0000","last_active":"2024-05-09T06:34:05.333667+0000","last_peered":"2024-05-09T06:34:05.333667+0000","last_clean":"2024-05-09T06:34:05.333667+0000","last_became_active":"2024-05-09T06:34:03.335333+0000","last_became_peered":"2024-05-09T06:34:03.335333+0000","last_unstale":"2024-05-09T06:34:05.333667+0000","last_undegraded":"2024-05-09T06:34:05.333667+0000","last_fullsized":"2024-05-09T06:34:05.333667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:57.435105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329706+0000","last_change":"2024-05-09T06:34:03.345585+0000","last_active":"2024-05-09T06:34:04.329706+0000","last_peered":"2024-05-09T06:34:04.329706+0000","last_clean":"2024-05-09T06:34:04.329706+0000","last_became_active":"2024-05-09T06:34:03.345225+0000","last_became_peered":"2024-05-09T06:34:03.345225+0000","last_unstale":"2024-05-09T06:34:04.329706+0000","last_undegraded":"2024-05-09T06:34:04.329706+0000","last_fullsized":"2024-05-09T06:34:04.329706+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:59.641223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.454748+0000","last_change":"2024-05-09T06:34:03.335481+0000","last_active":"2024-05-09T06:34:04.454748+0000","last_peered":"2024-05-09T06:34:04.454748+0000","last_clean":"2024-05-09T06:34:04.454748+0000","last_became_active":"2024-05-09T06:34:03.335203+0000","last_became_peered":"2024-05-09T06:34:03.335203+0000","last_unstale":"2024-05-09T06:34:04.454748+0000","last_undegraded":"2024-05-09T06:34:04.454748+0000","last_fullsized":"2024-05-09T06:34:04.454748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:58.920472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337243+0000","last_change":"2024-05-09T06:34:03.335778+0000","last_active":"2024-05-09T06:34:05.337243+0000","last_peered":"2024-05-09T06:34:05.337243+0000","last_clean":"2024-05-09T06:34:05.337243+0000","last_became_active":"2024-05-09T06:34:03.335419+0000","last_became_peered":"2024-05-09T06:34:03.335419+0000","last_unstale":"2024-05-09T06:34:05.337243+0000","last_undegraded":"2024-05-09T06:34:05.337243+0000","last_fullsized":"2024-05-09T06:34:05.337243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:23.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334264+0000","last_change":"2024-05-09T06:34:03.337376+0000","last_active":"2024-05-09T06:34:05.334264+0000","last_peered":"2024-05-09T06:34:05.334264+0000","last_clean":"2024-05-09T06:34:05.334264+0000","last_became_active":"2024-05-09T06:34:03.337085+0000","last_became_peered":"2024-05-09T06:34:03.337085+0000","last_unstale":"2024-05-09T06:34:05.334264+0000","last_undegraded":"2024-05-09T06:34:05.334264+0000","last_fullsized":"2024-05-09T06:34:05.334264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:45.149809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328313+0000","last_change":"2024-05-09T06:34:03.335298+0000","last_active":"2024-05-09T06:34:04.328313+0000","last_peered":"2024-05-09T06:34:04.328313+0000","last_clean":"2024-05-09T06:34:04.328313+0000","last_became_active":"2024-05-09T06:34:03.335016+0000","last_became_peered":"2024-05-09T06:34:03.335016+0000","last_unstale":"2024-05-09T06:34:04.328313+0000","last_undegraded":"2024-05-09T06:34:04.328313+0000","last_fullsized":"2024-05-09T06:34:04.328313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:31:46.211246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150791+0000","last_change":"2024-05-09T06:34:03.349366+0000","last_active":"2024-05-09T06:34:05.150791+0000","last_peered":"2024-05-09T06:34:05.150791+0000","last_clean":"2024-05-09T06:34:05.150791+0000","last_became_active":"2024-05-09T06:34:03.349081+0000","last_became_peered":"2024-05-09T06:34:03.349081+0000","last_unstale":"2024-05-09T06:34:05.150791+0000","last_undegraded":"2024-05-09T06:34:05.150791+0000","last_fullsized":"2024-05-09T06:34:05.150791+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:26.506076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012868+0000","last_change":"2024-05-09T06:34:03.333788+0000","last_active":"2024-05-09T06:34:06.012868+0000","last_peered":"2024-05-09T06:34:06.012868+0000","last_clean":"2024-05-09T06:34:06.012868+0000","last_became_active":"2024-05-09T06:34:03.333503+0000","last_became_peered":"2024-05-09T06:34:03.333503+0000","last_unstale":"2024-05-09T06:34:06.012868+0000","last_undegraded":"2024-05-09T06:34:06.012868+0000","last_fullsized":"2024-05-09T06:34:06.012868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:26.506076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333963+0000","last_change":"2024-05-09T06:34:03.337894+0000","last_active":"2024-05-09T06:34:05.333963+0000","last_peered":"2024-05-09T06:34:05.333963+0000","last_clean":"2024-05-09T06:34:05.333963+0000","last_became_active":"2024-05-09T06:34:03.337610+0000","last_became_peered":"2024-05-09T06:34:03.337610+0000","last_unstale":"2024-05-09T06:34:05.333963+0000","last_undegraded":"2024-05-09T06:34:05.333963+0000","last_fullsized":"2024-05-09T06:34:05.333963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:33.616196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337080+0000","last_change":"2024-05-09T06:34:03.332237+0000","last_active":"2024-05-09T06:34:05.337080+0000","last_peered":"2024-05-09T06:34:05.337080+0000","last_clean":"2024-05-09T06:34:05.337080+0000","last_became_active":"2024-05-09T06:34:03.331945+0000","last_became_peered":"2024-05-09T06:34:03.331945+0000","last_unstale":"2024-05-09T06:34:05.337080+0000","last_undegraded":"2024-05-09T06:34:05.337080+0000","last_fullsized":"2024-05-09T06:34:05.337080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:06.156746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911590+0000","last_change":"2024-05-09T06:34:03.326906+0000","last_active":"2024-05-09T06:34:04.911590+0000","last_peered":"2024-05-09T06:34:04.911590+0000","last_clean":"2024-05-09T06:34:04.911590+0000","last_became_active":"2024-05-09T06:34:03.326639+0000","last_became_peered":"2024-05-09T06:34:03.326639+0000","last_unstale":"2024-05-09T06:34:04.911590+0000","last_undegraded":"2024-05-09T06:34:04.911590+0000","last_fullsized":"2024-05-09T06:34:04.911590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:47.681594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911751+0000","last_change":"2024-05-09T06:34:03.337520+0000","last_active":"2024-05-09T06:34:04.911751+0000","last_peered":"2024-05-09T06:34:04.911751+0000","last_clean":"2024-05-09T06:34:04.911751+0000","last_became_active":"2024-05-09T06:34:03.337199+0000","last_became_peered":"2024-05-09T06:34:03.337199+0000","last_unstale":"2024-05-09T06:34:04.911751+0000","last_undegraded":"2024-05-09T06:34:04.911751+0000","last_fullsized":"2024-05-09T06:34:04.911751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:55.087723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.330102+0000","last_change":"2024-05-09T06:34:02.315932+0000","last_active":"2024-05-09T06:34:04.330102+0000","last_peered":"2024-05-09T06:34:04.330102+0000","last_clean":"2024-05-09T06:34:04.330102+0000","last_became_active":"2024-05-09T06:34:02.315556+0000","last_became_peered":"2024-05-09T06:34:02.315556+0000","last_unstale":"2024-05-09T06:34:04.330102+0000","last_undegraded":"2024-05-09T06:34:04.330102+0000","last_fullsized":"2024-05-09T06:34:04.330102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911832+0000","last_change":"2024-05-09T06:34:03.327664+0000","last_active":"2024-05-09T06:34:04.911832+0000","last_peered":"2024-05-09T06:34:04.911832+0000","last_clean":"2024-05-09T06:34:04.911832+0000","last_became_active":"2024-05-09T06:34:03.327408+0000","last_became_peered":"2024-05-09T06:34:03.327408+0000","last_unstale":"2024-05-09T06:34:04.911832+0000","last_undegraded":"2024-05-09T06:34:04.911832+0000","last_fullsized":"2024-05-09T06:34:04.911832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:45.744781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334342+0000","last_change":"2024-05-09T06:34:02.317231+0000","last_active":"2024-05-09T06:34:05.334342+0000","last_peered":"2024-05-09T06:34:05.334342+0000","last_clean":"2024-05-09T06:34:05.334342+0000","last_became_active":"2024-05-09T06:34:02.316853+0000","last_became_peered":"2024-05-09T06:34:02.316853+0000","last_unstale":"2024-05-09T06:34:05.334342+0000","last_undegraded":"2024-05-09T06:34:05.334342+0000","last_fullsized":"2024-05-09T06:34:05.334342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:39.290796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012951+0000","last_change":"2024-05-09T06:34:03.341031+0000","last_active":"2024-05-09T06:34:06.012951+0000","last_peered":"2024-05-09T06:34:06.012951+0000","last_clean":"2024-05-09T06:34:06.012951+0000","last_became_active":"2024-05-09T06:34:03.340728+0000","last_became_peered":"2024-05-09T06:34:03.340728+0000","last_unstale":"2024-05-09T06:34:06.012951+0000","last_undegraded":"2024-05-09T06:34:06.012951+0000","last_fullsized":"2024-05-09T06:34:06.012951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:50.595775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.013011+0000","last_change":"2024-05-09T06:34:02.322212+0000","last_active":"2024-05-09T06:34:06.013011+0000","last_peered":"2024-05-09T06:34:06.013011+0000","last_clean":"2024-05-09T06:34:06.013011+0000","last_became_active":"2024-05-09T06:34:02.321940+0000","last_became_peered":"2024-05-09T06:34:02.321940+0000","last_unstale":"2024-05-09T06:34:06.013011+0000","last_undegraded":"2024-05-09T06:34:06.013011+0000","last_fullsized":"2024-05-09T06:34:06.013011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:54:57.046208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.912126+0000","last_change":"2024-05-09T06:34:03.334515+0000","last_active":"2024-05-09T06:34:04.912126+0000","last_peered":"2024-05-09T06:34:04.912126+0000","last_clean":"2024-05-09T06:34:04.912126+0000","last_became_active":"2024-05-09T06:34:03.334200+0000","last_became_peered":"2024-05-09T06:34:03.334200+0000","last_unstale":"2024-05-09T06:34:04.912126+0000","last_undegraded":"2024-05-09T06:34:04.912126+0000","last_fullsized":"2024-05-09T06:34:04.912126+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:41.545159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.330036+0000","last_change":"2024-05-09T06:34:02.328490+0000","last_active":"2024-05-09T06:34:04.330036+0000","last_peered":"2024-05-09T06:34:04.330036+0000","last_clean":"2024-05-09T06:34:04.330036+0000","last_became_active":"2024-05-09T06:34:02.328118+0000","last_became_peered":"2024-05-09T06:34:02.328118+0000","last_unstale":"2024-05-09T06:34:04.330036+0000","last_undegraded":"2024-05-09T06:34:04.330036+0000","last_fullsized":"2024-05-09T06:34:04.330036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:51.881961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150094+0000","last_change":"2024-05-09T06:34:03.346506+0000","last_active":"2024-05-09T06:34:05.150094+0000","last_peered":"2024-05-09T06:34:05.150094+0000","last_clean":"2024-05-09T06:34:05.150094+0000","last_became_active":"2024-05-09T06:34:03.345985+0000","last_became_peered":"2024-05-09T06:34:03.345985+0000","last_unstale":"2024-05-09T06:34:05.150094+0000","last_undegraded":"2024-05-09T06:34:05.150094+0000","last_fullsized":"2024-05-09T06:34:05.150094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:45.149809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334039+0000","last_change":"2024-05-09T06:34:02.320157+0000","last_active":"2024-05-09T06:34:05.334039+0000","last_peered":"2024-05-09T06:34:05.334039+0000","last_clean":"2024-05-09T06:34:05.334039+0000","last_became_active":"2024-05-09T06:34:02.319814+0000","last_became_peered":"2024-05-09T06:34:02.319814+0000","last_unstale":"2024-05-09T06:34:05.334039+0000","last_undegraded":"2024-05-09T06:34:05.334039+0000","last_fullsized":"2024-05-09T06:34:05.334039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:44.543374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329719+0000","last_change":"2024-05-09T06:34:03.332819+0000","last_active":"2024-05-09T06:34:04.329719+0000","last_peered":"2024-05-09T06:34:04.329719+0000","last_clean":"2024-05-09T06:34:04.329719+0000","last_became_active":"2024-05-09T06:34:03.332104+0000","last_became_peered":"2024-05-09T06:34:03.332104+0000","last_unstale":"2024-05-09T06:34:04.329719+0000","last_undegraded":"2024-05-09T06:34:04.329719+0000","last_fullsized":"2024-05-09T06:34:04.329719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:45.149809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150728+0000","last_change":"2024-05-09T06:34:02.329390+0000","last_active":"2024-05-09T06:34:05.150728+0000","last_peered":"2024-05-09T06:34:05.150728+0000","last_clean":"2024-05-09T06:34:05.150728+0000","last_became_active":"2024-05-09T06:34:02.329108+0000","last_became_peered":"2024-05-09T06:34:02.329108+0000","last_unstale":"2024-05-09T06:34:05.150728+0000","last_undegraded":"2024-05-09T06:34:05.150728+0000","last_fullsized":"2024-05-09T06:34:05.150728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:12.987046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911627+0000","last_change":"2024-05-09T06:34:03.335411+0000","last_active":"2024-05-09T06:34:04.911627+0000","last_peered":"2024-05-09T06:34:04.911627+0000","last_clean":"2024-05-09T06:34:04.911627+0000","last_became_active":"2024-05-09T06:34:03.335143+0000","last_became_peered":"2024-05-09T06:34:03.335143+0000","last_unstale":"2024-05-09T06:34:04.911627+0000","last_undegraded":"2024-05-09T06:34:04.911627+0000","last_fullsized":"2024-05-09T06:34:04.911627+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:42.052029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.454845+0000","last_change":"2024-05-09T06:34:02.324159+0000","last_active":"2024-05-09T06:34:04.454845+0000","last_peered":"2024-05-09T06:34:04.454845+0000","last_clean":"2024-05-09T06:34:04.454845+0000","last_became_active":"2024-05-09T06:34:02.323875+0000","last_became_peered":"2024-05-09T06:34:02.323875+0000","last_unstale":"2024-05-09T06:34:04.454845+0000","last_undegraded":"2024-05-09T06:34:04.454845+0000","last_fullsized":"2024-05-09T06:34:04.454845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:25.837322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150346+0000","last_change":"2024-05-09T06:34:03.344503+0000","last_active":"2024-05-09T06:34:05.150346+0000","last_peered":"2024-05-09T06:34:05.150346+0000","last_clean":"2024-05-09T06:34:05.150346+0000","last_became_active":"2024-05-09T06:34:03.344171+0000","last_became_peered":"2024-05-09T06:34:03.344171+0000","last_unstale":"2024-05-09T06:34:05.150346+0000","last_undegraded":"2024-05-09T06:34:05.150346+0000","last_fullsized":"2024-05-09T06:34:05.150346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:06.156746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012664+0000","last_change":"2024-05-09T06:34:02.324681+0000","last_active":"2024-05-09T06:34:06.012664+0000","last_peered":"2024-05-09T06:34:06.012664+0000","last_clean":"2024-05-09T06:34:06.012664+0000","last_became_active":"2024-05-09T06:34:02.324383+0000","last_became_peered":"2024-05-09T06:34:02.324383+0000","last_unstale":"2024-05-09T06:34:06.012664+0000","last_undegraded":"2024-05-09T06:34:06.012664+0000","last_fullsized":"2024-05-09T06:34:06.012664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:42.429949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334005+0000","last_change":"2024-05-09T06:34:03.332610+0000","last_active":"2024-05-09T06:34:05.334005+0000","last_peered":"2024-05-09T06:34:05.334005+0000","last_clean":"2024-05-09T06:34:05.334005+0000","last_became_active":"2024-05-09T06:34:03.332293+0000","last_became_peered":"2024-05-09T06:34:03.332293+0000","last_unstale":"2024-05-09T06:34:05.334005+0000","last_undegraded":"2024-05-09T06:34:05.334005+0000","last_fullsized":"2024-05-09T06:34:05.334005+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:54:58.052845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337380+0000","last_change":"2024-05-09T06:34:02.322716+0000","last_active":"2024-05-09T06:34:05.337380+0000","last_peered":"2024-05-09T06:34:05.337380+0000","last_clean":"2024-05-09T06:34:05.337380+0000","last_became_active":"2024-05-09T06:34:02.322190+0000","last_became_peered":"2024-05-09T06:34:02.322190+0000","last_unstale":"2024-05-09T06:34:05.337380+0000","last_undegraded":"2024-05-09T06:34:05.337380+0000","last_fullsized":"2024-05-09T06:34:05.337380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.013101+0000","last_change":"2024-05-09T06:34:03.343470+0000","last_active":"2024-05-09T06:34:06.013101+0000","last_peered":"2024-05-09T06:34:06.013101+0000","last_clean":"2024-05-09T06:34:06.013101+0000","last_became_active":"2024-05-09T06:34:03.343152+0000","last_became_peered":"2024-05-09T06:34:03.343152+0000","last_unstale":"2024-05-09T06:34:06.013101+0000","last_undegraded":"2024-05-09T06:34:06.013101+0000","last_fullsized":"2024-05-09T06:34:06.013101+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:54.032165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.455075+0000","last_change":"2024-05-09T06:34:02.328152+0000","last_active":"2024-05-09T06:34:04.455075+0000","last_peered":"2024-05-09T06:34:04.455075+0000","last_clean":"2024-05-09T06:34:04.455075+0000","last_became_active":"2024-05-09T06:34:02.327851+0000","last_became_peered":"2024-05-09T06:34:02.327851+0000","last_unstale":"2024-05-09T06:34:04.455075+0000","last_undegraded":"2024-05-09T06:34:04.455075+0000","last_fullsized":"2024-05-09T06:34:04.455075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:53.025528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.455018+0000","last_change":"2024-05-09T06:34:03.329569+0000","last_active":"2024-05-09T06:34:04.455018+0000","last_peered":"2024-05-09T06:34:04.455018+0000","last_clean":"2024-05-09T06:34:04.455018+0000","last_became_active":"2024-05-09T06:34:03.329302+0000","last_became_peered":"2024-05-09T06:34:03.329302+0000","last_unstale":"2024-05-09T06:34:04.455018+0000","last_undegraded":"2024-05-09T06:34:04.455018+0000","last_fullsized":"2024-05-09T06:34:04.455018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:38:04.390825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334487+0000","last_change":"2024-05-09T06:34:02.318316+0000","last_active":"2024-05-09T06:34:05.334487+0000","last_peered":"2024-05-09T06:34:05.334487+0000","last_clean":"2024-05-09T06:34:05.334487+0000","last_became_active":"2024-05-09T06:34:02.318030+0000","last_became_peered":"2024-05-09T06:34:02.318030+0000","last_unstale":"2024-05-09T06:34:05.334487+0000","last_undegraded":"2024-05-09T06:34:05.334487+0000","last_fullsized":"2024-05-09T06:34:05.334487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:18:23.995419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150639+0000","last_change":"2024-05-09T06:34:03.340898+0000","last_active":"2024-05-09T06:34:05.150639+0000","last_peered":"2024-05-09T06:34:05.150639+0000","last_clean":"2024-05-09T06:34:05.150639+0000","last_became_active":"2024-05-09T06:34:03.340370+0000","last_became_peered":"2024-05-09T06:34:03.340370+0000","last_unstale":"2024-05-09T06:34:05.150639+0000","last_undegraded":"2024-05-09T06:34:05.150639+0000","last_fullsized":"2024-05-09T06:34:05.150639+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:23.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911958+0000","last_change":"2024-05-09T06:34:02.318480+0000","last_active":"2024-05-09T06:34:04.911958+0000","last_peered":"2024-05-09T06:34:04.911958+0000","last_clean":"2024-05-09T06:34:04.911958+0000","last_became_active":"2024-05-09T06:34:02.318175+0000","last_became_peered":"2024-05-09T06:34:02.318175+0000","last_unstale":"2024-05-09T06:34:04.911958+0000","last_undegraded":"2024-05-09T06:34:04.911958+0000","last_fullsized":"2024-05-09T06:34:04.911958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:56.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150286+0000","last_change":"2024-05-09T06:34:03.342387+0000","last_active":"2024-05-09T06:34:05.150286+0000","last_peered":"2024-05-09T06:34:05.150286+0000","last_clean":"2024-05-09T06:34:05.150286+0000","last_became_active":"2024-05-09T06:34:03.341150+0000","last_became_peered":"2024-05-09T06:34:03.341150+0000","last_unstale":"2024-05-09T06:34:05.150286+0000","last_undegraded":"2024-05-09T06:34:05.150286+0000","last_fullsized":"2024-05-09T06:34:05.150286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:59.641223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012829+0000","last_change":"2024-05-09T06:34:02.325683+0000","last_active":"2024-05-09T06:34:06.012829+0000","last_peered":"2024-05-09T06:34:06.012829+0000","last_clean":"2024-05-09T06:34:06.012829+0000","last_became_active":"2024-05-09T06:34:02.325346+0000","last_became_peered":"2024-05-09T06:34:02.325346+0000","last_unstale":"2024-05-09T06:34:06.012829+0000","last_undegraded":"2024-05-09T06:34:06.012829+0000","last_fullsized":"2024-05-09T06:34:06.012829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:46.670097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.333628+0000","last_change":"2024-05-09T06:34:03.334300+0000","last_active":"2024-05-09T06:34:05.333628+0000","last_peered":"2024-05-09T06:34:05.333628+0000","last_clean":"2024-05-09T06:34:05.333628+0000","last_became_active":"2024-05-09T06:34:03.334032+0000","last_became_peered":"2024-05-09T06:34:03.334032+0000","last_unstale":"2024-05-09T06:34:05.333628+0000","last_undegraded":"2024-05-09T06:34:05.333628+0000","last_fullsized":"2024-05-09T06:34:05.333628+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:22.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911744+0000","last_change":"2024-05-09T06:34:02.318607+0000","last_active":"2024-05-09T06:34:04.911744+0000","last_peered":"2024-05-09T06:34:04.911744+0000","last_clean":"2024-05-09T06:34:04.911744+0000","last_became_active":"2024-05-09T06:34:02.318297+0000","last_became_peered":"2024-05-09T06:34:02.318297+0000","last_unstale":"2024-05-09T06:34:04.911744+0000","last_undegraded":"2024-05-09T06:34:04.911744+0000","last_fullsized":"2024-05-09T06:34:04.911744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:40.667656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328186+0000","last_change":"2024-05-09T06:34:03.343365+0000","last_active":"2024-05-09T06:34:04.328186+0000","last_peered":"2024-05-09T06:34:04.328186+0000","last_clean":"2024-05-09T06:34:04.328186+0000","last_became_active":"2024-05-09T06:34:03.342875+0000","last_became_peered":"2024-05-09T06:34:03.342875+0000","last_unstale":"2024-05-09T06:34:04.328186+0000","last_undegraded":"2024-05-09T06:34:04.328186+0000","last_fullsized":"2024-05-09T06:34:04.328186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:37.587422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150252+0000","last_change":"2024-05-09T06:34:02.326169+0000","last_active":"2024-05-09T06:34:05.150252+0000","last_peered":"2024-05-09T06:34:05.150252+0000","last_clean":"2024-05-09T06:34:05.150252+0000","last_became_active":"2024-05-09T06:34:02.325706+0000","last_became_peered":"2024-05-09T06:34:02.325706+0000","last_unstale":"2024-05-09T06:34:05.150252+0000","last_undegraded":"2024-05-09T06:34:05.150252+0000","last_fullsized":"2024-05-09T06:34:05.150252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:29.480353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328341+0000","last_change":"2024-05-09T06:34:03.337365+0000","last_active":"2024-05-09T06:34:04.328341+0000","last_peered":"2024-05-09T06:34:04.328341+0000","last_clean":"2024-05-09T06:34:04.328341+0000","last_became_active":"2024-05-09T06:34:03.337084+0000","last_became_peered":"2024-05-09T06:34:03.337084+0000","last_unstale":"2024-05-09T06:34:04.328341+0000","last_undegraded":"2024-05-09T06:34:04.328341+0000","last_fullsized":"2024-05-09T06:34:04.328341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:41.954611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150466+0000","last_change":"2024-05-09T06:34:02.330110+0000","last_active":"2024-05-09T06:34:05.150466+0000","last_peered":"2024-05-09T06:34:05.150466+0000","last_clean":"2024-05-09T06:34:05.150466+0000","last_became_active":"2024-05-09T06:34:02.329847+0000","last_became_peered":"2024-05-09T06:34:02.329847+0000","last_unstale":"2024-05-09T06:34:05.150466+0000","last_undegraded":"2024-05-09T06:34:05.150466+0000","last_fullsized":"2024-05-09T06:34:05.150466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:15.178716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911928+0000","last_change":"2024-05-09T06:34:03.334670+0000","last_active":"2024-05-09T06:34:04.911928+0000","last_peered":"2024-05-09T06:34:04.911928+0000","last_clean":"2024-05-09T06:34:04.911928+0000","last_became_active":"2024-05-09T06:34:03.334396+0000","last_became_peered":"2024-05-09T06:34:03.334396+0000","last_unstale":"2024-05-09T06:34:04.911928+0000","last_undegraded":"2024-05-09T06:34:04.911928+0000","last_fullsized":"2024-05-09T06:34:04.911928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:49.814421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.330228+0000","last_change":"2024-05-09T06:34:02.322056+0000","last_active":"2024-05-09T06:34:04.330228+0000","last_peered":"2024-05-09T06:34:04.330228+0000","last_clean":"2024-05-09T06:34:04.330228+0000","last_became_active":"2024-05-09T06:34:02.321765+0000","last_became_peered":"2024-05-09T06:34:02.321765+0000","last_unstale":"2024-05-09T06:34:04.330228+0000","last_undegraded":"2024-05-09T06:34:04.330228+0000","last_fullsized":"2024-05-09T06:34:04.330228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:56.428468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329918+0000","last_change":"2024-05-09T06:34:03.336058+0000","last_active":"2024-05-09T06:34:04.329918+0000","last_peered":"2024-05-09T06:34:04.329918+0000","last_clean":"2024-05-09T06:34:04.329918+0000","last_became_active":"2024-05-09T06:34:03.335750+0000","last_became_peered":"2024-05-09T06:34:03.335750+0000","last_unstale":"2024-05-09T06:34:04.329918+0000","last_undegraded":"2024-05-09T06:34:04.329918+0000","last_fullsized":"2024-05-09T06:34:04.329918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:50:42.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012919+0000","last_change":"2024-05-09T06:34:02.327732+0000","last_active":"2024-05-09T06:34:06.012919+0000","last_peered":"2024-05-09T06:34:06.012919+0000","last_clean":"2024-05-09T06:34:06.012919+0000","last_became_active":"2024-05-09T06:34:02.327311+0000","last_became_peered":"2024-05-09T06:34:02.327311+0000","last_unstale":"2024-05-09T06:34:06.012919+0000","last_undegraded":"2024-05-09T06:34:06.012919+0000","last_fullsized":"2024-05-09T06:34:06.012919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:32.609559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334233+0000","last_change":"2024-05-09T06:34:03.333355+0000","last_active":"2024-05-09T06:34:05.334233+0000","last_peered":"2024-05-09T06:34:05.334233+0000","last_clean":"2024-05-09T06:34:05.334233+0000","last_became_active":"2024-05-09T06:34:03.333043+0000","last_became_peered":"2024-05-09T06:34:03.333043+0000","last_unstale":"2024-05-09T06:34:05.334233+0000","last_undegraded":"2024-05-09T06:34:05.334233+0000","last_fullsized":"2024-05-09T06:34:05.334233+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.256448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334534+0000","last_change":"2024-05-09T06:34:02.329189+0000","last_active":"2024-05-09T06:34:05.334534+0000","last_peered":"2024-05-09T06:34:05.334534+0000","last_clean":"2024-05-09T06:34:05.334534+0000","last_became_active":"2024-05-09T06:34:02.328869+0000","last_became_peered":"2024-05-09T06:34:02.328869+0000","last_unstale":"2024-05-09T06:34:05.334534+0000","last_undegraded":"2024-05-09T06:34:05.334534+0000","last_fullsized":"2024-05-09T06:34:05.334534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:28.744749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911758+0000","last_change":"2024-05-09T06:34:03.334654+0000","last_active":"2024-05-09T06:34:04.911758+0000","last_peered":"2024-05-09T06:34:04.911758+0000","last_clean":"2024-05-09T06:34:04.911758+0000","last_became_active":"2024-05-09T06:34:03.334375+0000","last_became_peered":"2024-05-09T06:34:03.334375+0000","last_unstale":"2024-05-09T06:34:04.911758+0000","last_undegraded":"2024-05-09T06:34:04.911758+0000","last_fullsized":"2024-05-09T06:34:04.911758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:25.609771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.336829+0000","last_change":"2024-05-09T06:34:02.327306+0000","last_active":"2024-05-09T06:34:05.336829+0000","last_peered":"2024-05-09T06:34:05.336829+0000","last_clean":"2024-05-09T06:34:05.336829+0000","last_became_active":"2024-05-09T06:34:02.326990+0000","last_became_peered":"2024-05-09T06:34:02.326990+0000","last_unstale":"2024-05-09T06:34:05.336829+0000","last_undegraded":"2024-05-09T06:34:05.336829+0000","last_fullsized":"2024-05-09T06:34:05.336829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:42.429949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.336929+0000","last_change":"2024-05-09T06:34:03.328711+0000","last_active":"2024-05-09T06:34:05.336929+0000","last_peered":"2024-05-09T06:34:05.336929+0000","last_clean":"2024-05-09T06:34:05.336929+0000","last_became_active":"2024-05-09T06:34:03.328175+0000","last_became_peered":"2024-05-09T06:34:03.328175+0000","last_unstale":"2024-05-09T06:34:05.336929+0000","last_undegraded":"2024-05-09T06:34:05.336929+0000","last_fullsized":"2024-05-09T06:34:05.336929+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:59.641223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328212+0000","last_change":"2024-05-09T06:34:02.328106+0000","last_active":"2024-05-09T06:34:04.328212+0000","last_peered":"2024-05-09T06:34:04.328212+0000","last_clean":"2024-05-09T06:34:04.328212+0000","last_became_active":"2024-05-09T06:34:02.327820+0000","last_became_peered":"2024-05-09T06:34:02.327820+0000","last_unstale":"2024-05-09T06:34:04.328212+0000","last_undegraded":"2024-05-09T06:34:04.328212+0000","last_fullsized":"2024-05-09T06:34:04.328212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:00.882173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337002+0000","last_change":"2024-05-09T06:34:03.342185+0000","last_active":"2024-05-09T06:34:05.337002+0000","last_peered":"2024-05-09T06:34:05.337002+0000","last_clean":"2024-05-09T06:34:05.337002+0000","last_became_active":"2024-05-09T06:34:03.341909+0000","last_became_peered":"2024-05-09T06:34:03.341909+0000","last_unstale":"2024-05-09T06:34:05.337002+0000","last_undegraded":"2024-05-09T06:34:05.337002+0000","last_fullsized":"2024-05-09T06:34:05.337002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:45.149809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337053+0000","last_change":"2024-05-09T06:34:02.327204+0000","last_active":"2024-05-09T06:34:05.337053+0000","last_peered":"2024-05-09T06:34:05.337053+0000","last_clean":"2024-05-09T06:34:05.337053+0000","last_became_active":"2024-05-09T06:34:02.326876+0000","last_became_peered":"2024-05-09T06:34:02.326876+0000","last_unstale":"2024-05-09T06:34:05.337053+0000","last_undegraded":"2024-05-09T06:34:05.337053+0000","last_fullsized":"2024-05-09T06:34:05.337053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911668+0000","last_change":"2024-05-09T06:34:03.326408+0000","last_active":"2024-05-09T06:34:04.911668+0000","last_peered":"2024-05-09T06:34:04.911668+0000","last_clean":"2024-05-09T06:34:04.911668+0000","last_became_active":"2024-05-09T06:34:03.326140+0000","last_became_peered":"2024-05-09T06:34:03.326140+0000","last_unstale":"2024-05-09T06:34:04.911668+0000","last_undegraded":"2024-05-09T06:34:04.911668+0000","last_fullsized":"2024-05-09T06:34:04.911668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:45.021390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328090+0000","last_change":"2024-05-09T06:34:02.327053+0000","last_active":"2024-05-09T06:34:04.328090+0000","last_peered":"2024-05-09T06:34:04.328090+0000","last_clean":"2024-05-09T06:34:04.328090+0000","last_became_active":"2024-05-09T06:34:02.326751+0000","last_became_peered":"2024-05-09T06:34:02.326751+0000","last_unstale":"2024-05-09T06:34:04.328090+0000","last_undegraded":"2024-05-09T06:34:04.328090+0000","last_fullsized":"2024-05-09T06:34:04.328090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:56.992622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329810+0000","last_change":"2024-05-09T06:34:03.334873+0000","last_active":"2024-05-09T06:34:04.329810+0000","last_peered":"2024-05-09T06:34:04.329810+0000","last_clean":"2024-05-09T06:34:04.329810+0000","last_became_active":"2024-05-09T06:34:03.334418+0000","last_became_peered":"2024-05-09T06:34:03.334418+0000","last_unstale":"2024-05-09T06:34:04.329810+0000","last_undegraded":"2024-05-09T06:34:04.329810+0000","last_fullsized":"2024-05-09T06:34:04.329810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:06.156746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.013016+0000","last_change":"2024-05-09T06:34:02.326680+0000","last_active":"2024-05-09T06:34:06.013016+0000","last_peered":"2024-05-09T06:34:06.013016+0000","last_clean":"2024-05-09T06:34:06.013016+0000","last_became_active":"2024-05-09T06:34:02.326353+0000","last_became_peered":"2024-05-09T06:34:02.326353+0000","last_unstale":"2024-05-09T06:34:06.013016+0000","last_undegraded":"2024-05-09T06:34:06.013016+0000","last_fullsized":"2024-05-09T06:34:06.013016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:54.502456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911928+0000","last_change":"2024-05-09T06:34:03.335586+0000","last_active":"2024-05-09T06:34:04.911928+0000","last_peered":"2024-05-09T06:34:04.911928+0000","last_clean":"2024-05-09T06:34:04.911928+0000","last_became_active":"2024-05-09T06:34:03.335275+0000","last_became_peered":"2024-05-09T06:34:03.335275+0000","last_unstale":"2024-05-09T06:34:04.911928+0000","last_undegraded":"2024-05-09T06:34:04.911928+0000","last_fullsized":"2024-05-09T06:34:04.911928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:32:34.039109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334432+0000","last_change":"2024-05-09T06:34:02.327790+0000","last_active":"2024-05-09T06:34:05.334432+0000","last_peered":"2024-05-09T06:34:05.334432+0000","last_clean":"2024-05-09T06:34:05.334432+0000","last_became_active":"2024-05-09T06:34:02.327409+0000","last_became_peered":"2024-05-09T06:34:02.327409+0000","last_unstale":"2024-05-09T06:34:05.334432+0000","last_undegraded":"2024-05-09T06:34:05.334432+0000","last_fullsized":"2024-05-09T06:34:05.334432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:44.911079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911869+0000","last_change":"2024-05-09T06:34:03.327975+0000","last_active":"2024-05-09T06:34:04.911869+0000","last_peered":"2024-05-09T06:34:04.911869+0000","last_clean":"2024-05-09T06:34:04.911869+0000","last_became_active":"2024-05-09T06:34:03.327683+0000","last_became_peered":"2024-05-09T06:34:03.327683+0000","last_unstale":"2024-05-09T06:34:04.911869+0000","last_undegraded":"2024-05-09T06:34:04.911869+0000","last_fullsized":"2024-05-09T06:34:04.911869+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:21.643651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.334600+0000","last_change":"2024-05-09T06:34:02.314084+0000","last_active":"2024-05-09T06:34:05.334600+0000","last_peered":"2024-05-09T06:34:05.334600+0000","last_clean":"2024-05-09T06:34:05.334600+0000","last_became_active":"2024-05-09T06:34:02.313093+0000","last_became_peered":"2024-05-09T06:34:02.313093+0000","last_unstale":"2024-05-09T06:34:05.334600+0000","last_undegraded":"2024-05-09T06:34:05.334600+0000","last_fullsized":"2024-05-09T06:34:05.334600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:59.281394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337269+0000","last_change":"2024-05-09T06:34:03.338212+0000","last_active":"2024-05-09T06:34:05.337269+0000","last_peered":"2024-05-09T06:34:05.337269+0000","last_clean":"2024-05-09T06:34:05.337269+0000","last_became_active":"2024-05-09T06:34:03.337879+0000","last_became_peered":"2024-05-09T06:34:03.337879+0000","last_unstale":"2024-05-09T06:34:05.337269+0000","last_undegraded":"2024-05-09T06:34:05.337269+0000","last_fullsized":"2024-05-09T06:34:05.337269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:50:42.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.330182+0000","last_change":"2024-05-09T06:34:02.319167+0000","last_active":"2024-05-09T06:34:04.330182+0000","last_peered":"2024-05-09T06:34:04.330182+0000","last_clean":"2024-05-09T06:34:04.330182+0000","last_became_active":"2024-05-09T06:34:02.318846+0000","last_became_peered":"2024-05-09T06:34:02.318846+0000","last_unstale":"2024-05-09T06:34:04.330182+0000","last_undegraded":"2024-05-09T06:34:04.330182+0000","last_fullsized":"2024-05-09T06:34:04.330182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:32.602571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.328079+0000","last_change":"2024-05-09T06:34:03.329957+0000","last_active":"2024-05-09T06:34:04.328079+0000","last_peered":"2024-05-09T06:34:04.328079+0000","last_clean":"2024-05-09T06:34:04.328079+0000","last_became_active":"2024-05-09T06:34:03.329697+0000","last_became_peered":"2024-05-09T06:34:03.329697+0000","last_unstale":"2024-05-09T06:34:04.328079+0000","last_undegraded":"2024-05-09T06:34:04.328079+0000","last_fullsized":"2024-05-09T06:34:04.328079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:41:51.755400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911836+0000","last_change":"2024-05-09T06:34:02.317990+0000","last_active":"2024-05-09T06:34:04.911836+0000","last_peered":"2024-05-09T06:34:04.911836+0000","last_clean":"2024-05-09T06:34:04.911836+0000","last_became_active":"2024-05-09T06:34:02.317688+0000","last_became_peered":"2024-05-09T06:34:02.317688+0000","last_unstale":"2024-05-09T06:34:04.911836+0000","last_undegraded":"2024-05-09T06:34:04.911836+0000","last_fullsized":"2024-05-09T06:34:04.911836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:53.025528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.454827+0000","last_change":"2024-05-09T06:34:03.338515+0000","last_active":"2024-05-09T06:34:04.454827+0000","last_peered":"2024-05-09T06:34:04.454827+0000","last_clean":"2024-05-09T06:34:04.454827+0000","last_became_active":"2024-05-09T06:34:03.338237+0000","last_became_peered":"2024-05-09T06:34:03.338237+0000","last_unstale":"2024-05-09T06:34:04.454827+0000","last_undegraded":"2024-05-09T06:34:04.454827+0000","last_fullsized":"2024-05-09T06:34:04.454827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:27.416635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.337192+0000","last_change":"2024-05-09T06:34:02.326540+0000","last_active":"2024-05-09T06:34:05.337192+0000","last_peered":"2024-05-09T06:34:05.337192+0000","last_clean":"2024-05-09T06:34:05.337192+0000","last_became_active":"2024-05-09T06:34:02.326253+0000","last_became_peered":"2024-05-09T06:34:02.326253+0000","last_unstale":"2024-05-09T06:34:05.337192+0000","last_undegraded":"2024-05-09T06:34:05.337192+0000","last_fullsized":"2024-05-09T06:34:05.337192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:21.329704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329670+0000","last_change":"2024-05-09T06:34:03.343671+0000","last_active":"2024-05-09T06:34:04.329670+0000","last_peered":"2024-05-09T06:34:04.329670+0000","last_clean":"2024-05-09T06:34:04.329670+0000","last_became_active":"2024-05-09T06:34:03.343393+0000","last_became_peered":"2024-05-09T06:34:03.343393+0000","last_unstale":"2024-05-09T06:34:04.329670+0000","last_undegraded":"2024-05-09T06:34:04.329670+0000","last_fullsized":"2024-05-09T06:34:04.329670+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.256448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.329827+0000","last_change":"2024-05-09T06:34:02.323636+0000","last_active":"2024-05-09T06:34:04.329827+0000","last_peered":"2024-05-09T06:34:04.329827+0000","last_clean":"2024-05-09T06:34:04.329827+0000","last_became_active":"2024-05-09T06:34:02.323275+0000","last_became_peered":"2024-05-09T06:34:02.323275+0000","last_unstale":"2024-05-09T06:34:04.329827+0000","last_undegraded":"2024-05-09T06:34:04.329827+0000","last_fullsized":"2024-05-09T06:34:04.329827+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:38.268999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:04.911740+0000","last_change":"2024-05-09T06:34:03.326941+0000","last_active":"2024-05-09T06:34:04.911740+0000","last_peered":"2024-05-09T06:34:04.911740+0000","last_clean":"2024-05-09T06:34:04.911740+0000","last_became_active":"2024-05-09T06:34:03.326696+0000","last_became_peered":"2024-05-09T06:34:03.326696+0000","last_unstale":"2024-05-09T06:34:04.911740+0000","last_undegraded":"2024-05-09T06:34:04.911740+0000","last_fullsized":"2024-05-09T06:34:04.911740+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:24.900392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012975+0000","last_change":"2024-05-09T06:34:02.328232+0000","last_active":"2024-05-09T06:34:06.012975+0000","last_peered":"2024-05-09T06:34:06.012975+0000","last_clean":"2024-05-09T06:34:06.012975+0000","last_became_active":"2024-05-09T06:34:02.327886+0000","last_became_peered":"2024-05-09T06:34:02.327886+0000","last_unstale":"2024-05-09T06:34:06.012975+0000","last_undegraded":"2024-05-09T06:34:06.012975+0000","last_fullsized":"2024-05-09T06:34:06.012975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:21.871549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.013016+0000","last_change":"2024-05-09T06:34:03.339738+0000","last_active":"2024-05-09T06:34:06.013016+0000","last_peered":"2024-05-09T06:34:06.013016+0000","last_clean":"2024-05-09T06:34:06.013016+0000","last_became_active":"2024-05-09T06:34:03.339378+0000","last_became_peered":"2024-05-09T06:34:03.339378+0000","last_unstale":"2024-05-09T06:34:06.013016+0000","last_undegraded":"2024-05-09T06:34:06.013016+0000","last_fullsized":"2024-05-09T06:34:06.013016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:02.301426+0000","last_clean_scrub_stamp":"2024-05-09T06:34:02.301426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:26.843959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:05.150798+0000","last_change":"2024-05-09T06:34:02.331212+0000","last_active":"2024-05-09T06:34:05.150798+0000","last_peered":"2024-05-09T06:34:05.150798+0000","last_clean":"2024-05-09T06:34:05.150798+0000","last_became_active":"2024-05-09T06:34:02.330939+0000","last_became_peered":"2024-05-09T06:34:02.330939+0000","last_unstale":"2024-05-09T06:34:05.150798+0000","last_undegraded":"2024-05-09T06:34:05.150798+0000","last_fullsized":"2024-05-09T06:34:05.150798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:34:01.294790+0000","last_clean_scrub_stamp":"2024-05-09T06:34:01.294790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:50.355406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":299,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-09T06:34:06.012818+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:34:06.012818+0000","last_peered":"2024-05-09T06:34:06.012818+0000","last_clean":"2024-05-09T06:34:06.012818+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:34:06.012818+0000","last_undegraded":"2024-05-09T06:34:06.012818+0000","last_fullsized":"2024-05-09T06:34:06.012818+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1108010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":519802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":517586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":517746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":517586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":517586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":517656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1110272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:12.124 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:34:12.124 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:34:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.289+0000 7f00f8148640 1 Processor -- start 2024-05-09T06:34:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.289+0000 7f00f8148640 1 -- start start 2024-05-09T06:34:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f8148640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f00f00b1680 0x7f00f00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f8148640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f015e900 con 0x7f00f015e390 2024-05-09T06:34:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f8148640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f015ebe0 con 0x7f00f00b11f0 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f8148640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f00f015eec0 con 0x7f00f00b1680 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f66e5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f00f00b1680 0x7f00f00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f5ee4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f00f015e390 0x7f00f00cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59244/0 (socket says 172.21.15.1:59244) 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f66e5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f00f00b1680 0x7f00f00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42552/0 (socket says 172.21.15.1:42552) 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f5ee4640 1 -- 172.21.15.1:0/3902182126 learned_addr learned my addr 172.21.15.1:0/3902182126 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.290+0000 7f00f66e5640 1 -- 172.21.15.1:0/3902182126 >> v1:172.21.15.6:6789/0 conn(0x7f00f00b11f0 legacy=0x7f00f00cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f66e5640 1 -- 172.21.15.1:0/3902182126 >> v1:172.21.15.1:6789/0 conn(0x7f00f015e390 legacy=0x7f00f00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f66e5640 1 -- 172.21.15.1:0/3902182126 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f015f1a0 con 0x7f00f00b1680 2024-05-09T06:34:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f4ee2640 1 -- 172.21.15.1:0/3902182126 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205634190 0 0) 0x7f00f015e900 con 0x7f00f015e390 2024-05-09T06:34:12.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f66e5640 1 --2- 172.21.15.1:0/3902182126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00ccff0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009d90 tx=0x7f00e002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566939f5d3150258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f4ee2640 1 -- 172.21.15.1:0/3902182126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0007740 con 0x7f00f00b1680 2024-05-09T06:34:12.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f4ee2640 1 -- 172.21.15.1:0/3902182126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e00078e0 con 0x7f00f00b1680 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.291+0000 7f00f4ee2640 1 -- 172.21.15.1:0/3902182126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0007cb0 con 0x7f00f00b1680 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 -- 172.21.15.1:0/3902182126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 msgr2=0x7f00f00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 --2- 172.21.15.1:0/3902182126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00ccff0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009d90 tx=0x7f00e002a6f0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 -- 172.21.15.1:0/3902182126 shutdown_connections 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 --2- 172.21.15.1:0/3902182126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00ccff0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 -- 172.21.15.1:0/3902182126 >> 172.21.15.1:0/3902182126 conn(0x7f00f00b1da0 msgr2=0x7f00f00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 -- 172.21.15.1:0/3902182126 shutdown_connections 2024-05-09T06:34:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.292+0000 7f00f8148640 1 -- 172.21.15.1:0/3902182126 wait complete. 2024-05-09T06:34:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.293+0000 7f00f8148640 1 Processor -- start 2024-05-09T06:34:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.293+0000 7f00f8148640 1 -- start start 2024-05-09T06:34:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f00b11f0 0x7f00f014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00f015e390 0x7f00f00adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0162ba0 con 0x7f00f00b1680 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00b1a80 con 0x7f00f00b11f0 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f8148640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00cffb0 con 0x7f00f015e390 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42556/0 (socket says 172.21.15.1:42556) 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 -- 172.21.15.1:0/2771741030 learned_addr learned my addr 172.21.15.1:0/2771741030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f66e5640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00f015e390 0x7f00f00adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f5ee4640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f00b11f0 0x7f00f014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 -- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00f015e390 msgr2=0x7f00f00adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00f015e390 0x7f00f00adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 -- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f00b11f0 msgr2=0x7f00f014cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f00b11f0 0x7f00f014cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.294+0000 7f00f56e3640 1 -- 172.21.15.1:0/2771741030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f014d470 con 0x7f00f00b1680 2024-05-09T06:34:12.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00f56e3640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f00e400c450 tx=0x7f00e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40052d0 con 0x7f00f00b1680 2024-05-09T06:34:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e4005470 con 0x7f00f00b1680 2024-05-09T06:34:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f00b3f80 con 0x7f00f00b1680 2024-05-09T06:34:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4012870 con 0x7f00f00b1680 2024-05-09T06:34:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.295+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f00b47c0 con 0x7f00f00b1680 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.296+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f00752b0 con 0x7f00f00b1680 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.296+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00e4012a10 con 0x7f00f00b1680 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.297+0000 7f00ee7fc640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 0x7f00dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.297+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f00e408aca0 con 0x7f00f00b1680 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.297+0000 7f00f5ee4640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 0x7f00dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.297+0000 7f00f5ee4640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 0x7f00dc06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f00e8000f60 tx=0x7f00e8007d00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.301+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00f00b3f80 con 0x7f00f00b1680 2024-05-09T06:34:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.557+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f00f00b2ae0 con 0x7f00f00b1680 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.559+0000 7f00ee7fc640 1 -- 172.21.15.1:0/2771741030 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f00e4059430 con 0x7f00f00b1680 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:34:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:34:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 msgr2=0x7f00dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 0x7f00dc06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f00e8000f60 tx=0x7f00e8007d00 comp rx=0 tx=0).stop 2024-05-09T06:34:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 msgr2=0x7f00f00c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f00e400c450 tx=0x7f00e400c910 comp rx=0 tx=0).stop 2024-05-09T06:34:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 shutdown_connections 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00dc06be30 0x7f00dc06e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00f015e390 0x7f00f00adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00f00b1680 0x7f00f00c3d80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 --2- 172.21.15.1:0/2771741030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00f00b11f0 0x7f00f014cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 >> 172.21.15.1:0/2771741030 conn(0x7f00f00b1da0 msgr2=0x7f00f00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 shutdown_connections 2024-05-09T06:34:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.561+0000 7f00f8148640 1 -- 172.21.15.1:0/2771741030 wait complete. 2024-05-09T06:34:12.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.675+0000 7fe3fd55a640 1 Processor -- start 2024-05-09T06:34:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.675+0000 7fe3fd55a640 1 -- start start 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3fd55a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3f80d2b10 0x7fe3f80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3fd55a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80b33c0 con 0x7fe3f8161950 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3fd55a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80b3660 con 0x7fe3f80d4940 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3fd55a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe3f80d4dd0 con 0x7fe3f80d2b10 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3f6ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe3f8161950 0x7fe3f80d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59248/0 (socket says 172.21.15.1:59248) 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.676+0000 7fe3f6ffd640 1 -- 172.21.15.1:0/107550268 learned_addr learned my addr 172.21.15.1:0/107550268 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f77fe640 1 --2- 172.21.15.1:0/107550268 >> v2:172.21.15.6:3300/0 conn(0x7fe3f80d2b10 0x7fe3f80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f5ffb640 1 -- 172.21.15.1:0/107550268 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341194709 0 0) 0x7fe3f80b33c0 con 0x7fe3f8161950 2024-05-09T06:34:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f5ffb640 1 -- 172.21.15.1:0/107550268 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e0003520 con 0x7fe3f8161950 2024-05-09T06:34:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f77fe640 1 -- 172.21.15.1:0/107550268 >> v1:172.21.15.6:6789/0 conn(0x7fe3f80d4940 legacy=0x7fe3f80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f77fe640 1 -- 172.21.15.1:0/107550268 >> v1:172.21.15.1:6789/0 conn(0x7fe3f8161950 legacy=0x7fe3f80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.677+0000 7fe3f77fe640 1 -- 172.21.15.1:0/107550268 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f80b3940 con 0x7fe3f80d2b10 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3f77fe640 1 --2- 172.21.15.1:0/107550268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80b4850 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00097a0 tx=0x7fe3f002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fcd3a06563aa7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3f5ffb640 1 -- 172.21.15.1:0/107550268 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0037070 con 0x7fe3f80d2b10 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3f5ffb640 1 -- 172.21.15.1:0/107550268 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f0032440 con 0x7fe3f80d2b10 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3f5ffb640 1 -- 172.21.15.1:0/107550268 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0032810 con 0x7fe3f80d2b10 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3fd55a640 1 -- 172.21.15.1:0/107550268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 msgr2=0x7fe3f80b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.678+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/107550268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80b4850 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00097a0 tx=0x7fe3f002a230 comp rx=0 tx=0).stop 2024-05-09T06:34:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.679+0000 7fe3fd55a640 1 -- 172.21.15.1:0/107550268 shutdown_connections 2024-05-09T06:34:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.679+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/107550268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80b4850 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.679+0000 7fe3fd55a640 1 -- 172.21.15.1:0/107550268 >> 172.21.15.1:0/107550268 conn(0x7fe3f80d3e20 msgr2=0x7fe3f80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.679+0000 7fe3fd55a640 1 -- 172.21.15.1:0/107550268 shutdown_connections 2024-05-09T06:34:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.679+0000 7fe3fd55a640 1 -- 172.21.15.1:0/107550268 wait complete. 2024-05-09T06:34:12.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.680+0000 7fe3fd55a640 1 Processor -- start 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.680+0000 7fe3fd55a640 1 -- start start 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 0x7fe3f80b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80d1100 con 0x7fe3f81f3a20 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80d0f80 con 0x7fe3f80d2b10 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3fd55a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f80d1c00 con 0x7fe3f8161950 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42562/0 (socket says 172.21.15.1:42562) 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 -- 172.21.15.1:0/2322359653 learned_addr learned my addr 172.21.15.1:0/2322359653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f6ffd640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f67fc640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 0x7fe3f80b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 -- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 msgr2=0x7fe3f80b9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 0x7fe3f80b9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 -- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 msgr2=0x7fe3f80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f77fe640 1 -- 172.21.15.1:0/2322359653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81f42c0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.681+0000 7fe3f67fc640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 0x7fe3f80b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3f77fe640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0035040 tx=0x7fe3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0037040 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f0042070 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f81f45a0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00308a0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.682+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f80cbf40 con 0x7fe3f81f3a20 2024-05-09T06:34:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.683+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f80752b0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.683+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3f0030a40 con 0x7fe3f81f3a20 2024-05-09T06:34:12.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.684+0000 7fe3df7fe640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 0x7fe3e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.684+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fe3f00ab830 con 0x7fe3f81f3a20 2024-05-09T06:34:12.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.684+0000 7fe3f6ffd640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 0x7fe3e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.684+0000 7fe3f6ffd640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 0x7fe3e406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8001560 tx=0x7fe3e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.688+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3f007a0e0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.910+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3f80b4290 con 0x7fe3f81f3a20 2024-05-09T06:34:12.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.911+0000 7fe3df7fe640 1 -- 172.21.15.1:0/2322359653 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5168 (secure 0 0 0) 0x7fe3f00799e0 con 0x7fe3f81f3a20 2024-05-09T06:34:12.892 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:12.892 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7899,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:05.704044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7911},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7911":{"gid":7911,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/1974102470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1974102470},{"type":"v1","addr":"172.21.15.6:6835","nonce":1974102470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7911,"qdb_cluster":[7911]},"id":6}]} 2024-05-09T06:34:12.892 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 61 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 msgr2=0x7fe3e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 0x7fe3e406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8001560 tx=0x7fe3e8009040 comp rx=0 tx=0).stop 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 msgr2=0x7fe3f80cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0035040 tx=0x7fe3f00046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 shutdown_connections 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe3e406be60 0x7fe3e406e320 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3f81f3a20 0x7fe3f80cc740 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3f8161950 0x7fe3f80b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 --2- 172.21.15.1:0/2322359653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3f80d2b10 0x7fe3f80cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 >> 172.21.15.1:0/2322359653 conn(0x7fe3f80d3e20 msgr2=0x7fe3f80b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 shutdown_connections 2024-05-09T06:34:12.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:12.913+0000 7fe3fd55a640 1 -- 172.21.15.1:0/2322359653 wait complete. 2024-05-09T06:34:13.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:34:05.704044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7911}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7911': {'gid': 7911, 'name': 'd', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6835/1974102470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6833', 'nonce': 1974102470}, {'type': 'v1', 'addr': '172.21.15.6:6835', 'nonce': 1974102470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7911, 'qdb_cluster': [7911]} 2024-05-09T06:34:13.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:34:13.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T06:34:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.434+0000 7fdb38215640 1 Processor -- start 2024-05-09T06:34:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.434+0000 7fdb38215640 1 -- start start 2024-05-09T06:34:13.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb38215640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb30070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb38215640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb300aec10 0x7fdb30162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb38215640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb300cdfd0 con 0x7fdb3015e9f0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb38215640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdb300ceb50 con 0x7fdb300aec10 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb38215640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb300ce2b0 con 0x7fdb3015e610 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb357b0640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdb3015e9f0 0x7fdb30161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59252/0 (socket says 172.21.15.1:59252) 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb357b0640 1 -- 172.21.15.1:0/2059997672 learned_addr learned my addr 172.21.15.1:0/2059997672 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb367b2640 1 --2- 172.21.15.1:0/2059997672 >> v2:172.21.15.6:3300/0 conn(0x7fdb300aec10 0x7fdb30162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963037341 0 0) 0x7fdb300cdfd0 con 0x7fdb3015e9f0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb30070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb1c003520 con 0x7fdb3015e9f0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 947181263 0 0) 0x7fdb1c003520 con 0x7fdb3015e9f0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 msgr2=0x7fdb30070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb30070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 >> v2:172.21.15.6:3300/0 conn(0x7fdb300aec10 msgr2=0x7fdb30162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 --2- 172.21.15.1:0/2059997672 >> v2:172.21.15.6:3300/0 conn(0x7fdb300aec10 0x7fdb30162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb300ce590 con 0x7fdb3015e9f0 2024-05-09T06:34:13.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fdb20001670 con 0x7fdb3015e9f0 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 >> v1:172.21.15.1:6789/0 conn(0x7fdb3015e9f0 legacy=0x7fdb30161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb1c0039b0 0x7fdb1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.435+0000 7fdb34faf640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c004d40 0x7fdb1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb300ceb50 con 0x7fdb1c0039b0 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdb300ce2b0 con 0x7fdb1c004d40 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c00b9c0 con 0x7fdb3015e610 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb1c0039b0 0x7fdb1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb367b2640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c004d40 0x7fdb1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb357b0640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 -- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 msgr2=0x7fdb1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb3015e610 0x7fdb1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 -- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c004d40 msgr2=0x7fdb1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c004d40 0x7fdb1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.436+0000 7fdb35fb1640 1 -- 172.21.15.1:0/2059997672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c003520 con 0x7fdb1c0039b0 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb1c0039b0 0x7fdb1c003dd0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009ac0 tx=0x7fdb1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7592dcaa239abe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c036070 con 0x7fdb1c0039b0 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2c031400 con 0x7fdb1c0039b0 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb34faf640 1 -- 172.21.15.1:0/2059997672 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c009690 con 0x7fdb1c0039b0 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 -- 172.21.15.1:0/2059997672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb1c0039b0 msgr2=0x7fdb1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 --2- 172.21.15.1:0/2059997672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb1c0039b0 0x7fdb1c003dd0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009ac0 tx=0x7fdb1c005830 comp rx=0 tx=0).stop 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb367b2640 1 -- 172.21.15.1:0/2059997672 reap_dead start 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 -- 172.21.15.1:0/2059997672 shutdown_connections 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 -- 172.21.15.1:0/2059997672 >> 172.21.15.1:0/2059997672 conn(0x7fdb300cc4b0 msgr2=0x7fdb300cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 -- 172.21.15.1:0/2059997672 shutdown_connections 2024-05-09T06:34:13.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.437+0000 7fdb38215640 1 -- 172.21.15.1:0/2059997672 wait complete. 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 Processor -- start 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 -- start start 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 0x7fdb301f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 0x7fdb301f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3015ee30 con 0x7fdb3013eec0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdb300718c0 con 0x7fdb301f7ac0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.439+0000 7fdb38215640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb301f6ec0 con 0x7fdb1c0039b0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42574/0 (socket says 172.21.15.1:42574) 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 -- 172.21.15.1:0/2970702442 learned_addr learned my addr 172.21.15.1:0/2970702442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 0x7fdb301f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb357b0640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 0x7fdb301f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 -- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 msgr2=0x7fdb301f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 0x7fdb301f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 -- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 msgr2=0x7fdb301f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 0x7fdb301f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 -- 172.21.15.1:0/2970702442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb300b2810 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb357b0640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 0x7fdb301f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 0x7fdb301f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb367b2640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdb3004ff70 tx=0x7fdb2800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb267fc640 1 -- 172.21.15.1:0/2970702442 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28019070 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb267fc640 1 -- 172.21.15.1:0/2970702442 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28011e90 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb300b2ab0 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb267fc640 1 -- 172.21.15.1:0/2970702442 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28003bf0 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.440+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb300b3070 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.442+0000 7fdb267fc640 1 -- 172.21.15.1:0/2970702442 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb2801c070 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.442+0000 7fdb267fc640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 0x7fdb1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.442+0000 7fdb267fc640 1 -- 172.21.15.1:0/2970702442 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fdb2808b830 con 0x7fdb3013eec0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.442+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 0x7fdb1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.443+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 0x7fdb1406e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c033000 tx=0x7fdb2c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.443+0000 7fdb3b000740 1 Processor -- start 2024-05-09T06:34:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.443+0000 7fdb3b000740 1 -- start start 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 0x55a3fbe02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe1ff10 0x55a3fbe20460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbd3b4d0 con 0x55a3fbdd8350 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbce33e0 con 0x55a3fbe02670 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a3fbd37490 con 0x55a3fbe1ff10 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb367b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb367b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42580/0 (socket says 172.21.15.1:42580) 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.444+0000 7fdb367b2640 1 -- 172.21.15.1:0/3539752218 learned_addr learned my addr 172.21.15.1:0/3539752218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 0x55a3fbe02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb357b0640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe1ff10 0x55a3fbe20460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 -- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe1ff10 msgr2=0x55a3fbe20460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe1ff10 0x55a3fbe20460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 -- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 msgr2=0x55a3fbe02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 0x55a3fbe02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 -- 172.21.15.1:0/3539752218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3fba4f2d0 con 0x55a3fbdd8350 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 0x55a3fbe02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb367b2640 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdb28055e40 tx=0x7fdb28059aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c55b43b4e3223 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb137fe640 1 -- 172.21.15.1:0/3539752218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28023040 con 0x55a3fbdd8350 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb137fe640 1 -- 172.21.15.1:0/3539752218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28003d90 con 0x55a3fbdd8350 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.445+0000 7fdb137fe640 1 -- 172.21.15.1:0/3539752218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28056050 con 0x55a3fbdd8350 2024-05-09T06:34:13.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- 172.21.15.1:0/3539752218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 msgr2=0x55a3fbe1e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdb28055e40 tx=0x7fdb28059aa0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- 172.21.15.1:0/3539752218 shutdown_connections 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbdd8350 0x55a3fbe1e5d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe1ff10 0x55a3fbe20460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 --2- 172.21.15.1:0/3539752218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe02670 0x55a3fbe02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- 172.21.15.1:0/3539752218 >> 172.21.15.1:0/3539752218 conn(0x55a3fbdd4c80 msgr2=0x55a3fbe1d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- 172.21.15.1:0/3539752218 shutdown_connections 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- 172.21.15.1:0/3539752218 wait complete. 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 -- start start 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.446+0000 7fdb3b000740 1 Processor -- start 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbdfede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 0x55a3fbe1e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbdd90f0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbcf68e0 con 0x55a3fbe2d8c0 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a3fbdd2bb0 con 0x55a3fbe02670 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb3b000740 10 client.0 _ll_register_callbacks cb 0x55a3fbcc39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42586/0 (socket says 172.21.15.1:42586) 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.447+0000 7fdb357b0640 1 -- 172.21.15.1:0/1522902789 learned_addr learned my addr 172.21.15.1:0/1522902789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbdfede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb367b2640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 0x55a3fbe1e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 -- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 msgr2=0x55a3fbdfede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbdfede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 -- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 msgr2=0x55a3fbe1e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 0x55a3fbe1e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 -- 172.21.15.1:0/1522902789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3fbb196b0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb367b2640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 0x55a3fbe1e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdb30074210 tx=0x7fdb20004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3681734c035abb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2001c060 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fdb20017c80 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb28003d90 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fdb200155e0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb3b000740 10 client.8052 fetch_fsmap learned FSMap version 61 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55a3fbe169c0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28003d90 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.448+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fdb20015780 con 0x55a3fbe1ff10 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 10 client.8052 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 10 client.8052 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 20 client.8052 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe31080 0x55a3fbe04be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 4 client.8052 mds_command: new command op to 7911 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 --> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3fbd48c00 con 0x55a3fbe31080 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe31080 0x55a3fbe04be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe31080 0x55a3fbe04be0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603606d7751faa14 server_cookie=4a401796c29a27c3 in_seq=0 out_seq=0 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.449+0000 7fdb10ff9640 10 client.8052 ms_handle_connect on v2:172.21.15.6:6833/1974102470 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.450+0000 7fdb10ff9640 1 -- 172.21.15.1:0/1522902789 <== mds.0 v2:172.21.15.6:6833/1974102470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3fbd48c00 con 0x55a3fbe31080 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.450+0000 7fdb10ff9640 10 client.8052 handle_command_reply: tid=0 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.450+0000 7fdb3b000740 1 client.8052 shutdown 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.450+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 msgr2=0x55a3fbe16480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.450+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdb30074210 tx=0x7fdb20004ea0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 shutdown_connections 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe2d8c0 0x55a3fbe1e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe31080 0x55a3fbe04be0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe16480 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb10ff9640 0 client.8052 ms_handle_reset on v2:172.21.15.6:6833/1974102470 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1522902789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbdfede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 >> 172.21.15.1:0/1522902789 conn(0x55a3fbdd8350 msgr2=0x55a3fbe20a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 shutdown_connections 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 1 -- 172.21.15.1:0/1522902789 wait complete. 2024-05-09T06:34:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.451+0000 7fdb3b000740 20 client.8052 trim_cache size 0 max 16384 2024-05-09T06:34:13.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.469+0000 7fdb3b000740 1 Processor -- start 2024-05-09T06:34:13.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.469+0000 7fdb3b000740 1 -- start start 2024-05-09T06:34:13.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1f920 0x55a3fbe20b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30200320 con 0x55a3fbe1ff10 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301ff6d0 con 0x55a3fbe1f920 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb301ffd20 con 0x55a3fbe02670 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42590/0 (socket says 172.21.15.1:42590) 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 -- 172.21.15.1:0/1613423988 learned_addr learned my addr 172.21.15.1:0/1613423988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 -- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 msgr2=0x55a3fbe01bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 -- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1f920 msgr2=0x55a3fbe20b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1f920 0x55a3fbe20b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 -- 172.21.15.1:0/1613423988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3fbe15090 con 0x55a3fbe1ff10 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.470+0000 7fdb357b0640 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdb20000f50 tx=0x7fdb2000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb16a33a4a153d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb12ffd640 1 -- 172.21.15.1:0/1613423988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200153e0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb12ffd640 1 -- 172.21.15.1:0/1613423988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb20015780 con 0x55a3fbe1ff10 2024-05-09T06:34:13.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb12ffd640 1 -- 172.21.15.1:0/1613423988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20016440 con 0x55a3fbe1ff10 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 -- 172.21.15.1:0/1613423988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 msgr2=0x55a3fbe1e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdb20000f50 tx=0x7fdb2000c000 comp rx=0 tx=0).stop 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 -- 172.21.15.1:0/1613423988 shutdown_connections 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1f920 0x55a3fbe20b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe1e190 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 --2- 172.21.15.1:0/1613423988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe01bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 -- 172.21.15.1:0/1613423988 >> 172.21.15.1:0/1613423988 conn(0x55a3fbdd8350 msgr2=0x55a3fbdfebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 -- 172.21.15.1:0/1613423988 shutdown_connections 2024-05-09T06:34:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.471+0000 7fdb3b000740 1 -- 172.21.15.1:0/1613423988 wait complete. 2024-05-09T06:34:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.472+0000 7fdb3b000740 1 -- start start 2024-05-09T06:34:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.472+0000 7fdb3b000740 1 Processor -- start 2024-05-09T06:34:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe34850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe2dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbe30f00 con 0x55a3fbe1f920 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a3fbe16cb0 con 0x55a3fbe1ff10 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a3fbe314f0 con 0x55a3fbe02670 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb3b000740 10 client.0 _ll_register_callbacks cb 0x7fdb20007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42598/0 (socket says 172.21.15.1:42598) 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 -- 172.21.15.1:0/142549379 learned_addr learned my addr 172.21.15.1:0/142549379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb367b2640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe2dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 -- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 msgr2=0x55a3fbe34850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe34850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 -- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1ff10 msgr2=0x55a3fbe2dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe2dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.473+0000 7fdb357b0640 1 -- 172.21.15.1:0/142549379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3fbe17000 con 0x55a3fbe1f920 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb357b0640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005fe0 tx=0x7fdb300ce1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876ccb87d19b821f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20037070 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fdb2001c9b0 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb20015780 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fdb20040050 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 10 client.8058 fetch_fsmap learned FSMap version 61 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55a3fbe405b0 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb20015780 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fdb20043020 con 0x55a3fbe1f920 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 10 client.8058 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 10 client.8058 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 20 client.8058 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe40fb0 0x55a3fbe12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 4 client.8058 mds_command: new command op to 7911 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 --> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] -- command(tid 0: {"prefix": "status"}) -- 0x55a3fbd42f50 con 0x55a3fbe40fb0 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.474+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe40fb0 0x55a3fbe12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb35fb1640 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe40fb0 0x55a3fbe12e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89304a3d5e2b20c server_cookie=73296272f7590c9d in_seq=0 out_seq=0 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb10ff9640 10 client.8058 ms_handle_connect on v2:172.21.15.6:6833/1974102470 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb10ff9640 1 -- 172.21.15.1:0/142549379 <== mds.0 v2:172.21.15.6:6833/1974102470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a3fbd42f50 con 0x55a3fbe40fb0 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb10ff9640 10 client.8058 handle_command_reply: tid=0 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 client.8058 shutdown 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 msgr2=0x55a3fbe97bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005fe0 tx=0x7fdb300ce1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 shutdown_connections 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3fbe1ff10 0x55a3fbe2dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:6833/1974102470,v1:172.21.15.6:6835/1974102470] conn(0x55a3fbe40fb0 0x55a3fbe12e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3fbe1f920 0x55a3fbe97bb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 --2- 172.21.15.1:0/142549379 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a3fbe02670 0x55a3fbe34850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 >> 172.21.15.1:0/142549379 conn(0x55a3fbdd8350 msgr2=0x55a3fbe20b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb10ff9640 0 client.8058 ms_handle_reset on v2:172.21.15.6:6833/1974102470 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 shutdown_connections 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 1 -- 172.21.15.1:0/142549379 wait complete. 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.475+0000 7fdb3b000740 20 client.8058 trim_cache size 0 max 16384 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7911, 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:34:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 10.150031026000001, 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 61, 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 91, 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 91, 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 16.142107516999999 2024-05-09T06:34:13.463 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 msgr2=0x7fdb1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 0x7fdb1406e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c033000 tx=0x7fdb2c008030 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 msgr2=0x7fdb3013f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdb3004ff70 tx=0x7fdb2800f580 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 shutdown_connections 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb1406be60 0x7fdb1406e320 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb3013eec0 0x7fdb3013f340 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb301f7ac0 0x7fdb301f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 --2- 172.21.15.1:0/2970702442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c0039b0 0x7fdb301f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 >> 172.21.15.1:0/2970702442 conn(0x7fdb300cc4b0 msgr2=0x7fdb300af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 shutdown_connections 2024-05-09T06:34:13.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.476+0000 7fdb38215640 1 -- 172.21.15.1:0/2970702442 wait complete. 2024-05-09T06:34:13.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:34:13.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.591+0000 7ff97d807640 1 Processor -- start 2024-05-09T06:34:13.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.591+0000 7ff97d807640 1 -- start start 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6860 0x7ff978099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9780b2aa0 0x7ff97815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff9780c6c40 0x7ff97815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff9780b1e20 con 0x7ff9780b2aa0 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff97815ee90 con 0x7ff9780c6c40 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff97d807640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff978099930 con 0x7ff9780c6860 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9780b2aa0 0x7ff97815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9780b2aa0 0x7ff97815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42614/0 (socket says 172.21.15.1:42614) 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 -- 172.21.15.1:0/3908905313 learned_addr learned my addr 172.21.15.1:0/3908905313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff976ffd640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6860 0x7ff978099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff9777fe640 1 --2- 172.21.15.1:0/3908905313 >> v2:172.21.15.6:3300/0 conn(0x7ff9780c6c40 0x7ff97815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 -- 172.21.15.1:0/3908905313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6860 msgr2=0x7ff978099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6860 0x7ff978099b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 -- 172.21.15.1:0/3908905313 >> v2:172.21.15.6:3300/0 conn(0x7ff9780c6c40 msgr2=0x7ff97815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 --2- 172.21.15.1:0/3908905313 >> v2:172.21.15.6:3300/0 conn(0x7ff9780c6c40 0x7ff97815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.592+0000 7ff96ffff640 1 -- 172.21.15.1:0/3908905313 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978162930 con 0x7ff9780b2aa0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.593+0000 7ff96ffff640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff97815e4b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff978099570 tx=0x7ff96002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3645bed4e0d0e38a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.593+0000 7ff9767fc640 1 -- 172.21.15.1:0/3908905313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960037070 con 0x7ff9780b2aa0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.593+0000 7ff9767fc640 1 -- 172.21.15.1:0/3908905313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960032440 con 0x7ff9780b2aa0 2024-05-09T06:34:13.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.593+0000 7ff9767fc640 1 -- 172.21.15.1:0/3908905313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960032810 con 0x7ff9780b2aa0 2024-05-09T06:34:13.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 -- 172.21.15.1:0/3908905313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 msgr2=0x7ff97815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff97815e4b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff978099570 tx=0x7ff96002a140 comp rx=0 tx=0).stop 2024-05-09T06:34:13.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 -- 172.21.15.1:0/3908905313 shutdown_connections 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 --2- 172.21.15.1:0/3908905313 >> v2:172.21.15.6:3300/0 conn(0x7ff9780c6c40 0x7ff97815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff97815e4b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 --2- 172.21.15.1:0/3908905313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6860 0x7ff978099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 -- 172.21.15.1:0/3908905313 >> 172.21.15.1:0/3908905313 conn(0x7ff9780b1fb0 msgr2=0x7ff9780b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 -- 172.21.15.1:0/3908905313 shutdown_connections 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.594+0000 7ff97d807640 1 -- 172.21.15.1:0/3908905313 wait complete. 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.595+0000 7ff97d807640 1 Processor -- start 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.595+0000 7ff97d807640 1 -- start start 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9780c6860 0x7ff9780cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 0x7ff9781526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780c70d0 con 0x7ff9780b2aa0 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff97815eb20 con 0x7ff9780c6860 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff97815f860 con 0x7ff9780c6c40 2024-05-09T06:34:13.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42630/0 (socket says 172.21.15.1:42630) 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 -- 172.21.15.1:0/2259842209 learned_addr learned my addr 172.21.15.1:0/2259842209 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff96ffff640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9780c6860 0x7ff9780cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff9777fe640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 0x7ff9781526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 -- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 msgr2=0x7ff9781526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 0x7ff9781526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 -- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9780c6860 msgr2=0x7ff9780cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9780c6860 0x7ff9780cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 -- 172.21.15.1:0/2259842209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9781f8850 con 0x7ff9780b2aa0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff9777fe640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 0x7ff9781526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff976ffd640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d800 tx=0x7ff96800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968018030 con 0x7ff9780b2aa0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff968009640 con 0x7ff9780b2aa0 2024-05-09T06:34:13.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9780b3760 con 0x7ff9780b2aa0 2024-05-09T06:34:13.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.596+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968009810 con 0x7ff9780b2aa0 2024-05-09T06:34:13.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.597+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781f7d70 con 0x7ff9780b2aa0 2024-05-09T06:34:13.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.597+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780752b0 con 0x7ff9780b2aa0 2024-05-09T06:34:13.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.598+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff96801b070 con 0x7ff9780b2aa0 2024-05-09T06:34:13.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.598+0000 7ff974ff9640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 0x7ff95806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:13.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.599+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff96808b180 con 0x7ff9780b2aa0 2024-05-09T06:34:13.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.599+0000 7ff96ffff640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 0x7ff95806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:13.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.599+0000 7ff96ffff640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 0x7ff95806e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff960034000 tx=0x7ff96002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:13.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.603+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9780b3760 con 0x7ff9780b2aa0 2024-05-09T06:34:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.827+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff97806ce30 con 0x7ff9780b2aa0 2024-05-09T06:34:13.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.827+0000 7ff974ff9640 1 -- 172.21.15.1:0/2259842209 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7ff968059a30 con 0x7ff9780b2aa0 2024-05-09T06:34:13.808 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 msgr2=0x7ff95806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 0x7ff95806e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff960034000 tx=0x7ff96002e000 comp rx=0 tx=0).stop 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 msgr2=0x7ff9780cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d800 tx=0x7ff96800dcc0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 shutdown_connections 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff95806be30 0x7ff95806e2f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9780c6c40 0x7ff9781526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9780c6860 0x7ff9780cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 --2- 172.21.15.1:0/2259842209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9780b2aa0 0x7ff9780cb470 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:13.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 >> 172.21.15.1:0/2259842209 conn(0x7ff9780b1fb0 msgr2=0x7ff97806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:13.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 shutdown_connections 2024-05-09T06:34:13.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:13.829+0000 7ff97d807640 1 -- 172.21.15.1:0/2259842209 wait complete. 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:34:13.821 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:13.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:13.822 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:34:13.838 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:34:13.838 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:34:13.839 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42288sec preferred_lft 42288sec 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:13.840 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:13.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:13.841 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:34:13.903 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:34:13.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:13.905 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:34:13.961 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:34:13.962 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:34:13.962 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:34:13.962 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:34:13.962 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:34:13.962 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:34:13.962 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:34:13.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:34:13.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:34:13.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:34:13.963 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:34:14.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:34:14.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:34:14.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.092 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:34:14.108 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:34:14.108 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:34:14.108 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:14.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:34:14.173 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:34:14.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:34:14.174 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:14.230 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:14.230 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:34:14.230 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:14.285 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:34:14.353 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:34:14.410 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:34:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:14.412 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:14.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.413 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:34:14.469 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:34:14.470 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:34:14.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:34:14.538 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:34:14.595 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:14.598 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:14.598 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:14.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.598 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:34:14.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:34:14.617+0000 7f54a6670480 -1 init, newargv = 0x5555763b41e0 newargc=15 2024-05-09T06:34:14.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[68342]: starting ceph client 2024-05-09T06:34:14.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[68342]: starting fuse 2024-05-09T06:34:14.615 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:34:14.615 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:34:14.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:34:14.615 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:34:14.616 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:34:14.617 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:34:14.737 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.68342.asok 2024-05-09T06:34:14.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:34:14.748 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.68342.asok 2024-05-09T06:34:14.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.68342.asok status 2024-05-09T06:34:14.886 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:14.886 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:34:14.886 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:34:14.886 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "68342", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 8067, 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 8067 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:14.887 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1434467045 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1434467045 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.8067 192.168.0.1:0/1434467045", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1434467045", 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 61, 2024-05-09T06:34:14.888 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 91, 2024-05-09T06:34:14.889 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:34:14.889 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:34:14.889 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:34:14.889 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:14.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.894 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:34:14.895 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:14.911 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:34:14.911 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:14.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:14.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:14.978 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-09T06:34:15.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:15.041 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-09T06:34:15.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:15.100 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-09T06:34:15.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:15.158 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-09T06:34:15.221 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:34:15.329 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:34:15.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:34:15.338 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:34:15.338 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:34:15.395 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:34:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.557+0000 7fbecb33b640 1 Processor -- start 2024-05-09T06:34:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.557+0000 7fbecb33b640 1 -- start start 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbecb33b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cf180 0x7fbec40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbecb33b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbec40cdab0 0x7fbec40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbecb33b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec40ce0e0 con 0x7fbec406e6d0 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbecb33b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbec40ce3c0 con 0x7fbec40cdab0 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbecb33b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbec40ce6a0 con 0x7fbec40cf180 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbec98d8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbec40cdab0 0x7fbec40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbec98d8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbec40cdab0 0x7fbec40b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42640/0 (socket says 172.21.15.1:42640) 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.558+0000 7fbec98d8640 1 -- 172.21.15.1:0/954576898 learned_addr learned my addr 172.21.15.1:0/954576898 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec88d6640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cf180 0x7fbec40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec98d8640 1 -- 172.21.15.1:0/954576898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cf180 msgr2=0x7fbec40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec98d8640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cf180 0x7fbec40cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec98d8640 1 -- 172.21.15.1:0/954576898 >> v1:172.21.15.6:6789/0 conn(0x7fbec406e6d0 legacy=0x7fbec406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec98d8640 1 -- 172.21.15.1:0/954576898 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec40ce980 con 0x7fbec40cdab0 2024-05-09T06:34:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.559+0000 7fbec98d8640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cdab0 0x7fbec40b3300 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87eb7e89cbd54735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbec3fff640 1 -- 172.21.15.1:0/954576898 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4032610 con 0x7fbec40cdab0 2024-05-09T06:34:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbec3fff640 1 -- 172.21.15.1:0/954576898 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb4037070 con 0x7fbec40cdab0 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbec3fff640 1 -- 172.21.15.1:0/954576898 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40329d0 con 0x7fbec40cdab0 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 -- 172.21.15.1:0/954576898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cdab0 msgr2=0x7fbec40b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cdab0 0x7fbec40b3300 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb402c920 comp rx=0 tx=0).stop 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 -- 172.21.15.1:0/954576898 shutdown_connections 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cdab0 0x7fbec40b3300 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 --2- 172.21.15.1:0/954576898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cf180 0x7fbec40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 -- 172.21.15.1:0/954576898 >> 172.21.15.1:0/954576898 conn(0x7fbec40cc4b0 msgr2=0x7fbec40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 -- 172.21.15.1:0/954576898 shutdown_connections 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.560+0000 7fbecb33b640 1 -- 172.21.15.1:0/954576898 wait complete. 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.561+0000 7fbecb33b640 1 Processor -- start 2024-05-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 -- start start 2024-05-09T06:34:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbec406e6d0 0x7fbec41f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cdab0 0x7fbec415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbec41447c0 con 0x7fbec40cf180 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbec4070ed0 con 0x7fbec406e6d0 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbecb33b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbec4144940 con 0x7fbec40cdab0 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbec98d8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbec98d8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42650/0 (socket says 172.21.15.1:42650) 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbec98d8640 1 -- 172.21.15.1:0/2757343489 learned_addr learned my addr 172.21.15.1:0/2757343489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbec90d7640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbec406e6d0 0x7fbec41f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.562+0000 7fbec88d6640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cdab0 0x7fbec415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 -- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cdab0 msgr2=0x7fbec415bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cdab0 0x7fbec415bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 -- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbec406e6d0 msgr2=0x7fbec41f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbec406e6d0 0x7fbec41f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 -- 172.21.15.1:0/2757343489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec415e320 con 0x7fbec40cf180 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.563+0000 7fbec98d8640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.564+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4041070 con 0x7fbec40cf180 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.564+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb403b620 con 0x7fbec40cf180 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.564+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec415e600 con 0x7fbec40cf180 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.564+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4009510 con 0x7fbec40cf180 2024-05-09T06:34:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.564+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec415ebc0 con 0x7fbec40cf180 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.565+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec40752b0 con 0x7fbec40cf180 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.566+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeb4005d10 con 0x7fbec40cf180 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.566+0000 7fbec1ffb640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 0x7fbeb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.566+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fbeb40ab630 con 0x7fbec40cf180 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.566+0000 7fbec90d7640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 0x7fbeb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.567+0000 7fbec90d7640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 0x7fbeb006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0016e0 tx=0x7fbebc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.571+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbeb404b030 con 0x7fbec40cf180 2024-05-09T06:34:15.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.754+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbec406cf50 con 0x7fbec40cf180 2024-05-09T06:34:15.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.755+0000 7fbec1ffb640 1 -- 172.21.15.1:0/2757343489 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16753 (secure 0 0 0) 0x7fbeb402f070 con 0x7fbec40cf180 2024-05-09T06:34:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:15.736 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:34:04.325165+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:15.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 msgr2=0x7fbeb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 0x7fbeb006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0016e0 tx=0x7fbebc009040 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 msgr2=0x7fbec415c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009e50 tx=0x7fbeb4004a40 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 shutdown_connections 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeb006be30 0x7fbeb006e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbec40cf180 0x7fbec415c120 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbec40cdab0 0x7fbec415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 --2- 172.21.15.1:0/2757343489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbec406e6d0 0x7fbec41f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 >> 172.21.15.1:0/2757343489 conn(0x7fbec40cc4b0 msgr2=0x7fbec40b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 shutdown_connections 2024-05-09T06:34:15.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.757+0000 7fbecb33b640 1 -- 172.21.15.1:0/2757343489 wait complete. 2024-05-09T06:34:15.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:34:15.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:34:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.875+0000 7f1f70942640 1 Processor -- start 2024-05-09T06:34:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.875+0000 7f1f70942640 1 -- start start 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f70942640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 0x7f1f6c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f70942640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 0x7f1f6c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f70942640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6c0b1340 con 0x7f1f6c0ce9d0 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f70942640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0cc260 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f70942640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c0d1020 con 0x7f1f6c15e270 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 0x7f1f6c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f61d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 0x7f1f6c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f6ad9d640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1f6c0ce9d0 0x7f1f6c06c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:35782/0 (socket says 172.21.15.1:35782) 2024-05-09T06:34:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.876+0000 7f1f6ad9d640 1 -- 172.21.15.1:0/2109535608 learned_addr learned my addr 172.21.15.1:0/2109535608 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f69d9b640 1 -- 172.21.15.1:0/2109535608 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672269749 0 0) 0x7f1f64001490 con 0x7f1f6c0ce9d0 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f69d9b640 1 -- 172.21.15.1:0/2109535608 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f58003520 con 0x7f1f6c0ce9d0 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f6a59c640 1 -- 172.21.15.1:0/2109535608 >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 msgr2=0x7f1f6c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f6a59c640 1 --2- 172.21.15.1:0/2109535608 >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 0x7f1f6c0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f6a59c640 1 -- 172.21.15.1:0/2109535608 >> v1:172.21.15.6:6789/0 conn(0x7f1f6c0ce9d0 legacy=0x7f1f6c06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f6a59c640 1 -- 172.21.15.1:0/2109535608 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0cc4b0 con 0x7f1f6c15e270 2024-05-09T06:34:15.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f61d9b640 1 --2- 172.21.15.1:0/2109535608 >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 0x7f1f6c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.877+0000 7f1f6a59c640 1 --2- 172.21.15.1:0/2109535608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 0x7f1f6c0cf180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097a0 tx=0x7f1f5c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4578c3a5e1078c32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f69d9b640 1 -- 172.21.15.1:0/2109535608 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037070 con 0x7f1f6c15e270 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f69d9b640 1 -- 172.21.15.1:0/2109535608 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c032440 con 0x7f1f6c15e270 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f69d9b640 1 -- 172.21.15.1:0/2109535608 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c032810 con 0x7f1f6c15e270 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 -- 172.21.15.1:0/2109535608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 msgr2=0x7f1f6c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 --2- 172.21.15.1:0/2109535608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 0x7f1f6c0cf180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097a0 tx=0x7f1f5c02a210 comp rx=0 tx=0).stop 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 -- 172.21.15.1:0/2109535608 shutdown_connections 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 --2- 172.21.15.1:0/2109535608 >> v2:172.21.15.6:3300/0 conn(0x7f1f6c0d08b0 0x7f1f6c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 --2- 172.21.15.1:0/2109535608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c15e270 0x7f1f6c0cf180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 -- 172.21.15.1:0/2109535608 >> 172.21.15.1:0/2109535608 conn(0x7f1f6c0d1c10 msgr2=0x7f1f6c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 -- 172.21.15.1:0/2109535608 shutdown_connections 2024-05-09T06:34:15.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.878+0000 7f1f70942640 1 -- 172.21.15.1:0/2109535608 wait complete. 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 Processor -- start 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 -- start start 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c0ce9d0 0x7f1f6c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c15e270 0x7f1f6c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c15e650 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0cca70 con 0x7f1f6c15e270 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.880+0000 7f1f70942640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c0d0cf0 con 0x7f1f6c0ce9d0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c0ce9d0 0x7f1f6c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42664/0 (socket says 172.21.15.1:42664) 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 -- 172.21.15.1:0/924825613 learned_addr learned my addr 172.21.15.1:0/924825613 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f6ad9d640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c15e270 0x7f1f6c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 -- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c0ce9d0 msgr2=0x7f1f6c0c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c0ce9d0 0x7f1f6c0c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 -- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c15e270 msgr2=0x7f1f6c146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c15e270 0x7f1f6c146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 -- 172.21.15.1:0/924825613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0c6ef0 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f61d9b640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400ebe0 tx=0x7f1f5400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54017070 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5400cc30 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54013470 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c0c71d0 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.881+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c0c7790 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.882+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c0752b0 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.883+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f5401a070 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.883+0000 7f1f637fe640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.884+0000 7f1f6a59c640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.884+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f1f5408b1a0 con 0x7f1f6c0d08b0 2024-05-09T06:34:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.884+0000 7f1f6a59c640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c034000 tx=0x7f1f5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:15.888+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f54059400 con 0x7f1f6c0d08b0 2024-05-09T06:34:16.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.069+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f6c0d29e0 con 0x7f1f6c0d08b0 2024-05-09T06:34:16.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.070+0000 7f1f637fe640 1 -- 172.21.15.1:0/924825613 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f1f5405d700 con 0x7f1f6c0d08b0 2024-05-09T06:34:16.051 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:34:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 msgr2=0x7f1f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c034000 tx=0x7f1f5c02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 msgr2=0x7f1f6c154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400ebe0 tx=0x7f1f5400c520 comp rx=0 tx=0).stop 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 shutdown_connections 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f4c06be30 0x7f1f4c06e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c15e270 0x7f1f6c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f6c0d08b0 0x7f1f6c154480 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 --2- 172.21.15.1:0/924825613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f6c0ce9d0 0x7f1f6c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 >> 172.21.15.1:0/924825613 conn(0x7f1f6c0d1c10 msgr2=0x7f1f6c0af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 shutdown_connections 2024-05-09T06:34:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.072+0000 7f1f70942640 1 -- 172.21.15.1:0/924825613 wait complete. 2024-05-09T06:34:16.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:34:16.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.190+0000 7f0962c6b640 1 Processor -- start 2024-05-09T06:34:16.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.190+0000 7f0962c6b640 1 -- start start 2024-05-09T06:34:16.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0962c6b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f095c0d06f0 0x7f095c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0962c6b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095c1629a0 con 0x7f095c0cf8e0 2024-05-09T06:34:16.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0962c6b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095c072050 con 0x7f095c0c41e0 2024-05-09T06:34:16.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0962c6b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f095c072330 con 0x7f095c0d06f0 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0960a07640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f095c0d06f0 0x7f095c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0961208640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f095c0cf8e0 0x7f095c162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59268/0 (socket says 172.21.15.1:59268) 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0961208640 1 -- 172.21.15.1:0/3155264465 learned_addr learned my addr 172.21.15.1:0/3155264465 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0960a07640 1 -- 172.21.15.1:0/3155264465 >> v1:172.21.15.6:6789/0 conn(0x7f095c0c41e0 legacy=0x7f095c161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0960a07640 1 -- 172.21.15.1:0/3155264465 >> v1:172.21.15.1:6789/0 conn(0x7f095c0cf8e0 legacy=0x7f095c162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.191+0000 7f0960a07640 1 -- 172.21.15.1:0/3155264465 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c072610 con 0x7f095c0d06f0 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.192+0000 7f0960a07640 1 --2- 172.21.15.1:0/3155264465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0d06f0 0x7f095c0cc6e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f09580097a0 tx=0x7f095802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c92254c4b75be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.192+0000 7f09537fe640 1 -- 172.21.15.1:0/3155264465 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958037070 con 0x7f095c0d06f0 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.192+0000 7f09537fe640 1 -- 172.21.15.1:0/3155264465 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0958032440 con 0x7f095c0d06f0 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.192+0000 7f09537fe640 1 -- 172.21.15.1:0/3155264465 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958032810 con 0x7f095c0d06f0 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 -- 172.21.15.1:0/3155264465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0d06f0 msgr2=0x7f095c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3155264465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0d06f0 0x7f095c0cc6e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f09580097a0 tx=0x7f095802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 -- 172.21.15.1:0/3155264465 shutdown_connections 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3155264465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0d06f0 0x7f095c0cc6e0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 -- 172.21.15.1:0/3155264465 >> 172.21.15.1:0/3155264465 conn(0x7f095c0d0b50 msgr2=0x7f095c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 -- 172.21.15.1:0/3155264465 shutdown_connections 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.193+0000 7f0962c6b640 1 -- 172.21.15.1:0/3155264465 wait complete. 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.194+0000 7f0962c6b640 1 Processor -- start 2024-05-09T06:34:16.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.194+0000 7f0962c6b640 1 -- start start 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f095c0cf8e0 0x7f095c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f095c0d06f0 0x7f095c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f095c06b1e0 con 0x7f095c0c41e0 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f095c071ce0 con 0x7f095c0cf8e0 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0962c6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f095c06b360 con 0x7f095c0d06f0 2024-05-09T06:34:16.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42686/0 (socket says 172.21.15.1:42686) 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 -- 172.21.15.1:0/3478065120 learned_addr learned my addr 172.21.15.1:0/3478065120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 -- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f095c0d06f0 msgr2=0x7f095c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f095c0d06f0 0x7f095c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 -- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f095c0cf8e0 msgr2=0x7f095c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f095c0cf8e0 0x7f095c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.195+0000 7f0960a07640 1 -- 172.21.15.1:0/3478065120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c0b4720 con 0x7f095c0c41e0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f0960a07640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0958035040 tx=0x7f09580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958037040 con 0x7f095c0c41e0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0958030770 con 0x7f095c0c41e0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c0b4bd0 con 0x7f095c0c41e0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958030b40 con 0x7f095c0c41e0 2024-05-09T06:34:16.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.196+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c0b5190 con 0x7f095c0c41e0 2024-05-09T06:34:16.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.197+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c0a2050 con 0x7f095c0c41e0 2024-05-09T06:34:16.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.197+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0958044020 con 0x7f095c0c41e0 2024-05-09T06:34:16.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.198+0000 7f09517fa640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 0x7f094806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.198+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f09580ab570 con 0x7f095c0c41e0 2024-05-09T06:34:16.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.198+0000 7f0953fff640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 0x7f094806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.198+0000 7f0953fff640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 0x7f094806e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f094c000f60 tx=0x7f094c0079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.202+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f095c0a2050 con 0x7f095c0c41e0 2024-05-09T06:34:16.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.385+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f095c0752b0 con 0x7f095c0c41e0 2024-05-09T06:34:16.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.385+0000 7f09517fa640 1 -- 172.21.15.1:0/3478065120 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f0958032e20 con 0x7f095c0c41e0 2024-05-09T06:34:16.366 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:34:16.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 msgr2=0x7f094806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 0x7f094806e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f094c000f60 tx=0x7f094c0079d0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 msgr2=0x7f095c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0958035040 tx=0x7f09580046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 shutdown_connections 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f094806be30 0x7f094806e2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f095c0d06f0 0x7f095c0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f095c0cf8e0 0x7f095c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 --2- 172.21.15.1:0/3478065120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f095c0c41e0 0x7f095c159110 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 >> 172.21.15.1:0/3478065120 conn(0x7f095c0d0b50 msgr2=0x7f095c0b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 shutdown_connections 2024-05-09T06:34:16.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.387+0000 7f0962c6b640 1 -- 172.21.15.1:0/3478065120 wait complete. 2024-05-09T06:34:16.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:34:16.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.505+0000 7f2721b1b640 1 Processor -- start 2024-05-09T06:34:16.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.505+0000 7f2721b1b640 1 -- start start 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f2721b1b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f271c15c4c0 0x7f271c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f2721b1b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f271c15cfb0 0x7f271c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f2721b1b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f271c06cf40 con 0x7f271c0b4de0 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f2721b1b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f271c156980 con 0x7f271c15c4c0 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f2721b1b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f271c06d870 con 0x7f271c15cfb0 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f271b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f271c15c4c0 0x7f271c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f271b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f271c15c4c0 0x7f271c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42700/0 (socket says 172.21.15.1:42700) 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f271affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f271c0b4de0 0x7f271c15c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59284/0 (socket says 172.21.15.1:59284) 2024-05-09T06:34:16.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f271b7fe640 1 -- 172.21.15.1:0/2840871063 learned_addr learned my addr 172.21.15.1:0/2840871063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.506+0000 7f271bfff640 1 --2- 172.21.15.1:0/2840871063 >> v2:172.21.15.6:3300/0 conn(0x7f271c15cfb0 0x7f271c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271b7fe640 1 -- 172.21.15.1:0/2840871063 >> v2:172.21.15.6:3300/0 conn(0x7f271c15cfb0 msgr2=0x7f271c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2840871063 >> v2:172.21.15.6:3300/0 conn(0x7f271c15cfb0 0x7f271c070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271b7fe640 1 -- 172.21.15.1:0/2840871063 >> v1:172.21.15.1:6789/0 conn(0x7f271c0b4de0 legacy=0x7f271c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271a7fc640 1 -- 172.21.15.1:0/2840871063 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1288205361 0 0) 0x7f271c06cf40 con 0x7f271c0b4de0 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271b7fe640 1 -- 172.21.15.1:0/2840871063 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c06d220 con 0x7f271c15c4c0 2024-05-09T06:34:16.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2840871063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15c4c0 0x7f271c06dd00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f271c15c450 tx=0x7f270c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9575c6c87aa996 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271a7fc640 1 -- 172.21.15.1:0/2840871063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c037070 con 0x7f271c15c4c0 2024-05-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271a7fc640 1 -- 172.21.15.1:0/2840871063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c032440 con 0x7f271c15c4c0 2024-05-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.507+0000 7f271a7fc640 1 -- 172.21.15.1:0/2840871063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c032810 con 0x7f271c15c4c0 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 -- 172.21.15.1:0/2840871063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15c4c0 msgr2=0x7f271c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2840871063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15c4c0 0x7f271c06dd00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f271c15c450 tx=0x7f270c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 -- 172.21.15.1:0/2840871063 shutdown_connections 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2840871063 >> v2:172.21.15.6:3300/0 conn(0x7f271c15cfb0 0x7f271c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2840871063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15c4c0 0x7f271c06dd00 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 -- 172.21.15.1:0/2840871063 >> 172.21.15.1:0/2840871063 conn(0x7f271c0cc4d0 msgr2=0x7f271c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 -- 172.21.15.1:0/2840871063 shutdown_connections 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.508+0000 7f2721b1b640 1 -- 172.21.15.1:0/2840871063 wait complete. 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.509+0000 7f2721b1b640 1 Processor -- start 2024-05-09T06:34:16.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.509+0000 7f2721b1b640 1 -- start start 2024-05-09T06:34:16.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 0x7f271c0aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 0x7f271c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f271c0b2dc0 con 0x7f271c15cfb0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f271c0b2c40 con 0x7f271c0b4de0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f2721b1b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f271c0b5260 con 0x7f271c15c4c0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f271bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f271bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42716/0 (socket says 172.21.15.1:42716) 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.510+0000 7f271bfff640 1 -- 172.21.15.1:0/2875544186 learned_addr learned my addr 172.21.15.1:0/2875544186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 0x7f271c0aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271affd640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 0x7f271c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 -- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 msgr2=0x7f271c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 0x7f271c0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 -- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 msgr2=0x7f271c0aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 0x7f271c0aae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 -- 172.21.15.1:0/2875544186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c0a9dd0 con 0x7f271c15cfb0 2024-05-09T06:34:16.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271affd640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 0x7f271c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 0x7f271c0aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f271bfff640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2714007e60 tx=0x7f271400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714014420 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2714019070 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c0aa0b0 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714014820 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.511+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c0aa670 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.512+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c0752b0 con 0x7f271c15cfb0 2024-05-09T06:34:16.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.513+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27140149c0 con 0x7f271c15cfb0 2024-05-09T06:34:16.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.513+0000 7f2703fff640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 0x7f270806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.513+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f271408b840 con 0x7f271c15cfb0 2024-05-09T06:34:16.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.513+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 0x7f270806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.514+0000 7f271b7fe640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 0x7f270806e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f270c034000 tx=0x7f270c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.518+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f271405a060 con 0x7f271c15cfb0 2024-05-09T06:34:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.701+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f271c15e440 con 0x7f271c15cfb0 2024-05-09T06:34:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.702+0000 7f2703fff640 1 -- 172.21.15.1:0/2875544186 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f2714059960 con 0x7f271c15cfb0 2024-05-09T06:34:16.683 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 msgr2=0x7f270806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 0x7f270806e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f270c034000 tx=0x7f270c02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 msgr2=0x7f271c0a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2714007e60 tx=0x7f271400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 shutdown_connections 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f270806be60 0x7f270806e320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f271c15cfb0 0x7f271c0a9480 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f271c15c4c0 0x7f271c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 --2- 172.21.15.1:0/2875544186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c0b4de0 0x7f271c0aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 >> 172.21.15.1:0/2875544186 conn(0x7f271c0cc4d0 msgr2=0x7f271c15e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 shutdown_connections 2024-05-09T06:34:16.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.704+0000 7f2721b1b640 1 -- 172.21.15.1:0/2875544186 wait complete. 2024-05-09T06:34:16.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:34:16.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.820+0000 7f6bd1676640 1 Processor -- start 2024-05-09T06:34:16.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.820+0000 7f6bd1676640 1 -- start start 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bcc0b0080 0x7f6bcc071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 0x7f6bcc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc15fd70 0x7f6bcc15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc0cbfe0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc0d04a0 con 0x7f6bcc0cbb60 2024-05-09T06:34:16.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bd1676640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc0722c0 con 0x7f6bcc15fd70 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bcc0b0080 0x7f6bcc071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6bcc0b0080 0x7f6bcc071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42732/0 (socket says 172.21.15.1:42732) 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 -- 172.21.15.1:0/2288842950 learned_addr learned my addr 172.21.15.1:0/2288842950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/2288842950 >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 0x7f6bcc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcb7fe640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc15fd70 0x7f6bcc15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 -- 172.21.15.1:0/2288842950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc15fd70 msgr2=0x7f6bcc15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc15fd70 0x7f6bcc15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 -- 172.21.15.1:0/2288842950 >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 msgr2=0x7f6bcc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/2288842950 >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 0x7f6bcc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bcaffd640 1 -- 172.21.15.1:0/2288842950 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0725a0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.821+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/2288842950 >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 0x7f6bcc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.822+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc071ce0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc074b60 tx=0x7f6bbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829afb860f84983f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.822+0000 7f6bca7fc640 1 -- 172.21.15.1:0/2288842950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc037070 con 0x7f6bcc0b0080 2024-05-09T06:34:16.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.822+0000 7f6bca7fc640 1 -- 172.21.15.1:0/2288842950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc032440 con 0x7f6bcc0b0080 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.822+0000 7f6bca7fc640 1 -- 172.21.15.1:0/2288842950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc032810 con 0x7f6bcc0b0080 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 -- 172.21.15.1:0/2288842950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 msgr2=0x7f6bcc071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc071ce0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc074b60 tx=0x7f6bbc02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 -- 172.21.15.1:0/2288842950 shutdown_connections 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc15fd70 0x7f6bcc15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 --2- 172.21.15.1:0/2288842950 >> v2:172.21.15.6:3300/0 conn(0x7f6bcc0cbb60 0x7f6bcc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 --2- 172.21.15.1:0/2288842950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc071ce0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 -- 172.21.15.1:0/2288842950 >> 172.21.15.1:0/2288842950 conn(0x7f6bcc0d0b50 msgr2=0x7f6bcc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 -- 172.21.15.1:0/2288842950 shutdown_connections 2024-05-09T06:34:16.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.823+0000 7f6bd1676640 1 -- 172.21.15.1:0/2288842950 wait complete. 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.824+0000 7f6bd1676640 1 Processor -- start 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.824+0000 7f6bd1676640 1 -- start start 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 0x7f6bcc0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bcc15fd70 0x7f6bcc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc15e7c0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc072b60 con 0x7f6bcc15fd70 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bd1676640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc15f220 con 0x7f6bcc0cbb60 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42734/0 (socket says 172.21.15.1:42734) 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 -- 172.21.15.1:0/809433370 learned_addr learned my addr 172.21.15.1:0/809433370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcb7fe640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bcc15fd70 0x7f6bcc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 0x7f6bcc0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 -- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 msgr2=0x7f6bcc0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 0x7f6bcc0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 -- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bcc15fd70 msgr2=0x7f6bcc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bcc15fd70 0x7f6bcc0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 -- 172.21.15.1:0/809433370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0c7c20 con 0x7f6bcc0b0080 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 0x7f6bcc0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.825+0000 7f6bcaffd640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc02a220 tx=0x7f6bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.826+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc037040 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.826+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc042070 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.826+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc0c7f00 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.826+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc030bb0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.826+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc0c84c0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.827+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0752b0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.827+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bbc047460 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.828+0000 7f6bc3fff640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.828+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f6bbc0abbd0 con 0x7f6bcc0b0080 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.828+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.828+0000 7f6bc27fc640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 0x7f6bac06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b65f0 tx=0x7f6bb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:16.832+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6bbc07a360 con 0x7f6bcc0b0080 2024-05-09T06:34:17.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.029+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6bcc158f30 con 0x7f6bcc0b0080 2024-05-09T06:34:17.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.030+0000 7f6bc3fff640 1 -- 172.21.15.1:0/809433370 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f6bbc079c60 con 0x7f6bcc0b0080 2024-05-09T06:34:17.011 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 msgr2=0x7f6bac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 0x7f6bac06e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b65f0 tx=0x7f6bb4008040 comp rx=0 tx=0).stop 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 msgr2=0x7f6bcc1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc02a220 tx=0x7f6bbc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 shutdown_connections 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6bac06be30 0x7f6bac06e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bcc15fd70 0x7f6bcc0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6bcc0cbb60 0x7f6bcc0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 --2- 172.21.15.1:0/809433370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bcc0b0080 0x7f6bcc1589f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 >> 172.21.15.1:0/809433370 conn(0x7f6bcc0d0b50 msgr2=0x7f6bcc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 shutdown_connections 2024-05-09T06:34:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.032+0000 7f6bd1676640 1 -- 172.21.15.1:0/809433370 wait complete. 2024-05-09T06:34:17.024 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:34:17.024 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:34:17.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:34:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.203+0000 7f903d114640 1 Processor -- start 2024-05-09T06:34:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 -- start start 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 0x7f903806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90380b3300 0x7f90380b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90380ce0e0 con 0x7f90380cf180 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f90380ce3c0 con 0x7f90380b3300 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.204+0000 7f903d114640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f90380ce6a0 con 0x7f903806e6d0 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903759e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90380b3300 0x7f90380b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f90380cf180 0x7f90380cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59294/0 (socket says 172.21.15.1:59294) 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903659c640 1 -- 172.21.15.1:0/3554824188 learned_addr learned my addr 172.21.15.1:0/3554824188 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3554824188 >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 0x7f903806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903759e640 1 -- 172.21.15.1:0/3554824188 >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 msgr2=0x7f903806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903759e640 1 --2- 172.21.15.1:0/3554824188 >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 0x7f903806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903759e640 1 -- 172.21.15.1:0/3554824188 >> v1:172.21.15.1:6789/0 conn(0x7f90380cf180 legacy=0x7f90380cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f903759e640 1 -- 172.21.15.1:0/3554824188 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90380ce980 con 0x7f90380b3300 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f9035d9b640 1 -- 172.21.15.1:0/3554824188 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263869959 0 0) 0x7f90380ce0e0 con 0x7f90380cf180 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.205+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3554824188 >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 0x7f903806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903759e640 1 --2- 172.21.15.1:0/3554824188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90380b3710 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9038074650 tx=0x7f903002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6153913609f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f9035d9b640 1 -- 172.21.15.1:0/3554824188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030037070 con 0x7f90380b3300 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f9035d9b640 1 -- 172.21.15.1:0/3554824188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030032440 con 0x7f90380b3300 2024-05-09T06:34:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f9035d9b640 1 -- 172.21.15.1:0/3554824188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030032810 con 0x7f90380b3300 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 -- 172.21.15.1:0/3554824188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 msgr2=0x7f90380b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 --2- 172.21.15.1:0/3554824188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90380b3710 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9038074650 tx=0x7f903002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 -- 172.21.15.1:0/3554824188 shutdown_connections 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 --2- 172.21.15.1:0/3554824188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90380b3710 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 --2- 172.21.15.1:0/3554824188 >> v2:172.21.15.6:3300/0 conn(0x7f903806e6d0 0x7f903806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 -- 172.21.15.1:0/3554824188 >> 172.21.15.1:0/3554824188 conn(0x7f90380cc4b0 msgr2=0x7f90380cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 -- 172.21.15.1:0/3554824188 shutdown_connections 2024-05-09T06:34:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.206+0000 7f903d114640 1 -- 172.21.15.1:0/3554824188 wait complete. 2024-05-09T06:34:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.208+0000 7f903d114640 1 Processor -- start 2024-05-09T06:34:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.208+0000 7f903d114640 1 -- start start 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 0x7f903815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90380cf180 0x7f90380ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9038070ed0 con 0x7f90380b3300 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f903806eff0 con 0x7f903806e6d0 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90381447c0 con 0x7f90380cf180 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42754/0 (socket says 172.21.15.1:42754) 2024-05-09T06:34:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 -- 172.21.15.1:0/3979659796 learned_addr learned my addr 172.21.15.1:0/3979659796 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 0x7f903815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 -- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90380cf180 msgr2=0x7f90380ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903759e640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90380cf180 0x7f90380ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90380cf180 0x7f90380ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 -- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 msgr2=0x7f903815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 0x7f903815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 -- 172.21.15.1:0/3979659796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903815c410 con 0x7f90380b3300 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 0x7f903815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903659c640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f902000c4b0 tx=0x7f902000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020018070 con 0x7f90380b3300 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9020007500 con 0x7f90380b3300 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.209+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903815c6b0 con 0x7f90380b3300 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.210+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020004a20 con 0x7f90380b3300 2024-05-09T06:34:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.210+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903815cc70 con 0x7f90380b3300 2024-05-09T06:34:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.211+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90380752b0 con 0x7f90380b3300 2024-05-09T06:34:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.211+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f902001b070 con 0x7f90380b3300 2024-05-09T06:34:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.212+0000 7f902f7fe640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 0x7f901806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.212+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f902008a370 con 0x7f90380b3300 2024-05-09T06:34:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.212+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 0x7f901806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.212+0000 7f9036d9d640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 0x7f901806e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f90380a9f30 tx=0x7f902800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.216+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9020058c50 con 0x7f90380b3300 2024-05-09T06:34:17.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.404+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f903806cf50 con 0x7f90380b3300 2024-05-09T06:34:17.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.405+0000 7f902f7fe640 1 -- 172.21.15.1:0/3979659796 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16753 (secure 0 0 0) 0x7f9020058550 con 0x7f90380b3300 2024-05-09T06:34:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:17.386 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:34:04.325165+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:17.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 msgr2=0x7f901806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 0x7f901806e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f90380a9f30 tx=0x7f902800b040 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 msgr2=0x7f90381f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f902000c4b0 tx=0x7f902000c970 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 shutdown_connections 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f901806be30 0x7f901806e2f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90380cf180 0x7f90380ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90380b3300 0x7f90381f1e30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 --2- 172.21.15.1:0/3979659796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f903806e6d0 0x7f903815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 >> 172.21.15.1:0/3979659796 conn(0x7f90380cc4b0 msgr2=0x7f90380b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 shutdown_connections 2024-05-09T06:34:17.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.407+0000 7f903d114640 1 -- 172.21.15.1:0/3979659796 wait complete. 2024-05-09T06:34:17.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:34:17.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:34:17.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.525+0000 7f3cc587d640 1 Processor -- start 2024-05-09T06:34:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.525+0000 7f3cc587d640 1 -- start start 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cc587d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc00cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cc587d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3cc00d08b0 0x7f3cc00ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cc587d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc00b1340 con 0x7f3cc015e270 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cc587d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc00cc260 con 0x7f3cc00d08b0 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cc587d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc00d0d60 con 0x7f3cc00ceba0 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbeffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3cc015e270 0x7f3cc00ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:59304/0 (socket says 172.21.15.1:59304) 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbeffd640 1 -- 172.21.15.1:0/1160387732 learned_addr learned my addr 172.21.15.1:0/1160387732 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1160387732 >> v2:172.21.15.6:3300/0 conn(0x7f3cc00d08b0 0x7f3cc00ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc00cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260034542 0 0) 0x7f3cb0001490 con 0x7f3cc015e270 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.526+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8003520 con 0x7f3cc015e270 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 msgr2=0x7f3cc00cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3349821407 0 0) 0x7f3ca8003520 con 0x7f3cc015e270 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc00cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1160387732 >> v1:172.21.15.1:6789/0 conn(0x7f3cc015e270 legacy=0x7f3cc00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1160387732 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc00d1020 con 0x7f3cc00d08b0 2024-05-09T06:34:17.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.527+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc00cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:17.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc00ccf90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b6291015abc4ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8037070 con 0x7f3cc00d08b0 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb802ac30 con 0x7f3cc00d08b0 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cbdffb640 1 -- 172.21.15.1:0/1160387732 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0001bc0 con 0x7f3cc00d08b0 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 -- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 msgr2=0x7f3cc00ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc00ccf90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 -- 172.21.15.1:0/1160387732 shutdown_connections 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc00ccf90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 --2- 172.21.15.1:0/1160387732 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc00cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 -- 172.21.15.1:0/1160387732 >> 172.21.15.1:0/1160387732 conn(0x7f3cc00d1c10 msgr2=0x7f3cc00d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 -- 172.21.15.1:0/1160387732 shutdown_connections 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.528+0000 7f3cc587d640 1 -- 172.21.15.1:0/1160387732 wait complete. 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.529+0000 7f3cc587d640 1 Processor -- start 2024-05-09T06:34:17.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 -- start start 2024-05-09T06:34:17.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc015fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc0151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00cca70 con 0x7f3cc015e270 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00cc790 con 0x7f3cc00d08b0 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.530+0000 7f3cc587d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc015e650 con 0x7f3cc00ceba0 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42762/0 (socket says 172.21.15.1:42762) 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/614512509 learned_addr learned my addr 172.21.15.1:0/614512509 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc0151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc015fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 msgr2=0x7f3cc015fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc015fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 msgr2=0x7f3cc0151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc0151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/614512509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc015e9c0 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8041030 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc015eca0 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb803d420 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8037040 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.531+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc015f260 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.532+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc00752b0 con 0x7f3cc015e270 2024-05-09T06:34:17.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.533+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cb8044070 con 0x7f3cc015e270 2024-05-09T06:34:17.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.533+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 0x7f3cac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.533+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3cb80ab400 con 0x7f3cc015e270 2024-05-09T06:34:17.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.533+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 0x7f3cac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.534+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 0x7f3cac06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005c00 tx=0x7f3cb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.537+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cb8079670 con 0x7f3cc015e270 2024-05-09T06:34:17.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.718+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cc00d29e0 con 0x7f3cc015e270 2024-05-09T06:34:17.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.718+0000 7f3ca7fff640 1 -- 172.21.15.1:0/614512509 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f3cb807d970 con 0x7f3cc015e270 2024-05-09T06:34:17.700 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:34:17.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 msgr2=0x7f3cac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 0x7f3cac06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005c00 tx=0x7f3cb0009040 comp rx=0 tx=0).stop 2024-05-09T06:34:17.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 msgr2=0x7f3cc00cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb80046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 shutdown_connections 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be30 0x7f3cac06e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015e270 0x7f3cc00cdef0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cc00d08b0 0x7f3cc0151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 --2- 172.21.15.1:0/614512509 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc00ceba0 0x7f3cc015fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 >> 172.21.15.1:0/614512509 conn(0x7f3cc00d1c10 msgr2=0x7f3cc00aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 shutdown_connections 2024-05-09T06:34:17.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.721+0000 7f3cc587d640 1 -- 172.21.15.1:0/614512509 wait complete. 2024-05-09T06:34:17.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:34:17.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.839+0000 7faa763c8640 1 Processor -- start 2024-05-09T06:34:17.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.839+0000 7faa763c8640 1 -- start start 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faa7015c4c0 0x7faa7006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa7015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa700b5260 0x7faa7015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7faa70156980 con 0x7faa700b5260 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7faa7006d870 con 0x7faa7015c4c0 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa763c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7faa7015d320 con 0x7faa700b4de0 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa700b5260 0x7faa7015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa700b5260 0x7faa7015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35590/0 (socket says 172.21.15.1:35590) 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 -- 172.21.15.1:0/2539450533 learned_addr learned my addr 172.21.15.1:0/2539450533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa6ffff640 1 --2- 172.21.15.1:0/2539450533 >> v2:172.21.15.6:3300/0 conn(0x7faa7015c4c0 0x7faa7006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa6f7fe640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa7015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 -- 172.21.15.1:0/2539450533 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 msgr2=0x7faa7015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa7015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 -- 172.21.15.1:0/2539450533 >> v2:172.21.15.6:3300/0 conn(0x7faa7015c4c0 msgr2=0x7faa7006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 --2- 172.21.15.1:0/2539450533 >> v2:172.21.15.6:3300/0 conn(0x7faa7015c4c0 0x7faa7006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa74965640 1 -- 172.21.15.1:0/2539450533 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7006cf40 con 0x7faa700b5260 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.840+0000 7faa6f7fe640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa7015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.841+0000 7faa74965640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa700b5260 0x7faa7015cde0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7faa7004fb80 tx=0x7faa6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ed3ae471b95e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.841+0000 7faa6effd640 1 -- 172.21.15.1:0/2539450533 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68036070 con 0x7faa700b5260 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.841+0000 7faa6effd640 1 -- 172.21.15.1:0/2539450533 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa68031600 con 0x7faa700b5260 2024-05-09T06:34:17.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.841+0000 7faa6effd640 1 -- 172.21.15.1:0/2539450533 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680319d0 con 0x7faa700b5260 2024-05-09T06:34:17.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 -- 172.21.15.1:0/2539450533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa700b5260 msgr2=0x7faa7015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa700b5260 0x7faa7015cde0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7faa7004fb80 tx=0x7faa6802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 -- 172.21.15.1:0/2539450533 shutdown_connections 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa700b5260 0x7faa7015cde0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 --2- 172.21.15.1:0/2539450533 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa7015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 --2- 172.21.15.1:0/2539450533 >> v2:172.21.15.6:3300/0 conn(0x7faa7015c4c0 0x7faa7006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 -- 172.21.15.1:0/2539450533 >> 172.21.15.1:0/2539450533 conn(0x7faa700cc4d0 msgr2=0x7faa700cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 -- 172.21.15.1:0/2539450533 shutdown_connections 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.842+0000 7faa763c8640 1 -- 172.21.15.1:0/2539450533 wait complete. 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.843+0000 7faa763c8640 1 Processor -- start 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.843+0000 7faa763c8640 1 -- start start 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa700a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa700b5260 0x7faa700a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa700b2c40 con 0x7faa7015c4c0 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faa7006d500 con 0x7faa700b5260 2024-05-09T06:34:17.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa763c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7faa700b2dc0 con 0x7faa700b4de0 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35600/0 (socket says 172.21.15.1:35600) 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 -- 172.21.15.1:0/1925383604 learned_addr learned my addr 172.21.15.1:0/1925383604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa6ffff640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa700a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 -- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 msgr2=0x7faa700a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa700a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 -- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa700b5260 msgr2=0x7faa700a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa700b5260 0x7faa700a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 -- 172.21.15.1:0/1925383604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa70159440 con 0x7faa7015c4c0 2024-05-09T06:34:17.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa74965640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7faa6802ac90 tx=0x7faa68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.844+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6803f020 con 0x7faa7015c4c0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.845+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa68037040 con 0x7faa7015c4c0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.845+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa70159720 con 0x7faa7015c4c0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.845+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68036070 con 0x7faa7015c4c0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.845+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa701f5bf0 con 0x7faa7015c4c0 2024-05-09T06:34:17.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.846+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa70073da0 con 0x7faa7015c4c0 2024-05-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.846+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa68042020 con 0x7faa7015c4c0 2024-05-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.846+0000 7faa6cff9640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 0x7faa5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.847+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7faa680ab760 con 0x7faa7015c4c0 2024-05-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.847+0000 7faa6ffff640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 0x7faa5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.847+0000 7faa6ffff640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 0x7faa5c06e320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faa700b18f0 tx=0x7faa60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:17.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:17.851+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa70073da0 con 0x7faa7015c4c0 2024-05-09T06:34:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.032+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa700752b0 con 0x7faa7015c4c0 2024-05-09T06:34:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.032+0000 7faa6cff9640 1 -- 172.21.15.1:0/1925383604 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7faa68049020 con 0x7faa7015c4c0 2024-05-09T06:34:18.013 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:34:18.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 msgr2=0x7faa5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 0x7faa5c06e320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7faa700b18f0 tx=0x7faa60008040 comp rx=0 tx=0).stop 2024-05-09T06:34:18.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 msgr2=0x7faa701f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7faa6802ac90 tx=0x7faa68004a40 comp rx=0 tx=0).stop 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 shutdown_connections 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7faa5c06be60 0x7faa5c06e320 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7015c4c0 0x7faa701f56b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa700b5260 0x7faa700a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 --2- 172.21.15.1:0/1925383604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faa700b4de0 0x7faa700a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 >> 172.21.15.1:0/1925383604 conn(0x7faa700cc4d0 msgr2=0x7faa700af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 shutdown_connections 2024-05-09T06:34:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.034+0000 7faa763c8640 1 -- 172.21.15.1:0/1925383604 wait complete. 2024-05-09T06:34:18.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:34:18.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.152+0000 7f3f4ba10640 1 Processor -- start 2024-05-09T06:34:18.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.152+0000 7f3f4ba10640 1 -- start start 2024-05-09T06:34:18.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f4ba10640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3f440ae9c0 0x7f3f4406f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f4ba10640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4406c420 con 0x7f3f4415e300 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f4ba10640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4406c700 con 0x7f3f4406fde0 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f4ba10640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4406f3f0 con 0x7f3f440ae9c0 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f497ac640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3f4415e300 0x7f3f440d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37430/0 (socket says 172.21.15.1:37430) 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.153+0000 7f3f497ac640 1 -- 172.21.15.1:0/2124088180 learned_addr learned my addr 172.21.15.1:0/2124088180 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786557834 0 0) 0x7f3f4406c420 con 0x7f3f4415e300 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f48fab640 1 --2- 172.21.15.1:0/2124088180 >> v2:172.21.15.6:3300/0 conn(0x7f3f440ae9c0 0x7f3f4406f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c003520 con 0x7f3f4415e300 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3181961440 0 0) 0x7f3f2c003520 con 0x7f3f4415e300 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 >> v2:172.21.15.6:3300/0 conn(0x7f3f440ae9c0 msgr2=0x7f3f4406f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 --2- 172.21.15.1:0/2124088180 >> v2:172.21.15.6:3300/0 conn(0x7f3f440ae9c0 0x7f3f4406f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 >> v1:172.21.15.6:6789/0 conn(0x7f3f4406fde0 legacy=0x7f3f4406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4406c9e0 con 0x7f3f4415e300 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3f34001670 con 0x7f3f4415e300 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 >> v1:172.21.15.1:6789/0 conn(0x7f3f4415e300 legacy=0x7f3f440d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0039b0 0x7f3f2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 0x7f3f2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c00c180 con 0x7f3f2c0052e0 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c00c330 con 0x7f3f2c004800 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c00c530 con 0x7f3f2c0039b0 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 0x7f3f2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f497ac640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0039b0 0x7f3f2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f48fab640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 -- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0039b0 msgr2=0x7f3f2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0039b0 0x7f3f2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 -- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f2c004800 msgr2=0x7f3f2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.154+0000 7f3f49fad640 1 -- 172.21.15.1:0/2124088180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c003520 con 0x7f3f2c0052e0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.155+0000 7f3f49fad640 1 -- 172.21.15.1:0/2124088180 reap_dead start 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.155+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 0x7f3f2c00b9c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c001e20 tx=0x7f3f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5288a76fc93011a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.155+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c001a70 con 0x7f3f2c0052e0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.155+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c02ae40 con 0x7f3f2c0052e0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.155+0000 7f3f43fff640 1 -- 172.21.15.1:0/2124088180 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c003bb0 con 0x7f3f2c0052e0 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 msgr2=0x7f3f2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 0x7f3f2c00b9c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c001e20 tx=0x7f3f2c005830 comp rx=0 tx=0).stop 2024-05-09T06:34:18.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2124088180 shutdown_connections 2024-05-09T06:34:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2124088180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f2c0052e0 0x7f3f2c00b9c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2124088180 >> 172.21.15.1:0/2124088180 conn(0x7f3f440d1c50 msgr2=0x7f3f440d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2124088180 shutdown_connections 2024-05-09T06:34:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.156+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2124088180 wait complete. 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.157+0000 7f3f4ba10640 1 Processor -- start 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.157+0000 7f3f4ba10640 1 -- start start 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0052e0 0x7f3f441f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f441f4170 0x7f3f440b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4406cfa0 con 0x7f3f440b59b0 2024-05-09T06:34:18.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44070210 con 0x7f3f441f4170 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f4ba10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4406f8a0 con 0x7f3f2c0052e0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35610/0 (socket says 172.21.15.1:35610) 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 -- 172.21.15.1:0/2380165848 learned_addr learned my addr 172.21.15.1:0/2380165848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f497ac640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0052e0 0x7f3f441f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f48fab640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f441f4170 0x7f3f440b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 -- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0052e0 msgr2=0x7f3f441f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0052e0 0x7f3f441f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 -- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f441f4170 msgr2=0x7f3f440b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f441f4170 0x7f3f440b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.158+0000 7f3f49fad640 1 -- 172.21.15.1:0/2380165848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f440b5f30 con 0x7f3f440b59b0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f49fad640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c005040 tx=0x7f3f3c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007db0 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c041070 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c001a70 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f440b6210 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.159+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f440b67d0 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.160+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f440752b0 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.160+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f3c032660 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.161+0000 7f3f41ffb640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 0x7f3f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.161+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3f3c0aadd0 con 0x7f3f440b59b0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.161+0000 7f3f497ac640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 0x7f3f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.161+0000 7f3f497ac640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 0x7f3f3006e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3f34001cd0 tx=0x7f3f34001b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.165+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f3c039070 con 0x7f3f440b59b0 2024-05-09T06:34:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.348+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f4406f680 con 0x7f3f440b59b0 2024-05-09T06:34:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.348+0000 7f3f41ffb640 1 -- 172.21.15.1:0/2380165848 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f3f3c0796a0 con 0x7f3f440b59b0 2024-05-09T06:34:18.329 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:34:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 msgr2=0x7f3f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 0x7f3f3006e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3f34001cd0 tx=0x7f3f34001b60 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 msgr2=0x7f3f44159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c005040 tx=0x7f3f3c0014a0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 shutdown_connections 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f3006be30 0x7f3f3006e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f440b59b0 0x7f3f44159e40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f441f4170 0x7f3f440b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 --2- 172.21.15.1:0/2380165848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f2c0052e0 0x7f3f441f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 >> 172.21.15.1:0/2380165848 conn(0x7f3f440d1c50 msgr2=0x7f3f44158490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 shutdown_connections 2024-05-09T06:34:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.350+0000 7f3f4ba10640 1 -- 172.21.15.1:0/2380165848 wait complete. 2024-05-09T06:34:18.342 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:34:18.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.465+0000 7ff92cee7640 1 Processor -- start 2024-05-09T06:34:18.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.465+0000 7ff92cee7640 1 -- start start 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff92cee7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff9280b2a60 0x7ff9280b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff92cee7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9280b4f70 0x7ff928070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff92cee7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92806eb40 con 0x7ff9281454b0 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff92cee7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff92806ee20 con 0x7ff9280b4f70 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff92cee7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff92806d8e0 con 0x7ff9280b2a60 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff926d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9280b4f70 0x7ff928070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff926d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9280b4f70 0x7ff928070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35618/0 (socket says 172.21.15.1:35618) 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.466+0000 7ff926d9d640 1 -- 172.21.15.1:0/1933968587 learned_addr learned my addr 172.21.15.1:0/1933968587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff925d9b640 1 --2- 172.21.15.1:0/1933968587 >> v2:172.21.15.6:3300/0 conn(0x7ff9280b2a60 0x7ff9280b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff926d9d640 1 -- 172.21.15.1:0/1933968587 >> v2:172.21.15.6:3300/0 conn(0x7ff9280b2a60 msgr2=0x7ff9280b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff926d9d640 1 --2- 172.21.15.1:0/1933968587 >> v2:172.21.15.6:3300/0 conn(0x7ff9280b2a60 0x7ff9280b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff926d9d640 1 -- 172.21.15.1:0/1933968587 >> v1:172.21.15.6:6789/0 conn(0x7ff9281454b0 legacy=0x7ff9280b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff926d9d640 1 -- 172.21.15.1:0/1933968587 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92806dbc0 con 0x7ff9280b4f70 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff926d9d640 1 --2- 172.21.15.1:0/1933968587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9280b4f70 0x7ff928070c10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff92804f830 tx=0x7ff91802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60aa11a0f86ae07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.467+0000 7ff92559a640 1 -- 172.21.15.1:0/1933968587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff9280b4f70 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92559a640 1 -- 172.21.15.1:0/1933968587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032440 con 0x7ff9280b4f70 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92559a640 1 -- 172.21.15.1:0/1933968587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032810 con 0x7ff9280b4f70 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 -- 172.21.15.1:0/1933968587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9280b4f70 msgr2=0x7ff928070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 --2- 172.21.15.1:0/1933968587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9280b4f70 0x7ff928070c10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff92804f830 tx=0x7ff91802a230 comp rx=0 tx=0).stop 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 -- 172.21.15.1:0/1933968587 shutdown_connections 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 --2- 172.21.15.1:0/1933968587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9280b4f70 0x7ff928070c10 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 --2- 172.21.15.1:0/1933968587 >> v2:172.21.15.6:3300/0 conn(0x7ff9280b2a60 0x7ff9280b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 -- 172.21.15.1:0/1933968587 >> 172.21.15.1:0/1933968587 conn(0x7ff9280cc4b0 msgr2=0x7ff9280cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 -- 172.21.15.1:0/1933968587 shutdown_connections 2024-05-09T06:34:18.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.468+0000 7ff92cee7640 1 -- 172.21.15.1:0/1933968587 wait complete. 2024-05-09T06:34:18.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.469+0000 7ff92cee7640 1 Processor -- start 2024-05-09T06:34:18.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.469+0000 7ff92cee7640 1 -- start start 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 0x7ff9280cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9280b4f70 0x7ff92813f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff92806e260 con 0x7ff9281454b0 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff92806e0e0 con 0x7ff9280b4f70 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92cee7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff928071150 con 0x7ff9280b2a60 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff926d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff926d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35630/0 (socket says 172.21.15.1:35630) 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff926d9d640 1 -- 172.21.15.1:0/2112156706 learned_addr learned my addr 172.21.15.1:0/2112156706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff925d9b640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9280b4f70 0x7ff92813f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.470+0000 7ff92659c640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 0x7ff9280cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 -- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 msgr2=0x7ff9280cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 0x7ff9280cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 -- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9280b4f70 msgr2=0x7ff92813f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9280b4f70 0x7ff92813f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 -- 172.21.15.1:0/2112156706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9280ce340 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff92659c640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 0x7ff9280cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff926d9d640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff91802a270 tx=0x7ff9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037040 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918042070 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9280ce620 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918030c90 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.471+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9280cebe0 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.472+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9280752b0 con 0x7ff9281454b0 2024-05-09T06:34:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.472+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff918047460 con 0x7ff9281454b0 2024-05-09T06:34:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.473+0000 7ff91effd640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 0x7ff90806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.473+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff9180abc00 con 0x7ff9281454b0 2024-05-09T06:34:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.473+0000 7ff92659c640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 0x7ff90806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.474+0000 7ff92659c640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 0x7ff90806e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff920007eb0 tx=0x7ff9200073c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.478+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9180b2080 con 0x7ff9281454b0 2024-05-09T06:34:18.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:34:18.481 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:34:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.675+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9280c5050 con 0x7ff9281454b0 2024-05-09T06:34:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.675+0000 7ff91effd640 1 -- 172.21.15.1:0/2112156706 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7ff918079e60 con 0x7ff9281454b0 2024-05-09T06:34:18.658 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:34:18.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 msgr2=0x7ff90806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 0x7ff90806e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff920007eb0 tx=0x7ff9200073c0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 msgr2=0x7ff9280c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff91802a270 tx=0x7ff9180046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 shutdown_connections 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff90806be30 0x7ff90806e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9281454b0 0x7ff9280c76e0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9280b4f70 0x7ff92813f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 --2- 172.21.15.1:0/2112156706 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9280b2a60 0x7ff9280cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 >> 172.21.15.1:0/2112156706 conn(0x7ff9280cc4b0 msgr2=0x7ff9280af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.677+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 shutdown_connections 2024-05-09T06:34:18.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.678+0000 7ff92cee7640 1 -- 172.21.15.1:0/2112156706 wait complete. 2024-05-09T06:34:18.669 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:34:18.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:18.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:18.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[68342]: fuse finished with error 0 and tester_r 0 2024-05-09T06:34:18.695 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:34:18.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:34:18.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:18.790 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:34:18.806 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:18.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.970+0000 7f2dd241c640 1 Processor -- start 2024-05-09T06:34:18.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.970+0000 7f2dd241c640 1 -- start start 2024-05-09T06:34:18.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dd241c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2dcc161950 0x7f2dcc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dd241c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2dcc0d4940 0x7f2dcc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dd241c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dcc0b33c0 con 0x7f2dcc0d0f80 2024-05-09T06:34:18.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dd241c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2dcc0d4dd0 con 0x7f2dcc161950 2024-05-09T06:34:18.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dd241c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2dcc0b3660 con 0x7f2dcc0d4940 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2dcc161950 0x7f2dcc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2dcc161950 0x7f2dcc0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35636/0 (socket says 172.21.15.1:35636) 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 -- 172.21.15.1:0/299166148 learned_addr learned my addr 172.21.15.1:0/299166148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dc3fff640 1 --2- 172.21.15.1:0/299166148 >> v2:172.21.15.6:3300/0 conn(0x7f2dcc0d4940 0x7f2dcc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 -- 172.21.15.1:0/299166148 >> v2:172.21.15.6:3300/0 conn(0x7f2dcc0d4940 msgr2=0x7f2dcc0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 --2- 172.21.15.1:0/299166148 >> v2:172.21.15.6:3300/0 conn(0x7f2dcc0d4940 0x7f2dcc0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 -- 172.21.15.1:0/299166148 >> v1:172.21.15.6:6789/0 conn(0x7f2dcc0d0f80 legacy=0x7f2dcc0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.971+0000 7f2dcbfff640 1 -- 172.21.15.1:0/299166148 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc0b3940 con 0x7f2dcc161950 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.972+0000 7f2dcbfff640 1 --2- 172.21.15.1:0/299166148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc0d2400 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06e3d0 tx=0x7f2dbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8821957e8b9062 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.972+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/299166148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037070 con 0x7f2dcc161950 2024-05-09T06:34:18.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.972+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/299166148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc032440 con 0x7f2dcc161950 2024-05-09T06:34:18.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.972+0000 7f2dcb7fe640 1 -- 172.21.15.1:0/299166148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc032810 con 0x7f2dcc161950 2024-05-09T06:34:18.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 -- 172.21.15.1:0/299166148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 msgr2=0x7f2dcc0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 --2- 172.21.15.1:0/299166148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc0d2400 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06e3d0 tx=0x7f2dbc02a270 comp rx=0 tx=0).stop 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 -- 172.21.15.1:0/299166148 shutdown_connections 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 --2- 172.21.15.1:0/299166148 >> v2:172.21.15.6:3300/0 conn(0x7f2dcc0d4940 0x7f2dcc0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 --2- 172.21.15.1:0/299166148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc0d2400 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 -- 172.21.15.1:0/299166148 >> 172.21.15.1:0/299166148 conn(0x7f2dcc0d3e20 msgr2=0x7f2dcc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 -- 172.21.15.1:0/299166148 shutdown_connections 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.973+0000 7f2dd241c640 1 -- 172.21.15.1:0/299166148 wait complete. 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.974+0000 7f2dd241c640 1 Processor -- start 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.974+0000 7f2dd241c640 1 -- start start 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2dcc0d0f80 0x7f2dcc15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 0x7f2dcc0b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc0b52d0 con 0x7f2dcc161950 2024-05-09T06:34:18.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc0b5150 con 0x7f2dcc15da40 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd241c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2dcc0b5450 con 0x7f2dcc0d0f80 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35648/0 (socket says 172.21.15.1:35648) 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 -- 172.21.15.1:0/3922417128 learned_addr learned my addr 172.21.15.1:0/3922417128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd09b9640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 0x7f2dcc0b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dcbfff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2dcc0d0f80 0x7f2dcc15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 -- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2dcc0d0f80 msgr2=0x7f2dcc15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2dcc0d0f80 0x7f2dcc15fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 -- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 msgr2=0x7f2dcc0b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 0x7f2dcc0b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 -- 172.21.15.1:0/3922417128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc160420 con 0x7f2dcc161950 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dd09b9640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 0x7f2dcc0b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc3fff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e9b0 tx=0x7f2db400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.975+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017070 con 0x7f2dcc161950 2024-05-09T06:34:18.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.976+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db400c9a0 con 0x7f2dcc161950 2024-05-09T06:34:18.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.976+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc0b6bb0 con 0x7f2dcc161950 2024-05-09T06:34:18.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.976+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db400cd90 con 0x7f2dcc161950 2024-05-09T06:34:18.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.976+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc160110 con 0x7f2dcc161950 2024-05-09T06:34:18.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.977+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc0752b0 con 0x7f2dcc161950 2024-05-09T06:34:18.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.977+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2db401a070 con 0x7f2dcc161950 2024-05-09T06:34:18.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.978+0000 7f2dc97fa640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 0x7f2dac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:18.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.978+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f2db408b240 con 0x7f2dcc161950 2024-05-09T06:34:18.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.978+0000 7f2dcbfff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 0x7f2dac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:18.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.978+0000 7f2dcbfff640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 0x7f2dac06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc034000 tx=0x7f2dbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:18.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:18.982+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2db40594a0 con 0x7f2dcc161950 2024-05-09T06:34:19.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.204+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dcc0d21e0 con 0x7f2dcc161950 2024-05-09T06:34:19.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.204+0000 7f2dc97fa640 1 -- 172.21.15.1:0/3922417128 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5168 (secure 0 0 0) 0x7f2db405d7a0 con 0x7f2dcc161950 2024-05-09T06:34:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:19.186 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7899,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:05.704044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7911},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7911":{"gid":7911,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6835/1974102470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1974102470},{"type":"v1","addr":"172.21.15.6:6835","nonce":1974102470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7911,"qdb_cluster":[7911]},"id":6}]} 2024-05-09T06:34:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 61 2024-05-09T06:34:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 msgr2=0x7f2dac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 0x7f2dac06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc034000 tx=0x7f2dbc02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 msgr2=0x7f2dcc06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e9b0 tx=0x7f2db400ee70 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 shutdown_connections 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dac06be30 0x7f2dac06e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dcc15da40 0x7f2dcc0b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2dcc161950 0x7f2dcc06c4e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 --2- 172.21.15.1:0/3922417128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2dcc0d0f80 0x7f2dcc15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 >> 172.21.15.1:0/3922417128 conn(0x7f2dcc0d3e20 msgr2=0x7f2dcc160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 shutdown_connections 2024-05-09T06:34:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.206+0000 7f2dd241c640 1 -- 172.21.15.1:0/3922417128 wait complete. 2024-05-09T06:34:19.198 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-09T06:34:19.352 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2024-05-09T06:34:19.352 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:34:19.353 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:34:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.477+0000 7f5c1abb7640 1 Processor -- start 2024-05-09T06:34:19.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.477+0000 7f5c1abb7640 1 -- start start 2024-05-09T06:34:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c1abb7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c140b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c1abb7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c140b4f70 0x7f5c14070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c1abb7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1406eb40 con 0x7f5c141454b0 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c1abb7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5c1406ee20 con 0x7f5c140b4f70 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c1abb7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1406d8e0 con 0x7f5c140b2a60 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c18953640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5c141454b0 0x7f5c140b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37436/0 (socket says 172.21.15.1:37436) 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c18953640 1 -- 172.21.15.1:0/3863347189 learned_addr learned my addr 172.21.15.1:0/3863347189 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c13fff640 1 --2- 172.21.15.1:0/3863347189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c140b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c19154640 1 --2- 172.21.15.1:0/3863347189 >> v2:172.21.15.6:3300/0 conn(0x7f5c140b4f70 0x7f5c14070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c137fe640 1 -- 172.21.15.1:0/3863347189 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553246386 0 0) 0x7f5c1406eb40 con 0x7f5c141454b0 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c137fe640 1 -- 172.21.15.1:0/3863347189 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c04003520 con 0x7f5c141454b0 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c13fff640 1 -- 172.21.15.1:0/3863347189 >> v2:172.21.15.6:3300/0 conn(0x7f5c140b4f70 msgr2=0x7f5c14070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c13fff640 1 --2- 172.21.15.1:0/3863347189 >> v2:172.21.15.6:3300/0 conn(0x7f5c140b4f70 0x7f5c14070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c13fff640 1 -- 172.21.15.1:0/3863347189 >> v1:172.21.15.1:6789/0 conn(0x7f5c141454b0 legacy=0x7f5c140b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.478+0000 7f5c13fff640 1 -- 172.21.15.1:0/3863347189 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1406dbc0 con 0x7f5c140b2a60 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.479+0000 7f5c13fff640 1 --2- 172.21.15.1:0/3863347189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c140b4a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097a0 tx=0x7f5c0002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e6b83310f41047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.479+0000 7f5c137fe640 1 -- 172.21.15.1:0/3863347189 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00037070 con 0x7f5c140b2a60 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.479+0000 7f5c137fe640 1 -- 172.21.15.1:0/3863347189 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c00032440 con 0x7f5c140b2a60 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.479+0000 7f5c137fe640 1 -- 172.21.15.1:0/3863347189 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000327f0 con 0x7f5c140b2a60 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 -- 172.21.15.1:0/3863347189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 msgr2=0x7f5c140b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/3863347189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c140b4a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097a0 tx=0x7f5c0002a180 comp rx=0 tx=0).stop 2024-05-09T06:34:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 -- 172.21.15.1:0/3863347189 shutdown_connections 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/3863347189 >> v2:172.21.15.6:3300/0 conn(0x7f5c140b4f70 0x7f5c14070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/3863347189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c140b4a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 -- 172.21.15.1:0/3863347189 >> 172.21.15.1:0/3863347189 conn(0x7f5c140cc4b0 msgr2=0x7f5c140cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 -- 172.21.15.1:0/3863347189 shutdown_connections 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.480+0000 7f5c1abb7640 1 -- 172.21.15.1:0/3863347189 wait complete. 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.481+0000 7f5c1abb7640 1 Processor -- start 2024-05-09T06:34:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.481+0000 7f5c1abb7640 1 -- start start 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c1415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140b4f70 0x7f5c141e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406e260 con 0x7f5c141454b0 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406e0e0 con 0x7f5c140b4f70 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c1abb7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5c14071150 con 0x7f5c140b2a60 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c19154640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c19154640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35660/0 (socket says 172.21.15.1:35660) 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c19154640 1 -- 172.21.15.1:0/346906873 learned_addr learned my addr 172.21.15.1:0/346906873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c1415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c13fff640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140b4f70 0x7f5c141e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 -- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140b4f70 msgr2=0x7f5c141e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140b4f70 0x7f5c141e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 -- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 msgr2=0x7f5c141e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.482+0000 7f5c18953640 1 -- 172.21.15.1:0/346906873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c140ad230 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c19154640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c18953640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c1415b710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800c390 tx=0x7f5c08001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08017070 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c08005ce0 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080072f0 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c140ad510 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.483+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141e1490 con 0x7f5c140b2a60 2024-05-09T06:34:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.484+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c141e4630 con 0x7f5c140b2a60 2024-05-09T06:34:19.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.484+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c0801a070 con 0x7f5c140b2a60 2024-05-09T06:34:19.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.485+0000 7f5c117fa640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 0x7f5bf806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.485+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f5c0808a260 con 0x7f5c140b2a60 2024-05-09T06:34:19.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.485+0000 7f5c13fff640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 0x7f5bf806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.485+0000 7f5c13fff640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 0x7f5bf806e320 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c0002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.490+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c08007e20 con 0x7f5c140b2a60 2024-05-09T06:34:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.711+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c140752b0 con 0x7f5c140b2a60 2024-05-09T06:34:19.693 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:34:19.712+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:34:19.699 INFO:tasks.ceph.mds.d.smithi006.stderr: -3913> 2024-05-09T06:34:08.299+0000 7fb75e56f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:34:19.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.721+0000 7f5c117fa640 1 -- 172.21.15.1:0/346906873 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f5c08058390 con 0x7f5c140b2a60 2024-05-09T06:34:19.702 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:34:19.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 msgr2=0x7f5bf806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 0x7f5bf806e320 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c0002e000 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 msgr2=0x7f5c1415b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c1415b710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800c390 tx=0x7f5c08001e70 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 shutdown_connections 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bf806be60 0x7f5bf806e320 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c141454b0 0x7f5c141e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c140b4f70 0x7f5c141e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 --2- 172.21.15.1:0/346906873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c140b2a60 0x7f5c1415b710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 >> 172.21.15.1:0/346906873 conn(0x7f5c140cc4b0 msgr2=0x7f5c140b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 shutdown_connections 2024-05-09T06:34:19.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.723+0000 7f5c1abb7640 1 -- 172.21.15.1:0/346906873 wait complete. 2024-05-09T06:34:19.715 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-09T06:34:19.729 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:34:19.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.778+0000 7f5439d1ef40 1 Processor -- start 2024-05-09T06:34:19.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.778+0000 7f5439d1ef40 1 -- start start 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149485c00 0x55f149536cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f149541850 0x55f14955def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f149486010 con 0x55f1494b4bc0 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f14942ec80 con 0x55f149485c00 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f14955ee90 con 0x55f149541850 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35690/0 (socket says 172.21.15.1:35690) 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 -- 172.21.15.1:0/1062043791 learned_addr learned my addr 172.21.15.1:0/1062043791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436d51640 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149485c00 0x55f149536cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5437552640 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f149541850 0x55f14955def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 -- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f149541850 msgr2=0x55f14955def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f149541850 0x55f14955def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 -- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149485c00 msgr2=0x55f149536cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149485c00 0x55f149536cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 -- 172.21.15.1:0/1062043791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1493b3cb0 con 0x55f1494b4bc0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.779+0000 7f5436550640 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f542c00a4f0 tx=0x7f542c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52df6840c6eec165 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f543554e640 1 -- 172.21.15.1:0/1062043791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c059040 con 0x55f1494b4bc0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f543554e640 1 -- 172.21.15.1:0/1062043791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c064070 con 0x55f1494b4bc0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f543554e640 1 -- 172.21.15.1:0/1062043791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c054bf0 con 0x55f1494b4bc0 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 -- 172.21.15.1:0/1062043791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 msgr2=0x55f149489190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f542c00a4f0 tx=0x7f542c04e000 comp rx=0 tx=0).stop 2024-05-09T06:34:19.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 -- 172.21.15.1:0/1062043791 shutdown_connections 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f149541850 0x55f14955def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1494b4bc0 0x55f149489190 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/1062043791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149485c00 0x55f149536cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 -- 172.21.15.1:0/1062043791 >> 172.21.15.1:0/1062043791 conn(0x55f1494b37c0 msgr2=0x55f14955cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 -- 172.21.15.1:0/1062043791 shutdown_connections 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.780+0000 7f5439d1ef40 1 -- 172.21.15.1:0/1062043791 wait complete. 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 Processor -- start 2024-05-09T06:34:19.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 -- start start 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 0x55f1495579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f1494b4bc0 0x55f1494cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149541850 0x55f149489190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f1494b6be0 con 0x55f149485c00 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f149489f00 con 0x55f149541850 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.781+0000 7f5439d1ef40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f1494b6ec0 con 0x55f1494b4bc0 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 0x55f1495579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436550640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f1494b4bc0 0x55f1494cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 0x55f1495579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35694/0 (socket says 172.21.15.1:35694) 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 -- 172.21.15.1:0/703808363 learned_addr learned my addr 172.21.15.1:0/703808363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5437552640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149541850 0x55f149489190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 -- 172.21.15.1:0/703808363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f1494b4bc0 msgr2=0x55f1494cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f1494b4bc0 0x55f1494cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 -- 172.21.15.1:0/703808363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149541850 msgr2=0x55f149489190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f149541850 0x55f149489190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 -- 172.21.15.1:0/703808363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f149557f30 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5436d51640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 0x55f1495579f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5428003f10 tx=0x7f54280018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f541ffff640 1 -- 172.21.15.1:0/703808363 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280123f0 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f541ffff640 1 -- 172.21.15.1:0/703808363 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428017070 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f1494cc180 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f541ffff640 1 -- 172.21.15.1:0/703808363 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280127d0 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.782+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1494cc740 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.784+0000 7f541ffff640 1 -- 172.21.15.1:0/703808363 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5428012970 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.784+0000 7f541ffff640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f542406a810 0x7f542406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.784+0000 7f541ffff640 1 -- 172.21.15.1:0/703808363 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f5428089af0 con 0x55f149485c00 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.784+0000 7f5436550640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f542406a810 0x7f542406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.785+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55f14955e440 0x55f149561510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.785+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e92) -- 0x55f1494cbd40 con 0x55f14955e440 2024-05-09T06:34:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.785+0000 7f5437552640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55f14955e440 0x55f149561510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.785+0000 7f5436550640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f542406a810 0x7f542406ccd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f542c045900 tx=0x7f542c051000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.785+0000 7f5437552640 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55f14955e440 0x55f149561510 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5437552640 1 -- 172.21.15.1:0/703808363 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v92'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5430009b80 con 0x55f14955e440 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55f14955e440 msgr2=0x55f149561510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55f14955e440 0x55f149561510 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f542406a810 msgr2=0x7f542406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f542406a810 0x7f542406ccd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f542c045900 tx=0x7f542c051000 comp rx=0 tx=0).stop 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 msgr2=0x55f1495579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 --2- 172.21.15.1:0/703808363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f149485c00 0x55f1495579f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5428003f10 tx=0x7f54280018a0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5437552640 1 -- 172.21.15.1:0/703808363 reap_dead start 2024-05-09T06:34:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 shutdown_connections 2024-05-09T06:34:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.788+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 >> 172.21.15.1:0/703808363 conn(0x55f1494b37c0 msgr2=0x55f1494b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.789+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 shutdown_connections 2024-05-09T06:34:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.789+0000 7f5439d1ef40 1 -- 172.21.15.1:0/703808363 wait complete. 2024-05-09T06:34:19.773 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-09T06:34:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.874+0000 7f8502f55280 1 -- start start 2024-05-09T06:34:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.874+0000 7f8502f55280 1 Processor -- start 2024-05-09T06:34:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 0x55c702e64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 0x55c702e64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4180 con 0x55c702df5c00 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4300 con 0x55c702df4c00 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c703bd4480 con 0x55c702df5800 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35702/0 (socket says 172.21.15.1:35702) 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 -- 172.21.15.1:0/3187301066 learned_addr learned my addr 172.21.15.1:0/3187301066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8500f00640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 0x55c702e64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501701640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 0x55c702e64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 -- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 msgr2=0x55c702e64b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 0x55c702e64b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 -- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 msgr2=0x55c702e64580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 0x55c702e64580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 -- 172.21.15.1:0/3187301066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c702e2a780 con 0x55c702df5c00 2024-05-09T06:34:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501701640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 0x55c702e64580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8500f00640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 0x55c702e64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8501f02640 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x55c703c11350 tx=0x55c703b8a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbd37b1d8355fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f84feefc640 1 -- 172.21.15.1:0/3187301066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1a680 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f84feefc640 1 -- 172.21.15.1:0/3187301066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c702e20fc0 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f84feefc640 1 -- 172.21.15.1:0/3187301066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1ab60 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.875+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c702e2a3c0 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c702e2b680 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 4 waiting for MDS map... 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f84feefc640 1 -- 172.21.15.1:0/3187301066 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x55c703ccc600 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f84feefc640 1 -- 172.21.15.1:0/3187301066 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55c702e4f180 con 0x55c702df5c00 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 4 Got MDS map 62 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 10 main: JournalTool::main 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 4 main: JournalTool: connecting to RADOS... 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 1 Processor -- start 2024-05-09T06:34:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.876+0000 7f8502f55280 1 -- start start 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703ccfc00 0x55c702e65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4600 con 0x55c703d2a000 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4780 con 0x55c703d2a400 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c703bd4900 con 0x55c703ccfc00 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35710/0 (socket says 172.21.15.1:35710) 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 -- 172.21.15.1:0/2320743072 learned_addr learned my addr 172.21.15.1:0/2320743072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8500f00640 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703ccfc00 0x55c702e65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8500f00640 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 -- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703ccfc00 msgr2=0x55c702e65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703ccfc00 0x55c702e65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 -- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 msgr2=0x55c702e66100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.877+0000 7f8501701640 1 -- 172.21.15.1:0/2320743072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c702e2b4a0 con 0x55c703d2a000 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8501701640 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55c703ce2690 tx=0x55c703ce4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b7b062fe5efcd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f84fcef8640 1 -- 172.21.15.1:0/2320743072 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1a680 con 0x55c703d2a000 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f84fcef8640 1 -- 172.21.15.1:0/2320743072 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c702e20a80 con 0x55c703d2a000 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f84fcef8640 1 -- 172.21.15.1:0/2320743072 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1ab60 con 0x55c703d2a000 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 -- 172.21.15.1:0/2320743072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 msgr2=0x55c702e65b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55c703ce2690 tx=0x55c703ce4c00 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 -- 172.21.15.1:0/2320743072 shutdown_connections 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2a000 0x55c702e65b80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 --2- 172.21.15.1:0/2320743072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703ccfc00 0x55c702e65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 -- 172.21.15.1:0/2320743072 >> 172.21.15.1:0/2320743072 conn(0x55c703ccf800 msgr2=0x55c702ee1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 -- 172.21.15.1:0/2320743072 shutdown_connections 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.878+0000 7f8502f55280 1 -- 172.21.15.1:0/2320743072 wait complete. 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 Processor -- start 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 -- start start 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703d2a800 0x55c702e66c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd5b00 con 0x55c703d2ac00 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd5980 con 0x55c703d2a400 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c703bd5800 con 0x55c703d2a800 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8500f00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8500f00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35712/0 (socket says 172.21.15.1:35712) 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.879+0000 7f8500f00640 1 -- 172.21.15.1:0/154568757 learned_addr learned my addr 172.21.15.1:0/154568757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8501701640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703d2a800 0x55c702e66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 -- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703d2a800 msgr2=0x55c702e66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703d2a800 0x55c702e66c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 -- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 msgr2=0x55c702e66680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 -- 172.21.15.1:0/154568757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c702e2a780 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8500f00640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x55c703ce3320 tx=0x55c703b88c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f84fb6f5640 1 -- 172.21.15.1:0/154568757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1a9c0 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f84fb6f5640 1 -- 172.21.15.1:0/154568757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c702e20e00 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c702e2a3c0 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f84fb6f5640 1 -- 172.21.15.1:0/154568757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1a680 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c702e2a000 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.880+0000 7f8502f55280 4 main: JournalTool: resolving pool 13 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.881+0000 7f84fb6f5640 1 -- 172.21.15.1:0/154568757 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55c703beec00 con 0x55c703d2ac00 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f84fb6f5640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 0x55c702e67700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f84fb6f5640 1 -- 172.21.15.1:0/154568757 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x55c703becc00 con 0x55c703d2ac00 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8502f55280 4 main: JournalTool: creating IoCtx.. 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8502f55280 4 main: Executing for rank 0 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8501701640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 0x55c702e67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8501701640 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 0x55c702e67700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55c703ce2690 tx=0x55c703b89ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8502f55280 1 -- start start 2024-05-09T06:34:19.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.882+0000 7f8502f55280 1 Processor -- start 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 0x55c702e66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703ccf800 0x55c702e65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c7031c6000 0x55c702e65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4600 con 0x55c703ccfc00 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c703bd4780 con 0x55c703ccf800 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8502f55280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c703bd4a80 con 0x55c7031c6000 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 0x55c702e66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 0x55c702e66100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35714/0 (socket says 172.21.15.1:35714) 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 learned_addr learned my addr 172.21.15.1:0/1409273672 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:19.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8500f00640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c7031c6000 0x55c702e65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703ccf800 0x55c702e65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c7031c6000 msgr2=0x55c702e65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c7031c6000 0x55c702e65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703ccf800 msgr2=0x55c702e65b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703ccf800 0x55c702e65b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c702e2a960 con 0x55c703ccfc00 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.883+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703ccf800 0x55c702e65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 0x55c702e66100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55c7031397a0 tx=0x55c703b88a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41a917d48245ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f84f86ef640 1 -- 172.21.15.1:0/1409273672 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c702e1a820 con 0x55c703ccfc00 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f84f86ef640 1 -- 172.21.15.1:0/1409273672 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c702e201c0 con 0x55c703ccfc00 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c702e2b860 con 0x55c703ccfc00 2024-05-09T06:34:19.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c702e2ba40 con 0x55c703ccfc00 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 4 waiting for MDS map... 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f84f86ef640 1 -- 172.21.15.1:0/1409273672 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c703d7bba0 con 0x55c703ccfc00 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f84f86ef640 1 -- 172.21.15.1:0/1409273672 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x55c7031ccc00 con 0x55c703ccfc00 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f84f86ef640 1 -- 172.21.15.1:0/1409273672 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55c702e4f500 con 0x55c703ccfc00 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 4 Got MDS map 62 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55c7031c6400 0x55c7031ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x55c7031c6800 con 0x55c7031c6400 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.884+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55c7031c6400 0x55c7031ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.885+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55c7031c6400 0x55c7031ee000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=f10901c348855664 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.887+0000 7f8501701640 1 -- 172.21.15.1:0/1409273672 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55c702ef6000 con 0x55c7031c6400 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.887+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55c7031c6c00 0x55c7031ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.887+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x55c7031c7000 con 0x55c7031c6c00 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.888+0000 7f8500f00640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55c7031c6c00 0x55c7031ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.888+0000 7f8500f00640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55c7031c6c00 0x55c7031ee580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=ff2e2225ded328df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55c702ef6500 con 0x55c7031c6c00 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f84f6eec640 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) -- 0x55c7031ec400 con 0x55c7031c6c00 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f84f6eec640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55c7031ec800 0x55c7031f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f84f6eec640 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) -- 0x55c7031ecc00 con 0x55c7031ec800 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55c702ef6500 con 0x55c7031c6c00 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.890+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55c7031ec800 0x55c7031f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.891+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55c7031ec800 0x55c7031f0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=9dde1a9dc9d4f926 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.877 INFO:teuthology.orchestra.run.smithi001.stdout:old journal was 4208696~2030 2024-05-09T06:34:19.878 INFO:teuthology.orchestra.run.smithi001.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-05-09T06:34:19.878 INFO:teuthology.orchestra.run.smithi001.stdout:writing journal head 2024-05-09T06:34:19.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.893+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6780 con 0x55c7031ec800 2024-05-09T06:34:19.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.893+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031c7400 con 0x55c7031c6c00 2024-05-09T06:34:19.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v92'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6500 con 0x55c7031c6c00 2024-05-09T06:34:19.879 INFO:teuthology.orchestra.run.smithi001.stdout:writing EResetJournal entry 2024-05-09T06:34:19.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031c7800 con 0x55c7031ec800 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55c7031c7c00 0x55c7031eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031ed000 con 0x55c7031c7c00 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031ed400 con 0x55c7031c6400 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55c7031ed800 0x55c7031ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031edc00 con 0x55c7031ed800 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c703202000 con 0x55c7031ec800 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55c703202400 0x55c7031ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c703202800 con 0x55c703202400 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8500f00640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55c7031ed800 0x55c7031ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.894+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55c7031c7c00 0x55c7031eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.895+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55c703202400 0x55c7031ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.895+0000 7f8500f00640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55c7031ed800 0x55c7031ef080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=4af9770ad4081c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.895+0000 7f8501701640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55c7031c7c00 0x55c7031eeb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=baa53110fbdeae8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.895+0000 7f8501f02640 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55c703202400 0x55c7031ef600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=9be4b19296fbf8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.895+0000 7f8501701640 1 -- 172.21.15.1:0/1409273672 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(8 200.00000004 [delete] v92'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6000 con 0x55c7031c6400 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.896+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(6 200.00000002 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6780 con 0x55c7031ec800 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.896+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(10 200.00000006 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6780 con 0x55c7031ec800 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.896+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(9 200.00000005 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6500 con 0x55c7031ed800 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.896+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(11 200.00000007 [delete] v92'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6780 con 0x55c703202400 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.898+0000 7f8501701640 1 -- 172.21.15.1:0/1409273672 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(7 200.00000003 [delete] v92'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6000 con 0x55c7031c7c00 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.898+0000 7f84f6eec640 1 -- 172.21.15.1:0/1409273672 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55c7031d5800 con 0x55c7031ec800 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8501f02640 1 -- 172.21.15.1:0/1409273672 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v92'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c702ef6780 con 0x55c7031ec800 2024-05-09T06:34:19.885 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-05-09T06:34:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55c7031c6c00 msgr2=0x55c7031ee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55c7031c6c00 0x55c7031ee580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55c7031ed800 msgr2=0x55c7031ef080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55c7031ed800 0x55c7031ef080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55c7031ec800 msgr2=0x55c7031f0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55c7031ec800 0x55c7031f0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55c7031c7c00 msgr2=0x55c7031eeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55c7031c7c00 0x55c7031eeb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55c7031c6400 msgr2=0x55c7031ee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55c7031c6400 0x55c7031ee000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55c703202400 msgr2=0x55c7031ef600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55c703202400 0x55c7031ef600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 msgr2=0x55c702e66100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/1409273672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703ccfc00 0x55c702e66100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55c7031397a0 tx=0x55c703b88a80 comp rx=0 tx=0).stop 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 reap_dead start 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8500f00640 1 -- 172.21.15.1:0/1409273672 reap_dead start 2024-05-09T06:34:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 shutdown_connections 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 >> 172.21.15.1:0/1409273672 conn(0x55c703d2a000 msgr2=0x55c702ee1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 shutdown_connections 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/1409273672 wait complete. 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 msgr2=0x55c702e67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 0x55c702e67700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55c703ce2690 tx=0x55c703b89ec0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 msgr2=0x55c702e67180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x55c703ce3320 tx=0x55c703b88c00 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 shutdown_connections 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55c703d2bc00 0x55c702e67700 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c703d2a400 0x55c702e66680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.899+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c703d2a800 0x55c702e66c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 --2- 172.21.15.1:0/154568757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c703d2ac00 0x55c702e67180 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 >> 172.21.15.1:0/154568757 conn(0x55c703d2b000 msgr2=0x55c702ee1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 shutdown_connections 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/154568757 wait complete. 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 msgr2=0x55c702e65080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x55c703c11350 tx=0x55c703b8a8a0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 shutdown_connections 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c702df5c00 0x55c702e65080 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c702df5800 0x55c702e64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 --2- 172.21.15.1:0/3187301066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c702df4c00 0x55c702e64580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 >> 172.21.15.1:0/3187301066 conn(0x55c702df5000 msgr2=0x55c702ee0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 shutdown_connections 2024-05-09T06:34:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:19.900+0000 7f8502f55280 1 -- 172.21.15.1:0/3187301066 wait complete. 2024-05-09T06:34:19.891 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.118820, result: old journal was 4208696~2030 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-09T06:34:19.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:34:20.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.055+0000 7f6e847d5640 1 Processor -- start 2024-05-09T06:34:20.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.055+0000 7f6e847d5640 1 -- start start 2024-05-09T06:34:20.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e847d5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e847d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c15c4a0 0x7f6e7c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e847d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7c15b420 con 0x7f6e7c158a50 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e847d5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7c158e30 con 0x7f6e7c071fd0 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e847d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7c072480 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e82571640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6e7c158a50 0x7f6e7c0718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37458/0 (socket says 172.21.15.1:37458) 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35718/0 (socket says 172.21.15.1:35718) 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e82571640 1 -- 172.21.15.1:0/751476739 learned_addr learned my addr 172.21.15.1:0/751476739 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e82d72640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c15c4a0 0x7f6e7c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 -- 172.21.15.1:0/751476739 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c15c4a0 msgr2=0x7f6e7c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c15c4a0 0x7f6e7c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 -- 172.21.15.1:0/751476739 >> v1:172.21.15.1:6789/0 conn(0x7f6e7c158a50 legacy=0x7f6e7c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e81d70640 1 -- 172.21.15.1:0/751476739 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c15b6e0 con 0x7f6e7c071fd0 2024-05-09T06:34:20.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.056+0000 7f6e8156f640 1 -- 172.21.15.1:0/751476739 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201702895 0 0) 0x7f6e7c15b420 con 0x7f6e7c158a50 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.057+0000 7f6e81d70640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097a0 tx=0x7f6e6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01fac07fe3348b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.057+0000 7f6e8156f640 1 -- 172.21.15.1:0/751476739 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c037070 con 0x7f6e7c071fd0 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.057+0000 7f6e8156f640 1 -- 172.21.15.1:0/751476739 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6c032440 con 0x7f6e7c071fd0 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.057+0000 7f6e8156f640 1 -- 172.21.15.1:0/751476739 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c032810 con 0x7f6e7c071fd0 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 -- 172.21.15.1:0/751476739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c071fd0 msgr2=0x7f6e7c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097a0 tx=0x7f6e6c02a180 comp rx=0 tx=0).stop 2024-05-09T06:34:20.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 -- 172.21.15.1:0/751476739 shutdown_connections 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c15c4a0 0x7f6e7c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 --2- 172.21.15.1:0/751476739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c071fd0 0x7f6e7c06e6d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 -- 172.21.15.1:0/751476739 >> 172.21.15.1:0/751476739 conn(0x7f6e7c0cc4d0 msgr2=0x7f6e7c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 -- 172.21.15.1:0/751476739 shutdown_connections 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.058+0000 7f6e847d5640 1 -- 172.21.15.1:0/751476739 wait complete. 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.059+0000 7f6e847d5640 1 Processor -- start 2024-05-09T06:34:20.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.059+0000 7f6e847d5640 1 -- start start 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c071fd0 0x7f6e7c143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7c158a50 0x7f6e7c167470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c15bf80 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c15bca0 con 0x7f6e7c158a50 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e847d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7c070ea0 con 0x7f6e7c071fd0 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82571640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c071fd0 0x7f6e7c143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35726/0 (socket says 172.21.15.1:35726) 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 -- 172.21.15.1:0/4113920224 learned_addr learned my addr 172.21.15.1:0/4113920224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e81d70640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7c158a50 0x7f6e7c167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 -- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c071fd0 msgr2=0x7f6e7c143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c071fd0 0x7f6e7c143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 -- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7c158a50 msgr2=0x7f6e7c167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7c158a50 0x7f6e7c167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 -- 172.21.15.1:0/4113920224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c0af770 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e82d72640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400efc0 tx=0x7f6e7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018020 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.060+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e74014480 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.061+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c0afa50 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.061+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009e60 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.061+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c0b0a70 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.061+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7c0752b0 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.062+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e74005d10 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.063+0000 7f6e72ffd640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 0x7f6e6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.063+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f6e7408bbc0 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.063+0000 7f6e82571640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 0x7f6e6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.063+0000 7f6e82571640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 0x7f6e6006e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000f60 tx=0x7f6e78007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.067+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e74022030 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.288+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6e7c158060 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.715+0000 7f6e72ffd640 1 -- 172.21.15.1:0/4113920224 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7f6e74010040 con 0x7f6e7c15c4a0 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 msgr2=0x7f6e6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 0x7f6e6006e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000f60 tx=0x7f6e78007d50 comp rx=0 tx=0).stop 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 msgr2=0x7f6e7c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400efc0 tx=0x7f6e7400c490 comp rx=0 tx=0).stop 2024-05-09T06:34:20.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 shutdown_connections 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e6006be30 0x7f6e6006e2f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7c15c4a0 0x7f6e7c0b0530 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7c158a50 0x7f6e7c167470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 --2- 172.21.15.1:0/4113920224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7c071fd0 0x7f6e7c143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 >> 172.21.15.1:0/4113920224 conn(0x7f6e7c0cc4d0 msgr2=0x7f6e7c06dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 shutdown_connections 2024-05-09T06:34:20.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.717+0000 7f6e847d5640 1 -- 172.21.15.1:0/4113920224 wait complete. 2024-05-09T06:34:20.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:20.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.840+0000 7fb34f076640 1 Processor -- start 2024-05-09T06:34:20.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.840+0000 7fb34f076640 1 -- start start 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.841+0000 7fb34f076640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.841+0000 7fb34f076640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb34815c680 0x7fb3480afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.841+0000 7fb34f076640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb348074a70 con 0x7fb3480b0170 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.841+0000 7fb34f076640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb34815fd30 con 0x7fb34815c680 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.841+0000 7fb34f076640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb34815cc40 con 0x7fb3481565e0 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb347fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb34815c680 0x7fb3480afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:40040/0 (socket says 172.21.15.1:40040) 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 -- 172.21.15.1:0/1622083463 learned_addr learned my addr 172.21.15.1:0/1622083463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 -- 172.21.15.1:0/1622083463 >> v2:172.21.15.6:3300/0 conn(0x7fb34815c680 msgr2=0x7fb3480afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 --2- 172.21.15.1:0/1622083463 >> v2:172.21.15.6:3300/0 conn(0x7fb34815c680 0x7fb3480afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 -- 172.21.15.1:0/1622083463 >> v1:172.21.15.6:6789/0 conn(0x7fb3480b0170 legacy=0x7fb3481601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.842+0000 7fb34ce12640 1 -- 172.21.15.1:0/1622083463 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34815cf20 con 0x7fb3481565e0 2024-05-09T06:34:20.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb34ce12640 1 --2- 172.21.15.1:0/1622083463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009870 tx=0x7fb33c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a9204fda6fc123 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb3477fe640 1 -- 172.21.15.1:0/1622083463 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c037070 con 0x7fb3481565e0 2024-05-09T06:34:20.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb3477fe640 1 -- 172.21.15.1:0/1622083463 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c032440 con 0x7fb3481565e0 2024-05-09T06:34:20.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb3477fe640 1 -- 172.21.15.1:0/1622083463 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c032810 con 0x7fb3481565e0 2024-05-09T06:34:20.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb34f076640 1 -- 172.21.15.1:0/1622083463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 msgr2=0x7fb348074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.843+0000 7fb34f076640 1 --2- 172.21.15.1:0/1622083463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009870 tx=0x7fb33c02a270 comp rx=0 tx=0).stop 2024-05-09T06:34:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 -- 172.21.15.1:0/1622083463 shutdown_connections 2024-05-09T06:34:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 --2- 172.21.15.1:0/1622083463 >> v2:172.21.15.6:3300/0 conn(0x7fb34815c680 0x7fb3480afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 --2- 172.21.15.1:0/1622083463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3481565e0 0x7fb348074530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 -- 172.21.15.1:0/1622083463 >> 172.21.15.1:0/1622083463 conn(0x7fb3480b1fb0 msgr2=0x7fb3480b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 -- 172.21.15.1:0/1622083463 shutdown_connections 2024-05-09T06:34:20.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.844+0000 7fb34f076640 1 -- 172.21.15.1:0/1622083463 wait complete. 2024-05-09T06:34:20.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.845+0000 7fb34f076640 1 Processor -- start 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.845+0000 7fb34f076640 1 -- start start 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3480b0170 0x7fb3481511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3481565e0 0x7fb3480aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3481569c0 con 0x7fb34815c680 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb34815d4e0 con 0x7fb3481565e0 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34f076640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb3481608d0 con 0x7fb3480b0170 2024-05-09T06:34:20.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35742/0 (socket says 172.21.15.1:35742) 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 -- 172.21.15.1:0/281958244 learned_addr learned my addr 172.21.15.1:0/281958244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34ce12640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3480b0170 0x7fb3481511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb347fff640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3481565e0 0x7fb3480aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 -- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3480b0170 msgr2=0x7fb3481511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3480b0170 0x7fb3481511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 -- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3481565e0 msgr2=0x7fb3480aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3481565e0 0x7fb3480aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 -- 172.21.15.1:0/281958244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3480af140 con 0x7fb34815c680 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.846+0000 7fb34d613640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c6b0 tx=0x7fb34000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.847+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340003b70 con 0x7fb34815c680 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.847+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb340003d30 con 0x7fb34815c680 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.847+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb348152230 con 0x7fb34815c680 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.847+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340005310 con 0x7fb34815c680 2024-05-09T06:34:20.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.847+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3481527f0 con 0x7fb34815c680 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.848+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3480752b0 con 0x7fb34815c680 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.848+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3400054b0 con 0x7fb34815c680 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.848+0000 7fb3457fa640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 0x7fb32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.849+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7fb34008aef0 con 0x7fb34815c680 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.849+0000 7fb34ce12640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 0x7fb32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.849+0000 7fb34ce12640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 0x7fb32c06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:20.853+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3400596c0 con 0x7fb34815c680 2024-05-09T06:34:21.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.075+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3480cda50 con 0x7fb34815c680 2024-05-09T06:34:21.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.075+0000 7fb3457fa640 1 -- 172.21.15.1:0/281958244 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5162 (secure 0 0 0) 0x7fb340058fc0 con 0x7fb34815c680 2024-05-09T06:34:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:21.056 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:20.718416+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-09T06:34:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 64 2024-05-09T06:34:21.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 msgr2=0x7fb32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:21.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 0x7fb32c06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:21.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 msgr2=0x7fb3480998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:21.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c6b0 tx=0x7fb34000cb70 comp rx=0 tx=0).stop 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 shutdown_connections 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb32c06be30 0x7fb32c06e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34815c680 0x7fb3480998e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3481565e0 0x7fb3480aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 --2- 172.21.15.1:0/281958244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3480b0170 0x7fb3481511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 >> 172.21.15.1:0/281958244 conn(0x7fb3480b1fb0 msgr2=0x7fb3481601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 shutdown_connections 2024-05-09T06:34:21.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:21.077+0000 7fb34f076640 1 -- 172.21.15.1:0/281958244 wait complete. 2024-05-09T06:34:21.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:34:20.718416+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6836/424906117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 424906117}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 424906117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:34:21.472 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7899:up:replay 2024-05-09T06:34:22.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:22.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.608+0000 7f1c6184c640 1 Processor -- start 2024-05-09T06:34:22.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.609+0000 7f1c6184c640 1 -- start start 2024-05-09T06:34:22.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.609+0000 7f1c6184c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:22.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.609+0000 7f1c6184c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c0b33c0 con 0x7f1c5c161950 2024-05-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.609+0000 7f1c6184c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c0b3660 con 0x7f1c5c0d4940 2024-05-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.609+0000 7f1c6184c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d4dd0 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1c5c161950 0x7f1c5c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37466/0 (socket says 172.21.15.1:37466) 2024-05-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5affd640 1 -- 172.21.15.1:0/591645833 learned_addr learned my addr 172.21.15.1:0/591645833 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/591645833 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182614684 0 0) 0x7f1c5c0b33c0 con 0x7f1c5c161950 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c44003520 con 0x7f1c5c161950 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732715114 0 0) 0x7f1c5c0b3660 con 0x7f1c5c0d4940 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c5c0b33c0 con 0x7f1c5c0d4940 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3152435262 0 0) 0x7f1c44003520 con 0x7f1c5c161950 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/591645833 >> v1:172.21.15.6:6789/0 conn(0x7f1c5c0d4940 legacy=0x7f1c5c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/591645833 >> v1:172.21.15.1:6789/0 conn(0x7f1c5c161950 legacy=0x7f1c5c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.610+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/591645833 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c0b3940 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.611+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/591645833 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c0b4850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c540097d0 tx=0x7f1c5402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd454182c68e1a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.611+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54037070 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.611+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5402ac30 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.611+0000 7f1c59ffb640 1 -- 172.21.15.1:0/591645833 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c001a10 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 -- 172.21.15.1:0/591645833 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 msgr2=0x7f1c5c0b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 --2- 172.21.15.1:0/591645833 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c0b4850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c540097d0 tx=0x7f1c5402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 -- 172.21.15.1:0/591645833 shutdown_connections 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 --2- 172.21.15.1:0/591645833 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c0b4850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 -- 172.21.15.1:0/591645833 >> 172.21.15.1:0/591645833 conn(0x7f1c5c0d3e20 msgr2=0x7f1c5c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 -- 172.21.15.1:0/591645833 shutdown_connections 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.612+0000 7f1c6184c640 1 -- 172.21.15.1:0/591645833 wait complete. 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.613+0000 7f1c6184c640 1 Processor -- start 2024-05-09T06:34:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.613+0000 7f1c6184c640 1 -- start start 2024-05-09T06:34:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c1f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c5c161950 0x7f1c5c09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d1100 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d0f80 con 0x7f1c5c161950 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c6184c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0d1c00 con 0x7f1c5c0d2b10 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35746/0 (socket says 172.21.15.1:35746) 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/958732555 learned_addr learned my addr 172.21.15.1:0/958732555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c5affd640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c1f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.614+0000 7f1c5a7fc640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c5c161950 0x7f1c5c09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 msgr2=0x7f1c5c1f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c1f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c5c161950 msgr2=0x7f1c5c09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c5c161950 0x7f1c5c09daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 -- 172.21.15.1:0/958732555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c09fa20 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1c54035040 tx=0x7f1c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54037040 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54004230 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c0cd660 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54006e10 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.615+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c09ff80 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.616+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c0752b0 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.617+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c54003610 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.617+0000 7f1c43fff640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.617+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f1c540abdd0 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.617+0000 7f1c5affd640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:22.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.618+0000 7f1c5affd640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c007d70 tx=0x7f1c4c007d00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.622+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c5c09fa20 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.842+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5c0d21e0 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.843+0000 7f1c43fff640 1 -- 172.21.15.1:0/958732555 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5162 (secure 0 0 0) 0x7f1c5407a080 con 0x7f1c5c09dfe0 2024-05-09T06:34:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:22.824 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:21.727748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-09T06:34:22.825 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 66 2024-05-09T06:34:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 msgr2=0x7f1c4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c007d70 tx=0x7f1c4c007d00 comp rx=0 tx=0).stop 2024-05-09T06:34:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 msgr2=0x7f1c5c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1c54035040 tx=0x7f1c540046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 shutdown_connections 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c09dfe0 0x7f1c5c0cc950 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c5c161950 0x7f1c5c09daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 --2- 172.21.15.1:0/958732555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c5c0d2b10 0x7f1c5c1f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 >> 172.21.15.1:0/958732555 conn(0x7f1c5c0d3e20 msgr2=0x7f1c5c0b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 shutdown_connections 2024-05-09T06:34:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:22.845+0000 7f1c6184c640 1 -- 172.21.15.1:0/958732555 wait complete. 2024-05-09T06:34:23.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:34:21.727748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.1:6836/424906117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 424906117}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 424906117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:34:23.240 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7899:up:rejoin 2024-05-09T06:34:24.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:24.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.377+0000 7f663f729640 1 Processor -- start 2024-05-09T06:34:24.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.377+0000 7f663f729640 1 -- start start 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663f729640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f663815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663f729640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6638099930 con 0x7f66380c6860 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663f729640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6638162930 con 0x7f6638099b30 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663f729640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f66380b1e20 con 0x7f66380b2aa0 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663d4c5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f66380c6860 0x7f66380c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37468/0 (socket says 172.21.15.1:37468) 2024-05-09T06:34:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663d4c5640 1 -- 172.21.15.1:0/1348422778 learned_addr learned my addr 172.21.15.1:0/1348422778 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f663dcc6640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f663815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235615589 0 0) 0x7f6634001490 con 0x7f66380c6860 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6624003520 con 0x7f66380c6860 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 808504246 0 0) 0x7f6624003520 con 0x7f66380c6860 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 msgr2=0x7f663815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f663815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 >> v1:172.21.15.6:6789/0 conn(0x7f6638099b30 legacy=0x7f663815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6638162c10 con 0x7f66380c6860 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.378+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6634001bc0 con 0x7f66380c6860 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 >> v1:172.21.15.1:6789/0 conn(0x7f66380c6860 legacy=0x7f66380c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66240039e0 0x7f6624003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6624004800 0x7f6624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f6624005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f662400baf0 con 0x7f6624004800 2024-05-09T06:34:24.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f662400bca0 con 0x7f66240039e0 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f662400bea0 con 0x7f66380b2aa0 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6624004800 0x7f6624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663d4c5640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66240039e0 0x7f6624003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663dcc6640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f6624005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 -- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 msgr2=0x7f6624005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380b2aa0 0x7f6624005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 -- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66240039e0 msgr2=0x7f6624003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f66240039e0 0x7f6624003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.379+0000 7f663ccc4640 1 -- 172.21.15.1:0/1348422778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624003520 con 0x7f6624004800 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f663ccc4640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6624004800 0x7f6624004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6628001750 tx=0x7f6624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5397bd734fd03a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628001370 con 0x7f6624004800 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66280052d0 con 0x7f6624004800 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f662ffff640 1 -- 172.21.15.1:0/1348422778 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662802abe0 con 0x7f6624004800 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f663f729640 1 -- 172.21.15.1:0/1348422778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6624004800 msgr2=0x7f6624004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f663f729640 1 --2- 172.21.15.1:0/1348422778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6624004800 0x7f6624004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6628001750 tx=0x7f6624005830 comp rx=0 tx=0).stop 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.380+0000 7f663dcc6640 1 -- 172.21.15.1:0/1348422778 reap_dead start 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.381+0000 7f663f729640 1 -- 172.21.15.1:0/1348422778 shutdown_connections 2024-05-09T06:34:24.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.381+0000 7f663f729640 1 -- 172.21.15.1:0/1348422778 >> 172.21.15.1:0/1348422778 conn(0x7f66380b1fb0 msgr2=0x7f66380b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.381+0000 7f663f729640 1 -- 172.21.15.1:0/1348422778 shutdown_connections 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.381+0000 7f663f729640 1 -- 172.21.15.1:0/1348422778 wait complete. 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.382+0000 7f663f729640 1 Processor -- start 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.382+0000 7f663f729640 1 -- start start 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6624004800 0x7f66380c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380ca990 0x7f66381469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f663815fa30 con 0x7f66380c7450 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f663815ee90 con 0x7f6624004800 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663f729640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f663815e4b0 con 0x7f66380ca990 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35770/0 (socket says 172.21.15.1:35770) 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 -- 172.21.15.1:0/2345438332 learned_addr learned my addr 172.21.15.1:0/2345438332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663d4c5640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6624004800 0x7f66380c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663ccc4640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380ca990 0x7f66381469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 -- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380ca990 msgr2=0x7f66381469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380ca990 0x7f66381469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 -- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6624004800 msgr2=0x7f66380c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6624004800 0x7f66380c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 -- 172.21.15.1:0/2345438332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66381473e0 con 0x7f66380c7450 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f663dcc6640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6624004780 tx=0x7f6630011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.383+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630004240 con 0x7f66380c7450 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.384+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663001b070 con 0x7f66380c7450 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.384+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66381476c0 con 0x7f66380c7450 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.384+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663000f7a0 con 0x7f66380c7450 2024-05-09T06:34:24.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.384+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6638147c80 con 0x7f66380c7450 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.385+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66380752b0 con 0x7f66380c7450 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.385+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6630012040 con 0x7f66380c7450 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.386+0000 7f662dffb640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 0x7f661c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.386+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f663008add0 con 0x7f66380c7450 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.386+0000 7f663d4c5640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 0x7f661c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:24.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.386+0000 7f663d4c5640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 0x7f661c06e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6634002230 tx=0x7f6634006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:24.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.391+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6630058ec0 con 0x7f66380c7450 2024-05-09T06:34:24.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.612+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f663806ce30 con 0x7f66380c7450 2024-05-09T06:34:24.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.613+0000 7f662dffb640 1 -- 172.21.15.1:0/2345438332 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5169 (secure 0 0 0) 0x7f663008c020 con 0x7f66380c7450 2024-05-09T06:34:24.594 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:24.594 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:23.731691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":9,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":6}]} 2024-05-09T06:34:24.594 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 67 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 msgr2=0x7f661c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 0x7f661c06e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6634002230 tx=0x7f6634006040 comp rx=0 tx=0).stop 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 msgr2=0x7f66380c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6624004780 tx=0x7f6630011cf0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 shutdown_connections 2024-05-09T06:34:24.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f661c06be60 0x7f661c06e320 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66380c7450 0x7f66380c7900 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f66380ca990 0x7f66381469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 --2- 172.21.15.1:0/2345438332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6624004800 0x7f66380c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 >> 172.21.15.1:0/2345438332 conn(0x7f66380b1fb0 msgr2=0x7f6638161ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 shutdown_connections 2024-05-09T06:34:24.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:24.615+0000 7f663f729640 1 -- 172.21.15.1:0/2345438332 wait complete. 2024-05-09T06:34:25.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:34:23.731691+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.1:6836/424906117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 424906117}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 424906117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7899, 'qdb_cluster': [7899]} 2024-05-09T06:34:25.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:34:25.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T06:34:25.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.137+0000 7f94dbab3640 1 Processor -- start 2024-05-09T06:34:25.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.137+0000 7f94dbab3640 1 -- start start 2024-05-09T06:34:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d406e6d0 0x7f94d406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d40cf180 0x7f94d40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94d40cdab0 0x7f94d415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f94d40ce0e0 con 0x7f94d40cf180 2024-05-09T06:34:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f94d40ce3c0 con 0x7f94d40cdab0 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94dbab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94d40ce6a0 con 0x7f94d406e6d0 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d40cf180 0x7f94d40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d40cf180 0x7f94d40cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35780/0 (socket says 172.21.15.1:35780) 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 -- 172.21.15.1:0/2694169324 learned_addr learned my addr 172.21.15.1:0/2694169324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d984f640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d406e6d0 0x7f94d406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94da050640 1 --2- 172.21.15.1:0/2694169324 >> v2:172.21.15.6:3300/0 conn(0x7f94d40cdab0 0x7f94d415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 -- 172.21.15.1:0/2694169324 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d406e6d0 msgr2=0x7f94d406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d406e6d0 0x7f94d406eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 -- 172.21.15.1:0/2694169324 >> v2:172.21.15.6:3300/0 conn(0x7f94d40cdab0 msgr2=0x7f94d415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 --2- 172.21.15.1:0/2694169324 >> v2:172.21.15.6:3300/0 conn(0x7f94d40cdab0 0x7f94d415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.138+0000 7f94d904e640 1 -- 172.21.15.1:0/2694169324 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d40ce980 con 0x7f94d40cf180 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.139+0000 7f94d904e640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cf180 0x7f94d40cd570 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f94d40b16f0 tx=0x7f94c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1107a4bfd5af05f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.139+0000 7f94d884d640 1 -- 172.21.15.1:0/2694169324 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4037070 con 0x7f94d40cf180 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.139+0000 7f94d884d640 1 -- 172.21.15.1:0/2694169324 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4032440 con 0x7f94d40cf180 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.139+0000 7f94d884d640 1 -- 172.21.15.1:0/2694169324 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4032810 con 0x7f94d40cf180 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 -- 172.21.15.1:0/2694169324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cf180 msgr2=0x7f94d40cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cf180 0x7f94d40cd570 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f94d40b16f0 tx=0x7f94c402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 -- 172.21.15.1:0/2694169324 shutdown_connections 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 --2- 172.21.15.1:0/2694169324 >> v2:172.21.15.6:3300/0 conn(0x7f94d40cdab0 0x7f94d415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cf180 0x7f94d40cd570 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 --2- 172.21.15.1:0/2694169324 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d406e6d0 0x7f94d406eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 -- 172.21.15.1:0/2694169324 >> 172.21.15.1:0/2694169324 conn(0x7f94d40cc4b0 msgr2=0x7f94d40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 -- 172.21.15.1:0/2694169324 shutdown_connections 2024-05-09T06:34:25.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.140+0000 7f94dbab3640 1 -- 172.21.15.1:0/2694169324 wait complete. 2024-05-09T06:34:25.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.141+0000 7f94dbab3640 1 Processor -- start 2024-05-09T06:34:25.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.141+0000 7f94dbab3640 1 -- start start 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 0x7f94d41f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d40cf180 0x7f94d41f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4070ed0 con 0x7f94d40cdab0 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94d406eff0 con 0x7f94d406e6d0 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94dbab3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94d41447c0 con 0x7f94d40cf180 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35788/0 (socket says 172.21.15.1:35788) 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 -- 172.21.15.1:0/1283731850 learned_addr learned my addr 172.21.15.1:0/1283731850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d984f640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 0x7f94d41f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94da050640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d40cf180 0x7f94d41f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 -- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d40cf180 msgr2=0x7f94d41f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d40cf180 0x7f94d41f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 -- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 msgr2=0x7f94d41f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 0x7f94d41f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 -- 172.21.15.1:0/1283731850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d40ae0f0 con 0x7f94d40cdab0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d984f640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 0x7f94d41f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.142+0000 7f94d904e640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f94c402a220 tx=0x7f94c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.143+0000 7f94ca7fc640 1 -- 172.21.15.1:0/1283731850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4041070 con 0x7f94d40cdab0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.143+0000 7f94ca7fc640 1 -- 172.21.15.1:0/1283731850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c403c890 con 0x7f94d40cdab0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.143+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41f7f90 con 0x7f94d40cdab0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.143+0000 7f94ca7fc640 1 -- 172.21.15.1:0/1283731850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4037040 con 0x7f94d40cdab0 2024-05-09T06:34:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.143+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41f4180 con 0x7f94d40cdab0 2024-05-09T06:34:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.145+0000 7f94ca7fc640 1 -- 172.21.15.1:0/1283731850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94c4044070 con 0x7f94d40cdab0 2024-05-09T06:34:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.145+0000 7f94ca7fc640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.145+0000 7f94ca7fc640 1 -- 172.21.15.1:0/1283731850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f94c40abce0 con 0x7f94d40cdab0 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.145+0000 7f94d984f640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.145+0000 7f94d984f640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 0x7f94b806e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006fd0 tx=0x7f94d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.146+0000 7f94de7c1740 1 Processor -- start 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.146+0000 7f94de7c1740 1 -- start start 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 0x55b103200460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 0x55b1031fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b10311b4d0 con 0x55b1031e2670 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b1030c33e0 con 0x55b1031fff10 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b103117490 con 0x55b1031b8350 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35804/0 (socket says 172.21.15.1:35804) 2024-05-09T06:34:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 -- 172.21.15.1:0/3983981860 learned_addr learned my addr 172.21.15.1:0/3983981860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d904e640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 0x55b103200460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94da050640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 0x55b1031fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 -- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 msgr2=0x55b1031fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 0x55b1031fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 -- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 msgr2=0x55b103200460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 0x55b103200460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 -- 172.21.15.1:0/3983981860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b102e2f2d0 con 0x55b1031e2670 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d904e640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 0x55b103200460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94da050640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 0x55b1031fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94d984f640 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f94d000f9f0 tx=0x7f94d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78678c01ca954254 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94b77fe640 1 -- 172.21.15.1:0/3983981860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d001b050 con 0x55b1031e2670 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94b77fe640 1 -- 172.21.15.1:0/3983981860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d0025070 con 0x55b1031e2670 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.147+0000 7f94b77fe640 1 -- 172.21.15.1:0/3983981860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0014e20 con 0x55b1031e2670 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 -- 172.21.15.1:0/3983981860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 msgr2=0x55b1031e2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f94d000f9f0 tx=0x7f94d0012000 comp rx=0 tx=0).stop 2024-05-09T06:34:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 -- 172.21.15.1:0/3983981860 shutdown_connections 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031b8350 0x55b1031fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031fff10 0x55b103200460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3983981860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031e2a50 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 -- 172.21.15.1:0/3983981860 >> 172.21.15.1:0/3983981860 conn(0x55b1031b4c80 msgr2=0x55b1031fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 -- 172.21.15.1:0/3983981860 shutdown_connections 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.148+0000 7f94de7c1740 1 -- 172.21.15.1:0/3983981860 wait complete. 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.149+0000 7f94de7c1740 1 -- start start 2024-05-09T06:34:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.149+0000 7f94de7c1740 1 Processor -- start 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031fff10 0x55b1031fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b1031b90f0 con 0x55b1031e2670 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b1030d68e0 con 0x55b103211000 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b1031b2bb0 con 0x55b1031fff10 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94de7c1740 10 client.0 _ll_register_callbacks cb 0x55b1030a39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35808/0 (socket says 172.21.15.1:35808) 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 -- 172.21.15.1:0/2878664454 learned_addr learned my addr 172.21.15.1:0/2878664454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d904e640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031fff10 0x55b1031fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94da050640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 -- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031fff10 msgr2=0x55b1031fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031fff10 0x55b1031fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 -- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 msgr2=0x55b1031fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 -- 172.21.15.1:0/2878664454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b102ef96b0 con 0x55b1031e2670 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94da050640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94d984f640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f94d00041e0 tx=0x7f94d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9951c54d2d2869 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.150+0000 7f94b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d00209d0 con 0x55b1031e2670 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f94d0020b70 con 0x55b1031e2670 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d0025070 con 0x55b1031e2670 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f94d001b020 con 0x55b1031e2670 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 10 client.8211 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:25.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b1031fc3b0 con 0x55b1031e2670 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d0025070 con 0x55b1031e2670 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f94d0005990 con 0x55b1031e2670 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 10 client.8211 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 10 client.8211 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 20 client.8211 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031e3640 0x55b1031e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 4 client.8211 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b103128c00 con 0x55b1031e3640 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94d904e640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031e3640 0x55b1031e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94b4ff9640 10 client.8211 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.151+0000 7f94d904e640 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031e3640 0x55b1031e3a20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2be950311709f7 server_cookie=34076036d05eada in_seq=0 out_seq=0 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94b4ff9640 1 -- 172.21.15.1:0/2878664454 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b103128c00 con 0x55b1031e3640 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94b4ff9640 10 client.8211 handle_command_reply: tid=0 2024-05-09T06:34:25.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 client.8211 shutdown 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 msgr2=0x55b1031f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f94d00041e0 tx=0x7f94d0004730 comp rx=0 tx=0).stop 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 shutdown_connections 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031e3640 0x55b1031e3a20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031fff10 0x55b1031fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2878664454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031e2670 0x55b1031f5bf0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94b4ff9640 0 client.8211 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 >> 172.21.15.1:0/2878664454 conn(0x55b1031b8350 msgr2=0x55b1031fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.152+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 shutdown_connections 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.153+0000 7f94de7c1740 1 -- 172.21.15.1:0/2878664454 wait complete. 2024-05-09T06:34:25.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.153+0000 7f94de7c1740 20 client.8211 trim_cache size 0 max 16384 2024-05-09T06:34:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 Processor -- start 2024-05-09T06:34:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 -- start start 2024-05-09T06:34:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031e2670 0x55b1031b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b103211000 0x55b1031b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ffc50 con 0x55b1031fff10 2024-05-09T06:34:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94d40b0f70 con 0x55b1031e2670 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.171+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94d41ff600 con 0x55b103211000 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35822/0 (socket says 172.21.15.1:35822) 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 -- 172.21.15.1:0/2138654863 learned_addr learned my addr 172.21.15.1:0/2138654863 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94da050640 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b103211000 0x55b1031b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d984f640 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031e2670 0x55b1031b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 -- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b103211000 msgr2=0x55b1031b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b103211000 0x55b1031b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 -- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031e2670 msgr2=0x55b1031b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031e2670 0x55b1031b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 -- 172.21.15.1:0/2138654863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1031e2090 con 0x55b1031fff10 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94d904e640 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94c4079ef0 tx=0x7f94c407a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687fe90529389de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94b6ffd640 1 -- 172.21.15.1:0/2138654863 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c404b040 con 0x55b1031fff10 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94b6ffd640 1 -- 172.21.15.1:0/2138654863 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c40a8070 con 0x55b1031fff10 2024-05-09T06:34:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.172+0000 7f94b6ffd640 1 -- 172.21.15.1:0/2138654863 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4082b40 con 0x55b1031fff10 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- 172.21.15.1:0/2138654863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 msgr2=0x55b1031b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94c4079ef0 tx=0x7f94c407a720 comp rx=0 tx=0).stop 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- 172.21.15.1:0/2138654863 shutdown_connections 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b103211000 0x55b1031b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031b8730 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 --2- 172.21.15.1:0/2138654863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1031e2670 0x55b1031b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- 172.21.15.1:0/2138654863 >> 172.21.15.1:0/2138654863 conn(0x55b1031b8350 msgr2=0x55b1031fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- 172.21.15.1:0/2138654863 shutdown_connections 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- 172.21.15.1:0/2138654863 wait complete. 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 -- start start 2024-05-09T06:34:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.173+0000 7f94de7c1740 1 Processor -- start 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031e2670 0x55b1032775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b103211610 con 0x55b1031fff10 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b1031f56d0 con 0x55b103211000 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b1031e54b0 con 0x55b1031e2670 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94de7c1740 10 client.0 _ll_register_callbacks cb 0x7f94d000f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94d904e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35838/0 (socket says 172.21.15.1:35838) 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94d904e640 1 -- 172.21.15.1:0/3193329264 learned_addr learned my addr 172.21.15.1:0/3193329264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94d984f640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031e2670 0x55b1032775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.174+0000 7f94da050640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 -- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031e2670 msgr2=0x55b1032775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031e2670 0x55b1032775b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 -- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 msgr2=0x55b1031b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 -- 172.21.15.1:0/3193329264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b103211440 con 0x55b1031fff10 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94d904e640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f94c4032fd0 tx=0x7f94c4048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d046e7ec98dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4088070 con 0x55b1031fff10 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f94c4078260 con 0x55b1031fff10 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c40a8070 con 0x55b1031fff10 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f94c404b040 con 0x55b1031fff10 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 10 client.8217 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b103201420 con 0x55b1031fff10 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c40a8070 con 0x55b1031fff10 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f94c40765d0 con 0x55b1031fff10 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 10 client.8217 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 10 client.8217 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 20 client.8217 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031fca20 0x55b1031fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 4 client.8217 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.175+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "status"}) -- 0x55b1031f71c0 con 0x55b1031fca20 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94d984f640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031fca20 0x55b1031fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94d984f640 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031fca20 0x55b1031fce00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38eaccc01aa62d2e server_cookie=3fe3e605169612ed in_seq=0 out_seq=0 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94b4ff9640 10 client.8217 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94b4ff9640 1 -- 172.21.15.1:0/3193329264 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b1031f71c0 con 0x55b1031fca20 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94b4ff9640 10 client.8217 handle_command_reply: tid=0 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 client.8217 shutdown 2024-05-09T06:34:25.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 msgr2=0x55b1031e5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f94c4032fd0 tx=0x7f94c4048040 comp rx=0 tx=0).stop 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 shutdown_connections 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b103211000 0x55b1031b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1031fff10 0x55b1031e5ff0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55b1031fca20 0x55b1031fce00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 --2- 172.21.15.1:0/3193329264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b1031e2670 0x55b1032775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94b4ff9640 0 client.8217 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 >> 172.21.15.1:0/3193329264 conn(0x55b1031b8350 msgr2=0x55b1032103c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.176+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 shutdown_connections 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94de7c1740 1 -- 172.21.15.1:0/3193329264 wait complete. 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94de7c1740 20 client.8217 trim_cache size 0 max 16384 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 7899, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4574537220000003, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 67, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 92, 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 27.852636330999999 2024-05-09T06:34:25.164 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 msgr2=0x7f94b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 0x7f94b806e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006fd0 tx=0x7f94d0008040 comp rx=0 tx=0).stop 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 msgr2=0x7f94d40aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f94c402a220 tx=0x7f94c40046c0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 shutdown_connections 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d40cf180 0x7f94d41f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d40cdab0 0x7f94d40aee90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 --2- 172.21.15.1:0/1283731850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d406e6d0 0x7f94d41f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 >> 172.21.15.1:0/1283731850 conn(0x7f94d40cc4b0 msgr2=0x7f94d4141a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 shutdown_connections 2024-05-09T06:34:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.177+0000 7f94dbab3640 1 -- 172.21.15.1:0/1283731850 wait complete. 2024-05-09T06:34:25.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:34:25.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.292+0000 7f2fd6a8c640 1 Processor -- start 2024-05-09T06:34:25.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.292+0000 7f2fd6a8c640 1 -- start start 2024-05-09T06:34:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.292+0000 7f2fd6a8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fd006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fd6a8c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd00ce0e0 con 0x7f2fd00cf180 2024-05-09T06:34:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fd6a8c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd00ce3c0 con 0x7f2fd00b3300 2024-05-09T06:34:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fd6a8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd00ce6a0 con 0x7f2fd006e6d0 2024-05-09T06:34:25.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2fd00cf180 0x7f2fd00cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37478/0 (socket says 172.21.15.1:37478) 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcffff640 1 -- 172.21.15.1:0/3955655207 learned_addr learned my addr 172.21.15.1:0/3955655207 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fd4828640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fd006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068738475 0 0) 0x7f2fd00ce0e0 con 0x7f2fd00cf180 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fbc005730 con 0x7f2fd00cf180 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3521060596 0 0) 0x7f2fbc005730 con 0x7f2fd00cf180 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 msgr2=0x7f2fd006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fd006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 >> v1:172.21.15.6:6789/0 conn(0x7f2fd00b3300 legacy=0x7f2fd00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd00ce980 con 0x7f2fd00cf180 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f2fc0001670 con 0x7f2fd00cf180 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 >> v1:172.21.15.1:6789/0 conn(0x7f2fd00cf180 legacy=0x7f2fd00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc005bf0 0x7f2fbc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fbc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fbc006f50 0x7f2fbc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd00ce6a0 con 0x7f2fbc006f50 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc00dc80 con 0x7f2fbc005bf0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2fbc00de80 con 0x7f2fd006e6d0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fd5029640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fbc006f50 0x7f2fbc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.293+0000 7f2fcffff640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fbc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd4828640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc005bf0 0x7f2fbc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd5029640 1 -- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 msgr2=0x7f2fbc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd5029640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd006e6d0 0x7f2fbc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd5029640 1 -- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc005bf0 msgr2=0x7f2fbc006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd5029640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc005bf0 0x7f2fbc006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd5029640 1 -- 172.21.15.1:0/3955655207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc005730 con 0x7f2fbc006f50 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.294+0000 7f2fd4828640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc005bf0 0x7f2fbc006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd5029640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fbc006f50 0x7f2fbc0073d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8001720 tx=0x7f2fbc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8e722ff19ce4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8001370 con 0x7f2fbc006f50 2024-05-09T06:34:25.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb80052d0 con 0x7f2fbc006f50 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fcf7fe640 1 -- 172.21.15.1:0/3955655207 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb802abc0 con 0x7f2fbc006f50 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/3955655207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fbc006f50 msgr2=0x7f2fbc0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/3955655207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fbc006f50 0x7f2fbc0073d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8001720 tx=0x7f2fbc007a40 comp rx=0 tx=0).stop 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd5029640 1 -- 172.21.15.1:0/3955655207 reap_dead start 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/3955655207 shutdown_connections 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/3955655207 >> 172.21.15.1:0/3955655207 conn(0x7f2fd00cc4b0 msgr2=0x7f2fd00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/3955655207 shutdown_connections 2024-05-09T06:34:25.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.295+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/3955655207 wait complete. 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.296+0000 7f2fd6a8c640 1 Processor -- start 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.296+0000 7f2fd6a8c640 1 -- start start 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc006f50 0x7f2fd013a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 0x7f2fd0139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01447c0 con 0x7f2fd01538f0 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0070ed0 con 0x7f2fbc006f50 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fd6a8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd0144940 con 0x7f2fd0153d70 2024-05-09T06:34:25.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fcffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fcffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35866/0 (socket says 172.21.15.1:35866) 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.297+0000 7f2fcffff640 1 -- 172.21.15.1:0/1956099679 learned_addr learned my addr 172.21.15.1:0/1956099679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fd4828640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc006f50 0x7f2fd013a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fd5029640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 0x7f2fd0139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 -- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 msgr2=0x7f2fd0139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 0x7f2fd0139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 -- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc006f50 msgr2=0x7f2fd013a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc006f50 0x7f2fd013a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 -- 172.21.15.1:0/1956099679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01f7940 con 0x7f2fd01538f0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fd5029640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 0x7f2fd0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcffff640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00072c0 tx=0x7f2fc00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000f960 con 0x7f2fd01538f0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0018070 con 0x7f2fd01538f0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd01f7c20 con 0x7f2fd01538f0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000fd60 con 0x7f2fd01538f0 2024-05-09T06:34:25.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.298+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd01f81e0 con 0x7f2fd01538f0 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.299+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd00752b0 con 0x7f2fd01538f0 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.300+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fc001d460 con 0x7f2fd01538f0 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.300+0000 7f2fcd7fa640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 0x7f2fb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.300+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f2fc008b0f0 con 0x7f2fd01538f0 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.300+0000 7f2fd4828640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 0x7f2fb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.301+0000 7f2fd4828640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 0x7f2fb006e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009fd0 tx=0x7f2fc800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:25.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.304+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2fc00598c0 con 0x7f2fd01538f0 2024-05-09T06:34:25.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.526+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2fd01409f0 con 0x7f2fd01538f0 2024-05-09T06:34:25.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.526+0000 7f2fcd7fa640 1 -- 172.21.15.1:0/1956099679 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f2fc00591c0 con 0x7f2fd01538f0 2024-05-09T06:34:25.507 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 msgr2=0x7f2fb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 0x7f2fb006e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009fd0 tx=0x7f2fc800b040 comp rx=0 tx=0).stop 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 msgr2=0x7f2fd0139050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00072c0 tx=0x7f2fc00076c0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 shutdown_connections 2024-05-09T06:34:25.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fb006be60 0x7f2fb006e320 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fd0153d70 0x7f2fd0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fd01538f0 0x7f2fd0139050 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 --2- 172.21.15.1:0/1956099679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc006f50 0x7f2fd013a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 >> 172.21.15.1:0/1956099679 conn(0x7f2fd00cc4b0 msgr2=0x7f2fd00cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 shutdown_connections 2024-05-09T06:34:25.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25.529+0000 7f2fd6a8c640 1 -- 172.21.15.1:0/1956099679 wait complete. 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:34:25.521 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:25.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:25.521 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:34:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:34:25.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:25.541 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:34:25.603 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:34:25.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:25.605 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:34:25.661 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:34:25.661 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:34:25.661 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:34:25.662 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:34:25.662 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:34:25.662 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:34:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:34:25.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:34:25.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:25.791 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:34:25.807 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:34:25.807 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:34:25.807 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:25.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:25.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:34:25.874 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:34:25.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:34:25.874 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:25.930 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:34:25.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:34:25.931 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:25.986 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:34:26.053 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:34:26.111 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:34:26.112 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:26.113 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:26.113 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:26.113 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:26.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.113 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:34:26.166 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:34:26.167 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:34:26.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:34:26.235 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:34:26.292 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:34:26.292 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:34:26.293 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:34:26.294 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:34:26.295 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:26.295 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:34:26.295 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:26.295 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:34:26.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.295 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:34:26.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:34:26.313+0000 7f14517d5480 -1 init, newargv = 0x55cf0c1fc040 newargc=15 2024-05-09T06:34:26.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[69769]: starting ceph client 2024-05-09T06:34:26.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[69769]: starting fuse 2024-05-09T06:34:26.312 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:34:26.312 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:34:26.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:34:26.312 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:34:26.312 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:34:26.313 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:34:26.314 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:34:26.433 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.69769.asok 2024-05-09T06:34:26.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:34:26.444 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69769.asok 2024-05-09T06:34:26.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69769.asok status 2024-05-09T06:34:26.582 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "69769", 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 8232, 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:34:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 8232 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2949046019 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2949046019 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.8232 192.168.0.1:0/2949046019", 2024-05-09T06:34:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/2949046019", 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 67, 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 92, 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:34:26.585 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:26.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.591 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:34:26.591 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:26.607 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:34:26.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:26.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.608 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:34:26.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.674 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-09T06:34:26.732 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:34:26.752+0000 7f0ee16dc640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-09T06:34:26.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:34:26.734 INFO:teuthology.orchestra.run.smithi001.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-09T06:34:26.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.734 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-09T06:34:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.901+0000 7f7c2c9ce640 1 Processor -- start 2024-05-09T06:34:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.901+0000 7f7c2c9ce640 1 -- start start 2024-05-09T06:34:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.901+0000 7f7c2c9ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c28143bb0 0x7f7c280cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c2c9ce640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7c2806e320 0x7f7c280d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c2c9ce640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c280cdd50 con 0x7f7c281446d0 2024-05-09T06:34:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c2c9ce640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7c280ce8d0 con 0x7f7c2806e320 2024-05-09T06:34:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c2c9ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7c280ce030 con 0x7f7c28143bb0 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7c2806e320 0x7f7c280d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7c2806e320 0x7f7c280d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35874/0 (socket says 172.21.15.1:35874) 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 -- 172.21.15.1:0/136588795 learned_addr learned my addr 172.21.15.1:0/136588795 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c2659c640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c28143bb0 0x7f7c280cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 -- 172.21.15.1:0/136588795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c28143bb0 msgr2=0x7f7c280cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c28143bb0 0x7f7c280cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 -- 172.21.15.1:0/136588795 >> v1:172.21.15.6:6789/0 conn(0x7f7c281446d0 legacy=0x7f7c2815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.902+0000 7f7c26d9d640 1 -- 172.21.15.1:0/136588795 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c280ce310 con 0x7f7c2806e320 2024-05-09T06:34:26.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c2806e320 0x7f7c280d01f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7c280cea50 tx=0x7f7c2002c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31316c6f82c71c7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c25d9b640 1 -- 172.21.15.1:0/136588795 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20009600 con 0x7f7c2806e320 2024-05-09T06:34:26.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c25d9b640 1 -- 172.21.15.1:0/136588795 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c200097a0 con 0x7f7c2806e320 2024-05-09T06:34:26.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c25d9b640 1 -- 172.21.15.1:0/136588795 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20009b70 con 0x7f7c2806e320 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/136588795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c2806e320 msgr2=0x7f7c280d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c2806e320 0x7f7c280d01f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7c280cea50 tx=0x7f7c2002c830 comp rx=0 tx=0).stop 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/136588795 shutdown_connections 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c2806e320 0x7f7c280d01f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/136588795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c28143bb0 0x7f7c280cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.903+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/136588795 >> 172.21.15.1:0/136588795 conn(0x7f7c2806b3c0 msgr2=0x7f7c2806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.904+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/136588795 shutdown_connections 2024-05-09T06:34:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.904+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/136588795 wait complete. 2024-05-09T06:34:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.905+0000 7f7c2c9ce640 1 Processor -- start 2024-05-09T06:34:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.905+0000 7f7c2c9ce640 1 -- start start 2024-05-09T06:34:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c2806e320 0x7f7c280cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c281446d0 0x7f7c2813c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c280cdaf0 con 0x7f7c28143bb0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c280cd970 con 0x7f7c281446d0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2c9ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7c28144110 con 0x7f7c2806e320 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35884/0 (socket says 172.21.15.1:35884) 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 -- 172.21.15.1:0/1880619797 learned_addr learned my addr 172.21.15.1:0/1880619797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c2659c640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c2806e320 0x7f7c280cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c281446d0 0x7f7c2813c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 -- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c2806e320 msgr2=0x7f7c280cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c2806e320 0x7f7c280cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 -- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c281446d0 msgr2=0x7f7c2813c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c281446d0 0x7f7c2813c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 -- 172.21.15.1:0/1880619797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c28158bb0 con 0x7f7c28143bb0 2024-05-09T06:34:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000b420 tx=0x7f7c1000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1effd640 1 -- 172.21.15.1:0/1880619797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011830 con 0x7f7c28143bb0 2024-05-09T06:34:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.906+0000 7f7c1effd640 1 -- 172.21.15.1:0/1880619797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10016070 con 0x7f7c28143bb0 2024-05-09T06:34:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.907+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c280b3ae0 con 0x7f7c28143bb0 2024-05-09T06:34:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.907+0000 7f7c1effd640 1 -- 172.21.15.1:0/1880619797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011c10 con 0x7f7c28143bb0 2024-05-09T06:34:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.907+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c28141620 con 0x7f7c28143bb0 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.908+0000 7f7c1effd640 1 -- 172.21.15.1:0/1880619797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c1001b460 con 0x7f7c28143bb0 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.909+0000 7f7c1effd640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 0x7f7c0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.909+0000 7f7c1effd640 1 -- 172.21.15.1:0/1880619797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f7c1008aa70 con 0x7f7c28143bb0 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.909+0000 7f7c2659c640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 0x7f7c0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.910+0000 7f7c2659c640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 0x7f7c0806e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.910+0000 7f7c2f7b0740 1 Processor -- start 2024-05-09T06:34:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.910+0000 7f7c2f7b0740 1 -- start start 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a5894ef0 0x55a6a589ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6a57fd880 con 0x55a6a58c4a50 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a6a57a5520 con 0x55a6a5894ef0 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a6a57f9840 con 0x55a6a58e09b0 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35896/0 (socket says 172.21.15.1:35896) 2024-05-09T06:34:26.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 -- 172.21.15.1:0/2468280996 learned_addr learned my addr 172.21.15.1:0/2468280996 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 -- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 msgr2=0x55a6a58e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58e0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a5894ef0 0x55a6a589ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 -- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a5894ef0 msgr2=0x55a6a589ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a5894ef0 0x55a6a589ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 -- 172.21.15.1:0/2468280996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a5799b40 con 0x55a6a58c4a50 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.911+0000 7f7c2659c640 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7c18010bb0 tx=0x7f7c18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff391b25ff6d786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c07fff640 1 -- 172.21.15.1:0/2468280996 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1801c050 con 0x55a6a58c4a50 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c07fff640 1 -- 172.21.15.1:0/2468280996 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c18026070 con 0x55a6a58c4a50 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c07fff640 1 -- 172.21.15.1:0/2468280996 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18021440 con 0x55a6a58c4a50 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2468280996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 msgr2=0x55a6a58c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7c18010bb0 tx=0x7f7c18013000 comp rx=0 tx=0).stop 2024-05-09T06:34:26.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2468280996 shutdown_connections 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a5894ef0 0x55a6a589ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2468280996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58c4e30 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2468280996 >> 172.21.15.1:0/2468280996 conn(0x55a6a58971b0 msgr2=0x55a6a58df860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2468280996 shutdown_connections 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.912+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2468280996 wait complete. 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.913+0000 7f7c2f7b0740 1 -- start start 2024-05-09T06:34:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.913+0000 7f7c2f7b0740 1 Processor -- start 2024-05-09T06:34:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58c4a50 0x55a6a58a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6a589b220 con 0x55a6a58be140 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a6a57b8ab0 con 0x55a6a58c4a50 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a6a589a850 con 0x55a6a58e09b0 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2f7b0740 10 client.0 _ll_register_callbacks cb 0x55a6a5510b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35910/0 (socket says 172.21.15.1:35910) 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 -- 172.21.15.1:0/3638705528 learned_addr learned my addr 172.21.15.1:0/3638705528 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c2659c640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58c4a50 0x55a6a58a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 -- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 msgr2=0x55a6a58a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 -- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58c4a50 msgr2=0x55a6a58a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58c4a50 0x55a6a58a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 -- 172.21.15.1:0/3638705528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a55dd7b0 con 0x55a6a58be140 2024-05-09T06:34:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7c2002ce20 tx=0x7f7c20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac3c9712cae0ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.914+0000 7f7c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20035900 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f7c20035aa0 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c18026070 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f7c20035ca0 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 10 client.8244 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55a6a563df40 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c18026070 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f7c20008040 con 0x55a6a58be140 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 10 client.8244 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 10 client.8244 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 20 client.8244 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:26.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58a7470 0x55a6a59019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 4 client.8244 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6a580afb0 con 0x55a6a58a7470 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.915+0000 7f7c2659c640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58a7470 0x55a6a59019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2659c640 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58a7470 0x55a6a59019f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658d3fbba2730f1c server_cookie=ef708198c8da88f8 in_seq=0 out_seq=0 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c057fa640 10 client.8244 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c057fa640 1 -- 172.21.15.1:0/3638705528 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6a580afb0 con 0x55a6a58a7470 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c057fa640 10 client.8244 handle_command_reply: tid=0 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 client.8244 shutdown 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 msgr2=0x55a6a58be5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7c2002ce20 tx=0x7f7c20004ab0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 shutdown_connections 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a58be5c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58a7470 0x55a6a59019f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58e09b0 0x55a6a58a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c057fa640 0 client.8244 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.916+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/3638705528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58c4a50 0x55a6a58a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.917+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 >> 172.21.15.1:0/3638705528 conn(0x55a6a5894ef0 msgr2=0x55a6a58e2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.917+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 shutdown_connections 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.917+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/3638705528 wait complete. 2024-05-09T06:34:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.917+0000 7f7c2f7b0740 20 client.8244 trim_cache size 0 max 16384 2024-05-09T06:34:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.935+0000 7f7c2f7b0740 1 Processor -- start 2024-05-09T06:34:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.935+0000 7f7c2f7b0740 1 -- start start 2024-05-09T06:34:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58be140 0x55a6a58b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5906c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281fc810 con 0x55a6a58c4a50 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281fb820 con 0x55a6a58e09b0 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7c280b3290 con 0x55a6a58be140 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35926/0 (socket says 172.21.15.1:35926) 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 -- 172.21.15.1:0/2842530842 learned_addr learned my addr 172.21.15.1:0/2842530842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c2659c640 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58be140 0x55a6a58b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5906c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 -- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58be140 msgr2=0x55a6a58b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58be140 0x55a6a58b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 -- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 msgr2=0x55a6a5906c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5906c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 -- 172.21.15.1:0/2842530842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a54d10a0 con 0x55a6a58c4a50 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f7c10058ae0 tx=0x7f7c1000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba05d7dcc32948e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c077fe640 1 -- 172.21.15.1:0/2842530842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10070070 con 0x55a6a58c4a50 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c077fe640 1 -- 172.21.15.1:0/2842530842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1008a040 con 0x55a6a58c4a50 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.936+0000 7f7c077fe640 1 -- 172.21.15.1:0/2842530842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1005f0b0 con 0x55a6a58c4a50 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2842530842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 msgr2=0x55a6a58b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f7c10058ae0 tx=0x7f7c1000bf90 comp rx=0 tx=0).stop 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2842530842 shutdown_connections 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5906c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58c4a50 0x55a6a58b8000 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/2842530842 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58be140 0x55a6a58b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2842530842 >> 172.21.15.1:0/2842530842 conn(0x55a6a5894ef0 msgr2=0x55a6a58947e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2842530842 shutdown_connections 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/2842530842 wait complete. 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.937+0000 7f7c2f7b0740 1 -- start start 2024-05-09T06:34:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 Processor -- start 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58c4a50 0x55a6a59608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5960e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6a58a5230 con 0x55a6a58be140 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281fb320 con 0x55a6a58e09b0 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a6a58e39c0 con 0x55a6a58c4a50 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2f7b0740 10 client.0 _ll_register_callbacks cb 0x7f7c2000d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.938+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35934/0 (socket says 172.21.15.1:35934) 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 -- 172.21.15.1:0/826027985 learned_addr learned my addr 172.21.15.1:0/826027985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58c4a50 0x55a6a59608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c26d9d640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5960e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 -- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58c4a50 msgr2=0x55a6a59608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58c4a50 0x55a6a59608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 -- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 msgr2=0x55a6a5960e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5960e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 -- 172.21.15.1:0/826027985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a58a4fd0 con 0x55a6a58be140 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2659c640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7c18011010 tx=0x7f7c18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb51ee8c12c4ae0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1803a060 con 0x55a6a58be140 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f7c18004390 con 0x55a6a58be140 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1008a040 con 0x55a6a58be140 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f7c18040050 con 0x55a6a58be140 2024-05-09T06:34:26.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2f7b0740 10 client.8250 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55a6a58f6540 con 0x55a6a58be140 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.939+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1008a040 con 0x55a6a58be140 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f7c18043020 con 0x55a6a58be140 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 10 client.8250 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 10 client.8250 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 20 client.8250 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58977d0 0x55a6a5898330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 4 client.8250 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55a6a5805300 con 0x55a6a58977d0 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58977d0 0x55a6a5898330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c057fa640 10 client.8250 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.940+0000 7f7c1ffff640 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58977d0 0x55a6a5898330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2677f3768069eac server_cookie=1aadd4c02a1e5e2a in_seq=0 out_seq=0 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c057fa640 1 -- 172.21.15.1:0/826027985 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6a5805300 con 0x55a6a58977d0 2024-05-09T06:34:26.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c057fa640 10 client.8250 handle_command_reply: tid=0 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 client.8250 shutdown 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 msgr2=0x55a6a5907100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7c18011010 tx=0x7f7c18004730 comp rx=0 tx=0).stop 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 shutdown_connections 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6a58e09b0 0x55a6a5960e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55a6a58977d0 0x55a6a5898330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a6a58c4a50 0x55a6a59608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 --2- 172.21.15.1:0/826027985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6a58be140 0x55a6a5907100 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 >> 172.21.15.1:0/826027985 conn(0x55a6a5894ef0 msgr2=0x55a6a58a4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c057fa640 0 client.8250 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:26.927 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f0ee16dc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0) n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x564e3d142c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-09T06:34:26.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 shutdown_connections 2024-05-09T06:34:26.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 1 -- 172.21.15.1:0/826027985 wait complete. 2024-05-09T06:34:26.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2f7b0740 20 client.8250 trim_cache size 0 max 16384 2024-05-09T06:34:26.928 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:34:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "e4a7b0d1-0558-4449-962b-0ffdde1767df", 2024-05-09T06:34:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:34:26.929 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:26.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 msgr2=0x7f7c0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 0x7f7c0806e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 msgr2=0x7f7c28158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.941+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000b420 tx=0x7f7c1000b8e0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 shutdown_connections 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7c0806be30 0x7f7c0806e2f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c281446d0 0x7f7c2813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c28143bb0 0x7f7c28158670 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 --2- 172.21.15.1:0/1880619797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c2806e320 0x7f7c280cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 >> 172.21.15.1:0/1880619797 conn(0x7f7c2806b3c0 msgr2=0x7f7c2815d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 shutdown_connections 2024-05-09T06:34:26.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:26.942+0000 7f7c2c9ce640 1 -- 172.21.15.1:0/1880619797 wait complete. 2024-05-09T06:34:26.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:26.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:34:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.058+0000 7f3c14509640 1 Processor -- start 2024-05-09T06:34:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.058+0000 7f3c14509640 1 -- start start 2024-05-09T06:34:27.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.059+0000 7f3c14509640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3c0c15e190 0x7f3c0c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.059+0000 7f3c14509640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3c0c0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c14509640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c0c161fe0 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c14509640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c15f580 con 0x7f3c0c15e190 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c14509640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c1622c0 con 0x7f3c0c0d1c70 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c12aa6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3c0c0b56d0 0x7f3c0c0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37492/0 (socket says 172.21.15.1:37492) 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2136285241 learned_addr learned my addr 172.21.15.1:0/2136285241 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3c0c0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c122a5640 1 --2- 172.21.15.1:0/2136285241 >> v2:172.21.15.6:3300/0 conn(0x7f3c0c15e190 0x7f3c0c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759973636 0 0) 0x7f3c0c161fe0 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf8003520 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4215034093 0 0) 0x7f3bf8003520 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 msgr2=0x7f3c0c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3c0c0d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 >> v2:172.21.15.6:3300/0 conn(0x7f3c0c15e190 msgr2=0x7f3c0c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 --2- 172.21.15.1:0/2136285241 >> v2:172.21.15.6:3300/0 conn(0x7f3c0c15e190 0x7f3c0c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c1625a0 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3c040019f0 con 0x7f3c0c0b56d0 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 >> v1:172.21.15.1:6789/0 conn(0x7f3c0c0b56d0 legacy=0x7f3c0c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3bf80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bf8004d40 0x7f3bf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1622c0 con 0x7f3bf8003f30 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf800b9c0 con 0x7f3bf8004d40 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.060+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf800bb40 con 0x7f3c0c0d1c70 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c122a5640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3bf80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bf8004d40 0x7f3bf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 -- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 msgr2=0x7f3bf80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c0d1c70 0x7f3bf80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 -- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bf8004d40 msgr2=0x7f3bf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bf8004d40 0x7f3bf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.061+0000 7f3c11aa4640 1 -- 172.21.15.1:0/2136285241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf8003520 con 0x7f3bf8003f30 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009a90 tx=0x7f3bf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52d1438508658b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc036070 con 0x7f3bf8003f30 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc031400 con 0x7f3bf8003f30 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c112a3640 1 -- 172.21.15.1:0/2136285241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc009690 con 0x7f3bf8003f30 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 -- 172.21.15.1:0/2136285241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 msgr2=0x7f3bf8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 --2- 172.21.15.1:0/2136285241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009a90 tx=0x7f3bf8005830 comp rx=0 tx=0).stop 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2136285241 reap_dead start 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 -- 172.21.15.1:0/2136285241 shutdown_connections 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 -- 172.21.15.1:0/2136285241 >> 172.21.15.1:0/2136285241 conn(0x7f3c0c073340 msgr2=0x7f3c0c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 -- 172.21.15.1:0/2136285241 shutdown_connections 2024-05-09T06:34:27.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.062+0000 7f3c14509640 1 -- 172.21.15.1:0/2136285241 wait complete. 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 Processor -- start 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 -- start start 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c14b7e0 0x7f3c0c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c15ac10 0x7f3c0c14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b44f0 con 0x7f3bf8003f30 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c161c60 con 0x7f3c0c14b7e0 2024-05-09T06:34:27.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c14509640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b4670 con 0x7f3c0c15ac10 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c122a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c122a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35958/0 (socket says 172.21.15.1:35958) 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c122a5640 1 -- 172.21.15.1:0/676647544 learned_addr learned my addr 172.21.15.1:0/676647544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c15ac10 0x7f3c0c14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.064+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c14b7e0 0x7f3c0c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 -- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c15ac10 msgr2=0x7f3c0c14aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c15ac10 0x7f3c0c14aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 -- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c14b7e0 msgr2=0x7f3c0c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c14b7e0 0x7f3c0c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 -- 172.21.15.1:0/676647544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c159b90 con 0x7f3bf8003f30 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c122a5640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011a70 tx=0x7f3c08011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c02ffd640 1 -- 172.21.15.1:0/676647544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800fa90 con 0x7f3bf8003f30 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c02ffd640 1 -- 172.21.15.1:0/676647544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0800fc30 con 0x7f3bf8003f30 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c159e50 con 0x7f3bf8003f30 2024-05-09T06:34:27.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c02ffd640 1 -- 172.21.15.1:0/676647544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08014440 con 0x7f3bf8003f30 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.065+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c15a410 con 0x7f3bf8003f30 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.067+0000 7f3c02ffd640 1 -- 172.21.15.1:0/676647544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c08012040 con 0x7f3bf8003f30 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.067+0000 7f3c02ffd640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 0x7f3bf006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.067+0000 7f3c02ffd640 1 -- 172.21.15.1:0/676647544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f3c0808b6e0 con 0x7f3bf8003f30 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.067+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 0x7f3bf006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.068+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 0x7f3bf006e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc033000 tx=0x7f3bfc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.068+0000 7f3c17213740 1 Processor -- start 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 -- start start 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f8962fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f8992af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5643f88ccd00 con 0x5643f8993030 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5643f88744c0 con 0x5643f89b0650 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5643f88c8cc0 con 0x5643f8993770 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.069+0000 7f3c12aa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35960/0 (socket says 172.21.15.1:35960) 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2235098988 learned_addr learned my addr 172.21.15.1:0/2235098988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f8992af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c122a5640 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f8962fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 msgr2=0x5643f8992af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f8992af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 msgr2=0x5643f8962fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f8962fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 -- 172.21.15.1:0/2235098988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f86aa7d0 con 0x5643f8993030 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b5b60 tx=0x7f3c0400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc808f1906669ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3beffff640 1 -- 172.21.15.1:0/2235098988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401b070 con 0x5643f8993030 2024-05-09T06:34:27.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3beffff640 1 -- 172.21.15.1:0/2235098988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04015440 con 0x5643f8993030 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.070+0000 7f3beffff640 1 -- 172.21.15.1:0/2235098988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c040157f0 con 0x5643f8993030 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- 172.21.15.1:0/2235098988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 msgr2=0x5643f8967e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b5b60 tx=0x7f3c0400a060 comp rx=0 tx=0).stop 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- 172.21.15.1:0/2235098988 shutdown_connections 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993030 0x5643f8967e40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f8992af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 --2- 172.21.15.1:0/2235098988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f8962fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- 172.21.15.1:0/2235098988 >> 172.21.15.1:0/2235098988 conn(0x5643f8965250 msgr2=0x5643f89adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- 172.21.15.1:0/2235098988 shutdown_connections 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- 172.21.15.1:0/2235098988 wait complete. 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.071+0000 7f3c17213740 1 -- start start 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 Processor -- start 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f89af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5643f8967400 con 0x5643f89afd60 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5643f8887b90 con 0x5643f89b0650 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.072+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5643f89667f0 con 0x5643f8993770 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c17213740 10 client.0 _ll_register_callbacks cb 0x5643f88c3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35962/0 (socket says 172.21.15.1:35962) 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 -- 172.21.15.1:0/831201152 learned_addr learned my addr 172.21.15.1:0/831201152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c122a5640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f89af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 -- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 msgr2=0x5643f89837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 -- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 msgr2=0x5643f89af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f89af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 -- 172.21.15.1:0/831201152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f85e03f0 con 0x5643f89afd60 2024-05-09T06:34:27.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f89af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c04007920 tx=0x7f3c04001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14b49a28674d7a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401b040 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f3c04017270 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04015440 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3c0402e050 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c17213740 10 client.8262 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.073+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5643f85a00f0 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04015440 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f3c04031020 con 0x5643f89afd60 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 10 client.8262 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 10 client.8262 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 20 client.8262 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8980ff0 0x5643f89813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 4 client.8262 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:27.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f88da430 con 0x5643f8980ff0 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c122a5640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8980ff0 0x5643f89813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3bed7fa640 10 client.8262 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.074+0000 7f3c122a5640 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8980ff0 0x5643f89813d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aac81d07c4da1f8 server_cookie=f26aabafa8b43a06 in_seq=0 out_seq=0 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3bed7fa640 1 -- 172.21.15.1:0/831201152 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643f88da430 con 0x5643f8980ff0 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3bed7fa640 10 client.8262 handle_command_reply: tid=0 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 client.8262 shutdown 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 msgr2=0x5643f8967f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c04007920 tx=0x7f3c04001dc0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 shutdown_connections 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f8967f70 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f89af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8980ff0 0x5643f89813d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 --2- 172.21.15.1:0/831201152 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3bed7fa640 0 client.8262 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:27.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 >> 172.21.15.1:0/831201152 conn(0x5643f8993030 msgr2=0x5643f8967800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 shutdown_connections 2024-05-09T06:34:27.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 1 -- 172.21.15.1:0/831201152 wait complete. 2024-05-09T06:34:27.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.075+0000 7f3c17213740 20 client.8262 trim_cache size 0 max 16384 2024-05-09T06:34:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.093+0000 7f3c17213740 1 Processor -- start 2024-05-09T06:34:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.093+0000 7f3c17213740 1 -- start start 2024-05-09T06:34:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.093+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.093+0000 7f3c17213740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f898f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c17213740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c2013f0 con 0x5643f89afd60 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c200410 con 0x5643f89b0650 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0d1430 con 0x5643f8993770 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35966/0 (socket says 172.21.15.1:35966) 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 -- 172.21.15.1:0/3671745146 learned_addr learned my addr 172.21.15.1:0/3671745146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c122a5640 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 -- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 msgr2=0x5643f89ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 -- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 msgr2=0x5643f898f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f898f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 -- 172.21.15.1:0/3671745146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f8995880 con 0x5643f89afd60 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c122a5640 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc02ab50 tx=0x7f3bfc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71d3e256544f6f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3bef7fe640 1 -- 172.21.15.1:0/3671745146 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc055070 con 0x5643f89afd60 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3bef7fe640 1 -- 172.21.15.1:0/3671745146 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc048c20 con 0x5643f89afd60 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.094+0000 7f3bef7fe640 1 -- 172.21.15.1:0/3671745146 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc036040 con 0x5643f89afd60 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 -- 172.21.15.1:0/3671745146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 msgr2=0x5643f89ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc02ab50 tx=0x7f3bfc02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 -- 172.21.15.1:0/3671745146 shutdown_connections 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89b0650 0x5643f898f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f89afd60 0x5643f89ade30 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 --2- 172.21.15.1:0/3671745146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f8993770 0x5643f89ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 -- 172.21.15.1:0/3671745146 >> 172.21.15.1:0/3671745146 conn(0x5643f8993030 msgr2=0x5643f89af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 -- 172.21.15.1:0/3671745146 shutdown_connections 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.095+0000 7f3c17213740 1 -- 172.21.15.1:0/3671745146 wait complete. 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 -- start start 2024-05-09T06:34:27.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 Processor -- start 2024-05-09T06:34:27.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 0x5643f89827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f89b0650 0x5643f898d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5643f8964740 con 0x5643f8993770 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.096+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1ffdf0 con 0x5643f89afd60 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c17213740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5643f886a4d0 con 0x5643f89b0650 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c17213740 10 client.0 _ll_register_callbacks cb 0x7f3c04019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35982/0 (socket says 172.21.15.1:35982) 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 -- 172.21.15.1:0/1561863751 learned_addr learned my addr 172.21.15.1:0/1561863751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 0x5643f89827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 -- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f89b0650 msgr2=0x5643f898d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c12aa6640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f89b0650 0x5643f898d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f89b0650 0x5643f898d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 -- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 msgr2=0x5643f89827f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 0x5643f89827f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 -- 172.21.15.1:0/1561863751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f898e890 con 0x5643f8993770 2024-05-09T06:34:27.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 0x5643f89827f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c122a5640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3c08059610 tx=0x7f3c08089860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a29fa798b4ff6b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08055cf0 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f3c08061e60 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc048c20 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3c08055e90 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c17213740 10 client.8268 fetch_fsmap learned FSMap version 67 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.097+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5643f898fe70 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc048c20 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f3c08084070 con 0x5643f8993770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 10 client.8268 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 10 client.8268 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 20 client.8268 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8a11770 0x5643f898db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 4 client.8268 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643f88d4780 con 0x5643f8a11770 2024-05-09T06:34:27.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8a11770 0x5643f898db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3bed7fa640 10 client.8268 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c11aa4640 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8a11770 0x5643f898db50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29041e9476d781cc server_cookie=bd58050c4609aa86 in_seq=0 out_seq=0 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3bed7fa640 1 -- 172.21.15.1:0/1561863751 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5643f88d4780 con 0x5643f8a11770 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3bed7fa640 10 client.8268 handle_command_reply: tid=0 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 1 client.8268 shutdown 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 msgr2=0x5643f89822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.098+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3c08059610 tx=0x7f3c08089860 comp rx=0 tx=0).stop 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 shutdown_connections 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x5643f8a11770 0x5643f898db50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5643f89b0650 0x5643f898d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5643f89afd60 0x5643f89827f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 --2- 172.21.15.1:0/1561863751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5643f8993770 0x5643f89822b0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3bed7fa640 0 client.8268 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 >> 172.21.15.1:0/1561863751 conn(0x5643f8993030 msgr2=0x5643f89a3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 shutdown_connections 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 1 -- 172.21.15.1:0/1561863751 wait complete. 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c17213740 20 client.8268 trim_cache size 0 max 16384 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "e4a7b0d1-0558-4449-962b-0ffdde1767df": { 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir", 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "e4a7b0d1-0558-4449-962b-0ffdde1767df", 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive" 2024-05-09T06:34:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:27.087 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 msgr2=0x7f3bf006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 0x7f3bf006e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc033000 tx=0x7f3bfc008030 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 msgr2=0x7f3c0c1481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011a70 tx=0x7f3c08011e70 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 shutdown_connections 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3bf006be30 0x7f3bf006e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3c0c15ac10 0x7f3c0c14aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c14b7e0 0x7f3c0c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 --2- 172.21.15.1:0/676647544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bf8003f30 0x7f3c0c1481c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 >> 172.21.15.1:0/676647544 conn(0x7f3c0c073340 msgr2=0x7f3c0c06cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 shutdown_connections 2024-05-09T06:34:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:27.099+0000 7f3c14509640 1 -- 172.21.15.1:0/676647544 wait complete. 2024-05-09T06:34:27.091 INFO:tasks.cephfs.filesystem:scrub status for tag:e4a7b0d1-0558-4449-962b-0ffdde1767df - {'path': '/dir', 'tag': 'e4a7b0d1-0558-4449-962b-0ffdde1767df', 'options': 'recursive'} 2024-05-09T06:34:28.279 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:34:28.298+0000 7f0ee56e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:34:30.727 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:34:30.746+0000 7f5fea9bd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T06:34:48.558 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:34:48.562 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:34:57.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:57.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:34:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 Processor -- start 2024-05-09T06:34:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 -- start start 2024-05-09T06:34:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5bec15e190 0x7f5bec15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0b56d0 0x7f5bec0d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.231+0000 7f5bf1d3b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5bec161fe0 con 0x7f5bec0d1c70 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5bf1d3b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5bec1622c0 con 0x7f5bec15e190 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5bf1d3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec1625a0 con 0x7f5bec0b56d0 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44208/0 (socket says 172.21.15.1:44208) 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 -- 172.21.15.1:0/1886623349 learned_addr learned my addr 172.21.15.1:0/1886623349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0b56d0 0x7f5bec0d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1886623349 >> v2:172.21.15.6:3300/0 conn(0x7f5bec15e190 0x7f5bec15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 -- 172.21.15.1:0/1886623349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0b56d0 msgr2=0x7f5bec0d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0b56d0 0x7f5bec0d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 -- 172.21.15.1:0/1886623349 >> v2:172.21.15.6:3300/0 conn(0x7f5bec15e190 msgr2=0x7f5bec15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1886623349 >> v2:172.21.15.6:3300/0 conn(0x7f5bec15e190 0x7f5bec15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.232+0000 7f5beaffd640 1 -- 172.21.15.1:0/1886623349 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec162880 con 0x7f5bec0d1c70 2024-05-09T06:34:57.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.233+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0d1bf0 tx=0x7f5be002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ac6164cdab929a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.233+0000 7f5bea7fc640 1 -- 172.21.15.1:0/1886623349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0036070 con 0x7f5bec0d1c70 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.233+0000 7f5bea7fc640 1 -- 172.21.15.1:0/1886623349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0031600 con 0x7f5bec0d1c70 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.233+0000 7f5bea7fc640 1 -- 172.21.15.1:0/1886623349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00319d0 con 0x7f5bec0d1c70 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1886623349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec0d1c70 msgr2=0x7f5bec0d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0d1bf0 tx=0x7f5be002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1886623349 shutdown_connections 2024-05-09T06:34:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0b56d0 0x7f5bec0d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1886623349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec0d1c70 0x7f5bec0d3ee0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1886623349 >> v2:172.21.15.6:3300/0 conn(0x7f5bec15e190 0x7f5bec15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1886623349 >> 172.21.15.1:0/1886623349 conn(0x7f5bec073340 msgr2=0x7f5bec09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1886623349 shutdown_connections 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.234+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1886623349 wait complete. 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.235+0000 7f5bf1d3b640 1 Processor -- start 2024-05-09T06:34:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.235+0000 7f5bf1d3b640 1 -- start start 2024-05-09T06:34:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bec0b56d0 0x7f5bec0b9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 0x7f5bec0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0b3f60 con 0x7f5bec15e190 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0b3de0 con 0x7f5bec0b56d0 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bf1d3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec0b40e0 con 0x7f5bec0d1c70 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44220/0 (socket says 172.21.15.1:44220) 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 -- 172.21.15.1:0/1768635713 learned_addr learned my addr 172.21.15.1:0/1768635713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bec0b56d0 0x7f5bec0b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 0x7f5bec0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 -- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 msgr2=0x7f5bec0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 0x7f5bec0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 -- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bec0b56d0 msgr2=0x7f5bec0b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bec0b56d0 0x7f5bec0b9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5bebfff640 1 -- 172.21.15.1:0/1768635713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec0cbd40 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.236+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 0x7f5bec0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d620 tx=0x7f5be400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bd3fff640 1 -- 172.21.15.1:0/1768635713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4009600 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bd3fff640 1 -- 172.21.15.1:0/1768635713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4019070 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec0cc020 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bd3fff640 1 -- 172.21.15.1:0/1768635713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4009a00 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.237+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec0c9f40 con 0x7f5bec15e190 2024-05-09T06:34:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.239+0000 7f5bd3fff640 1 -- 172.21.15.1:0/1768635713 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5be4009ba0 con 0x7f5bec15e190 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.239+0000 7f5bd3fff640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 0x7f5bd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.239+0000 7f5bd3fff640 1 -- 172.21.15.1:0/1768635713 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f5be408b300 con 0x7f5bec15e190 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.239+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 0x7f5bd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.240+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 0x7f5bd806e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0079a0 tx=0x7f5bdc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.240+0000 7f5bf4a46740 1 Processor -- start 2024-05-09T06:34:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.240+0000 7f5bf4a46740 1 -- start start 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d378a650 0x55f2d373cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d376d770 0x55f2d376caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f2d36a6d00 con 0x55f2d376d030 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f2d364e4c0 con 0x55f2d378a650 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f2d36a2cc0 con 0x55f2d376d770 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44230/0 (socket says 172.21.15.1:44230) 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5bebfff640 1 -- 172.21.15.1:0/1017941231 learned_addr learned my addr 172.21.15.1:0/1017941231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d376d770 0x55f2d376caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.241+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d378a650 0x55f2d373cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 -- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d376d770 msgr2=0x55f2d376caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d376d770 0x55f2d376caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 -- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d378a650 msgr2=0x55f2d373cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d378a650 0x55f2d373cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 -- 172.21.15.1:0/1017941231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d34847d0 con 0x55f2d376d030 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5be4061b10 tx=0x7f5be4059b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d1ec3e22940ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bd17fa640 1 -- 172.21.15.1:0/1017941231 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be405d810 con 0x55f2d376d030 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bd17fa640 1 -- 172.21.15.1:0/1017941231 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4062040 con 0x55f2d376d030 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.242+0000 7f5bd17fa640 1 -- 172.21.15.1:0/1017941231 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4057760 con 0x55f2d376d030 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1017941231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 msgr2=0x55f2d3741e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5be4061b10 tx=0x7f5be4059b60 comp rx=0 tx=0).stop 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1017941231 shutdown_connections 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d030 0x55f2d3741e40 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d376d770 0x55f2d376caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1017941231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d378a650 0x55f2d373cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1017941231 >> 172.21.15.1:0/1017941231 conn(0x55f2d373f250 msgr2=0x55f2d3787af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1017941231 shutdown_connections 2024-05-09T06:34:57.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1017941231 wait complete. 2024-05-09T06:34:57.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 -- start start 2024-05-09T06:34:57.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.243+0000 7f5bf4a46740 1 Processor -- start 2024-05-09T06:34:57.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d3770970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d376c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f2d3741400 con 0x55f2d376e130 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f2d3661b90 con 0x55f2d376d770 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f2d37407f0 con 0x55f2d378a650 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bf4a46740 10 client.0 _ll_register_callbacks cb 0x55f2d369dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44246/0 (socket says 172.21.15.1:44246) 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5bebfff640 1 -- 172.21.15.1:0/1910210510 learned_addr learned my addr 172.21.15.1:0/1910210510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d3770970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.244+0000 7f5beaffd640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d376c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 -- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 msgr2=0x55f2d376c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d376c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 -- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 msgr2=0x55f2d3770970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d3770970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 -- 172.21.15.1:0/1910210510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d33ba3f0 con 0x55f2d376e130 2024-05-09T06:34:57.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bebfff640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5be4089730 tx=0x7f5be4006d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef625e7e08cd921b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4089370 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f5be4055ad0 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4062040 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f5be4078050 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 10 client.8280 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55f2d337a0f0 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4062040 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f5be4013070 con 0x55f2d376e130 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 10 client.8280 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 10 client.8280 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 20 client.8280 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d375da10 0x55f2d375b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 4 client.8280 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.245+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2d36b4430 con 0x55f2d375da10 2024-05-09T06:34:57.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.246+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d375da10 0x55f2d375b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.246+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d375da10 0x55f2d375b220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e246355955b827f server_cookie=40e0984e2edd39b1 in_seq=0 out_seq=0 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.246+0000 7f5bca7fc640 10 client.8280 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bca7fc640 1 -- 172.21.15.1:0/1910210510 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2d36b4430 con 0x55f2d375da10 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bca7fc640 10 client.8280 handle_command_reply: tid=0 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 client.8280 shutdown 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 msgr2=0x55f2d376c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5be4089730 tx=0x7f5be4006d30 comp rx=0 tx=0).stop 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 shutdown_connections 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d376c8e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d375da10 0x55f2d375b220 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d376c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/1910210510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d3770970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 >> 172.21.15.1:0/1910210510 conn(0x55f2d376d030 msgr2=0x55f2d373de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bca7fc640 0 client.8280 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 shutdown_connections 2024-05-09T06:34:57.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 1 -- 172.21.15.1:0/1910210510 wait complete. 2024-05-09T06:34:57.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.247+0000 7f5bf4a46740 20 client.8280 trim_cache size 0 max 16384 2024-05-09T06:34:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 Processor -- start 2024-05-09T06:34:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 -- start start 2024-05-09T06:34:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d379a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d3767fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec200e50 con 0x55f2d376e130 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1ffe50 con 0x55f2d376d770 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.265+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec0b2e60 con 0x55f2d378a650 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44248/0 (socket says 172.21.15.1:44248) 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 -- 172.21.15.1:0/3295977034 learned_addr learned my addr 172.21.15.1:0/3295977034 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5bebfff640 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d3767fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d379a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 -- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 msgr2=0x55f2d3767fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d3767fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 -- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 msgr2=0x55f2d379a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d379a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 -- 172.21.15.1:0/3295977034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d3789270 con 0x55f2d376e130 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009f40 tx=0x7f5be002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9632a4d5552ae41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5bd0ff9640 1 -- 172.21.15.1:0/3295977034 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0036040 con 0x55f2d376e130 2024-05-09T06:34:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5bd0ff9640 1 -- 172.21.15.1:0/3295977034 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0042070 con 0x55f2d376e130 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.266+0000 7f5bd0ff9640 1 -- 172.21.15.1:0/3295977034 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00319d0 con 0x55f2d376e130 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3295977034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 msgr2=0x55f2d3767a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009f40 tx=0x7f5be002e000 comp rx=0 tx=0).stop 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3295977034 shutdown_connections 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d3767fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376e130 0x55f2d3767a60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3295977034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376d770 0x55f2d379a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3295977034 >> 172.21.15.1:0/3295977034 conn(0x55f2d376d030 msgr2=0x55f2d37ab7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3295977034 shutdown_connections 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.267+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3295977034 wait complete. 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 -- start start 2024-05-09T06:34:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 Processor -- start 2024-05-09T06:34:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376e130 0x55f2d3801580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d37e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0b34b0 con 0x55f2d376d770 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f2d37709b0 con 0x55f2d376e130 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f2d3787ea0 con 0x55f2d378a650 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.268+0000 7f5bf4a46740 10 client.0 _ll_register_callbacks cb 0x7f5be4070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44258/0 (socket says 172.21.15.1:44258) 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 -- 172.21.15.1:0/3651964953 learned_addr learned my addr 172.21.15.1:0/3651964953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376e130 0x55f2d3801580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bebfff640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d37e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 -- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 msgr2=0x55f2d37e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d37e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 -- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376e130 msgr2=0x55f2d3801580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376e130 0x55f2d3801580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 -- 172.21.15.1:0/3651964953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d37e6c00 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5beb7fe640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc010c50 tx=0x7f5bdc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b8b12a340e6c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc030060 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f5bdc02bb50 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be0042070 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc036050 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bf4a46740 10 client.8286 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.269+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55f2d3787b80 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0042070 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f5bdc039020 con 0x55f2d376d770 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 10 client.8286 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 10 client.8286 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 20 client.8286 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d376fee0 0x55f2d3767a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 4 client.8286 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2d36ae780 con 0x55f2d376fee0 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d376fee0 0x55f2d3767a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bca7fc640 10 client.8286 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5beaffd640 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d376fee0 0x55f2d3767a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d803156c04dde348 server_cookie=76ec66be58eaf0cc in_seq=0 out_seq=0 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bca7fc640 1 -- 172.21.15.1:0/3651964953 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f2d36ae780 con 0x55f2d376fee0 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bca7fc640 10 client.8286 handle_command_reply: tid=0 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 1 client.8286 shutdown 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 msgr2=0x55f2d37ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.270+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc010c50 tx=0x7f5bdc013000 comp rx=0 tx=0).stop 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 shutdown_connections 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55f2d376fee0 0x55f2d3767a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f2d378a650 0x55f2d37e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f2d376e130 0x55f2d3801580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bca7fc640 0 client.8286 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 --2- 172.21.15.1:0/3651964953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2d376d770 0x55f2d37ab1f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 >> 172.21.15.1:0/3651964953 conn(0x55f2d376d030 msgr2=0x55f2d379a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 shutdown_connections 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 1 -- 172.21.15.1:0/3651964953 wait complete. 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf4a46740 20 client.8286 trim_cache size 0 max 16384 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:34:57.258 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:57.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 msgr2=0x7f5bd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 0x7f5bd806e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0079a0 tx=0x7f5bdc008040 comp rx=0 tx=0).stop 2024-05-09T06:34:57.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 msgr2=0x7f5bec0cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5be400d620 tx=0x7f5be400dae0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 shutdown_connections 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5bd806be30 0x7f5bd806e2f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bec15e190 0x7f5bec0cce40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5bec0d1c70 0x7f5bec0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 --2- 172.21.15.1:0/1768635713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bec0b56d0 0x7f5bec0b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 >> 172.21.15.1:0/1768635713 conn(0x7f5bec073340 msgr2=0x7f5bec0d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 shutdown_connections 2024-05-09T06:34:57.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.271+0000 7f5bf1d3b640 1 -- 172.21.15.1:0/1768635713 wait complete. 2024-05-09T06:34:57.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:34:57.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:34:57.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.386+0000 7f4343909640 1 Processor -- start 2024-05-09T06:34:57.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.386+0000 7f4343909640 1 -- start start 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4343909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4343909640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f433c0cdab0 0x7f433c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4343909640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c0ce0e0 con 0x7f433c06e6d0 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4343909640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f433c0ce3c0 con 0x7f433c0cdab0 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4343909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f433c0ce6a0 con 0x7f433c0cf180 2024-05-09T06:34:57.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4341ea6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f433c0cdab0 0x7f433c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.387+0000 7f4340ea4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f43416a5640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f433c06e6d0 0x7f433c06eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:44074/0 (socket says 172.21.15.1:44074) 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4340ea4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:55578/0 (socket says 172.21.15.1:55578) 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f43416a5640 1 -- 172.21.15.1:0/3514434366 learned_addr learned my addr 172.21.15.1:0/3514434366 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4341ea6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f433c0cdab0 0x7f433c0b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44274/0 (socket says 172.21.15.1:44274) 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4341ea6640 1 -- 172.21.15.1:0/3514434366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 msgr2=0x7f433c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4341ea6640 1 --2- 172.21.15.1:0/3514434366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4341ea6640 1 -- 172.21.15.1:0/3514434366 >> v1:172.21.15.6:6789/0 conn(0x7f433c06e6d0 legacy=0x7f433c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f4341ea6640 1 -- 172.21.15.1:0/3514434366 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c0ce980 con 0x7f433c0cdab0 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.388+0000 7f433bfff640 1 -- 172.21.15.1:0/3514434366 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893024974 0 0) 0x7f433c0ce0e0 con 0x7f433c06e6d0 2024-05-09T06:34:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4341ea6640 1 --2- 172.21.15.1:0/3514434366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c0b3300 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f432c009d90 tx=0x7f432c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce43ac0640b457f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f433bfff640 1 -- 172.21.15.1:0/3514434366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c007740 con 0x7f433c0cdab0 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f433bfff640 1 -- 172.21.15.1:0/3514434366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c0078e0 con 0x7f433c0cdab0 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f433bfff640 1 -- 172.21.15.1:0/3514434366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c007cb0 con 0x7f433c0cdab0 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 -- 172.21.15.1:0/3514434366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 msgr2=0x7f433c0b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 --2- 172.21.15.1:0/3514434366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c0b3300 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f432c009d90 tx=0x7f432c02a660 comp rx=0 tx=0).stop 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 -- 172.21.15.1:0/3514434366 shutdown_connections 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 --2- 172.21.15.1:0/3514434366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c0b3300 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 --2- 172.21.15.1:0/3514434366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 -- 172.21.15.1:0/3514434366 >> 172.21.15.1:0/3514434366 conn(0x7f433c0cc4b0 msgr2=0x7f433c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 -- 172.21.15.1:0/3514434366 shutdown_connections 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.389+0000 7f4343909640 1 -- 172.21.15.1:0/3514434366 wait complete. 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 Processor -- start 2024-05-09T06:34:57.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 -- start start 2024-05-09T06:34:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f433c06e6d0 0x7f433c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f433c1447c0 con 0x7f433c0cdab0 2024-05-09T06:34:57.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f433c070ed0 con 0x7f433c06e6d0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4343909640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f433c144940 con 0x7f433c0cf180 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4340ea4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4340ea4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44284/0 (socket says 172.21.15.1:44284) 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.391+0000 7f4340ea4640 1 -- 172.21.15.1:0/949145016 learned_addr learned my addr 172.21.15.1:0/949145016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f43416a5640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f433c06e6d0 0x7f433c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4341ea6640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 -- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 msgr2=0x7f433c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 -- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f433c06e6d0 msgr2=0x7f433c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f433c06e6d0 0x7f433c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 -- 172.21.15.1:0/949145016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c0aa390 con 0x7f433c0cdab0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4341ea6640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4340ea4640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9b0 tx=0x7f433000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017070 con 0x7f433c0cdab0 2024-05-09T06:34:57.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433000c9a0 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c0aabb0 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd90 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.392+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c0ab170 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.393+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c0aa9d0 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.394+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f433001a070 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.394+0000 7f4339ffb640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.394+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f433008b390 con 0x7f433c0cdab0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.394+0000 7f43416a5640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.395+0000 7f43416a5640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 0x7f432806e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4334000f60 tx=0x7f4334007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.399+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f433c0aa9d0 con 0x7f433c0cdab0 2024-05-09T06:34:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.617+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f433c0ce0e0 con 0x7f433c0cdab0 2024-05-09T06:34:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.618+0000 7f4339ffb640 1 -- 172.21.15.1:0/949145016 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5170 (secure 0 0 0) 0x7f433c0ce0e0 con 0x7f433c0cdab0 2024-05-09T06:34:57.599 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:57.599 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:30.731462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":11,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":6}]} 2024-05-09T06:34:57.600 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 68 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 msgr2=0x7f432806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 0x7f432806e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4334000f60 tx=0x7f4334007e40 comp rx=0 tx=0).stop 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 msgr2=0x7f433c15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9b0 tx=0x7f433000ee70 comp rx=0 tx=0).stop 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 shutdown_connections 2024-05-09T06:34:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f432806be30 0x7f432806e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f433c0cf180 0x7f433c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f433c0cdab0 0x7f433c15e3c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 --2- 172.21.15.1:0/949145016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f433c06e6d0 0x7f433c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 >> 172.21.15.1:0/949145016 conn(0x7f433c0cc4b0 msgr2=0x7f433c0af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 shutdown_connections 2024-05-09T06:34:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.620+0000 7f4343909640 1 -- 172.21.15.1:0/949145016 wait complete. 2024-05-09T06:34:57.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-09T06:34:57.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 Processor -- start 2024-05-09T06:34:57.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 -- start start 2024-05-09T06:34:57.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f36b815e190 0x7f36b815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f36b80d1c70 0x7f36b80d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b8161fe0 con 0x7f36b80b56d0 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.737+0000 7f36bdc37640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f36b815f580 con 0x7f36b815e190 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36bdc37640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f36b81622c0 con 0x7f36b80d1c70 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f36b815e190 0x7f36b815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f36b815e190 0x7f36b815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57452/0 (socket says 172.21.15.1:57452) 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 -- 172.21.15.1:0/3987907556 learned_addr learned my addr 172.21.15.1:0/3987907556 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/3987907556 >> v2:172.21.15.6:3300/0 conn(0x7f36b80d1c70 0x7f36b80d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 -- 172.21.15.1:0/3987907556 >> v2:172.21.15.6:3300/0 conn(0x7f36b80d1c70 msgr2=0x7f36b80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 --2- 172.21.15.1:0/3987907556 >> v2:172.21.15.6:3300/0 conn(0x7f36b80d1c70 0x7f36b80d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 -- 172.21.15.1:0/3987907556 >> v1:172.21.15.6:6789/0 conn(0x7f36b80b56d0 legacy=0x7f36b80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.738+0000 7f36b77fe640 1 -- 172.21.15.1:0/3987907556 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b81625a0 con 0x7f36b815e190 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36b77fe640 1 --2- 172.21.15.1:0/3987907556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b815f780 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f36b80b6c30 tx=0x7f36a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c0be1496fc27cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36b67fc640 1 -- 172.21.15.1:0/3987907556 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8037070 con 0x7f36b815e190 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36b67fc640 1 -- 172.21.15.1:0/3987907556 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8032440 con 0x7f36b815e190 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36b67fc640 1 -- 172.21.15.1:0/3987907556 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8032810 con 0x7f36b815e190 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 -- 172.21.15.1:0/3987907556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 msgr2=0x7f36b815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 --2- 172.21.15.1:0/3987907556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b815f780 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f36b80b6c30 tx=0x7f36a802a270 comp rx=0 tx=0).stop 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 -- 172.21.15.1:0/3987907556 shutdown_connections 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 --2- 172.21.15.1:0/3987907556 >> v2:172.21.15.6:3300/0 conn(0x7f36b80d1c70 0x7f36b80d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 --2- 172.21.15.1:0/3987907556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b815f780 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 -- 172.21.15.1:0/3987907556 >> 172.21.15.1:0/3987907556 conn(0x7f36b8073340 msgr2=0x7f36b809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 -- 172.21.15.1:0/3987907556 shutdown_connections 2024-05-09T06:34:57.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.739+0000 7f36bdc37640 1 -- 172.21.15.1:0/3987907556 wait complete. 2024-05-09T06:34:57.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.741+0000 7f36bdc37640 1 Processor -- start 2024-05-09T06:34:57.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.741+0000 7f36bdc37640 1 -- start start 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.741+0000 7f36bdc37640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36b80b56d0 0x7f36b80ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.741+0000 7f36bdc37640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36b80d1c70 0x7f36b80b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.741+0000 7f36bdc37640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36bdc37640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80d4420 con 0x7f36b815e190 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36bdc37640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8161c60 con 0x7f36b80b56d0 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36bdc37640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f36b80d45a0 con 0x7f36b80d1c70 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57454/0 (socket says 172.21.15.1:57454) 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 -- 172.21.15.1:0/2009754491 learned_addr learned my addr 172.21.15.1:0/2009754491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b77fe640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36b80b56d0 0x7f36b80ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36b80d1c70 0x7f36b80b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 -- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36b80d1c70 msgr2=0x7f36b80b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36b80d1c70 0x7f36b80b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 -- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36b80b56d0 msgr2=0x7f36b80ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36b80b56d0 0x7f36b80ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 -- 172.21.15.1:0/2009754491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8152a10 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36b7fff640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c760 tx=0x7f36b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f369ffff640 1 -- 172.21.15.1:0/2009754491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0003b70 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f369ffff640 1 -- 172.21.15.1:0/2009754491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b0003d10 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b80caa20 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f369ffff640 1 -- 172.21.15.1:0/2009754491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b000aa30 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.742+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b80cef30 con 0x7f36b815e190 2024-05-09T06:34:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.744+0000 7f369ffff640 1 -- 172.21.15.1:0/2009754491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36b000abd0 con 0x7f36b815e190 2024-05-09T06:34:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.744+0000 7f369ffff640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 0x7f36a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.745+0000 7f369ffff640 1 -- 172.21.15.1:0/2009754491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f36b008adf0 con 0x7f36b815e190 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.745+0000 7f36b77fe640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 0x7f36a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.745+0000 7f36b77fe640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 0x7f36a406e320 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36a8034000 tx=0x7f36a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.745+0000 7f36c0a17740 1 Processor -- start 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.745+0000 7f36c0a17740 1 -- start start 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec56b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec86b60 0x55c57ec5b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c57ebbfa30 con 0x55c57ec872a0 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c57eb5d590 con 0x55c57ec86b60 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c57ebbb9f0 con 0x55c57eca4180 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57462/0 (socket says 172.21.15.1:57462) 2024-05-09T06:34:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.746+0000 7f36b6ffd640 1 -- 172.21.15.1:0/250056639 learned_addr learned my addr 172.21.15.1:0/250056639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b77fe640 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec56b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b7fff640 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec86b60 0x55c57ec5b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 -- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 msgr2=0x55c57ec56b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec56b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 -- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec86b60 msgr2=0x55c57ec5b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec86b60 0x55c57ec5b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 -- 172.21.15.1:0/250056639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e99f7c0 con 0x55c57ec872a0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f36ac007900 tx=0x7f36ac007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4758d44b1489fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f369d7fa640 1 -- 172.21.15.1:0/250056639 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac010bc0 con 0x55c57ec872a0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f369d7fa640 1 -- 172.21.15.1:0/250056639 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ac010d60 con 0x55c57ec872a0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.747+0000 7f369d7fa640 1 -- 172.21.15.1:0/250056639 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004230 con 0x55c57ec872a0 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- 172.21.15.1:0/250056639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 msgr2=0x55c57ec86620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f36ac007900 tx=0x7f36ac007dc0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- 172.21.15.1:0/250056639 shutdown_connections 2024-05-09T06:34:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec86b60 0x55c57ec5b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec86620 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 --2- 172.21.15.1:0/250056639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec56b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- 172.21.15.1:0/250056639 >> 172.21.15.1:0/250056639 conn(0x55c57ec58d80 msgr2=0x55c57eca1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- 172.21.15.1:0/250056639 shutdown_connections 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- 172.21.15.1:0/250056639 wait complete. 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 -- start start 2024-05-09T06:34:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.748+0000 7f36c0a17740 1 Processor -- start 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ec89070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c57ec5af30 con 0x55c57ec872a0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c57eb5b060 con 0x55c57ec886f0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c57ec5a320 con 0x55c57eca4180 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36c0a17740 10 client.0 _ll_register_callbacks cb 0x55c57eb7fdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57464/0 (socket says 172.21.15.1:57464) 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36b77fe640 1 -- 172.21.15.1:0/3174312012 learned_addr learned my addr 172.21.15.1:0/3174312012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.749+0000 7f36b7fff640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ec89070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 -- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 msgr2=0x55c57ec74650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec74650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 -- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 msgr2=0x55c57ec89070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ec89070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 -- 172.21.15.1:0/3174312012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e9fff50 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36b77fe640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000c00 tx=0x7f36a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e804da2d8df455 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a804d020 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f36a8049650 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ac010d60 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f36a8057050 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 10 client.8304 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c57e8930b0 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ac010d60 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f36a805a020 con 0x55c57ec872a0 2024-05-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 10 client.8304 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 10 client.8304 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 20 client.8304 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec84c70 0x55c57ec85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 4 client.8304 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.750+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57ebcd160 con 0x55c57ec84c70 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec84c70 0x55c57ec85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec84c70 0x55c57ec85050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e3907e44b8eba9 server_cookie=113fb93c183592b0 in_seq=0 out_seq=0 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36967fc640 10 client.8304 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36967fc640 1 -- 172.21.15.1:0/3174312012 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c57ebcd160 con 0x55c57ec84c70 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36967fc640 10 client.8304 handle_command_reply: tid=0 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.751+0000 7f36c0a17740 1 client.8304 shutdown 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 msgr2=0x55c57ec76b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f36a8000c00 tx=0x7f36a8004730 comp rx=0 tx=0).stop 2024-05-09T06:34:57.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 shutdown_connections 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ec89070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec84c70 0x55c57ec85050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57ec74650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36967fc640 0 client.8304 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 --2- 172.21.15.1:0/3174312012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec76b50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 >> 172.21.15.1:0/3174312012 conn(0x55c57ec86b60 msgr2=0x55c57ec56240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 shutdown_connections 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 1 -- 172.21.15.1:0/3174312012 wait complete. 2024-05-09T06:34:57.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.752+0000 7f36c0a17740 20 client.8304 trim_cache size 0 max 16384 2024-05-09T06:34:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.765+0000 7f36c0a17740 1 Processor -- start 2024-05-09T06:34:57.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.765+0000 7f36c0a17740 1 -- start start 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57eca1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57eca1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ffca0 con 0x55c57ec872a0 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80d0660 con 0x55c57ec886f0 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f36b80d0cb0 con 0x55c57eca4180 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57478/0 (socket says 172.21.15.1:57478) 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 -- 172.21.15.1:0/516994557 learned_addr learned my addr 172.21.15.1:0/516994557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 -- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 msgr2=0x55c57eca1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57eca1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 -- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 msgr2=0x55c57eca1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57eca1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.766+0000 7f36b77fe640 1 -- 172.21.15.1:0/516994557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57ec8aaf0 con 0x55c57ec872a0 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36b77fe640 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f36a802a860 tx=0x7f36a8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50183ab18d35749 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f369cff9640 1 -- 172.21.15.1:0/516994557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a804d040 con 0x55c57ec872a0 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f369cff9640 1 -- 172.21.15.1:0/516994557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8049650 con 0x55c57ec872a0 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f369cff9640 1 -- 172.21.15.1:0/516994557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8037040 con 0x55c57ec872a0 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 -- 172.21.15.1:0/516994557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 msgr2=0x55c57ec9e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f36a802a860 tx=0x7f36a8045000 comp rx=0 tx=0).stop 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 -- 172.21.15.1:0/516994557 shutdown_connections 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57eca4180 0x55c57eca1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57eca1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 --2- 172.21.15.1:0/516994557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57ec872a0 0x55c57ec9e020 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 -- 172.21.15.1:0/516994557 >> 172.21.15.1:0/516994557 conn(0x55c57ec86b60 msgr2=0x55c57ec74480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 -- 172.21.15.1:0/516994557 shutdown_connections 2024-05-09T06:34:57.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.767+0000 7f36c0a17740 1 -- 172.21.15.1:0/516994557 wait complete. 2024-05-09T06:34:57.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.768+0000 7f36c0a17740 1 -- start start 2024-05-09T06:34:57.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.768+0000 7f36c0a17740 1 Processor -- start 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57ec872a0 0x55c57ecc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ed1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c57ec88b70 con 0x55c57eca4180 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c57ec87ef0 con 0x55c57ec886f0 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55c57ec5b850 con 0x55c57ec872a0 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36c0a17740 10 client.0 _ll_register_callbacks cb 0x7f36a80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57494/0 (socket says 172.21.15.1:57494) 2024-05-09T06:34:57.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 -- 172.21.15.1:0/31386714 learned_addr learned my addr 172.21.15.1:0/31386714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b77fe640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57ec872a0 0x55c57ecc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 -- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57ec872a0 msgr2=0x55c57ecc4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57ec872a0 0x55c57ecc4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b6ffd640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ed1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 -- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 msgr2=0x55c57ed1e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ed1e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 -- 172.21.15.1:0/31386714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57ed1ee80 con 0x55c57eca4180 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36b7fff640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f36b0058e70 tx=0x7f36b0058900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe9a63cc99db435 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.769+0000 7f36967fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0055550 con 0x55c57eca4180 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f36b0058330 con 0x55c57eca4180 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a8049650 con 0x55c57eca4180 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f36b00556f0 con 0x55c57eca4180 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 10 client.8310 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:57.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c57eca5500 con 0x55c57eca4180 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8049650 con 0x55c57eca4180 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f36b005f3e0 con 0x55c57eca4180 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 10 client.8310 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 10 client.8310 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 20 client.8310 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec97d40 0x55c57eca0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 4 client.8310 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55c57ebc74b0 con 0x55c57ec97d40 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36b77fe640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec97d40 0x55c57eca0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36b77fe640 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec97d40 0x55c57eca0730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda012acb9d6bcb5 server_cookie=56c0b598b0a328d0 in_seq=0 out_seq=0 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.770+0000 7f36967fc640 10 client.8310 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36967fc640 1 -- 172.21.15.1:0/31386714 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x55c57ebc74b0 con 0x55c57ec97d40 2024-05-09T06:34:57.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36967fc640 10 client.8310 handle_command_reply: tid=0 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 client.8310 shutdown 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 msgr2=0x55c57eca4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f36b0058e70 tx=0x7f36b0058900 comp rx=0 tx=0).stop 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 shutdown_connections 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c57eca4180 0x55c57eca4a40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c57ec886f0 0x55c57ed1e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55c57ec97d40 0x55c57eca0730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 --2- 172.21.15.1:0/31386714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55c57ec872a0 0x55c57ecc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 >> 172.21.15.1:0/31386714 conn(0x55c57ec86b60 msgr2=0x55c57ec5a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36967fc640 0 client.8310 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 shutdown_connections 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 1 -- 172.21.15.1:0/31386714 wait complete. 2024-05-09T06:34:57.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36c0a17740 20 client.8310 trim_cache size 0 max 16384 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "backtrace", 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 2338625048, 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627776, 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir" 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:34:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "dir_frag", 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 3126914579, 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627776, 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "frag": "*", 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir" 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "backtrace", 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 3645043045, 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/testdir/hardlink" 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:57.760 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 msgr2=0x7f36a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 0x7f36a406e320 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36a8034000 tx=0x7f36a802e000 comp rx=0 tx=0).stop 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 msgr2=0x7f36b80c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.771+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c760 tx=0x7f36b000cc20 comp rx=0 tx=0).stop 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 shutdown_connections 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f36a406be60 0x7f36a406e320 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36b815e190 0x7f36b80c7bd0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36b80d1c70 0x7f36b80b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 --2- 172.21.15.1:0/2009754491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36b80b56d0 0x7f36b80ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 >> 172.21.15.1:0/2009754491 conn(0x7f36b8073340 msgr2=0x7f36b815c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 shutdown_connections 2024-05-09T06:34:57.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.772+0000 7f36bdc37640 1 -- 172.21.15.1:0/2009754491 wait complete. 2024-05-09T06:34:57.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:34:57.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.885+0000 7f8aabdc8640 1 Processor -- start 2024-05-09T06:34:57.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.886+0000 7f8aabdc8640 1 -- start start 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aabdc8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8aa4163c70 0x7f8aa4166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aabdc8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa40ce0e0 con 0x7f8aa40cf180 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aabdc8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa40ce3c0 con 0x7f8aa406e6d0 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aabdc8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa40ce6a0 con 0x7f8aa4163c70 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aaa365640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8aa4163c70 0x7f8aa4166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aa9363640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8aa40cf180 0x7f8aa40cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42308/0 (socket says 172.21.15.1:42308) 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aa9363640 1 -- 172.21.15.1:0/1613741599 learned_addr learned my addr 172.21.15.1:0/1613741599 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aaa365640 1 -- 172.21.15.1:0/1613741599 >> v1:172.21.15.6:6789/0 conn(0x7f8aa406e6d0 legacy=0x7f8aa406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aaa365640 1 -- 172.21.15.1:0/1613741599 >> v1:172.21.15.1:6789/0 conn(0x7f8aa40cf180 legacy=0x7f8aa40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aaa365640 1 -- 172.21.15.1:0/1613741599 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa40ce980 con 0x7f8aa4163c70 2024-05-09T06:34:57.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.887+0000 7f8aa8b62640 1 -- 172.21.15.1:0/1613741599 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944210932 0 0) 0x7f8aa40ce0e0 con 0x7f8aa40cf180 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aaa365640 1 --2- 172.21.15.1:0/1613741599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4163c70 0x7f8aa4166060 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d90 tx=0x7f8a9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656938e0a88702a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aa8b62640 1 -- 172.21.15.1:0/1613741599 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94007740 con 0x7f8aa4163c70 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aa8b62640 1 -- 172.21.15.1:0/1613741599 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a940078e0 con 0x7f8aa4163c70 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aa8b62640 1 -- 172.21.15.1:0/1613741599 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94007cb0 con 0x7f8aa4163c70 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aabdc8640 1 -- 172.21.15.1:0/1613741599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4163c70 msgr2=0x7f8aa4166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/1613741599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4163c70 0x7f8aa4166060 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d90 tx=0x7f8a9402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aabdc8640 1 -- 172.21.15.1:0/1613741599 shutdown_connections 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/1613741599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4163c70 0x7f8aa4166060 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.888+0000 7f8aabdc8640 1 -- 172.21.15.1:0/1613741599 >> 172.21.15.1:0/1613741599 conn(0x7f8aa40cc4b0 msgr2=0x7f8aa40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.889+0000 7f8aabdc8640 1 -- 172.21.15.1:0/1613741599 shutdown_connections 2024-05-09T06:34:57.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.889+0000 7f8aabdc8640 1 -- 172.21.15.1:0/1613741599 wait complete. 2024-05-09T06:34:57.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 Processor -- start 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 -- start start 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8aa40cf180 0x7f8aa4167a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8aa4163c70 0x7f8aa415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41447c0 con 0x7f8aa406e6d0 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4070ed0 con 0x7f8aa4163c70 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.890+0000 7f8aabdc8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa4144940 con 0x7f8aa40cf180 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57508/0 (socket says 172.21.15.1:57508) 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 -- 172.21.15.1:0/587226405 learned_addr learned my addr 172.21.15.1:0/587226405 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9363640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8aa40cf180 0x7f8aa4167a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 -- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8aa40cf180 msgr2=0x7f8aa4167a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8aa40cf180 0x7f8aa4167a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 -- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8aa4163c70 msgr2=0x7f8aa415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aaa365640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8aa4163c70 0x7f8aa415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8aa4163c70 0x7f8aa415a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 -- 172.21.15.1:0/587226405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa40ae440 con 0x7f8aa406e6d0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa9b64640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c430 tx=0x7f8a9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c017070 con 0x7f8aa406e6d0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c0052d0 con 0x7f8aa406e6d0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa40ae720 con 0x7f8aa406e6d0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c003830 con 0x7f8aa406e6d0 2024-05-09T06:34:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.891+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa40aece0 con 0x7f8aa406e6d0 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.892+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa406f680 con 0x7f8aa406e6d0 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.893+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a9c01a070 con 0x7f8aa406e6d0 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.893+0000 7f8aa27fc640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 0x7f8a9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.893+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8a9c08a320 con 0x7f8aa406e6d0 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.893+0000 7f8aa9363640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 0x7f8a9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.894+0000 7f8aa9363640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 0x7f8a9006e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007fd0 tx=0x7f8a98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:57.898+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8aa406f680 con 0x7f8aa406e6d0 2024-05-09T06:34:58.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.120+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8aa40752b0 con 0x7f8aa406e6d0 2024-05-09T06:34:58.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.120+0000 7f8aa27fc640 1 -- 172.21.15.1:0/587226405 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f8a9c058b80 con 0x7f8aa406e6d0 2024-05-09T06:34:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:34:58.101 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-09T06:34:58.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 msgr2=0x7f8a9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 0x7f8a9006e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007fd0 tx=0x7f8a98009040 comp rx=0 tx=0).stop 2024-05-09T06:34:58.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 msgr2=0x7f8aa4166f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c430 tx=0x7f8a9c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 shutdown_connections 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8a9006be30 0x7f8a9006e2f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8aa4163c70 0x7f8aa415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8aa40cf180 0x7f8aa4167a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 --2- 172.21.15.1:0/587226405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa406e6d0 0x7f8aa4166f30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 >> 172.21.15.1:0/587226405 conn(0x7f8aa40cc4b0 msgr2=0x7f8aa41436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 shutdown_connections 2024-05-09T06:34:58.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.122+0000 7f8aabdc8640 1 -- 172.21.15.1:0/587226405 wait complete. 2024-05-09T06:34:58.114 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:34:58.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:58.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-09T06:34:58.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.241+0000 7f8c21eca640 1 Processor -- start 2024-05-09T06:34:58.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.241+0000 7f8c21eca640 1 -- start start 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c21eca640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c21eca640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c1c0d2a10 0x7f8c1c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c21eca640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1c0d1cc0 con 0x7f8c1c0b2ed0 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c21eca640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1c0d4d10 con 0x7f8c1c0d2630 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c21eca640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1c0d1f80 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.242+0000 7f8c1b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8c1c0b2ed0 0x7f8c1c0b1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42310/0 (socket says 172.21.15.1:42310) 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57514/0 (socket says 172.21.15.1:57514) 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1affd640 1 -- 172.21.15.1:0/3887711596 learned_addr learned my addr 172.21.15.1:0/3887711596 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/3887711596 >> v2:172.21.15.6:3300/0 conn(0x7f8c1c0d2a10 0x7f8c1c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3887711596 >> v2:172.21.15.6:3300/0 conn(0x7f8c1c0d2a10 msgr2=0x7f8c1c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/3887711596 >> v2:172.21.15.6:3300/0 conn(0x7f8c1c0d2a10 0x7f8c1c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3887711596 >> v1:172.21.15.1:6789/0 conn(0x7f8c1c0b2ed0 legacy=0x7f8c1c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3887711596 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c0b0c00 con 0x7f8c1c0d2630 2024-05-09T06:34:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.243+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/3887711596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c08e0e0 tx=0x7f8c0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83abad7ce9692e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c1a7fc640 1 -- 172.21.15.1:0/3887711596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037070 con 0x7f8c1c0d2630 2024-05-09T06:34:58.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c1a7fc640 1 -- 172.21.15.1:0/3887711596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c032440 con 0x7f8c1c0d2630 2024-05-09T06:34:58.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c1a7fc640 1 -- 172.21.15.1:0/3887711596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c032810 con 0x7f8c1c0d2630 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 -- 172.21.15.1:0/3887711596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2630 msgr2=0x7f8c1c15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 --2- 172.21.15.1:0/3887711596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c08e0e0 tx=0x7f8c0c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 -- 172.21.15.1:0/3887711596 shutdown_connections 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 --2- 172.21.15.1:0/3887711596 >> v2:172.21.15.6:3300/0 conn(0x7f8c1c0d2a10 0x7f8c1c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 --2- 172.21.15.1:0/3887711596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2630 0x7f8c1c15ddf0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 -- 172.21.15.1:0/3887711596 >> 172.21.15.1:0/3887711596 conn(0x7f8c1c0d3ee0 msgr2=0x7f8c1c0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 -- 172.21.15.1:0/3887711596 shutdown_connections 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.244+0000 7f8c21eca640 1 -- 172.21.15.1:0/3887711596 wait complete. 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.245+0000 7f8c21eca640 1 Processor -- start 2024-05-09T06:34:58.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 -- start start 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c1c0b2ed0 0x7f8c1c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c1c0d2630 0x7f8c1c09fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15e4b0 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15e330 con 0x7f8c1c0b2ed0 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.246+0000 7f8c21eca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c15e630 con 0x7f8c1c0d2630 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57520/0 (socket says 172.21.15.1:57520) 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 -- 172.21.15.1:0/160042439 learned_addr learned my addr 172.21.15.1:0/160042439 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c1c0b2ed0 0x7f8c1c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1affd640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c1c0d2630 0x7f8c1c09fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 -- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c1c0d2630 msgr2=0x7f8c1c09fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c1c0d2630 0x7f8c1c09fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 -- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c1c0b2ed0 msgr2=0x7f8c1c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c1c0b2ed0 0x7f8c1c06ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 -- 172.21.15.1:0/160042439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c0a0760 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007e60 tx=0x7f8c1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c03fff640 1 -- 172.21.15.1:0/160042439 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14014420 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c03fff640 1 -- 172.21.15.1:0/160042439 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c14019070 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c0a0cb0 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c03fff640 1 -- 172.21.15.1:0/160042439 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14014820 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.247+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c0a1270 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.249+0000 7f8c03fff640 1 -- 172.21.15.1:0/160042439 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c140149c0 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.249+0000 7f8c03fff640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 0x7f8c0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.249+0000 7f8c03fff640 1 -- 172.21.15.1:0/160042439 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8c1408b870 con 0x7f8c1c0d2a10 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.249+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 0x7f8c0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.250+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 0x7f8c0806e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c034000 tx=0x7f8c0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.250+0000 7f8c24bd6740 1 Processor -- start 2024-05-09T06:34:58.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.250+0000 7f8c24bd6740 1 -- start start 2024-05-09T06:34:58.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300aebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560300a4f990 con 0x560300aeb9b0 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5603009f75b0 con 0x560300b16470 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560300a4b950 con 0x560300b28080 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57528/0 (socket says 172.21.15.1:57528) 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/483291390 learned_addr learned my addr 172.21.15.1:0/483291390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c1affd640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300aebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.251+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 msgr2=0x560300aebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300aebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 msgr2=0x560300b168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/483291390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030082b7e0 con 0x560300aeb9b0 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a1e0 tx=0x7f8c0c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6b832fc735e90a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c017fa640 1 -- 172.21.15.1:0/483291390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c04e070 con 0x560300aeb9b0 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c017fa640 1 -- 172.21.15.1:0/483291390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c032d90 con 0x560300aeb9b0 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.252+0000 7f8c017fa640 1 -- 172.21.15.1:0/483291390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037040 con 0x560300aeb9b0 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- 172.21.15.1:0/483291390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 msgr2=0x560300aedb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a1e0 tx=0x7f8c0c02c000 comp rx=0 tx=0).stop 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- 172.21.15.1:0/483291390 shutdown_connections 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300aebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/483291390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300aedb10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- 172.21.15.1:0/483291390 >> 172.21.15.1:0/483291390 conn(0x560300ab30a0 msgr2=0x560300b31470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- 172.21.15.1:0/483291390 shutdown_connections 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- 172.21.15.1:0/483291390 wait complete. 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 -- start start 2024-05-09T06:34:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.253+0000 7f8c24bd6740 1 Processor -- start 2024-05-09T06:34:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300b2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560300a0ab40 con 0x560300aeb9b0 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560300b13fc0 con 0x560300b16470 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560300b2ca20 con 0x560300b28080 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c24bd6740 10 client.0 _ll_register_callbacks cb 0x5603009ff0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57540/0 (socket says 172.21.15.1:57540) 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.254+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/4029883957 learned_addr learned my addr 172.21.15.1:0/4029883957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1affd640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300b2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 msgr2=0x560300b2dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300b2dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 msgr2=0x560300b2da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b2da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/4029883957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030088bf70 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a790 tx=0x7f8c0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07782670a491890 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c04e030 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8c0c045040 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c032d90 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8c0c037040 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 10 client.8334 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560300723170 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c032d90 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f8c0c0498c0 con 0x560300aeb9b0 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 10 client.8334 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 10 client.8334 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 20 client.8334 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2b980 0x560300b2bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 4 client.8334 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.255+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560300a5d0c0 con 0x560300b2b980 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.256+0000 7f8c1affd640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2b980 0x560300b2bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.256+0000 7f8bfa7fc640 10 client.8334 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.256+0000 7f8c1affd640 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2b980 0x560300b2bd60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f652d712980471a6 server_cookie=b6b8193c0623c7f5 in_seq=0 out_seq=0 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.256+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/4029883957 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560300a5d0c0 con 0x560300b2b980 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.256+0000 7f8bfa7fc640 10 client.8334 handle_command_reply: tid=0 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 client.8334 shutdown 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 msgr2=0x560300b13090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a790 tx=0x7f8c0c004730 comp rx=0 tx=0).stop 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 shutdown_connections 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2b980 0x560300b2bd60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b28080 0x560300b2dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b16470 0x560300b2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8bfa7fc640 0 client.8334 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/4029883957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13090 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 >> 172.21.15.1:0/4029883957 conn(0x560300ab30a0 msgr2=0x560300b141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 shutdown_connections 2024-05-09T06:34:58.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 1 -- 172.21.15.1:0/4029883957 wait complete. 2024-05-09T06:34:58.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.257+0000 7f8c24bd6740 20 client.8334 trim_cache size 0 max 16384 2024-05-09T06:34:58.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.275+0000 7f8c24bd6740 1 Processor -- start 2024-05-09T06:34:58.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.275+0000 7f8c24bd6740 1 -- start start 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b54e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c200cc0 con 0x560300aeb9b0 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ffd10 con 0x560300b28080 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c1f4ac0 con 0x560300b16470 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57548/0 (socket says 172.21.15.1:57548) 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3110293287 learned_addr learned my addr 172.21.15.1:0/3110293287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1affd640 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b54e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 msgr2=0x560300b54e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b54e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 msgr2=0x560300b13b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b13b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.276+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/3110293287 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560300b16a00 con 0x560300aeb9b0 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0494a0 tx=0x7f8c0c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8811bdfb695af451 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c00ff9640 1 -- 172.21.15.1:0/3110293287 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c04e070 con 0x560300aeb9b0 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c00ff9640 1 -- 172.21.15.1:0/3110293287 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c050050 con 0x560300aeb9b0 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c00ff9640 1 -- 172.21.15.1:0/3110293287 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c03e4b0 con 0x560300aeb9b0 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 -- 172.21.15.1:0/3110293287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 msgr2=0x560300b0ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0494a0 tx=0x7f8c0c040000 comp rx=0 tx=0).stop 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 -- 172.21.15.1:0/3110293287 shutdown_connections 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b54e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/3110293287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b0ff60 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 -- 172.21.15.1:0/3110293287 >> 172.21.15.1:0/3110293287 conn(0x560300ab30a0 msgr2=0x560300ae1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.277+0000 7f8c24bd6740 1 -- 172.21.15.1:0/3110293287 shutdown_connections 2024-05-09T06:34:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.278+0000 7f8c24bd6740 1 -- 172.21.15.1:0/3110293287 wait complete. 2024-05-09T06:34:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.278+0000 7f8c24bd6740 1 -- start start 2024-05-09T06:34:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.278+0000 7f8c24bd6740 1 Processor -- start 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560300b34290 con 0x560300aeb9b0 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ff6c0 con 0x560300b28080 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560300aecac0 con 0x560300b16470 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c24bd6740 10 client.0 _ll_register_callbacks cb 0x7f8c0c04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57552/0 (socket says 172.21.15.1:57552) 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/1478823244 learned_addr learned my addr 172.21.15.1:0/1478823244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1affd640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1bfff640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 msgr2=0x560300b140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 msgr2=0x560300b3fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b3fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 -- 172.21.15.1:0/1478823244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560300b27050 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8c1b7fe640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c035040 tx=0x7f8c0c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6cc028bbc7fc85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.279+0000 7f8bfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c04e070 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f8c0c05fd10 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c050050 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8c0c03e670 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 10 client.8340 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560300b40af0 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c050050 con 0x560300aeb9b0 2024-05-09T06:34:58.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f8c0c03e810 con 0x560300aeb9b0 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 10 client.8340 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 10 client.8340 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 20 client.8340 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2de60 0x560300b400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 4 client.8340 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x560300a57410 con 0x560300b2de60 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c1affd640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2de60 0x560300b400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8bfa7fc640 10 client.8340 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.280+0000 7f8c1affd640 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2de60 0x560300b400d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c61cab0beff961e server_cookie=e9d4ac1aeb8f52cb in_seq=0 out_seq=0 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8bfa7fc640 1 -- 172.21.15.1:0/1478823244 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560300a57410 con 0x560300b2de60 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8bfa7fc640 10 client.8340 handle_command_reply: tid=0 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 client.8340 shutdown 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 msgr2=0x560300b13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c035040 tx=0x7f8c0c008030 comp rx=0 tx=0).stop 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 shutdown_connections 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560300b28080 0x560300b3fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x560300b2de60 0x560300b400d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560300b16470 0x560300b140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 --2- 172.21.15.1:0/1478823244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560300aeb9b0 0x560300b13b70 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 >> 172.21.15.1:0/1478823244 conn(0x560300ab30a0 msgr2=0x560300b33240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8bfa7fc640 0 client.8340 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 shutdown_connections 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 1 -- 172.21.15.1:0/1478823244 wait complete. 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.281+0000 7f8c24bd6740 20 client.8340 trim_cache size 0 max 16384 2024-05-09T06:34:58.268 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:58.269 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:34:58.269 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "d13ae69c-4fb3-47f6-acc9-4f362b1f126c", 2024-05-09T06:34:58.269 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:34:58.269 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:58.269 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f0ee16dc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)->f(v1 m2024-05-09T06:34:15.118383+0000) n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)->n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564e3d142c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 msgr2=0x7f8c0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 0x7f8c0806e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c034000 tx=0x7f8c0c02e000 comp rx=0 tx=0).stop 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 msgr2=0x7f8c1c0a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007e60 tx=0x7f8c1400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 shutdown_connections 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8c0806be30 0x7f8c0806e2f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c1c0d2a10 0x7f8c1c0a0220 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c1c0d2630 0x7f8c1c09fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 --2- 172.21.15.1:0/160042439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c1c0b2ed0 0x7f8c1c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 >> 172.21.15.1:0/160042439 conn(0x7f8c1c0d3ee0 msgr2=0x7f8c1c160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 shutdown_connections 2024-05-09T06:34:58.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.282+0000 7f8c21eca640 1 -- 172.21.15.1:0/160042439 wait complete. 2024-05-09T06:34:58.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:34:58.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:34:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.401+0000 7f653aa72640 1 Processor -- start 2024-05-09T06:34:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.401+0000 7f653aa72640 1 -- start start 2024-05-09T06:34:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653aa72640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6534158a50 0x7f65340718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653aa72640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f653406ede0 0x7f6534071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653aa72640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653415b420 con 0x7f653415c4a0 2024-05-09T06:34:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653aa72640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6534158e30 con 0x7f6534158a50 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653aa72640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6534072480 con 0x7f653406ede0 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6534158a50 0x7f65340718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f6533fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f653415c4a0 0x7f653406e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42324/0 (socket says 172.21.15.1:42324) 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6534158a50 0x7f65340718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57568/0 (socket says 172.21.15.1:57568) 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f6533fff640 1 -- 172.21.15.1:0/3993034604 learned_addr learned my addr 172.21.15.1:0/3993034604 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653900f640 1 --2- 172.21.15.1:0/3993034604 >> v2:172.21.15.6:3300/0 conn(0x7f653406ede0 0x7f6534071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 -- 172.21.15.1:0/3993034604 >> v2:172.21.15.6:3300/0 conn(0x7f653406ede0 msgr2=0x7f6534071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 --2- 172.21.15.1:0/3993034604 >> v2:172.21.15.6:3300/0 conn(0x7f653406ede0 0x7f6534071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 -- 172.21.15.1:0/3993034604 >> v1:172.21.15.1:6789/0 conn(0x7f653415c4a0 legacy=0x7f653406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f65337fe640 1 -- 172.21.15.1:0/3993034604 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614173670 0 0) 0x7f653415b420 con 0x7f653415c4a0 2024-05-09T06:34:58.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.402+0000 7f653880e640 1 -- 172.21.15.1:0/3993034604 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653415b6e0 con 0x7f6534158a50 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.403+0000 7f653880e640 1 --2- 172.21.15.1:0/3993034604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340718c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f65340741b0 tx=0x7f652802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64dacc971e1d6ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.403+0000 7f65337fe640 1 -- 172.21.15.1:0/3993034604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037070 con 0x7f6534158a50 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.403+0000 7f65337fe640 1 -- 172.21.15.1:0/3993034604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528032440 con 0x7f6534158a50 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.403+0000 7f65337fe640 1 -- 172.21.15.1:0/3993034604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528032810 con 0x7f6534158a50 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 -- 172.21.15.1:0/3993034604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 msgr2=0x7f65340718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 --2- 172.21.15.1:0/3993034604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340718c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f65340741b0 tx=0x7f652802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 -- 172.21.15.1:0/3993034604 shutdown_connections 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 --2- 172.21.15.1:0/3993034604 >> v2:172.21.15.6:3300/0 conn(0x7f653406ede0 0x7f6534071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 --2- 172.21.15.1:0/3993034604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340718c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 -- 172.21.15.1:0/3993034604 >> 172.21.15.1:0/3993034604 conn(0x7f65340cc4d0 msgr2=0x7f65340cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 -- 172.21.15.1:0/3993034604 shutdown_connections 2024-05-09T06:34:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.404+0000 7f653aa72640 1 -- 172.21.15.1:0/3993034604 wait complete. 2024-05-09T06:34:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.405+0000 7f653aa72640 1 Processor -- start 2024-05-09T06:34:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.405+0000 7f653aa72640 1 -- start start 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f653406ede0 0x7f65340c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 0x7f653414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f653415bf80 con 0x7f6534158a50 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f653415bca0 con 0x7f653415c4a0 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653aa72640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6534070ea0 con 0x7f653406ede0 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57578/0 (socket says 172.21.15.1:57578) 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 -- 172.21.15.1:0/482155066 learned_addr learned my addr 172.21.15.1:0/482155066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653880e640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f653406ede0 0x7f65340c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653900f640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 0x7f653414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 -- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f653406ede0 msgr2=0x7f65340c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f653406ede0 0x7f65340c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 -- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 msgr2=0x7f653414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 0x7f653414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 -- 172.21.15.1:0/482155066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65340ae780 con 0x7f6534158a50 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f653900f640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 0x7f653414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.406+0000 7f6533fff640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f652000c430 tx=0x7f652000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.407+0000 7f65317fa640 1 -- 172.21.15.1:0/482155066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520017070 con 0x7f6534158a50 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.407+0000 7f65317fa640 1 -- 172.21.15.1:0/482155066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6520005ce0 con 0x7f6534158a50 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.407+0000 7f65317fa640 1 -- 172.21.15.1:0/482155066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520007310 con 0x7f6534158a50 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.407+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65340aea60 con 0x7f6534158a50 2024-05-09T06:34:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.407+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65340af020 con 0x7f6534158a50 2024-05-09T06:34:58.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.408+0000 7f65317fa640 1 -- 172.21.15.1:0/482155066 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f652001a070 con 0x7f6534158a50 2024-05-09T06:34:58.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.409+0000 7f65317fa640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 0x7f651806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.409+0000 7f65317fa640 1 -- 172.21.15.1:0/482155066 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f652008a400 con 0x7f6534158a50 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.409+0000 7f653880e640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 0x7f651806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.409+0000 7f653880e640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 0x7f651806e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f652802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.409+0000 7f653d77d740 1 Processor -- start 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 -- start start 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 0x55bf4f0d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf4f03dd00 con 0x55bf4f104770 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55bf4efe54c0 con 0x55bf4f104030 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55bf4f039cc0 con 0x55bf4f121650 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57580/0 (socket says 172.21.15.1:57580) 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.410+0000 7f6533fff640 1 -- 172.21.15.1:0/695580214 learned_addr learned my addr 172.21.15.1:0/695580214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f653880e640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f653900f640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 0x55bf4f0d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 -- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 msgr2=0x55bf4f0d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 -- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 msgr2=0x55bf4f0d8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 0x55bf4f0d8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 -- 172.21.15.1:0/695580214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4ee1b7d0 con 0x55bf4f104770 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f653900f640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 0x55bf4f0d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f6533fff640 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6520058570 tx=0x7f6520058660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb30c1598f0b7d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f651e7fc640 1 -- 172.21.15.1:0/695580214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200567f0 con 0x55bf4f104770 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f651e7fc640 1 -- 172.21.15.1:0/695580214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652008a0d0 con 0x55bf4f104770 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f651e7fc640 1 -- 172.21.15.1:0/695580214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200074b0 con 0x55bf4f104770 2024-05-09T06:34:58.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f653d77d740 1 -- 172.21.15.1:0/695580214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 msgr2=0x55bf4f103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.411+0000 7f653d77d740 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6520058570 tx=0x7f6520058660 comp rx=0 tx=0).stop 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 -- 172.21.15.1:0/695580214 shutdown_connections 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104030 0x55bf4f0d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f103af0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 --2- 172.21.15.1:0/695580214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 -- 172.21.15.1:0/695580214 >> 172.21.15.1:0/695580214 conn(0x55bf4f0d6250 msgr2=0x55bf4f11eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 -- 172.21.15.1:0/695580214 shutdown_connections 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 -- 172.21.15.1:0/695580214 wait complete. 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 -- start start 2024-05-09T06:34:58.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.412+0000 7f653d77d740 1 Processor -- start 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f105bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f120240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf4f0d8400 con 0x55bf4f104770 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55bf4eff8b90 con 0x55bf4f121650 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55bf4f0d77f0 con 0x55bf4f11d020 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653d77d740 10 client.0 _ll_register_callbacks cb 0x55bf4f034da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653880e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653880e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57584/0 (socket says 172.21.15.1:57584) 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653880e640 1 -- 172.21.15.1:0/4056791988 learned_addr learned my addr 172.21.15.1:0/4056791988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f6533fff640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f105bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.413+0000 7f653900f640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f120240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 -- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 msgr2=0x55bf4f120240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f120240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 -- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 msgr2=0x55bf4f105bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f105bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 -- 172.21.15.1:0/4056791988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4ed513f0 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653880e640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f652802a790 tx=0x7f652802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cb88d2cbdcedae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037040 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f652803c850 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652008a0d0 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6528056050 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 10 client.8355 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55bf4ed110f0 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652008a0d0 con 0x55bf4f104770 2024-05-09T06:34:58.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f6528059020 con 0x55bf4f104770 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 10 client.8355 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 10 client.8355 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 20 client.8355 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f120780 0x55bf4f120b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 4 client.8355 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.414+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf4f04b430 con 0x55bf4f120780 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f6533fff640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f120780 0x55bf4f120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f65137fe640 10 client.8355 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f6533fff640 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f120780 0x55bf4f120b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adee609042545939 server_cookie=84d648720d059ac1 in_seq=0 out_seq=0 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f65137fe640 1 -- 172.21.15.1:0/4056791988 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf4f04b430 con 0x55bf4f120780 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f65137fe640 10 client.8355 handle_command_reply: tid=0 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.415+0000 7f653d77d740 1 client.8355 shutdown 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 msgr2=0x55bf4f1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f652802a790 tx=0x7f652802c070 comp rx=0 tx=0).stop 2024-05-09T06:34:58.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 shutdown_connections 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f120780 0x55bf4f120b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f120240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f65137fe640 0 client.8355 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 --2- 172.21.15.1:0/4056791988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f1056b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 >> 172.21.15.1:0/4056791988 conn(0x55bf4f104030 msgr2=0x55bf4f100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 shutdown_connections 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 1 -- 172.21.15.1:0/4056791988 wait complete. 2024-05-09T06:34:58.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.416+0000 7f653d77d740 20 client.8355 trim_cache size 0 max 16384 2024-05-09T06:34:58.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.433+0000 7f653d77d740 1 Processor -- start 2024-05-09T06:34:58.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.433+0000 7f653d77d740 1 -- start start 2024-05-09T06:34:58.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 0x55bf4f120b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6534200af0 con 0x55bf4f11d020 2024-05-09T06:34:58.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65341ffbb0 con 0x55bf4f104770 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f65340b47c0 con 0x55bf4f121650 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57592/0 (socket says 172.21.15.1:57592) 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 -- 172.21.15.1:0/1797206609 learned_addr learned my addr 172.21.15.1:0/1797206609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 -- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 msgr2=0x55bf4f0d4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653880e640 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 0x55bf4f120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 -- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 msgr2=0x55bf4f120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 0x55bf4f120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f6533fff640 1 -- 172.21.15.1:0/1797206609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4f102b70 con 0x55bf4f11d020 2024-05-09T06:34:58.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.434+0000 7f653880e640 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 0x55bf4f120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f6533fff640 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f652005c9b0 tx=0x7f6520089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba06f1315370d8d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f651dffb640 1 -- 172.21.15.1:0/1797206609 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520054b60 con 0x55bf4f11d020 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f651dffb640 1 -- 172.21.15.1:0/1797206609 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200074b0 con 0x55bf4f11d020 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f651dffb640 1 -- 172.21.15.1:0/1797206609 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520054d00 con 0x55bf4f11d020 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 -- 172.21.15.1:0/1797206609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 msgr2=0x55bf4f0fd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f652005c9b0 tx=0x7f6520089000 comp rx=0 tx=0).stop 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 -- 172.21.15.1:0/1797206609 shutdown_connections 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f121650 0x55bf4f0d4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f11d020 0x55bf4f0fd9d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 --2- 172.21.15.1:0/1797206609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f104770 0x55bf4f120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 -- 172.21.15.1:0/1797206609 >> 172.21.15.1:0/1797206609 conn(0x55bf4f104030 msgr2=0x55bf4f106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 -- 172.21.15.1:0/1797206609 shutdown_connections 2024-05-09T06:34:58.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.435+0000 7f653d77d740 1 -- 172.21.15.1:0/1797206609 wait complete. 2024-05-09T06:34:58.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.436+0000 7f653d77d740 1 -- start start 2024-05-09T06:34:58.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.436+0000 7f653d77d740 1 Processor -- start 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f105990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf4f142f80 con 0x55bf4f104770 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65340b4e10 con 0x55bf4f121650 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55bf4f1224b0 con 0x55bf4f11d020 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653d77d740 10 client.0 _ll_register_callbacks cb 0x7f6528009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57604/0 (socket says 172.21.15.1:57604) 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 -- 172.21.15.1:0/309912006 learned_addr learned my addr 172.21.15.1:0/309912006 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f6533fff640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653900f640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 -- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 msgr2=0x55bf4f105990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f105990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 -- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 msgr2=0x55bf4f106c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f106c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 -- 172.21.15.1:0/309912006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4f102f90 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653900f640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f653880e640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6528009560 tx=0x7f6528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8a9aceed496e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.437+0000 7f65137fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528037070 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6187+0+0 (secure 0 0 0) 0x7f6528049670 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65200074b0 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6528067050 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 10 client.8361 fetch_fsmap learned FSMap version 68 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55bf4f198250 con 0x55bf4f104770 2024-05-09T06:34:58.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200074b0 con 0x55bf4f104770 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f652806a020 con 0x55bf4f104770 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 10 client.8361 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 10 client.8361 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 20 client.8361 populate_metadata read hostname 'smithi001' 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f1053d0 0x55bf4f1071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 4 client.8361 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf4f045780 con 0x55bf4f1053d0 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f6533fff640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f1053d0 0x55bf4f1071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.438+0000 7f65137fe640 10 client.8361 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f6533fff640 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f1053d0 0x55bf4f1071c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d07bd27ba4434b server_cookie=52b031ac905afcc7 in_seq=0 out_seq=0 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f65137fe640 1 -- 172.21.15.1:0/309912006 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55bf4f045780 con 0x55bf4f1053d0 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f65137fe640 10 client.8361 handle_command_reply: tid=0 2024-05-09T06:34:58.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 client.8361 shutdown 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 msgr2=0x55bf4f18e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6528009560 tx=0x7f6528004730 comp rx=0 tx=0).stop 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 shutdown_connections 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf4f121650 0x55bf4f106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55bf4f1053d0 0x55bf4f1071c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bf4f11d020 0x55bf4f105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 --2- 172.21.15.1:0/309912006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf4f104770 0x55bf4f18e890 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f65137fe640 0 client.8361 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 >> 172.21.15.1:0/309912006 conn(0x55bf4f104030 msgr2=0x55bf4f0d99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 shutdown_connections 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 1 -- 172.21.15.1:0/309912006 wait complete. 2024-05-09T06:34:58.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.439+0000 7f653d77d740 20 client.8361 trim_cache size 0 max 16384 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "d13ae69c-4fb3-47f6-acc9-4f362b1f126c": { 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir", 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "d13ae69c-4fb3-47f6-acc9-4f362b1f126c", 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:34:58.428 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 msgr2=0x7f651806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 0x7f651806e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f652802e000 comp rx=0 tx=0).stop 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 msgr2=0x7f65340c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f652000c430 tx=0x7f652000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 shutdown_connections 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f651806be30 0x7f651806e2f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f653415c4a0 0x7f653414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6534158a50 0x7f65340c5a10 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 --2- 172.21.15.1:0/482155066 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f653406ede0 0x7f65340c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 >> 172.21.15.1:0/482155066 conn(0x7f65340cc4d0 msgr2=0x7f65340cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 shutdown_connections 2024-05-09T06:34:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:34:58.440+0000 7f653aa72640 1 -- 172.21.15.1:0/482155066 wait complete. 2024-05-09T06:34:58.432 INFO:tasks.cephfs.filesystem:scrub status for tag:d13ae69c-4fb3-47f6-acc9-4f362b1f126c - {'path': '/dir', 'tag': 'd13ae69c-4fb3-47f6-acc9-4f362b1f126c', 'options': 'recursive,repair'} 2024-05-09T06:35:18.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:18.624 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:28.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:28.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:35:28.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.571+0000 7efda9f09640 1 Processor -- start 2024-05-09T06:35:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 -- start start 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda40d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efda40d0a10 0x7efda40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda40d1350 con 0x7efda40ceaa0 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efda40b6e50 con 0x7efda40d0a10 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.572+0000 7efda9f09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efda40d2760 con 0x7efda415e0f0 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efda40d0a10 0x7efda40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efda40d0a10 0x7efda40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44416/0 (socket says 172.21.15.1:44416) 2024-05-09T06:35:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 -- 172.21.15.1:0/1975775700 learned_addr learned my addr 172.21.15.1:0/1975775700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda37fe640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda40d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 -- 172.21.15.1:0/1975775700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 msgr2=0x7efda40d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda40d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 -- 172.21.15.1:0/1975775700 >> v1:172.21.15.6:6789/0 conn(0x7efda40ceaa0 legacy=0x7efda40ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda2ffd640 1 -- 172.21.15.1:0/1975775700 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40d2a40 con 0x7efda40d0a10 2024-05-09T06:35:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.573+0000 7efda37fe640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda40d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda40d0e10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac459cf16b71b035 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda27fc640 1 -- 172.21.15.1:0/1975775700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98037070 con 0x7efda40d0a10 2024-05-09T06:35:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda27fc640 1 -- 172.21.15.1:0/1975775700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98032440 con 0x7efda40d0a10 2024-05-09T06:35:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda27fc640 1 -- 172.21.15.1:0/1975775700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98032810 con 0x7efda40d0a10 2024-05-09T06:35:28.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 -- 172.21.15.1:0/1975775700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 msgr2=0x7efda40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda40d0e10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd9802a210 comp rx=0 tx=0).stop 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 -- 172.21.15.1:0/1975775700 shutdown_connections 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda40d0e10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 --2- 172.21.15.1:0/1975775700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda40d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 -- 172.21.15.1:0/1975775700 >> 172.21.15.1:0/1975775700 conn(0x7efda409d3b0 msgr2=0x7efda40d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.574+0000 7efda9f09640 1 -- 172.21.15.1:0/1975775700 shutdown_connections 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.575+0000 7efda9f09640 1 -- 172.21.15.1:0/1975775700 wait complete. 2024-05-09T06:35:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.576+0000 7efda9f09640 1 Processor -- start 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.576+0000 7efda9f09640 1 -- start start 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda40ceaa0 0x7efda414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda406c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efda415e4d0 con 0x7efda40d0a10 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efda40b70a0 con 0x7efda40ceaa0 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda9f09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efda415e650 con 0x7efda415e0f0 2024-05-09T06:35:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44422/0 (socket says 172.21.15.1:44422) 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 -- 172.21.15.1:0/1873479053 learned_addr learned my addr 172.21.15.1:0/1873479053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda3fff640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda406c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda37fe640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda40ceaa0 0x7efda414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 -- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 msgr2=0x7efda406c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda406c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 -- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda40ceaa0 msgr2=0x7efda414afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda40ceaa0 0x7efda414afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 -- 172.21.15.1:0/1873479053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4160ca0 con 0x7efda40d0a10 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.577+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.578+0000 7efd8bfff640 1 -- 172.21.15.1:0/1873479053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98041020 con 0x7efda40d0a10 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.578+0000 7efd8bfff640 1 -- 172.21.15.1:0/1873479053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98030970 con 0x7efda40d0a10 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.578+0000 7efd8bfff640 1 -- 172.21.15.1:0/1873479053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98037040 con 0x7efda40d0a10 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.578+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda4160f80 con 0x7efda40d0a10 2024-05-09T06:35:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.578+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda4161540 con 0x7efda40d0a10 2024-05-09T06:35:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.579+0000 7efd8bfff640 1 -- 172.21.15.1:0/1873479053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd98044020 con 0x7efda40d0a10 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.580+0000 7efd8bfff640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 0x7efd9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.580+0000 7efd8bfff640 1 -- 172.21.15.1:0/1873479053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7efd980ab990 con 0x7efda40d0a10 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.580+0000 7efda37fe640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 0x7efd9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.580+0000 7efda37fe640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 0x7efd9006e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efd9400aac0 tx=0x7efd94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.581+0000 7efdacc10740 1 Processor -- start 2024-05-09T06:35:28.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.581+0000 7efdacc10740 1 -- start start 2024-05-09T06:35:28.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0beedaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee030 0x559e0bec2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e0be27d00 con 0x559e0bf0b650 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e0bdcf4c0 con 0x559e0beee030 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e0be23cc0 con 0x559e0beee770 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44430/0 (socket says 172.21.15.1:44430) 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 -- 172.21.15.1:0/2660308310 learned_addr learned my addr 172.21.15.1:0/2660308310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 -- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 msgr2=0x559e0beedaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0beedaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 -- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee030 msgr2=0x559e0bec2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee030 0x559e0bec2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 -- 172.21.15.1:0/2660308310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0bc057d0 con 0x559e0bf0b650 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efda37fe640 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efd9400fd80 tx=0x7efd94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d02b18403c61bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.582+0000 7efd897fa640 1 -- 172.21.15.1:0/2660308310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94023070 con 0x559e0bf0b650 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efd897fa640 1 -- 172.21.15.1:0/2660308310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd94019070 con 0x559e0bf0b650 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efd897fa640 1 -- 172.21.15.1:0/2660308310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94013820 con 0x559e0bf0b650 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 -- 172.21.15.1:0/2660308310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 msgr2=0x559e0bebdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efd9400fd80 tx=0x7efd94012000 comp rx=0 tx=0).stop 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 -- 172.21.15.1:0/2660308310 shutdown_connections 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee030 0x559e0bec2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0beedaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 --2- 172.21.15.1:0/2660308310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf0b650 0x559e0bebdfe0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 -- 172.21.15.1:0/2660308310 >> 172.21.15.1:0/2660308310 conn(0x559e0bec0250 msgr2=0x559e0bf08af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 -- 172.21.15.1:0/2660308310 shutdown_connections 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.583+0000 7efdacc10740 1 -- 172.21.15.1:0/2660308310 wait complete. 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.584+0000 7efdacc10740 1 -- start start 2024-05-09T06:35:28.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.584+0000 7efdacc10740 1 Processor -- start 2024-05-09T06:35:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bef0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 0x559e0bf016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e0bec2400 con 0x559e0beee770 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e0bde2b90 con 0x559e0bf0b650 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e0bec17f0 con 0x559e0bf01260 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efdacc10740 10 client.0 _ll_register_callbacks cb 0x559e0be1eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44446/0 (socket says 172.21.15.1:44446) 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 -- 172.21.15.1:0/2490267380 learned_addr learned my addr 172.21.15.1:0/2490267380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda2ffd640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bef0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda3fff640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 0x559e0bf016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 -- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 msgr2=0x559e0bf016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 0x559e0bf016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 -- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 msgr2=0x559e0bef0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bef0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 -- 172.21.15.1:0/2490267380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0bb3b3f0 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda3fff640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 0x559e0bf016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda2ffd640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bef0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efda37fe640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd9400fd20 tx=0x7efd94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25cf9b0a2a842aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.585+0000 7efd827fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94035060 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7efd94013830 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94019070 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7efd9403b050 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 10 client.8373 fetch_fsmap learned FSMap version 68 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x559e0bafb0f0 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd94019070 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7efd9403e020 con 0x559e0beee770 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 10 client.8373 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 10 client.8373 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 20 client.8373 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bf1c770 0x559e0bf1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 4 client.8373 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0be35430 con 0x559e0bf1c770 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.586+0000 7efda2ffd640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bf1c770 0x559e0bf1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efd827fc640 10 client.8373 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efda2ffd640 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bf1c770 0x559e0bf1cb90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8522d475512995 server_cookie=b971307b868a019b in_seq=0 out_seq=0 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efd827fc640 1 -- 172.21.15.1:0/2490267380 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e0be35430 con 0x559e0bf1c770 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efd827fc640 10 client.8373 handle_command_reply: tid=0 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 client.8373 shutdown 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 msgr2=0x559e0bee9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd9400fd20 tx=0x7efd94004730 comp rx=0 tx=0).stop 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 shutdown_connections 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bf1c770 0x559e0bf1cb90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf01260 0x559e0bf016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bef0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 --2- 172.21.15.1:0/2490267380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0beee770 0x559e0bee9670 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 >> 172.21.15.1:0/2490267380 conn(0x559e0beee030 msgr2=0x559e0bf08af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efd827fc640 0 client.8373 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.587+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 shutdown_connections 2024-05-09T06:35:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.588+0000 7efdacc10740 1 -- 172.21.15.1:0/2490267380 wait complete. 2024-05-09T06:35:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.588+0000 7efdacc10740 20 client.8373 trim_cache size 0 max 16384 2024-05-09T06:35:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.605+0000 7efdacc10740 1 Processor -- start 2024-05-09T06:35:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.605+0000 7efdacc10740 1 -- start start 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee770 0x559e0bec1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf0b650 0x559e0bee9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efda4200f10 con 0x559e0bf01260 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ffef0 con 0x559e0beee770 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efda40c8f50 con 0x559e0bf0b650 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44452/0 (socket says 172.21.15.1:44452) 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.606+0000 7efda2ffd640 1 -- 172.21.15.1:0/1610323433 learned_addr learned my addr 172.21.15.1:0/1610323433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 -- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf0b650 msgr2=0x559e0bee9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf0b650 0x559e0bee9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 -- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee770 msgr2=0x559e0bec1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda37fe640 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee770 0x559e0bec1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee770 0x559e0bec1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 -- 172.21.15.1:0/1610323433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0bf0cd90 con 0x559e0bf01260 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efd98079790 tx=0x7efd98048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0774265e976da38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efd88ff9640 1 -- 172.21.15.1:0/1610323433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98096070 con 0x559e0bf01260 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efd88ff9640 1 -- 172.21.15.1:0/1610323433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98077d90 con 0x559e0bf01260 2024-05-09T06:35:28.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efd88ff9640 1 -- 172.21.15.1:0/1610323433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98091030 con 0x559e0bf01260 2024-05-09T06:35:28.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efdacc10740 1 -- 172.21.15.1:0/1610323433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 msgr2=0x559e0bf08fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.607+0000 7efdacc10740 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efd98079790 tx=0x7efd98048040 comp rx=0 tx=0).stop 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 -- 172.21.15.1:0/1610323433 shutdown_connections 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0bf0b650 0x559e0bee9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bf08fc0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 --2- 172.21.15.1:0/1610323433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0beee770 0x559e0bec1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 -- 172.21.15.1:0/1610323433 >> 172.21.15.1:0/1610323433 conn(0x559e0beee030 msgr2=0x559e0bf2c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 -- 172.21.15.1:0/1610323433 shutdown_connections 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 -- 172.21.15.1:0/1610323433 wait complete. 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 -- start start 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.608+0000 7efdacc10740 1 Processor -- start 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0bf82a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bee99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e0bdc54d0 con 0x559e0bf01260 2024-05-09T06:35:28.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ff880 con 0x559e0bf0b650 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e0bf0c880 con 0x559e0beee770 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efdacc10740 10 client.0 _ll_register_callbacks cb 0x7efd9400fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44460/0 (socket says 172.21.15.1:44460) 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 -- 172.21.15.1:0/1304183612 learned_addr learned my addr 172.21.15.1:0/1304183612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 -- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 msgr2=0x559e0bf82a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0bf82a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 -- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 msgr2=0x559e0bee99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bee99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.609+0000 7efda2ffd640 1 -- 172.21.15.1:0/1304183612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0bec56a0 con 0x559e0bf01260 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efda2ffd640 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7efd9802a800 tx=0x7efd98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f631d5dcaaeff9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98096070 con 0x559e0bf01260 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7efd980a9c30 con 0x559e0bf01260 2024-05-09T06:35:28.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd98077d90 con 0x559e0bf01260 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7efd98082130 con 0x559e0bf01260 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 10 client.8379 fetch_fsmap learned FSMap version 68 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x559e0bf02b50 con 0x559e0bf01260 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98077d90 con 0x559e0bf01260 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7efd9803a030 con 0x559e0bf01260 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 10 client.8379 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 10 client.8379 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 20 client.8379 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bee8ba0 0x559e0bf07640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 4 client.8379 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e0be2f780 con 0x559e0bee8ba0 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.610+0000 7efda37fe640 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bee8ba0 0x559e0bf07640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efda37fe640 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bee8ba0 0x559e0bf07640 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f621d0e473c3a21 server_cookie=44cb44afe6e8f664 in_seq=0 out_seq=0 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efd827fc640 10 client.8379 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efd827fc640 1 -- 172.21.15.1:0/1304183612 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e0be2f780 con 0x559e0bee8ba0 2024-05-09T06:35:28.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efd827fc640 10 client.8379 handle_command_reply: tid=0 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 client.8379 shutdown 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 msgr2=0x559e0bee9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7efd9802a800 tx=0x7efd98000bc0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 shutdown_connections 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e0bf0b650 0x559e0bee99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0bf01260 0x559e0bee9480 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x559e0bee8ba0 0x559e0bf07640 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 --2- 172.21.15.1:0/1304183612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e0beee770 0x559e0bf82a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efd827fc640 0 client.8379 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 >> 172.21.15.1:0/1304183612 conn(0x559e0beee030 msgr2=0x559e0bf0bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 shutdown_connections 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 1 -- 172.21.15.1:0/1304183612 wait complete. 2024-05-09T06:35:28.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.611+0000 7efdacc10740 20 client.8379 trim_cache size 0 max 16384 2024-05-09T06:35:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:35:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:35:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:35:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 msgr2=0x7efd9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 0x7efd9006e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efd9400aac0 tx=0x7efd94008040 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 msgr2=0x7efda406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd980046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 shutdown_connections 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efd9006be30 0x7efd9006e2f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efda415e0f0 0x7efda406c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efda40d0a10 0x7efda406bda0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 --2- 172.21.15.1:0/1873479053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda40ceaa0 0x7efda414afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 >> 172.21.15.1:0/1873479053 conn(0x7efda409d3b0 msgr2=0x7efda415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 shutdown_connections 2024-05-09T06:35:28.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.612+0000 7efda9f09640 1 -- 172.21.15.1:0/1873479053 wait complete. 2024-05-09T06:35:28.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:35:28.603 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.727+0000 7f2d283e7640 1 Processor -- start 2024-05-09T06:35:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.727+0000 7f2d283e7640 1 -- start start 2024-05-09T06:35:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d283e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d200b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d283e7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2d2015e4b0 0x7f2d2015f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d283e7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d20099930 con 0x7f2d200c6860 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d283e7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2d200b1e20 con 0x7f2d2015e4b0 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d283e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2015ee90 con 0x7f2d20099b30 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d26183640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2d200c6860 0x7f2d200c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42954/0 (socket says 172.21.15.1:42954) 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d26183640 1 -- 172.21.15.1:0/449861953 learned_addr learned my addr 172.21.15.1:0/449861953 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882916484 0 0) 0x7f2d1c001110 con 0x7f2d200c6860 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25982640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d200b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d26984640 1 --2- 172.21.15.1:0/449861953 >> v2:172.21.15.6:3300/0 conn(0x7f2d2015e4b0 0x7f2d2015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d0c003520 con 0x7f2d200c6860 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2854134584 0 0) 0x7f2d0c003520 con 0x7f2d200c6860 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 msgr2=0x7f2d200b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d200b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 >> v2:172.21.15.6:3300/0 conn(0x7f2d2015e4b0 msgr2=0x7f2d2015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 --2- 172.21.15.1:0/449861953 >> v2:172.21.15.6:3300/0 conn(0x7f2d2015e4b0 0x7f2d2015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20162930 con 0x7f2d200c6860 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f2d1c001840 con 0x7f2d200c6860 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 >> v1:172.21.15.1:6789/0 conn(0x7f2d200c6860 legacy=0x7f2d200c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d0c0039b0 0x7f2d0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2015ee90 con 0x7f2d0c004d40 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c00b9c0 con 0x7f2d0c0039b0 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.728+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c00bb40 con 0x7f2d20099b30 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26183640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d0c0039b0 0x7f2d0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d25982640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 -- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 msgr2=0x7f2d0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d20099b30 0x7f2d0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 -- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d0c0039b0 msgr2=0x7f2d0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d0c0039b0 0x7f2d0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.729+0000 7f2d26984640 1 -- 172.21.15.1:0/449861953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c003520 con 0x7f2d0c004d40 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d26984640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d0c0051c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fc0 tx=0x7f2d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b47a9d54ea01852 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18009c10 con 0x7f2d0c004d40 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d18036070 con 0x7f2d0c004d40 2024-05-09T06:35:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d25181640 1 -- 172.21.15.1:0/449861953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1802d070 con 0x7f2d0c004d40 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 -- 172.21.15.1:0/449861953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 msgr2=0x7f2d0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 --2- 172.21.15.1:0/449861953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d0c0051c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fc0 tx=0x7f2d0c005830 comp rx=0 tx=0).stop 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d26984640 1 -- 172.21.15.1:0/449861953 reap_dead start 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 -- 172.21.15.1:0/449861953 shutdown_connections 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 -- 172.21.15.1:0/449861953 >> 172.21.15.1:0/449861953 conn(0x7f2d200b1fb0 msgr2=0x7f2d200b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 -- 172.21.15.1:0/449861953 shutdown_connections 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.730+0000 7f2d283e7640 1 -- 172.21.15.1:0/449861953 wait complete. 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 Processor -- start 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 -- start start 2024-05-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d200cd590 0x7f2d2015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d2015bc00 0x7f2d200c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2015f860 con 0x7f2d0c004d40 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2015eb20 con 0x7f2d200cd590 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.732+0000 7f2d283e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2015f9e0 con 0x7f2d2015bc00 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44486/0 (socket says 172.21.15.1:44486) 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 -- 172.21.15.1:0/2446851453 learned_addr learned my addr 172.21.15.1:0/2446851453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d25982640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d200cd590 0x7f2d2015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26984640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d2015bc00 0x7f2d200c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 -- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d2015bc00 msgr2=0x7f2d200c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d2015bc00 0x7f2d200c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 -- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d200cd590 msgr2=0x7f2d2015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d200cd590 0x7f2d2015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 -- 172.21.15.1:0/2446851453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d200cc4f0 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d26183640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007290 tx=0x7f2d1c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00f940 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c018070 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d200cc7d0 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00fd20 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.733+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d200ccd90 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.734+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d200752b0 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.735+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d1c01d460 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.735+0000 7f2d16ffd640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.735+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f2d1c08b100 con 0x7f2d0c004d40 2024-05-09T06:35:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.735+0000 7f2d25982640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.736+0000 7f2d25982640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 0x7f2d0406e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b19b0 tx=0x7f2d10009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.740+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d20099930 con 0x7f2d0c004d40 2024-05-09T06:35:28.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.960+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d2006ce30 con 0x7f2d0c004d40 2024-05-09T06:35:28.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.960+0000 7f2d16ffd640 1 -- 172.21.15.1:0/2446851453 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5170 (secure 0 0 0) 0x7f2d1c0597c0 con 0x7f2d0c004d40 2024-05-09T06:35:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:28.940 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:30.731462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":11,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":6}]} 2024-05-09T06:35:28.941 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 68 2024-05-09T06:35:28.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 msgr2=0x7f2d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 0x7f2d0406e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b19b0 tx=0x7f2d10009190 comp rx=0 tx=0).stop 2024-05-09T06:35:28.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 msgr2=0x7f2d200cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007290 tx=0x7f2d1c007690 comp rx=0 tx=0).stop 2024-05-09T06:35:28.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 shutdown_connections 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d2015bc00 0x7f2d200c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d200cd590 0x7f2d2015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 --2- 172.21.15.1:0/2446851453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d0c004d40 0x7f2d200cbd50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 >> 172.21.15.1:0/2446851453 conn(0x7f2d200b1fb0 msgr2=0x7f2d200cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 shutdown_connections 2024-05-09T06:35:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:28.962+0000 7f2d283e7640 1 -- 172.21.15.1:0/2446851453 wait complete. 2024-05-09T06:35:28.953 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-09T06:35:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.078+0000 7f2e0eed4640 1 Processor -- start 2024-05-09T06:35:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 -- start start 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2e0815c4a0 0x7f2e08071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0815b420 con 0x7f2e0806e6d0 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2e08158e30 con 0x7f2e0815c4a0 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.079+0000 7f2e0eed4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e08072480 con 0x7f2e08158a50 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e07fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2e0815c4a0 0x7f2e08071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e0d471640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2e0806e6d0 0x7f2e080cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42962/0 (socket says 172.21.15.1:42962) 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e0d471640 1 -- 172.21.15.1:0/3507602202 learned_addr learned my addr 172.21.15.1:0/3507602202 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e07fff640 1 -- 172.21.15.1:0/3507602202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 msgr2=0x7f2e080718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e07fff640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e077fe640 1 -- 172.21.15.1:0/3507602202 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331162107 0 0) 0x7f2e0815b420 con 0x7f2e0806e6d0 2024-05-09T06:35:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e07fff640 1 -- 172.21.15.1:0/3507602202 >> v1:172.21.15.1:6789/0 conn(0x7f2e0806e6d0 legacy=0x7f2e080cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.080+0000 7f2e07fff640 1 -- 172.21.15.1:0/3507602202 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0815b6e0 con 0x7f2e0815c4a0 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e07fff640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0815c4a0 0x7f2e08071e00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2df40097a0 tx=0x7f2df402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333fe95784e1f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e077fe640 1 -- 172.21.15.1:0/3507602202 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4037070 con 0x7f2e0815c4a0 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e077fe640 1 -- 172.21.15.1:0/3507602202 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4032440 con 0x7f2e0815c4a0 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e077fe640 1 -- 172.21.15.1:0/3507602202 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4032810 con 0x7f2e0815c4a0 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 -- 172.21.15.1:0/3507602202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0815c4a0 msgr2=0x7f2e08071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0815c4a0 0x7f2e08071e00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2df40097a0 tx=0x7f2df402a180 comp rx=0 tx=0).stop 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 -- 172.21.15.1:0/3507602202 shutdown_connections 2024-05-09T06:35:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0815c4a0 0x7f2e08071e00 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/3507602202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 -- 172.21.15.1:0/3507602202 >> 172.21.15.1:0/3507602202 conn(0x7f2e080cc4d0 msgr2=0x7f2e080cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 -- 172.21.15.1:0/3507602202 shutdown_connections 2024-05-09T06:35:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.081+0000 7f2e0eed4640 1 -- 172.21.15.1:0/3507602202 wait complete. 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.083+0000 7f2e0eed4640 1 Processor -- start 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.083+0000 7f2e0eed4640 1 -- start start 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.083+0000 7f2e0eed4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e0815c4a0 0x7f2e0814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815bf80 con 0x7f2e0806e6d0 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815bca0 con 0x7f2e0815c4a0 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e08070ea0 con 0x7f2e08158a50 2024-05-09T06:35:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44502/0 (socket says 172.21.15.1:44502) 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 -- 172.21.15.1:0/1470985000 learned_addr learned my addr 172.21.15.1:0/1470985000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 -- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 msgr2=0x7f2e080cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 -- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e0815c4a0 msgr2=0x7f2e0814fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e0815c4a0 0x7f2e0814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 -- 172.21.15.1:0/1470985000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e08151dc0 con 0x7f2e0806e6d0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00d800 tx=0x7f2dfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e057fa640 1 -- 172.21.15.1:0/1470985000 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00b6c0 con 0x7f2e0806e6d0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e057fa640 1 -- 172.21.15.1:0/1470985000 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc019070 con 0x7f2e0806e6d0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081520a0 con 0x7f2e0806e6d0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e057fa640 1 -- 172.21.15.1:0/1470985000 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00ba80 con 0x7f2e0806e6d0 2024-05-09T06:35:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.084+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e08152660 con 0x7f2e0806e6d0 2024-05-09T06:35:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.086+0000 7f2e057fa640 1 -- 172.21.15.1:0/1470985000 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dfc00bc20 con 0x7f2e0806e6d0 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.086+0000 7f2e057fa640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 0x7f2dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.086+0000 7f2e057fa640 1 -- 172.21.15.1:0/1470985000 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f2dfc08b580 con 0x7f2e0806e6d0 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.086+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 0x7f2dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.087+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 0x7f2dec06e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2df4034000 tx=0x7f2df402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.087+0000 7f2e11be8740 1 Processor -- start 2024-05-09T06:35:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.087+0000 7f2e11be8740 1 -- start start 2024-05-09T06:35:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ec6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 0x55e038ecb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e038e2fa30 con 0x55e038ef72a0 2024-05-09T06:35:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e038dcd590 con 0x55e038ef6b60 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e038e2b9f0 con 0x55e038f14180 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44512/0 (socket says 172.21.15.1:44512) 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 -- 172.21.15.1:0/1940310105 learned_addr learned my addr 172.21.15.1:0/1940310105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e0d471640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 0x55e038ecb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ec6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 -- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 msgr2=0x55e038ec6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ec6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 -- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 msgr2=0x55e038ecb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.088+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 0x55e038ecb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e07fff640 1 -- 172.21.15.1:0/1940310105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e038c0f7c0 con 0x55e038ef72a0 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e0d471640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 0x55e038ecb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f2df4032670 tx=0x7f2df4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7292c76aba7b23b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2df27fc640 1 -- 172.21.15.1:0/1940310105 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df404c070 con 0x55e038ef72a0 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2df27fc640 1 -- 172.21.15.1:0/1940310105 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4004230 con 0x55e038ef72a0 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2df27fc640 1 -- 172.21.15.1:0/1940310105 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4037040 con 0x55e038ef72a0 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 -- 172.21.15.1:0/1940310105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 msgr2=0x55e038ef6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f2df4032670 tx=0x7f2df4008030 comp rx=0 tx=0).stop 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 -- 172.21.15.1:0/1940310105 shutdown_connections 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef6b60 0x55e038ecb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038ef6620 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.089+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1940310105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ec6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.090+0000 7f2e11be8740 1 -- 172.21.15.1:0/1940310105 >> 172.21.15.1:0/1940310105 conn(0x55e038ec8d80 msgr2=0x55e038f11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.090+0000 7f2e11be8740 1 -- 172.21.15.1:0/1940310105 shutdown_connections 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.090+0000 7f2e11be8740 1 -- 172.21.15.1:0/1940310105 wait complete. 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.090+0000 7f2e11be8740 1 -- start start 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.090+0000 7f2e11be8740 1 Processor -- start 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 0x55e038f10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e038ecaf30 con 0x55e038f14180 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e038dcb060 con 0x55e038ef72a0 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e038eca320 con 0x55e038ef9a00 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e11be8740 10 client.0 _ll_register_callbacks cb 0x55e038defdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44514/0 (socket says 172.21.15.1:44514) 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e07fff640 1 -- 172.21.15.1:0/1102028101 learned_addr learned my addr 172.21.15.1:0/1102028101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e0d471640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 0x55e038f10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.091+0000 7f2e07fff640 1 -- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 msgr2=0x55e038f10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 0x55e038f10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e07fff640 1 -- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 msgr2=0x55e038ef81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e07fff640 1 -- 172.21.15.1:0/1102028101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e038c6ff50 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e0d471640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 0x55e038f10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e07fff640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2df402a770 tx=0x7f2df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525fd9aadc18c786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df404c070 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df4004230 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f2df4003810 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2df405c050 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 10 client.8400 fetch_fsmap learned FSMap version 68 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55e038b030b0 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4004230 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2df405f020 con 0x55e038f14180 2024-05-09T06:35:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 10 client.8400 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 10 client.8400 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 20 client.8400 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f11950 0x55e038f11d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 4 client.8400 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.092+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e038e3d160 con 0x55e038f11950 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f11950 0x55e038f11d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f11950 0x55e038f11d30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf06b448464c74a server_cookie=81522778ebf46286 in_seq=0 out_seq=0 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2de77fe640 10 client.8400 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2de77fe640 1 -- 172.21.15.1:0/1102028101 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e038e3d160 con 0x55e038f11950 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2de77fe640 10 client.8400 handle_command_reply: tid=0 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.093+0000 7f2e11be8740 1 client.8400 shutdown 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 msgr2=0x55e038f13060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2df402a770 tx=0x7f2df4004730 comp rx=0 tx=0).stop 2024-05-09T06:35:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 shutdown_connections 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f11950 0x55e038f11d30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038ef9a00 0x55e038f10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038f14180 0x55e038f13060 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2de77fe640 0 client.8400 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 --2- 172.21.15.1:0/1102028101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 >> 172.21.15.1:0/1102028101 conn(0x55e038ef6b60 msgr2=0x55e038f14a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 shutdown_connections 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 1 -- 172.21.15.1:0/1102028101 wait complete. 2024-05-09T06:35:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.094+0000 7f2e11be8740 20 client.8400 trim_cache size 0 max 16384 2024-05-09T06:35:29.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.107+0000 7f2e11be8740 1 Processor -- start 2024-05-09T06:35:29.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.107+0000 7f2e11be8740 1 -- start start 2024-05-09T06:35:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081fdaa0 con 0x55e038ef9a00 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081fce30 con 0x55e038ef72a0 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e081fd450 con 0x55e038f14180 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44518/0 (socket says 172.21.15.1:44518) 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 -- 172.21.15.1:0/3223837738 learned_addr learned my addr 172.21.15.1:0/3223837738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 -- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 msgr2=0x55e038ef9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 -- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 msgr2=0x55e038ef7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.108+0000 7f2e07fff640 1 -- 172.21.15.1:0/3223837738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e038f15aa0 con 0x55e038ef9a00 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e07fff640 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2df4006f00 tx=0x7f2df404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9256c20bb2625b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2df1ffb640 1 -- 172.21.15.1:0/3223837738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4037040 con 0x55e038ef9a00 2024-05-09T06:35:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2df1ffb640 1 -- 172.21.15.1:0/3223837738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4003610 con 0x55e038ef9a00 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2df1ffb640 1 -- 172.21.15.1:0/3223837738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4003b80 con 0x55e038ef9a00 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 -- 172.21.15.1:0/3223837738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 msgr2=0x55e038ef8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2df4006f00 tx=0x7f2df404c000 comp rx=0 tx=0).stop 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 -- 172.21.15.1:0/3223837738 shutdown_connections 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef9a00 0x55e038ef8220 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 --2- 172.21.15.1:0/3223837738 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef72a0 0x55e038ef7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 -- 172.21.15.1:0/3223837738 >> 172.21.15.1:0/3223837738 conn(0x55e038ef6b60 msgr2=0x55e038ec1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 -- 172.21.15.1:0/3223837738 shutdown_connections 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.109+0000 7f2e11be8740 1 -- 172.21.15.1:0/3223837738 wait complete. 2024-05-09T06:35:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.110+0000 7f2e11be8740 1 -- start start 2024-05-09T06:35:29.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.110+0000 7f2e11be8740 1 Processor -- start 2024-05-09T06:35:29.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 0x55e038f0ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e038ef9580 con 0x55e038ef72a0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e038ecaf30 con 0x55e038ef9a00 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e038f216f0 con 0x55e038f14180 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 10 client.0 _ll_register_callbacks cb 0x7f2df404a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44534/0 (socket says 172.21.15.1:44534) 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 -- 172.21.15.1:0/986084011 learned_addr learned my addr 172.21.15.1:0/986084011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e07fff640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 0x55e038f0ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0d471640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 -- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 msgr2=0x55e038ef7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 -- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 msgr2=0x55e038f0ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 0x55e038f0ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 -- 172.21.15.1:0/986084011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e038f14ae0 con 0x55e038ef72a0 2024-05-09T06:35:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e07fff640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 0x55e038f0ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e0cc70640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc055a40 tx=0x7f2dfc05da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b2b264c0f30232 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc071030 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f2dfc055c50 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.111+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df4003610 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc061d30 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 10 client.8406 fetch_fsmap learned FSMap version 68 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55e038f2afb0 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4003610 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2dfc013070 con 0x55e038ef72a0 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 10 client.8406 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 10 client.8406 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 20 client.8406 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f06fb0 0x55e038f07390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 4 client.8406 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 --> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e038ec81d0 con 0x55e038f06fb0 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e07fff640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f06fb0 0x55e038f07390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2e07fff640 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f06fb0 0x55e038f07390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd38a1e26d392661 server_cookie=ec85f7c86c88fe35 in_seq=0 out_seq=0 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2de77fe640 10 client.8406 ms_handle_connect on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.112+0000 7f2de77fe640 1 -- 172.21.15.1:0/986084011 <== mds.0 v2:172.21.15.1:6834/424906117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55e038ec81d0 con 0x55e038f06fb0 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2de77fe640 10 client.8406 handle_command_reply: tid=0 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 client.8406 shutdown 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 msgr2=0x55e038f0e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc055a40 tx=0x7f2dfc05da50 comp rx=0 tx=0).stop 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 shutdown_connections 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:6834/424906117,v1:172.21.15.1:6836/424906117] conn(0x55e038f06fb0 0x55e038f07390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e038f14180 0x55e038ef7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e038ef9a00 0x55e038f0ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2de77fe640 0 client.8406 ms_handle_reset on v2:172.21.15.1:6834/424906117 2024-05-09T06:35:29.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 --2- 172.21.15.1:0/986084011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e038ef72a0 0x55e038f0e790 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 >> 172.21.15.1:0/986084011 conn(0x55e038ef6b60 msgr2=0x55e038efa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 shutdown_connections 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 1 -- 172.21.15.1:0/986084011 wait complete. 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e11be8740 20 client.8406 trim_cache size 0 max 16384 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "damage_type": "backtrace", 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 3645043045, 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1099511627777, 2024-05-09T06:35:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/testdir/hardlink" 2024-05-09T06:35:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:35:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 msgr2=0x7f2dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 0x7f2dec06e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2df4034000 tx=0x7f2df402c000 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 msgr2=0x7f2e0813f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.113+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00d800 tx=0x7f2dfc00dcc0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 shutdown_connections 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2dec06be30 0x7f2dec06e2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e0815c4a0 0x7f2e0814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e08158a50 0x7f2e080cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 --2- 172.21.15.1:0/1470985000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e0806e6d0 0x7f2e0813f4e0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 >> 172.21.15.1:0/1470985000 conn(0x7f2e080cc4d0 msgr2=0x7f2e080b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 shutdown_connections 2024-05-09T06:35:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.114+0000 7f2e0eed4640 1 -- 172.21.15.1:0/1470985000 wait complete. 2024-05-09T06:35:29.104 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:35:29.104 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:35:29.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:29.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.285+0000 7f1c1fb5c640 1 Processor -- start 2024-05-09T06:35:29.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.285+0000 7f1c1fb5c640 1 -- start start 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1fb5c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 0x7f1c1806f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1fb5c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c180d2890 0x7f1c180d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1fb5c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1806c2f0 con 0x7f1c1815e5b0 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1fb5c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1c180cc3c0 con 0x7f1c180d2890 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1fb5c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c180ccc60 con 0x7f1c1815e1d0 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1d8f8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 0x7f1c1806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1e0f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c180d2890 0x7f1c180d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1d0f7640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1c1815e5b0 0x7f1c180cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:52896/0 (socket says 172.21.15.1:52896) 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.286+0000 7f1c1d0f7640 1 -- 172.21.15.1:0/3382756911 learned_addr learned my addr 172.21.15.1:0/3382756911 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:29.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1c8f6640 1 -- 172.21.15.1:0/3382756911 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529617068 0 0) 0x7f1c08001110 con 0x7f1c1815e5b0 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1c8f6640 1 -- 172.21.15.1:0/3382756911 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c04003520 con 0x7f1c1815e5b0 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1e0f9640 1 -- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 msgr2=0x7f1c1806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1e0f9640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 0x7f1c1806f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1e0f9640 1 -- 172.21.15.1:0/3382756911 >> v1:172.21.15.6:6789/0 conn(0x7f1c1815e5b0 legacy=0x7f1c180cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1e0f9640 1 -- 172.21.15.1:0/3382756911 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1806f500 con 0x7f1c180d2890 2024-05-09T06:35:29.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1d8f8640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 0x7f1c1806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:29.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.287+0000 7f1c1e0f9640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c180d2890 0x7f1c180d0fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009dc0 tx=0x7f1c1002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482bc210c0f5dae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1c8f6640 1 -- 172.21.15.1:0/3382756911 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10007740 con 0x7f1c180d2890 2024-05-09T06:35:29.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1c8f6640 1 -- 172.21.15.1:0/3382756911 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c100078e0 con 0x7f1c180d2890 2024-05-09T06:35:29.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1c8f6640 1 -- 172.21.15.1:0/3382756911 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10007cb0 con 0x7f1c180d2890 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c180d2890 msgr2=0x7f1c180d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c180d2890 0x7f1c180d0fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009dc0 tx=0x7f1c1002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/3382756911 shutdown_connections 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c180d2890 0x7f1c180d0fc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/3382756911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e1d0 0x7f1c1806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/3382756911 >> 172.21.15.1:0/3382756911 conn(0x7f1c180d1d70 msgr2=0x7f1c180d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/3382756911 shutdown_connections 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.288+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/3382756911 wait complete. 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.289+0000 7f1c1fb5c640 1 Processor -- start 2024-05-09T06:35:29.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 -- start start 2024-05-09T06:35:29.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c1815e1d0 0x7f1c180cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e5b0 0x7f1c181466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1806f1b0 con 0x7f1c180d2890 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180cc8f0 con 0x7f1c1815e1d0 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1fb5c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1806fdd0 con 0x7f1c1815e5b0 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1d8f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1d8f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44542/0 (socket says 172.21.15.1:44542) 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1d8f8640 1 -- 172.21.15.1:0/664754656 learned_addr learned my addr 172.21.15.1:0/664754656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.290+0000 7f1c1e0f9640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e5b0 0x7f1c181466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d0f7640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c1815e1d0 0x7f1c180cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 -- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e5b0 msgr2=0x7f1c181466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e5b0 0x7f1c181466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 -- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c1815e1d0 msgr2=0x7f1c180cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c1815e1d0 0x7f1c180cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 -- 172.21.15.1:0/664754656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c180cb1f0 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1d8f8640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ec10 tx=0x7f1c1400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14017070 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1400cc30 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c180cb490 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14013470 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.291+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c180cba50 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.292+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c180752b0 con 0x7f1c180d2890 2024-05-09T06:35:29.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.293+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c1401a070 con 0x7f1c180d2890 2024-05-09T06:35:29.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.293+0000 7f1c0e7fc640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 0x7f1bfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.294+0000 7f1c1d0f7640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 0x7f1bfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.294+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f1c1408af00 con 0x7f1c180d2890 2024-05-09T06:35:29.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.294+0000 7f1c1d0f7640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 0x7f1bfc06e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c180c8940 tx=0x7f1c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.298+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c140590a0 con 0x7f1c180d2890 2024-05-09T06:35:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.483+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c180b0f40 con 0x7f1c180d2890 2024-05-09T06:35:29.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.484+0000 7f1c0e7fc640 1 -- 172.21.15.1:0/664754656 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17073 (secure 0 0 0) 0x7f1c1405d3a0 con 0x7f1c180d2890 2024-05-09T06:35:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:29.464 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":94,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:03.818180+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:29.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 msgr2=0x7f1bfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 0x7f1bfc06e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c180c8940 tx=0x7f1c08009040 comp rx=0 tx=0).stop 2024-05-09T06:35:29.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 msgr2=0x7f1c181615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ec10 tx=0x7f1c1400c520 comp rx=0 tx=0).stop 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 shutdown_connections 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bfc06be30 0x7f1bfc06e2f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c1815e5b0 0x7f1c181466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c1815e1d0 0x7f1c180cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 --2- 172.21.15.1:0/664754656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c180d2890 0x7f1c181615a0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 >> 172.21.15.1:0/664754656 conn(0x7f1c180d1d70 msgr2=0x7f1c180aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 shutdown_connections 2024-05-09T06:35:29.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.486+0000 7f1c1fb5c640 1 -- 172.21.15.1:0/664754656 wait complete. 2024-05-09T06:35:29.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:35:29.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:35:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.605+0000 7f4b778bd640 1 Processor -- start 2024-05-09T06:35:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b778bd640 1 -- start start 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b778bd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4b700d1c70 0x7f4b7015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b778bd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b70161fe0 con 0x7f4b7015e190 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b778bd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b701622c0 con 0x7f4b700d45f0 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b778bd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4b7015f580 con 0x7f4b700d1c70 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.606+0000 7f4b74e58640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4b700d1c70 0x7f4b7015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b75659640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4b7015e190 0x7f4b700d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42970/0 (socket says 172.21.15.1:42970) 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b75659640 1 -- 172.21.15.1:0/3294896606 learned_addr learned my addr 172.21.15.1:0/3294896606 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b74e58640 1 -- 172.21.15.1:0/3294896606 >> v1:172.21.15.6:6789/0 conn(0x7f4b700d45f0 legacy=0x7f4b700b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b74e58640 1 -- 172.21.15.1:0/3294896606 >> v1:172.21.15.1:6789/0 conn(0x7f4b7015e190 legacy=0x7f4b700d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b6ffff640 1 -- 172.21.15.1:0/3294896606 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087335646 0 0) 0x7f4b70161fe0 con 0x7f4b7015e190 2024-05-09T06:35:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.607+0000 7f4b74e58640 1 -- 172.21.15.1:0/3294896606 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b701625a0 con 0x7f4b700d1c70 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b74e58640 1 --2- 172.21.15.1:0/3294896606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d1c70 0x7f4b7015f780 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4b640097a0 tx=0x7f4b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b0c499adb6be31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b6ffff640 1 -- 172.21.15.1:0/3294896606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64037070 con 0x7f4b700d1c70 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b6ffff640 1 -- 172.21.15.1:0/3294896606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64032440 con 0x7f4b700d1c70 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b6ffff640 1 -- 172.21.15.1:0/3294896606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64032810 con 0x7f4b700d1c70 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 -- 172.21.15.1:0/3294896606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d1c70 msgr2=0x7f4b7015f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 --2- 172.21.15.1:0/3294896606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d1c70 0x7f4b7015f780 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4b640097a0 tx=0x7f4b6402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 -- 172.21.15.1:0/3294896606 shutdown_connections 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 --2- 172.21.15.1:0/3294896606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d1c70 0x7f4b7015f780 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 -- 172.21.15.1:0/3294896606 >> 172.21.15.1:0/3294896606 conn(0x7f4b70073340 msgr2=0x7f4b7009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 -- 172.21.15.1:0/3294896606 shutdown_connections 2024-05-09T06:35:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.608+0000 7f4b778bd640 1 -- 172.21.15.1:0/3294896606 wait complete. 2024-05-09T06:35:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 Processor -- start 2024-05-09T06:35:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 -- start start 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 0x7f4b70159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 0x7f4b700b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700b56d0 con 0x7f4b700d45f0 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70161c60 con 0x7f4b700d1c70 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b778bd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4b700b5850 con 0x7f4b7015e190 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b74e58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b74e58640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44556/0 (socket says 172.21.15.1:44556) 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.610+0000 7f4b74e58640 1 -- 172.21.15.1:0/2190214782 learned_addr learned my addr 172.21.15.1:0/2190214782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b75659640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 0x7f4b70159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b75e5a640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 0x7f4b700b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 -- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 msgr2=0x7f4b700b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 0x7f4b700b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 -- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 msgr2=0x7f4b70159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 0x7f4b70159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 -- 172.21.15.1:0/2190214782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b700c9460 con 0x7f4b700d45f0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b75659640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 0x7f4b70159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b75e5a640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 0x7f4b700b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b74e58640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4b640097a0 tx=0x7f4b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64041070 con 0x7f4b700d45f0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6403c870 con 0x7f4b700d45f0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b700c9180 con 0x7f4b700d45f0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64037040 con 0x7f4b700d45f0 2024-05-09T06:35:29.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.611+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b701f7650 con 0x7f4b700d45f0 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.612+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700752b0 con 0x7f4b700d45f0 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.613+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b64044070 con 0x7f4b700d45f0 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.613+0000 7f4b6dffb640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 0x7f4b5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.613+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f4b640abe80 con 0x7f4b700d45f0 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.614+0000 7f4b75659640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 0x7f4b5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.614+0000 7f4b75659640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 0x7f4b5c06e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000f60 tx=0x7f4b68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.618+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b6407a6f0 con 0x7f4b700d45f0 2024-05-09T06:35:29.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.799+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b7008e1c0 con 0x7f4b700d45f0 2024-05-09T06:35:29.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.799+0000 7f4b6dffb640 1 -- 172.21.15.1:0/2190214782 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f4b64079ff0 con 0x7f4b700d45f0 2024-05-09T06:35:29.779 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 msgr2=0x7f4b5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 0x7f4b5c06e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000f60 tx=0x7f4b68009040 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 msgr2=0x7f4b700b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4b640097a0 tx=0x7f4b640046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 shutdown_connections 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b5c06be30 0x7f4b5c06e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b7015e190 0x7f4b700b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b700d45f0 0x7f4b700b2e40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 --2- 172.21.15.1:0/2190214782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b700d1c70 0x7f4b70159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 >> 172.21.15.1:0/2190214782 conn(0x7f4b70073340 msgr2=0x7f4b700d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 shutdown_connections 2024-05-09T06:35:29.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.801+0000 7f4b778bd640 1 -- 172.21.15.1:0/2190214782 wait complete. 2024-05-09T06:35:29.792 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:35:29.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.918+0000 7f6b48163640 1 Processor -- start 2024-05-09T06:35:29.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.918+0000 7f6b48163640 1 -- start start 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.919+0000 7f6b48163640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 0x7f6b4006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.919+0000 7f6b48163640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6b4015cfb0 0x7f6b40070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.919+0000 7f6b48163640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4006cf40 con 0x7f6b400b4de0 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.919+0000 7f6b48163640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6b40156980 con 0x7f6b4015cfb0 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.919+0000 7f6b48163640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4006d870 con 0x7f6b4015c4c0 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6b4015cfb0 0x7f6b40070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6b4015cfb0 0x7f6b40070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44566/0 (socket says 172.21.15.1:44566) 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 -- 172.21.15.1:0/2965167549 learned_addr learned my addr 172.21.15.1:0/2965167549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b45eff640 1 --2- 172.21.15.1:0/2965167549 >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 0x7f6b4006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 -- 172.21.15.1:0/2965167549 >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 msgr2=0x7f6b4006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 --2- 172.21.15.1:0/2965167549 >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 0x7f6b4006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 -- 172.21.15.1:0/2965167549 >> v1:172.21.15.6:6789/0 conn(0x7f6b400b4de0 legacy=0x7f6b4015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b46700640 1 -- 172.21.15.1:0/2965167549 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4006d220 con 0x7f6b4015cfb0 2024-05-09T06:35:29.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.920+0000 7f6b45eff640 1 --2- 172.21.15.1:0/2965167549 >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 0x7f6b4006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b46700640 1 --2- 172.21.15.1:0/2965167549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b4015cfb0 0x7f6b40070c10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6b4004fb80 tx=0x7f6b3002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea100e9de05c88d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b44efd640 1 -- 172.21.15.1:0/2965167549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30036070 con 0x7f6b4015cfb0 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b44efd640 1 -- 172.21.15.1:0/2965167549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b30031600 con 0x7f6b4015cfb0 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b44efd640 1 -- 172.21.15.1:0/2965167549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b300319d0 con 0x7f6b4015cfb0 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 -- 172.21.15.1:0/2965167549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b4015cfb0 msgr2=0x7f6b40070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 --2- 172.21.15.1:0/2965167549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b4015cfb0 0x7f6b40070c10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6b4004fb80 tx=0x7f6b3002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 -- 172.21.15.1:0/2965167549 shutdown_connections 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 --2- 172.21.15.1:0/2965167549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b4015cfb0 0x7f6b40070c10 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 --2- 172.21.15.1:0/2965167549 >> v2:172.21.15.6:3300/0 conn(0x7f6b4015c4c0 0x7f6b4006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 -- 172.21.15.1:0/2965167549 >> 172.21.15.1:0/2965167549 conn(0x7f6b400cc4d0 msgr2=0x7f6b400cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 -- 172.21.15.1:0/2965167549 shutdown_connections 2024-05-09T06:35:29.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.921+0000 7f6b48163640 1 -- 172.21.15.1:0/2965167549 wait complete. 2024-05-09T06:35:29.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 Processor -- start 2024-05-09T06:35:29.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 -- start start 2024-05-09T06:35:29.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b4015c4c0 0x7f6b4015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 0x7f6b4015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400b2dc0 con 0x7f6b400b4de0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400b2c40 con 0x7f6b4015c4c0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.923+0000 7f6b48163640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6b400b5260 con 0x7f6b4015cfb0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44576/0 (socket says 172.21.15.1:44576) 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 -- 172.21.15.1:0/3630901024 learned_addr learned my addr 172.21.15.1:0/3630901024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b46700640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 0x7f6b4015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b456fe640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b4015c4c0 0x7f6b4015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 -- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 msgr2=0x7f6b4015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 0x7f6b4015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 -- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b4015c4c0 msgr2=0x7f6b4015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b4015c4c0 0x7f6b4015aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 -- 172.21.15.1:0/3630901024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b400a9a60 con 0x7f6b400b4de0 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b46700640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 0x7f6b4015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:29.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b45eff640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e7e0 tx=0x7f6b3800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800f040 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b38004030 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b400a9d40 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38004400 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.924+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b400aa300 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.925+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b400752b0 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.926+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b3800c6d0 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.926+0000 7f6b3e7fc640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 0x7f6b2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.926+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f6b3808ae20 con 0x7f6b400b4de0 2024-05-09T06:35:29.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.926+0000 7f6b456fe640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 0x7f6b2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.927+0000 7f6b456fe640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 0x7f6b2c06e320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b40144de0 tx=0x7f6b34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:29.930+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b38017020 con 0x7f6b400b4de0 2024-05-09T06:35:30.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.113+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b400cbc50 con 0x7f6b400b4de0 2024-05-09T06:35:30.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.114+0000 7f6b3e7fc640 1 -- 172.21.15.1:0/3630901024 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f6b38058f20 con 0x7f6b400b4de0 2024-05-09T06:35:30.094 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.115+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 msgr2=0x7f6b2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 0x7f6b2c06e320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b40144de0 tx=0x7f6b34009040 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 msgr2=0x7f6b400ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e7e0 tx=0x7f6b3800eca0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 shutdown_connections 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b2c06be60 0x7f6b2c06e320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b4015cfb0 0x7f6b4015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b4015c4c0 0x7f6b4015aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 --2- 172.21.15.1:0/3630901024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b400b4de0 0x7f6b400ac650 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 >> 172.21.15.1:0/3630901024 conn(0x7f6b400cc4d0 msgr2=0x7f6b4015c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 shutdown_connections 2024-05-09T06:35:30.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.116+0000 7f6b48163640 1 -- 172.21.15.1:0/3630901024 wait complete. 2024-05-09T06:35:30.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:35:30.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.233+0000 7f1798015640 1 Processor -- start 2024-05-09T06:35:30.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.233+0000 7f1798015640 1 -- start start 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1798015640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f179015a4d0 0x7f17900cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1798015640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1790071610 0x7f179006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1798015640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17900cf1b0 con 0x7f17900cdb90 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1798015640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f179015a8b0 con 0x7f179015a4d0 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1798015640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f179006dc10 con 0x7f1790071610 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1795db1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f179015a4d0 0x7f17900cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1795db1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f179015a4d0 0x7f17900cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44578/0 (socket says 172.21.15.1:44578) 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f1795db1640 1 -- 172.21.15.1:0/41568857 learned_addr learned my addr 172.21.15.1:0/41568857 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.234+0000 7f17965b2640 1 --2- 172.21.15.1:0/41568857 >> v2:172.21.15.6:3300/0 conn(0x7f1790071610 0x7f179006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1795db1640 1 -- 172.21.15.1:0/41568857 >> v2:172.21.15.6:3300/0 conn(0x7f1790071610 msgr2=0x7f179006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1795db1640 1 --2- 172.21.15.1:0/41568857 >> v2:172.21.15.6:3300/0 conn(0x7f1790071610 0x7f179006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1795db1640 1 -- 172.21.15.1:0/41568857 >> v1:172.21.15.6:6789/0 conn(0x7f17900cdb90 legacy=0x7f1790070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1795db1640 1 -- 172.21.15.1:0/41568857 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900cf490 con 0x7f179015a4d0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1795db1640 1 --2- 172.21.15.1:0/41568857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f17900cea80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f179006c2a0 tx=0x7f178802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4afa7c7fda59486 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.235+0000 7f1794daf640 1 -- 172.21.15.1:0/41568857 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788037070 con 0x7f179015a4d0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1794daf640 1 -- 172.21.15.1:0/41568857 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1788032440 con 0x7f179015a4d0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1794daf640 1 -- 172.21.15.1:0/41568857 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788032810 con 0x7f179015a4d0 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 -- 172.21.15.1:0/41568857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 msgr2=0x7f17900cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 --2- 172.21.15.1:0/41568857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f17900cea80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f179006c2a0 tx=0x7f178802a270 comp rx=0 tx=0).stop 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 -- 172.21.15.1:0/41568857 shutdown_connections 2024-05-09T06:35:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 --2- 172.21.15.1:0/41568857 >> v2:172.21.15.6:3300/0 conn(0x7f1790071610 0x7f179006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 --2- 172.21.15.1:0/41568857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f17900cea80 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 -- 172.21.15.1:0/41568857 >> 172.21.15.1:0/41568857 conn(0x7f17900cc4d0 msgr2=0x7f17900cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 -- 172.21.15.1:0/41568857 shutdown_connections 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.236+0000 7f1798015640 1 -- 172.21.15.1:0/41568857 wait complete. 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.237+0000 7f1798015640 1 Processor -- start 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.237+0000 7f1798015640 1 -- start start 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1790071610 0x7f179014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 0x7f179013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17900b2be0 con 0x7f179015a4d0 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f17900b2a60 con 0x7f1790071610 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1798015640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f179006cd30 con 0x7f17900cdb90 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f17965b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f17965b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44582/0 (socket says 172.21.15.1:44582) 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f17965b2640 1 -- 172.21.15.1:0/3884234531 learned_addr learned my addr 172.21.15.1:0/3884234531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f17955b0640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 0x7f179013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.238+0000 7f1795db1640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1790071610 0x7f179014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 -- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 msgr2=0x7f179013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 0x7f179013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 -- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1790071610 msgr2=0x7f179014eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1790071610 0x7f179014eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 -- 172.21.15.1:0/3884234531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179015ab60 con 0x7f179015a4d0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17955b0640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 0x7f179013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f17965b2640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1780007de0 tx=0x7f178000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780014620 con 0x7f179015a4d0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780019070 con 0x7f179015a4d0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780014a20 con 0x7f179015a4d0 2024-05-09T06:35:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179015ae40 con 0x7f179015a4d0 2024-05-09T06:35:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.239+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1790142b30 con 0x7f179015a4d0 2024-05-09T06:35:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.240+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17900752b0 con 0x7f179015a4d0 2024-05-09T06:35:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.241+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1780014bc0 con 0x7f179015a4d0 2024-05-09T06:35:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.241+0000 7f178e7fc640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 0x7f177c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.241+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f178008b910 con 0x7f179015a4d0 2024-05-09T06:35:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.241+0000 7f1795db1640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 0x7f177c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.242+0000 7f1795db1640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 0x7f177c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1788034000 tx=0x7f178802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.246+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f178005a100 con 0x7f179015a4d0 2024-05-09T06:35:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.429+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1790068560 con 0x7f179015a4d0 2024-05-09T06:35:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.430+0000 7f178e7fc640 1 -- 172.21.15.1:0/3884234531 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f1780059a00 con 0x7f179015a4d0 2024-05-09T06:35:30.411 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 msgr2=0x7f177c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 0x7f177c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1788034000 tx=0x7f178802e000 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 msgr2=0x7f179015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1780007de0 tx=0x7f178000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 shutdown_connections 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f177c06be30 0x7f177c06e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f179015a4d0 0x7f179015b920 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17900cdb90 0x7f179013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 --2- 172.21.15.1:0/3884234531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1790071610 0x7f179014eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 >> 172.21.15.1:0/3884234531 conn(0x7f17900cc4d0 msgr2=0x7f17901502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 shutdown_connections 2024-05-09T06:35:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.432+0000 7f1798015640 1 -- 172.21.15.1:0/3884234531 wait complete. 2024-05-09T06:35:30.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:35:30.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.549+0000 7fa17e81c640 1 Processor -- start 2024-05-09T06:35:30.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.549+0000 7fa17e81c640 1 -- start start 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17e81c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1780cf180 0x7fa1780cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17e81c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1780cdab0 0x7fa1780b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17e81c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1780ce0e0 con 0x7fa17806e6d0 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17e81c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa1780ce3c0 con 0x7fa1780cdab0 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17e81c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa1780ce6a0 con 0x7fa1780cf180 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa17cdb9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1780cdab0 0x7fa1780b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.550+0000 7fa1777fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1780cf180 0x7fa1780cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1780cdab0 0x7fa1780b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:40606/0 (socket says 172.21.15.1:40606) 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 -- 172.21.15.1:0/1707812871 learned_addr learned my addr 172.21.15.1:0/1707812871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:30.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 -- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1780cf180 msgr2=0x7fa1780cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:30.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 --2- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1780cf180 0x7fa1780cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 -- 172.21.15.1:0/1707812871 >> v1:172.21.15.6:6789/0 conn(0x7fa17806e6d0 legacy=0x7fa17806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa17cdb9640 1 -- 172.21.15.1:0/1707812871 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1780ce980 con 0x7fa1780cdab0 2024-05-09T06:35:30.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.551+0000 7fa176ffd640 1 -- 172.21.15.1:0/1707812871 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310902085 0 0) 0x7fa1780ce0e0 con 0x7fa17806e6d0 2024-05-09T06:35:30.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17cdb9640 1 --2- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa1780b3300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d150 tx=0x7fa17002c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c214f670d4dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa176ffd640 1 -- 172.21.15.1:0/1707812871 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170009600 con 0x7fa1780cdab0 2024-05-09T06:35:30.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa176ffd640 1 -- 172.21.15.1:0/1707812871 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa170037070 con 0x7fa1780cdab0 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa176ffd640 1 -- 172.21.15.1:0/1707812871 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700099c0 con 0x7fa1780cdab0 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 -- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 msgr2=0x7fa1780b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 --2- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa1780b3300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d150 tx=0x7fa17002c930 comp rx=0 tx=0).stop 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 -- 172.21.15.1:0/1707812871 shutdown_connections 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 --2- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa1780b3300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 --2- 172.21.15.1:0/1707812871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1780cf180 0x7fa1780cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 -- 172.21.15.1:0/1707812871 >> 172.21.15.1:0/1707812871 conn(0x7fa1780cc4b0 msgr2=0x7fa1780cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 -- 172.21.15.1:0/1707812871 shutdown_connections 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.552+0000 7fa17e81c640 1 -- 172.21.15.1:0/1707812871 wait complete. 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.554+0000 7fa17e81c640 1 Processor -- start 2024-05-09T06:35:30.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.554+0000 7fa17e81c640 1 -- start start 2024-05-09T06:35:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.554+0000 7fa17e81c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa17806e6d0 0x7fa1780aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa17815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781447c0 con 0x7fa1780cf180 2024-05-09T06:35:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa178070ed0 con 0x7fa1780cdab0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa178144940 con 0x7fa17806e6d0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44598/0 (socket says 172.21.15.1:44598) 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 -- 172.21.15.1:0/2950564690 learned_addr learned my addr 172.21.15.1:0/2950564690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa1777fe640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa17815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa177fff640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa17806e6d0 0x7fa1780aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 -- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa17806e6d0 msgr2=0x7fa1780aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa17806e6d0 0x7fa1780aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 -- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 msgr2=0x7fa17815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa17815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 -- 172.21.15.1:0/2950564690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17815d710 con 0x7fa1780cf180 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17cdb9640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa170034040 tx=0x7fa170004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000b720 con 0x7fa1780cf180 2024-05-09T06:35:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa170009e40 con 0x7fa1780cf180 2024-05-09T06:35:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa178151020 con 0x7fa1780cf180 2024-05-09T06:35:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.555+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170032660 con 0x7fa1780cf180 2024-05-09T06:35:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.556+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17815dc70 con 0x7fa1780cf180 2024-05-09T06:35:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.557+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1780b5750 con 0x7fa1780cf180 2024-05-09T06:35:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.557+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa170043030 con 0x7fa1780cf180 2024-05-09T06:35:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.557+0000 7fa174ff9640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 0x7fa15806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.557+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fa1700ab720 con 0x7fa1780cf180 2024-05-09T06:35:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.557+0000 7fa177fff640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 0x7fa15806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.558+0000 7fa177fff640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 0x7fa15806e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa168000f60 tx=0x7fa168009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.562+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1780b5750 con 0x7fa1780cf180 2024-05-09T06:35:30.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.756+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1780ce0e0 con 0x7fa1780cf180 2024-05-09T06:35:30.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.756+0000 7fa174ff9640 1 -- 172.21.15.1:0/2950564690 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) ==== 80+0+120 (secure 0 0 0) 0x7fa1780ce0e0 con 0x7fa1780cf180 2024-05-09T06:35:30.737 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6835/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:35:30.737 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6833/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:35:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:35:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 msgr2=0x7fa15806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 0x7fa15806e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa168000f60 tx=0x7fa168009040 comp rx=0 tx=0).stop 2024-05-09T06:35:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 msgr2=0x7fa17813d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa170034040 tx=0x7fa170004a40 comp rx=0 tx=0).stop 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 shutdown_connections 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa15806be30 0x7fa15806e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1780cf180 0x7fa17813d430 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1780cdab0 0x7fa17815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 --2- 172.21.15.1:0/2950564690 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa17806e6d0 0x7fa1780aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 >> 172.21.15.1:0/2950564690 conn(0x7fa1780cc4b0 msgr2=0x7fa1780b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 shutdown_connections 2024-05-09T06:35:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.758+0000 7fa17e81c640 1 -- 172.21.15.1:0/2950564690 wait complete. 2024-05-09T06:35:30.750 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:35:30.750 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:35:30.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:30.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.929+0000 7fcf85fb0640 1 Processor -- start 2024-05-09T06:35:30.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.929+0000 7fcf85fb0640 1 -- start start 2024-05-09T06:35:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf85fb0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf800b2a60 0x7fcf800b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf85fb0640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8006eb40 con 0x7fcf800b4f70 2024-05-09T06:35:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf85fb0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8006ee20 con 0x7fcf801454b0 2024-05-09T06:35:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf85fb0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8006d8e0 con 0x7fcf800b2a60 2024-05-09T06:35:30.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf7ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcf800b4f70 0x7fcf80070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42982/0 (socket says 172.21.15.1:42982) 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.930+0000 7fcf7ffff640 1 -- 172.21.15.1:0/1976389390 learned_addr learned my addr 172.21.15.1:0/1976389390 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> v2:172.21.15.6:3300/0 conn(0x7fcf800b2a60 0x7fcf800b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042438874 0 0) 0x7fcf8006eb40 con 0x7fcf800b4f70 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf68003520 con 0x7fcf800b4f70 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013465818 0 0) 0x7fcf8006ee20 con 0x7fcf801454b0 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf8006eb40 con 0x7fcf801454b0 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2615916019 0 0) 0x7fcf68003520 con 0x7fcf800b4f70 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 >> v2:172.21.15.6:3300/0 conn(0x7fcf800b2a60 msgr2=0x7fcf800b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 --2- 172.21.15.1:0/1976389390 >> v2:172.21.15.6:3300/0 conn(0x7fcf800b2a60 0x7fcf800b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 >> v1:172.21.15.6:6789/0 conn(0x7fcf801454b0 legacy=0x7fcf800b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8006dbc0 con 0x7fcf800b4f70 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fcf780019f0 con 0x7fcf800b4f70 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 >> v1:172.21.15.1:6789/0 conn(0x7fcf800b4f70 legacy=0x7fcf80070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68003b90 0x7fcf68003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 0x7fcf68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf680051c0 0x7fcf6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8006d8e0 con 0x7fcf68004800 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6800c090 con 0x7fcf680051c0 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6800c290 con 0x7fcf68003b90 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> v2:172.21.15.6:3300/0 conn(0x7fcf800b2a60 0x7fcf800b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 0x7fcf68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7ffff640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf680051c0 0x7fcf6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.931+0000 7fcf7f7fe640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68003b90 0x7fcf68003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 -- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68003b90 msgr2=0x7fcf68003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68003b90 0x7fcf68003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 -- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf680051c0 msgr2=0x7fcf6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf680051c0 0x7fcf6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 -- 172.21.15.1:0/1976389390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf68003520 con 0x7fcf68004800 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7ffff640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf680051c0 0x7fcf6800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:30.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7ffff640 1 -- 172.21.15.1:0/1976389390 reap_dead start 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7effd640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 0x7fcf68004c80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c70 tx=0x7fcf68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a977784b9f1a2358 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74009840 con 0x7fcf68004800 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf74004020 con 0x7fcf68004800 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.932+0000 7fcf7e7fc640 1 -- 172.21.15.1:0/1976389390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740043d0 con 0x7fcf68004800 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 -- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 msgr2=0x7fcf68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 0x7fcf68004c80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009c70 tx=0x7fcf68005830 comp rx=0 tx=0).stop 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 -- 172.21.15.1:0/1976389390 shutdown_connections 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/1976389390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf68004800 0x7fcf68004c80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 -- 172.21.15.1:0/1976389390 >> 172.21.15.1:0/1976389390 conn(0x7fcf800cc4b0 msgr2=0x7fcf800cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 -- 172.21.15.1:0/1976389390 shutdown_connections 2024-05-09T06:35:30.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.933+0000 7fcf85fb0640 1 -- 172.21.15.1:0/1976389390 wait complete. 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.934+0000 7fcf85fb0640 1 Processor -- start 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.934+0000 7fcf85fb0640 1 -- start start 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68004800 0x7fcf8013fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf8013c870 0x7fcf8014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8006e260 con 0x7fcf8013ccf0 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8006e0e0 con 0x7fcf8013c870 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf85fb0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcf800b4320 con 0x7fcf68004800 2024-05-09T06:35:30.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf7ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf7ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44622/0 (socket says 172.21.15.1:44622) 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.935+0000 7fcf7ffff640 1 -- 172.21.15.1:0/3322112581 learned_addr learned my addr 172.21.15.1:0/3322112581 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 -- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68004800 msgr2=0x7fcf8013fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68004800 0x7fcf8013fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 -- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf8013c870 msgr2=0x7fcf8014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf8013c870 0x7fcf8014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 -- 172.21.15.1:0/3322112581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8013d260 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf7ffff640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b5d20 tx=0x7fcf7800af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800a350 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78019070 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf800cde00 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780133f0 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.936+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf80150b20 con 0x7fcf8013ccf0 2024-05-09T06:35:30.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.937+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf800752b0 con 0x7fcf8013ccf0 2024-05-09T06:35:30.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.937+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf78013590 con 0x7fcf8013ccf0 2024-05-09T06:35:30.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.938+0000 7fcf67fff640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 0x7fcf6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.938+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fcf7808b570 con 0x7fcf8013ccf0 2024-05-09T06:35:30.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.938+0000 7fcf7f7fe640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 0x7fcf6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.938+0000 7fcf7f7fe640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 0x7fcf6c06e320 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf70001850 tx=0x7fcf70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:30.942+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf78059c30 con 0x7fcf8013ccf0 2024-05-09T06:35:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.129+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf800686f0 con 0x7fcf8013ccf0 2024-05-09T06:35:31.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.130+0000 7fcf67fff640 1 -- 172.21.15.1:0/3322112581 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17073 (secure 0 0 0) 0x7fcf78059530 con 0x7fcf8013ccf0 2024-05-09T06:35:31.111 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:31.111 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":94,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:03.818180+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:31.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 msgr2=0x7fcf6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 0x7fcf6c06e320 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf70001850 tx=0x7fcf70009040 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 msgr2=0x7fcf800adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b5d20 tx=0x7fcf7800af20 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 shutdown_connections 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcf6c06be60 0x7fcf6c06e320 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf8013ccf0 0x7fcf800adee0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf8013c870 0x7fcf8014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 --2- 172.21.15.1:0/3322112581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf68004800 0x7fcf8013fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 >> 172.21.15.1:0/3322112581 conn(0x7fcf800cc4b0 msgr2=0x7fcf800aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 shutdown_connections 2024-05-09T06:35:31.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.132+0000 7fcf85fb0640 1 -- 172.21.15.1:0/3322112581 wait complete. 2024-05-09T06:35:31.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:35:31.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:35:31.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.251+0000 7f5f0cc4d640 1 Processor -- start 2024-05-09T06:35:31.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.251+0000 7f5f0cc4d640 1 -- start start 2024-05-09T06:35:31.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0cc4d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f080ceba0 0x7f5f080cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0cc4d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080b1340 con 0x7f5f0815e270 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0cc4d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080d1020 con 0x7f5f080d08b0 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0cc4d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5f080cc260 con 0x7f5f080ceba0 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5f0815e270 0x7f5f080ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42990/0 (socket says 172.21.15.1:42990) 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0659c640 1 -- 172.21.15.1:0/1951565672 learned_addr learned my addr 172.21.15.1:0/1951565672 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f05d9b640 1 --2- 172.21.15.1:0/1951565672 >> v2:172.21.15.6:3300/0 conn(0x7f5f080ceba0 0x7f5f080cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873334995 0 0) 0x7f5ef8001110 con 0x7f5f0815e270 2024-05-09T06:35:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.252+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef0003520 con 0x7f5f0815e270 2024-05-09T06:35:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.253+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1692466568 0 0) 0x7f5ef0003520 con 0x7f5f0815e270 2024-05-09T06:35:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.253+0000 7f5f05d9b640 1 -- 172.21.15.1:0/1951565672 >> v1:172.21.15.6:6789/0 conn(0x7f5f080d08b0 legacy=0x7f5f080ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.253+0000 7f5f05d9b640 1 -- 172.21.15.1:0/1951565672 >> v1:172.21.15.1:6789/0 conn(0x7f5f0815e270 legacy=0x7f5f080ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.253+0000 7f5f05d9b640 1 -- 172.21.15.1:0/1951565672 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080cc4b0 con 0x7f5f080ceba0 2024-05-09T06:35:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.253+0000 7f5f05d9b640 1 --2- 172.21.15.1:0/1951565672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080ceba0 0x7f5f080cf180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0097a0 tx=0x7f5efc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad67d8f4440c5197 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc037070 con 0x7f5f080ceba0 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc02ace0 con 0x7f5f080ceba0 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0559a640 1 -- 172.21.15.1:0/1951565672 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8001840 con 0x7f5f080ceba0 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/1951565672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080ceba0 msgr2=0x7f5f080cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/1951565672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080ceba0 0x7f5f080cf180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0097a0 tx=0x7f5efc02a250 comp rx=0 tx=0).stop 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/1951565672 shutdown_connections 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/1951565672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080ceba0 0x7f5f080cf180 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/1951565672 >> 172.21.15.1:0/1951565672 conn(0x7f5f080d1c10 msgr2=0x7f5f080d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/1951565672 shutdown_connections 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.254+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/1951565672 wait complete. 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 Processor -- start 2024-05-09T06:35:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 -- start start 2024-05-09T06:35:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f080ceba0 0x7f5f080c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080d08b0 0x7f5f080c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0815e650 con 0x7f5f0815e270 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5f080cca70 con 0x7f5f080d08b0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f0cc4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5f080d0cc0 con 0x7f5f080ceba0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.256+0000 7f5f06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44636/0 (socket says 172.21.15.1:44636) 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 -- 172.21.15.1:0/2365586070 learned_addr learned my addr 172.21.15.1:0/2365586070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f0659c640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f080ceba0 0x7f5f080c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f05d9b640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080d08b0 0x7f5f080c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 -- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f080ceba0 msgr2=0x7f5f080c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f080ceba0 0x7f5f080c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 -- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080d08b0 msgr2=0x7f5f080c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080d08b0 0x7f5f080c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 -- 172.21.15.1:0/2365586070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080c61b0 con 0x7f5f0815e270 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f06d9d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c7a0 tx=0x7f5f0000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00003b70 con 0x7f5f0815e270 2024-05-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f00003d30 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f080c7750 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000aa90 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.257+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f080c6450 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.258+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f080cf9e0 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.259+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f0000ac30 con 0x7f5f0815e270 2024-05-09T06:35:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.259+0000 7f5ef6ffd640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 0x7f5ee806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.259+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f5f0008ae30 con 0x7f5f0815e270 2024-05-09T06:35:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.259+0000 7f5f0659c640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 0x7f5ee806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.260+0000 7f5f0659c640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 0x7f5ee806e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005880 tx=0x7f5ef80011d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.264+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f080cf9e0 con 0x7f5f0815e270 2024-05-09T06:35:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.452+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f080752b0 con 0x7f5f0815e270 2024-05-09T06:35:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.452+0000 7f5ef6ffd640 1 -- 172.21.15.1:0/2365586070 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f5f00058f50 con 0x7f5f0815e270 2024-05-09T06:35:31.433 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 msgr2=0x7f5ee806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 0x7f5ee806e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005880 tx=0x7f5ef80011d0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 msgr2=0x7f5f0815ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c7a0 tx=0x7f5f0000cc60 comp rx=0 tx=0).stop 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 shutdown_connections 2024-05-09T06:35:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5ee806be60 0x7f5ee806e320 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0815e270 0x7f5f0815ac60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f080d08b0 0x7f5f080c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 --2- 172.21.15.1:0/2365586070 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f080ceba0 0x7f5f080c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 >> 172.21.15.1:0/2365586070 conn(0x7f5f080d1c10 msgr2=0x7f5f080aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 shutdown_connections 2024-05-09T06:35:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.454+0000 7f5f0cc4d640 1 -- 172.21.15.1:0/2365586070 wait complete. 2024-05-09T06:35:31.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:35:31.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.572+0000 7f5b7f43f640 1 Processor -- start 2024-05-09T06:35:31.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.572+0000 7f5b7f43f640 1 -- start start 2024-05-09T06:35:31.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7f43f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b78071fd0 0x7f5b7806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7f43f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 0x7f5b7806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7f43f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7815b420 con 0x7f5b78158a50 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7f43f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5b78158e30 con 0x7f5b78071fd0 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7f43f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5b78072480 con 0x7f5b7815c4a0 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b78071fd0 0x7f5b7806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b78071fd0 0x7f5b7806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44638/0 (socket says 172.21.15.1:44638) 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7d1db640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5b78158a50 0x7f5b780718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:43002/0 (socket says 172.21.15.1:43002) 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 -- 172.21.15.1:0/3131267664 learned_addr learned my addr 172.21.15.1:0/3131267664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7d9dc640 1 --2- 172.21.15.1:0/3131267664 >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 0x7f5b7806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 -- 172.21.15.1:0/3131267664 >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 msgr2=0x7f5b7806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 --2- 172.21.15.1:0/3131267664 >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 0x7f5b7806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 -- 172.21.15.1:0/3131267664 >> v1:172.21.15.1:6789/0 conn(0x7f5b78158a50 legacy=0x7f5b780718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7c9da640 1 -- 172.21.15.1:0/3131267664 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7815b6e0 con 0x7f5b78071fd0 2024-05-09T06:35:31.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.573+0000 7f5b7d9dc640 1 --2- 172.21.15.1:0/3131267664 >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 0x7f5b7806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.574+0000 7f5b7c9da640 1 --2- 172.21.15.1:0/3131267664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b7806e6d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806d050 tx=0x7f5b6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bdd9ccfd5842dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.574+0000 7f5b77fff640 1 -- 172.21.15.1:0/3131267664 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c037070 con 0x7f5b78071fd0 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.574+0000 7f5b77fff640 1 -- 172.21.15.1:0/3131267664 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c032440 con 0x7f5b78071fd0 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.574+0000 7f5b77fff640 1 -- 172.21.15.1:0/3131267664 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c032810 con 0x7f5b78071fd0 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 -- 172.21.15.1:0/3131267664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 msgr2=0x7f5b7806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/3131267664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b7806e6d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806d050 tx=0x7f5b6c02a270 comp rx=0 tx=0).stop 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 -- 172.21.15.1:0/3131267664 shutdown_connections 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/3131267664 >> v2:172.21.15.6:3300/0 conn(0x7f5b7815c4a0 0x7f5b7806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/3131267664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b7806e6d0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 -- 172.21.15.1:0/3131267664 >> 172.21.15.1:0/3131267664 conn(0x7f5b780cc4d0 msgr2=0x7f5b780cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 -- 172.21.15.1:0/3131267664 shutdown_connections 2024-05-09T06:35:31.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.575+0000 7f5b7f43f640 1 -- 172.21.15.1:0/3131267664 wait complete. 2024-05-09T06:35:31.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.576+0000 7f5b7f43f640 1 Processor -- start 2024-05-09T06:35:31.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.576+0000 7f5b7f43f640 1 -- start start 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78158a50 0x7f5b780b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 0x7f5b78152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7815bf80 con 0x7f5b78071fd0 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7815bca0 con 0x7f5b7815c4a0 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7f43f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5b78070ea0 con 0x7f5b78158a50 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44644/0 (socket says 172.21.15.1:44644) 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 -- 172.21.15.1:0/1721270883 learned_addr learned my addr 172.21.15.1:0/1721270883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7c9da640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78158a50 0x7f5b780b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d9dc640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 0x7f5b78152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 -- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78158a50 msgr2=0x7f5b780b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78158a50 0x7f5b780b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 -- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 msgr2=0x7f5b78152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 0x7f5b78152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.577+0000 7f5b7d1db640 1 -- 172.21.15.1:0/1721270883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b781552f0 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b7d9dc640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 0x7f5b78152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b7d1db640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c430 tx=0x7f5b7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700074e0 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70007680 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70004a00 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b78156cf0 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.578+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781572b0 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.579+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b78150950 con 0x7f5b78071fd0 2024-05-09T06:35:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.579+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b7001a020 con 0x7f5b78071fd0 2024-05-09T06:35:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.580+0000 7f5b75ffb640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 0x7f5b6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.580+0000 7f5b7c9da640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 0x7f5b6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.580+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f5b7008a8c0 con 0x7f5b78071fd0 2024-05-09T06:35:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.581+0000 7f5b7c9da640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 0x7f5b6406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c034000 tx=0x7f5b6c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.585+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b70059020 con 0x7f5b78071fd0 2024-05-09T06:35:31.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.766+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b780752b0 con 0x7f5b78071fd0 2024-05-09T06:35:31.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.767+0000 7f5b75ffb640 1 -- 172.21.15.1:0/1721270883 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f5b70058920 con 0x7f5b78071fd0 2024-05-09T06:35:31.748 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.768+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 msgr2=0x7f5b6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.768+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 0x7f5b6406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c034000 tx=0x7f5b6c02e000 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.768+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 msgr2=0x7f5b78154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.768+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c430 tx=0x7f5b7000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 shutdown_connections 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b6406be30 0x7f5b6406e2f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b7815c4a0 0x7f5b78152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78158a50 0x7f5b780b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 --2- 172.21.15.1:0/1721270883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78071fd0 0x7f5b78154a10 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 >> 172.21.15.1:0/1721270883 conn(0x7f5b780cc4d0 msgr2=0x7f5b78159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 shutdown_connections 2024-05-09T06:35:31.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.769+0000 7f5b7f43f640 1 -- 172.21.15.1:0/1721270883 wait complete. 2024-05-09T06:35:31.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:35:31.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.886+0000 7fbfdf14c640 1 Processor -- start 2024-05-09T06:35:31.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 -- start start 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbfd815e190 0x7fbfd815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbfd80d1c70 0x7fbfd80d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd8161fe0 con 0x7fbfd80b56d0 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd815f580 con 0x7fbfd80d1c70 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfdf14c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd81622c0 con 0x7fbfd815e190 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.887+0000 7fbfd7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbfd80d1c70 0x7fbfd80d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfdd6e9640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbfd80b56d0 0x7fbfd80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:43010/0 (socket says 172.21.15.1:43010) 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfdd6e9640 1 -- 172.21.15.1:0/4019496645 learned_addr learned my addr 172.21.15.1:0/4019496645 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfdcee8640 1 --2- 172.21.15.1:0/4019496645 >> v2:172.21.15.6:3300/0 conn(0x7fbfd815e190 0x7fbfd815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfd7fff640 1 -- 172.21.15.1:0/4019496645 >> v2:172.21.15.6:3300/0 conn(0x7fbfd815e190 msgr2=0x7fbfd815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfd7fff640 1 --2- 172.21.15.1:0/4019496645 >> v2:172.21.15.6:3300/0 conn(0x7fbfd815e190 0x7fbfd815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfd7fff640 1 -- 172.21.15.1:0/4019496645 >> v1:172.21.15.1:6789/0 conn(0x7fbfd80b56d0 legacy=0x7fbfd80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.888+0000 7fbfd7fff640 1 -- 172.21.15.1:0/4019496645 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd81625a0 con 0x7fbfd80d1c70 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfd7fff640 1 --2- 172.21.15.1:0/4019496645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80d3ee0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80d1bf0 tx=0x7fbfcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e6fdeb54b8005d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfd77fe640 1 -- 172.21.15.1:0/4019496645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc037070 con 0x7fbfd80d1c70 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfd77fe640 1 -- 172.21.15.1:0/4019496645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfcc032440 con 0x7fbfd80d1c70 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfd77fe640 1 -- 172.21.15.1:0/4019496645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc032810 con 0x7fbfd80d1c70 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 -- 172.21.15.1:0/4019496645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 msgr2=0x7fbfd80d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/4019496645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80d3ee0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80d1bf0 tx=0x7fbfcc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 -- 172.21.15.1:0/4019496645 shutdown_connections 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/4019496645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80d3ee0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/4019496645 >> v2:172.21.15.6:3300/0 conn(0x7fbfd815e190 0x7fbfd815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 -- 172.21.15.1:0/4019496645 >> 172.21.15.1:0/4019496645 conn(0x7fbfd8073340 msgr2=0x7fbfd809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 -- 172.21.15.1:0/4019496645 shutdown_connections 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.889+0000 7fbfdf14c640 1 -- 172.21.15.1:0/4019496645 wait complete. 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.890+0000 7fbfdf14c640 1 Processor -- start 2024-05-09T06:35:31.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.890+0000 7fbfdf14c640 1 -- start start 2024-05-09T06:35:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfd80b56d0 0x7fbfd81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfd815e190 0x7fbfd80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd80d4420 con 0x7fbfd80d1c70 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8161c60 con 0x7fbfd815e190 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfdf14c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd80d45a0 con 0x7fbfd80b56d0 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.891+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44654/0 (socket says 172.21.15.1:44654) 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 -- 172.21.15.1:0/3672470350 learned_addr learned my addr 172.21.15.1:0/3672470350 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfdcee8640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfd80b56d0 0x7fbfd81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfdd6e9640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfd815e190 0x7fbfd80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 -- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfd80b56d0 msgr2=0x7fbfd81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfd80b56d0 0x7fbfd81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 -- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfd815e190 msgr2=0x7fbfd80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfd815e190 0x7fbfd80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 -- 172.21.15.1:0/3672470350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80cd060 con 0x7fbfd80d1c70 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd7fff640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000c00 tx=0x7fbfcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc037040 con 0x7fbfd80d1c70 2024-05-09T06:35:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfcc042070 con 0x7fbfd80d1c70 2024-05-09T06:35:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd80cd340 con 0x7fbfd80d1c70 2024-05-09T06:35:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc030bd0 con 0x7fbfd80d1c70 2024-05-09T06:35:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.892+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80b1d70 con 0x7fbfd80d1c70 2024-05-09T06:35:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.893+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd80752b0 con 0x7fbfd80d1c70 2024-05-09T06:35:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.894+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfcc047460 con 0x7fbfd80d1c70 2024-05-09T06:35:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.894+0000 7fbfd57fa640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 0x7fbfc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.894+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fbfcc0aba80 con 0x7fbfd80d1c70 2024-05-09T06:35:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.894+0000 7fbfdcee8640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 0x7fbfc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.895+0000 7fbfdcee8640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 0x7fbfc406e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0003aa0 tx=0x7fbfd0009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:31.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:31.899+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbfcc032e20 con 0x7fbfd80d1c70 2024-05-09T06:35:32.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.079+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfd808e1c0 con 0x7fbfd80d1c70 2024-05-09T06:35:32.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.080+0000 7fbfd57fa640 1 -- 172.21.15.1:0/3672470350 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fbfcc079c20 con 0x7fbfd80d1c70 2024-05-09T06:35:32.061 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.081+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 msgr2=0x7fbfc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.081+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 0x7fbfc406e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0003aa0 tx=0x7fbfd0009830 comp rx=0 tx=0).stop 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.081+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 msgr2=0x7fbfd80b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.081+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000c00 tx=0x7fbfcc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 shutdown_connections 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbfc406be30 0x7fbfc406e2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfd815e190 0x7fbfd80ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfd80d1c70 0x7fbfd80b2290 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 --2- 172.21.15.1:0/3672470350 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfd80b56d0 0x7fbfd81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 >> 172.21.15.1:0/3672470350 conn(0x7fbfd8073340 msgr2=0x7fbfd815c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:32.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 shutdown_connections 2024-05-09T06:35:32.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.082+0000 7fbfdf14c640 1 -- 172.21.15.1:0/3672470350 wait complete. 2024-05-09T06:35:32.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:35:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.198+0000 7fefe1458640 1 Processor -- start 2024-05-09T06:35:32.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.198+0000 7fefe1458640 1 -- start start 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefe1458640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fefdc06e6d0 0x7fefdc06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefe1458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc0b3300 0x7fefdc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefe1458640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc0ce0e0 con 0x7fefdc0cf180 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefe1458640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fefdc0ce3c0 con 0x7fefdc06e6d0 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefe1458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc0ce6a0 con 0x7fefdc0b3300 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fefdc06e6d0 0x7fefdc06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefd27fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fefdc0cf180 0x7fefdc0cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:43018/0 (socket says 172.21.15.1:43018) 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefd27fc640 1 -- 172.21.15.1:0/2448538481 learned_addr learned my addr 172.21.15.1:0/2448538481 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdb7fe640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc0b3300 0x7fefdc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdaffd640 1 -- 172.21.15.1:0/2448538481 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc0b3300 msgr2=0x7fefdc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:32.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdaffd640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc0b3300 0x7fefdc0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdaffd640 1 -- 172.21.15.1:0/2448538481 >> v1:172.21.15.1:6789/0 conn(0x7fefdc0cf180 legacy=0x7fefdc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefdaffd640 1 -- 172.21.15.1:0/2448538481 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc0ce980 con 0x7fefdc06e6d0 2024-05-09T06:35:32.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.199+0000 7fefda7fc640 1 -- 172.21.15.1:0/2448538481 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623879433 0 0) 0x7fefdc0ce0e0 con 0x7fefdc0cf180 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefdaffd640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc06e6d0 0x7fefdc06eab0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097a0 tx=0x7fefcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41724858e60f7ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefda7fc640 1 -- 172.21.15.1:0/2448538481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037070 con 0x7fefdc06e6d0 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefda7fc640 1 -- 172.21.15.1:0/2448538481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc032440 con 0x7fefdc06e6d0 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefda7fc640 1 -- 172.21.15.1:0/2448538481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc032810 con 0x7fefdc06e6d0 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefe1458640 1 -- 172.21.15.1:0/2448538481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc06e6d0 msgr2=0x7fefdc06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.200+0000 7fefe1458640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc06e6d0 0x7fefdc06eab0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097a0 tx=0x7fefcc02a180 comp rx=0 tx=0).stop 2024-05-09T06:35:32.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 -- 172.21.15.1:0/2448538481 shutdown_connections 2024-05-09T06:35:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc0b3300 0x7fefdc0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 --2- 172.21.15.1:0/2448538481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc06e6d0 0x7fefdc06eab0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 -- 172.21.15.1:0/2448538481 >> 172.21.15.1:0/2448538481 conn(0x7fefdc0cc4b0 msgr2=0x7fefdc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 -- 172.21.15.1:0/2448538481 shutdown_connections 2024-05-09T06:35:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.201+0000 7fefe1458640 1 -- 172.21.15.1:0/2448538481 wait complete. 2024-05-09T06:35:32.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.202+0000 7fefe1458640 1 Processor -- start 2024-05-09T06:35:32.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.202+0000 7fefe1458640 1 -- start start 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc06e6d0 0x7fefdc1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 0x7fefdc0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc070ed0 con 0x7fefdc0b3300 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc06eff0 con 0x7fefdc0cf180 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc1447c0 con 0x7fefdc06e6d0 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44680/0 (socket says 172.21.15.1:44680) 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 -- 172.21.15.1:0/1053101360 learned_addr learned my addr 172.21.15.1:0/1053101360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefdaffd640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc06e6d0 0x7fefdc1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefdb7fe640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 0x7fefdc0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 -- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc06e6d0 msgr2=0x7fefdc1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc06e6d0 0x7fefdc1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 -- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 msgr2=0x7fefdc0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 0x7fefdc0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 -- 172.21.15.1:0/1053101360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc0ae4b0 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefdb7fe640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 0x7fefdc0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd27fc640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c4b0 tx=0x7fefc400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4018070 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc4007500 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4004a20 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.203+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc0ae790 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.204+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc0aed50 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.205+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefdc0752b0 con 0x7fefdc0b3300 2024-05-09T06:35:32.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.205+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefc401b070 con 0x7fefdc0b3300 2024-05-09T06:35:32.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.205+0000 7fefd3fff640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 0x7fefbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:32.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.205+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7fefc408a2f0 con 0x7fefdc0b3300 2024-05-09T06:35:32.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.205+0000 7fefdaffd640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 0x7fefbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:32.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.206+0000 7fefdaffd640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 0x7fefbc06e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fefcc034000 tx=0x7fefcc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:32.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.210+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fefc40584c0 con 0x7fefdc0b3300 2024-05-09T06:35:32.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.411+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fefdc06cf50 con 0x7fefdc0b3300 2024-05-09T06:35:32.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.411+0000 7fefd3fff640 1 -- 172.21.15.1:0/1053101360 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) ==== 80+0+120 (secure 0 0 0) 0x7fefc405c7c0 con 0x7fefdc0b3300 2024-05-09T06:35:32.392 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6835/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:35:32.392 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6833/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:35:32.392 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:35:32.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 msgr2=0x7fefbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 0x7fefbc06e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fefcc034000 tx=0x7fefcc02e000 comp rx=0 tx=0).stop 2024-05-09T06:35:32.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 msgr2=0x7fefdc0ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c4b0 tx=0x7fefc400c970 comp rx=0 tx=0).stop 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 shutdown_connections 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fefbc06be30 0x7fefbc06e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fefdc0cf180 0x7fefdc0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefdc0b3300 0x7fefdc0ad960 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 --2- 172.21.15.1:0/1053101360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fefdc06e6d0 0x7fefdc1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 >> 172.21.15.1:0/1053101360 conn(0x7fefdc0cc4b0 msgr2=0x7fefdc0cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 shutdown_connections 2024-05-09T06:35:32.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:32.413+0000 7fefe1458640 1 -- 172.21.15.1:0/1053101360 wait complete. 2024-05-09T06:35:32.405 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:35:32.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:32.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:32.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[69769]: fuse finished with error 0 and tester_r 0 2024-05-09T06:35:32.432 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:35:38.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:35:38.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:38.533 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:35:38.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.728+0000 7f5305102640 1 Processor -- start 2024-05-09T06:35:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.728+0000 7f5305102640 1 -- start start 2024-05-09T06:35:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f5305102640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f530006ffa0 0x7f53000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f5305102640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f53000d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f5305102640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f530006c2f0 con 0x7f530015e1d0 2024-05-09T06:35:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f5305102640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f53000cc3c0 con 0x7f530006ffa0 2024-05-09T06:35:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f5305102640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f53000ccc60 con 0x7f530015e5b0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f52fe59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f530006ffa0 0x7f53000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f52fe59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f530006ffa0 0x7f53000cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39692/0 (socket says 172.21.15.1:39692) 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f52fe59c640 1 -- 172.21.15.1:0/3772932320 learned_addr learned my addr 172.21.15.1:0/3772932320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.729+0000 7f52fed9d640 1 --1- 172.21.15.1:0/3772932320 >> v1:172.21.15.1:6789/0 conn(0x7f530015e1d0 0x7f530006f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56238/0 (socket says 172.21.15.1:56238) 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fe59c640 1 -- 172.21.15.1:0/3772932320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 msgr2=0x7f53000d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52ff59e640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f53000d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fdd9b640 1 -- 172.21.15.1:0/3772932320 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431067164 0 0) 0x7f52f0001110 con 0x7f530015e1d0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fe59c640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f53000d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fe59c640 1 -- 172.21.15.1:0/3772932320 >> v1:172.21.15.1:6789/0 conn(0x7f530015e1d0 legacy=0x7f530006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fe59c640 1 -- 172.21.15.1:0/3772932320 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530006f500 con 0x7f530006ffa0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52ff59e640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f53000d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.730+0000 7f52fe59c640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53000cffb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009870 tx=0x7f52f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c935f927c81c9acc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f52fdd9b640 1 -- 172.21.15.1:0/3772932320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4037070 con 0x7f530006ffa0 2024-05-09T06:35:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f52fdd9b640 1 -- 172.21.15.1:0/3772932320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f4032440 con 0x7f530006ffa0 2024-05-09T06:35:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f52fdd9b640 1 -- 172.21.15.1:0/3772932320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4032810 con 0x7f530006ffa0 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 -- 172.21.15.1:0/3772932320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 msgr2=0x7f53000cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53000cffb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009870 tx=0x7f52f402a1c0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 -- 172.21.15.1:0/3772932320 shutdown_connections 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f53000d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 --2- 172.21.15.1:0/3772932320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53000cffb0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 -- 172.21.15.1:0/3772932320 >> 172.21.15.1:0/3772932320 conn(0x7f53000d1d70 msgr2=0x7f53000d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 -- 172.21.15.1:0/3772932320 shutdown_connections 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.731+0000 7f5305102640 1 -- 172.21.15.1:0/3772932320 wait complete. 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.733+0000 7f5305102640 1 Processor -- start 2024-05-09T06:35:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.733+0000 7f5305102640 1 -- start start 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530015e1d0 0x7f53000d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f5300143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f530006f1b0 con 0x7f530006ffa0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f53000cc8f0 con 0x7f530015e1d0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f53000ce100 con 0x7f530015e5b0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39704/0 (socket says 172.21.15.1:39704) 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 -- 172.21.15.1:0/1589185737 learned_addr learned my addr 172.21.15.1:0/1589185737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52ff59e640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f5300143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fe59c640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530015e1d0 0x7f53000d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 -- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 msgr2=0x7f5300143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f5300143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 -- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530015e1d0 msgr2=0x7f53000d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530015e1d0 0x7f53000d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 -- 172.21.15.1:0/1589185737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53001f7640 con 0x7f530006ffa0 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52fed9d640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c780 tx=0x7f52f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0018070 con 0x7f530006ffa0 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f00072d0 con 0x7f530006ffa0 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.734+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001f78e0 con 0x7f530006ffa0 2024-05-09T06:35:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.735+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f00047e0 con 0x7f530006ffa0 2024-05-09T06:35:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.735+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53000cb260 con 0x7f530006ffa0 2024-05-09T06:35:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.736+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53000752b0 con 0x7f530006ffa0 2024-05-09T06:35:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.736+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52f001b070 con 0x7f530006ffa0 2024-05-09T06:35:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.737+0000 7f52e77fe640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 0x7f52ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.737+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f52f008a2a0 con 0x7f530006ffa0 2024-05-09T06:35:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.737+0000 7f52fe59c640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 0x7f52ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.737+0000 7f52fe59c640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 0x7f52ec06e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f52f4034000 tx=0x7f52f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:38.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.741+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f530006c2f0 con 0x7f530006ffa0 2024-05-09T06:35:38.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.966+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53000b0c90 con 0x7f530006ffa0 2024-05-09T06:35:38.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.967+0000 7f52e77fe640 1 -- 172.21.15.1:0/1589185737 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5170 (secure 0 0 0) 0x7f52f008a0c0 con 0x7f530006ffa0 2024-05-09T06:35:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:38.948 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:34:30.731462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":11,"addr":"172.21.15.1:6836/424906117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":424906117},{"type":"v1","addr":"172.21.15.1:6836","nonce":424906117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":6}]} 2024-05-09T06:35:38.948 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 68 2024-05-09T06:35:38.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 msgr2=0x7f52ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 0x7f52ec06e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f52f4034000 tx=0x7f52f402e000 comp rx=0 tx=0).stop 2024-05-09T06:35:38.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 msgr2=0x7f53001f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c780 tx=0x7f52f000cc40 comp rx=0 tx=0).stop 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 shutdown_connections 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f52ec06be30 0x7f52ec06e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f530015e5b0 0x7f5300143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f530015e1d0 0x7f53000d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 --2- 172.21.15.1:0/1589185737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f530006ffa0 0x7f53001f7100 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 >> 172.21.15.1:0/1589185737 conn(0x7f53000d1d70 msgr2=0x7f53000afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 shutdown_connections 2024-05-09T06:35:38.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:38.969+0000 7f5305102640 1 -- 172.21.15.1:0/1589185737 wait complete. 2024-05-09T06:35:38.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T06:35:39.090 INFO:teuthology.orchestra.run.smithi001.stdout:{"message":"","return_code":0} 2024-05-09T06:35:39.091 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:35:39.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:35:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.215+0000 7f96d84fc640 1 Processor -- start 2024-05-09T06:35:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.215+0000 7f96d84fc640 1 -- start start 2024-05-09T06:35:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d84fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f96d0163c70 0x7f96d0166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d84fc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d00ce0e0 con 0x7f96d006e6d0 2024-05-09T06:35:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d84fc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d00ce3c0 con 0x7f96d00cf180 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d84fc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f96d00ce6a0 con 0x7f96d0163c70 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d6a99640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f96d0163c70 0x7f96d0166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d6298640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f96d006e6d0 0x7f96d006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56244/0 (socket says 172.21.15.1:56244) 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.216+0000 7f96d6298640 1 -- 172.21.15.1:0/2070173278 learned_addr learned my addr 172.21.15.1:0/2070173278 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d6a99640 1 -- 172.21.15.1:0/2070173278 >> v1:172.21.15.6:6789/0 conn(0x7f96d00cf180 legacy=0x7f96d00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d6a99640 1 -- 172.21.15.1:0/2070173278 >> v1:172.21.15.1:6789/0 conn(0x7f96d006e6d0 legacy=0x7f96d006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d5296640 1 -- 172.21.15.1:0/2070173278 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202961204 0 0) 0x7f96d00ce0e0 con 0x7f96d006e6d0 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d6a99640 1 -- 172.21.15.1:0/2070173278 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d00ce980 con 0x7f96d0163c70 2024-05-09T06:35:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d6a99640 1 --2- 172.21.15.1:0/2070173278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d0166060 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f3cbbae5ba9fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.217+0000 7f96d5296640 1 -- 172.21.15.1:0/2070173278 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8037070 con 0x7f96d0163c70 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d5296640 1 -- 172.21.15.1:0/2070173278 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c8032440 con 0x7f96d0163c70 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d5296640 1 -- 172.21.15.1:0/2070173278 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8032810 con 0x7f96d0163c70 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 -- 172.21.15.1:0/2070173278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 msgr2=0x7f96d0166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2070173278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d0166060 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802a230 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 -- 172.21.15.1:0/2070173278 shutdown_connections 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2070173278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d0166060 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 -- 172.21.15.1:0/2070173278 >> 172.21.15.1:0/2070173278 conn(0x7f96d00cc4b0 msgr2=0x7f96d00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 -- 172.21.15.1:0/2070173278 shutdown_connections 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.218+0000 7f96d84fc640 1 -- 172.21.15.1:0/2070173278 wait complete. 2024-05-09T06:35:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.219+0000 7f96d84fc640 1 Processor -- start 2024-05-09T06:35:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.219+0000 7f96d84fc640 1 -- start start 2024-05-09T06:35:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d006e6d0 0x7f96d013b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d00cf180 0x7f96d0143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01447c0 con 0x7f96d0163c70 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0070ed0 con 0x7f96d00cf180 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d84fc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96d0144940 con 0x7f96d006e6d0 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d6a99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d6a99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39722/0 (socket says 172.21.15.1:39722) 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d6a99640 1 -- 172.21.15.1:0/2982120908 learned_addr learned my addr 172.21.15.1:0/2982120908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d6298640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d006e6d0 0x7f96d013b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.220+0000 7f96d5a97640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d00cf180 0x7f96d0143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 -- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d006e6d0 msgr2=0x7f96d013b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d006e6d0 0x7f96d013b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 -- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d00cf180 msgr2=0x7f96d0143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d00cf180 0x7f96d0143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 -- 172.21.15.1:0/2982120908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d015f110 con 0x7f96d0163c70 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d6a99640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8037040 con 0x7f96d0163c70 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c8042070 con 0x7f96d0163c70 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d01f3a40 con 0x7f96d0163c70 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8030b30 con 0x7f96d0163c70 2024-05-09T06:35:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.221+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01f4000 con 0x7f96d0163c70 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.222+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d00752b0 con 0x7f96d0163c70 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.223+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96c8030cd0 con 0x7f96d0163c70 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.223+0000 7f96ceffd640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 0x7f96bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.223+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6303+0+0 (secure 0 0 0) 0x7f96c80abd10 con 0x7f96d0163c70 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.223+0000 7f96d6298640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 0x7f96bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.224+0000 7f96d6298640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 0x7f96bc06e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f96c00012e0 tx=0x7f96c0005bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.227+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96c80b1020 con 0x7f96d0163c70 2024-05-09T06:35:39.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.451+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f96d006cf50 con 0x7f96d0163c70 2024-05-09T06:35:39.432 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:35:39.451+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:35:39.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.453+0000 7f96ceffd640 1 -- 172.21.15.1:0/2982120908 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7f96c8079d80 con 0x7f96d0163c70 2024-05-09T06:35:39.433 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:35:39.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 msgr2=0x7f96bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 0x7f96bc06e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f96c00012e0 tx=0x7f96c0005bf0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 msgr2=0x7f96d015e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c80046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 shutdown_connections 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f96bc06be60 0x7f96bc06e320 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96d0163c70 0x7f96d015e8e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d00cf180 0x7f96d0143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 --2- 172.21.15.1:0/2982120908 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d006e6d0 0x7f96d013b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.455+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 >> 172.21.15.1:0/2982120908 conn(0x7f96d00cc4b0 msgr2=0x7f96d00b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.456+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 shutdown_connections 2024-05-09T06:35:39.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.456+0000 7f96d84fc640 1 -- 172.21.15.1:0/2982120908 wait complete. 2024-05-09T06:35:39.440 INFO:tasks.ceph.mds.c.smithi001.stderr: -4435> 2024-05-09T06:34:26.752+0000 7f0ee16dc640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-09T06:35:39.440 INFO:tasks.ceph.mds.c.smithi001.stderr: -4273> 2024-05-09T06:34:26.941+0000 7f0ee16dc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0) n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x564e3d142c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-09T06:35:39.440 INFO:tasks.ceph.mds.c.smithi001.stderr: -4131> 2024-05-09T06:34:28.298+0000 7f0ee56e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:35:39.442 INFO:tasks.ceph.mds.c.smithi001.stderr: -2539> 2024-05-09T06:34:58.282+0000 7f0ee16dc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)->f(v1 m2024-05-09T06:34:15.118383+0000) n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)->n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564e3d142c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-09T06:34:15.118383+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-09T06:34:15.238907+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-09T06:35:39.448 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-09T06:35:39.470 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:35:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.509+0000 7fe407a18740 1 -- start start 2024-05-09T06:35:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.509+0000 7fe407a18740 1 Processor -- start 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 0x56080c844580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 0x56080c844b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a6180 con 0x56080c7d1c00 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a6300 con 0x56080c7d0c00 2024-05-09T06:35:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56080d5a6480 con 0x56080c7d1800 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39748/0 (socket says 172.21.15.1:39748) 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 -- 172.21.15.1:0/1914082529 learned_addr learned my addr 172.21.15.1:0/1914082529 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405731640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 0x56080c844580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 -- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 msgr2=0x56080c844b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe404f30640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 0x56080c844b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 0x56080c844b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 -- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 msgr2=0x56080c844580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 0x56080c844580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 -- 172.21.15.1:0/1914082529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7fd680 con 0x56080c7d1c00 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe404f30640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 0x56080c844b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405731640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 0x56080c844580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe405f32640 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x56080d5e5c50 tx=0x56080d56a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9b783d1379c0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe402f2c640 1 -- 172.21.15.1:0/1914082529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f6680 con 0x56080c7d1c00 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe402f2c640 1 -- 172.21.15.1:0/1914082529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56080c81ae00 con 0x56080c7d1c00 2024-05-09T06:35:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.510+0000 7fe402f2c640 1 -- 172.21.15.1:0/1914082529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f6b60 con 0x56080c7d1c00 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c7fd4a0 con 0x56080c7d1c00 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56080c7fda40 con 0x56080c7d1c00 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 4 waiting for MDS map... 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe402f2c640 1 -- 172.21.15.1:0/1914082529 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x56080d6a0600 con 0x56080c7d1c00 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe402f2c640 1 -- 172.21.15.1:0/1914082529 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x56080c829180 con 0x56080c7d1c00 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 4 Got MDS map 69 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 4 datascan.main: Using metadata pool output 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 4 datascan.main: connecting to RADOS... 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 1 Processor -- start 2024-05-09T06:35:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.511+0000 7fe407a18740 1 -- start start 2024-05-09T06:35:39.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3400 0x56080c845600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c845b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a6600 con 0x56080d6a3c00 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a6780 con 0x56080d6a3400 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56080d5a6900 con 0x56080d6a3800 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39752/0 (socket says 172.21.15.1:39752) 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 -- 172.21.15.1:0/3792698936 learned_addr learned my addr 172.21.15.1:0/3792698936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3400 0x56080c845600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe405731640 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c845b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 -- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 msgr2=0x56080c845b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c845b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 -- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3400 msgr2=0x56080c845600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3400 0x56080c845600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.512+0000 7fe404f30640 1 -- 172.21.15.1:0/3792698936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7fd860 con 0x56080d6a3c00 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe404f30640 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x56080d5e5200 tx=0x56080d56bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7184fd1cddf8be12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe400f28640 1 -- 172.21.15.1:0/3792698936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f69c0 con 0x56080d6a3c00 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe400f28640 1 -- 172.21.15.1:0/3792698936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56080c81ac40 con 0x56080d6a3c00 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe400f28640 1 -- 172.21.15.1:0/3792698936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f6680 con 0x56080d6a3c00 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 -- 172.21.15.1:0/3792698936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 msgr2=0x56080c846100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x56080d5e5200 tx=0x56080d56bf20 comp rx=0 tx=0).stop 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 -- 172.21.15.1:0/3792698936 shutdown_connections 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3c00 0x56080c846100 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c845b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 --2- 172.21.15.1:0/3792698936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3400 0x56080c845600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 -- 172.21.15.1:0/3792698936 >> 172.21.15.1:0/3792698936 conn(0x56080d6a3000 msgr2=0x56080d555000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 -- 172.21.15.1:0/3792698936 shutdown_connections 2024-05-09T06:35:39.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.513+0000 7fe407a18740 1 -- 172.21.15.1:0/3792698936 wait complete. 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 Processor -- start 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 -- start start 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c846c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3c00 0x56080c846680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a7980 con 0x56080d6a3400 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56080d5a7800 con 0x56080d6a3c00 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe407a18740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56080d5a7680 con 0x56080d6a3800 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe404f30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe404f30640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39756/0 (socket says 172.21.15.1:39756) 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 learned_addr learned my addr 172.21.15.1:0/3130549781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.514+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c846c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3c00 0x56080c846680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 msgr2=0x56080c846c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080d6a3800 0x56080c846c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3c00 msgr2=0x56080c846680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080d6a3c00 0x56080c846680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7fd680 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x56080d6c3710 tx=0x56080d56bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe3ff725640 1 -- 172.21.15.1:0/3130549781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f6b60 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe3ff725640 1 -- 172.21.15.1:0/3130549781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56080c81aa80 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56080c7fd4a0 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe3ff725640 1 -- 172.21.15.1:0/3130549781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56080c7f64e0 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c7fd2c0 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.515+0000 7fe407a18740 4 datascan.init: resolving metadata pool 13 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.516+0000 7fe3ff725640 1 -- 172.21.15.1:0/3130549781 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x56080d698c00 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.516+0000 7fe3ff725640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 0x56080c847700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe3ff725640 1 -- 172.21.15.1:0/3130549781 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x56080d5c8600 con 0x56080d6a3400 2024-05-09T06:35:39.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe407a18740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe407a18740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe407a18740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 0x56080c847700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56080cb42400 0x56080c846100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb42800 con 0x56080cb42400 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56080cb42400 0x56080c846100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 0x56080c847700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56080d56d020 tx=0x56080cad2ae0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.517+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56080cb42400 0x56080c846100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.518+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.518+0000 7fe400727640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56080cb56000 0x56080cb5c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.518+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56400 con 0x56080cb56000 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.518+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56080cb56000 0x56080cb5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.518+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56080cb56000 0x56080cb5c000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.520+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.520+0000 7fe400f28640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56080cb56c00 0x56080cb5c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.520+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57000 con 0x56080cb56c00 2024-05-09T06:35:39.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.520+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56080cb56c00 0x56080cb5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.521+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56080cb56c00 0x56080cb5c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56800 con 0x56080cb56000 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57400 con 0x56080cb56000 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe400727640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56080cb57800 0x56080cb5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57c00 con 0x56080cb57800 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.523+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56080cb57800 0x56080cb5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.524+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56080cb57800 0x56080cb5cb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.525+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.525+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6e000 con 0x56080cb56c00 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6ec00 con 0x56080cb56c00 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(8 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe400f28640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56080cb6e400 0x56080cb5d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6e800 con 0x56080cb6e400 2024-05-09T06:35:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.526+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56080cb6e400 0x56080cb5d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.527+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56080cb6e400 0x56080cb5d600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.529+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6e400 2024-05-09T06:35:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.529+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f000 con 0x56080cb56c00 2024-05-09T06:35:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.529+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.529+0000 7fe400f28640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56080cb57000 0x56080cb5db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.529+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56400 con 0x56080cb57000 2024-05-09T06:35:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.530+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56080cb57000 0x56080cb5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.530+0000 7fe405731640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56080cb57000 0x56080cb5db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.530+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.530+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f400 con 0x56080cb42400 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56800 con 0x56080cb56000 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(13 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f800 con 0x56080cb56c00 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.531+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d6a2800 con 0x56080cb56000 2024-05-09T06:35:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.532+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(15 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.532+0000 7fe400727640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56080cb6e000 0x56080cb5d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.532+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6ec00 con 0x56080cb6e000 2024-05-09T06:35:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.532+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56080cb6e000 0x56080cb5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.532+0000 7fe404f30640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56080cb6e000 0x56080cb5d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f400 con 0x56080cb6e400 2024-05-09T06:35:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6e400 2024-05-09T06:35:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f000 con 0x56080cb57000 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.535+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6ec00 con 0x56080cb42400 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.536+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.536+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6e800 con 0x56080cb57000 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.536+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.536+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57c00 con 0x56080cb6e000 2024-05-09T06:35:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.536+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57400 con 0x56080cb57000 2024-05-09T06:35:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb42800 con 0x56080cb56000 2024-05-09T06:35:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(23 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f800 con 0x56080cb56000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(24 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.537+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d5a9400 con 0x56080cb56000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f000 con 0x56080cb57000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d5a9400 con 0x56080cb57800 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(27 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.538+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6e800 con 0x56080cb42400 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f400 con 0x56080cb57000 2024-05-09T06:35:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6fc00 con 0x56080cb57800 2024-05-09T06:35:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe400f28640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56080cb6f800 0x56080cb5e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb42800 con 0x56080cb6f800 2024-05-09T06:35:39.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.539+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56080cb6f800 0x56080cb5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.540+0000 7fe405f32640 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56080cb6f800 0x56080cb5e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.542+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(31 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.542+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56400 con 0x56080cb6e000 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.542+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(32 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d6a2800 con 0x56080cb57800 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(33 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57400 con 0x56080cb6e000 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(34 [pgnls start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56800 con 0x56080cb57000 2024-05-09T06:35:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 7 ==== osd_op_reply(35 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.543+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57c00 con 0x56080cb56c00 2024-05-09T06:35:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.544+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(36 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.544+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88000 con 0x56080cb6e000 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.544+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(37 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.544+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56400 con 0x56080cb6e400 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.544+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(38 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6e400 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88400 con 0x56080cb6e400 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(39 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6e400 2024-05-09T06:35:39.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6ec00 con 0x56080cb6f800 2024-05-09T06:35:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(40 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88800 con 0x56080cb56000 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 9 ==== osd_op_reply(41 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.545+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6e800 con 0x56080cb42400 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.546+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 5 ==== osd_op_reply(42 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.546+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88c00 con 0x56080cb6f800 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.546+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(43 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.546+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6fc00 con 0x56080cb57000 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 8 ==== osd_op_reply(44 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb89000 con 0x56080cb6e400 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.6 v2:172.21.15.6:6817/1315796905 5 ==== osd_op_reply(45 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6e400 2024-05-09T06:35:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d5a9400 con 0x56080cb57000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 9 ==== osd_op_reply(46 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.547+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb89400 con 0x56080cb56000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 10 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f000 con 0x56080cb57800 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 5 ==== osd_op_reply(48 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb89800 con 0x56080cb56000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 11 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.548+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6f400 con 0x56080cb6f800 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88400 con 0x56080cb42400 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.2 v2:172.21.15.1:6808/1632480843 6 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb42400 2024-05-09T06:35:39.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6fc00 con 0x56080cb56c00 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(52 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88800 con 0x56080cb56000 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 12 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.549+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56800 con 0x56080cb57000 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 10 ==== osd_op_reply(54 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88c00 con 0x56080cb6f800 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb57c00 con 0x56080cb57000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 11 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.550+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb89000 con 0x56080cb6e000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.551+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v91'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.551+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56080cb42c00 con 0x56080cb6e000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.551+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.551+0000 7fe407a18740 10 datascan.operator(): handling object 1099511627777.0 2024-05-09T06:35:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.551+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56080cb43000 con 0x56080cb6e000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.553+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(59 10000000001.00000000 [call] v95'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.553+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb43400 con 0x56080cb6e000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.553+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.5 v2:172.21.15.6:6801/2308051940 9 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb6e000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.553+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb89c00 con 0x56080cb56c00 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.554+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(61 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb56c00 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.554+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb56400 con 0x56080cb57000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.554+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 <== osd.4 v2:172.21.15.6:6800/3928343737 12 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544780 con 0x56080cb57000 2024-05-09T06:35:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.554+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb6ec00 con 0x56080cb6f800 2024-05-09T06:35:39.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb6f800 2024-05-09T06:35:39.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb42800 con 0x56080cb56000 2024-05-09T06:35:39.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe400f28640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080cb88000 con 0x56080cb56000 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe405f32640 1 -- 172.21.15.1:0/3130549781 <== osd.0 v2:172.21.15.1:6819/3367065226 14 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544500 con 0x56080cb56000 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe400727640 1 -- 172.21.15.1:0/3130549781 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56080d5a9400 con 0x56080cb57800 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe404f30640 1 -- 172.21.15.1:0/3130549781 <== osd.3 v2:172.21.15.1:6800/2131549113 6 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56080d544000 con 0x56080cb57800 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56080cb56000 msgr2=0x56080cb5c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56080cb56000 0x56080cb5c000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56080cb56c00 msgr2=0x56080cb5c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56080cb56c00 0x56080cb5c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56080cb42400 msgr2=0x56080c846100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56080cb42400 0x56080c846100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56080cb57800 msgr2=0x56080cb5cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56080cb57800 0x56080cb5cb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56080cb57000 msgr2=0x56080cb5db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56080cb57000 0x56080cb5db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56080cb6e000 msgr2=0x56080cb5d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56080cb6e000 0x56080cb5d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 reap_dead start 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56080cb6e400 msgr2=0x56080cb5d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56080cb6e400 0x56080cb5d600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 reap_dead start 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56080cb6f800 msgr2=0x56080cb5e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.555+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56080cb6f800 0x56080cb5e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe405731640 1 -- 172.21.15.1:0/3130549781 reap_dead start 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 msgr2=0x56080c847700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 0x56080c847700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56080d56d020 tx=0x56080cad2ae0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 msgr2=0x56080c847180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x56080d6c3710 tx=0x56080d56bf80 comp rx=0 tx=0).stop 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 shutdown_connections 2024-05-09T06:35:39.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56080cb42000 0x56080c847700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/3130549781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080d6a3400 0x56080c847180 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 >> 172.21.15.1:0/3130549781 conn(0x56080d6a3000 msgr2=0x56080d555000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 shutdown_connections 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/3130549781 wait complete. 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 msgr2=0x56080c845080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x56080d5e5c50 tx=0x56080d56a660 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 shutdown_connections 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56080c7d1c00 0x56080c845080 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56080c7d1800 0x56080c844b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 --2- 172.21.15.1:0/1914082529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56080c7d0c00 0x56080c844580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 >> 172.21.15.1:0/1914082529 conn(0x56080c7d1000 msgr2=0x56080d554800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 shutdown_connections 2024-05-09T06:35:39.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.556+0000 7fe407a18740 1 -- 172.21.15.1:0/1914082529 wait complete. 2024-05-09T06:35:39.542 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.093955, result: 2024-05-09T06:35:39.543 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-09T06:35:39.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 -- start start 2024-05-09T06:35:39.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 Processor -- start 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x563465a73800 0x563465ae6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563465a73c00 0x563465ae7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56346685c600 con 0x563465a72c00 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56346685c480 con 0x563465a73c00 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56346685c300 con 0x563465a73800 2024-05-09T06:35:39.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39762/0 (socket says 172.21.15.1:39762) 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27b7c95640 1 -- 172.21.15.1:0/1502040220 learned_addr learned my addr 172.21.15.1:0/1502040220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.644+0000 7f27b7494640 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x563465a73800 0x563465ae6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b8496640 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563465a73c00 0x563465ae7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 -- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x563465a73800 msgr2=0x563465ae6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x563465a73800 0x563465ae6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 -- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563465a73c00 msgr2=0x563465ae7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563465a73c00 0x563465ae7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 -- 172.21.15.1:0/1502040220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563465a9f680 con 0x563465a72c00 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b7c95640 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5634668971a0 tx=0x563466810d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70117ae1d6f0e687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b5490640 1 -- 172.21.15.1:0/1502040220 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a989c0 con 0x563465a72c00 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b5490640 1 -- 172.21.15.1:0/1502040220 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563465abce00 con 0x563465a72c00 2024-05-09T06:35:39.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563465a9f4a0 con 0x563465a72c00 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563465a9fa40 con 0x563465a72c00 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27ba00d740 4 waiting for MDS map... 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b5490640 1 -- 172.21.15.1:0/1502040220 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a98680 con 0x563465a72c00 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b5490640 1 -- 172.21.15.1:0/1502040220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x563466948200 con 0x563465a72c00 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27b5490640 1 -- 172.21.15.1:0/1502040220 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x563465acb180 con 0x563465a72c00 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.645+0000 7f27ba00d740 4 Got MDS map 69 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.646+0000 7f27ba00d740 4 datascan.main: Using metadata pool output 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.646+0000 7f27ba00d740 4 datascan.main: connecting to RADOS... 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.646+0000 7f27ba00d740 1 Processor -- start 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.646+0000 7f27ba00d740 1 -- start start 2024-05-09T06:35:39.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 0x563465ae7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac800 0x563465ae8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56346685c000 con 0x5634669ac400 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563465ad2180 con 0x5634669ac800 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x563465ad2900 con 0x5634669ac000 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39770/0 (socket says 172.21.15.1:39770) 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 -- 172.21.15.1:0/3923335312 learned_addr learned my addr 172.21.15.1:0/3923335312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b8496640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 0x563465ae7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7494640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac800 0x563465ae8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 -- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 msgr2=0x563465ae7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 0x563465ae7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 -- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac800 msgr2=0x563465ae8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac800 0x563465ae8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 -- 172.21.15.1:0/3923335312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563465a9f860 con 0x5634669ac400 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b8496640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 0x563465ae7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b7c95640 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x563466897a10 tx=0x563466810e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8336abf6e7f9394 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b348c640 1 -- 172.21.15.1:0/3923335312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a98b60 con 0x5634669ac400 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b348c640 1 -- 172.21.15.1:0/3923335312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563465abcc40 con 0x5634669ac400 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.647+0000 7f27b348c640 1 -- 172.21.15.1:0/3923335312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a98820 con 0x5634669ac400 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- 172.21.15.1:0/3923335312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 msgr2=0x563465ae7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x563466897a10 tx=0x563466810e40 comp rx=0 tx=0).stop 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- 172.21.15.1:0/3923335312 shutdown_connections 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac800 0x563465ae8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae7b80 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 --2- 172.21.15.1:0/3923335312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac000 0x563465ae7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- 172.21.15.1:0/3923335312 >> 172.21.15.1:0/3923335312 conn(0x563466865c00 msgr2=0x5634667f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- 172.21.15.1:0/3923335312 shutdown_connections 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- 172.21.15.1:0/3923335312 wait complete. 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 Processor -- start 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.648+0000 7f27ba00d740 1 -- start start 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac000 0x563465ae9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac800 0x563465ae8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56346685d380 con 0x5634669ac400 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56346685d200 con 0x5634669ac000 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27ba00d740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56346685d500 con 0x5634669ac800 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39780/0 (socket says 172.21.15.1:39780) 2024-05-09T06:35:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 learned_addr learned my addr 172.21.15.1:0/2606344983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac000 0x563465ae9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac800 0x563465ae8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac800 msgr2=0x563465ae8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5634669ac800 0x563465ae8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac000 msgr2=0x563465ae9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5634669ac000 0x563465ae9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.649+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563465a9f2c0 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x563466954a50 tx=0x563466810060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27b1c89640 1 -- 172.21.15.1:0/2606344983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a98b60 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27b1c89640 1 -- 172.21.15.1:0/2606344983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563465abca80 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563465a9f0e0 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27b1c89640 1 -- 172.21.15.1:0/2606344983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563465a98820 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563465a9e3c0 con 0x5634669ac400 2024-05-09T06:35:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.650+0000 7f27ba00d740 4 datascan.init: resolving metadata pool 13 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27b1c89640 1 -- 172.21.15.1:0/2606344983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x5634669c2c00 con 0x5634669ac400 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27b1c89640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 0x563465ae9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27b1c89640 1 -- 172.21.15.1:0/2606344983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x5634669c0c00 con 0x5634669ac400 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27ba00d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27ba00d740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27ba00d740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 0x563465ae9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.651+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x5634669acc00 0x563465ae8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5634669ad000 con 0x5634669acc00 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x5634669acc00 0x563465ae8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 0x563465ae9700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x56346680ee40 tx=0x563465d86ae0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x5634669acc00 0x563465ae8100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563465df5000 0x563465ae7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.652+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465df5000 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563465df5000 0x563465ae7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563465df5000 0x563465ae7b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563465df5800 0x563465ae7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5c00 con 0x563465df5800 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.653+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563465df5800 0x563465ae7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563465df5800 0x563465ae7600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x5634669acc00 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b348c640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5634669ad000 0x563465e1a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x5634669ad000 2024-05-09T06:35:39.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.654+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5634669ad000 0x563465e1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5634669ad000 0x563465e1a000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12800 con 0x5634669acc00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x5634669acc00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b2c8b640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x563465e12c00 0x563465e1a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.655+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x563465e12c00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x563465e12c00 0x563465e1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x563465e12c00 0x563465e1a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x5634669ad000 2024-05-09T06:35:39.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.656+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12800 con 0x5634669ad000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.657+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.657+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465df5000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.657+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.657+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x5634669ad000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b348c640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563465e13800 0x563465e1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x563465e13800 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563465e13800 0x563465e1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b7c95640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563465e13800 0x563465e1ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.658+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x563465df5800 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x5634669acc00 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x5634669ad000 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df4c00 con 0x5634669acc00 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b2c8b640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563465e12800 0x563465e1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.659+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465e12800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.660+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563465e12800 0x563465e1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.660+0000 7f27b7494640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563465e12800 0x563465e1b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.660+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.660+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465df5000 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465e13800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x563465df5800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.661+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465e13800 2024-05-09T06:35:39.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.662+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.662+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e400 con 0x563465e12800 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.662+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.662+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f800 con 0x563465e13800 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e800 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2fc00 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2ec00 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.663+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x563465e13800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x563465e12c00 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465df5800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.664+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e400 con 0x563465e13800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x563465e12c00 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b348c640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x563465e2e800 0x563465e1b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x563465e2e800 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.665+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x563465e2e800 0x563465e1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.666+0000 7f27b8496640 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x563465e2e800 0x563465e1b080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.666+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.666+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.666+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.666+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465e12c00 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x563465e13800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.667+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df4c00 con 0x5634669ad000 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5c00 con 0x563465df5000 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.668+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465df5000 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2fc00 con 0x563465e2e800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x5634669acc00 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.669+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2ec00 con 0x563465df5800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465e2e800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x563465e13800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.670+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f800 con 0x563465df5000 2024-05-09T06:35:39.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df4c00 con 0x563465e13800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e400 con 0x5634669acc00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.671+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x563465e12c00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x5634669acc00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465e2e800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.672+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2ec00 con 0x563465df5800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x5634669ad000 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x5634669acc00 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.673+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465e13800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x563465e2e800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(57 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2fc00 con 0x563465e13800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.674+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465e12800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3c000 con 0x563465df5800 2024-05-09T06:35:39.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3dc00 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.675+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f800 con 0x5634669ad000 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(62 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3dc00 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5c00 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x563465e12c00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.676+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x5634669ad000 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42000 con 0x563465df5000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.677+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 11 ==== osd_op_reply(69 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.678+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42400 con 0x563465e13800 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.678+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.678+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e400 con 0x563465df5800 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.678+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.678+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42800 con 0x5634669acc00 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42c00 con 0x5634669acc00 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x563465e12800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.679+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43000 con 0x563465df5000 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f800 con 0x563465e13800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43400 con 0x563465df5800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.680+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x563465e13800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43800 con 0x563465e12800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42800 con 0x563465e13800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43c00 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.681+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2ec00 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465e13800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465e12c00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.682+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5c00 con 0x563465df5800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2fc00 con 0x563465e13800 2024-05-09T06:35:39.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465e12c00 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.683+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e400 con 0x563465e2e800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(90 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3dc00 con 0x563465e12800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f800 con 0x563465e12c00 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465e12800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.684+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42400 con 0x563465e13800 2024-05-09T06:35:39.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x5634669ad000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 13 ==== osd_op_reply(95 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3c000 con 0x563465e12800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 11 ==== osd_op_reply(96 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.685+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42000 con 0x563465df5000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df4c00 con 0x563465df5000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43000 con 0x563465e2e800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 7 ==== osd_op_reply(99 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.686+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x5634669acc00 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465df5800 2024-05-09T06:35:39.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2ec00 con 0x563465e2e800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 8 ==== osd_op_reply(102 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x563465e13800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.687+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f400 con 0x563465df5000 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12000 con 0x563465e13800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x5634669acc00 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.688+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465e12c00 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e3dc00 con 0x5634669acc00 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13400 con 0x563465e2e800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 9 ==== osd_op_reply(109 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.689+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13c00 con 0x563465df5800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.2 v2:172.21.15.1:6808/1632480843 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465df5800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x5634669ad000 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42c00 con 0x5634669acc00 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43400 con 0x563465e13800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.690+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43800 con 0x563465e2e800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.691+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 10 ==== osd_op_reply(114 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.691+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42800 con 0x563465e13800 2024-05-09T06:35:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.691+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.691+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e43c00 con 0x563465e12800 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 10 datascan.operator(): handling object 10000000001.0 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465e3c800 con 0x563465e12800 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465e3cc00 con 0x5634669ad000 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 15 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.692+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465e3d000 con 0x563465df5000 2024-05-09T06:35:39.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.6 v2:172.21.15.6:6817/1315796905 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x563465be6500 con 0x563465df5000 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.get_frag_of: frag is * 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465e3d400 con 0x5634669ad000 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 16 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x563465e3d800 con 0x5634669ad000 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 17 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 20 datascan.read_dentry: file_head not found in result 2024-05-09T06:35:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-09T06:35:39.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.693+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x563465e42c00 con 0x5634669ad000 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.694+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 18 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v95'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.694+0000 7f27ba00d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.694+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465df5400 con 0x563465e12800 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.695+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.5 v2:172.21.15.6:6801/2308051940 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x563465e12800 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.695+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e13000 con 0x5634669ad000 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.695+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.1 v2:172.21.15.1:6816/1858675679 19 ==== osd_op_reply(124 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669ad000 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.695+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e42400 con 0x563465e13800 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.695+0000 7f27b7c95640 1 -- 172.21.15.1:0/2606344983 <== osd.4 v2:172.21.15.6:6800/3928343737 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6780 con 0x563465e13800 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e12400 con 0x563465e2e800 2024-05-09T06:35:39.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.7 v2:172.21.15.6:6816/3240730649 11 ==== osd_op_reply(126 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e2e800 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2f000 con 0x5634669acc00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b2c8b640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2e000 con 0x5634669acc00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 <== osd.0 v2:172.21.15.1:6819/3367065226 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6000 con 0x5634669acc00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.696+0000 7f27b348c640 1 -- 172.21.15.1:0/2606344983 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x563465e2fc00 con 0x563465e12c00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27b8496640 1 -- 172.21.15.1:0/2606344983 <== osd.3 v2:172.21.15.1:6800/2131549113 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563465be6500 con 0x563465e12c00 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x5634669acc00 msgr2=0x563465ae8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x5634669acc00 0x563465ae8100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5634669ad000 msgr2=0x563465e1a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5634669ad000 0x563465e1a000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563465df5800 msgr2=0x563465ae7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563465df5800 0x563465ae7600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x563465e12c00 msgr2=0x563465e1a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x563465e12c00 0x563465e1a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563465e13800 msgr2=0x563465e1ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563465e13800 0x563465e1ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563465e12800 msgr2=0x563465e1b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563465e12800 0x563465e1b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563465df5000 msgr2=0x563465ae7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563465df5000 0x563465ae7b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x563465e2e800 msgr2=0x563465e1b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x563465e2e800 0x563465e1b080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 reap_dead start 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 reap_dead start 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 reap_dead start 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27b7494640 1 -- 172.21.15.1:0/2606344983 reap_dead start 2024-05-09T06:35:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 msgr2=0x563465ae9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 0x563465ae9700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x56346680ee40 tx=0x563465d86ae0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 msgr2=0x563465ae8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x563466954a50 tx=0x563466810060 comp rx=0 tx=0).stop 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 shutdown_connections 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563465df4000 0x563465ae9700 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/2606344983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634669ac400 0x563465ae8c00 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 >> 172.21.15.1:0/2606344983 conn(0x563466865c00 msgr2=0x5634667f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 shutdown_connections 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/2606344983 wait complete. 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 msgr2=0x563465ae6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5634668971a0 tx=0x563466810d80 comp rx=0 tx=0).stop 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 shutdown_connections 2024-05-09T06:35:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563465a73c00 0x563465ae7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x563465a73800 0x563465ae6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.697+0000 7f27ba00d740 1 --2- 172.21.15.1:0/1502040220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563465a72c00 0x563465ae6580 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.698+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 >> 172.21.15.1:0/1502040220 conn(0x563465a73000 msgr2=0x5634667f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.698+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 shutdown_connections 2024-05-09T06:35:39.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.698+0000 7f27ba00d740 1 -- 172.21.15.1:0/1502040220 wait complete. 2024-05-09T06:35:39.702 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.159445, result: 2024-05-09T06:35:39.702 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-09T06:35:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 -- start start 2024-05-09T06:35:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 Processor -- start 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf4400 0x56474ce91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0000 0x56474ce90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2000 con 0x56474dbf5c00 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2180 con 0x56474dbf0000 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56474dbf2300 con 0x56474dbf4400 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.799+0000 7fed2ba06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39796/0 (socket says 172.21.15.1:39796) 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 -- 172.21.15.1:0/2328936614 learned_addr learned my addr 172.21.15.1:0/2328936614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2c207640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf4400 0x56474ce91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ca08640 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0000 0x56474ce90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 -- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf4400 msgr2=0x56474ce91600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf4400 0x56474ce91600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 -- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0000 msgr2=0x56474ce90580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0000 0x56474ce90580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 -- 172.21.15.1:0/2328936614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474ce49680 con 0x56474dbf5c00 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2ba06640 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x56474dc459e0 tx=0x56474dbb6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab4c776fa565a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed29a02640 1 -- 172.21.15.1:0/2328936614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42d00 con 0x56474dbf5c00 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed29a02640 1 -- 172.21.15.1:0/2328936614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56474ce66c40 con 0x56474dbf5c00 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed29a02640 1 -- 172.21.15.1:0/2328936614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42680 con 0x56474dbf5c00 2024-05-09T06:35:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474ce49860 con 0x56474dbf5c00 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56474ce490e0 con 0x56474dbf5c00 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2e60c740 4 waiting for MDS map... 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed29a02640 1 -- 172.21.15.1:0/2328936614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x56474dcfe400 con 0x56474dbf5c00 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed29a02640 1 -- 172.21.15.1:0/2328936614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x56474ce75180 con 0x56474dbf5c00 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.800+0000 7fed2e60c740 4 Got MDS map 69 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.801+0000 7fed2e60c740 4 datascan.main: Using metadata pool output 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.801+0000 7fed2e60c740 4 datascan.main: connecting to RADOS... 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.801+0000 7fed2e60c740 1 Processor -- start 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.801+0000 7fed2e60c740 1 -- start start 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0800 0x56474ce91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0c00 0x56474ce92100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2480 con 0x56474dbf1000 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2600 con 0x56474dbf0c00 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56474dbf2780 con 0x56474dbf0800 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39804/0 (socket says 172.21.15.1:39804) 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 -- 172.21.15.1:0/3228944772 learned_addr learned my addr 172.21.15.1:0/3228944772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ca08640 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0800 0x56474ce91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2c207640 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0c00 0x56474ce92100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 -- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0800 msgr2=0x56474ce91b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0800 0x56474ce91b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 -- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0c00 msgr2=0x56474ce92100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0c00 0x56474ce92100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 -- 172.21.15.1:0/3228944772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474ce494a0 con 0x56474dbf1000 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed2ba06640 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x56474dc202d0 tx=0x56474dbb6a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18de21535b43b7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed279fe640 1 -- 172.21.15.1:0/3228944772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42b60 con 0x56474dbf1000 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed279fe640 1 -- 172.21.15.1:0/3228944772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56474ce66a80 con 0x56474dbf1000 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.802+0000 7fed279fe640 1 -- 172.21.15.1:0/3228944772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42d00 con 0x56474dbf1000 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 -- 172.21.15.1:0/3228944772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 msgr2=0x56474ce92680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x56474dc202d0 tx=0x56474dbb6a80 comp rx=0 tx=0).stop 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 -- 172.21.15.1:0/3228944772 shutdown_connections 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf1000 0x56474ce92680 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0c00 0x56474ce92100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 --2- 172.21.15.1:0/3228944772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0800 0x56474ce91b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 -- 172.21.15.1:0/3228944772 >> 172.21.15.1:0/3228944772 conn(0x56474dbf0400 msgr2=0x56474dba1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 -- 172.21.15.1:0/3228944772 shutdown_connections 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 -- 172.21.15.1:0/3228944772 wait complete. 2024-05-09T06:35:39.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.803+0000 7fed2e60c740 1 Processor -- start 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 -- start start 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0c00 0x56474ce93180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf1000 0x56474ce92c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2a80 con 0x56474dbf0800 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56474dbf2900 con 0x56474dbf1000 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2e60c740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56474dbf3500 con 0x56474dbf0c00 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2ca08640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2ca08640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39810/0 (socket says 172.21.15.1:39810) 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 learned_addr learned my addr 172.21.15.1:0/1090472545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0c00 0x56474ce93180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.804+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf1000 0x56474ce92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0c00 msgr2=0x56474ce93180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf0c00 0x56474ce93180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf1000 msgr2=0x56474ce92c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf1000 0x56474ce92c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474ce49680 con 0x56474dbf0800 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf1000 0x56474ce92c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x56474dc21e90 tx=0x56474dc25800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed261fb640 1 -- 172.21.15.1:0/1090472545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42b60 con 0x56474dbf0800 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed261fb640 1 -- 172.21.15.1:0/1090472545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56474ce661c0 con 0x56474dbf0800 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56474ce492c0 con 0x56474dbf0800 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed261fb640 1 -- 172.21.15.1:0/1090472545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56474ce42d00 con 0x56474dbf0800 2024-05-09T06:35:39.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474ce49a40 con 0x56474dbf0800 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.805+0000 7fed2e60c740 4 datascan.init: resolving metadata pool 13 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.806+0000 7fed261fb640 1 -- 172.21.15.1:0/1090472545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x56474dd6cc00 con 0x56474dbf0800 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.806+0000 7fed261fb640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 0x56474d1a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.806+0000 7fed261fb640 1 -- 172.21.15.1:0/1090472545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x56474dd72600 con 0x56474dbf0800 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2e60c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2e60c740 4 datascan.main: resolving metadata pool 13 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 0x56474d1a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 0x56474ce92680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75000 con 0x56474dd74c00 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 0x56474ce92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 0x56474d1a6000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56474dc45860 tx=0x56474d134e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.807+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 0x56474ce92680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56474dd75000 0x56474d1a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dbf1c00 con 0x56474dd75000 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56474dd75000 0x56474d1a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56474dd75000 0x56474d1a6580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56474dbf1c00 0x56474d1a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.808+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd74400 con 0x56474dbf1c00 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56474dbf1c00 0x56474d1a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56474dbf1c00 0x56474d1a6b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd74400 con 0x56474dd74c00 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56474dd74400 0x56474d1a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.809+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dbf1800 con 0x56474dd74400 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.810+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56474dd74400 0x56474d1a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.810+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56474dd74400 0x56474d1a7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.810+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.810+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56474dbf1800 0x56474d1a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.810+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0400 con 0x56474dbf1800 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56474dbf1800 0x56474d1a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56474dbf1800 0x56474d1a7b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0400 con 0x56474dd75000 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.811+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0400 con 0x56474dd74400 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56474d1b0400 0x56474d1a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0800 con 0x56474d1b0400 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56474d1b0400 0x56474d1a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2c207640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56474d1b0400 0x56474d1a8100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.812+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0800 con 0x56474dbf1800 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0800 con 0x56474dbf1c00 2024-05-09T06:35:39.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(11 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0800 con 0x56474dd75000 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56474d1b0800 0x56474d1a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1800 con 0x56474d1b0800 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.813+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56474d1b0800 0x56474d1a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.814+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56474d1b0800 0x56474d1a7080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.814+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.814+0000 7fed2ba06640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56474d1b1800 0x56474d1a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.814+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b1800 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.814+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56474d1b1800 0x56474d1a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ca08640 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56474d1b1800 0x56474d1a8680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474d1b1800 2024-05-09T06:35:39.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74c00 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0800 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.815+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74400 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd75000 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0400 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.816+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b1800 2024-05-09T06:35:39.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474d1b1800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.817+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74c00 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dbf1800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.818+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74c00 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0800 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(28 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.819+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74c00 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474d1b0400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.820+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dd74400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 5 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75400 con 0x56474dd75000 2024-05-09T06:35:39.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75800 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.821+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 20 datascan.scan_links: injected ino detected 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6000 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6400 con 0x56474dd75000 2024-05-09T06:35:39.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6800 con 0x56474dbf1800 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.822+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6c00 con 0x56474dd74400 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-09T06:35:39.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7000 con 0x56474dbf1c00 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7400 con 0x56474dd74c00 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.823+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7800 con 0x56474d1b0800 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-09T06:35:39.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6000 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.824+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7c00 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75400 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75800 con 0x56474dd74400 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.825+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 11 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd75000 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 10 ==== osd_op_reply(48 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dbf1c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 12 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.826+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 8 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dbf1800 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(52 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd75000 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 11 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.827+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 9 ==== osd_op_reply(54 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dbf1800 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.828+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dbf1c00 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 5 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd75000 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 12 ==== osd_op_reply(58 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 8 ==== osd_op_reply(59 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b1800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474d1b1800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.829+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 13 ==== osd_op_reply(61 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 9 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74400 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 10 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.830+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd75000 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0400 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b1800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474d1b1800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.831+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 10 ==== osd_op_reply(67 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 11 ==== osd_op_reply(68 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 12 ==== osd_op_reply(69 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 14 ==== osd_op_reply(70 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.832+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dbf1800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(71 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 15 ==== osd_op_reply(72 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0400 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(73 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.833+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0800 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 13 ==== osd_op_reply(74 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74c00 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 16 ==== osd_op_reply(75 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474d1b0400 2024-05-09T06:35:39.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(76 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56474db90780 con 0x56474d1b0400 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74400 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 11 ==== osd_op_reply(77 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.834+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75c00 con 0x56474dd74400 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 12 ==== osd_op_reply(78 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1c00 con 0x56474dd75000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.835+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6c00 con 0x56474dd75000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7400 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 20 datascan.scan_links: injected ino detected 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7800 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6000 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-09T06:35:39.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.836+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1400 con 0x56474dbf1800 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90500 con 0x56474dbf1800 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1000 con 0x56474dd74400 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.837+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b0c00 con 0x56474dbf1c00 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 <== osd.2 v2:172.21.15.1:6808/1632480843 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x56474db90780 con 0x56474dbf1c00 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-09T06:35:39.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6400 con 0x56474dd74c00 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.838+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6800 con 0x56474d1b0800 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-09T06:35:39.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6c00 con 0x56474dd74c00 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7400 con 0x56474dd74c00 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-09T06:35:39.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.839+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7800 con 0x56474dd74c00 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1c00 con 0x56474dd74400 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.6 v2:172.21.15.6:6817/1315796905 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x56474db90000 con 0x56474dd74400 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d7c00 con 0x56474dd75000 2024-05-09T06:35:39.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.840+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.841+0000 7fed2e60c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.841+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6400 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v95'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 10 datascan.scan_links: processing 1 injected_inos 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6800 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 20 datascan.scan_links: first is now 2 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.842+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56474d1d6c00 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2ca08640 1 -- 172.21.15.1:0/1090472545 <== osd.1 v2:172.21.15.1:6816/1858675679 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v95'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56474db90500 con 0x56474dd75000 2024-05-09T06:35:39.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 10 datascan.scan_links: updating inotable 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474d1b1000 con 0x56474dd74c00 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.843+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75400 con 0x56474dd74c00 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.7 v2:172.21.15.6:6816/3240730649 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v95'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x56474db90000 con 0x56474dd74c00 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 10 datascan.scan_links: updating snaptable 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56474dd75800 con 0x56474d1b0800 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2ba06640 1 -- 172.21.15.1:0/1090472545 <== osd.4 v2:172.21.15.6:6800/3928343737 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x56474db90000 con 0x56474d1b0800 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56474dbf1800 msgr2=0x56474d1a7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56474dbf1800 0x56474d1a7b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56474dd75000 msgr2=0x56474d1a6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56474dd75000 0x56474d1a6580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56474dbf1c00 msgr2=0x56474d1a6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x56474dbf1c00 0x56474d1a6b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56474d1b1800 msgr2=0x56474d1a8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x56474d1b1800 0x56474d1a8680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 reap_dead start 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56474d1b0800 msgr2=0x56474d1a7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56474d1b0800 0x56474d1a7080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56474d1b0400 msgr2=0x56474d1a8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x56474d1b0400 0x56474d1a8100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56474dd74400 msgr2=0x56474d1a7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56474dd74400 0x56474d1a7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2c207640 1 -- 172.21.15.1:0/1090472545 reap_dead start 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 msgr2=0x56474ce92680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 0x56474ce92680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 msgr2=0x56474d1a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 0x56474d1a6000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56474dc45860 tx=0x56474d134e40 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 msgr2=0x56474ce93700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x56474dc21e90 tx=0x56474dc25800 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 shutdown_connections 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x56474dd74c00 0x56474ce92680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56474dd74800 0x56474d1a6000 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 --2- 172.21.15.1:0/1090472545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf0800 0x56474ce93700 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 >> 172.21.15.1:0/1090472545 conn(0x56474dbf0400 msgr2=0x56474dba1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 shutdown_connections 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.845+0000 7fed2e60c740 1 -- 172.21.15.1:0/1090472545 wait complete. 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 msgr2=0x56474ce90b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x56474dc459e0 tx=0x56474dbb6a20 comp rx=0 tx=0).stop 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 shutdown_connections 2024-05-09T06:35:39.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56474dbf0000 0x56474ce90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56474dbf5c00 0x56474ce90b00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 --2- 172.21.15.1:0/2328936614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56474dbf4400 0x56474ce91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 >> 172.21.15.1:0/2328936614 conn(0x56474ce1d000 msgr2=0x56474dba0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 shutdown_connections 2024-05-09T06:35:39.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:39.846+0000 7fed2e60c740 1 -- 172.21.15.1:0/2328936614 wait complete. 2024-05-09T06:35:39.855 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.152699, result: 2024-05-09T06:35:39.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:35:39.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.011+0000 7f88cacca640 1 Processor -- start 2024-05-09T06:35:39.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.011+0000 7f88cacca640 1 -- start start 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88cacca640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88c415c4c0 0x7f88c406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88cacca640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f88c40b4de0 0x7f88c415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88cacca640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c406cf40 con 0x7f88c40b5260 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88cacca640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f88c4156980 con 0x7f88c415c4c0 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88cacca640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f88c406d870 con 0x7f88c40b4de0 2024-05-09T06:35:39.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88c415c4c0 0x7f88c406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88c415c4c0 0x7f88c406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39818/0 (socket says 172.21.15.1:39818) 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 -- 172.21.15.1:0/1958602858 learned_addr learned my addr 172.21.15.1:0/1958602858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88bbfff640 1 --2- 172.21.15.1:0/1958602858 >> v2:172.21.15.6:3300/0 conn(0x7f88c40b4de0 0x7f88c415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 -- 172.21.15.1:0/1958602858 >> v2:172.21.15.6:3300/0 conn(0x7f88c40b4de0 msgr2=0x7f88c415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 --2- 172.21.15.1:0/1958602858 >> v2:172.21.15.6:3300/0 conn(0x7f88c40b4de0 0x7f88c415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 -- 172.21.15.1:0/1958602858 >> v1:172.21.15.6:6789/0 conn(0x7f88c40b5260 legacy=0x7f88c4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.012+0000 7f88c8a66640 1 -- 172.21.15.1:0/1958602858 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c406d220 con 0x7f88c415c4c0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.013+0000 7f88c8a66640 1 --2- 172.21.15.1:0/1958602858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c415c4c0 0x7f88c406dd00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f88c40cd190 tx=0x7f88c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b4a8588c61c34d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.013+0000 7f88bb7fe640 1 -- 172.21.15.1:0/1958602858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0037070 con 0x7f88c415c4c0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.013+0000 7f88bb7fe640 1 -- 172.21.15.1:0/1958602858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c0032440 con 0x7f88c415c4c0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.013+0000 7f88bb7fe640 1 -- 172.21.15.1:0/1958602858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0032810 con 0x7f88c415c4c0 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 -- 172.21.15.1:0/1958602858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c415c4c0 msgr2=0x7f88c406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 --2- 172.21.15.1:0/1958602858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c415c4c0 0x7f88c406dd00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f88c40cd190 tx=0x7f88c002a270 comp rx=0 tx=0).stop 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 -- 172.21.15.1:0/1958602858 shutdown_connections 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 --2- 172.21.15.1:0/1958602858 >> v2:172.21.15.6:3300/0 conn(0x7f88c40b4de0 0x7f88c415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 --2- 172.21.15.1:0/1958602858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c415c4c0 0x7f88c406dd00 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 -- 172.21.15.1:0/1958602858 >> 172.21.15.1:0/1958602858 conn(0x7f88c40cc4d0 msgr2=0x7f88c40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 -- 172.21.15.1:0/1958602858 shutdown_connections 2024-05-09T06:35:39.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.014+0000 7f88cacca640 1 -- 172.21.15.1:0/1958602858 wait complete. 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.015+0000 7f88cacca640 1 Processor -- start 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.015+0000 7f88cacca640 1 -- start start 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 0x7f88c41f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 0x7f88c40ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88c40b2dc0 con 0x7f88c40b4de0 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88c40b2c40 con 0x7f88c40b5260 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88cacca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88c415cde0 con 0x7f88c415c4c0 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39828/0 (socket says 172.21.15.1:39828) 2024-05-09T06:35:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 -- 172.21.15.1:0/653665990 learned_addr learned my addr 172.21.15.1:0/653665990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c9267640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 0x7f88c40ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88bbfff640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 0x7f88c41f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 -- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 msgr2=0x7f88c40ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 0x7f88c40ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 -- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 msgr2=0x7f88c41f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 0x7f88c41f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 -- 172.21.15.1:0/653665990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c40aa5c0 con 0x7f88c40b4de0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c9267640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 0x7f88c40ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88bbfff640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 0x7f88c41f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.016+0000 7f88c8a66640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f88c002a2b0 tx=0x7f88c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.017+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0041020 con 0x7f88c40b4de0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.017+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00306c0 con 0x7f88c40b4de0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.017+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c40aa8a0 con 0x7f88c40b4de0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.017+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0037040 con 0x7f88c40b4de0 2024-05-09T06:35:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.017+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c40aae60 con 0x7f88c40b4de0 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.018+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c40752b0 con 0x7f88c40b4de0 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.018+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88c0044020 con 0x7f88c40b4de0 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.019+0000 7f88b97fa640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 0x7f88b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.019+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f88c00ab950 con 0x7f88c40b4de0 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.019+0000 7f88bbfff640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 0x7f88b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.019+0000 7f88bbfff640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 0x7f88b006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f88c40af660 tx=0x7f88b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.023+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f88c004b030 con 0x7f88c40b4de0 2024-05-09T06:35:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.244+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f88c40b1550 con 0x7f88c40b4de0 2024-05-09T06:35:40.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.454+0000 7f88b97fa640 1 -- 172.21.15.1:0/653665990 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v70) ==== 162+0+0 (secure 0 0 0) 0x7f88c007a060 con 0x7f88c40b4de0 2024-05-09T06:35:40.435 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:35:40.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 msgr2=0x7f88b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 0x7f88b006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f88c40af660 tx=0x7f88b4008040 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 msgr2=0x7f88c415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f88c002a2b0 tx=0x7f88c00046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 shutdown_connections 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88b006be30 0x7f88b006e2f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88c415c4c0 0x7f88c40ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.456+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88c40b5260 0x7f88c41f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.457+0000 7f88cacca640 1 --2- 172.21.15.1:0/653665990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88c40b4de0 0x7f88c415dba0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.457+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 >> 172.21.15.1:0/653665990 conn(0x7f88c40cc4d0 msgr2=0x7f88c406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.457+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 shutdown_connections 2024-05-09T06:35:40.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.457+0000 7f88cacca640 1 -- 172.21.15.1:0/653665990 wait complete. 2024-05-09T06:35:40.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:40.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.574+0000 7fe99727d640 1 Processor -- start 2024-05-09T06:35:40.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.574+0000 7fe99727d640 1 -- start start 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99727d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9900c6860 0x7fe990099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99727d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe99015e4b0 0x7fe99015f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99727d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe990099930 con 0x7fe9900b2aa0 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99727d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe9900b1e20 con 0x7fe9900c6860 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99727d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe99015ee90 con 0x7fe99015e4b0 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe995019640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9900c6860 0x7fe990099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe995019640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe9900c6860 0x7fe990099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39832/0 (socket says 172.21.15.1:39832) 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe995019640 1 -- 172.21.15.1:0/238224090 learned_addr learned my addr 172.21.15.1:0/238224090 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.575+0000 7fe99581a640 1 --2- 172.21.15.1:0/238224090 >> v2:172.21.15.6:3300/0 conn(0x7fe99015e4b0 0x7fe99015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.576+0000 7fe995019640 1 -- 172.21.15.1:0/238224090 >> v2:172.21.15.6:3300/0 conn(0x7fe99015e4b0 msgr2=0x7fe99015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.576+0000 7fe995019640 1 --2- 172.21.15.1:0/238224090 >> v2:172.21.15.6:3300/0 conn(0x7fe99015e4b0 0x7fe99015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.576+0000 7fe995019640 1 -- 172.21.15.1:0/238224090 >> v1:172.21.15.6:6789/0 conn(0x7fe9900b2aa0 legacy=0x7fe9900c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:40.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.576+0000 7fe995019640 1 -- 172.21.15.1:0/238224090 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990162930 con 0x7fe9900c6860 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.576+0000 7fe995019640 1 --2- 172.21.15.1:0/238224090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900c6860 0x7fe990099b30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cf800 tx=0x7fe98802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c06b6e68afc84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe98ffff640 1 -- 172.21.15.1:0/238224090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988037070 con 0x7fe9900c6860 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe98ffff640 1 -- 172.21.15.1:0/238224090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988032440 con 0x7fe9900c6860 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe98ffff640 1 -- 172.21.15.1:0/238224090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988032810 con 0x7fe9900c6860 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 -- 172.21.15.1:0/238224090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900c6860 msgr2=0x7fe990099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 --2- 172.21.15.1:0/238224090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900c6860 0x7fe990099b30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cf800 tx=0x7fe98802a270 comp rx=0 tx=0).stop 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 -- 172.21.15.1:0/238224090 shutdown_connections 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 --2- 172.21.15.1:0/238224090 >> v2:172.21.15.6:3300/0 conn(0x7fe99015e4b0 0x7fe99015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 --2- 172.21.15.1:0/238224090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900c6860 0x7fe990099b30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 -- 172.21.15.1:0/238224090 >> 172.21.15.1:0/238224090 conn(0x7fe9900b1fb0 msgr2=0x7fe9900b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 -- 172.21.15.1:0/238224090 shutdown_connections 2024-05-09T06:35:40.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.577+0000 7fe99727d640 1 -- 172.21.15.1:0/238224090 wait complete. 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 Processor -- start 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 -- start start 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9900c6860 0x7fe9901f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe99015e4b0 0x7fe990158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe99015f860 con 0x7fe9900b2aa0 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe99015eb20 con 0x7fe9900c6860 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe99727d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe99015f9e0 con 0x7fe99015e4b0 2024-05-09T06:35:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.579+0000 7fe995019640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39844/0 (socket says 172.21.15.1:39844) 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 -- 172.21.15.1:0/3175313762 learned_addr learned my addr 172.21.15.1:0/3175313762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe994818640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9900c6860 0x7fe9901f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe99581a640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe99015e4b0 0x7fe990158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 -- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe99015e4b0 msgr2=0x7fe990158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe99015e4b0 0x7fe990158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 -- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9900c6860 msgr2=0x7fe9901f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9900c6860 0x7fe9901f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 -- 172.21.15.1:0/3175313762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9901f4c40 con 0x7fe9900b2aa0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.580+0000 7fe995019640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe98802a2b0 tx=0x7fe9880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.581+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988037040 con 0x7fe9900b2aa0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.581+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988042070 con 0x7fe9900b2aa0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.581+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988030880 con 0x7fe9900b2aa0 2024-05-09T06:35:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.581+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9901f4f20 con 0x7fe9900b2aa0 2024-05-09T06:35:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.581+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901f54e0 con 0x7fe9900b2aa0 2024-05-09T06:35:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.582+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99015c430 con 0x7fe9900b2aa0 2024-05-09T06:35:40.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.582+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe988030a20 con 0x7fe9900b2aa0 2024-05-09T06:35:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.583+0000 7fe98dffb640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 0x7fe97c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.583+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fe9880ab8f0 con 0x7fe9900b2aa0 2024-05-09T06:35:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.583+0000 7fe994818640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 0x7fe97c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.583+0000 7fe994818640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 0x7fe97c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cc560 tx=0x7fe984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.587+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe99015c430 con 0x7fe9900b2aa0 2024-05-09T06:35:40.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.808+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe99006dc90 con 0x7fe9900b2aa0 2024-05-09T06:35:40.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.808+0000 7fe98dffb640 1 -- 172.21.15.1:0/3175313762 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5162 (secure 0 0 0) 0x7fe99006dc90 con 0x7fe9900b2aa0 2024-05-09T06:35:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:40.789 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:40.457839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-09T06:35:40.790 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 71 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 msgr2=0x7fe97c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 0x7fe97c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cc560 tx=0x7fe984009040 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 msgr2=0x7fe9900b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe98802a2b0 tx=0x7fe9880046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 shutdown_connections 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe97c06be30 0x7fe97c06e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe99015e4b0 0x7fe990158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9900c6860 0x7fe9901f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 --2- 172.21.15.1:0/3175313762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9900b2aa0 0x7fe9900b42b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.810+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 >> 172.21.15.1:0/3175313762 conn(0x7fe9900b1fb0 msgr2=0x7fe990073170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.811+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 shutdown_connections 2024-05-09T06:35:40.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:40.811+0000 7fe99727d640 1 -- 172.21.15.1:0/3175313762 wait complete. 2024-05-09T06:35:41.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:35:40.457839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6836/779210140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 779210140}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 779210140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:35:41.205 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8499:up:replay 2024-05-09T06:35:42.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:42.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.343+0000 7f5af63b7640 1 Processor -- start 2024-05-09T06:35:42.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.343+0000 7f5af63b7640 1 -- start start 2024-05-09T06:35:42.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5af63b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5af00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5af63b7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af00ce0e0 con 0x7f5af006e6d0 2024-05-09T06:35:42.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5af63b7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af00ce3c0 con 0x7f5af00cdab0 2024-05-09T06:35:42.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5af63b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5af00ce6a0 con 0x7f5af00cf180 2024-05-09T06:35:42.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5aeffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5af006e6d0 0x7f5af006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56258/0 (socket says 172.21.15.1:56258) 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5aeffff640 1 -- 172.21.15.1:0/1180243790 learned_addr learned my addr 172.21.15.1:0/1180243790 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5aef7fe640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5af00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104620142 0 0) 0x7f5af00ce0e0 con 0x7f5af006e6d0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.344+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ad8003520 con 0x7f5af006e6d0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2513024188 0 0) 0x7f5ad8003520 con 0x7f5af006e6d0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 msgr2=0x7f5af00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5af00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 >> v1:172.21.15.6:6789/0 conn(0x7f5af00cdab0 legacy=0x7f5af00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af00ce980 con 0x7f5af006e6d0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f5ae0001670 con 0x7f5af006e6d0 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 >> v1:172.21.15.1:6789/0 conn(0x7f5af006e6d0 legacy=0x7f5af006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad80039e0 0x7f5ad8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5ad8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ad8004d40 0x7f5ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5af00ce6a0 con 0x7f5ad80039e0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad800b9f0 con 0x7f5ad8004d40 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad800bbf0 con 0x7f5af00cf180 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad80039e0 0x7f5ad8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aef7fe640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5ad8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5af4954640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ad8004d40 0x7f5ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 -- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 msgr2=0x7f5ad8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5af00cf180 0x7f5ad8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 -- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ad8004d40 msgr2=0x7f5ad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ad8004d40 0x7f5ad80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5aeffff640 1 -- 172.21.15.1:0/1180243790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad8003520 con 0x7f5ad80039e0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.345+0000 7f5af4954640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ad8004d40 0x7f5ad80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.346+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad80039e0 0x7f5ad8003dc0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c290 tx=0x7f5ad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e626f9887c46fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.346+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0007d50 con 0x7f5ad80039e0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.346+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae0005af0 con 0x7f5ad80039e0 2024-05-09T06:35:42.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.346+0000 7f5aeeffd640 1 -- 172.21.15.1:0/1180243790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0007070 con 0x7f5ad80039e0 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 -- 172.21.15.1:0/1180243790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad80039e0 msgr2=0x7f5ad8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1180243790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad80039e0 0x7f5ad8003dc0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c290 tx=0x7f5ad8005830 comp rx=0 tx=0).stop 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af4954640 1 -- 172.21.15.1:0/1180243790 reap_dead start 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 -- 172.21.15.1:0/1180243790 shutdown_connections 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 -- 172.21.15.1:0/1180243790 >> 172.21.15.1:0/1180243790 conn(0x7f5af00cc4b0 msgr2=0x7f5af00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 -- 172.21.15.1:0/1180243790 shutdown_connections 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.347+0000 7f5af63b7640 1 -- 172.21.15.1:0/1180243790 wait complete. 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.348+0000 7f5af63b7640 1 Processor -- start 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.348+0000 7f5af63b7640 1 -- start start 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 0x7f5af00c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 0x7f5af01405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01447c0 con 0x7f5af015df70 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0070ed0 con 0x7f5af01f43a0 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af63b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5af0144940 con 0x7f5ad80039e0 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39868/0 (socket says 172.21.15.1:39868) 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 -- 172.21.15.1:0/1932689317 learned_addr learned my addr 172.21.15.1:0/1932689317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5aef7fe640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 0x7f5af01405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 0x7f5af00c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 -- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 msgr2=0x7f5af00c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 0x7f5af00c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 -- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 msgr2=0x7f5af01405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 0x7f5af01405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5af4954640 1 -- 172.21.15.1:0/1932689317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af00c4a70 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5aef7fe640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 0x7f5af01405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.349+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 0x7f5af00c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5af4954640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5af00689b0 tx=0x7f5ae800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800ee50 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae8018070 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af00c4d50 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8003dc0 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.350+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01f4b80 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.351+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af01f4d80 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.351+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ae80050d0 con 0x7f5af015df70 2024-05-09T06:35:42.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.352+0000 7f5aecff9640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 0x7f5adc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.352+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f5ae808af00 con 0x7f5af015df70 2024-05-09T06:35:42.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.352+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 0x7f5adc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.353+0000 7f5aeffff640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 0x7f5adc06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0032000 tx=0x7f5ae002d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:42.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.356+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5af01f4d80 con 0x7f5af015df70 2024-05-09T06:35:42.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.581+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af00752b0 con 0x7f5af015df70 2024-05-09T06:35:42.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.581+0000 7f5aecff9640 1 -- 172.21.15.1:0/1932689317 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5162 (secure 0 0 0) 0x7f5ae8059600 con 0x7f5af015df70 2024-05-09T06:35:42.562 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:42.563 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:41.463442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-09T06:35:42.563 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 73 2024-05-09T06:35:42.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.583+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 msgr2=0x7f5adc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.583+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 0x7f5adc06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0032000 tx=0x7f5ae002d000 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.583+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 msgr2=0x7f5af00ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.583+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5af00689b0 tx=0x7f5ae800ec00 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 shutdown_connections 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5adc06be30 0x7f5adc06e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5af015df70 0x7f5af00ae8a0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af01f43a0 0x7f5af01405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 --2- 172.21.15.1:0/1932689317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5ad80039e0 0x7f5af00c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 >> 172.21.15.1:0/1932689317 conn(0x7f5af00cc4b0 msgr2=0x7f5af00ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 shutdown_connections 2024-05-09T06:35:42.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:42.584+0000 7f5af63b7640 1 -- 172.21.15.1:0/1932689317 wait complete. 2024-05-09T06:35:42.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:35:41.463442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.1:6836/779210140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 779210140}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 779210140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:35:42.980 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8499:up:rejoin 2024-05-09T06:35:43.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:44.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.116+0000 7f69593d5640 1 Processor -- start 2024-05-09T06:35:44.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.116+0000 7f69593d5640 1 -- start start 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69593d5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f695406e6d0 0x7f695406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69593d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 0x7f69540b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69593d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69540ce0e0 con 0x7f69540cf180 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69593d5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69540ce3c0 con 0x7f695406e6d0 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69593d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f69540ce6a0 con 0x7f69540b3300 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f6952ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f695406e6d0 0x7f695406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f6952ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f695406e6d0 0x7f695406eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39878/0 (socket says 172.21.15.1:39878) 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f6952ffd640 1 -- 172.21.15.1:0/862525564 learned_addr learned my addr 172.21.15.1:0/862525564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.117+0000 7f69527fc640 1 --1- 172.21.15.1:0/862525564 >> v1:172.21.15.1:6789/0 conn(0x7f69540cf180 0x7f69540cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56266/0 (socket says 172.21.15.1:56266) 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f69537fe640 1 --2- 172.21.15.1:0/862525564 >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 0x7f69540b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f6952ffd640 1 -- 172.21.15.1:0/862525564 >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 msgr2=0x7f69540b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f6952ffd640 1 --2- 172.21.15.1:0/862525564 >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 0x7f69540b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f6952ffd640 1 -- 172.21.15.1:0/862525564 >> v1:172.21.15.1:6789/0 conn(0x7f69540cf180 legacy=0x7f69540cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f6952ffd640 1 -- 172.21.15.1:0/862525564 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540ce980 con 0x7f695406e6d0 2024-05-09T06:35:44.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f69537fe640 1 --2- 172.21.15.1:0/862525564 >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 0x7f69540b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:44.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.118+0000 7f6952ffd640 1 --2- 172.21.15.1:0/862525564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f695406eab0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f69540b16f0 tx=0x7f694402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc485939f2964dae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f6951ffb640 1 -- 172.21.15.1:0/862525564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944036020 con 0x7f695406e6d0 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f6951ffb640 1 -- 172.21.15.1:0/862525564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6944032640 con 0x7f695406e6d0 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f6951ffb640 1 -- 172.21.15.1:0/862525564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944032a10 con 0x7f695406e6d0 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 -- 172.21.15.1:0/862525564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 msgr2=0x7f695406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 --2- 172.21.15.1:0/862525564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f695406eab0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f69540b16f0 tx=0x7f694402c940 comp rx=0 tx=0).stop 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 -- 172.21.15.1:0/862525564 shutdown_connections 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 --2- 172.21.15.1:0/862525564 >> v2:172.21.15.6:3300/0 conn(0x7f69540b3300 0x7f69540b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 --2- 172.21.15.1:0/862525564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f695406eab0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 -- 172.21.15.1:0/862525564 >> 172.21.15.1:0/862525564 conn(0x7f69540cc4b0 msgr2=0x7f69540cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 -- 172.21.15.1:0/862525564 shutdown_connections 2024-05-09T06:35:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.119+0000 7f69593d5640 1 -- 172.21.15.1:0/862525564 wait complete. 2024-05-09T06:35:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 Processor -- start 2024-05-09T06:35:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 -- start start 2024-05-09T06:35:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69540b3300 0x7f695415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69540cf180 0x7f695413e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6954070ed0 con 0x7f695406e6d0 2024-05-09T06:35:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f695406eff0 con 0x7f69540b3300 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.121+0000 7f69593d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69541447c0 con 0x7f69540cf180 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39892/0 (socket says 172.21.15.1:39892) 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 -- 172.21.15.1:0/2526433149 learned_addr learned my addr 172.21.15.1:0/2526433149 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f69527fc640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69540b3300 0x7f695415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f69537fe640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69540cf180 0x7f695413e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 -- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69540cf180 msgr2=0x7f695413e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69540cf180 0x7f695413e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 -- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69540b3300 msgr2=0x7f695415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69540b3300 0x7f695415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 -- 172.21.15.1:0/2526433149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695415e130 con 0x7f695406e6d0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f6952ffd640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f694402c980 tx=0x7f69440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944036040 con 0x7f695406e6d0 2024-05-09T06:35:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694403bcb0 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695415e410 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694403be60 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.122+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541efb90 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.123+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695415ea90 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.124+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f694404c060 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.124+0000 7f694b7fe640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 0x7f693406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.125+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f69440adf90 con 0x7f695406e6d0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.125+0000 7f69527fc640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 0x7f693406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.125+0000 7f69527fc640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 0x7f693406e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f695415d1b0 tx=0x7f693c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.129+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f695415ea90 con 0x7f695406e6d0 2024-05-09T06:35:44.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.350+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69540752b0 con 0x7f695406e6d0 2024-05-09T06:35:44.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.350+0000 7f694b7fe640 1 -- 172.21.15.1:0/2526433149 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7f6944040070 con 0x7f695406e6d0 2024-05-09T06:35:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:35:44.331 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:35:44.332 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:35:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 msgr2=0x7f693406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 0x7f693406e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f695415d1b0 tx=0x7f693c007e80 comp rx=0 tx=0).stop 2024-05-09T06:35:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 msgr2=0x7f6954142990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f694402c980 tx=0x7f69440046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 shutdown_connections 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be30 0x7f693406e2f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69540cf180 0x7f695413e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69540b3300 0x7f695415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 --2- 172.21.15.1:0/2526433149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f695406e6d0 0x7f6954142990 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 >> 172.21.15.1:0/2526433149 conn(0x7f69540cc4b0 msgr2=0x7f69540b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 shutdown_connections 2024-05-09T06:35:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.352+0000 7f69593d5640 1 -- 172.21.15.1:0/2526433149 wait complete. 2024-05-09T06:35:44.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:34:02.686762+0000', 'modified': '2024-05-09T06:35:43.462274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.1:6836/779210140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 779210140}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 779210140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8499, 'qdb_cluster': [8499]} 2024-05-09T06:35:44.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:35:44.747 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T06:35:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.873+0000 7fb60bee1640 1 Processor -- start 2024-05-09T06:35:44.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.874+0000 7fb60bee1640 1 -- start start 2024-05-09T06:35:44.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.874+0000 7fb60bee1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb6040cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60bee1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6040cdab0 0x7fb6040b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60bee1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6040ce0e0 con 0x7fb60406e6d0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60bee1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb6040ce3c0 con 0x7fb6040cdab0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60bee1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040ce6a0 con 0x7fb6040cf180 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb609c7d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb60406e6d0 0x7fb60406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56270/0 (socket says 172.21.15.1:56270) 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb609c7d640 1 -- 172.21.15.1:0/625965864 learned_addr learned my addr 172.21.15.1:0/625965864 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60a47e640 1 --2- 172.21.15.1:0/625965864 >> v2:172.21.15.6:3300/0 conn(0x7fb6040cdab0 0x7fb6040b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb60947c640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb6040cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563472765 0 0) 0x7fb6040ce0e0 con 0x7fb60406e6d0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5ec003520 con 0x7fb60406e6d0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4174962737 0 0) 0x7fb5ec003520 con 0x7fb60406e6d0 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 msgr2=0x7fb6040cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb6040cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 >> v2:172.21.15.6:3300/0 conn(0x7fb6040cdab0 msgr2=0x7fb6040b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 --2- 172.21.15.1:0/625965864 >> v2:172.21.15.6:3300/0 conn(0x7fb6040cdab0 0x7fb6040b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040ce980 con 0x7fb60406e6d0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb5fc001670 con 0x7fb60406e6d0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 >> v1:172.21.15.1:6789/0 conn(0x7fb60406e6d0 legacy=0x7fb60406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5ec0039b0 0x7fb5ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec004800 0x7fb5ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb5ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040ce3c0 con 0x7fb5ec0039b0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040ce6a0 con 0x7fb5ec004800 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec00b9c0 con 0x7fb6040cf180 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.875+0000 7fb609c7d640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5ec0039b0 0x7fb5ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb60947c640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec004800 0x7fb5ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb60a47e640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb5ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 -- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 msgr2=0x7fb5ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6040cf180 0x7fb5ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 -- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec004800 msgr2=0x7fb5ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec004800 0x7fb5ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 -- 172.21.15.1:0/625965864 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec003520 con 0x7fb5ec0039b0 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.876+0000 7fb609c7d640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5ec0039b0 0x7fb5ec003dd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c210 tx=0x7fb5ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a25d752cc28bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc007d50 con 0x7fb5ec0039b0 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc005af0 con 0x7fb5ec0039b0 2024-05-09T06:35:44.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb608c7b640 1 -- 172.21.15.1:0/625965864 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc007070 con 0x7fb5ec0039b0 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 -- 172.21.15.1:0/625965864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5ec0039b0 msgr2=0x7fb5ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 --2- 172.21.15.1:0/625965864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5ec0039b0 0x7fb5ec003dd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c210 tx=0x7fb5ec005830 comp rx=0 tx=0).stop 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60a47e640 1 -- 172.21.15.1:0/625965864 reap_dead start 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 -- 172.21.15.1:0/625965864 shutdown_connections 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 -- 172.21.15.1:0/625965864 >> 172.21.15.1:0/625965864 conn(0x7fb6040cc4b0 msgr2=0x7fb6040cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 -- 172.21.15.1:0/625965864 shutdown_connections 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.877+0000 7fb60bee1640 1 -- 172.21.15.1:0/625965864 wait complete. 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.878+0000 7fb60bee1640 1 Processor -- start 2024-05-09T06:35:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 -- start start 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec0039b0 0x7fb60415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 0x7fb6041f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6041447c0 con 0x7fb60415e150 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb604070ed0 con 0x7fb5ec0039b0 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60bee1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb604144940 con 0x7fb6041f1d40 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60947c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60947c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39910/0 (socket says 172.21.15.1:39910) 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.879+0000 7fb60947c640 1 -- 172.21.15.1:0/1891929247 learned_addr learned my addr 172.21.15.1:0/1891929247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60a47e640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 0x7fb6041f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb609c7d640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec0039b0 0x7fb60415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 -- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 msgr2=0x7fb6041f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 0x7fb6041f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 -- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec0039b0 msgr2=0x7fb60415a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec0039b0 0x7fb60415a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 -- 172.21.15.1:0/1891929247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60415d0b0 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60a47e640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 0x7fb6041f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60947c640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8011910 tx=0x7fb5f8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb6027fc640 1 -- 172.21.15.1:0/1891929247 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800fa90 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb6027fc640 1 -- 172.21.15.1:0/1891929247 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f800fc30 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60415d390 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb6027fc640 1 -- 172.21.15.1:0/1891929247 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8014440 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.880+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60415d950 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.882+0000 7fb6027fc640 1 -- 172.21.15.1:0/1891929247 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5f8012040 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.882+0000 7fb6027fc640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.882+0000 7fb6027fc640 1 -- 172.21.15.1:0/1891929247 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb5f808b6f0 con 0x7fb60415e150 2024-05-09T06:35:44.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.882+0000 7fb609c7d640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.883+0000 7fb609c7d640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc033000 tx=0x7fb5fc009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.883+0000 7fb60ebf3740 1 Processor -- start 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.883+0000 7fb60ebf3740 1 -- start start 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2ac20 0x5572d2a03280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5572d29664d0 con 0x5572d2a2b000 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5572d290e3e0 con 0x5572d2a2ac20 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5572d2962490 con 0x5572d2a2d9c0 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39916/0 (socket says 172.21.15.1:39916) 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 -- 172.21.15.1:0/3973885337 learned_addr learned my addr 172.21.15.1:0/3973885337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 -- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 msgr2=0x5572d2a01220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 -- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2ac20 msgr2=0x5572d2a03280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2ac20 0x5572d2a03280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.884+0000 7fb60947c640 1 -- 172.21.15.1:0/3973885337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d267a2d0 con 0x5572d2a2b000 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60947c640 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8055b50 tx=0x7fb5f8059eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4971d5767b5bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb5eb7fe640 1 -- 172.21.15.1:0/3973885337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80622d0 con 0x5572d2a2b000 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb5eb7fe640 1 -- 172.21.15.1:0/3973885337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80579f0 con 0x5572d2a2b000 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb5eb7fe640 1 -- 172.21.15.1:0/3973885337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8012330 con 0x5572d2a2b000 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 -- 172.21.15.1:0/3973885337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 msgr2=0x5572d2a2d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8055b50 tx=0x7fb5f8059eb0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 -- 172.21.15.1:0/3973885337 shutdown_connections 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a2d480 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/3973885337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2ac20 0x5572d2a03280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 -- 172.21.15.1:0/3973885337 >> 172.21.15.1:0/3973885337 conn(0x5572d29ffc80 msgr2=0x5572d2a48480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 -- 172.21.15.1:0/3973885337 shutdown_connections 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.885+0000 7fb60ebf3740 1 -- 172.21.15.1:0/3973885337 wait complete. 2024-05-09T06:35:44.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.886+0000 7fb60ebf3740 1 -- start start 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.886+0000 7fb60ebf3740 1 Processor -- start 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a47670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a47bb0 0x5572d2a29360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5572d29218e0 con 0x5572d2a2b000 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5572d2a46180 con 0x5572d2a47bb0 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5572d2a2f450 con 0x5572d2a2d9c0 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb60ebf3740 10 client.0 _ll_register_callbacks cb 0x5572d28ee9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39932/0 (socket says 172.21.15.1:39932) 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 -- 172.21.15.1:0/2269560428 learned_addr learned my addr 172.21.15.1:0/2269560428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 -- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 msgr2=0x5572d2a47670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a47670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 -- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a47bb0 msgr2=0x5572d2a29360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a47bb0 0x5572d2a29360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.887+0000 7fb609c7d640 1 -- 172.21.15.1:0/2269560428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d27446b0 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb609c7d640 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc02ae50 tx=0x7fb5fc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ce85339bdd8c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc04d070 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb5fc03f430 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f80579f0 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc058050 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 10 client.8574 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5572d2a30a60 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80579f0 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb5fc05b020 con 0x5572d2a2b000 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 10 client.8574 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 10 client.8574 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:35:44.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 20 client.8574 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d2a66620 0x5572d2a6a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 4 client.8574 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572d2973c00 con 0x5572d2a66620 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.888+0000 7fb60947c640 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d2a66620 0x5572d2a6a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60947c640 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d2a66620 0x5572d2a6a990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5aae424d627572c server_cookie=abbed2a24af4862f in_seq=0 out_seq=0 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb5e8ff9640 10 client.8574 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/2269560428 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572d2973c00 con 0x5572d2a66620 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb5e8ff9640 10 client.8574 handle_command_reply: tid=0 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60ebf3740 1 client.8574 shutdown 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 msgr2=0x5572d2a314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc02ae50 tx=0x7fb5fc001a40 comp rx=0 tx=0).stop 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 shutdown_connections 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.889+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d2a66620 0x5572d2a6a990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a47bb0 0x5572d2a29360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a47670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb5e8ff9640 0 client.8574 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/2269560428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a2b000 0x5572d2a314a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 >> 172.21.15.1:0/2269560428 conn(0x5572d2a2ac20 msgr2=0x5572d29fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 shutdown_connections 2024-05-09T06:35:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 1 -- 172.21.15.1:0/2269560428 wait complete. 2024-05-09T06:35:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.890+0000 7fb60ebf3740 20 client.8574 trim_cache size 0 max 16384 2024-05-09T06:35:44.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.908+0000 7fb60ebf3740 1 Processor -- start 2024-05-09T06:35:44.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.908+0000 7fb60ebf3740 1 -- start start 2024-05-09T06:35:44.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a28e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a5eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6041fc2f0 con 0x5572d2a47bb0 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040b1390 con 0x5572d2a2b000 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040b19e0 con 0x5572d2a2d9c0 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39934/0 (socket says 172.21.15.1:39934) 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 -- 172.21.15.1:0/896824016 learned_addr learned my addr 172.21.15.1:0/896824016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 -- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 msgr2=0x5572d2a5eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a5eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 -- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 msgr2=0x5572d2a28e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a28e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 -- 172.21.15.1:0/896824016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d2a2f150 con 0x5572d2a47bb0 2024-05-09T06:35:44.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb60a47e640 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4004af0 tx=0x7fb5f4013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf4a21663a90e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.909+0000 7fb5eaffd640 1 -- 172.21.15.1:0/896824016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f401c020 con 0x5572d2a47bb0 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb5eaffd640 1 -- 172.21.15.1:0/896824016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4003da0 con 0x5572d2a47bb0 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb5eaffd640 1 -- 172.21.15.1:0/896824016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004270 con 0x5572d2a47bb0 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 -- 172.21.15.1:0/896824016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 msgr2=0x5572d2a5f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4004af0 tx=0x7fb5f4013740 comp rx=0 tx=0).stop 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 -- 172.21.15.1:0/896824016 shutdown_connections 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a5f050 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a5eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/896824016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a28e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 -- 172.21.15.1:0/896824016 >> 172.21.15.1:0/896824016 conn(0x5572d2a2ac20 msgr2=0x5572d2a3ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 -- 172.21.15.1:0/896824016 shutdown_connections 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.910+0000 7fb60ebf3740 1 -- 172.21.15.1:0/896824016 wait complete. 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.911+0000 7fb60ebf3740 1 -- start start 2024-05-09T06:35:44.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.911+0000 7fb60ebf3740 1 Processor -- start 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5572d2a664a0 con 0x5572d2a47bb0 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5572d2a66050 con 0x5572d2a2b000 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5572d2a66ab0 con 0x5572d2a2d9c0 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 10 client.0 _ll_register_callbacks cb 0x7fb5fc02c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39936/0 (socket says 172.21.15.1:39936) 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 -- 172.21.15.1:0/1635256697 learned_addr learned my addr 172.21.15.1:0/1635256697 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 -- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 msgr2=0x5572d2a01630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 -- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 msgr2=0x5572d2a010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 -- 172.21.15.1:0/1635256697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d29fdb60 con 0x5572d2a47bb0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60a47e640 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b010 tx=0x7fb5f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515383673e634c70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb5e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4009cb0 con 0x5572d2a47bb0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fb5f402d480 con 0x5572d2a47bb0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.912+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f4003da0 con 0x5572d2a47bb0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb5f4035050 con 0x5572d2a47bb0 2024-05-09T06:35:44.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 10 client.8580 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5572d2a02e00 con 0x5572d2a47bb0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4003da0 con 0x5572d2a47bb0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb5f4038020 con 0x5572d2a47bb0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 10 client.8580 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 10 client.8580 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 20 client.8580 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d29fcae0 0x5572d29fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 4 client.8580 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "status"}) -- 0x5572d296df50 con 0x5572d29fcae0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb609c7d640 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d29fcae0 0x5572d29fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 10 client.8580 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb609c7d640 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d29fcae0 0x5572d29fcec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4162d6527a7383b server_cookie=cb2c3706daf3ddeb in_seq=0 out_seq=0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 1 -- 172.21.15.1:0/1635256697 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5572d296df50 con 0x5572d29fcae0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb5e8ff9640 10 client.8580 handle_command_reply: tid=0 2024-05-09T06:35:44.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.913+0000 7fb60ebf3740 1 client.8580 shutdown 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 msgr2=0x5572d2a46140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b010 tx=0x7fb5f4009750 comp rx=0 tx=0).stop 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 shutdown_connections 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5572d29fcae0 0x5572d29fcec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572d2a47bb0 0x5572d2a46140 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5572d2a2d9c0 0x5572d2a01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb5e8ff9640 0 client.8580 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 --2- 172.21.15.1:0/1635256697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5572d2a2b000 0x5572d2a010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 >> 172.21.15.1:0/1635256697 conn(0x5572d2a2ac20 msgr2=0x5572d2a3ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 shutdown_connections 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 1 -- 172.21.15.1:0/1635256697 wait complete. 2024-05-09T06:35:44.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60ebf3740 20 client.8580 trim_cache size 0 max 16384 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 8499, 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4524910440000003, 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 74, 2024-05-09T06:35:44.901 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 95, 2024-05-09T06:35:44.902 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 95, 2024-05-09T06:35:44.902 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 5.4229341829999997 2024-05-09T06:35:44.902 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:35:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 msgr2=0x7fb5f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc033000 tx=0x7fb5fc009030 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 msgr2=0x7fb60415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8011910 tx=0x7fb5f8011dd0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 shutdown_connections 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6041f1d40 0x7fb6041f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb60415e150 0x7fb60415e5d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 --2- 172.21.15.1:0/1891929247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5ec0039b0 0x7fb60415a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.914+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 >> 172.21.15.1:0/1891929247 conn(0x7fb6040cc4b0 msgr2=0x7fb6040b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.915+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 shutdown_connections 2024-05-09T06:35:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:44.915+0000 7fb60bee1640 1 -- 172.21.15.1:0/1891929247 wait complete. 2024-05-09T06:35:44.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:35:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.029+0000 7f14c991d640 1 Processor -- start 2024-05-09T06:35:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.029+0000 7f14c991d640 1 -- start start 2024-05-09T06:35:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c991d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f14c41454b0 0x7f14c40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c991d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14c4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c991d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c406eb40 con 0x7f14c40b2a60 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c991d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f14c406ee20 con 0x7f14c41454b0 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c991d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f14c406d8e0 con 0x7f14c40b4f70 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c27fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f14c40b2a60 0x7f14c40b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56278/0 (socket says 172.21.15.1:56278) 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c27fc640 1 -- 172.21.15.1:0/3067557328 learned_addr learned my addr 172.21.15.1:0/3067557328 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c37fe640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14c4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/3067557328 >> v2:172.21.15.6:3300/0 conn(0x7f14c41454b0 0x7f14c40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498426195 0 0) 0x7f14c406eb40 con 0x7f14c40b2a60 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ac003520 con 0x7f14c40b2a60 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2703425142 0 0) 0x7f14ac003520 con 0x7f14c40b2a60 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 msgr2=0x7f14c4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14c4070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 >> v2:172.21.15.6:3300/0 conn(0x7f14c41454b0 msgr2=0x7f14c40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 --2- 172.21.15.1:0/3067557328 >> v2:172.21.15.6:3300/0 conn(0x7f14c41454b0 0x7f14c40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c406dbc0 con 0x7f14c40b2a60 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f14b8001670 con 0x7f14c40b2a60 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 >> v1:172.21.15.1:6789/0 conn(0x7f14c40b2a60 legacy=0x7f14c40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.030+0000 7f14c1ffb640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14ac003f30 0x7f14ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c1ffb640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac004d40 0x7f14ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406ee20 con 0x7f14ac003f30 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406d8e0 con 0x7f14ac004d40 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac00b9c0 con 0x7f14c40b4f70 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14ac003f30 0x7f14ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c37fe640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac004d40 0x7f14ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 -- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 msgr2=0x7f14ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 -- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac004d40 msgr2=0x7f14ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac004d40 0x7f14ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c27fc640 1 -- 172.21.15.1:0/3067557328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac003520 con 0x7f14ac003f30 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c37fe640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac004d40 0x7f14ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.031+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14c40b4f70 0x7f14ac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c27fc640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14ac003f30 0x7f14ac004800 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f14b800c290 tx=0x7f14ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c144a03ffbdcf9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8007d50 con 0x7f14ac003f30 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b802e070 con 0x7f14ac003f30 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c1ffb640 1 -- 172.21.15.1:0/3067557328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8005d70 con 0x7f14ac003f30 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 -- 172.21.15.1:0/3067557328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14ac003f30 msgr2=0x7f14ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 --2- 172.21.15.1:0/3067557328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14ac003f30 0x7f14ac004800 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f14b800c290 tx=0x7f14ac005830 comp rx=0 tx=0).stop 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c37fe640 1 -- 172.21.15.1:0/3067557328 reap_dead start 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 -- 172.21.15.1:0/3067557328 shutdown_connections 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 -- 172.21.15.1:0/3067557328 >> 172.21.15.1:0/3067557328 conn(0x7f14c40cc4b0 msgr2=0x7f14c40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 -- 172.21.15.1:0/3067557328 shutdown_connections 2024-05-09T06:35:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.032+0000 7f14c991d640 1 -- 172.21.15.1:0/3067557328 wait complete. 2024-05-09T06:35:45.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 Processor -- start 2024-05-09T06:35:45.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 -- start start 2024-05-09T06:35:45.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14ac003f30 0x7f14c41f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14c40c4b90 0x7f14c40c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406e260 con 0x7f14c4142120 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406e0e0 con 0x7f14c40c4b90 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.034+0000 7f14c991d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f14c4070c10 con 0x7f14ac003f30 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39958/0 (socket says 172.21.15.1:39958) 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 -- 172.21.15.1:0/1902537296 learned_addr learned my addr 172.21.15.1:0/1902537296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14ac003f30 0x7f14c41f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c37fe640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14c40c4b90 0x7f14c40c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 -- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14ac003f30 msgr2=0x7f14c41f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14ac003f30 0x7f14c41f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 -- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14c40c4b90 msgr2=0x7f14c40c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14c40c4b90 0x7f14c40c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 -- 172.21.15.1:0/1902537296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c4139b80 con 0x7f14c4142120 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c27fc640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f14b8001970 tx=0x7f14c415c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8042020 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b8007270 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c4138dc0 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8001c80 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.035+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c4139380 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.036+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c40752b0 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.037+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14b8045020 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.037+0000 7f14abfff640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 0x7f14b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.037+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f14b80ab2d0 con 0x7f14c4142120 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.038+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 0x7f14b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.038+0000 7f14c2ffd640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 0x7f14b006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009b40 tx=0x7f14b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.042+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14b8004e20 con 0x7f14c4142120 2024-05-09T06:35:45.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.265+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f14c4140e90 con 0x7f14c4142120 2024-05-09T06:35:45.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.265+0000 7f14abfff640 1 -- 172.21.15.1:0/1902537296 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f14b8079530 con 0x7f14c4142120 2024-05-09T06:35:45.246 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:35:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 msgr2=0x7f14b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 0x7f14b006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009b40 tx=0x7f14b4008040 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 msgr2=0x7f14c41425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f14b8001970 tx=0x7f14c415c370 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 shutdown_connections 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f14b006be30 0x7f14b006e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14c40c4b90 0x7f14c40c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14c4142120 0x7f14c41425a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 --2- 172.21.15.1:0/1902537296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14ac003f30 0x7f14c41f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.267+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 >> 172.21.15.1:0/1902537296 conn(0x7f14c40cc4b0 msgr2=0x7f14c40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.268+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 shutdown_connections 2024-05-09T06:35:45.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45.268+0000 7f14c991d640 1 -- 172.21.15.1:0/1902537296 wait complete. 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:35:45.260 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:35:45.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.261 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:45.277 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:35:45.278 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:35:45.279 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:45.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.280 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:35:45.342 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:35:45.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.343 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:35:45.400 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:35:45.401 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:35:45.401 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:35:45.401 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:35:45.401 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:35:45.401 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:45.401 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:35:45.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:35:45.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:35:45.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:35:45.402 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:35:45.443 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:35:45.462+0000 7efc715d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:35:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:45.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.530 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:35:45.546 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:35:45.546 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:35:45.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:35:45.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:35:45.613 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:35:45.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:45.613 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:45.670 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:35:45.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:45.670 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:45.725 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:35:45.792 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:35:45.850 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:45.851 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:45.852 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:35:45.853 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:35:45.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:45.853 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:35:45.906 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:35:45.907 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:35:45.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:35:45.975 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:35:46.032 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:35:46.033 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:35:46.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.035 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:35:46.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:35:46.052+0000 7f7de6464480 -1 init, newargv = 0x5594782ba320 newargc=15 2024-05-09T06:35:46.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[71881]: starting ceph client 2024-05-09T06:35:46.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[71881]: starting fuse 2024-05-09T06:35:46.051 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:35:46.051 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:35:46.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:35:46.051 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:35:46.052 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:35:46.053 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:35:46.173 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.71881.asok 2024-05-09T06:35:46.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:46.186 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.71881.asok 2024-05-09T06:35:46.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.71881.asok status 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:35:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "71881", 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 8592, 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 8592 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1590354446 2024-05-09T06:35:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1590354446 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.8592 192.168.0.1:0/1590354446", 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1590354446", 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 74, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 95, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:35:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:35:46.326 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:35:46.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.332 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:35:46.332 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:46.348 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:35:46.348 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:46.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:46.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-09T06:35:46.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.582+0000 7f2fefccd640 1 Processor -- start 2024-05-09T06:35:46.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.582+0000 7f2fefccd640 1 -- start start 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fefccd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2fe80b3820 0x7f2fe80ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fefccd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe80b3cc0 con 0x7f2fe8145880 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fefccd640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe80b3fa0 con 0x7f2fe80ced00 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fefccd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe80b4280 con 0x7f2fe80b3820 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2fe80b3820 0x7f2fe80ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2fe80b3820 0x7f2fe80ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39974/0 (socket says 172.21.15.1:39974) 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2feda69640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2fe8145880 0x7f2fe815cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56288/0 (socket says 172.21.15.1:56288) 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 -- 172.21.15.1:0/2324143154 learned_addr learned my addr 172.21.15.1:0/2324143154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 -- 172.21.15.1:0/2324143154 >> v1:172.21.15.6:6789/0 conn(0x7f2fe80ced00 legacy=0x7f2fe8070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 -- 172.21.15.1:0/2324143154 >> v1:172.21.15.1:6789/0 conn(0x7f2fe8145880 legacy=0x7f2fe815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2fed268640 1 -- 172.21.15.1:0/2324143154 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe80b4560 con 0x7f2fe80b3820 2024-05-09T06:35:46.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.583+0000 7f2feca67640 1 -- 172.21.15.1:0/2324143154 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548679034 0 0) 0x7f2fe80b3cc0 con 0x7f2fe8145880 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2fed268640 1 --2- 172.21.15.1:0/2324143154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe80ce7c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e1d9f2b3d2da39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2feca67640 1 -- 172.21.15.1:0/2324143154 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8037070 con 0x7f2fe80b3820 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2feca67640 1 -- 172.21.15.1:0/2324143154 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd8032440 con 0x7f2fe80b3820 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2feca67640 1 -- 172.21.15.1:0/2324143154 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8032810 con 0x7f2fe80b3820 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2fefccd640 1 -- 172.21.15.1:0/2324143154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 msgr2=0x7f2fe80ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.584+0000 7f2fefccd640 1 --2- 172.21.15.1:0/2324143154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe80ce7c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd802a2a0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.585+0000 7f2fefccd640 1 -- 172.21.15.1:0/2324143154 shutdown_connections 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.585+0000 7f2fefccd640 1 --2- 172.21.15.1:0/2324143154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe80ce7c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.585+0000 7f2fefccd640 1 -- 172.21.15.1:0/2324143154 >> 172.21.15.1:0/2324143154 conn(0x7f2fe806b3c0 msgr2=0x7f2fe806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.585+0000 7f2fefccd640 1 -- 172.21.15.1:0/2324143154 shutdown_connections 2024-05-09T06:35:46.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.585+0000 7f2fefccd640 1 -- 172.21.15.1:0/2324143154 wait complete. 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.586+0000 7f2fefccd640 1 Processor -- start 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.586+0000 7f2fefccd640 1 -- start start 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 0x7f2fe80aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 0x7f2fe813ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8071440 con 0x7f2fe80b3820 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe80cfee0 con 0x7f2fe8145880 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe80715c0 con 0x7f2fe80ced00 2024-05-09T06:35:46.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39982/0 (socket says 172.21.15.1:39982) 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 -- 172.21.15.1:0/1216166120 learned_addr learned my addr 172.21.15.1:0/1216166120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fed268640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 0x7f2fe80aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fee26a640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 0x7f2fe813ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 -- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 msgr2=0x7f2fe80aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 0x7f2fe80aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 -- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 msgr2=0x7f2fe813ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 0x7f2fe813ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 -- 172.21.15.1:0/1216166120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe80ab270 con 0x7f2fe80b3820 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fed268640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 0x7f2fe80aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fee26a640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 0x7f2fe813ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2feda69640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c430 tx=0x7f2fe400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fde7fc640 1 -- 172.21.15.1:0/1216166120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40074e0 con 0x7f2fe80b3820 2024-05-09T06:35:46.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fde7fc640 1 -- 172.21.15.1:0/1216166120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe40047e0 con 0x7f2fe80b3820 2024-05-09T06:35:46.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe8143010 con 0x7f2fe80b3820 2024-05-09T06:35:46.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fde7fc640 1 -- 172.21.15.1:0/1216166120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4004bd0 con 0x7f2fe80b3820 2024-05-09T06:35:46.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.587+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81435d0 con 0x7f2fe80b3820 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.589+0000 7f2fde7fc640 1 -- 172.21.15.1:0/1216166120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fe4011070 con 0x7f2fe80b3820 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.590+0000 7f2fde7fc640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.590+0000 7f2fed268640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.590+0000 7f2fde7fc640 1 -- 172.21.15.1:0/1216166120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f2fe408adb0 con 0x7f2fe80b3820 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.590+0000 7f2fed268640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8034000 tx=0x7f2fd802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.591+0000 7f2ff29d9740 1 Processor -- start 2024-05-09T06:35:46.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.591+0000 7f2ff29d9740 1 -- start start 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bbd9b0 0x556707bbfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707bbdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556707b21990 con 0x556707be8470 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556707ac95b0 con 0x556707bbd9b0 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556707b1d950 con 0x556707bfa080 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39990/0 (socket says 172.21.15.1:39990) 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 -- 172.21.15.1:0/3739773978 learned_addr learned my addr 172.21.15.1:0/3739773978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fed268640 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707bbdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 -- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 msgr2=0x556707bbdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707bbdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 -- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bbd9b0 msgr2=0x556707bbfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bbd9b0 0x556707bbfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 -- 172.21.15.1:0/3739773978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567078fd7e0 con 0x556707be8470 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fee26a640 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0004580 tx=0x7f2fe000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fcee619f1a7c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fcb7fe640 1 -- 172.21.15.1:0/3739773978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000ddd0 con 0x556707be8470 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fcb7fe640 1 -- 172.21.15.1:0/3739773978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0006b50 con 0x556707be8470 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.592+0000 7f2fcb7fe640 1 -- 172.21.15.1:0/3739773978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0007030 con 0x556707be8470 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3739773978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 msgr2=0x556707be88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0004580 tx=0x7f2fe000d520 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3739773978 shutdown_connections 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707be8470 0x556707be88f0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707bbdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3739773978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bbd9b0 0x556707bbfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3739773978 >> 172.21.15.1:0/3739773978 conn(0x556707b850a0 msgr2=0x556707c03470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3739773978 shutdown_connections 2024-05-09T06:35:46.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.593+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3739773978 wait complete. 2024-05-09T06:35:46.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.594+0000 7f2ff29d9740 1 -- start start 2024-05-09T06:35:46.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.594+0000 7f2ff29d9740 1 Processor -- start 2024-05-09T06:35:46.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707bfc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707be4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556707adcb40 con 0x556707bbd9b0 2024-05-09T06:35:46.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556707be5fc0 con 0x556707bfa080 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556707bea670 con 0x556707be8470 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2ff29d9740 10 client.0 _ll_register_callbacks cb 0x556707ad10f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39992/0 (socket says 172.21.15.1:39992) 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 -- 172.21.15.1:0/3303793067 learned_addr learned my addr 172.21.15.1:0/3303793067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2fed268640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707bfc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2fee26a640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707be4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 -- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 msgr2=0x556707bfc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707bfc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 -- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 msgr2=0x556707be4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707be4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 -- 172.21.15.1:0/3303793067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670795df70 con 0x556707bbd9b0 2024-05-09T06:35:46.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2feda69640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4020030 tx=0x7f2fe4058e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f1a109934b339 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.595+0000 7f2fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401ad60 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f2fe4058200 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe0006b50 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f2fe4058400 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 10 client.8607 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5567077f5170 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe4075080 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f2fe405f040 con 0x556707bbd9b0 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 10 client.8607 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 10 client.8607 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.596+0000 7f2ff29d9740 20 client.8607 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707be9f70 0x556707c139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2ff29d9740 4 client.8607 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:35:46.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556707b2f0c0 con 0x556707be9f70 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2fed268640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707be9f70 0x556707c139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2fc8ff9640 10 client.8607 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.597+0000 7f2fed268640 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707be9f70 0x556707c139c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c901229baf28628 server_cookie=c702138d169e2a61 in_seq=0 out_seq=0 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/3303793067 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556707b2f0c0 con 0x556707be9f70 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2fc8ff9640 10 client.8607 handle_command_reply: tid=0 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 client.8607 shutdown 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 msgr2=0x556707c17590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4020030 tx=0x7f2fe4058e70 comp rx=0 tx=0).stop 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 shutdown_connections 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707be9f70 0x556707c139c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707be4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707bfc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3303793067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c17590 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2fc8ff9640 0 client.8607 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 >> 172.21.15.1:0/3303793067 conn(0x556707b850a0 msgr2=0x556707be5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 shutdown_connections 2024-05-09T06:35:46.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3303793067 wait complete. 2024-05-09T06:35:46.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.598+0000 7f2ff29d9740 20 client.8607 trim_cache size 0 max 16384 2024-05-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.616+0000 7f2ff29d9740 1 Processor -- start 2024-05-09T06:35:46.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.616+0000 7f2ff29d9740 1 -- start start 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707be8470 0x556707c01ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707c02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81febe0 con 0x556707bbd9b0 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81fdbf0 con 0x556707be8470 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe80ac750 con 0x556707bfa080 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40002/0 (socket says 172.21.15.1:40002) 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 -- 172.21.15.1:0/3268839843 learned_addr learned my addr 172.21.15.1:0/3268839843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2fee26a640 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707c02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 -- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 msgr2=0x556707c02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707c02430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 -- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707be8470 msgr2=0x556707c01ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2fed268640 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707be8470 0x556707c01ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707be8470 0x556707c01ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 -- 172.21.15.1:0/3268839843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556707be5680 con 0x556707bbd9b0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2feda69640 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400cdf0 tx=0x7f2fe4058cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdc5f793290defb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2fcaffd640 1 -- 172.21.15.1:0/3268839843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401ad60 con 0x556707bbd9b0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2fcaffd640 1 -- 172.21.15.1:0/3268839843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0006b50 con 0x556707bbd9b0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.617+0000 7f2fcaffd640 1 -- 172.21.15.1:0/3268839843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe405f450 con 0x556707bbd9b0 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3268839843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 msgr2=0x556707c13bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400cdf0 tx=0x7f2fe4058cb0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3268839843 shutdown_connections 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707bfa080 0x556707c02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707be8470 0x556707c01ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/3268839843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707c13bb0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3268839843 >> 172.21.15.1:0/3268839843 conn(0x556707b850a0 msgr2=0x556707c12f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3268839843 shutdown_connections 2024-05-09T06:35:46.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.618+0000 7f2ff29d9740 1 -- 172.21.15.1:0/3268839843 wait complete. 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.619+0000 7f2ff29d9740 1 -- start start 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.619+0000 7f2ff29d9740 1 Processor -- start 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707c65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707bbc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81fd5f0 con 0x556707bbd9b0 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556707b2f0c0 con 0x556707bfa080 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556707bfa9a0 con 0x556707be8470 2024-05-09T06:35:46.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 10 client.0 _ll_register_callbacks cb 0x7f2fe4055050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40012/0 (socket says 172.21.15.1:40012) 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 -- 172.21.15.1:0/992820972 learned_addr learned my addr 172.21.15.1:0/992820972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fed268640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707c65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fee26a640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707bbc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 -- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 msgr2=0x556707c65cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707c65cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 -- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 msgr2=0x556707bbc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707bbc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 -- 172.21.15.1:0/992820972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556707c661f0 con 0x556707bbd9b0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fee26a640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707bbc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2feda69640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4058ab0 tx=0x7f2fe4001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70550827fbfcc54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:46.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4058710 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f2fe4058200 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe0006b50 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f2fe405f690 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 10 client.8613 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.620+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x556707c661f0 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0006b50 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f2fe405f040 con 0x556707bbd9b0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 10 client.8613 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 10 client.8613 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 20 client.8613 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707c657c0 0x556707bbc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 4 client.8613 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x556707b29410 con 0x556707c657c0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2fed268640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707c657c0 0x556707bbc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2fc8ff9640 10 client.8613 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.621+0000 7f2fed268640 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707c657c0 0x556707bbc6b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66594763c5583e7d server_cookie=e9a5cf1d62edb8d4 in_seq=0 out_seq=0 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2fc8ff9640 1 -- 172.21.15.1:0/992820972 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556707b29410 con 0x556707c657c0 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2fc8ff9640 10 client.8613 handle_command_reply: tid=0 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 client.8613 shutdown 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 msgr2=0x556707bfa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4058ab0 tx=0x7f2fe4001060 comp rx=0 tx=0).stop 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 shutdown_connections 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x556707c657c0 0x556707bbc6b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556707bfa080 0x556707bbc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556707be8470 0x556707c65cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 --2- 172.21.15.1:0/992820972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556707bbd9b0 0x556707bfa460 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 >> 172.21.15.1:0/992820972 conn(0x556707b850a0 msgr2=0x556707bb77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2fc8ff9640 0 client.8613 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 shutdown_connections 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 1 -- 172.21.15.1:0/992820972 wait complete. 2024-05-09T06:35:46.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.622+0000 7f2ff29d9740 20 client.8613 trim_cache size 0 max 16384 2024-05-09T06:35:46.609 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:35:46.609 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:35:46.609 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "fa7da6d5-fecf-4fed-a75b-79a811e58a10", 2024-05-09T06:35:46.609 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:35:46.609 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 msgr2=0x7f2fcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8034000 tx=0x7f2fd802e000 comp rx=0 tx=0).stop 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 msgr2=0x7f2fe81f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c430 tx=0x7f2fe400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 shutdown_connections 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2fcc06be30 0x7f2fcc06e2f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe8145880 0x7f2fe813ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe80ced00 0x7f2fe80aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 --2- 172.21.15.1:0/1216166120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2fe80b3820 0x7f2fe81f1e30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 >> 172.21.15.1:0/1216166120 conn(0x7f2fe806b3c0 msgr2=0x7f2fe80af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 shutdown_connections 2024-05-09T06:35:46.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.623+0000 7f2fefccd640 1 -- 172.21.15.1:0/1216166120 wait complete. 2024-05-09T06:35:46.611 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T06:35:46.624+0000 7efc6d5cd640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55f34a2f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:35:46.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:46.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:35:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.740+0000 7f5983f09640 1 Processor -- start 2024-05-09T06:35:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.740+0000 7f5983f09640 1 -- start start 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 0x7f597c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f597c15c4a0 0x7f597c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f597c06e6d0 0x7f597c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f597c158e30 con 0x7f597c06e6d0 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f597c072480 con 0x7f597c15c4a0 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5983f09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f597c15b420 con 0x7f597c158a50 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f597c06e6d0 0x7f597c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f597c06e6d0 0x7f597c06eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40026/0 (socket says 172.21.15.1:40026) 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 -- 172.21.15.1:0/627810948 learned_addr learned my addr 172.21.15.1:0/627810948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5981ca5640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 0x7f597c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59814a4640 1 --2- 172.21.15.1:0/627810948 >> v2:172.21.15.6:3300/0 conn(0x7f597c15c4a0 0x7f597c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 -- 172.21.15.1:0/627810948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 msgr2=0x7f597c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 0x7f597c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 -- 172.21.15.1:0/627810948 >> v2:172.21.15.6:3300/0 conn(0x7f597c15c4a0 msgr2=0x7f597c071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 --2- 172.21.15.1:0/627810948 >> v2:172.21.15.6:3300/0 conn(0x7f597c15c4a0 0x7f597c071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f59824a6640 1 -- 172.21.15.1:0/627810948 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c15b6e0 con 0x7f597c06e6d0 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.741+0000 7f5981ca5640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 0x7f597c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.742+0000 7f59824a6640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c06e6d0 0x7f597c06eb80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f597c0b2770 tx=0x7f597402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c147fdbfb5bde3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.742+0000 7f5980ca3640 1 -- 172.21.15.1:0/627810948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974036070 con 0x7f597c06e6d0 2024-05-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.742+0000 7f5980ca3640 1 -- 172.21.15.1:0/627810948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5974031600 con 0x7f597c06e6d0 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.742+0000 7f5980ca3640 1 -- 172.21.15.1:0/627810948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740319d0 con 0x7f597c06e6d0 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 -- 172.21.15.1:0/627810948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c06e6d0 msgr2=0x7f597c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c06e6d0 0x7f597c06eb80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f597c0b2770 tx=0x7f597402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 -- 172.21.15.1:0/627810948 shutdown_connections 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c06e6d0 0x7f597c06eb80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 --2- 172.21.15.1:0/627810948 >> v2:172.21.15.6:3300/0 conn(0x7f597c15c4a0 0x7f597c071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 --2- 172.21.15.1:0/627810948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c158a50 0x7f597c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 -- 172.21.15.1:0/627810948 >> 172.21.15.1:0/627810948 conn(0x7f597c0cc4d0 msgr2=0x7f597c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 -- 172.21.15.1:0/627810948 shutdown_connections 2024-05-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.743+0000 7f5983f09640 1 -- 172.21.15.1:0/627810948 wait complete. 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.744+0000 7f5983f09640 1 Processor -- start 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.744+0000 7f5983f09640 1 -- start start 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 0x7f597c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 0x7f597c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f597c15bf80 con 0x7f597c15c4a0 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f597c15bca0 con 0x7f597c158a50 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5983f09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f597c070ea0 con 0x7f597c06e6d0 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40042/0 (socket says 172.21.15.1:40042) 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 -- 172.21.15.1:0/2426438784 learned_addr learned my addr 172.21.15.1:0/2426438784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 0x7f597c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59814a4640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 0x7f597c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 -- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 msgr2=0x7f597c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 0x7f597c0b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 -- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 msgr2=0x7f597c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 0x7f597c0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 -- 172.21.15.1:0/2426438784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c152de0 con 0x7f597c15c4a0 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59814a4640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 0x7f597c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 0x7f597c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.745+0000 7f59824a6640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5974034040 tx=0x7f5974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.746+0000 7f59727fc640 1 -- 172.21.15.1:0/2426438784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974041020 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.746+0000 7f59727fc640 1 -- 172.21.15.1:0/2426438784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59740077a0 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.746+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1530c0 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.746+0000 7f59727fc640 1 -- 172.21.15.1:0/2426438784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974036040 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.746+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c153680 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.747+0000 7f59727fc640 1 -- 172.21.15.1:0/2426438784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5974044020 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.748+0000 7f59727fc640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 0x7f596006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.748+0000 7f59727fc640 1 -- 172.21.15.1:0/2426438784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f59740abda0 con 0x7f597c15c4a0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.748+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 0x7f596006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.748+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 0x7f596006e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f597c152d10 tx=0x7f597800e360 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 Processor -- start 2024-05-09T06:35:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 -- start start 2024-05-09T06:35:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e4940 0x5586d57e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57bc650 0x5586d57e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586d5721d00 con 0x5586d57e5260 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5586d56c94c0 con 0x5586d57e4940 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.749+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5586d571dcc0 con 0x5586d57bc650 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40058/0 (socket says 172.21.15.1:40058) 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 -- 172.21.15.1:0/2498773455 learned_addr learned my addr 172.21.15.1:0/2498773455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59814a4640 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57bc650 0x5586d57e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e4940 0x5586d57e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 -- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57bc650 msgr2=0x5586d57e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57bc650 0x5586d57e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 -- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e4940 msgr2=0x5586d57e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e4940 0x5586d57e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 -- 172.21.15.1:0/2498773455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d54ff7d0 con 0x5586d57e5260 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f59824a6640 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5974079a10 tx=0x7f5974079a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f8a00ddfecae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f595f7fe640 1 -- 172.21.15.1:0/2498773455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740a8070 con 0x5586d57e5260 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f595f7fe640 1 -- 172.21.15.1:0/2498773455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597407a4d0 con 0x5586d57e5260 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.750+0000 7f595f7fe640 1 -- 172.21.15.1:0/2498773455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974079dd0 con 0x5586d57e5260 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- 172.21.15.1:0/2498773455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 msgr2=0x5586d57e7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5974079a10 tx=0x7f5974079a40 comp rx=0 tx=0).stop 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- 172.21.15.1:0/2498773455 shutdown_connections 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e5260 0x5586d57e7af0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57bc650 0x5586d57e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 --2- 172.21.15.1:0/2498773455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e4940 0x5586d57e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- 172.21.15.1:0/2498773455 >> 172.21.15.1:0/2498773455 conn(0x5586d57ba250 msgr2=0x5586d5802af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- 172.21.15.1:0/2498773455 shutdown_connections 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- 172.21.15.1:0/2498773455 wait complete. 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 -- start start 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.751+0000 7f5986c14740 1 Processor -- start 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 0x5586d57da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586d56c94c0 con 0x5586d5817b70 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5586d56dcb90 con 0x5586d57e5260 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5586d5721d00 con 0x5586d57e4940 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f5986c14740 10 client.0 _ll_register_callbacks cb 0x5586d5718da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.752+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40068/0 (socket says 172.21.15.1:40068) 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 -- 172.21.15.1:0/4011502439 learned_addr learned my addr 172.21.15.1:0/4011502439 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f5981ca5640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 0x5586d57da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59814a4640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 -- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 msgr2=0x5586d57da320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 0x5586d57da320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 -- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 msgr2=0x5586d57def20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57def20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 -- 172.21.15.1:0/4011502439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d54353f0 con 0x5586d5817b70 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f5981ca5640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 0x5586d57da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f59824a6640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5974031780 tx=0x7f5974009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bc7f0f3183be1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:46.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597409a030 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f59740780d0 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f597407a4d0 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f59740a3050 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f5986c14740 10 client.8625 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5586d53f50f0 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597407a4d0 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.753+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f5974007b90 con 0x5586d5817b70 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 10 client.8625 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 10 client.8625 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 20 client.8625 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d57dadb0 0x5586d57db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 4 client.8625 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d572f430 con 0x5586d57dadb0 2024-05-09T06:35:46.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5981ca5640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d57dadb0 0x5586d57db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f5981ca5640 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d57dadb0 0x5586d57db1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f128dc77a0b46c server_cookie=db730b3c4e04a694 in_seq=0 out_seq=0 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.754+0000 7f595cff9640 10 client.8625 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f595cff9640 1 -- 172.21.15.1:0/4011502439 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586d572f430 con 0x5586d57dadb0 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f595cff9640 10 client.8625 handle_command_reply: tid=0 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 client.8625 shutdown 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 msgr2=0x5586d57dd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5974031780 tx=0x7f5974009810 comp rx=0 tx=0).stop 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 shutdown_connections 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d57dadb0 0x5586d57db1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d5817b70 0x5586d57dd810 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f595cff9640 0 client.8625 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 --2- 172.21.15.1:0/4011502439 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e4940 0x5586d57da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 >> 172.21.15.1:0/4011502439 conn(0x5586d57bc650 msgr2=0x5586d57b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 shutdown_connections 2024-05-09T06:35:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 1 -- 172.21.15.1:0/4011502439 wait complete. 2024-05-09T06:35:46.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.755+0000 7f5986c14740 20 client.8625 trim_cache size 0 max 16384 2024-05-09T06:35:46.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 Processor -- start 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 -- start start 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 0x5586d5804a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 0x5586d57e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1fca90 con 0x5586d57e4940 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1fba70 con 0x5586d5817b70 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f597c1fadb0 con 0x5586d57e5260 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5981ca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5981ca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40072/0 (socket says 172.21.15.1:40072) 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.773+0000 7f5981ca5640 1 -- 172.21.15.1:0/2224945491 learned_addr learned my addr 172.21.15.1:0/2224945491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f59814a4640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 0x5586d5804a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f59824a6640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 0x5586d57e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 -- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 msgr2=0x5586d5804a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 0x5586d5804a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 -- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 msgr2=0x5586d57e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 0x5586d57e9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 -- 172.21.15.1:0/2224945491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d57db270 con 0x5586d57e4940 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f59814a4640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 0x5586d5804a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f59824a6640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 0x5586d57e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f5981ca5640 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5978012c60 tx=0x7f5978015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e6f7f1dba5f00d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f595effd640 1 -- 172.21.15.1:0/2224945491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597802d070 con 0x5586d57e4940 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f595effd640 1 -- 172.21.15.1:0/2224945491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5978028630 con 0x5586d57e4940 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.774+0000 7f595effd640 1 -- 172.21.15.1:0/2224945491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597801d280 con 0x5586d57e4940 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- 172.21.15.1:0/2224945491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 msgr2=0x5586d5804540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5978012c60 tx=0x7f5978015000 comp rx=0 tx=0).stop 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- 172.21.15.1:0/2224945491 shutdown_connections 2024-05-09T06:35:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d5817b70 0x5586d57e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d57e5260 0x5586d5804a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 --2- 172.21.15.1:0/2224945491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d5804540 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- 172.21.15.1:0/2224945491 >> 172.21.15.1:0/2224945491 conn(0x5586d57bc650 msgr2=0x5586d57da130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- 172.21.15.1:0/2224945491 shutdown_connections 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- 172.21.15.1:0/2224945491 wait complete. 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 -- start start 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.775+0000 7f5986c14740 1 Processor -- start 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d5817b70 0x5586d5884270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586d582b110 con 0x5586d57e4940 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1fb400 con 0x5586d57e5260 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5586d57b6f50 con 0x5586d5817b70 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5986c14740 10 client.0 _ll_register_callbacks cb 0x7f59740a6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:35:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5981ca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5981ca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40088/0 (socket says 172.21.15.1:40088) 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f5981ca5640 1 -- 172.21.15.1:0/655111460 learned_addr learned my addr 172.21.15.1:0/655111460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f59824a6640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d5817b70 0x5586d5884270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.776+0000 7f59814a4640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 -- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d5817b70 msgr2=0x5586d5884270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d5817b70 0x5586d5884270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 -- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 msgr2=0x5586d57dcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57dcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 -- 172.21.15.1:0/655111460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d58698f0 con 0x5586d57e4940 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f59814a4640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5981ca5640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f597801df00 tx=0x7f5978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59676c25443dd5a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597802d040 con 0x5586d57e4940 2024-05-09T06:35:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f5978033430 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5978028630 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f597803d050 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 10 client.8631 fetch_fsmap learned FSMap version 74 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5586d57bcbc0 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5978028630 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f5978040020 con 0x5586d57e4940 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 10 client.8631 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 10 client.8631 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.777+0000 7f5986c14740 20 client.8631 populate_metadata read hostname 'smithi001' 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d5869310 0x5586d58847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 4 client.8631 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586d5729780 con 0x5586d5869310 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f59814a4640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d5869310 0x5586d58847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f595cff9640 10 client.8631 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f59814a4640 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d5869310 0x5586d58847b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dabc4d1596075ec server_cookie=11b83bc7e5d8f3a4 in_seq=0 out_seq=0 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f595cff9640 1 -- 172.21.15.1:0/655111460 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5586d5729780 con 0x5586d5869310 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f595cff9640 10 client.8631 handle_command_reply: tid=0 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 client.8631 shutdown 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 msgr2=0x5586d57dc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f597801df00 tx=0x7f5978004730 comp rx=0 tx=0).stop 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 shutdown_connections 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x5586d5869310 0x5586d58847b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5586d5817b70 0x5586d5884270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5586d57e5260 0x5586d57dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f595cff9640 0 client.8631 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 --2- 172.21.15.1:0/655111460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586d57e4940 0x5586d57dc710 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 >> 172.21.15.1:0/655111460 conn(0x5586d57bc650 msgr2=0x5586d57bf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 shutdown_connections 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.778+0000 7f5986c14740 1 -- 172.21.15.1:0/655111460 wait complete. 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5986c14740 20 client.8631 trim_cache size 0 max 16384 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "fa7da6d5-fecf-4fed-a75b-79a811e58a10": { 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/dir", 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "fa7da6d5-fecf-4fed-a75b-79a811e58a10", 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:35:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:35:46.768 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:35:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 msgr2=0x7f596006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 0x7f596006e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f597c152d10 tx=0x7f597800e360 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 msgr2=0x7f597c1542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5974034040 tx=0x7f5974004a40 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 shutdown_connections 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f596006be30 0x7f596006e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f597c15c4a0 0x7f597c1542c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f597c158a50 0x7f597c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 --2- 172.21.15.1:0/2426438784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f597c06e6d0 0x7f597c0b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 >> 172.21.15.1:0/2426438784 conn(0x7f597c0cc4d0 msgr2=0x7f597c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 shutdown_connections 2024-05-09T06:35:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:35:46.779+0000 7f5983f09640 1 -- 172.21.15.1:0/2426438784 wait complete. 2024-05-09T06:35:46.771 INFO:tasks.cephfs.filesystem:scrub status for tag:fa7da6d5-fecf-4fed-a75b-79a811e58a10 - {'path': '/dir', 'tag': 'fa7da6d5-fecf-4fed-a75b-79a811e58a10', 'options': 'recursive,repair'} 2024-05-09T06:35:48.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:48.689 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:16.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:16.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-09T06:36:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.911+0000 7f111b6f5640 1 Processor -- start 2024-05-09T06:36:16.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.911+0000 7f111b6f5640 1 -- start start 2024-05-09T06:36:16.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f111b6f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f11140b11f0 0x7f11140b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f111b6f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f11140cc8e0 0x7f11140ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f111b6f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111415e900 con 0x7f111415e390 2024-05-09T06:36:16.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f111b6f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f111415ebe0 con 0x7f11140b11f0 2024-05-09T06:36:16.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f111b6f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f111415eec0 con 0x7f11140cc8e0 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f11140b11f0 0x7f11140b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f11140b11f0 0x7f11140b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43674/0 (socket says 172.21.15.1:43674) 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 -- 172.21.15.1:0/1227045325 learned_addr learned my addr 172.21.15.1:0/1227045325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 -- 172.21.15.1:0/1227045325 >> v2:172.21.15.6:3300/0 conn(0x7f11140cc8e0 msgr2=0x7f11140ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 --2- 172.21.15.1:0/1227045325 >> v2:172.21.15.6:3300/0 conn(0x7f11140cc8e0 0x7f11140ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 -- 172.21.15.1:0/1227045325 >> v1:172.21.15.6:6789/0 conn(0x7f111415e390 legacy=0x7f11140cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.912+0000 7f1118c90640 1 -- 172.21.15.1:0/1227045325 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111415f1a0 con 0x7f11140b11f0 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.913+0000 7f1118c90640 1 --2- 172.21.15.1:0/1227045325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11140b1680 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f11140721e0 tx=0x7f110402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476c1182f23468ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.913+0000 7f110bfff640 1 -- 172.21.15.1:0/1227045325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104037070 con 0x7f11140b11f0 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.913+0000 7f110bfff640 1 -- 172.21.15.1:0/1227045325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1104032440 con 0x7f11140b11f0 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.913+0000 7f110bfff640 1 -- 172.21.15.1:0/1227045325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104032810 con 0x7f11140b11f0 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 -- 172.21.15.1:0/1227045325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 msgr2=0x7f11140b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 --2- 172.21.15.1:0/1227045325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11140b1680 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f11140721e0 tx=0x7f110402a230 comp rx=0 tx=0).stop 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 -- 172.21.15.1:0/1227045325 shutdown_connections 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 --2- 172.21.15.1:0/1227045325 >> v2:172.21.15.6:3300/0 conn(0x7f11140cc8e0 0x7f11140ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 --2- 172.21.15.1:0/1227045325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11140b1680 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 -- 172.21.15.1:0/1227045325 >> 172.21.15.1:0/1227045325 conn(0x7f11140b1da0 msgr2=0x7f11140b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 -- 172.21.15.1:0/1227045325 shutdown_connections 2024-05-09T06:36:16.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.914+0000 7f111b6f5640 1 -- 172.21.15.1:0/1227045325 wait complete. 2024-05-09T06:36:16.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.915+0000 7f111b6f5640 1 Processor -- start 2024-05-09T06:36:16.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.915+0000 7f111b6f5640 1 -- start start 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 0x7f11140999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 0x7f11141f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11140cffb0 con 0x7f11140b11f0 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1114162ba0 con 0x7f111415e390 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f111b6f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f11140cfb60 con 0x7f11140cc8e0 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43688/0 (socket says 172.21.15.1:43688) 2024-05-09T06:36:16.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 -- 172.21.15.1:0/3266378512 learned_addr learned my addr 172.21.15.1:0/3266378512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1118c90640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 0x7f11140999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119c92640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 0x7f11141f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 -- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 msgr2=0x7f11140999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 0x7f11140999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 -- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 msgr2=0x7f11141f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 0x7f11141f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1119491640 1 -- 172.21.15.1:0/3266378512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141f59c0 con 0x7f11140b11f0 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.916+0000 7f1118c90640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 0x7f11140999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f1119c92640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 0x7f11141f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f1119491640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f111000b410 tx=0x7f111000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f1109ffb640 1 -- 172.21.15.1:0/3266378512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110015020 con 0x7f11140b11f0 2024-05-09T06:36:16.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f1109ffb640 1 -- 172.21.15.1:0/3266378512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1110011830 con 0x7f11140b11f0 2024-05-09T06:36:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f1109ffb640 1 -- 172.21.15.1:0/3266378512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110011c20 con 0x7f11140b11f0 2024-05-09T06:36:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11140c3ed0 con 0x7f11140b11f0 2024-05-09T06:36:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.917+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11140c4490 con 0x7f11140b11f0 2024-05-09T06:36:16.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.918+0000 7f1109ffb640 1 -- 172.21.15.1:0/3266378512 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1110018020 con 0x7f11140b11f0 2024-05-09T06:36:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.919+0000 7f1109ffb640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 0x7f10f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.919+0000 7f1109ffb640 1 -- 172.21.15.1:0/3266378512 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f111008b020 con 0x7f11140b11f0 2024-05-09T06:36:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.919+0000 7f1118c90640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 0x7f10f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.920+0000 7f1118c90640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 0x7f10f806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1104034000 tx=0x7f110402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.920+0000 7f111e402740 1 Processor -- start 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.920+0000 7f111e402740 1 -- start start 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9b70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9ba0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562aa9adad00 con 0x562aa9ba1030 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562aa9a824c0 con 0x562aa9ba1770 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562aa9ad6cc0 con 0x562aa9bbe650 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43692/0 (socket says 172.21.15.1:43692) 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 -- 172.21.15.1:0/3546402550 learned_addr learned my addr 172.21.15.1:0/3546402550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119491640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9b70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1118c90640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9ba0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 -- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 msgr2=0x562aa9b70fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9b70fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 -- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 msgr2=0x562aa9ba0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9ba0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119c92640 1 -- 172.21.15.1:0/3546402550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa98b87d0 con 0x562aa9ba1030 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1119491640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9b70fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.921+0000 7f1118c90640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9ba0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f1119c92640 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f110c007f60 tx=0x7f110c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0569c6d2b17b02a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f10f6ffd640 1 -- 172.21.15.1:0/3546402550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c014070 con 0x562aa9ba1030 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f10f6ffd640 1 -- 172.21.15.1:0/3546402550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c00f850 con 0x562aa9ba1030 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f10f6ffd640 1 -- 172.21.15.1:0/3546402550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c00fc20 con 0x562aa9ba1030 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 -- 172.21.15.1:0/3546402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 msgr2=0x562aa9b75e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f110c007f60 tx=0x7f110c007f90 comp rx=0 tx=0).stop 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 -- 172.21.15.1:0/3546402550 shutdown_connections 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1030 0x562aa9b75e40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9ba0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 --2- 172.21.15.1:0/3546402550 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9b70fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 -- 172.21.15.1:0/3546402550 >> 172.21.15.1:0/3546402550 conn(0x562aa9b73250 msgr2=0x562aa9bbbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 -- 172.21.15.1:0/3546402550 shutdown_connections 2024-05-09T06:36:16.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.922+0000 7f111e402740 1 -- 172.21.15.1:0/3546402550 wait complete. 2024-05-09T06:36:16.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.923+0000 7f111e402740 1 -- start start 2024-05-09T06:36:16.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.923+0000 7f111e402740 1 Processor -- start 2024-05-09T06:36:16.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9bbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9b9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562aa9b75400 con 0x562aa9bbe650 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562aa9a95b90 con 0x562aa9ba1770 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562aa9b747f0 con 0x562aa9ba3f70 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f111e402740 10 client.0 _ll_register_callbacks cb 0x562aa9ad1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43694/0 (socket says 172.21.15.1:43694) 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 -- 172.21.15.1:0/2869794673 learned_addr learned my addr 172.21.15.1:0/2869794673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1119c92640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9b9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1119491640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9bbdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 -- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 msgr2=0x562aa9b9fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9b9fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 -- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 msgr2=0x562aa9bbdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9bbdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.924+0000 7f1118c90640 1 -- 172.21.15.1:0/2869794673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aa97ee3f0 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f1118c90640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f110402a820 tx=0x7f110402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44c779ebf54bdb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104037040 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f110403ebb0 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c00f850 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1104056050 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 10 client.8643 fetch_fsmap learned FSMap version 74 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x562aa97ae0f0 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c00f850 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f1104059020 con 0x562aa9bbe650 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 10 client.8643 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 10 client.8643 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 20 client.8643 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bcf700 0x562aa9bcfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 4 client.8643 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.925+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aa9ae8430 con 0x562aa9bcf700 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f1119491640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bcf700 0x562aa9bcfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f1119491640 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bcf700 0x562aa9bcfb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e2459f090be6ca server_cookie=c70c7d176da28bf5 in_seq=0 out_seq=0 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f10effff640 10 client.8643 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f10effff640 1 -- 172.21.15.1:0/2869794673 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562aa9ae8430 con 0x562aa9bcf700 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f10effff640 10 client.8643 handle_command_reply: tid=0 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.926+0000 7f111e402740 1 client.8643 shutdown 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 msgr2=0x562aa9ba3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f110402a820 tx=0x7f110402c070 comp rx=0 tx=0).stop 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 shutdown_connections 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bcf700 0x562aa9bcfb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9b9fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9bbe650 0x562aa9ba3a30 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f10effff640 0 client.8643 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 --2- 172.21.15.1:0/2869794673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba1770 0x562aa9bbdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 >> 172.21.15.1:0/2869794673 conn(0x562aa9ba1030 msgr2=0x562aa9bbef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 shutdown_connections 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 1 -- 172.21.15.1:0/2869794673 wait complete. 2024-05-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.927+0000 7f111e402740 20 client.8643 trim_cache size 0 max 16384 2024-05-09T06:36:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.945+0000 7f111e402740 1 Processor -- start 2024-05-09T06:36:16.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.945+0000 7f111e402740 1 -- start start 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9bbef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9bbe650 0x562aa9bbf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1114201150 con 0x562aa9ba1770 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1114200130 con 0x562aa9bbe650 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f11140b37e0 con 0x562aa9ba3f70 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43706/0 (socket says 172.21.15.1:43706) 2024-05-09T06:36:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 -- 172.21.15.1:0/507918038 learned_addr learned my addr 172.21.15.1:0/507918038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 -- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 msgr2=0x562aa9bbef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9bbef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 -- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9bbe650 msgr2=0x562aa9bbf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119c92640 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9bbe650 0x562aa9bbf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9bbe650 0x562aa9bbf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 -- 172.21.15.1:0/507918038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa9b78670 con 0x562aa9ba1770 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f1119491640 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1110089310 tx=0x7f1110058a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f850e8182e2b716e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f10f67fc640 1 -- 172.21.15.1:0/507918038 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110086070 con 0x562aa9ba1770 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f10f67fc640 1 -- 172.21.15.1:0/507918038 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111005d0b0 con 0x562aa9ba1770 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.946+0000 7f10f67fc640 1 -- 172.21.15.1:0/507918038 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100584e0 con 0x562aa9ba1770 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- 172.21.15.1:0/507918038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 msgr2=0x562aa9b9c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1110089310 tx=0x7f1110058a70 comp rx=0 tx=0).stop 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- 172.21.15.1:0/507918038 shutdown_connections 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9bbe650 0x562aa9bbf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9ba3f70 0x562aa9bbef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 --2- 172.21.15.1:0/507918038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9b9c1a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- 172.21.15.1:0/507918038 >> 172.21.15.1:0/507918038 conn(0x562aa9ba1030 msgr2=0x562aa9bcea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- 172.21.15.1:0/507918038 shutdown_connections 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- 172.21.15.1:0/507918038 wait complete. 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.947+0000 7f111e402740 1 -- start start 2024-05-09T06:36:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 Processor -- start 2024-05-09T06:36:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba3f70 0x562aa9bb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9bb7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11141ffb00 con 0x562aa9ba1770 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562aa9be0000 con 0x562aa9ba3f70 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562aa9bca880 con 0x562aa9bbe650 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.948+0000 7f111e402740 10 client.0 _ll_register_callbacks cb 0x7f1104035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43716/0 (socket says 172.21.15.1:43716) 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 -- 172.21.15.1:0/2139009266 learned_addr learned my addr 172.21.15.1:0/2139009266 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119c92640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9bb7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1118c90640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba3f70 0x562aa9bb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 -- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 msgr2=0x562aa9bb7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9bb7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 -- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba3f70 msgr2=0x562aa9bb7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba3f70 0x562aa9bb7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 -- 172.21.15.1:0/2139009266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aa9bdf6b0 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f1119491640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f11100554f0 tx=0x7f111006f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f1e26d0aca9736 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f10effff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111007d060 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f11100586e0 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111005d0b0 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1110086040 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f111e402740 10 client.8649 fetch_fsmap learned FSMap version 74 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x562aa9bca620 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.949+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111005d0b0 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f1110072070 con 0x562aa9ba1770 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 10 client.8649 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 10 client.8649 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-09T06:36:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 20 client.8649 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bbcbc0 0x562aa9bbf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 4 client.8649 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562aa9ae2780 con 0x562aa9bbcbc0 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f1118c90640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bbcbc0 0x562aa9bbf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f10effff640 10 client.8649 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f1118c90640 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bbcbc0 0x562aa9bbf3a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6500d2871df99fdf server_cookie=d36bb19d39085962 in_seq=0 out_seq=0 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f10effff640 1 -- 172.21.15.1:0/2139009266 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562aa9ae2780 con 0x562aa9bbcbc0 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f10effff640 10 client.8649 handle_command_reply: tid=0 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 client.8649 shutdown 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 msgr2=0x562aa9c353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f11100554f0 tx=0x7f111006f040 comp rx=0 tx=0).stop 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 shutdown_connections 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x562aa9bbcbc0 0x562aa9bbf3a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562aa9bbe650 0x562aa9bb7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f10effff640 0 client.8649 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562aa9ba3f70 0x562aa9bb7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 --2- 172.21.15.1:0/2139009266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562aa9ba1770 0x562aa9c353c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.950+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 >> 172.21.15.1:0/2139009266 conn(0x562aa9ba1030 msgr2=0x562aa9bbadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 shutdown_connections 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111e402740 1 -- 172.21.15.1:0/2139009266 wait complete. 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111e402740 20 client.8649 trim_cache size 0 max 16384 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:36:16.937 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:36:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 msgr2=0x7f10f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 0x7f10f806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1104034000 tx=0x7f110402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 msgr2=0x7f11141f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f111000b410 tx=0x7f111000b8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 shutdown_connections 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f10f806be30 0x7f10f806e2f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f111415e390 0x7f11141f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f11140cc8e0 0x7f11140999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 --2- 172.21.15.1:0/3266378512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11140b11f0 0x7f11141f4eb0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 >> 172.21.15.1:0/3266378512 conn(0x7f11140b1da0 msgr2=0x7f11141613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 shutdown_connections 2024-05-09T06:36:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:16.951+0000 7f111b6f5640 1 -- 172.21.15.1:0/3266378512 wait complete. 2024-05-09T06:36:16.943 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:36:16.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:17.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.066+0000 7f76098a3640 1 Processor -- start 2024-05-09T06:36:17.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.066+0000 7f76098a3640 1 -- start start 2024-05-09T06:36:17.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76098a3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f760415e1d0 0x7f760406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76098a3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f76040d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76098a3640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760406c2f0 con 0x7f760415e5b0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76098a3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f76040cc3c0 con 0x7f760415e1d0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76098a3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76040ccc60 con 0x7f76040d2890 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76027fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f760415e5b0 0x7f76040cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51910/0 (socket says 172.21.15.1:51910) 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76027fc640 1 -- 172.21.15.1:0/1072721511 learned_addr learned my addr 172.21.15.1:0/1072721511 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f76037fe640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f76040d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7602ffd640 1 --2- 172.21.15.1:0/1072721511 >> v2:172.21.15.6:3300/0 conn(0x7f760415e1d0 0x7f760406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856589486 0 0) 0x7f75f8001110 con 0x7f760415e5b0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75ec003520 con 0x7f760415e5b0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1550773312 0 0) 0x7f75ec003520 con 0x7f760415e5b0 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 msgr2=0x7f76040d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f76040d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 >> v2:172.21.15.6:3300/0 conn(0x7f760415e1d0 msgr2=0x7f760406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 --2- 172.21.15.1:0/1072721511 >> v2:172.21.15.6:3300/0 conn(0x7f760415e1d0 0x7f760406f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760406f500 con 0x7f760415e5b0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.067+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f75f8001840 con 0x7f760415e5b0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 >> v1:172.21.15.1:6789/0 conn(0x7f760415e5b0 legacy=0x7f76040cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75ec0039b0 0x7f75ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f75ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec004d40 0x7f75ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76040cc3c0 con 0x7f75ec0039b0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76040ccc60 con 0x7f75ec004d40 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec00b9f0 con 0x7f76040d2890 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75ec0039b0 0x7f75ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f76027fc640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f75ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f76037fe640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec004d40 0x7f75ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 -- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 msgr2=0x7f75ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040d2890 0x7f75ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 -- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec004d40 msgr2=0x7f75ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec004d40 0x7f75ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f7602ffd640 1 -- 172.21.15.1:0/1072721511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec003520 con 0x7f75ec0039b0 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.068+0000 7f76037fe640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec004d40 0x7f75ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f7602ffd640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75ec0039b0 0x7f75ec003dd0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009ac0 tx=0x7f75ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39851397b9b403e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4036070 con 0x7f75ec0039b0 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4031400 con 0x7f75ec0039b0 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f7601ffb640 1 -- 172.21.15.1:0/1072721511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4009690 con 0x7f75ec0039b0 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 -- 172.21.15.1:0/1072721511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75ec0039b0 msgr2=0x7f75ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 --2- 172.21.15.1:0/1072721511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75ec0039b0 0x7f75ec003dd0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009ac0 tx=0x7f75ec005830 comp rx=0 tx=0).stop 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76037fe640 1 -- 172.21.15.1:0/1072721511 reap_dead start 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 -- 172.21.15.1:0/1072721511 shutdown_connections 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 -- 172.21.15.1:0/1072721511 >> 172.21.15.1:0/1072721511 conn(0x7f76040d1d70 msgr2=0x7f76040d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 -- 172.21.15.1:0/1072721511 shutdown_connections 2024-05-09T06:36:17.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.069+0000 7f76098a3640 1 -- 172.21.15.1:0/1072721511 wait complete. 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.070+0000 7f76098a3640 1 Processor -- start 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 -- start start 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec0039b0 0x7f7604160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040c6dc0 0x7f76040c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f760406f1b0 con 0x7f76040c9530 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76040cc8f0 con 0x7f75ec0039b0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.071+0000 7f76098a3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76040d0fc0 con 0x7f76040c6dc0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43734/0 (socket says 172.21.15.1:43734) 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 -- 172.21.15.1:0/2933942359 learned_addr learned my addr 172.21.15.1:0/2933942359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76037fe640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040c6dc0 0x7f76040c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f7602ffd640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec0039b0 0x7f7604160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 -- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040c6dc0 msgr2=0x7f76040c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040c6dc0 0x7f76040c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 -- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec0039b0 msgr2=0x7f7604160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec0039b0 0x7f7604160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 -- 172.21.15.1:0/2933942359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76041574c0 con 0x7f76040c9530 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76027fc640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f75f8007370 tx=0x7f75f8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800f960 con 0x7f76040c9530 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f8018070 con 0x7f76040c9530 2024-05-09T06:36:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76040c9ad0 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800fd40 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.072+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7604154400 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.073+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76040752b0 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.074+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75f801d460 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.074+0000 7f75ebfff640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 0x7f75f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.074+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f75f808b1b0 con 0x7f76040c9530 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.074+0000 7f7602ffd640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 0x7f75f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.075+0000 7f7602ffd640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 0x7f75f006e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f75f4033000 tx=0x7f75f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.079+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f760406c2f0 con 0x7f76040c9530 2024-05-09T06:36:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.301+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f760415ccc0 con 0x7f76040c9530 2024-05-09T06:36:17.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.301+0000 7f75ebfff640 1 -- 172.21.15.1:0/2933942359 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7f75f80598b0 con 0x7f76040c9530 2024-05-09T06:36:17.283 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:17.283 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:36:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.303+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 msgr2=0x7f75f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.303+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 0x7f75f006e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f75f4033000 tx=0x7f75f4008030 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.303+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 msgr2=0x7f7604156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.303+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f75f8007370 tx=0x7f75f8007830 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 shutdown_connections 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f75f006be30 0x7f75f006e2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76040c6dc0 0x7f76040c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76040c9530 0x7f7604156f80 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 --2- 172.21.15.1:0/2933942359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75ec0039b0 0x7f7604160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 >> 172.21.15.1:0/2933942359 conn(0x7f76040d1d70 msgr2=0x7f76040af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 shutdown_connections 2024-05-09T06:36:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.304+0000 7f76098a3640 1 -- 172.21.15.1:0/2933942359 wait complete. 2024-05-09T06:36:17.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-09T06:36:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.421+0000 7f83cb9d5640 1 Processor -- start 2024-05-09T06:36:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.421+0000 7f83cb9d5640 1 -- start start 2024-05-09T06:36:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83cb9d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83cb9d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 0x7f83c4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83cb9d5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c40cf1b0 con 0x7f83c4071440 2024-05-09T06:36:17.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83cb9d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f83c415a8b0 con 0x7f83c40cdb90 2024-05-09T06:36:17.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83cb9d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f83c406dc10 con 0x7f83c415a4d0 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83c9771640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83c8f70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 0x7f83c4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83c9771640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:39756/0 (socket says 172.21.15.1:39756) 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83c8f70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 0x7f83c4070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:40892/0 (socket says 172.21.15.1:40892) 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.422+0000 7f83c9771640 1 -- 172.21.15.1:0/1025863237 learned_addr learned my addr 172.21.15.1:0/1025863237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.423+0000 7f83c9771640 1 -- 172.21.15.1:0/1025863237 >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 msgr2=0x7f83c4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.423+0000 7f83c9771640 1 --2- 172.21.15.1:0/1025863237 >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 0x7f83c4070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.423+0000 7f83c9771640 1 -- 172.21.15.1:0/1025863237 >> v1:172.21.15.6:6789/0 conn(0x7f83c4071440 legacy=0x7f83c415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.423+0000 7f83c9771640 1 -- 172.21.15.1:0/1025863237 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40cf490 con 0x7f83c415a4d0 2024-05-09T06:36:17.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.423+0000 7f83c9771640 1 --2- 172.21.15.1:0/1025863237 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009870 tx=0x7f83c002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fffbddffe16d4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83bbfff640 1 -- 172.21.15.1:0/1025863237 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0037070 con 0x7f83c415a4d0 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83bbfff640 1 -- 172.21.15.1:0/1025863237 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0032440 con 0x7f83c415a4d0 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83bbfff640 1 -- 172.21.15.1:0/1025863237 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0032810 con 0x7f83c415a4d0 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1025863237 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 msgr2=0x7f83c40cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1025863237 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009870 tx=0x7f83c002a270 comp rx=0 tx=0).stop 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1025863237 shutdown_connections 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1025863237 >> v2:172.21.15.6:3300/0 conn(0x7f83c40cdb90 0x7f83c4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1025863237 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c40cea80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1025863237 >> 172.21.15.1:0/1025863237 conn(0x7f83c40cc4d0 msgr2=0x7f83c40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1025863237 shutdown_connections 2024-05-09T06:36:17.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.424+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1025863237 wait complete. 2024-05-09T06:36:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.425+0000 7f83cb9d5640 1 Processor -- start 2024-05-09T06:36:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 -- start start 2024-05-09T06:36:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83c4071440 0x7f83c40a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b2be0 con 0x7f83c40cdb90 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b2a60 con 0x7f83c4071440 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83cb9d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f83c415e6e0 con 0x7f83c415a4d0 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43750/0 (socket says 172.21.15.1:43750) 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.426+0000 7f83c8f70640 1 -- 172.21.15.1:0/1446675872 learned_addr learned my addr 172.21.15.1:0/1446675872 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c9771640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83c4071440 0x7f83c40a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c9f72640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 -- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 msgr2=0x7f83c414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 -- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83c4071440 msgr2=0x7f83c40a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83c4071440 0x7f83c40a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 -- 172.21.15.1:0/1446675872 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c415aa30 con 0x7f83c40cdb90 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83c8f70640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83b9ffb640 1 -- 172.21.15.1:0/1446675872 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4017070 con 0x7f83c40cdb90 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83b9ffb640 1 -- 172.21.15.1:0/1446675872 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b400c9a0 con 0x7f83c40cdb90 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c415acf0 con 0x7f83c40cdb90 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83b9ffb640 1 -- 172.21.15.1:0/1446675872 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b400cd90 con 0x7f83c40cdb90 2024-05-09T06:36:17.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.427+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c415b2b0 con 0x7f83c40cdb90 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.429+0000 7f83b9ffb640 1 -- 172.21.15.1:0/1446675872 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83b401a070 con 0x7f83c40cdb90 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.429+0000 7f83b9ffb640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.429+0000 7f83b9ffb640 1 -- 172.21.15.1:0/1446675872 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f83b408b220 con 0x7f83c40cdb90 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.429+0000 7f83c9771640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.430+0000 7f83c9771640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 0x7f83a806e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f83c0034000 tx=0x7f83c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.430+0000 7f83ce7c4740 1 Processor -- start 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 -- start start 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0cfb60 0x55835a0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55835a008a30 con 0x55835a0ed180 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x558359fa6590 con 0x55835a0d02a0 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55835a0049f0 con 0x55835a0cfb60 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83c9771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83c9771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43752/0 (socket says 172.21.15.1:43752) 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83c9771640 1 -- 172.21.15.1:0/3032551595 learned_addr learned my addr 172.21.15.1:0/3032551595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83c8f70640 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.431+0000 7f83c9f72640 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0cfb60 0x55835a0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 -- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0cfb60 msgr2=0x55835a0a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0cfb60 0x55835a0a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 -- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 msgr2=0x55835a0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 -- 172.21.15.1:0/3032551595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558359de87c0 con 0x55835a0ed180 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83c9771640 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000c00 tx=0x7f83c002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc41513bf7769159 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83aeffd640 1 -- 172.21.15.1:0/3032551595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c004c070 con 0x55835a0ed180 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83aeffd640 1 -- 172.21.15.1:0/3032551595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0032d90 con 0x55835a0ed180 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.432+0000 7f83aeffd640 1 -- 172.21.15.1:0/3032551595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0037040 con 0x55835a0ed180 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3032551595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 msgr2=0x55835a09fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000c00 tx=0x7f83c002c000 comp rx=0 tx=0).stop 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3032551595 shutdown_connections 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0cfb60 0x55835a0a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3032551595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a09fb10 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3032551595 >> 172.21.15.1:0/3032551595 conn(0x55835a0a1d80 msgr2=0x55835a0ea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3032551595 shutdown_connections 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3032551595 wait complete. 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 -- start start 2024-05-09T06:36:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.433+0000 7f83ce7c4740 1 Processor -- start 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55835a0a3f30 con 0x55835a0ed180 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x558359fa4060 con 0x55835a0d02a0 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55835a0a3320 con 0x55835a0d2690 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83ce7c4740 10 client.0 _ll_register_callbacks cb 0x558359fc8dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43762/0 (socket says 172.21.15.1:43762) 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.434+0000 7f83c8f70640 1 -- 172.21.15.1:0/3749980936 learned_addr learned my addr 172.21.15.1:0/3749980936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c9771640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c9f72640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 -- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 msgr2=0x55835a0a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 -- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 msgr2=0x55835a0e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 -- 172.21.15.1:0/3749980936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558359e48f50 con 0x55835a0ed180 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c9f72640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83c8f70640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f83b4016000 tx=0x7f83b40599e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237f7ba1ff1e4a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4021040 con 0x55835a0ed180 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f83b405d170 con 0x55835a0ed180 2024-05-09T06:36:17.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c0032d90 con 0x55835a0ed180 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f83b4078070 con 0x55835a0ed180 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 10 client.8664 fetch_fsmap learned FSMap version 74 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x558359cdc0b0 con 0x55835a0ed180 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0032d90 con 0x55835a0ed180 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f83b4024060 con 0x55835a0ed180 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 10 client.8664 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 10 client.8664 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 20 client.8664 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.435+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0bded0 0x55835a0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83ce7c4740 4 client.8664 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55835a016160 con 0x55835a0bded0 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83c9771640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0bded0 0x55835a0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83c9771640 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0bded0 0x55835a0d0e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c2a23863493996 server_cookie=dc8c67699984955c in_seq=0 out_seq=0 2024-05-09T06:36:17.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83a3fff640 10 client.8664 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83a3fff640 1 -- 172.21.15.1:0/3749980936 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55835a016160 con 0x55835a0bded0 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.436+0000 7f83a3fff640 10 client.8664 handle_command_reply: tid=0 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 client.8664 shutdown 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 msgr2=0x55835a0a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f83b4016000 tx=0x7f83b40599e0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 shutdown_connections 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0bded0 0x55835a0d0e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0ed180 0x55835a0a4850 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83a3fff640 0 client.8664 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/3749980936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0d02a0 0x55835a0e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 >> 172.21.15.1:0/3749980936 conn(0x55835a0cfb60 msgr2=0x55835a0ea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 shutdown_connections 2024-05-09T06:36:17.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 1 -- 172.21.15.1:0/3749980936 wait complete. 2024-05-09T06:36:17.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.437+0000 7f83ce7c4740 20 client.8664 trim_cache size 0 max 16384 2024-05-09T06:36:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.450+0000 7f83ce7c4740 1 Processor -- start 2024-05-09T06:36:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.450+0000 7f83ce7c4740 1 -- start start 2024-05-09T06:36:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d02a0 0x55835a09ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41fddd0 con 0x55835a0d2690 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f83c414a0f0 con 0x55835a0ed180 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f83c41fd8d0 con 0x55835a0d02a0 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43774/0 (socket says 172.21.15.1:43774) 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 -- 172.21.15.1:0/2324167923 learned_addr learned my addr 172.21.15.1:0/2324167923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 -- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d02a0 msgr2=0x55835a09ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d02a0 0x55835a09ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 -- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 msgr2=0x55835a0a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.451+0000 7f83c8f70640 1 -- 172.21.15.1:0/2324167923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55835a0a71d0 con 0x55835a0d2690 2024-05-09T06:36:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83c8f70640 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f83b4059a50 tx=0x7f83b4078000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc792f039e19c3bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ae7fc640 1 -- 172.21.15.1:0/2324167923 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4021050 con 0x55835a0d2690 2024-05-09T06:36:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ae7fc640 1 -- 172.21.15.1:0/2324167923 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4078210 con 0x55835a0d2690 2024-05-09T06:36:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ae7fc640 1 -- 172.21.15.1:0/2324167923 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40585f0 con 0x55835a0d2690 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2324167923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 msgr2=0x55835a0e1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f83b4059a50 tx=0x7f83b4078000 comp rx=0 tx=0).stop 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2324167923 shutdown_connections 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d2690 0x55835a0e1230 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2324167923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d02a0 0x55835a09ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2324167923 >> 172.21.15.1:0/2324167923 conn(0x55835a0cfb60 msgr2=0x55835a0eca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2324167923 shutdown_connections 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.452+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2324167923 wait complete. 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.453+0000 7f83ce7c4740 1 -- start start 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.453+0000 7f83ce7c4740 1 Processor -- start 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55835a0bdd50 con 0x55835a0d02a0 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55835a0e2f70 con 0x55835a0ed180 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55835a0be2e0 con 0x55835a0d2690 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83ce7c4740 10 client.0 _ll_register_callbacks cb 0x7f83b408b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43784/0 (socket says 172.21.15.1:43784) 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 -- 172.21.15.1:0/2060932210 learned_addr learned my addr 172.21.15.1:0/2060932210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 -- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 msgr2=0x55835a0e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 -- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 msgr2=0x55835a0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 -- 172.21.15.1:0/2060932210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55835a0bd6b0 con 0x55835a0d02a0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83c9771640 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83c002a2b0 tx=0x7f83c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347625059f193d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.454+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c004c040 con 0x55835a0d02a0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f83c0032d90 con 0x55835a0d02a0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b4078210 con 0x55835a0d02a0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f83c0037040 con 0x55835a0d02a0 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 10 client.8670 fetch_fsmap learned FSMap version 74 2024-05-09T06:36:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55835a0f9760 con 0x55835a0d02a0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4078210 con 0x55835a0d02a0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f83c0046070 con 0x55835a0d02a0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 10 client.8670 fetch_fsmap finished waiting for FSMap version 74 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 10 client.8670 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 20 client.8670 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0cb0d0 0x55835a0dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 4 client.8670 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 --> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55835a0104b0 con 0x55835a0cb0d0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83c8f70640 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0cb0d0 0x55835a0dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83a3fff640 10 client.8670 ms_handle_connect on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.455+0000 7f83c8f70640 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0cb0d0 0x55835a0dffb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e660d36c27f7a5 server_cookie=dafc203b516334d0 in_seq=0 out_seq=0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83a3fff640 1 -- 172.21.15.1:0/2060932210 <== mds.0 v2:172.21.15.1:6834/779210140 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55835a0104b0 con 0x55835a0cb0d0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83a3fff640 10 client.8670 handle_command_reply: tid=0 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 client.8670 shutdown 2024-05-09T06:36:17.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 msgr2=0x55835a0e1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f83c002a2b0 tx=0x7f83c00046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 shutdown_connections 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:6834/779210140,v1:172.21.15.1:6836/779210140] conn(0x55835a0cb0d0 0x55835a0dffb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55835a0ed180 0x55835a0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55835a0d2690 0x55835a0e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83a3fff640 0 client.8670 ms_handle_reset on v2:172.21.15.1:6834/779210140 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 --2- 172.21.15.1:0/2060932210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55835a0d02a0 0x55835a0e1230 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 >> 172.21.15.1:0/2060932210 conn(0x55835a0cfb60 msgr2=0x55835a0a4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 shutdown_connections 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 1 -- 172.21.15.1:0/2060932210 wait complete. 2024-05-09T06:36:17.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83ce7c4740 20 client.8670 trim_cache size 0 max 16384 2024-05-09T06:36:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:17.443 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 msgr2=0x7f83a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 0x7f83a806e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f83c0034000 tx=0x7f83c002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 msgr2=0x7f83c41508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.456+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ee70 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 shutdown_connections 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f83c415a4d0 0x7f83c414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c40cdb90 0x7f83c41508e0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 --2- 172.21.15.1:0/1446675872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83c4071440 0x7f83c40a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 >> 172.21.15.1:0/1446675872 conn(0x7f83c40cc4d0 msgr2=0x7f83c40af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 shutdown_connections 2024-05-09T06:36:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.457+0000 7f83cb9d5640 1 -- 172.21.15.1:0/1446675872 wait complete. 2024-05-09T06:36:17.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:36:17.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.570+0000 7f4c24a10640 1 Processor -- start 2024-05-09T06:36:17.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.570+0000 7f4c24a10640 1 -- start start 2024-05-09T06:36:17.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c24a10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c2006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c24a10640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c200ce0e0 con 0x7f4c200b3300 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c24a10640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c200ce3c0 con 0x7f4c200cf180 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c24a10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c200ce6a0 con 0x7f4c2006e6d0 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c1ed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4c200b3300 0x7f4c200b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51922/0 (socket says 172.21.15.1:51922) 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c1ed9d640 1 -- 172.21.15.1:0/3773601714 learned_addr learned my addr 172.21.15.1:0/3773601714 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c2006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955003511 0 0) 0x7f4c200ce0e0 con 0x7f4c200b3300 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.571+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c0c005730 con 0x7f4c200b3300 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2752799068 0 0) 0x7f4c0c005730 con 0x7f4c200b3300 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 msgr2=0x7f4c2006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c2006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 >> v1:172.21.15.6:6789/0 conn(0x7f4c200cf180 legacy=0x7f4c200cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c200ce980 con 0x7f4c200b3300 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f4c180019f0 con 0x7f4c200b3300 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 >> v1:172.21.15.1:6789/0 conn(0x7f4c200b3300 legacy=0x7f4c200b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c0c005bf0 0x7f4c0c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c0c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c006f50 0x7f4c0c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200ce6a0 con 0x7f4c0c005bf0 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c003520 con 0x7f4c0c006f50 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c003720 con 0x7f4c2006e6d0 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c0c005bf0 0x7f4c0c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1ed9d640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c006f50 0x7f4c0c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c15d9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c0c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 -- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 msgr2=0x7f4c0c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c0c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 -- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c006f50 msgr2=0x7f4c0c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c0c006f50 0x7f4c0c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c1e59c640 1 -- 172.21.15.1:0/3773601714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c005730 con 0x7f4c0c005bf0 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.572+0000 7f4c15d9b640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c2006e6d0 0x7f4c0c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.573+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c0c005bf0 0x7f4c0c006010 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009a90 tx=0x7f4c0c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4db8d80a0e375f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.573+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10036070 con 0x7f4c0c005bf0 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.573+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10031400 con 0x7f4c0c005bf0 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.573+0000 7f4c1dd9b640 1 -- 172.21.15.1:0/3773601714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10009690 con 0x7f4c0c005bf0 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 -- 172.21.15.1:0/3773601714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c0c005bf0 msgr2=0x7f4c0c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 --2- 172.21.15.1:0/3773601714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c0c005bf0 0x7f4c0c006010 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009a90 tx=0x7f4c0c007a40 comp rx=0 tx=0).stop 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c1ed9d640 1 -- 172.21.15.1:0/3773601714 reap_dead start 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 -- 172.21.15.1:0/3773601714 shutdown_connections 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 -- 172.21.15.1:0/3773601714 >> 172.21.15.1:0/3773601714 conn(0x7f4c200cc4b0 msgr2=0x7f4c200cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 -- 172.21.15.1:0/3773601714 shutdown_connections 2024-05-09T06:36:17.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.574+0000 7f4c24a10640 1 -- 172.21.15.1:0/3773601714 wait complete. 2024-05-09T06:36:17.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.575+0000 7f4c24a10640 1 Processor -- start 2024-05-09T06:36:17.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.575+0000 7f4c24a10640 1 -- start start 2024-05-09T06:36:17.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c0c005bf0 0x7f4c20134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 0x7f4c2014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c201447c0 con 0x7f4c201365d0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20070ed0 con 0x7f4c2015a3e0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c24a10640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4c20144940 con 0x7f4c0c005bf0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43808/0 (socket says 172.21.15.1:43808) 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 -- 172.21.15.1:0/1355150411 learned_addr learned my addr 172.21.15.1:0/1355150411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c1ed9d640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 0x7f4c2014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c0c005bf0 0x7f4c20134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 -- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c0c005bf0 msgr2=0x7f4c20134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c0c005bf0 0x7f4c20134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 -- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 msgr2=0x7f4c2014ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 0x7f4c2014ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.576+0000 7f4c15d9b640 1 -- 172.21.15.1:0/1355150411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2015ac30 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c1ed9d640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 0x7f4c2014ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c15d9b640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e550 tx=0x7f4c0800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080052e0 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08018070 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08003830 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c20135530 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.577+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c20135af0 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.578+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c200722a0 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.578+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c080039d0 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.579+0000 7f4c177fe640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 0x7f4c0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.579+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f4c0808ae00 con 0x7f4c201365d0 2024-05-09T06:36:17.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.579+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 0x7f4c0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.579+0000 7f4c1e59c640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 0x7f4c0006e320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4c10033000 tx=0x7f4c10008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.584+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c200722a0 con 0x7f4c201365d0 2024-05-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.804+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4c200752b0 con 0x7f4c201365d0 2024-05-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.804+0000 7f4c177fe640 1 -- 172.21.15.1:0/1355150411 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4c08059500 con 0x7f4c201365d0 2024-05-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 msgr2=0x7f4c0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 0x7f4c0006e320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4c10033000 tx=0x7f4c10008030 comp rx=0 tx=0).stop 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 msgr2=0x7f4c20139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e550 tx=0x7f4c0800ea10 comp rx=0 tx=0).stop 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 shutdown_connections 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4c0006be60 0x7f4c0006e320 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2015a3e0 0x7f4c2014ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c201365d0 0x7f4c20139f90 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 --2- 172.21.15.1:0/1355150411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c0c005bf0 0x7f4c20134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 >> 172.21.15.1:0/1355150411 conn(0x7f4c200cc4b0 msgr2=0x7f4c2006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 shutdown_connections 2024-05-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:17.806+0000 7f4c24a10640 1 -- 172.21.15.1:0/1355150411 wait complete. 2024-05-09T06:36:17.798 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-09T06:36:17.798 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:36:17.798 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:17.844 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:17.892 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:36:17.893 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:36:17.893 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:17.893 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:17.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:18.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.072+0000 7f3268e12640 1 Processor -- start 2024-05-09T06:36:18.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.072+0000 7f3268e12640 1 -- start start 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3268e12640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f32640cf180 0x7f32640cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3268e12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cdab0 0x7f32640b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3268e12640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32640ce0e0 con 0x7f326406e6d0 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3268e12640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f32640ce3c0 con 0x7f32640cf180 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3268e12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f32640ce6a0 con 0x7f32640cdab0 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f325bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f32640cf180 0x7f32640cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f326259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f326406e6d0 0x7f326406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55778/0 (socket says 172.21.15.1:55778) 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f326259c640 1 -- 172.21.15.1:0/3913999208 learned_addr learned my addr 172.21.15.1:0/3913999208 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f3262d9d640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cdab0 0x7f32640b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f325bfff640 1 -- 172.21.15.1:0/3913999208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cdab0 msgr2=0x7f32640b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f325bfff640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cdab0 0x7f32640b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f325bfff640 1 -- 172.21.15.1:0/3913999208 >> v1:172.21.15.1:6789/0 conn(0x7f326406e6d0 legacy=0x7f326406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.073+0000 7f325bfff640 1 -- 172.21.15.1:0/3913999208 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32640ce980 con 0x7f32640cf180 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.074+0000 7f325bfff640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cf180 0x7f32640cd570 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f324c0097a0 tx=0x7f324c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3b98bb3e64316a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.074+0000 7f3261d9b640 1 -- 172.21.15.1:0/3913999208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c037070 con 0x7f32640cf180 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.074+0000 7f3261d9b640 1 -- 172.21.15.1:0/3913999208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c032440 con 0x7f32640cf180 2024-05-09T06:36:18.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.074+0000 7f3261d9b640 1 -- 172.21.15.1:0/3913999208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c032810 con 0x7f32640cf180 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 -- 172.21.15.1:0/3913999208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cf180 msgr2=0x7f32640cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cf180 0x7f32640cd570 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f324c0097a0 tx=0x7f324c02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 -- 172.21.15.1:0/3913999208 shutdown_connections 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cdab0 0x7f32640b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 --2- 172.21.15.1:0/3913999208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cf180 0x7f32640cd570 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 -- 172.21.15.1:0/3913999208 >> 172.21.15.1:0/3913999208 conn(0x7f32640cc4b0 msgr2=0x7f32640cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 -- 172.21.15.1:0/3913999208 shutdown_connections 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.075+0000 7f3268e12640 1 -- 172.21.15.1:0/3913999208 wait complete. 2024-05-09T06:36:18.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.076+0000 7f3268e12640 1 Processor -- start 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.076+0000 7f3268e12640 1 -- start start 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 0x7f326415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cf180 0x7f32640b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32641447c0 con 0x7f32640cdab0 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3264070ed0 con 0x7f326406e6d0 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3268e12640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3264144940 con 0x7f32640cf180 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55324/0 (socket says 172.21.15.1:55324) 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 -- 172.21.15.1:0/784108365 learned_addr learned my addr 172.21.15.1:0/784108365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f3262d9d640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cf180 0x7f32640b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f326259c640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 0x7f326415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 -- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cf180 msgr2=0x7f32640b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cf180 0x7f32640b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 -- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 msgr2=0x7f326415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 0x7f326415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f325bfff640 1 -- 172.21.15.1:0/784108365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326415ac30 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.077+0000 7f326259c640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 0x7f326415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f325bfff640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c037040 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c042070 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c030bd0 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32641f49b0 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.078+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32641f4f70 con 0x7f32640cdab0 2024-05-09T06:36:18.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.079+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32640752b0 con 0x7f32640cdab0 2024-05-09T06:36:18.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.079+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f324c047460 con 0x7f32640cdab0 2024-05-09T06:36:18.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.080+0000 7f325affd640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 0x7f324406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.080+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f324c0abcf0 con 0x7f32640cdab0 2024-05-09T06:36:18.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.080+0000 7f326259c640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 0x7f324406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.080+0000 7f326259c640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 0x7f324406e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3254000f60 tx=0x7f3254009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.084+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f324c0b0070 con 0x7f32640cdab0 2024-05-09T06:36:18.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.270+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3264143d30 con 0x7f32640cdab0 2024-05-09T06:36:18.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.272+0000 7f325affd640 1 -- 172.21.15.1:0/784108365 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17199 (secure 0 0 0) 0x7f324c032e20 con 0x7f32640cdab0 2024-05-09T06:36:18.252 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:18.253 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:39.452636+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.1:6834/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 msgr2=0x7f324406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 0x7f324406e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3254000f60 tx=0x7f3254009040 comp rx=0 tx=0).stop 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 msgr2=0x7f32641f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 shutdown_connections 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f324406be30 0x7f324406e2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32640cf180 0x7f32640b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32640cdab0 0x7f32641f4320 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 --2- 172.21.15.1:0/784108365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f326406e6d0 0x7f326415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 >> 172.21.15.1:0/784108365 conn(0x7f32640cc4b0 msgr2=0x7f326406eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 shutdown_connections 2024-05-09T06:36:18.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.274+0000 7f3268e12640 1 -- 172.21.15.1:0/784108365 wait complete. 2024-05-09T06:36:18.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:18.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:18.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.393+0000 7fafd7770640 1 Processor -- start 2024-05-09T06:36:18.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.393+0000 7fafd7770640 1 -- start start 2024-05-09T06:36:18.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd7770640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fafd00d06f0 0x7fafd00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd7770640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fafd00c41e0 0x7fafd00ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd7770640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd0072050 con 0x7fafd006b1e0 2024-05-09T06:36:18.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd7770640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fafd00cd160 con 0x7fafd00c41e0 2024-05-09T06:36:18.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd7770640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fafd0072330 con 0x7fafd00d06f0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd4d0b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fafd00c41e0 0x7fafd00ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd4d0b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fafd00c41e0 0x7fafd00ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55328/0 (socket says 172.21.15.1:55328) 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.394+0000 7fafd4d0b640 1 -- 172.21.15.1:0/2301439459 learned_addr learned my addr 172.21.15.1:0/2301439459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd550c640 1 --2- 172.21.15.1:0/2301439459 >> v2:172.21.15.6:3300/0 conn(0x7fafd00d06f0 0x7fafd00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd4d0b640 1 -- 172.21.15.1:0/2301439459 >> v2:172.21.15.6:3300/0 conn(0x7fafd00d06f0 msgr2=0x7fafd00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd4d0b640 1 --2- 172.21.15.1:0/2301439459 >> v2:172.21.15.6:3300/0 conn(0x7fafd00d06f0 0x7fafd00cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd4d0b640 1 -- 172.21.15.1:0/2301439459 >> v1:172.21.15.6:6789/0 conn(0x7fafd006b1e0 legacy=0x7fafd0161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd4d0b640 1 -- 172.21.15.1:0/2301439459 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd0072610 con 0x7fafd00c41e0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.395+0000 7fafd4d0b640 1 --2- 172.21.15.1:0/2301439459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00ccc20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fafd004ff70 tx=0x7fafc402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30363ed331d4d6be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafcffff640 1 -- 172.21.15.1:0/2301439459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4036070 con 0x7fafd00c41e0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafcffff640 1 -- 172.21.15.1:0/2301439459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4031600 con 0x7fafd00c41e0 2024-05-09T06:36:18.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafcffff640 1 -- 172.21.15.1:0/2301439459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40319d0 con 0x7fafd00c41e0 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 -- 172.21.15.1:0/2301439459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 msgr2=0x7fafd00ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 --2- 172.21.15.1:0/2301439459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00ccc20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fafd004ff70 tx=0x7fafc402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 -- 172.21.15.1:0/2301439459 shutdown_connections 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 --2- 172.21.15.1:0/2301439459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00ccc20 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 --2- 172.21.15.1:0/2301439459 >> v2:172.21.15.6:3300/0 conn(0x7fafd00d06f0 0x7fafd00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 -- 172.21.15.1:0/2301439459 >> 172.21.15.1:0/2301439459 conn(0x7fafd00d0b50 msgr2=0x7fafd00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 -- 172.21.15.1:0/2301439459 shutdown_connections 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.396+0000 7fafd7770640 1 -- 172.21.15.1:0/2301439459 wait complete. 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.397+0000 7fafd7770640 1 Processor -- start 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.397+0000 7fafd7770640 1 -- start start 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd006b1e0 0x7fafd00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 0x7fafd00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00cf8e0 con 0x7fafd00c41e0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0071ce0 con 0x7fafd006b1e0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd7770640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00cfa60 con 0x7fafd00d06f0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd4d0b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd4d0b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55340/0 (socket says 172.21.15.1:55340) 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd4d0b640 1 -- 172.21.15.1:0/3475486290 learned_addr learned my addr 172.21.15.1:0/3475486290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd550c640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd006b1e0 0x7fafd00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.398+0000 7fafd5d0d640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 0x7fafd00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 -- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 msgr2=0x7fafd00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 0x7fafd00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 -- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd006b1e0 msgr2=0x7fafd00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd006b1e0 0x7fafd00b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 -- 172.21.15.1:0/3475486290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd00c9ae0 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd5d0d640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 0x7fafd00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd4d0b640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fafc4034040 tx=0x7fafc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4036050 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4042070 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4009510 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd00c9dc0 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.399+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd00ca380 con 0x7fafd00c41e0 2024-05-09T06:36:18.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.400+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd00752b0 con 0x7fafd00c41e0 2024-05-09T06:36:18.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.401+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafc4007c50 con 0x7fafd00c41e0 2024-05-09T06:36:18.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.401+0000 7fafcdffb640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.401+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fafc40abe70 con 0x7fafd00c41e0 2024-05-09T06:36:18.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.401+0000 7fafd550c640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.402+0000 7fafd550c640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 0x7fafbc06e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fafc00079a0 tx=0x7fafc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.406+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fafc407a600 con 0x7fafd00c41e0 2024-05-09T06:36:18.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.587+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafd00cc6a0 con 0x7fafd00c41e0 2024-05-09T06:36:18.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.587+0000 7fafcdffb640 1 -- 172.21.15.1:0/3475486290 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fafc4079f00 con 0x7fafd00c41e0 2024-05-09T06:36:18.568 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 msgr2=0x7fafbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 0x7fafbc06e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fafc00079a0 tx=0x7fafc0008040 comp rx=0 tx=0).stop 2024-05-09T06:36:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 msgr2=0x7fafd00c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fafc4034040 tx=0x7fafc4004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 shutdown_connections 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafd00d06f0 0x7fafd00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafd00c41e0 0x7fafd00c8f40 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 --2- 172.21.15.1:0/3475486290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafd006b1e0 0x7fafd00b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 >> 172.21.15.1:0/3475486290 conn(0x7fafd00d0b50 msgr2=0x7fafd015c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 shutdown_connections 2024-05-09T06:36:18.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.589+0000 7fafd7770640 1 -- 172.21.15.1:0/3475486290 wait complete. 2024-05-09T06:36:18.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:18.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 Processor -- start 2024-05-09T06:36:18.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 -- start start 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f719815e190 0x7f719815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f71980d1c70 0x7f71980d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f71980b56d0 0x7f71980d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7198161fe0 con 0x7f71980d1c70 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f71981622c0 con 0x7f719815e190 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.708+0000 7f719ecbd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f71981625a0 con 0x7f71980b56d0 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f71980d1c70 0x7f71980d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f71980d1c70 0x7f71980d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55350/0 (socket says 172.21.15.1:55350) 2024-05-09T06:36:18.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 -- 172.21.15.1:0/4155923032 learned_addr learned my addr 172.21.15.1:0/4155923032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f719d25a640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f71980b56d0 0x7f71980d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f719ca59640 1 --2- 172.21.15.1:0/4155923032 >> v2:172.21.15.6:3300/0 conn(0x7f719815e190 0x7f719815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 -- 172.21.15.1:0/4155923032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f71980b56d0 msgr2=0x7f71980d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f71980b56d0 0x7f71980d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 -- 172.21.15.1:0/4155923032 >> v2:172.21.15.6:3300/0 conn(0x7f719815e190 msgr2=0x7f719815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 --2- 172.21.15.1:0/4155923032 >> v2:172.21.15.6:3300/0 conn(0x7f719815e190 0x7f719815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.709+0000 7f7197fff640 1 -- 172.21.15.1:0/4155923032 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7198162880 con 0x7f71980d1c70 2024-05-09T06:36:18.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f7197fff640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980d1c70 0x7f71980d3ee0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f71980d1bf0 tx=0x7f718802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a16e2c4a0ba59c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f71977fe640 1 -- 172.21.15.1:0/4155923032 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188037070 con 0x7f71980d1c70 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f71977fe640 1 -- 172.21.15.1:0/4155923032 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7188032440 con 0x7f71980d1c70 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f71977fe640 1 -- 172.21.15.1:0/4155923032 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188032810 con 0x7f71980d1c70 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 -- 172.21.15.1:0/4155923032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980d1c70 msgr2=0x7f71980d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980d1c70 0x7f71980d3ee0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f71980d1bf0 tx=0x7f718802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 -- 172.21.15.1:0/4155923032 shutdown_connections 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f71980b56d0 0x7f71980d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 --2- 172.21.15.1:0/4155923032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980d1c70 0x7f71980d3ee0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 --2- 172.21.15.1:0/4155923032 >> v2:172.21.15.6:3300/0 conn(0x7f719815e190 0x7f719815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 -- 172.21.15.1:0/4155923032 >> 172.21.15.1:0/4155923032 conn(0x7f7198073340 msgr2=0x7f719809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.710+0000 7f719ecbd640 1 -- 172.21.15.1:0/4155923032 shutdown_connections 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.711+0000 7f719ecbd640 1 -- 172.21.15.1:0/4155923032 wait complete. 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.712+0000 7f719ecbd640 1 Processor -- start 2024-05-09T06:36:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.712+0000 7f719ecbd640 1 -- start start 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 0x7f719814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f719815e190 0x7f719815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71980b3f60 con 0x7f71980b56d0 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f71980b3de0 con 0x7f71980d1c70 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f71980b40e0 con 0x7f719815e190 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55362/0 (socket says 172.21.15.1:55362) 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 -- 172.21.15.1:0/1266367032 learned_addr learned my addr 172.21.15.1:0/1266367032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f7197fff640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 0x7f719814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719d25a640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f719815e190 0x7f719815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 -- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f719815e190 msgr2=0x7f719815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f719815e190 0x7f719815b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 -- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 msgr2=0x7f719814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 0x7f719814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 -- 172.21.15.1:0/1266367032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719814f140 con 0x7f71980b56d0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f7197fff640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 0x7f719814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ca59640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7190007e70 tx=0x7f719000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190014620 con 0x7f71980b56d0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7190019070 con 0x7f71980b56d0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719815bac0 con 0x7f71980b56d0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190014a00 con 0x7f71980b56d0 2024-05-09T06:36:18.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.713+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71980cc260 con 0x7f71980b56d0 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.714+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71980752b0 con 0x7f71980b56d0 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.715+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7190014ba0 con 0x7f71980b56d0 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.715+0000 7f71957fa640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 0x7f717806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.715+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f719008b940 con 0x7f71980b56d0 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.715+0000 7f7197fff640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 0x7f717806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.716+0000 7f7197fff640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 0x7f717806e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7188034000 tx=0x7f718802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.720+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f719815bac0 con 0x7f71980b56d0 2024-05-09T06:36:18.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:18.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.904+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7198161440 con 0x7f71980b56d0 2024-05-09T06:36:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.904+0000 7f71957fa640 1 -- 172.21.15.1:0/1266367032 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f719005a040 con 0x7f71980b56d0 2024-05-09T06:36:18.885 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 msgr2=0x7f717806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 0x7f717806e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7188034000 tx=0x7f718802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 msgr2=0x7f719815abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7190007e70 tx=0x7f719000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 shutdown_connections 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f717806be30 0x7f717806e2f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f719815e190 0x7f719815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71980d1c70 0x7f719814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 --2- 172.21.15.1:0/1266367032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71980b56d0 0x7f719815abb0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 >> 172.21.15.1:0/1266367032 conn(0x7f7198073340 msgr2=0x7f719806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 shutdown_connections 2024-05-09T06:36:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:18.907+0000 7f719ecbd640 1 -- 172.21.15.1:0/1266367032 wait complete. 2024-05-09T06:36:18.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:19.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.024+0000 7fdbe3f22640 1 Processor -- start 2024-05-09T06:36:19.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.025+0000 7fdbe3f22640 1 -- start start 2024-05-09T06:36:19.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.025+0000 7fdbe3f22640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbdc071610 0x7fdbdc0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe3f22640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc0cf1b0 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe3f22640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc0cf490 con 0x7fdbdc0cdb90 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe3f22640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc15a8b0 con 0x7fdbdc071610 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe14bd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbdc071610 0x7fdbdc0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe1cbe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdbdc15a4d0 0x7fdbdc070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55784/0 (socket says 172.21.15.1:55784) 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe14bd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbdc071610 0x7fdbdc0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55366/0 (socket says 172.21.15.1:55366) 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe1cbe640 1 -- 172.21.15.1:0/3794345174 learned_addr learned my addr 172.21.15.1:0/3794345174 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe14bd640 1 -- 172.21.15.1:0/3794345174 >> v1:172.21.15.6:6789/0 conn(0x7fdbdc0cdb90 legacy=0x7fdbdc06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe14bd640 1 -- 172.21.15.1:0/3794345174 >> v1:172.21.15.1:6789/0 conn(0x7fdbdc15a4d0 legacy=0x7fdbdc070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe14bd640 1 -- 172.21.15.1:0/3794345174 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc06d8a0 con 0x7fdbdc071610 2024-05-09T06:36:19.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.026+0000 7fdbe0cbc640 1 -- 172.21.15.1:0/3794345174 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013975955 0 0) 0x7fdbdc0cf1b0 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe14bd640 1 --2- 172.21.15.1:0/3794345174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc071610 0x7fdbdc0cea80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0097a0 tx=0x7fdbcc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde7ff2c9cd7534 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe0cbc640 1 -- 172.21.15.1:0/3794345174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037070 con 0x7fdbdc071610 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe0cbc640 1 -- 172.21.15.1:0/3794345174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc02ae50 con 0x7fdbdc071610 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe0cbc640 1 -- 172.21.15.1:0/3794345174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc032660 con 0x7fdbdc071610 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3794345174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc071610 msgr2=0x7fdbdc0cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3794345174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc071610 0x7fdbdc0cea80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0097a0 tx=0x7fdbcc02a140 comp rx=0 tx=0).stop 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3794345174 shutdown_connections 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3794345174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc071610 0x7fdbdc0cea80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.027+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3794345174 >> 172.21.15.1:0/3794345174 conn(0x7fdbdc0cc4d0 msgr2=0x7fdbdc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.028+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3794345174 shutdown_connections 2024-05-09T06:36:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.028+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3794345174 wait complete. 2024-05-09T06:36:19.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 Processor -- start 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 -- start start 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbdc071610 0x7fdbdc14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbdc0cdb90 0x7fdbdc15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc06e7b0 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc06dc10 con 0x7fdbdc071610 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.029+0000 7fdbe3f22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc06e930 con 0x7fdbdc0cdb90 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55376/0 (socket says 172.21.15.1:55376) 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 -- 172.21.15.1:0/3499870622 learned_addr learned my addr 172.21.15.1:0/3499870622 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe14bd640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbdc0cdb90 0x7fdbdc15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe1cbe640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbdc071610 0x7fdbdc14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 -- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbdc0cdb90 msgr2=0x7fdbdc15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbdc0cdb90 0x7fdbdc15d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 -- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbdc071610 msgr2=0x7fdbdc14e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbdc071610 0x7fdbdc14e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 -- 172.21.15.1:0/3499870622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc15d950 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe24bf640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400c7b0 tx=0x7fdbd400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4004b60 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd4004d00 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc14d420 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400aa90 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.030+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc14d9e0 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.031+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc14ea00 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.032+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbd400ac30 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.032+0000 7fdbd27fc640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 0x7fdbc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.032+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fdbd408b420 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.033+0000 7fdbe1cbe640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 0x7fdbc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.033+0000 7fdbe1cbe640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 0x7fdbc006e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.037+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbd4059560 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.218+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbdc072120 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.218+0000 7fdbd27fc640 1 -- 172.21.15.1:0/3499870622 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fdbd405d860 con 0x7fdbdc15a4d0 2024-05-09T06:36:19.199 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 msgr2=0x7fdbc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 0x7fdbc006e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8007de0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 msgr2=0x7fdbdc14c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400c7b0 tx=0x7fdbd400cc70 comp rx=0 tx=0).stop 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 shutdown_connections 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdbc006be30 0x7fdbc006e2f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbdc15a4d0 0x7fdbdc14c7b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdbdc0cdb90 0x7fdbdc15d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 --2- 172.21.15.1:0/3499870622 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbdc071610 0x7fdbdc14e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 >> 172.21.15.1:0/3499870622 conn(0x7fdbdc0cc4d0 msgr2=0x7fdbdc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 shutdown_connections 2024-05-09T06:36:19.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.220+0000 7fdbe3f22640 1 -- 172.21.15.1:0/3499870622 wait complete. 2024-05-09T06:36:19.212 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:19.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.337+0000 7f7a09227640 1 Processor -- start 2024-05-09T06:36:19.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.337+0000 7f7a09227640 1 -- start start 2024-05-09T06:36:19.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a09227640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 0x7f7a0406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a09227640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0406ce40 con 0x7f7a040cedd0 2024-05-09T06:36:19.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a09227640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0406d120 con 0x7f7a0406c960 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a09227640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0406d400 con 0x7f7a040b10f0 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0359e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7a040cedd0 0x7f7a040cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55790/0 (socket says 172.21.15.1:55790) 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0359e640 1 -- 172.21.15.1:0/3287805686 learned_addr learned my addr 172.21.15.1:0/3287805686 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a02d9d640 1 --2- 172.21.15.1:0/3287805686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 0x7f7a0406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291017229 0 0) 0x7f7a0406ce40 con 0x7f7a040cedd0 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f0003520 con 0x7f7a040cedd0 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421054071 0 0) 0x7f7a0406d120 con 0x7f7a0406c960 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.338+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a0406ce40 con 0x7f7a0406c960 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.339+0000 7f7a02d9d640 1 -- 172.21.15.1:0/3287805686 >> v1:172.21.15.6:6789/0 conn(0x7f7a0406c960 legacy=0x7f7a040ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.339+0000 7f7a02d9d640 1 -- 172.21.15.1:0/3287805686 >> v1:172.21.15.1:6789/0 conn(0x7f7a040cedd0 legacy=0x7f7a040cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.339+0000 7f7a02d9d640 1 -- 172.21.15.1:0/3287805686 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0406d6e0 con 0x7f7a040b10f0 2024-05-09T06:36:19.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.339+0000 7f7a02d9d640 1 --2- 172.21.15.1:0/3287805686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 0x7f7a0406fb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f79f40097d0 tx=0x7f79f402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5ad43e0adfc492 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4037070 con 0x7f7a040b10f0 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f4032440 con 0x7f7a040b10f0 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a0259c640 1 -- 172.21.15.1:0/3287805686 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4032810 con 0x7f7a040b10f0 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 -- 172.21.15.1:0/3287805686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 msgr2=0x7f7a0406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 --2- 172.21.15.1:0/3287805686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 0x7f7a0406fb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f79f40097d0 tx=0x7f79f402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 -- 172.21.15.1:0/3287805686 shutdown_connections 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 --2- 172.21.15.1:0/3287805686 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040b10f0 0x7f7a0406fb10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 -- 172.21.15.1:0/3287805686 >> 172.21.15.1:0/3287805686 conn(0x7f7a040b37c0 msgr2=0x7f7a040b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 -- 172.21.15.1:0/3287805686 shutdown_connections 2024-05-09T06:36:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.340+0000 7f7a09227640 1 -- 172.21.15.1:0/3287805686 wait complete. 2024-05-09T06:36:19.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.342+0000 7f7a09227640 1 Processor -- start 2024-05-09T06:36:19.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.342+0000 7f7a09227640 1 -- start start 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a0406c960 0x7f7a040c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040cedd0 0x7f7a040d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a04070050 con 0x7f7a040b10f0 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7a040b14d0 con 0x7f7a0406c960 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a09227640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a040701d0 con 0x7f7a040cedd0 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55382/0 (socket says 172.21.15.1:55382) 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 -- 172.21.15.1:0/3044866137 learned_addr learned my addr 172.21.15.1:0/3044866137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a0359e640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040cedd0 0x7f7a040d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f7a02d9d640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a0406c960 0x7f7a040c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 -- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040cedd0 msgr2=0x7f7a040d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040cedd0 0x7f7a040d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 -- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a0406c960 msgr2=0x7f7a040c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a0406c960 0x7f7a040c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 -- 172.21.15.1:0/3044866137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a040c5dc0 con 0x7f7a040b10f0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.343+0000 7f79fbfff640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c430 tx=0x7f79ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.344+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec017070 con 0x7f7a040b10f0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.344+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec0052f0 con 0x7f7a040b10f0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.344+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a040c60a0 con 0x7f7a040b10f0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.344+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec003830 con 0x7f7a040b10f0 2024-05-09T06:36:19.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.344+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a040c6660 con 0x7f7a040b10f0 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.345+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a04073f40 con 0x7f7a040b10f0 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.345+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79ec01a070 con 0x7f7a040b10f0 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.346+0000 7f79fb7fe640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 0x7f79e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.346+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f79ec08a480 con 0x7f7a040b10f0 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.346+0000 7f7a02d9d640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 0x7f79e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.346+0000 7f7a02d9d640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 0x7f79e406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f79f4034000 tx=0x7f79f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.350+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a04073f40 con 0x7f7a040b10f0 2024-05-09T06:36:19.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.549+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a040752b0 con 0x7f7a040b10f0 2024-05-09T06:36:19.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.550+0000 7f79fb7fe640 1 -- 172.21.15.1:0/3044866137 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v95) ==== 80+0+238 (secure 0 0 0) 0x7f79ec0585c0 con 0x7f7a040b10f0 2024-05-09T06:36:19.531 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6836/424906117 2024-05-10T06:35:39.452614+0000 2024-05-09T06:36:19.531 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6834/424906117 2024-05-10T06:35:39.452614+0000 2024-05-09T06:36:19.531 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6835/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:36:19.531 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.6:6833/1974102470 2024-05-10T06:34:19.713641+0000 2024-05-09T06:36:19.531 INFO:teuthology.orchestra.run.smithi001.stderr:listed 4 entries 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 msgr2=0x7f79e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 0x7f79e406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f79f4034000 tx=0x7f79f402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 msgr2=0x7f7a04156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c430 tx=0x7f79ec00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 shutdown_connections 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f79e406be30 0x7f79e406e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a040cedd0 0x7f7a040d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a040b10f0 0x7f7a04156270 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 --2- 172.21.15.1:0/3044866137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a0406c960 0x7f7a040c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 >> 172.21.15.1:0/3044866137 conn(0x7f7a040b37c0 msgr2=0x7f7a040ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 shutdown_connections 2024-05-09T06:36:19.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:19.552+0000 7f7a09227640 1 -- 172.21.15.1:0/3044866137 wait complete. 2024-05-09T06:36:19.544 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:36:19.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:19.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:19.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[71881]: fuse finished with error 0 and tester_r 0 2024-05-09T06:36:19.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:36:19.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:19.571 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:36:19.669 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:36:19.669 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:36:19.669 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:19.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:36:19.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:36:19.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:19.689 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:36:19.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:19.749 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:36:19.750 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:36:19.750 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:19.750 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:19.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:36:19.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:36:19.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:19.767 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:36:19.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:19.824 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:36:19.824 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:36:19.825 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:36:19.825 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:19.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:36:19.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:36:19.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:19.845 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:36:19.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:19.905 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:36:19.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.028+0000 7fe6d4cd4640 1 Processor -- start 2024-05-09T06:36:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.028+0000 7fe6d4cd4640 1 -- start start 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe6d01454b0 0x7fe6d00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b2a60 0x7fe6d00b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6d00b4da0 0x7fe6d015f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d00b5230 con 0x7fe6d00b4da0 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d006eb40 con 0x7fe6d01454b0 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.029+0000 7fe6d4cd4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d006ee20 con 0x7fe6d00b2a60 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6d00b4da0 0x7fe6d015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe6d00b4da0 0x7fe6d015f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55396/0 (socket says 172.21.15.1:55396) 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 -- 172.21.15.1:0/937855582 learned_addr learned my addr 172.21.15.1:0/937855582 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 -- 172.21.15.1:0/937855582 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b2a60 msgr2=0x7fe6d00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6cdd9b640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b2a60 0x7fe6d00b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ce59c640 1 --2- 172.21.15.1:0/937855582 >> v2:172.21.15.6:3300/0 conn(0x7fe6d01454b0 0x7fe6d00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b2a60 0x7fe6d00b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 -- 172.21.15.1:0/937855582 >> v2:172.21.15.6:3300/0 conn(0x7fe6d01454b0 msgr2=0x7fe6d00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/937855582 >> v2:172.21.15.6:3300/0 conn(0x7fe6d01454b0 0x7fe6d00b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.030+0000 7fe6ced9d640 1 -- 172.21.15.1:0/937855582 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d006d8e0 con 0x7fe6d00b4da0 2024-05-09T06:36:20.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d00b4da0 0x7fe6d015f960 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00689d0 tx=0x7fe6c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096db115db64ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6cd59a640 1 -- 172.21.15.1:0/937855582 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8036070 con 0x7fe6d00b4da0 2024-05-09T06:36:20.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6cd59a640 1 -- 172.21.15.1:0/937855582 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c8031600 con 0x7fe6d00b4da0 2024-05-09T06:36:20.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6cd59a640 1 -- 172.21.15.1:0/937855582 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80319d0 con 0x7fe6d00b4da0 2024-05-09T06:36:20.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/937855582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d00b4da0 msgr2=0x7fe6d015f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d00b4da0 0x7fe6d015f960 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00689d0 tx=0x7fe6c802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/937855582 shutdown_connections 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d00b4da0 0x7fe6d015f960 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/937855582 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b2a60 0x7fe6d00b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/937855582 >> v2:172.21.15.6:3300/0 conn(0x7fe6d01454b0 0x7fe6d00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/937855582 >> 172.21.15.1:0/937855582 conn(0x7fe6d00cc4b0 msgr2=0x7fe6d00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/937855582 shutdown_connections 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.031+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/937855582 wait complete. 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.033+0000 7fe6d4cd4640 1 Processor -- start 2024-05-09T06:36:20.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.033+0000 7fe6d4cd4640 1 -- start start 2024-05-09T06:36:20.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.033+0000 7fe6d4cd4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 0x7fe6d0152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b4da0 0x7fe6d013eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d006e0e0 con 0x7fe6d01454b0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0145890 con 0x7fe6d00b2a60 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d006e260 con 0x7fe6d00b4da0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55398/0 (socket says 172.21.15.1:55398) 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 -- 172.21.15.1:0/3647092452 learned_addr learned my addr 172.21.15.1:0/3647092452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6cdd9b640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b4da0 0x7fe6d013eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ce59c640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 0x7fe6d0152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 -- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b4da0 msgr2=0x7fe6d013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b4da0 0x7fe6d013eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 -- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 msgr2=0x7fe6d0152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 0x7fe6d0152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 -- 172.21.15.1:0/3647092452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d013f5e0 con 0x7fe6d01454b0 2024-05-09T06:36:20.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ce59c640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 0x7fe6d0152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6ced9d640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009f40 tx=0x7fe6c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8041020 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c8007780 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8036040 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d013f8c0 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.034+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d013fe80 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.035+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d00752b0 con 0x7fe6d01454b0 2024-05-09T06:36:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.036+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6c8044020 con 0x7fe6d01454b0 2024-05-09T06:36:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.036+0000 7fe6beffd640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.036+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fe6c80abf80 con 0x7fe6d01454b0 2024-05-09T06:36:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.036+0000 7fe6ce59c640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.037+0000 7fe6ce59c640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 0x7fe6b006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b840 tx=0x7fe6c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.041+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6c804b030 con 0x7fe6d01454b0 2024-05-09T06:36:20.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.265+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d00b5b00 con 0x7fe6d01454b0 2024-05-09T06:36:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.266+0000 7fe6beffd640 1 -- 172.21.15.1:0/3647092452 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7fe6c807a730 con 0x7fe6d01454b0 2024-05-09T06:36:20.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:20.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:36:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 msgr2=0x7fe6b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 0x7fe6b006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b840 tx=0x7fe6c0008040 comp rx=0 tx=0).stop 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 msgr2=0x7fe6d01f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009f40 tx=0x7fe6c8004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 shutdown_connections 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6d01454b0 0x7fe6d01f1950 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe6d00b4da0 0x7fe6d013eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 --2- 172.21.15.1:0/3647092452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6d00b2a60 0x7fe6d0152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 >> 172.21.15.1:0/3647092452 conn(0x7fe6d00cc4b0 msgr2=0x7fe6d0144270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 shutdown_connections 2024-05-09T06:36:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.268+0000 7fe6d4cd4640 1 -- 172.21.15.1:0/3647092452 wait complete. 2024-05-09T06:36:20.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:20.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.383+0000 7f698ab8b640 1 Processor -- start 2024-05-09T06:36:20.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.383+0000 7f698ab8b640 1 -- start start 2024-05-09T06:36:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f698ab8b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f69840b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f698ab8b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698406eb40 con 0x7f69840b4f70 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f698ab8b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698406ee20 con 0x7f69840b2a60 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f698ab8b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f698406d8e0 con 0x7f69841454b0 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f6989128640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69840b4f70 0x7f6984070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55800/0 (socket says 172.21.15.1:55800) 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.384+0000 7f6989128640 1 -- 172.21.15.1:0/3740264199 learned_addr learned my addr 172.21.15.1:0/3740264199 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6988927640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f69840b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4198488012 0 0) 0x7f698406eb40 con 0x7f69840b4f70 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6970003520 con 0x7f69840b4f70 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1056055346 0 0) 0x7f6970003520 con 0x7f69840b4f70 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 msgr2=0x7f69840b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f69840b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 >> v1:172.21.15.6:6789/0 conn(0x7f69840b2a60 legacy=0x7f69840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698406dbc0 con 0x7f69840b4f70 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f696c001670 con 0x7f69840b4f70 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 >> v1:172.21.15.1:6789/0 conn(0x7f69840b4f70 legacy=0x7f6984070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69700039e0 0x7f6970003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f6970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f698406d8e0 con 0x7f6970004d40 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f697000ba70 con 0x7f69700039e0 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f697000bc70 con 0x7f69841454b0 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6988927640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69700039e0 0x7f6970003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6983fff640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f6970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 -- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 msgr2=0x7f6970004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69841454b0 0x7f6970004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 -- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69700039e0 msgr2=0x7f6970003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69700039e0 0x7f6970003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.385+0000 7f6989128640 1 -- 172.21.15.1:0/3740264199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6970003520 con 0x7f6970004d40 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.386+0000 7f6989128640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69700051c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f69840b5400 tx=0x7f6970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7652cc66944c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.386+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007d10 con 0x7f6970004d40 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.386+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c005ad0 con 0x7f6970004d40 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.386+0000 7f69837fe640 1 -- 172.21.15.1:0/3740264199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007070 con 0x7f6970004d40 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 -- 172.21.15.1:0/3740264199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 msgr2=0x7f69700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 --2- 172.21.15.1:0/3740264199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69700051c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f69840b5400 tx=0x7f6970005830 comp rx=0 tx=0).stop 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f6989128640 1 -- 172.21.15.1:0/3740264199 reap_dead start 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 -- 172.21.15.1:0/3740264199 shutdown_connections 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 -- 172.21.15.1:0/3740264199 >> 172.21.15.1:0/3740264199 conn(0x7f69840cc4b0 msgr2=0x7f69840cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 -- 172.21.15.1:0/3740264199 shutdown_connections 2024-05-09T06:36:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.387+0000 7f698ab8b640 1 -- 172.21.15.1:0/3740264199 wait complete. 2024-05-09T06:36:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.388+0000 7f698ab8b640 1 Processor -- start 2024-05-09T06:36:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.388+0000 7f698ab8b640 1 -- start start 2024-05-09T06:36:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b5f0 0x7f69840aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415ba70 0x7f69840aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f698406e260 con 0x7f6970004d40 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f698406e0e0 con 0x7f698415b5f0 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f698ab8b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69840b4320 con 0x7f698415ba70 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55414/0 (socket says 172.21.15.1:55414) 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 -- 172.21.15.1:0/2080040096 learned_addr learned my addr 172.21.15.1:0/2080040096 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6983fff640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b5f0 0x7f69840aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6989128640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415ba70 0x7f69840aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 -- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415ba70 msgr2=0x7f69840aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415ba70 0x7f69840aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 -- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b5f0 msgr2=0x7f69840aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.389+0000 7f6988927640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b5f0 0x7f69840aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f6988927640 1 -- 172.21.15.1:0/2080040096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69841e37c0 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f6988927640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f697c0119a0 tx=0x7f697c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c00faf0 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c00fc90 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841e3aa0 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c014440 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.390+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841e4060 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.391+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69840752b0 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.391+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f697c012040 con 0x7f6970004d40 2024-05-09T06:36:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.392+0000 7f69817fa640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 0x7f696406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.392+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f697c08b670 con 0x7f6970004d40 2024-05-09T06:36:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.392+0000 7f6983fff640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 0x7f696406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.392+0000 7f6983fff640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 0x7f696406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69840b4e20 tx=0x7f6974009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.396+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f697c019030 con 0x7f6970004d40 2024-05-09T06:36:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.580+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6984072fa0 con 0x7f6970004d40 2024-05-09T06:36:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.581+0000 7f69817fa640 1 -- 172.21.15.1:0/2080040096 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17199 (secure 0 0 0) 0x7f697c0597b0 con 0x7f6970004d40 2024-05-09T06:36:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:20.562 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:39.452636+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.1:6834/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 msgr2=0x7f696406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 0x7f696406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69840b4e20 tx=0x7f6974009040 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 msgr2=0x7f69840ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f697c0119a0 tx=0x7f697c011e60 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 shutdown_connections 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f696406be30 0x7f696406e2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f698415ba70 0x7f69840aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f698415b5f0 0x7f69840aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 --2- 172.21.15.1:0/2080040096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6970004d40 0x7f69840ac510 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 >> 172.21.15.1:0/2080040096 conn(0x7f69840cc4b0 msgr2=0x7f69840b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 shutdown_connections 2024-05-09T06:36:20.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.583+0000 7f698ab8b640 1 -- 172.21.15.1:0/2080040096 wait complete. 2024-05-09T06:36:20.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:34:00.356458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:34:01.667377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:20.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:20.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.702+0000 7f042d012640 1 Processor -- start 2024-05-09T06:36:20.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.702+0000 7f042d012640 1 -- start start 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042d012640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04280d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042d012640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042806c420 con 0x7f042806ffb0 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042d012640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042806c700 con 0x7f04280ae9c0 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042d012640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f042806f3f0 con 0x7f042815e300 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042759e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f042806ffb0 0x7f042806d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55810/0 (socket says 172.21.15.1:55810) 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f042759e640 1 -- 172.21.15.1:0/1075122505 learned_addr learned my addr 172.21.15.1:0/1075122505 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.703+0000 7f0426d9d640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04280d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501432744 0 0) 0x7f042806c420 con 0x7f042806ffb0 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0414003520 con 0x7f042806ffb0 2024-05-09T06:36:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3203136523 0 0) 0x7f0414003520 con 0x7f042806ffb0 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 msgr2=0x7f04280d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04280d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 >> v1:172.21.15.6:6789/0 conn(0x7f04280ae9c0 legacy=0x7f042806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042806c9e0 con 0x7f042806ffb0 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f0420001670 con 0x7f042806ffb0 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 >> v1:172.21.15.1:6789/0 conn(0x7f042806ffb0 legacy=0x7f042806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0414003f20 0x7f0414004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414004d40 0x7f04140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f041400bb50 con 0x7f0414003f20 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f041400bd00 con 0x7f0414004d40 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f041400bf00 con 0x7f042815e300 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0414003f20 0x7f0414004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f0426d9d640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042759e640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414004d40 0x7f04140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 -- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 msgr2=0x7f04140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f042815e300 0x7f04140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 -- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414004d40 msgr2=0x7f04140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414004d40 0x7f04140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.704+0000 7f042659c640 1 -- 172.21.15.1:0/1075122505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414003520 con 0x7f0414003f20 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.705+0000 7f042659c640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0414003f20 0x7f0414004800 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f0410001910 tx=0x7f0414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20515289f4bedcb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.705+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410037070 con 0x7f0414003f20 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.705+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041002ac40 con 0x7f0414003f20 2024-05-09T06:36:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.705+0000 7f0425d9b640 1 -- 172.21.15.1:0/1075122505 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100014e0 con 0x7f0414003f20 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 -- 172.21.15.1:0/1075122505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0414003f20 msgr2=0x7f0414004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 --2- 172.21.15.1:0/1075122505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0414003f20 0x7f0414004800 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f0410001910 tx=0x7f0414005830 comp rx=0 tx=0).stop 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042759e640 1 -- 172.21.15.1:0/1075122505 reap_dead start 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 -- 172.21.15.1:0/1075122505 shutdown_connections 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 -- 172.21.15.1:0/1075122505 >> 172.21.15.1:0/1075122505 conn(0x7f04280d1c50 msgr2=0x7f04280d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 -- 172.21.15.1:0/1075122505 shutdown_connections 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.706+0000 7f042d012640 1 -- 172.21.15.1:0/1075122505 wait complete. 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.707+0000 7f042d012640 1 Processor -- start 2024-05-09T06:36:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.707+0000 7f042d012640 1 -- start start 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414003f20 0x7f042814b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0428157f70 0x7f04280c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f042806dbb0 con 0x7f04280c5cc0 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f042806cfa0 con 0x7f0414003f20 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042d012640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f042806dd30 con 0x7f0428157f70 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55446/0 (socket says 172.21.15.1:55446) 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 -- 172.21.15.1:0/1122790901 learned_addr learned my addr 172.21.15.1:0/1122790901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f0426d9d640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414003f20 0x7f042814b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042659c640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0428157f70 0x7f04280c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 -- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0428157f70 msgr2=0x7f04280c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0428157f70 0x7f04280c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 -- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414003f20 msgr2=0x7f042814b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414003f20 0x7f042814b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.708+0000 7f042759e640 1 -- 172.21.15.1:0/1122790901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04280c6140 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f042759e640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f04280aeea0 tx=0x7f0420007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042001a070 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042000f600 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042000fa10 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04280c6420 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.709+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04280c69e0 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.710+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04280752b0 con 0x7f04280c5cc0 2024-05-09T06:36:20.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.710+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f042001d070 con 0x7f04280c5cc0 2024-05-09T06:36:20.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.711+0000 7f041f7fe640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 0x7f040806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.711+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f042008a5b0 con 0x7f04280c5cc0 2024-05-09T06:36:20.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.711+0000 7f0426d9d640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 0x7f040806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.711+0000 7f0426d9d640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 0x7f040806e320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0418007440 tx=0x7f041800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.715+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0420058cb0 con 0x7f04280c5cc0 2024-05-09T06:36:20.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.900+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f042815b7f0 con 0x7f04280c5cc0 2024-05-09T06:36:20.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.901+0000 7f041f7fe640 1 -- 172.21.15.1:0/1122790901 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f04200585b0 con 0x7f04280c5cc0 2024-05-09T06:36:20.882 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 msgr2=0x7f040806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 0x7f040806e320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0418007440 tx=0x7f041800b040 comp rx=0 tx=0).stop 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 msgr2=0x7f04280af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f04280aeea0 tx=0x7f0420007430 comp rx=0 tx=0).stop 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 shutdown_connections 2024-05-09T06:36:20.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f040806be60 0x7f040806e320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04280c5cc0 0x7f04280af350 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0428157f70 0x7f04280c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 --2- 172.21.15.1:0/1122790901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0414003f20 0x7f042814b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 >> 172.21.15.1:0/1122790901 conn(0x7f04280d1c50 msgr2=0x7f042815bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 shutdown_connections 2024-05-09T06:36:20.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:20.903+0000 7f042d012640 1 -- 172.21.15.1:0/1122790901 wait complete. 2024-05-09T06:36:20.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:21.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.021+0000 7fdda6d63640 1 Processor -- start 2024-05-09T06:36:21.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.021+0000 7fdda6d63640 1 -- start start 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda6d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda6d63640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdda00cc8e0 0x7fdda00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda6d63640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda015e900 con 0x7fdda015e390 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda6d63640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00b1b90 con 0x7fdda00cc8e0 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda6d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdda015ebe0 con 0x7fdda00b11f0 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdda00cc8e0 0x7fdda00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdda00cc8e0 0x7fdda00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:37254/0 (socket says 172.21.15.1:37254) 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda4aff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdda015e390 0x7fdda00cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:55200/0 (socket says 172.21.15.1:55200) 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdd9ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:39486/0 (socket says 172.21.15.1:39486) 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 -- 172.21.15.1:0/3679081202 learned_addr learned my addr 172.21.15.1:0/3679081202 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 -- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 msgr2=0x7fdda00b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 -- 172.21.15.1:0/3679081202 >> v1:172.21.15.6:6789/0 conn(0x7fdda015e390 legacy=0x7fdda00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdda5300640 1 -- 172.21.15.1:0/3679081202 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda015eec0 con 0x7fdda00cc8e0 2024-05-09T06:36:21.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.022+0000 7fdd9ffff640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.023+0000 7fdda5300640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00ccd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009870 tx=0x7fdd8802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff90ad96e8e482 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.023+0000 7fdd9f7fe640 1 -- 172.21.15.1:0/3679081202 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88037070 con 0x7fdda00cc8e0 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.023+0000 7fdd9f7fe640 1 -- 172.21.15.1:0/3679081202 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd88032440 con 0x7fdda00cc8e0 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.023+0000 7fdd9f7fe640 1 -- 172.21.15.1:0/3679081202 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88032810 con 0x7fdda00cc8e0 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 -- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 msgr2=0x7fdda00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00ccd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009870 tx=0x7fdd8802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 -- 172.21.15.1:0/3679081202 shutdown_connections 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00ccd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3679081202 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda00b11f0 0x7fdda00b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 -- 172.21.15.1:0/3679081202 >> 172.21.15.1:0/3679081202 conn(0x7fdda00b1da0 msgr2=0x7fdda00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 -- 172.21.15.1:0/3679081202 shutdown_connections 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.024+0000 7fdda6d63640 1 -- 172.21.15.1:0/3679081202 wait complete. 2024-05-09T06:36:21.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.025+0000 7fdda6d63640 1 Processor -- start 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.025+0000 7fdda6d63640 1 -- start start 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda015e390 0x7fdda00b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0162ba0 con 0x7fdda00b11f0 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdda015f6c0 con 0x7fdda00cc8e0 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda6d63640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdda00cffb0 con 0x7fdda015e390 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda4aff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda4aff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55454/0 (socket says 172.21.15.1:55454) 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.026+0000 7fdda4aff640 1 -- 172.21.15.1:0/3762572091 learned_addr learned my addr 172.21.15.1:0/3762572091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 -- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda015e390 msgr2=0x7fdda00b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdd9ffff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda5300640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda015e390 0x7fdda00b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda015e390 0x7fdda00b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 -- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 msgr2=0x7fdda00b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 -- 172.21.15.1:0/3762572091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda01579d0 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda4aff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c3f0 tx=0x7fdd9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980074e0 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98007680 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004a00 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda00b3330 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.027+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda0159b50 con 0x7fdda00b11f0 2024-05-09T06:36:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.028+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda00752b0 con 0x7fdda00b11f0 2024-05-09T06:36:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.028+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd9801a020 con 0x7fdda00b11f0 2024-05-09T06:36:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.029+0000 7fdd9d7fa640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 0x7fdd8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.029+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fdd9808a860 con 0x7fdda00b11f0 2024-05-09T06:36:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.029+0000 7fdd9ffff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 0x7fdd8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.029+0000 7fdd9ffff640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 0x7fdd8006e320 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdda01f79a0 tx=0x7fdd9000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.033+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd98058f50 con 0x7fdda00b11f0 2024-05-09T06:36:21.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.218+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdda00b2ae0 con 0x7fdda00b11f0 2024-05-09T06:36:21.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.218+0000 7fdd9d7fa640 1 -- 172.21.15.1:0/3762572091 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fdd98058850 con 0x7fdda00b11f0 2024-05-09T06:36:21.200 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 msgr2=0x7fdd8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 0x7fdd8006e320 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdda01f79a0 tx=0x7fdd9000b040 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 msgr2=0x7fdda0157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c3f0 tx=0x7fdd9800c8b0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 shutdown_connections 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd8006be60 0x7fdd8006e320 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdda015e390 0x7fdda00b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdda00cc8e0 0x7fdda00b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 --2- 172.21.15.1:0/3762572091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdda00b11f0 0x7fdda0157490 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 >> 172.21.15.1:0/3762572091 conn(0x7fdda00b1da0 msgr2=0x7fdda006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 shutdown_connections 2024-05-09T06:36:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.221+0000 7fdda6d63640 1 -- 172.21.15.1:0/3762572091 wait complete. 2024-05-09T06:36:21.213 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.339+0000 7fef4cb09640 1 Processor -- start 2024-05-09T06:36:21.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.339+0000 7fef4cb09640 1 -- start start 2024-05-09T06:36:21.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4cb09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4806dd00 0x7fef4815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4cb09640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fef4815cde0 0x7fef48070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4cb09640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4806cf40 con 0x7fef4815c4c0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4cb09640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fef48156980 con 0x7fef4815cde0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4cb09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef4806e180 con 0x7fef4806dd00 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4659c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fef4815c4c0 0x7fef480b4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55826/0 (socket says 172.21.15.1:55826) 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef4659c640 1 -- 172.21.15.1:0/2383735317 learned_addr learned my addr 172.21.15.1:0/2383735317 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef46d9d640 1 --2- 172.21.15.1:0/2383735317 >> v2:172.21.15.1:3300/0 conn(0x7fef4815cde0 0x7fef48070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.340+0000 7fef45d9b640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4806dd00 0x7fef4815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef4559a640 1 -- 172.21.15.1:0/2383735317 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787870073 0 0) 0x7fef4806cf40 con 0x7fef4815c4c0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef4559a640 1 -- 172.21.15.1:0/2383735317 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef34003520 con 0x7fef4815c4c0 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef46d9d640 1 -- 172.21.15.1:0/2383735317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4806dd00 msgr2=0x7fef4815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef46d9d640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4806dd00 0x7fef4815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef46d9d640 1 -- 172.21.15.1:0/2383735317 >> v1:172.21.15.1:6789/0 conn(0x7fef4815c4c0 legacy=0x7fef480b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef46d9d640 1 -- 172.21.15.1:0/2383735317 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4806d220 con 0x7fef4815cde0 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.341+0000 7fef46d9d640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef48070c10 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d90 tx=0x7fef3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a8827db37d8fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4559a640 1 -- 172.21.15.1:0/2383735317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38007740 con 0x7fef4815cde0 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4559a640 1 -- 172.21.15.1:0/2383735317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef380078e0 con 0x7fef4815cde0 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4559a640 1 -- 172.21.15.1:0/2383735317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38007cb0 con 0x7fef4815cde0 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 -- 172.21.15.1:0/2383735317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 msgr2=0x7fef48070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef48070c10 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d90 tx=0x7fef3802a660 comp rx=0 tx=0).stop 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 -- 172.21.15.1:0/2383735317 shutdown_connections 2024-05-09T06:36:21.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4806dd00 0x7fef4815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 --2- 172.21.15.1:0/2383735317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef48070c10 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 -- 172.21.15.1:0/2383735317 >> 172.21.15.1:0/2383735317 conn(0x7fef480cc4d0 msgr2=0x7fef480cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 -- 172.21.15.1:0/2383735317 shutdown_connections 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.342+0000 7fef4cb09640 1 -- 172.21.15.1:0/2383735317 wait complete. 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 Processor -- start 2024-05-09T06:36:21.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 -- start start 2024-05-09T06:36:21.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 0x7fef480ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4815c4c0 0x7fef480a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef4815d270 con 0x7fef4815cde0 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef4806d870 con 0x7fef4806dd00 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef4cb09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fef480b54f0 con 0x7fef4815c4c0 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef46d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55472/0 (socket says 172.21.15.1:55472) 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.344+0000 7fef46d9d640 1 -- 172.21.15.1:0/1061666498 learned_addr learned my addr 172.21.15.1:0/1061666498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef45d9b640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4815c4c0 0x7fef480a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef4659c640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 0x7fef480ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 -- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4815c4c0 msgr2=0x7fef480a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4815c4c0 0x7fef480a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 -- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 msgr2=0x7fef480ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 0x7fef480ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 -- 172.21.15.1:0/1061666498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef48159c10 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef4659c640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 0x7fef480ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef46d9d640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fef38009eb0 tx=0x7fef38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3803f070 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef38031bc0 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef48159eb0 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38009450 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.345+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4815a470 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.346+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef480b28f0 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.346+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef3803b410 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.347+0000 7fef3effd640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 0x7fef2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.347+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fef380ac690 con 0x7fef4815cde0 2024-05-09T06:36:21.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.347+0000 7fef4659c640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 0x7fef2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.347+0000 7fef4659c640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 0x7fef2806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fef40007fd0 tx=0x7fef40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.351+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef480b28f0 con 0x7fef4815cde0 2024-05-09T06:36:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.533+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef480752b0 con 0x7fef4815cde0 2024-05-09T06:36:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.533+0000 7fef3effd640 1 -- 172.21.15.1:0/1061666498 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fef3807ae20 con 0x7fef4815cde0 2024-05-09T06:36:21.514 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 msgr2=0x7fef2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 0x7fef2806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fef40007fd0 tx=0x7fef40009040 comp rx=0 tx=0).stop 2024-05-09T06:36:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 msgr2=0x7fef4815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fef38009eb0 tx=0x7fef38004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 shutdown_connections 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fef2806be30 0x7fef2806e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef4815cde0 0x7fef4815ac70 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef4815c4c0 0x7fef480a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 --2- 172.21.15.1:0/1061666498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef4806dd00 0x7fef480ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 >> 172.21.15.1:0/1061666498 conn(0x7fef480cc4d0 msgr2=0x7fef4815ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 shutdown_connections 2024-05-09T06:36:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.535+0000 7fef4cb09640 1 -- 172.21.15.1:0/1061666498 wait complete. 2024-05-09T06:36:21.527 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:21.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.651+0000 7f025875c640 1 Processor -- start 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.651+0000 7f025875c640 1 -- start start 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f025875c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f02500c6860 0x7f0250099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f025875c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 0x7f025015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f025875c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0250099930 con 0x7f02500c6c40 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f025875c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f02500b1e20 con 0x7f02500c6860 2024-05-09T06:36:21.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f025875c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f025015ee90 con 0x7f02500b2aa0 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f0255cf7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 0x7f025015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02564f8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f02500c6860 0x7f0250099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f0255cf7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 0x7f025015e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:39536/0 (socket says 172.21.15.1:39536) 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f0256cf9640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f02500c6c40 0x7f025015f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:55206/0 (socket says 172.21.15.1:55206) 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f0255cf7640 1 -- 172.21.15.1:0/1150745654 learned_addr learned my addr 172.21.15.1:0/1150745654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02554f6640 1 -- 172.21.15.1:0/1150745654 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284137454 0 0) 0x7f0248001490 con 0x7f02500c6c40 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02554f6640 1 -- 172.21.15.1:0/1150745654 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023c003520 con 0x7f02500c6c40 2024-05-09T06:36:21.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02564f8640 1 -- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 msgr2=0x7f025015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02564f8640 1 --2- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 0x7f025015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02564f8640 1 -- 172.21.15.1:0/1150745654 >> v1:172.21.15.6:6789/0 conn(0x7f02500c6c40 legacy=0x7f025015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.652+0000 7f02564f8640 1 -- 172.21.15.1:0/1150745654 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0250162930 con 0x7f02500c6860 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.653+0000 7f02564f8640 1 --2- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6860 0x7f0250099b30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77842d392f5b41f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.653+0000 7f02554f6640 1 -- 172.21.15.1:0/1150745654 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037070 con 0x7f02500c6860 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.653+0000 7f02554f6640 1 -- 172.21.15.1:0/1150745654 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c02ae50 con 0x7f02500c6860 2024-05-09T06:36:21.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f02554f6640 1 -- 172.21.15.1:0/1150745654 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c032660 con 0x7f02500c6860 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 -- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6860 msgr2=0x7f0250099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 --2- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6860 0x7f0250099b30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02a100 comp rx=0 tx=0).stop 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 -- 172.21.15.1:0/1150745654 shutdown_connections 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 --2- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500b2aa0 0x7f025015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 --2- 172.21.15.1:0/1150745654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6860 0x7f0250099b30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 -- 172.21.15.1:0/1150745654 >> 172.21.15.1:0/1150745654 conn(0x7f02500b1fb0 msgr2=0x7f02500b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 -- 172.21.15.1:0/1150745654 shutdown_connections 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.654+0000 7f025875c640 1 -- 172.21.15.1:0/1150745654 wait complete. 2024-05-09T06:36:21.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.655+0000 7f025875c640 1 Processor -- start 2024-05-09T06:36:21.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.655+0000 7f025875c640 1 -- start start 2024-05-09T06:36:21.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500c6860 0x7f02500b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6c40 0x7f0250161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f025015eb20 con 0x7f02500b2aa0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f025015fa30 con 0x7f02500c6c40 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f025875c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f02500c70d0 con 0x7f02500c6860 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f02564f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f02564f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55488/0 (socket says 172.21.15.1:55488) 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f02564f8640 1 -- 172.21.15.1:0/1931873950 learned_addr learned my addr 172.21.15.1:0/1931873950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f0255cf7640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500c6860 0x7f02500b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.656+0000 7f0256cf9640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6c40 0x7f0250161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 -- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500c6860 msgr2=0x7f02500b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500c6860 0x7f02500b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 -- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6c40 msgr2=0x7f0250161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6c40 0x7f0250161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 -- 172.21.15.1:0/1931873950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02500b3b60 con 0x7f02500b2aa0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f02564f8640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f024c035040 tx=0x7f024c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c041070 con 0x7f02500b2aa0 2024-05-09T06:36:21.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c03c460 con 0x7f02500b2aa0 2024-05-09T06:36:21.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02500b3e40 con 0x7f02500b2aa0 2024-05-09T06:36:21.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037040 con 0x7f02500b2aa0 2024-05-09T06:36:21.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.657+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02500c7450 con 0x7f02500b2aa0 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.658+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02500752b0 con 0x7f02500b2aa0 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.659+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f024c03a070 con 0x7f02500b2aa0 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.659+0000 7f0246ffd640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.659+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f024c0ab290 con 0x7f02500b2aa0 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.659+0000 7f0255cf7640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.660+0000 7f0255cf7640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 0x7f023406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02500cd9d0 tx=0x7f0240009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.665+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f024c079460 con 0x7f02500b2aa0 2024-05-09T06:36:21.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.888+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f025006ce30 con 0x7f02500b2aa0 2024-05-09T06:36:21.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.888+0000 7f0246ffd640 1 -- 172.21.15.1:0/1931873950 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7f024c07d760 con 0x7f02500b2aa0 2024-05-09T06:36:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:21.869 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:36:21.870 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 msgr2=0x7f023406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 0x7f023406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02500cd9d0 tx=0x7f0240009240 comp rx=0 tx=0).stop 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 msgr2=0x7f0250152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f024c035040 tx=0x7f024c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 shutdown_connections 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f02500c6c40 0x7f0250161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f02500c6860 0x7f02500b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 --2- 172.21.15.1:0/1931873950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02500b2aa0 0x7f0250152eb0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 >> 172.21.15.1:0/1931873950 conn(0x7f02500b1fb0 msgr2=0x7f025006de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 shutdown_connections 2024-05-09T06:36:21.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:21.891+0000 7f025875c640 1 -- 172.21.15.1:0/1931873950 wait complete. 2024-05-09T06:36:21.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:21.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.005+0000 7f4ebd929640 1 Processor -- start 2024-05-09T06:36:21.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.005+0000 7f4ebd929640 1 -- start start 2024-05-09T06:36:21.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4ebd929640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4eb80d4940 0x7f4eb80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4ebd929640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4eb80d2b10 0x7f4eb80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4ebd929640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb80b33c0 con 0x7f4eb8161950 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4ebd929640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb80d4dd0 con 0x7f4eb80d4940 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4ebd929640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb80b3660 con 0x7f4eb80d2b10 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4eb67fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4eb80d4940 0x7f4eb80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4eb67fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4eb80d4940 0x7f4eb80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55494/0 (socket says 172.21.15.1:55494) 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.006+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2662713387 learned_addr learned my addr 172.21.15.1:0/2662713387 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb77fe640 1 --2- 172.21.15.1:0/2662713387 >> v2:172.21.15.6:3300/0 conn(0x7f4eb80d2b10 0x7f4eb80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2662713387 >> v2:172.21.15.6:3300/0 conn(0x7f4eb80d2b10 msgr2=0x7f4eb80b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb67fc640 1 --2- 172.21.15.1:0/2662713387 >> v2:172.21.15.6:3300/0 conn(0x7f4eb80d2b10 0x7f4eb80b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2662713387 >> v1:172.21.15.6:6789/0 conn(0x7f4eb8161950 legacy=0x7f4eb80d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2662713387 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80b3940 con 0x7f4eb80d4940 2024-05-09T06:36:21.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.007+0000 7f4eb67fc640 1 --2- 172.21.15.1:0/2662713387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb80d4940 0x7f4eb80b4140 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f4eb808e540 tx=0x7f4ea002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a076fec3c6c660cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4eb5ffb640 1 -- 172.21.15.1:0/2662713387 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0037070 con 0x7f4eb80d4940 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4eb5ffb640 1 -- 172.21.15.1:0/2662713387 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea0032440 con 0x7f4eb80d4940 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4eb5ffb640 1 -- 172.21.15.1:0/2662713387 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0032810 con 0x7f4eb80d4940 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 -- 172.21.15.1:0/2662713387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb80d4940 msgr2=0x7f4eb80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2662713387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb80d4940 0x7f4eb80b4140 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f4eb808e540 tx=0x7f4ea002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 -- 172.21.15.1:0/2662713387 shutdown_connections 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2662713387 >> v2:172.21.15.6:3300/0 conn(0x7f4eb80d2b10 0x7f4eb80b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2662713387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb80d4940 0x7f4eb80b4140 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 -- 172.21.15.1:0/2662713387 >> 172.21.15.1:0/2662713387 conn(0x7f4eb80d3e20 msgr2=0x7f4eb80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 -- 172.21.15.1:0/2662713387 shutdown_connections 2024-05-09T06:36:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.008+0000 7f4ebd929640 1 -- 172.21.15.1:0/2662713387 wait complete. 2024-05-09T06:36:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.009+0000 7f4ebd929640 1 Processor -- start 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 -- start start 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4eb80d2b10 0x7f4eb814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4eb815abb0 0x7f4eb80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80d1100 con 0x7f4eb8161950 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80d0f80 con 0x7f4eb815abb0 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4ebd929640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80d1c00 con 0x7f4eb80d2b10 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4eb67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4eb67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55504/0 (socket says 172.21.15.1:55504) 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2546813725 learned_addr learned my addr 172.21.15.1:0/2546813725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.010+0000 7f4eb6ffd640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4eb80d2b10 0x7f4eb814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb77fe640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4eb815abb0 0x7f4eb80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4eb80d2b10 msgr2=0x7f4eb814d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4eb80d2b10 0x7f4eb814d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4eb815abb0 msgr2=0x7f4eb80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4eb815abb0 0x7f4eb80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 -- 172.21.15.1:0/2546813725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb814a6b0 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eb67fc640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4ea002a270 tx=0x7f4ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0041020 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea00306c0 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0037040 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb814a990 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.011+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb80b4d20 con 0x7f4eb8161950 2024-05-09T06:36:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.012+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80752b0 con 0x7f4eb8161950 2024-05-09T06:36:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.013+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ea0044020 con 0x7f4eb8161950 2024-05-09T06:36:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.013+0000 7f4eaffff640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 0x7f4e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.013+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f4ea00ab7a0 con 0x7f4eb8161950 2024-05-09T06:36:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.013+0000 7f4eb6ffd640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 0x7f4e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.014+0000 7f4eb6ffd640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 0x7f4e9806e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0006fd0 tx=0x7f4eb0009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.018+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ea004b030 con 0x7f4eb8161950 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.202+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4eb80cf920 con 0x7f4eb8161950 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.203+0000 7f4eaffff640 1 -- 172.21.15.1:0/2546813725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17199 (secure 0 0 0) 0x7f4ea00798e0 con 0x7f4eb8161950 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:39.452636+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.1:6834/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 msgr2=0x7f4e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 0x7f4e9806e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0006fd0 tx=0x7f4eb0009060 comp rx=0 tx=0).stop 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 msgr2=0x7f4eb80cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4ea002a270 tx=0x7f4ea00046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 shutdown_connections 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e9806be30 0x7f4e9806e2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4eb815abb0 0x7f4eb80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4eb8161950 0x7f4eb80cd660 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 --2- 172.21.15.1:0/2546813725 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4eb80d2b10 0x7f4eb814d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 >> 172.21.15.1:0/2546813725 conn(0x7f4eb80d3e20 msgr2=0x7f4eb80b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 shutdown_connections 2024-05-09T06:36:22.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.205+0000 7f4ebd929640 1 -- 172.21.15.1:0/2546813725 wait complete. 2024-05-09T06:36:22.197 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:36:22.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.323+0000 7f7e401a9640 1 Processor -- start 2024-05-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.323+0000 7f7e401a9640 1 -- start start 2024-05-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e401a9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7e38099b30 0x7f7e380b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e401a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 0x7f7e3815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e401a9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e38099930 con 0x7f7e380c6860 2024-05-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e401a9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7e380b1e20 con 0x7f7e38099b30 2024-05-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e401a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3815ee90 con 0x7f7e3815e4b0 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3df45640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7e380c6860 0x7f7e380c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55830/0 (socket says 172.21.15.1:55830) 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3df45640 1 -- 172.21.15.1:0/2201660413 learned_addr learned my addr 172.21.15.1:0/2201660413 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3d744640 1 --2- 172.21.15.1:0/2201660413 >> v2:172.21.15.6:3300/0 conn(0x7f7e38099b30 0x7f7e380b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3e746640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 0x7f7e3815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3cf43640 1 -- 172.21.15.1:0/2201660413 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506013889 0 0) 0x7f7e34001110 con 0x7f7e380c6860 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3cf43640 1 -- 172.21.15.1:0/2201660413 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e24003520 con 0x7f7e380c6860 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3d744640 1 -- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 msgr2=0x7f7e3815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3d744640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 0x7f7e3815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3d744640 1 -- 172.21.15.1:0/2201660413 >> v1:172.21.15.1:6789/0 conn(0x7f7e380c6860 legacy=0x7f7e380c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3d744640 1 -- 172.21.15.1:0/2201660413 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38162930 con 0x7f7e38099b30 2024-05-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.324+0000 7f7e3e746640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 0x7f7e3815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.325+0000 7f7e3d744640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e380b2aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01d1865abe7af44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.325+0000 7f7e3cf43640 1 -- 172.21.15.1:0/2201660413 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28037070 con 0x7f7e38099b30 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.325+0000 7f7e3cf43640 1 -- 172.21.15.1:0/2201660413 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e28032440 con 0x7f7e38099b30 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.325+0000 7f7e3cf43640 1 -- 172.21.15.1:0/2201660413 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28032810 con 0x7f7e38099b30 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 -- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 msgr2=0x7f7e380b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e380b2aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 -- 172.21.15.1:0/2201660413 shutdown_connections 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e3815e4b0 0x7f7e3815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 --2- 172.21.15.1:0/2201660413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e380b2aa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 -- 172.21.15.1:0/2201660413 >> 172.21.15.1:0/2201660413 conn(0x7f7e380b1fb0 msgr2=0x7f7e380b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 -- 172.21.15.1:0/2201660413 shutdown_connections 2024-05-09T06:36:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.326+0000 7f7e401a9640 1 -- 172.21.15.1:0/2201660413 wait complete. 2024-05-09T06:36:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.327+0000 7f7e401a9640 1 Processor -- start 2024-05-09T06:36:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.327+0000 7f7e401a9640 1 -- start start 2024-05-09T06:36:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e38155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e380c6860 0x7f7e3815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3815f860 con 0x7f7e3815e4b0 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3815eb20 con 0x7f7e38099b30 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e401a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3815f9e0 con 0x7f7e380c6860 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55518/0 (socket says 172.21.15.1:55518) 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 -- 172.21.15.1:0/467516388 learned_addr learned my addr 172.21.15.1:0/467516388 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3df45640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e38155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3d744640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e380c6860 0x7f7e3815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 -- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e380c6860 msgr2=0x7f7e3815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e380c6860 0x7f7e3815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 -- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 msgr2=0x7f7e38155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e38155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.328+0000 7f7e3e746640 1 -- 172.21.15.1:0/467516388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38155960 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e3e746640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000efc0 tx=0x7f7e3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30018030 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3000cd90 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e380c9e90 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30003bb0 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.329+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e380ca450 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.330+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e380752b0 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.330+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e30005d10 con 0x7f7e3815e4b0 2024-05-09T06:36:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.331+0000 7f7e2e7fc640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.331+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f7e3008b9b0 con 0x7f7e3815e4b0 2024-05-09T06:36:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.331+0000 7f7e3df45640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.331+0000 7f7e3df45640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005880 tx=0x7f7e34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.335+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e30022070 con 0x7f7e3815e4b0 2024-05-09T06:36:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.557+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e3806ce30 con 0x7f7e3815e4b0 2024-05-09T06:36:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.558+0000 7f7e2e7fc640 1 -- 172.21.15.1:0/467516388 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7f7e30010040 con 0x7f7e3815e4b0 2024-05-09T06:36:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:22.539 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:36:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 msgr2=0x7f7e1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005880 tx=0x7f7e34009040 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 msgr2=0x7f7e380c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000efc0 tx=0x7f7e3000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 shutdown_connections 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e1c06be30 0x7f7e1c06e2f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e3815e4b0 0x7f7e380c7500 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e380c6860 0x7f7e3815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 --2- 172.21.15.1:0/467516388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e38099b30 0x7f7e38155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 >> 172.21.15.1:0/467516388 conn(0x7f7e380b1fb0 msgr2=0x7f7e38156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 shutdown_connections 2024-05-09T06:36:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.560+0000 7f7e401a9640 1 -- 172.21.15.1:0/467516388 wait complete. 2024-05-09T06:36:22.552 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:22.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 Processor -- start 2024-05-09T06:36:22.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 -- start start 2024-05-09T06:36:22.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff42815f3f0 0x7ff4280cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280d04a0 con 0x7ff4280b0080 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280722c0 con 0x7ff428071ce0 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.676+0000 7ff42efee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280725a0 con 0x7ff42815f3f0 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42d58b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff42815f3f0 0x7ff4280cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42cd8a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff4280b0080 0x7ff42815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55842/0 (socket says 172.21.15.1:55842) 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42cd8a640 1 -- 172.21.15.1:0/3401294543 learned_addr learned my addr 172.21.15.1:0/3401294543 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42d58b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff42815f3f0 0x7ff4280cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55524/0 (socket says 172.21.15.1:55524) 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42d58b640 1 -- 172.21.15.1:0/3401294543 >> v1:172.21.15.6:6789/0 conn(0x7ff428071ce0 legacy=0x7ff42815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42d58b640 1 -- 172.21.15.1:0/3401294543 >> v1:172.21.15.1:6789/0 conn(0x7ff4280b0080 legacy=0x7ff42815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff42d58b640 1 -- 172.21.15.1:0/3401294543 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff428072880 con 0x7ff42815f3f0 2024-05-09T06:36:22.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.677+0000 7ff4277fe640 1 -- 172.21.15.1:0/3401294543 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722276845 0 0) 0x7ff4280d04a0 con 0x7ff4280b0080 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42d58b640 1 --2- 172.21.15.1:0/3401294543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42815f3f0 0x7ff4280cbb60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d90 tx=0x7ff42002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb50fc073687b6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff4277fe640 1 -- 172.21.15.1:0/3401294543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420007740 con 0x7ff42815f3f0 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff4277fe640 1 -- 172.21.15.1:0/3401294543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4200078e0 con 0x7ff42815f3f0 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff4277fe640 1 -- 172.21.15.1:0/3401294543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420007cb0 con 0x7ff42815f3f0 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 -- 172.21.15.1:0/3401294543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42815f3f0 msgr2=0x7ff4280cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 --2- 172.21.15.1:0/3401294543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42815f3f0 0x7ff4280cbb60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d90 tx=0x7ff42002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 -- 172.21.15.1:0/3401294543 shutdown_connections 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 --2- 172.21.15.1:0/3401294543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42815f3f0 0x7ff4280cbb60 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 -- 172.21.15.1:0/3401294543 >> 172.21.15.1:0/3401294543 conn(0x7ff4280d0b50 msgr2=0x7ff4280d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 -- 172.21.15.1:0/3401294543 shutdown_connections 2024-05-09T06:36:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.678+0000 7ff42efee640 1 -- 172.21.15.1:0/3401294543 wait complete. 2024-05-09T06:36:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.679+0000 7ff42efee640 1 Processor -- start 2024-05-09T06:36:22.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 -- start start 2024-05-09T06:36:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff428071ce0 0x7ff4280c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42815f3f0 0x7ff42815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff42815ff40 con 0x7ff4280b0080 2024-05-09T06:36:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff42815e7c0 con 0x7ff42815f3f0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.680+0000 7ff42efee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff42806b1e0 con 0x7ff428071ce0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55532/0 (socket says 172.21.15.1:55532) 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 -- 172.21.15.1:0/1664129784 learned_addr learned my addr 172.21.15.1:0/1664129784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff42d58b640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42815f3f0 0x7ff42815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff42cd8a640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff428071ce0 0x7ff4280c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 -- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff428071ce0 msgr2=0x7ff4280c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff428071ce0 0x7ff4280c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 -- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42815f3f0 msgr2=0x7ff42815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42815f3f0 0x7ff42815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 -- 172.21.15.1:0/1664129784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280c7c20 con 0x7ff4280b0080 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff427fff640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c430 tx=0x7ff41400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414011790 con 0x7ff4280b0080 2024-05-09T06:36:22.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff414017070 con 0x7ff4280b0080 2024-05-09T06:36:22.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414011bb0 con 0x7ff4280b0080 2024-05-09T06:36:22.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4280c7f00 con 0x7ff4280b0080 2024-05-09T06:36:22.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.681+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4280c84c0 con 0x7ff4280b0080 2024-05-09T06:36:22.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.682+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4280752b0 con 0x7ff4280b0080 2024-05-09T06:36:22.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.683+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff41401c460 con 0x7ff4280b0080 2024-05-09T06:36:22.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.683+0000 7ff4257fa640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.683+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7ff41408ada0 con 0x7ff4280b0080 2024-05-09T06:36:22.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.683+0000 7ff42cd8a640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.684+0000 7ff42cd8a640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 0x7ff40c06e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.688+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff414058ee0 con 0x7ff4280b0080 2024-05-09T06:36:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.909+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff42806b720 con 0x7ff4280b0080 2024-05-09T06:36:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.910+0000 7ff4257fa640 1 -- 172.21.15.1:0/1664129784 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5169 (secure 0 0 0) 0x7ff41405d1e0 con 0x7ff4280b0080 2024-05-09T06:36:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:22.891 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:34:02.686762+0000","modified":"2024-05-09T06:35:43.462274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6836/779210140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":779210140},{"type":"v1","addr":"172.21.15.1:6836","nonce":779210140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":6}]} 2024-05-09T06:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 74 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 msgr2=0x7ff40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 0x7ff40c06e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c007d50 comp rx=0 tx=0).stop 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 msgr2=0x7ff428153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c430 tx=0x7ff41400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 shutdown_connections 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42815f3f0 0x7ff42815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4280b0080 0x7ff428153f90 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 --2- 172.21.15.1:0/1664129784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff428071ce0 0x7ff4280c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 >> 172.21.15.1:0/1664129784 conn(0x7ff4280d0b50 msgr2=0x7ff4280b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 shutdown_connections 2024-05-09T06:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:22.912+0000 7ff42efee640 1 -- 172.21.15.1:0/1664129784 wait complete. 2024-05-09T06:36:22.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:23.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.026+0000 7fa324510640 1 Processor -- start 2024-05-09T06:36:23.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.026+0000 7fa324510640 1 -- start start 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.027+0000 7fa324510640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.027+0000 7fa324510640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa31c0d0f80 0x7fa31c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.027+0000 7fa324510640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31c0b33c0 con 0x7fa31c0d4940 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.027+0000 7fa324510640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa31c0d4dd0 con 0x7fa31c0d0f80 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.027+0000 7fa324510640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c0b3660 con 0x7fa31c161950 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa3222ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa31c0d0f80 0x7fa31c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa31c0d0f80 0x7fa31c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:37326/0 (socket says 172.21.15.1:37326) 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa3222ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:39596/0 (socket says 172.21.15.1:39596) 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 -- 172.21.15.1:0/4290605052 learned_addr learned my addr 172.21.15.1:0/4290605052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 -- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 msgr2=0x7fa31c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 -- 172.21.15.1:0/4290605052 >> v1:172.21.15.6:6789/0 conn(0x7fa31c0d4940 legacy=0x7fa31c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa322aad640 1 -- 172.21.15.1:0/4290605052 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c0b3940 con 0x7fa31c0d0f80 2024-05-09T06:36:23.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.028+0000 7fa3222ac640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.029+0000 7fa322aad640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c0d2940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b3590 tx=0x7fa31402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64591334969519da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.029+0000 7fa3212aa640 1 -- 172.21.15.1:0/4290605052 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314036070 con 0x7fa31c0d0f80 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.029+0000 7fa3212aa640 1 -- 172.21.15.1:0/4290605052 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314031600 con 0x7fa31c0d0f80 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.029+0000 7fa3212aa640 1 -- 172.21.15.1:0/4290605052 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140319d0 con 0x7fa31c0d0f80 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 -- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 msgr2=0x7fa31c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c0d2940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b3590 tx=0x7fa31402a710 comp rx=0 tx=0).stop 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 -- 172.21.15.1:0/4290605052 shutdown_connections 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c0d2940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 --2- 172.21.15.1:0/4290605052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 -- 172.21.15.1:0/4290605052 >> 172.21.15.1:0/4290605052 conn(0x7fa31c0d3e20 msgr2=0x7fa31c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:23.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 -- 172.21.15.1:0/4290605052 shutdown_connections 2024-05-09T06:36:23.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.030+0000 7fa324510640 1 -- 172.21.15.1:0/4290605052 wait complete. 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.031+0000 7fa324510640 1 Processor -- start 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.031+0000 7fa324510640 1 -- start start 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c1f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c1f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c0b52d0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c0b5150 con 0x7fa31c0d0f80 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa324510640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c0b5450 con 0x7fa31c161950 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55544/0 (socket says 172.21.15.1:55544) 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 -- 172.21.15.1:0/3566612630 learned_addr learned my addr 172.21.15.1:0/3566612630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa321aab640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c1f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa3222ac640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c1f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 -- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 msgr2=0x7fa31c1f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c1f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 -- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 msgr2=0x7fa31c1f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c1f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 -- 172.21.15.1:0/3566612630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1f5de0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.032+0000 7fa322aad640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa314009f40 tx=0x7fa314004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.033+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314036050 con 0x7fa31c1f6e80 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.033+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314042070 con 0x7fa31c1f6e80 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.033+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1f60c0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.033+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314009510 con 0x7fa31c1f6e80 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.033+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1f6960 con 0x7fa31c1f6e80 2024-05-09T06:36:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.034+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31c1f6290 con 0x7fa31c1f6e80 2024-05-09T06:36:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.034+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa314007c70 con 0x7fa31c1f6e80 2024-05-09T06:36:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.035+0000 7fa312ffd640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 0x7fa30006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.035+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7fa3140abe10 con 0x7fa31c1f6e80 2024-05-09T06:36:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.035+0000 7fa3222ac640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 0x7fa30006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.035+0000 7fa3222ac640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 0x7fa30006e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0cf5d0 tx=0x7fa318009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.039+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa314079ec0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.227+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa31c0752b0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.228+0000 7fa312ffd640 1 -- 172.21.15.1:0/3566612630 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17199 (secure 0 0 0) 0x7fa31407e1c0 con 0x7fa31c1f6e80 2024-05-09T06:36:23.209 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:23.209 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:35:39.452636+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:34:00.356458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-09T06:34:01.667377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.1:6834/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 msgr2=0x7fa30006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 0x7fa30006e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0cf5d0 tx=0x7fa318009190 comp rx=0 tx=0).stop 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 msgr2=0x7fa31c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa314009f40 tx=0x7fa314004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 shutdown_connections 2024-05-09T06:36:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa30006be30 0x7fa30006e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31c1f6e80 0x7fa31c0cbf40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa31c161950 0x7fa31c1f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 --2- 172.21.15.1:0/3566612630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa31c0d0f80 0x7fa31c1f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 >> 172.21.15.1:0/3566612630 conn(0x7fa31c0d3e20 msgr2=0x7fa31c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 shutdown_connections 2024-05-09T06:36:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.230+0000 7fa324510640 1 -- 172.21.15.1:0/3566612630 wait complete. 2024-05-09T06:36:23.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:36:23.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.346+0000 7f189ca5d640 1 Processor -- start 2024-05-09T06:36:23.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.346+0000 7f189ca5d640 1 -- start start 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f189ca5d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18980d4940 0x7f18980b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f189ca5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f18980b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f189ca5d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18980b33c0 con 0x7f1898161950 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f189ca5d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f18980d4dd0 con 0x7f18980d4940 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f189ca5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18980b3660 con 0x7f18980d2b10 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f1895d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18980d4940 0x7f18980b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f18980b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f1895d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18980d4940 0x7f18980b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:37336/0 (socket says 172.21.15.1:37336) 2024-05-09T06:36:23.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.347+0000 7f1895d9b640 1 -- 172.21.15.1:0/2829685753 learned_addr learned my addr 172.21.15.1:0/2829685753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.348+0000 7f1895d9b640 1 -- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 msgr2=0x7f18980b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.348+0000 7f1895d9b640 1 --2- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f18980b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.348+0000 7f1895d9b640 1 -- 172.21.15.1:0/2829685753 >> v1:172.21.15.6:6789/0 conn(0x7f1898161950 legacy=0x7f18980d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.348+0000 7f1895d9b640 1 -- 172.21.15.1:0/2829685753 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18980b3940 con 0x7f18980d4940 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f1895d9b640 1 --2- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18980d4940 0x7f18980b4140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b4cc2ac8be776e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189559a640 1 -- 172.21.15.1:0/2829685753 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c037070 con 0x7f18980d4940 2024-05-09T06:36:23.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189559a640 1 -- 172.21.15.1:0/2829685753 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188c032440 con 0x7f18980d4940 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189559a640 1 -- 172.21.15.1:0/2829685753 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c032810 con 0x7f18980d4940 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 -- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18980d4940 msgr2=0x7f18980b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 --2- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18980d4940 0x7f18980b4140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02a270 comp rx=0 tx=0).stop 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 -- 172.21.15.1:0/2829685753 shutdown_connections 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 --2- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f18980b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 --2- 172.21.15.1:0/2829685753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18980d4940 0x7f18980b4140 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 -- 172.21.15.1:0/2829685753 >> 172.21.15.1:0/2829685753 conn(0x7f18980d3e20 msgr2=0x7f18980d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 -- 172.21.15.1:0/2829685753 shutdown_connections 2024-05-09T06:36:23.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.349+0000 7f189ca5d640 1 -- 172.21.15.1:0/2829685753 wait complete. 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 Processor -- start 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 -- start start 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f189809f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f189809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18980d1100 con 0x7f1898151400 2024-05-09T06:36:23.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f18980d0f80 con 0x7f1898161950 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.351+0000 7f189ca5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18980d1c00 con 0x7f18980d2b10 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55560/0 (socket says 172.21.15.1:55560) 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 -- 172.21.15.1:0/1802692797 learned_addr learned my addr 172.21.15.1:0/1802692797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f189659c640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f189809f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1895d9b640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f189809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 -- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 msgr2=0x7f189809f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f189809f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 -- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 msgr2=0x7f189809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f189809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 -- 172.21.15.1:0/1802692797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981518b0 con 0x7f1898151400 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1896d9d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f189000ef90 tx=0x7f189000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890018020 con 0x7f1898151400 2024-05-09T06:36:23.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890014480 con 0x7f1898151400 2024-05-09T06:36:23.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189809fb10 con 0x7f1898151400 2024-05-09T06:36:23.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890014870 con 0x7f1898151400 2024-05-09T06:36:23.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.352+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18980a00d0 con 0x7f1898151400 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.353+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18980752b0 con 0x7f1898151400 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.354+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1890005d10 con 0x7f1898151400 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.355+0000 7f1886ffd640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 0x7f187806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.355+0000 7f189659c640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 0x7f187806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.355+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6389+0+0 (secure 0 0 0) 0x7f189008bee0 con 0x7f1898151400 2024-05-09T06:36:23.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.355+0000 7f189659c640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 0x7f187806e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1888000f60 tx=0x7f1888009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.359+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1890022030 con 0x7f1898151400 2024-05-09T06:36:23.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.582+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f18980d21e0 con 0x7f1898151400 2024-05-09T06:36:23.563 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:36:23.582+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:36:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.584+0000 7f1886ffd640 1 -- 172.21.15.1:0/1802692797 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f1890010040 con 0x7f1898151400 2024-05-09T06:36:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 msgr2=0x7f187806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 0x7f187806e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1888000f60 tx=0x7f1888009040 comp rx=0 tx=0).stop 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 msgr2=0x7f18981f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f189000ef90 tx=0x7f189000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 shutdown_connections 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187806be30 0x7f187806e2f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1898151400 0x7f18981f71f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f189809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 --2- 172.21.15.1:0/1802692797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d2b10 0x7f189809f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 >> 172.21.15.1:0/1802692797 conn(0x7f18980d3e20 msgr2=0x7f18980b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 shutdown_connections 2024-05-09T06:36:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.587+0000 7f189ca5d640 1 -- 172.21.15.1:0/1802692797 wait complete. 2024-05-09T06:36:23.569 INFO:tasks.ceph.mds.c.smithi001.stderr: -2310> 2024-05-09T06:35:45.462+0000 7efc715d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:36:23.570 INFO:tasks.ceph.mds.c.smithi001.stderr: -1923> 2024-05-09T06:35:46.624+0000 7efc6d5cd640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55f34a2f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T06:36:23.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:36:23.598 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:36:23.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.706+0000 7fb46921c640 1 Processor -- start 2024-05-09T06:36:23.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.706+0000 7fb46921c640 1 -- start start 2024-05-09T06:36:23.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46921c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb46415e390 0x7fb4640cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46921c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4640b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46921c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46415e900 con 0x7fb4640b1730 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46921c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb4640b1ba0 con 0x7fb46415e390 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46921c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb46415ebe0 con 0x7fb4640cf990 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb46415e390 0x7fb4640cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb46415e390 0x7fb4640cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55588/0 (socket says 172.21.15.1:55588) 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 -- 172.21.15.1:0/1023485512 learned_addr learned my addr 172.21.15.1:0/1023485512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb46259c640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4640b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 -- 172.21.15.1:0/1023485512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 msgr2=0x7fb4640b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4640b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 -- 172.21.15.1:0/1023485512 >> v1:172.21.15.6:6789/0 conn(0x7fb4640b1730 legacy=0x7fb464072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.707+0000 7fb462d9d640 1 -- 172.21.15.1:0/1023485512 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46415eec0 con 0x7fb46415e390 2024-05-09T06:36:23.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb462d9d640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4640cf450 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb454009870 tx=0x7fb45402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff9a7e33f15bdc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb461d9b640 1 -- 172.21.15.1:0/1023485512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454037070 con 0x7fb46415e390 2024-05-09T06:36:23.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb461d9b640 1 -- 172.21.15.1:0/1023485512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb454032440 con 0x7fb46415e390 2024-05-09T06:36:23.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb461d9b640 1 -- 172.21.15.1:0/1023485512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454032810 con 0x7fb46415e390 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb46921c640 1 -- 172.21.15.1:0/1023485512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 msgr2=0x7fb4640cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.708+0000 7fb46921c640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4640cf450 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb454009870 tx=0x7fb45402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 -- 172.21.15.1:0/1023485512 shutdown_connections 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4640b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 --2- 172.21.15.1:0/1023485512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4640cf450 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 -- 172.21.15.1:0/1023485512 >> 172.21.15.1:0/1023485512 conn(0x7fb4640b1da0 msgr2=0x7fb4640b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 -- 172.21.15.1:0/1023485512 shutdown_connections 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.709+0000 7fb46921c640 1 -- 172.21.15.1:0/1023485512 wait complete. 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.710+0000 7fb46921c640 1 Processor -- start 2024-05-09T06:36:23.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.710+0000 7fb46921c640 1 -- start start 2024-05-09T06:36:23.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4640b1730 0x7fb4640c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4641e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640cd8f0 con 0x7fb46415e390 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb46415f6c0 con 0x7fb4640b1730 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46921c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb464162ba0 con 0x7fb4640cf990 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55598/0 (socket says 172.21.15.1:55598) 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 -- 172.21.15.1:0/1171447839 learned_addr learned my addr 172.21.15.1:0/1171447839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46259c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4641e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb462d9d640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4640b1730 0x7fb4640c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 -- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 msgr2=0x7fb4641e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4641e84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 -- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4640b1730 msgr2=0x7fb4640c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4640b1730 0x7fb4640c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46359e640 1 -- 172.21.15.1:0/1171447839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464160a50 con 0x7fb46415e390 2024-05-09T06:36:23.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.711+0000 7fb46259c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4641e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb46359e640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c780 tx=0x7fb45c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004b60 con 0x7fb46415e390 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c004d00 con 0x7fb46415e390 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4640b5960 con 0x7fb46415e390 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00aa90 con 0x7fb46415e390 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.712+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4640b4950 con 0x7fb46415e390 2024-05-09T06:36:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.713+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4640752b0 con 0x7fb46415e390 2024-05-09T06:36:23.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.713+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb45c00ac30 con 0x7fb46415e390 2024-05-09T06:36:23.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.714+0000 7fb44b7fe640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 0x7fb45006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:23.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.714+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6475+0+0 (secure 0 0 0) 0x7fb45c08b460 con 0x7fb46415e390 2024-05-09T06:36:23.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.714+0000 7fb462d9d640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 0x7fb45006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:23.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.714+0000 7fb462d9d640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 0x7fb45006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb454034000 tx=0x7fb45402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:23.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.718+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb45c020070 con 0x7fb46415e390 2024-05-09T06:36:23.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:23.939+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb4640d0b80 con 0x7fb46415e390 2024-05-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.585+0000 7fb44b7fe640 1 -- 172.21.15.1:0/1171447839 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fb45c059b10 con 0x7fb46415e390 2024-05-09T06:36:24.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.588+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 msgr2=0x7fb45006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.588+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 0x7fb45006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb454034000 tx=0x7fb45402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.588+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 msgr2=0x7fb4641e8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.588+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c780 tx=0x7fb45c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 shutdown_connections 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb45006be30 0x7fb45006e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb46415e390 0x7fb4641e8a30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4640cf990 0x7fb4641e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 --2- 172.21.15.1:0/1171447839 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4640b1730 0x7fb4640c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 >> 172.21.15.1:0/1171447839 conn(0x7fb4640b1da0 msgr2=0x7fb46406ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 shutdown_connections 2024-05-09T06:36:24.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.589+0000 7fb46921c640 1 -- 172.21.15.1:0/1171447839 wait complete. 2024-05-09T06:36:24.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:36:24.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.760+0000 7fc60f8ca640 1 Processor -- start 2024-05-09T06:36:24.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.760+0000 7fc60f8ca640 1 -- start start 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60f8ca640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 0x7fc6080cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60f8ca640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6081446d0 0x7fc6080d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60f8ca640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6080cdd50 con 0x7fc608144110 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60f8ca640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc6080ce8d0 con 0x7fc6081446d0 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60f8ca640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc6080ce030 con 0x7fc608143bb0 2024-05-09T06:36:24.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6081446d0 0x7fc6080d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc6081446d0 0x7fc6080d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55610/0 (socket says 172.21.15.1:55610) 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 -- 172.21.15.1:0/2483559806 learned_addr learned my addr 172.21.15.1:0/2483559806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 -- 172.21.15.1:0/2483559806 >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 msgr2=0x7fc6080cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60d666640 1 --2- 172.21.15.1:0/2483559806 >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 0x7fc6080cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 --2- 172.21.15.1:0/2483559806 >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 0x7fc6080cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.761+0000 7fc60ce65640 1 -- 172.21.15.1:0/2483559806 >> v1:172.21.15.6:6789/0 conn(0x7fc608144110 legacy=0x7fc60815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.762+0000 7fc60ce65640 1 -- 172.21.15.1:0/2483559806 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6080ce310 con 0x7fc6081446d0 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.762+0000 7fc60d666640 1 --2- 172.21.15.1:0/2483559806 >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 0x7fc6080cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.762+0000 7fc60ce65640 1 --2- 172.21.15.1:0/2483559806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6081446d0 0x7fc6080d01f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc6080cef80 tx=0x7fc5f802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f308865b9d5bc42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.762+0000 7fc5fffff640 1 -- 172.21.15.1:0/2483559806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8032710 con 0x7fc6081446d0 2024-05-09T06:36:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc5fffff640 1 -- 172.21.15.1:0/2483559806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f80328b0 con 0x7fc6081446d0 2024-05-09T06:36:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc5fffff640 1 -- 172.21.15.1:0/2483559806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8032ca0 con 0x7fc6081446d0 2024-05-09T06:36:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 -- 172.21.15.1:0/2483559806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6081446d0 msgr2=0x7fc6080d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/2483559806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6081446d0 0x7fc6080d01f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc6080cef80 tx=0x7fc5f802c850 comp rx=0 tx=0).stop 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 -- 172.21.15.1:0/2483559806 shutdown_connections 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/2483559806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6081446d0 0x7fc6080d01f0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/2483559806 >> v2:172.21.15.6:3300/0 conn(0x7fc608143bb0 0x7fc6080cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 -- 172.21.15.1:0/2483559806 >> 172.21.15.1:0/2483559806 conn(0x7fc60806b3c0 msgr2=0x7fc60806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 -- 172.21.15.1:0/2483559806 shutdown_connections 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.763+0000 7fc60f8ca640 1 -- 172.21.15.1:0/2483559806 wait complete. 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.764+0000 7fc60f8ca640 1 Processor -- start 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.764+0000 7fc60f8ca640 1 -- start start 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 0x7fc60815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 0x7fc60815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080cdaf0 con 0x7fc608143bb0 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080cd970 con 0x7fc6081446d0 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60f8ca640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc60806e320 con 0x7fc608144110 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60d666640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60d666640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55618/0 (socket says 172.21.15.1:55618) 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60d666640 1 -- 172.21.15.1:0/1771222410 learned_addr learned my addr 172.21.15.1:0/1771222410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60ce65640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 0x7fc60815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.765+0000 7fc60de67640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 0x7fc60815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 -- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 msgr2=0x7fc60815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 0x7fc60815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 -- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 msgr2=0x7fc60815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 0x7fc60815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 -- 172.21.15.1:0/1771222410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6081327d0 con 0x7fc608143bb0 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60ce65640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 0x7fc60815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60de67640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 0x7fc60815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60d666640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c7f0 tx=0x7fc60400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604015070 con 0x7fc608143bb0 2024-05-09T06:36:24.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604007a30 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6080b3de0 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604007e00 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.766+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6080b43a0 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.767+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc608075320 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.768+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc604018070 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.768+0000 7fc5fdffb640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 0x7fc5ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.768+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6475+0+0 (secure 0 0 0) 0x7fc60408af90 con 0x7fc608143bb0 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.768+0000 7fc60ce65640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 0x7fc5ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.769+0000 7fc60ce65640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 0x7fc5ec06e370 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8037000 tx=0x7fc5f8036040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.773+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc6040597b0 con 0x7fc608143bb0 2024-05-09T06:36:24.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:24.956+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc608071050 con 0x7fc608143bb0 2024-05-09T06:36:25.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.672+0000 7fc5fdffb640 1 -- 172.21.15.1:0/1771222410 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7fc6040590b0 con 0x7fc608143bb0 2024-05-09T06:36:25.653 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:36:25.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.678+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 msgr2=0x7fc5ec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.678+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 0x7fc5ec06e370 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8037000 tx=0x7fc5f8036040 comp rx=0 tx=0).stop 2024-05-09T06:36:25.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.678+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 msgr2=0x7fc608132290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.678+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc60400c7f0 tx=0x7fc60400cbf0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 shutdown_connections 2024-05-09T06:36:25.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ec06beb0 0x7fc5ec06e370 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6081446d0 0x7fc60815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc608144110 0x7fc60815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 --2- 172.21.15.1:0/1771222410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc608143bb0 0x7fc608132290 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 >> 172.21.15.1:0/1771222410 conn(0x7fc60806b3c0 msgr2=0x7fc60815ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:25.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.679+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 shutdown_connections 2024-05-09T06:36:25.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.680+0000 7fc60f8ca640 1 -- 172.21.15.1:0/1771222410 wait complete. 2024-05-09T06:36:25.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:36:25.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.808+0000 7f30d799e640 1 Processor -- start 2024-05-09T06:36:25.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.808+0000 7f30d799e640 1 -- start start 2024-05-09T06:36:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f30d0143bb0 0x7f30d00cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f30d01446d0 0x7f30d00d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d015e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00ce8d0 con 0x7f30d01446d0 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00cdd50 con 0x7f30d0143bb0 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d799e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00ce030 con 0x7f30d0144110 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f30d01446d0 0x7f30d00d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f30d01446d0 0x7f30d00d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55630/0 (socket says 172.21.15.1:55630) 2024-05-09T06:36:25.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 -- 172.21.15.1:0/2640487712 learned_addr learned my addr 172.21.15.1:0/2640487712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 -- 172.21.15.1:0/2640487712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 msgr2=0x7f30d015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 --2- 172.21.15.1:0/2640487712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 -- 172.21.15.1:0/2640487712 >> v2:172.21.15.6:3300/0 conn(0x7f30d0143bb0 msgr2=0x7f30d00cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d573a640 1 --2- 172.21.15.1:0/2640487712 >> v2:172.21.15.6:3300/0 conn(0x7f30d0143bb0 0x7f30d00cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 --2- 172.21.15.1:0/2640487712 >> v2:172.21.15.6:3300/0 conn(0x7f30d0143bb0 0x7f30d00cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.809+0000 7f30d4f39640 1 -- 172.21.15.1:0/2640487712 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d00ce310 con 0x7f30d01446d0 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.810+0000 7f30d4f39640 1 --2- 172.21.15.1:0/2640487712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d00d01f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cef80 tx=0x7f30c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4c7eee3311cfa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.810+0000 7f30c7fff640 1 -- 172.21.15.1:0/2640487712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0037070 con 0x7f30d01446d0 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.810+0000 7f30c7fff640 1 -- 172.21.15.1:0/2640487712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c0032440 con 0x7f30d01446d0 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.810+0000 7f30c7fff640 1 -- 172.21.15.1:0/2640487712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0032810 con 0x7f30d01446d0 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 -- 172.21.15.1:0/2640487712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 msgr2=0x7f30d00d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 --2- 172.21.15.1:0/2640487712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d00d01f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cef80 tx=0x7f30c002a140 comp rx=0 tx=0).stop 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 -- 172.21.15.1:0/2640487712 shutdown_connections 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 --2- 172.21.15.1:0/2640487712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 --2- 172.21.15.1:0/2640487712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d00d01f0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 --2- 172.21.15.1:0/2640487712 >> v2:172.21.15.6:3300/0 conn(0x7f30d0143bb0 0x7f30d00cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 -- 172.21.15.1:0/2640487712 >> 172.21.15.1:0/2640487712 conn(0x7f30d006b3c0 msgr2=0x7f30d006b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 -- 172.21.15.1:0/2640487712 shutdown_connections 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.811+0000 7f30d799e640 1 -- 172.21.15.1:0/2640487712 wait complete. 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.812+0000 7f30d799e640 1 Processor -- start 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.812+0000 7f30d799e640 1 -- start start 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30d0143bb0 0x7f30d013dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d0142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00cdaf0 con 0x7f30d01446d0 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00cd970 con 0x7f30d0143bb0 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d799e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f30d015eea0 con 0x7f30d0144110 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55632/0 (socket says 172.21.15.1:55632) 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 -- 172.21.15.1:0/2337640854 learned_addr learned my addr 172.21.15.1:0/2337640854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d573a640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30d0143bb0 0x7f30d013dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d4f39640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d0142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 -- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 msgr2=0x7f30d0142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d0142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 -- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30d0143bb0 msgr2=0x7f30d013dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30d0143bb0 0x7f30d013dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d5f3b640 1 -- 172.21.15.1:0/2337640854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d0158250 con 0x7f30d01446d0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.813+0000 7f30d4f39640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d0142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30d5f3b640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011830 con 0x7f30d01446d0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c8016070 con 0x7f30d01446d0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d0158530 con 0x7f30d01446d0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011bf0 con 0x7f30d01446d0 2024-05-09T06:36:25.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.814+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0157c20 con 0x7f30d01446d0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.815+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0158920 con 0x7f30d01446d0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.815+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30c801b460 con 0x7f30d01446d0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.816+0000 7f30c5ffb640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.816+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6037+0+0 (secure 0 0 0) 0x7f30c808ace0 con 0x7f30d01446d0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.816+0000 7f30d573a640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.817+0000 7f30d573a640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0070b0 tx=0x7f30cc007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:25.820+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30c8059540 con 0x7f30d01446d0 2024-05-09T06:36:25.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.002+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30d0075320 con 0x7f30d01446d0 2024-05-09T06:36:26.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.677+0000 7f30c5ffb640 1 -- 172.21.15.1:0/2337640854 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7f30c8058e40 con 0x7f30d01446d0 2024-05-09T06:36:26.658 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:36:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.681+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 msgr2=0x7f30b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.681+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 0x7f30b406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0070b0 tx=0x7f30cc007040 comp rx=0 tx=0).stop 2024-05-09T06:36:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.681+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 msgr2=0x7f30d01f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.681+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c800b8e0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 shutdown_connections 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f30b406be30 0x7f30b406e2f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d01446d0 0x7f30d01f7a20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30d0144110 0x7f30d0142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 --2- 172.21.15.1:0/2337640854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30d0143bb0 0x7f30d013dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 >> 172.21.15.1:0/2337640854 conn(0x7f30d006b3c0 msgr2=0x7f30d00ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 shutdown_connections 2024-05-09T06:36:26.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.682+0000 7f30d799e640 1 -- 172.21.15.1:0/2337640854 wait complete. 2024-05-09T06:36:26.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:36:26.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.809+0000 7f33c5130640 1 Processor -- start 2024-05-09T06:36:26.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.809+0000 7f33c5130640 1 -- start start 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33c5130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33c015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33c5130640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c0161fe0 con 0x7f33c00d45f0 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33c5130640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c01622c0 con 0x7f33c00d1c70 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33c5130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33c015f580 con 0x7f33c015e190 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33bf59e640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f33c00d45f0 0x7f33c00b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55862/0 (socket says 172.21.15.1:55862) 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.810+0000 7f33bf59e640 1 -- 172.21.15.1:0/790854131 learned_addr learned my addr 172.21.15.1:0/790854131 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33c015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070533053 0 0) 0x7f33c0161fe0 con 0x7f33c00d45f0 2024-05-09T06:36:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33a8003520 con 0x7f33c00d45f0 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663394442 0 0) 0x7f33c01622c0 con 0x7f33c00d1c70 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33c0161fe0 con 0x7f33c00d1c70 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4279070402 0 0) 0x7f33a8003520 con 0x7f33c00d45f0 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 msgr2=0x7f33c015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33c015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 >> v1:172.21.15.6:6789/0 conn(0x7f33c00d1c70 legacy=0x7f33c00d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c01625a0 con 0x7f33c00d45f0 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f33b80019f0 con 0x7f33c00d45f0 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33c015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 >> v1:172.21.15.1:6789/0 conn(0x7f33c00d45f0 legacy=0x7f33c00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33a8003b70 0x7f33a8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33a8004800 0x7f33a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33a800b9c0 con 0x7f33a8003b70 2024-05-09T06:36:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f33a800bb70 con 0x7f33a8004800 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33a800bd70 con 0x7f33c015e190 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33a8003b70 0x7f33a8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.811+0000 7f33be59c640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33a8004800 0x7f33a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bf59e640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 -- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 msgr2=0x7f33a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33c015e190 0x7f33a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 -- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33a8004800 msgr2=0x7f33a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33a8004800 0x7f33a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 -- 172.21.15.1:0/790854131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a8003520 con 0x7f33a8003b70 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.812+0000 7f33bed9d640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33a8003b70 0x7f33a8003f70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009c60 tx=0x7f33a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ddebdcbfd8a00b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0036070 con 0x7f33a8003b70 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b0031400 con 0x7f33a8003b70 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33bdd9b640 1 -- 172.21.15.1:0/790854131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0009860 con 0x7f33a8003b70 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 -- 172.21.15.1:0/790854131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33a8003b70 msgr2=0x7f33a8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 --2- 172.21.15.1:0/790854131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33a8003b70 0x7f33a8003f70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009c60 tx=0x7f33a8005830 comp rx=0 tx=0).stop 2024-05-09T06:36:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33bf59e640 1 -- 172.21.15.1:0/790854131 reap_dead start 2024-05-09T06:36:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 -- 172.21.15.1:0/790854131 shutdown_connections 2024-05-09T06:36:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 -- 172.21.15.1:0/790854131 >> 172.21.15.1:0/790854131 conn(0x7f33c0073340 msgr2=0x7f33c009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 -- 172.21.15.1:0/790854131 shutdown_connections 2024-05-09T06:36:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.813+0000 7f33c5130640 1 -- 172.21.15.1:0/790854131 wait complete. 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.815+0000 7f33c5130640 1 Processor -- start 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.815+0000 7f33c5130640 1 -- start start 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 0x7f33c014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 0x7f33c00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00b56d0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0161c60 con 0x7f33c00b1cc0 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00b5850 con 0x7f33a8003b70 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55650/0 (socket says 172.21.15.1:55650) 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 -- 172.21.15.1:0/2521770219 learned_addr learned my addr 172.21.15.1:0/2521770219 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33bed9d640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 0x7f33c014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33bf59e640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 0x7f33c00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 -- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 msgr2=0x7f33c014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 0x7f33c014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 -- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 msgr2=0x7f33c00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 0x7f33c00b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 -- 172.21.15.1:0/2521770219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c00cf6f0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33bed9d640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 0x7f33c014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33bf59e640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 0x7f33c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33be59c640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f33b400f7a0 tx=0x7f33b400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b400fe50 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b4018070 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c00cf9d0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b400d730 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.816+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c00cf3e0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.818+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00752b0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.818+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33b400d8d0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.818+0000 7f33a77fe640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 0x7f33ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:26.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.819+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f33b408acc0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.819+0000 7f33bed9d640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 0x7f33ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:26.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.819+0000 7f33bed9d640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 0x7f33ac06e320 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:26.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:26.823+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33b4059730 con 0x7f33c00cd2c0 2024-05-09T06:36:26.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.010+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f33c008e1c0 con 0x7f33c00cd2c0 2024-05-09T06:36:26.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.014+0000 7f33a77fe640 1 -- 172.21.15.1:0/2521770219 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f33b4059030 con 0x7f33c00cd2c0 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 msgr2=0x7f33ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 0x7f33ac06e320 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b0008030 comp rx=0 tx=0).stop 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 msgr2=0x7f33c00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f33b400f7a0 tx=0x7f33b400fc60 comp rx=0 tx=0).stop 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 shutdown_connections 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33ac06be60 0x7f33ac06e320 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33c00b1cc0 0x7f33c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33c00cd2c0 0x7f33c00cc810 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 --2- 172.21.15.1:0/2521770219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33a8003b70 0x7f33c014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 >> 172.21.15.1:0/2521770219 conn(0x7f33c0073340 msgr2=0x7f33c015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 shutdown_connections 2024-05-09T06:36:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.016+0000 7f33c5130640 1 -- 172.21.15.1:0/2521770219 wait complete. 2024-05-09T06:36:27.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-09T06:36:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.133+0000 7f738b615640 1 Processor -- start 2024-05-09T06:36:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.133+0000 7f738b615640 1 -- start start 2024-05-09T06:36:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f738b615640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f738b615640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f73840b2a60 0x7f73840b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f738b615640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73840b5230 con 0x7f73840b4da0 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f738b615640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7384071c20 con 0x7f73840b2a60 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f738b615640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f738406eb40 con 0x7f73841454b0 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f73840b2a60 0x7f73840b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f73840b2a60 0x7f73840b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55656/0 (socket says 172.21.15.1:55656) 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 -- 172.21.15.1:0/2920571996 learned_addr learned my addr 172.21.15.1:0/2920571996 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f73893b1640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 -- 172.21.15.1:0/2920571996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 msgr2=0x7f73840b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 -- 172.21.15.1:0/2920571996 >> v1:172.21.15.6:6789/0 conn(0x7f73840b4da0 legacy=0x7f738415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.134+0000 7f7388bb0640 1 -- 172.21.15.1:0/2920571996 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738406ee20 con 0x7f73840b2a60 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.135+0000 7f7388bb0640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840b4860 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7374009870 tx=0x7f737402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37bdf9f85437e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.135+0000 7f737bfff640 1 -- 172.21.15.1:0/2920571996 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374037070 con 0x7f73840b2a60 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.135+0000 7f737bfff640 1 -- 172.21.15.1:0/2920571996 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7374032440 con 0x7f73840b2a60 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.135+0000 7f737bfff640 1 -- 172.21.15.1:0/2920571996 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374032810 con 0x7f73840b2a60 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 -- 172.21.15.1:0/2920571996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 msgr2=0x7f73840b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840b4860 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7374009870 tx=0x7f737402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 -- 172.21.15.1:0/2920571996 shutdown_connections 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840b4860 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 --2- 172.21.15.1:0/2920571996 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 -- 172.21.15.1:0/2920571996 >> 172.21.15.1:0/2920571996 conn(0x7f73840cc4b0 msgr2=0x7f73840cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 -- 172.21.15.1:0/2920571996 shutdown_connections 2024-05-09T06:36:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.136+0000 7f738b615640 1 -- 172.21.15.1:0/2920571996 wait complete. 2024-05-09T06:36:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.137+0000 7f738b615640 1 Processor -- start 2024-05-09T06:36:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.137+0000 7f738b615640 1 -- start start 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 0x7f738413ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7384145890 con 0x7f73840b2a60 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f738406dbc0 con 0x7f73840b4da0 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f738b615640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f738406e0e0 con 0x7f73841454b0 2024-05-09T06:36:27.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55660/0 (socket says 172.21.15.1:55660) 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 -- 172.21.15.1:0/1593720716 learned_addr learned my addr 172.21.15.1:0/1593720716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 -- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 msgr2=0x7f73840caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f7389bb2640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f7388bb0640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 0x7f738413ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 -- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 msgr2=0x7f738413ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 0x7f738413ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f73893b1640 1 -- 172.21.15.1:0/1593720716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73840cdab0 con 0x7f73840b2a60 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f7389bb2640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.138+0000 7f7388bb0640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 0x7f738413ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f73893b1640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f738000d800 tx=0x7f738000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738000b6c0 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380019070 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738000ba80 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73840cdd50 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.139+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73840ce310 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.140+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73840752b0 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.140+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f738000bc20 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.141+0000 7f7379ffb640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 0x7f736806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.141+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f738008b1e0 con 0x7f73840b2a60 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.141+0000 7f7388bb0640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 0x7f736806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.141+0000 7f7388bb0640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 0x7f736806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7374034000 tx=0x7f737402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.145+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7380059ce0 con 0x7f73840b2a60 2024-05-09T06:36:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.404+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f7384072fa0 con 0x7f73840b2a60 2024-05-09T06:36:27.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.679+0000 7f7379ffb640 1 -- 172.21.15.1:0/1593720716 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v431) ==== 167+0+0 (secure 0 0 0) 0x7f73800595e0 con 0x7f73840b2a60 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 msgr2=0x7f736806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 0x7f736806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7374034000 tx=0x7f737402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 msgr2=0x7f73840cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f738000d800 tx=0x7f738000dcc0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 shutdown_connections 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f736806be30 0x7f736806e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73841454b0 0x7f73840caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73840b4da0 0x7f738413ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 --2- 172.21.15.1:0/1593720716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73840b2a60 0x7f73840cd570 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 >> 172.21.15.1:0/1593720716 conn(0x7f73840cc4b0 msgr2=0x7f7384071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:27.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 shutdown_connections 2024-05-09T06:36:27.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.682+0000 7f738b615640 1 -- 172.21.15.1:0/1593720716 wait complete. 2024-05-09T06:36:27.675 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-09T06:36:27.675 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:36:27.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.800+0000 7f8b74a4d640 1 Processor -- start 2024-05-09T06:36:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.800+0000 7f8b74a4d640 1 -- start start 2024-05-09T06:36:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b74a4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b74a4d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8b700b4f70 0x7f8b70070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b74a4d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7006eb40 con 0x7f8b701454b0 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b74a4d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7006ee20 con 0x7f8b700b4f70 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b74a4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7006d8e0 con 0x7f8b700b2a60 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b6ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8b700b4f70 0x7f8b70070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b6ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8b700b4f70 0x7f8b70070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:57592/0 (socket says 172.21.15.1:57592) 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b67fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:46248/0 (socket says 172.21.15.1:46248) 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.801+0000 7f8b6ed9d640 1 -- 172.21.15.1:0/3530648516 learned_addr learned my addr 172.21.15.1:0/3530648516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6ed9d640 1 -- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 msgr2=0x7f8b700b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6ed9d640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6ed9d640 1 -- 172.21.15.1:0/3530648516 >> v1:172.21.15.6:6789/0 conn(0x7f8b701454b0 legacy=0x7f8b700b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6dd9b640 1 -- 172.21.15.1:0/3530648516 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873431846 0 0) 0x7f8b7006eb40 con 0x7f8b701454b0 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6ed9d640 1 -- 172.21.15.1:0/3530648516 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7006dbc0 con 0x7f8b700b4f70 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b67fff640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.802+0000 7f8b6ed9d640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b70070c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e50 tx=0x7f8b6802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12aa9533afa8d02f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b6dd9b640 1 -- 172.21.15.1:0/3530648516 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68036070 con 0x7f8b700b4f70 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b6dd9b640 1 -- 172.21.15.1:0/3530648516 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b68031600 con 0x7f8b700b4f70 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b6dd9b640 1 -- 172.21.15.1:0/3530648516 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680319d0 con 0x7f8b700b4f70 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 -- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 msgr2=0x7f8b70070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b70070c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e50 tx=0x7f8b6802a710 comp rx=0 tx=0).stop 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 -- 172.21.15.1:0/3530648516 shutdown_connections 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b70070c10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/3530648516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b700b2a60 0x7f8b700b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 -- 172.21.15.1:0/3530648516 >> 172.21.15.1:0/3530648516 conn(0x7f8b700cc4b0 msgr2=0x7f8b700cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 -- 172.21.15.1:0/3530648516 shutdown_connections 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.803+0000 7f8b74a4d640 1 -- 172.21.15.1:0/3530648516 wait complete. 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.804+0000 7f8b74a4d640 1 Processor -- start 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 -- start start 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b700ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b701454b0 0x7f8b700c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7006e260 con 0x7f8b700b2a60 2024-05-09T06:36:27.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7006e0e0 con 0x7f8b700b4f70 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b74a4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8b70071150 con 0x7f8b701454b0 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b6e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b6e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42252/0 (socket says 172.21.15.1:42252) 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.805+0000 7f8b6e59c640 1 -- 172.21.15.1:0/1956618354 learned_addr learned my addr 172.21.15.1:0/1956618354 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b67fff640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b700ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6ed9d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b701454b0 0x7f8b700c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 -- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b701454b0 msgr2=0x7f8b700c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b701454b0 0x7f8b700c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 -- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 msgr2=0x7f8b700ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b700ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 -- 172.21.15.1:0/1956618354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7013b4e0 con 0x7f8b700b2a60 2024-05-09T06:36:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b6e59c640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c410 tx=0x7f8b6000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b600035f0 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b70135a20 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600039e0 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.806+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b70135fe0 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.807+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b700752b0 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.808+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b6001a070 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.808+0000 7f8b66ffd640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 0x7f8b5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.808+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f8b6008a180 con 0x7f8b700b2a60 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.808+0000 7f8b67fff640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 0x7f8b5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.809+0000 7f8b67fff640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 0x7f8b5006e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009560 tx=0x7f8b58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.813+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b60058bf0 con 0x7f8b700b2a60 2024-05-09T06:36:27.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.998+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b70072fa0 con 0x7f8b700b2a60 2024-05-09T06:36:27.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:27.999+0000 7f8b66ffd640 1 -- 172.21.15.1:0/1956618354 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+13412 (secure 0 0 0) 0x7f8b600584f0 con 0x7f8b700b2a60 2024-05-09T06:36:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:27.979 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":98,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:26.674174+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6836/779210140":"2024-05-10T06:36:23.583523+0000","172.21.15.1:6834/779210140":"2024-05-10T06:36:23.583523+0000","172.21.15.1:6836/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.1:6834/424906117":"2024-05-10T06:35:39.452614+0000","172.21.15.6:6835/1974102470":"2024-05-10T06:34:19.713641+0000","172.21.15.6:6833/1974102470":"2024-05-10T06:34:19.713641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 msgr2=0x7f8b5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 0x7f8b5006e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009560 tx=0x7f8b58008040 comp rx=0 tx=0).stop 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 msgr2=0x7f8b7013afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c410 tx=0x7f8b6000c8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 shutdown_connections 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8b5006be60 0x7f8b5006e320 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b701454b0 0x7f8b700c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b700b4f70 0x7f8b700ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 --2- 172.21.15.1:0/1956618354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b700b2a60 0x7f8b7013afa0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 >> 172.21.15.1:0/1956618354 conn(0x7f8b700cc4b0 msgr2=0x7f8b70143a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:27.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 shutdown_connections 2024-05-09T06:36:27.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.001+0000 7f8b74a4d640 1 -- 172.21.15.1:0/1956618354 wait complete. 2024-05-09T06:36:27.993 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:27.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:28.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.120+0000 7f69582ba640 1 Processor -- start 2024-05-09T06:36:28.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.120+0000 7f69582ba640 1 -- start start 2024-05-09T06:36:28.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f69582ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f695015e270 0x7f69500cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f69582ba640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69500d08b0 0x7f69500ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f69582ba640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69500b1340 con 0x7f69500ce9d0 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f69582ba640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69500cc260 con 0x7f69500d08b0 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f69582ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69500d1020 con 0x7f695015e270 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955855640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69500d08b0 0x7f69500ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6956857640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69500ce9d0 0x7f695006c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46912/0 (socket says 172.21.15.1:46912) 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6956857640 1 -- 172.21.15.1:0/2150634466 learned_addr learned my addr 172.21.15.1:0/2150634466 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:28.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6956056640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f695015e270 0x7f69500cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955855640 1 -- 172.21.15.1:0/2150634466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f695015e270 msgr2=0x7f69500cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955855640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f695015e270 0x7f69500cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955855640 1 -- 172.21.15.1:0/2150634466 >> v1:172.21.15.1:6789/0 conn(0x7f69500ce9d0 legacy=0x7f695006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955855640 1 -- 172.21.15.1:0/2150634466 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69500cc4b0 con 0x7f69500d08b0 2024-05-09T06:36:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.121+0000 7f6955054640 1 -- 172.21.15.1:0/2150634466 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384473275 0 0) 0x7f6948001490 con 0x7f69500ce9d0 2024-05-09T06:36:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.122+0000 7f6955855640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500d08b0 0x7f69500ce490 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f69440097a0 tx=0x7f694402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed62d40ab116702b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.122+0000 7f6955054640 1 -- 172.21.15.1:0/2150634466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944037070 con 0x7f69500d08b0 2024-05-09T06:36:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.122+0000 7f6955054640 1 -- 172.21.15.1:0/2150634466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6944032440 con 0x7f69500d08b0 2024-05-09T06:36:28.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.122+0000 7f6955054640 1 -- 172.21.15.1:0/2150634466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944032810 con 0x7f69500d08b0 2024-05-09T06:36:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 -- 172.21.15.1:0/2150634466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500d08b0 msgr2=0x7f69500ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500d08b0 0x7f69500ce490 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f69440097a0 tx=0x7f694402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 -- 172.21.15.1:0/2150634466 shutdown_connections 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500d08b0 0x7f69500ce490 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 --2- 172.21.15.1:0/2150634466 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f695015e270 0x7f69500cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 -- 172.21.15.1:0/2150634466 >> 172.21.15.1:0/2150634466 conn(0x7f69500d1c10 msgr2=0x7f69500d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 -- 172.21.15.1:0/2150634466 shutdown_connections 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.123+0000 7f69582ba640 1 -- 172.21.15.1:0/2150634466 wait complete. 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.124+0000 7f69582ba640 1 Processor -- start 2024-05-09T06:36:28.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.124+0000 7f69582ba640 1 -- start start 2024-05-09T06:36:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500d08b0 0x7f69501f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f695015e270 0x7f6950151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f695015e650 con 0x7f69500ce9d0 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69500cca70 con 0x7f695015e270 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69500d0cf0 con 0x7f69500d08b0 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42264/0 (socket says 172.21.15.1:42264) 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 -- 172.21.15.1:0/631391737 learned_addr learned my addr 172.21.15.1:0/631391737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 -- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500d08b0 msgr2=0x7f69501f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6955855640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500d08b0 0x7f69501f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500d08b0 0x7f69501f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 -- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f695015e270 msgr2=0x7f6950151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f695015e270 0x7f6950151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 -- 172.21.15.1:0/631391737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69500c6e50 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6956056640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d800 tx=0x7f694c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00b6c0 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694c019070 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69500c7130 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00ba80 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.125+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69500c76f0 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.126+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69500752b0 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.127+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f694c00bc20 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.127+0000 7f6942ffd640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 0x7f693406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.128+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f694c08b2b0 con 0x7f69500ce9d0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.128+0000 7f6955855640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 0x7f693406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.128+0000 7f6955855640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 0x7f693406e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6944034000 tx=0x7f694402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.132+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f694c01c360 con 0x7f69500ce9d0 2024-05-09T06:36:28.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.316+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69500d29e0 con 0x7f69500ce9d0 2024-05-09T06:36:28.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.317+0000 7f6942ffd640 1 -- 172.21.15.1:0/631391737 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f694c059c80 con 0x7f69500ce9d0 2024-05-09T06:36:28.298 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 msgr2=0x7f693406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 0x7f693406e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6944034000 tx=0x7f694402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 msgr2=0x7f69500c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d800 tx=0x7f694c00dcc0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 shutdown_connections 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f693406be60 0x7f693406e320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f695015e270 0x7f6950151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69500d08b0 0x7f69501f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 --2- 172.21.15.1:0/631391737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69500ce9d0 0x7f69500c6750 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 >> 172.21.15.1:0/631391737 conn(0x7f69500d1c10 msgr2=0x7f69500affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:28.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 shutdown_connections 2024-05-09T06:36:28.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.319+0000 7f69582ba640 1 -- 172.21.15.1:0/631391737 wait complete. 2024-05-09T06:36:28.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-09T06:36:28.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.434+0000 7f80783b0640 1 Processor -- start 2024-05-09T06:36:28.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.434+0000 7f80783b0640 1 -- start start 2024-05-09T06:36:28.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f80783b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f807015c190 0x7f80700cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f80783b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f80700d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f80783b0640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80700cdbf0 con 0x7f80700cfd60 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f80783b0640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f80700c5c20 con 0x7f807015c190 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f80783b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80700cd630 con 0x7f80700c70c0 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f807614c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f807015c190 0x7f80700cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f807594b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f80700cfd60 0x7f80700ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46914/0 (socket says 172.21.15.1:46914) 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.435+0000 7f807594b640 1 -- 172.21.15.1:0/1077355266 learned_addr learned my addr 172.21.15.1:0/1077355266 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807694d640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f80700d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807614c640 1 -- 172.21.15.1:0/1077355266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 msgr2=0x7f80700d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807614c640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f80700d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807614c640 1 -- 172.21.15.1:0/1077355266 >> v1:172.21.15.1:6789/0 conn(0x7f80700cfd60 legacy=0x7f80700ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807614c640 1 -- 172.21.15.1:0/1077355266 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80700cd910 con 0x7f807015c190 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807514a640 1 -- 172.21.15.1:0/1077355266 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153044245 0 0) 0x7f80700cdbf0 con 0x7f80700cfd60 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.436+0000 7f807614c640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cf820 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f80680097a0 tx=0x7f806802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b43167795eeb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f807514a640 1 -- 172.21.15.1:0/1077355266 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068037070 con 0x7f807015c190 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f807514a640 1 -- 172.21.15.1:0/1077355266 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8068032440 con 0x7f807015c190 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f807514a640 1 -- 172.21.15.1:0/1077355266 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068032810 con 0x7f807015c190 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 -- 172.21.15.1:0/1077355266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 msgr2=0x7f80700cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cf820 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f80680097a0 tx=0x7f806802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 -- 172.21.15.1:0/1077355266 shutdown_connections 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f80700d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 --2- 172.21.15.1:0/1077355266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cf820 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 -- 172.21.15.1:0/1077355266 >> 172.21.15.1:0/1077355266 conn(0x7f80700d1c50 msgr2=0x7f80700d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:28.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 -- 172.21.15.1:0/1077355266 shutdown_connections 2024-05-09T06:36:28.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.437+0000 7f80783b0640 1 -- 172.21.15.1:0/1077355266 wait complete. 2024-05-09T06:36:28.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 Processor -- start 2024-05-09T06:36:28.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 -- start start 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f8070157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80700cfd60 0x7f80701f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80700cd270 con 0x7f807015c190 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f80700d0a30 con 0x7f80700cfd60 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f80783b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80700cd3f0 con 0x7f80700c70c0 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f807694d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f807694d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42278/0 (socket says 172.21.15.1:42278) 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.439+0000 7f807694d640 1 -- 172.21.15.1:0/2327867614 learned_addr learned my addr 172.21.15.1:0/2327867614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:28.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807614c640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f8070157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807594b640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80700cfd60 0x7f80701f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 -- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 msgr2=0x7f8070157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f8070157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 -- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80700cfd60 msgr2=0x7f80701f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80700cfd60 0x7f80701f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 -- 172.21.15.1:0/2327867614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80700b74c0 con 0x7f807015c190 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f807694d640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f806000efc0 tx=0x7f806000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060018020 con 0x7f807015c190 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8060014480 con 0x7f807015c190 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807009bee0 con 0x7f807015c190 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060009e60 con 0x7f807015c190 2024-05-09T06:36:28.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.440+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807009c4a0 con 0x7f807015c190 2024-05-09T06:36:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.441+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80700752b0 con 0x7f807015c190 2024-05-09T06:36:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.442+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8060005d10 con 0x7f807015c190 2024-05-09T06:36:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.442+0000 7f806effd640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 0x7f805c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.443+0000 7f807614c640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 0x7f805c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.443+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f806008b930 con 0x7f807015c190 2024-05-09T06:36:28.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.443+0000 7f807614c640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 0x7f805c06e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8068034000 tx=0x7f806802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:28.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.447+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8060022030 con 0x7f807015c190 2024-05-09T06:36:28.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:28.700+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f80700d24d0 con 0x7f807015c190 2024-05-09T06:36:29.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.686+0000 7f806effd640 1 -- 172.21.15.1:0/2327867614 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v433) ==== 149+0+0 (secure 0 0 0) 0x7f8060010040 con 0x7f807015c190 2024-05-09T06:36:29.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 msgr2=0x7f805c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 0x7f805c06e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8068034000 tx=0x7f806802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:29.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 msgr2=0x7f80700cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f806000efc0 tx=0x7f806000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 shutdown_connections 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f805c06be60 0x7f805c06e320 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f807015c190 0x7f80700cadb0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80700cfd60 0x7f80701f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 --2- 172.21.15.1:0/2327867614 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80700c70c0 0x7f8070157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.688+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 >> 172.21.15.1:0/2327867614 conn(0x7f80700d1c50 msgr2=0x7f80700c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.689+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 shutdown_connections 2024-05-09T06:36:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.689+0000 7f80783b0640 1 -- 172.21.15.1:0/2327867614 wait complete. 2024-05-09T06:36:29.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:36:29.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.816+0000 7f4b6fd70640 1 Processor -- start 2024-05-09T06:36:29.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.816+0000 7f4b6fd70640 1 -- start start 2024-05-09T06:36:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6fd70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4b6815c4c0 0x7f4b6806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6fd70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b68070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6fd70640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6806cf40 con 0x7f4b680b4de0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6fd70640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4b68156980 con 0x7f4b6815c4c0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6fd70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6806d870 con 0x7f4b6815cfb0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6d30b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4b680b4de0 0x7f4b6815c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46920/0 (socket says 172.21.15.1:46920) 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6d30b640 1 -- 172.21.15.1:0/44487767 learned_addr learned my addr 172.21.15.1:0/44487767 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6e30d640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b68070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665142567 0 0) 0x7f4b6806cf40 con 0x7f4b680b4de0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b54003520 con 0x7f4b680b4de0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6db0c640 1 --2- 172.21.15.1:0/44487767 >> v2:172.21.15.6:3300/0 conn(0x7f4b6815c4c0 0x7f4b6806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835525964 0 0) 0x7f4b54003520 con 0x7f4b680b4de0 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 msgr2=0x7f4b68070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b68070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 >> v2:172.21.15.6:3300/0 conn(0x7f4b6815c4c0 msgr2=0x7f4b6806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 --2- 172.21.15.1:0/44487767 >> v2:172.21.15.6:3300/0 conn(0x7f4b6815c4c0 0x7f4b6806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6806d220 con 0x7f4b680b4de0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.817+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f4b58001670 con 0x7f4b680b4de0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 >> v1:172.21.15.1:6789/0 conn(0x7f4b680b4de0 legacy=0x7f4b6815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b540039b0 0x7f4b54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b54004800 0x7f4b54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6806d870 con 0x7f4b54004800 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5400b9c0 con 0x7f4b540039b0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5400bb40 con 0x7f4b6815cfb0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b54004800 0x7f4b54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6db0c640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b540039b0 0x7f4b54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6e30d640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 -- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 msgr2=0x7f4b540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b6815cfb0 0x7f4b540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 -- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b540039b0 msgr2=0x7f4b54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b540039b0 0x7f4b54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.818+0000 7f4b6d30b640 1 -- 172.21.15.1:0/44487767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b54003520 con 0x7f4b54004800 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b54004800 0x7f4b54004c80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c2c0 tx=0x7f4b54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57c67259ec45fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58007d50 con 0x7f4b54004800 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5802e070 con 0x7f4b54004800 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6cb0a640 1 -- 172.21.15.1:0/44487767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58005d30 con 0x7f4b54004800 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6fd70640 1 -- 172.21.15.1:0/44487767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b54004800 msgr2=0x7f4b54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/44487767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b54004800 0x7f4b54004c80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800c2c0 tx=0x7f4b54005830 comp rx=0 tx=0).stop 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.819+0000 7f4b6e30d640 1 -- 172.21.15.1:0/44487767 reap_dead start 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.820+0000 7f4b6fd70640 1 -- 172.21.15.1:0/44487767 shutdown_connections 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.820+0000 7f4b6fd70640 1 -- 172.21.15.1:0/44487767 >> 172.21.15.1:0/44487767 conn(0x7f4b680cc4d0 msgr2=0x7f4b680cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.820+0000 7f4b6fd70640 1 -- 172.21.15.1:0/44487767 shutdown_connections 2024-05-09T06:36:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.820+0000 7f4b6fd70640 1 -- 172.21.15.1:0/44487767 wait complete. 2024-05-09T06:36:29.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.821+0000 7f4b6fd70640 1 Processor -- start 2024-05-09T06:36:29.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.821+0000 7f4b6fd70640 1 -- start start 2024-05-09T06:36:29.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b54004800 0x7f4b681f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 0x7f4b680b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b680b2dc0 con 0x7f4b6815b3c0 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b680b2c40 con 0x7f4b680cae30 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4b680b5260 con 0x7f4b54004800 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42290/0 (socket says 172.21.15.1:42290) 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 -- 172.21.15.1:0/538925197 learned_addr learned my addr 172.21.15.1:0/538925197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6db0c640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b54004800 0x7f4b681f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6e30d640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 0x7f4b680b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 -- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b54004800 msgr2=0x7f4b681f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b54004800 0x7f4b681f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 -- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 msgr2=0x7f4b680b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 0x7f4b680b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 -- 172.21.15.1:0/538925197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b681597f0 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6e30d640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 0x7f4b680b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6d30b640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4b58034040 tx=0x7f4b6806d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58041070 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b58007280 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b68159ad0 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58001c80 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.822+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b681f3ec0 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.824+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b680752b0 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.824+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b58044070 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.824+0000 7f4b5e7fc640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 0x7f4b4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.825+0000 7f4b6db0c640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 0x7f4b4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.825+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f4b580aac70 con 0x7f4b6815b3c0 2024-05-09T06:36:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.825+0000 7f4b6db0c640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 0x7f4b4c06e320 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b680cc320 tx=0x7f4b64009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:29.829+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b5804c020 con 0x7f4b6815b3c0 2024-05-09T06:36:29.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.012+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4b68071050 con 0x7f4b6815b3c0 2024-05-09T06:36:29.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.015+0000 7f4b5e7fc640 1 -- 172.21.15.1:0/538925197 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f4b5803b330 con 0x7f4b6815b3c0 2024-05-09T06:36:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 msgr2=0x7f4b4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 0x7f4b4c06e320 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b680cc320 tx=0x7f4b64009190 comp rx=0 tx=0).stop 2024-05-09T06:36:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 msgr2=0x7f4b68159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4b58034040 tx=0x7f4b6806d7e0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 shutdown_connections 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4b4c06be60 0x7f4b4c06e320 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b680cae30 0x7f4b680b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b6815b3c0 0x7f4b68159080 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 --2- 172.21.15.1:0/538925197 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4b54004800 0x7f4b681f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 >> 172.21.15.1:0/538925197 conn(0x7f4b680cc4d0 msgr2=0x7f4b68071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 shutdown_connections 2024-05-09T06:36:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.017+0000 7f4b6fd70640 1 -- 172.21.15.1:0/538925197 wait complete. 2024-05-09T06:36:30.009 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:36:30.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:30.049 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:30.095 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:36:30.095 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:30.096 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:30.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:36:30.112 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:36:30.112 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:30.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:36:30.132 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:30.132 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:30.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:36:30.169 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:36:30.169 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:30.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:36:30.189 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:36:30.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:30.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.332+0000 7f699eba5640 1 Processor -- start 2024-05-09T06:36:30.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.332+0000 7f699eba5640 1 -- start start 2024-05-09T06:36:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699eba5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69980c6860 0x7f6998099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699eba5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69980b2aa0 0x7f699815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699eba5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6998099930 con 0x7f69980c6c40 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699eba5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69980b1e20 con 0x7f69980c6860 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699eba5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f699815ee90 con 0x7f69980b2aa0 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69980c6860 0x7f6998099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f69980c6860 0x7f6998099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42296/0 (socket says 172.21.15.1:42296) 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 -- 172.21.15.1:0/1808459281 learned_addr learned my addr 172.21.15.1:0/1808459281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 -- 172.21.15.1:0/1808459281 >> v2:172.21.15.6:3300/0 conn(0x7f69980b2aa0 msgr2=0x7f699815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 --2- 172.21.15.1:0/1808459281 >> v2:172.21.15.6:3300/0 conn(0x7f69980b2aa0 0x7f699815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 -- 172.21.15.1:0/1808459281 >> v1:172.21.15.6:6789/0 conn(0x7f69980c6c40 legacy=0x7f699815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.333+0000 7f699c941640 1 -- 172.21.15.1:0/1808459281 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6998162930 con 0x7f69980c6860 2024-05-09T06:36:30.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.334+0000 7f699c941640 1 --2- 172.21.15.1:0/1808459281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980c6860 0x7f6998099b30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f699809e570 tx=0x7f698c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3be16463a8f582 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.334+0000 7f69977fe640 1 -- 172.21.15.1:0/1808459281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037070 con 0x7f69980c6860 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.334+0000 7f69977fe640 1 -- 172.21.15.1:0/1808459281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c032440 con 0x7f69980c6860 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.334+0000 7f69977fe640 1 -- 172.21.15.1:0/1808459281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c032810 con 0x7f69980c6860 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 -- 172.21.15.1:0/1808459281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980c6860 msgr2=0x7f6998099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 --2- 172.21.15.1:0/1808459281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980c6860 0x7f6998099b30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f699809e570 tx=0x7f698c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 -- 172.21.15.1:0/1808459281 shutdown_connections 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 --2- 172.21.15.1:0/1808459281 >> v2:172.21.15.6:3300/0 conn(0x7f69980b2aa0 0x7f699815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 --2- 172.21.15.1:0/1808459281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980c6860 0x7f6998099b30 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 -- 172.21.15.1:0/1808459281 >> 172.21.15.1:0/1808459281 conn(0x7f69980b1fb0 msgr2=0x7f69980b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 -- 172.21.15.1:0/1808459281 shutdown_connections 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.335+0000 7f699eba5640 1 -- 172.21.15.1:0/1808459281 wait complete. 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.336+0000 7f699eba5640 1 Processor -- start 2024-05-09T06:36:30.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.336+0000 7f699eba5640 1 -- start start 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69980c6860 0x7f69980c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69980c6c40 0x7f69981f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f699815eb20 con 0x7f69980b2aa0 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f699815fa30 con 0x7f69980c6c40 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699eba5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69980c70d0 con 0x7f69980c6860 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42304/0 (socket says 172.21.15.1:42304) 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 -- 172.21.15.1:0/826662216 learned_addr learned my addr 172.21.15.1:0/826662216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699d142640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69980c6c40 0x7f69981f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f6997fff640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69980c6860 0x7f69980c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 -- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69980c6860 msgr2=0x7f69980c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69980c6860 0x7f69980c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 -- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69980c6c40 msgr2=0x7f69981f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69980c6c40 0x7f69981f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 -- 172.21.15.1:0/826662216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69980c7b50 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.337+0000 7f699c941640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.338+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c041070 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.338+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c03c870 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.338+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69980c7e30 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.338+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037040 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.338+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981f4580 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.339+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69981f3e50 con 0x7f69980b2aa0 2024-05-09T06:36:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.339+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f698c044070 con 0x7f69980b2aa0 2024-05-09T06:36:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.340+0000 7f69957fa640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 0x7f697c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.340+0000 7f6997fff640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 0x7f697c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.340+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f698c0abb30 con 0x7f69980b2aa0 2024-05-09T06:36:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.340+0000 7f6997fff640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 0x7f697c06e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f699809e570 tx=0x7f6984008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.344+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6998099930 con 0x7f69980b2aa0 2024-05-09T06:36:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.571+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69980752b0 con 0x7f69980b2aa0 2024-05-09T06:36:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.571+0000 7f69957fa640 1 -- 172.21.15.1:0/826662216 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3857 (secure 0 0 0) 0x7f698c07a750 con 0x7f69980b2aa0 2024-05-09T06:36:30.552 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:30.552 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6164,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2790132177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":2790132177},{"type":"v1","addr":"172.21.15.1:6837","nonce":2790132177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6757,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/3520861965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3520861965},{"type":"v1","addr":"172.21.15.6:6834","nonce":3520861965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3651284286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":3651284286},{"type":"v1","addr":"172.21.15.6:6835","nonce":3651284286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":8781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1217978433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1217978433},{"type":"v1","addr":"172.21.15.1:6836","nonce":1217978433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-05-09T06:36:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 76 2024-05-09T06:36:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 msgr2=0x7f697c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 0x7f697c06e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f699809e570 tx=0x7f6984008040 comp rx=0 tx=0).stop 2024-05-09T06:36:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 msgr2=0x7f69980c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 shutdown_connections 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f697c06be30 0x7f697c06e2f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69980c6c40 0x7f69981f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69980c6860 0x7f69980c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 --2- 172.21.15.1:0/826662216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69980b2aa0 0x7f69980c9610 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.573+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 >> 172.21.15.1:0/826662216 conn(0x7f69980b1fb0 msgr2=0x7f699806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.574+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 shutdown_connections 2024-05-09T06:36:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.574+0000 7f699eba5640 1 -- 172.21.15.1:0/826662216 wait complete. 2024-05-09T06:36:30.565 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:36:30.566 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:36:30.566 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:36:30.566 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:30.566 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:36:30.566 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:36:30.566 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:36:30.566 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:30.567 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:36:30.567 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:36:30.567 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:36:30.567 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:30.567 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:36:30.567 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:36:30.567 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:36:30.567 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:30.576 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:36:30.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:36:30.577 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:36:30.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:36:30.579 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:36:30.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:36:30.580 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:36:30.580 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:36:30.619 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:36:30.620 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:30.620 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:36:30.621 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:30.622 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:36:30.622 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:30.626 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:36:30.626 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:30.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:36:30.697 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:36:30.699 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:36:30.708 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:36:30.709 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:36:30.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.756+0000 7f189ff5a640 1 Processor -- start 2024-05-09T06:36:30.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.756+0000 7f189ff5a640 1 -- start start 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ff5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 0x7f18980d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ff5a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 0x7f18980d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ff5a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18980b33c0 con 0x7f18980d4940 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ff5a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f18980d4dd0 con 0x7f18980d0f80 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ff5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18980b3660 con 0x7f1898161950 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189d4f5640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f18980d4940 0x7f18980cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46930/0 (socket says 172.21.15.1:46930) 2024-05-09T06:36:30.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189d4f5640 1 -- 172.21.15.1:0/3654912654 learned_addr learned my addr 172.21.15.1:0/3654912654 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189dcf6640 1 --2- 172.21.15.1:0/3654912654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 0x7f18980d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189e4f7640 1 --2- 172.21.15.1:0/3654912654 >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 0x7f18980d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ccf4640 1 -- 172.21.15.1:0/3654912654 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399497046 0 0) 0x7f18980b33c0 con 0x7f18980d4940 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189ccf4640 1 -- 172.21.15.1:0/3654912654 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1884003520 con 0x7f18980d4940 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189dcf6640 1 -- 172.21.15.1:0/3654912654 >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 msgr2=0x7f18980d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189dcf6640 1 --2- 172.21.15.1:0/3654912654 >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 0x7f18980d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189dcf6640 1 -- 172.21.15.1:0/3654912654 >> v1:172.21.15.1:6789/0 conn(0x7f18980d4940 legacy=0x7f18980cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189dcf6640 1 -- 172.21.15.1:0/3654912654 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18980b3940 con 0x7f1898161950 2024-05-09T06:36:30.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.757+0000 7f189e4f7640 1 --2- 172.21.15.1:0/3654912654 >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 0x7f18980d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.758+0000 7f189dcf6640 1 --2- 172.21.15.1:0/3654912654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 0x7f18980d2400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18980b3590 tx=0x7f189402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a1ef4f4e44e9774 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.758+0000 7f189ccf4640 1 -- 172.21.15.1:0/3654912654 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894037070 con 0x7f1898161950 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.758+0000 7f189ccf4640 1 -- 172.21.15.1:0/3654912654 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894032440 con 0x7f1898161950 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.758+0000 7f189ccf4640 1 -- 172.21.15.1:0/3654912654 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894032810 con 0x7f1898161950 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 -- 172.21.15.1:0/3654912654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 msgr2=0x7f18980d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 --2- 172.21.15.1:0/3654912654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 0x7f18980d2400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18980b3590 tx=0x7f189402a180 comp rx=0 tx=0).stop 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 -- 172.21.15.1:0/3654912654 shutdown_connections 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 --2- 172.21.15.1:0/3654912654 >> v2:172.21.15.6:3300/0 conn(0x7f18980d0f80 0x7f18980d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 --2- 172.21.15.1:0/3654912654 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1898161950 0x7f18980d2400 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 -- 172.21.15.1:0/3654912654 >> 172.21.15.1:0/3654912654 conn(0x7f18980d3e20 msgr2=0x7f18980d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 -- 172.21.15.1:0/3654912654 shutdown_connections 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.759+0000 7f189ff5a640 1 -- 172.21.15.1:0/3654912654 wait complete. 2024-05-09T06:36:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.760+0000 7f189ff5a640 1 Processor -- start 2024-05-09T06:36:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.760+0000 7f189ff5a640 1 -- start start 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d0f80 0x7f189809dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f18980caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18980b52d0 con 0x7f18980b6a10 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f18980b5150 con 0x7f1898161950 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189ff5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18980b5450 con 0x7f18980d0f80 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42348/0 (socket says 172.21.15.1:42348) 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 -- 172.21.15.1:0/2547547323 learned_addr learned my addr 172.21.15.1:0/2547547323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189dcf6640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d0f80 0x7f189809dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189d4f5640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f18980caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 -- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d0f80 msgr2=0x7f189809dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d0f80 0x7f189809dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 -- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 msgr2=0x7f18980caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f18980caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.761+0000 7f189e4f7640 1 -- 172.21.15.1:0/2547547323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18980cb740 con 0x7f18980b6a10 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f189e4f7640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f189000efb0 tx=0x7f189000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890018020 con 0x7f18980b6a10 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890014480 con 0x7f18980b6a10 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18980cba20 con 0x7f18980b6a10 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890014850 con 0x7f18980b6a10 2024-05-09T06:36:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.762+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18980cbfe0 con 0x7f18980b6a10 2024-05-09T06:36:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.763+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18980752b0 con 0x7f18980b6a10 2024-05-09T06:36:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.763+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1890005d10 con 0x7f18980b6a10 2024-05-09T06:36:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.764+0000 7f188e7fc640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.764+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7f189008bb50 con 0x7f18980b6a10 2024-05-09T06:36:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.764+0000 7f189dcf6640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.764+0000 7f189dcf6640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.768+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1890022030 con 0x7f18980b6a10 2024-05-09T06:36:30.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:30.963+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f189809e4e0 con 0x7f18980b6a10 2024-05-09T06:36:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.697+0000 7f188e7fc640 1 -- 172.21.15.1:0/2547547323 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) ==== 97+0+0 (secure 0 0 0) 0x7f1890010040 con 0x7f18980b6a10 2024-05-09T06:36:31.678 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:36:31.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.700+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 msgr2=0x7f187c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.700+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.700+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 msgr2=0x7f18980b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.700+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f189000efb0 tx=0x7f189000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 shutdown_connections 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18980b6a10 0x7f18980b8060 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1898161950 0x7f18980caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 --2- 172.21.15.1:0/2547547323 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18980d0f80 0x7f189809dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 >> 172.21.15.1:0/2547547323 conn(0x7f18980d3e20 msgr2=0x7f189806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 shutdown_connections 2024-05-09T06:36:31.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.701+0000 7f189ff5a640 1 -- 172.21.15.1:0/2547547323 wait complete. 2024-05-09T06:36:31.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:36:31.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.826+0000 7f176f5ed640 1 Processor -- start 2024-05-09T06:36:31.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.826+0000 7f176f5ed640 1 -- start start 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176f5ed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176f5ed640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 0x7f17680b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176f5ed640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17680b33c0 con 0x7f1768161950 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176f5ed640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f17680d4dd0 con 0x7f17680d2b10 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176f5ed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f17680b3660 con 0x7f17680d4940 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176cb88640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176db8a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 0x7f17680b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176cb88640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:46394/0 (socket says 172.21.15.1:46394) 2024-05-09T06:36:31.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176cb88640 1 -- 172.21.15.1:0/146654007 learned_addr learned my addr 172.21.15.1:0/146654007 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.827+0000 7f176d389640 1 --1- 172.21.15.1:0/146654007 >> v1:172.21.15.6:6789/0 conn(0x7f1768161950 0x7f17680d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:55948/0 (socket says 172.21.15.1:55948) 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176cb88640 1 -- 172.21.15.1:0/146654007 >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 msgr2=0x7f17680b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176cb88640 1 --2- 172.21.15.1:0/146654007 >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 0x7f17680b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176cb88640 1 -- 172.21.15.1:0/146654007 >> v1:172.21.15.6:6789/0 conn(0x7f1768161950 legacy=0x7f17680d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176cb88640 1 -- 172.21.15.1:0/146654007 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17680b3940 con 0x7f17680d4940 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176db8a640 1 --2- 172.21.15.1:0/146654007 >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 0x7f17680b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f176cb88640 1 --2- 172.21.15.1:0/146654007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1758009870 tx=0x7f175802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4e4d8cd453ffaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.828+0000 7f175ffff640 1 -- 172.21.15.1:0/146654007 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758037070 con 0x7f17680d4940 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f175ffff640 1 -- 172.21.15.1:0/146654007 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1758032440 con 0x7f17680d4940 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f175ffff640 1 -- 172.21.15.1:0/146654007 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758032810 con 0x7f17680d4940 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 -- 172.21.15.1:0/146654007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 msgr2=0x7f17680b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 --2- 172.21.15.1:0/146654007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1758009870 tx=0x7f175802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:31.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 -- 172.21.15.1:0/146654007 shutdown_connections 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 --2- 172.21.15.1:0/146654007 >> v2:172.21.15.6:3300/0 conn(0x7f17680d2b10 0x7f17680b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 --2- 172.21.15.1:0/146654007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680d4940 0x7f17680b4140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 -- 172.21.15.1:0/146654007 >> 172.21.15.1:0/146654007 conn(0x7f17680d3e20 msgr2=0x7f17680d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 -- 172.21.15.1:0/146654007 shutdown_connections 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.829+0000 7f176f5ed640 1 -- 172.21.15.1:0/146654007 wait complete. 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.830+0000 7f176f5ed640 1 Processor -- start 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.830+0000 7f176f5ed640 1 -- start start 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1768161950 0x7f17680b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680b6a10 0x7f176806c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17680d1100 con 0x7f17680d2b10 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f17680d0f80 con 0x7f1768161950 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176f5ed640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f17680d1c00 con 0x7f17680b6a10 2024-05-09T06:36:31.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176d389640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176d389640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42352/0 (socket says 172.21.15.1:42352) 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176d389640 1 -- 172.21.15.1:0/3076057835 learned_addr learned my addr 172.21.15.1:0/3076057835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176cb88640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1768161950 0x7f17680b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.831+0000 7f176db8a640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680b6a10 0x7f176806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 -- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680b6a10 msgr2=0x7f176806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680b6a10 0x7f176806c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 -- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1768161950 msgr2=0x7f17680b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1768161950 0x7f17680b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 -- 172.21.15.1:0/3076057835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17680b6fb0 con 0x7f17680d2b10 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176d389640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f176400c730 tx=0x7f176400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764018070 con 0x7f17680d2b10 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764004d80 con 0x7f17680d2b10 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17680b7290 con 0x7f17680d2b10 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176400a880 con 0x7f17680d2b10 2024-05-09T06:36:31.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.832+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17680b7850 con 0x7f17680d2b10 2024-05-09T06:36:31.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.833+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17680752b0 con 0x7f17680d2b10 2024-05-09T06:36:31.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.834+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f176401b070 con 0x7f17680d2b10 2024-05-09T06:36:31.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.834+0000 7f175dffb640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.834+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f176408a020 con 0x7f17680d2b10 2024-05-09T06:36:31.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.834+0000 7f176cb88640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.835+0000 7f176cb88640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 0x7f174c06e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1758034000 tx=0x7f175802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:31.839+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1764058540 con 0x7f17680d2b10 2024-05-09T06:36:32.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.090+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1768067440 con 0x7f17680d2b10 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.090+0000 7f175dffb640 1 -- 172.21.15.1:0/3076057835 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f176405c840 con 0x7f17680d2b10 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:36:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:36:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:36:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:36:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:36:32.086 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 msgr2=0x7f174c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 0x7f174c06e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1758034000 tx=0x7f175802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 msgr2=0x7f1768159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f176400c730 tx=0x7f176400cb30 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 shutdown_connections 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f17680b6a10 0x7f176806c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1768161950 0x7f17680b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 --2- 172.21.15.1:0/3076057835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17680d2b10 0x7f1768159630 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 >> 172.21.15.1:0/3076057835 conn(0x7f17680d3e20 msgr2=0x7f17680b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 shutdown_connections 2024-05-09T06:36:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.092+0000 7f176f5ed640 1 -- 172.21.15.1:0/3076057835 wait complete. 2024-05-09T06:36:32.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.212+0000 7f7a96789640 1 Processor -- start 2024-05-09T06:36:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.212+0000 7f7a96789640 1 -- start start 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a96789640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 0x7f7a900b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a96789640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9006eb40 con 0x7f7a900b2a60 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a96789640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9006ee20 con 0x7f7a900b4f70 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a96789640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9006d8e0 con 0x7f7a901454b0 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a87fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7a900b2a60 0x7f7a900b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46942/0 (socket says 172.21.15.1:46942) 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a87fff640 1 -- 172.21.15.1:0/2650917038 learned_addr learned my addr 172.21.15.1:0/2650917038 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.213+0000 7f7a8ffff640 1 --2- 172.21.15.1:0/2650917038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 0x7f7a900b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155575795 0 0) 0x7f7a9006eb40 con 0x7f7a900b2a60 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a7c003520 con 0x7f7a900b2a60 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520947416 0 0) 0x7f7a9006ee20 con 0x7f7a900b4f70 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a9006eb40 con 0x7f7a900b4f70 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8ffff640 1 -- 172.21.15.1:0/2650917038 >> v1:172.21.15.6:6789/0 conn(0x7f7a900b4f70 legacy=0x7f7a90070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8ffff640 1 -- 172.21.15.1:0/2650917038 >> v1:172.21.15.1:6789/0 conn(0x7f7a900b2a60 legacy=0x7f7a900b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3252368117 0 0) 0x7f7a7c003520 con 0x7f7a900b2a60 2024-05-09T06:36:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.214+0000 7f7a8ffff640 1 -- 172.21.15.1:0/2650917038 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9006dbc0 con 0x7f7a901454b0 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a8ffff640 1 --2- 172.21.15.1:0/2650917038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 0x7f7a900b4320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a8002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce10f4743e2d01a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037070 con 0x7f7a901454b0 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8002ac30 con 0x7f7a901454b0 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a8f7fe640 1 -- 172.21.15.1:0/2650917038 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78001670 con 0x7f7a901454b0 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 -- 172.21.15.1:0/2650917038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 msgr2=0x7f7a900b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 --2- 172.21.15.1:0/2650917038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 0x7f7a900b4320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a8002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 -- 172.21.15.1:0/2650917038 shutdown_connections 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 --2- 172.21.15.1:0/2650917038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a901454b0 0x7f7a900b4320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 -- 172.21.15.1:0/2650917038 >> 172.21.15.1:0/2650917038 conn(0x7f7a900cc4b0 msgr2=0x7f7a900cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 -- 172.21.15.1:0/2650917038 shutdown_connections 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.215+0000 7f7a96789640 1 -- 172.21.15.1:0/2650917038 wait complete. 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 Processor -- start 2024-05-09T06:36:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 -- start start 2024-05-09T06:36:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a900b4f70 0x7f7a9015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a901454b0 0x7f7a900adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006e260 con 0x7f7a900b2a60 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006e0e0 con 0x7f7a901454b0 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a96789640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7a90071320 con 0x7f7a900b4f70 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42356/0 (socket says 172.21.15.1:42356) 2024-05-09T06:36:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.217+0000 7f7a8ffff640 1 -- 172.21.15.1:0/68330001 learned_addr learned my addr 172.21.15.1:0/68330001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a87fff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a900b4f70 0x7f7a9015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a94d26640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a901454b0 0x7f7a900adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 -- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a900b4f70 msgr2=0x7f7a9015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a900b4f70 0x7f7a9015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 -- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a901454b0 msgr2=0x7f7a900adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a901454b0 0x7f7a900adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 -- 172.21.15.1:0/68330001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9013f0a0 con 0x7f7a900b2a60 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8ffff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037050 con 0x7f7a900b2a60 2024-05-09T06:36:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a80043070 con 0x7f7a900b2a60 2024-05-09T06:36:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9013f380 con 0x7f7a900b2a60 2024-05-09T06:36:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8003ca50 con 0x7f7a900b2a60 2024-05-09T06:36:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.218+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9015ba80 con 0x7f7a900b2a60 2024-05-09T06:36:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.219+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a900b30a0 con 0x7f7a900b2a60 2024-05-09T06:36:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.220+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a8003cbf0 con 0x7f7a900b2a60 2024-05-09T06:36:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.220+0000 7f7a8d7fa640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 0x7f7a7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.220+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f7a800ab620 con 0x7f7a900b2a60 2024-05-09T06:36:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.220+0000 7f7a87fff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 0x7f7a7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.221+0000 7f7a87fff640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 0x7f7a7006e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a900b16c0 tx=0x7f7a78001610 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.225+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a900b30a0 con 0x7f7a900b2a60 2024-05-09T06:36:32.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.412+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a900752b0 con 0x7f7a900b2a60 2024-05-09T06:36:32.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.412+0000 7f7a8d7fa640 1 -- 172.21.15.1:0/68330001 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+13034 (secure 0 0 0) 0x7f7a80079b40 con 0x7f7a900b2a60 2024-05-09T06:36:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:32.393 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":102,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:31.705933+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 msgr2=0x7f7a7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 0x7f7a7006e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a900b16c0 tx=0x7f7a78001610 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 msgr2=0x7f7a9013fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a800046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 shutdown_connections 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7a7006be30 0x7f7a7006e2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a901454b0 0x7f7a900adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a900b4f70 0x7f7a9015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 --2- 172.21.15.1:0/68330001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a900b2a60 0x7f7a9013fe60 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 >> 172.21.15.1:0/68330001 conn(0x7f7a900cc4b0 msgr2=0x7f7a900aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 shutdown_connections 2024-05-09T06:36:32.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.415+0000 7f7a96789640 1 -- 172.21.15.1:0/68330001 wait complete. 2024-05-09T06:36:32.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:32.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:32.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.533+0000 7fcedcc76640 1 Processor -- start 2024-05-09T06:36:32.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.533+0000 7fcedcc76640 1 -- start start 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fcedcc76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced815e390 0x7fced80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fcedcc76640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fced80b1900 0x7fced80cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fcedcc76640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced815e900 con 0x7fced80cf990 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fcedcc76640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fced815ebe0 con 0x7fced80b1900 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fcedcc76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fced815eec0 con 0x7fced815e390 2024-05-09T06:36:32.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fced6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fced80b1900 0x7fced80cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fced5d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fced80cf990 0x7fced80b11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46950/0 (socket says 172.21.15.1:46950) 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fced5d9b640 1 -- 172.21.15.1:0/1666409366 learned_addr learned my addr 172.21.15.1:0/1666409366 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fced659c640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced815e390 0x7fced80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.534+0000 7fced6d9d640 1 -- 172.21.15.1:0/1666409366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced815e390 msgr2=0x7fced80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced6d9d640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced815e390 0x7fced80cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced6d9d640 1 -- 172.21.15.1:0/1666409366 >> v1:172.21.15.1:6789/0 conn(0x7fced80cf990 legacy=0x7fced80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced559a640 1 -- 172.21.15.1:0/1666409366 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056097142 0 0) 0x7fced815e900 con 0x7fced80cf990 2024-05-09T06:36:32.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced6d9d640 1 -- 172.21.15.1:0/1666409366 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced815f1a0 con 0x7fced80b1900 2024-05-09T06:36:32.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced6d9d640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced80b1900 0x7fced80cc8e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fced0009d90 tx=0x7fced002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730dc711cedb7f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced559a640 1 -- 172.21.15.1:0/1666409366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0031600 con 0x7fced80b1900 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced559a640 1 -- 172.21.15.1:0/1666409366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced00317a0 con 0x7fced80b1900 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.535+0000 7fced559a640 1 -- 172.21.15.1:0/1666409366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0031b70 con 0x7fced80b1900 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 -- 172.21.15.1:0/1666409366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced80b1900 msgr2=0x7fced80cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced80b1900 0x7fced80cc8e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fced0009d90 tx=0x7fced002a5d0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 -- 172.21.15.1:0/1666409366 shutdown_connections 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced80b1900 0x7fced80cc8e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 --2- 172.21.15.1:0/1666409366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced815e390 0x7fced80cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 -- 172.21.15.1:0/1666409366 >> 172.21.15.1:0/1666409366 conn(0x7fced80b1da0 msgr2=0x7fced80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 -- 172.21.15.1:0/1666409366 shutdown_connections 2024-05-09T06:36:32.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.536+0000 7fcedcc76640 1 -- 172.21.15.1:0/1666409366 wait complete. 2024-05-09T06:36:32.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.537+0000 7fcedcc76640 1 Processor -- start 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.537+0000 7fcedcc76640 1 -- start start 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced80b1900 0x7fced81e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fced80cf990 0x7fced80c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fced80cffb0 con 0x7fced815e390 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fced8162ba0 con 0x7fced80cf990 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fcedcc76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fced80cce20 con 0x7fced80b1900 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fced6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fced6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42370/0 (socket says 172.21.15.1:42370) 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fced6d9d640 1 -- 172.21.15.1:0/2468845411 learned_addr learned my addr 172.21.15.1:0/2468845411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fced659c640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced80b1900 0x7fced81e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.538+0000 7fced5d9b640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fced80cf990 0x7fced80c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 -- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced80b1900 msgr2=0x7fced81e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced80b1900 0x7fced81e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 -- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fced80cf990 msgr2=0x7fced80c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fced80cf990 0x7fced80c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 -- 172.21.15.1:0/2468845411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81e8cd0 con 0x7fced815e390 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.539+0000 7fced6d9d640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fced0009eb0 tx=0x7fced0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.540+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0040070 con 0x7fced815e390 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.540+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced0030b60 con 0x7fced815e390 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.540+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced815b080 con 0x7fced815e390 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.540+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0045460 con 0x7fced815e390 2024-05-09T06:36:32.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.540+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced80c5c40 con 0x7fced815e390 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.541+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced80752b0 con 0x7fced815e390 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.541+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fced0039040 con 0x7fced815e390 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.542+0000 7fcec6ffd640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 0x7fceb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.542+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7fced00ac280 con 0x7fced815e390 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.542+0000 7fced659c640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 0x7fceb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.542+0000 7fced659c640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 0x7fceb806e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fced815b930 tx=0x7fcec8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.546+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fced81e8cd0 con 0x7fced815e390 2024-05-09T06:36:32.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.729+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fced80b2ae0 con 0x7fced815e390 2024-05-09T06:36:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.729+0000 7fcec6ffd640 1 -- 172.21.15.1:0/2468845411 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fced007a8f0 con 0x7fced815e390 2024-05-09T06:36:32.710 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 msgr2=0x7fceb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 0x7fceb806e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fced815b930 tx=0x7fcec8008040 comp rx=0 tx=0).stop 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 msgr2=0x7fced80b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fced0009eb0 tx=0x7fced0004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 shutdown_connections 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fceb806be30 0x7fceb806e2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fced815e390 0x7fced80b34d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fced80cf990 0x7fced80c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 --2- 172.21.15.1:0/2468845411 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fced80b1900 0x7fced81e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 >> 172.21.15.1:0/2468845411 conn(0x7fced80b1da0 msgr2=0x7fced806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 shutdown_connections 2024-05-09T06:36:32.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.731+0000 7fcedcc76640 1 -- 172.21.15.1:0/2468845411 wait complete. 2024-05-09T06:36:32.723 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:36:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.846+0000 7efe20212640 1 Processor -- start 2024-05-09T06:36:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.847+0000 7efe20212640 1 -- start start 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.847+0000 7efe20212640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe18161950 0x7efe180d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.847+0000 7efe20212640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe180b33c0 con 0x7efe180d0f80 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.847+0000 7efe20212640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe180b3660 con 0x7efe180d4940 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.847+0000 7efe20212640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efe180d4dd0 con 0x7efe18161950 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1e7af640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7efe180d0f80 0x7efe180cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46966/0 (socket says 172.21.15.1:46966) 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1e7af640 1 -- 172.21.15.1:0/704600799 learned_addr learned my addr 172.21.15.1:0/704600799 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1dfae640 1 --2- 172.21.15.1:0/704600799 >> v2:172.21.15.6:3300/0 conn(0x7efe18161950 0x7efe180d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 366014897 0 0) 0x7efe180b33c0 con 0x7efe180d0f80 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe00003520 con 0x7efe180d0f80 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 358430532 0 0) 0x7efe00003520 con 0x7efe180d0f80 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1dfae640 1 -- 172.21.15.1:0/704600799 >> v1:172.21.15.6:6789/0 conn(0x7efe180d4940 legacy=0x7efe180cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1dfae640 1 -- 172.21.15.1:0/704600799 >> v1:172.21.15.1:6789/0 conn(0x7efe180d0f80 legacy=0x7efe180cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.848+0000 7efe1dfae640 1 -- 172.21.15.1:0/704600799 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180b3940 con 0x7efe18161950 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.849+0000 7efe1dfae640 1 --2- 172.21.15.1:0/704600799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe18161950 0x7efe180d2400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe1002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51edf9daf5a6a4f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.849+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10037070 con 0x7efe18161950 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.849+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1002ac50 con 0x7efe18161950 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.849+0000 7efe1cfac640 1 -- 172.21.15.1:0/704600799 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08001a10 con 0x7efe18161950 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 -- 172.21.15.1:0/704600799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe18161950 msgr2=0x7efe180d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 --2- 172.21.15.1:0/704600799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe18161950 0x7efe180d2400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe1002a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 -- 172.21.15.1:0/704600799 shutdown_connections 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 --2- 172.21.15.1:0/704600799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe18161950 0x7efe180d2400 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 -- 172.21.15.1:0/704600799 >> 172.21.15.1:0/704600799 conn(0x7efe180d3e20 msgr2=0x7efe180d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 -- 172.21.15.1:0/704600799 shutdown_connections 2024-05-09T06:36:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.850+0000 7efe20212640 1 -- 172.21.15.1:0/704600799 wait complete. 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.851+0000 7efe20212640 1 Processor -- start 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.851+0000 7efe20212640 1 -- start start 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe180d0f80 0x7efe181e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe18161950 0x7efe180b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe180b5150 con 0x7efe1814e260 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efe180cf850 con 0x7efe180d0f80 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe20212640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe180b52d0 con 0x7efe18161950 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42382/0 (socket says 172.21.15.1:42382) 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 -- 172.21.15.1:0/2606274027 learned_addr learned my addr 172.21.15.1:0/2606274027 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1dfae640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe180d0f80 0x7efe181e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1d7ad640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe18161950 0x7efe180b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 -- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe18161950 msgr2=0x7efe180b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe18161950 0x7efe180b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 -- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe180d0f80 msgr2=0x7efe181e6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe180d0f80 0x7efe181e6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 -- 172.21.15.1:0/2606274027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180b7b50 con 0x7efe1814e260 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.852+0000 7efe1e7af640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7efe0800cb70 tx=0x7efe0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.853+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08018070 con 0x7efe1814e260 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.853+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08004d60 con 0x7efe1814e260 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.853+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0800ad40 con 0x7efe1814e260 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.853+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe180b7e30 con 0x7efe1814e260 2024-05-09T06:36:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.853+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe180b83f0 con 0x7efe1814e260 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.854+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe180752b0 con 0x7efe1814e260 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.854+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe0801b070 con 0x7efe1814e260 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.855+0000 7efe167fc640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.855+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7efe0808a3b0 con 0x7efe1814e260 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.855+0000 7efe1dfae640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.855+0000 7efe1dfae640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 0x7efe0406e320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:32.859+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe08058960 con 0x7efe1814e260 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.044+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efe1806c2f0 con 0x7efe1814e260 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.044+0000 7efe167fc640 1 -- 172.21.15.1:0/2606274027 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) ==== 84+0+65 (secure 0 0 0) 0x7efe0805cc60 con 0x7efe1814e260 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:36:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:36:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:36:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:33.026 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 msgr2=0x7efe0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 0x7efe0406e320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 msgr2=0x7efe180b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7efe0800cb70 tx=0x7efe0800a430 comp rx=0 tx=0).stop 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 shutdown_connections 2024-05-09T06:36:33.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe0406be60 0x7efe0406e320 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe1814e260 0x7efe180b8bf0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe18161950 0x7efe180b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 --2- 172.21.15.1:0/2606274027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe180d0f80 0x7efe181e6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 >> 172.21.15.1:0/2606274027 conn(0x7efe180d3e20 msgr2=0x7efe180cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 shutdown_connections 2024-05-09T06:36:33.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.046+0000 7efe20212640 1 -- 172.21.15.1:0/2606274027 wait complete. 2024-05-09T06:36:33.038 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:36:33.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:36:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.166+0000 7f6d4773b640 1 Processor -- start 2024-05-09T06:36:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.166+0000 7f6d4773b640 1 -- start start 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d4773b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d4015ec10 0x7f6d400b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d4773b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d400ccf20 con 0x7f6d40072580 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d4773b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d400ccc40 con 0x7f6d40157fa0 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d4773b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4015e130 con 0x7f6d4015ec10 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d44cd6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d4015ec10 0x7f6d400b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d45cd8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6d40072580 0x7f6d40072a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46978/0 (socket says 172.21.15.1:46978) 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d44cd6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d4015ec10 0x7f6d400b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42396/0 (socket says 172.21.15.1:42396) 2024-05-09T06:36:33.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d45cd8640 1 -- 172.21.15.1:0/3254705795 learned_addr learned my addr 172.21.15.1:0/3254705795 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:33.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d44cd6640 1 -- 172.21.15.1:0/3254705795 >> v1:172.21.15.6:6789/0 conn(0x7f6d40157fa0 legacy=0x7f6d400cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d44cd6640 1 -- 172.21.15.1:0/3254705795 >> v1:172.21.15.1:6789/0 conn(0x7f6d40072580 legacy=0x7f6d40072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d37fff640 1 -- 172.21.15.1:0/3254705795 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501042406 0 0) 0x7f6d400ccf20 con 0x7f6d40072580 2024-05-09T06:36:33.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.167+0000 7f6d44cd6640 1 -- 172.21.15.1:0/3254705795 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4015e410 con 0x7f6d4015ec10 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.168+0000 7f6d44cd6640 1 --2- 172.21.15.1:0/3254705795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d400b4bd0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000a940 tx=0x7f6d3002b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5e3c9c4c42ae06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.168+0000 7f6d37fff640 1 -- 172.21.15.1:0/3254705795 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000b040 con 0x7f6d4015ec10 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.168+0000 7f6d37fff640 1 -- 172.21.15.1:0/3254705795 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d30030510 con 0x7f6d4015ec10 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.168+0000 7f6d37fff640 1 -- 172.21.15.1:0/3254705795 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300308e0 con 0x7f6d4015ec10 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 -- 172.21.15.1:0/3254705795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 msgr2=0x7f6d400b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3254705795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d400b4bd0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000a940 tx=0x7f6d3002b5e0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 -- 172.21.15.1:0/3254705795 shutdown_connections 2024-05-09T06:36:33.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3254705795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d400b4bd0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 -- 172.21.15.1:0/3254705795 >> 172.21.15.1:0/3254705795 conn(0x7f6d4006b3c0 msgr2=0x7f6d4006b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 -- 172.21.15.1:0/3254705795 shutdown_connections 2024-05-09T06:36:33.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.169+0000 7f6d4773b640 1 -- 172.21.15.1:0/3254705795 wait complete. 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.170+0000 7f6d4773b640 1 Processor -- start 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.170+0000 7f6d4773b640 1 -- start start 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d40072580 0x7f6d40154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d40157fa0 0x7f6d40159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d400cfcb0 con 0x7f6d4015ec10 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40158380 con 0x7f6d40157fa0 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d4773b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d400cfe30 con 0x7f6d40072580 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42410/0 (socket says 172.21.15.1:42410) 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 -- 172.21.15.1:0/3322021780 learned_addr learned my addr 172.21.15.1:0/3322021780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d44cd6640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d40157fa0 0x7f6d40159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d454d7640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d40072580 0x7f6d40154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 -- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d40072580 msgr2=0x7f6d40154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d40072580 0x7f6d40154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 -- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d40157fa0 msgr2=0x7f6d40159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d40157fa0 0x7f6d40159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.171+0000 7f6d45cd8640 1 -- 172.21.15.1:0/3322021780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d40154f40 con 0x7f6d4015ec10 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d45cd8640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800c7a0 tx=0x7f6d3800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38018070 con 0x7f6d4015ec10 2024-05-09T06:36:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3800a640 con 0x7f6d4015ec10 2024-05-09T06:36:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d401f2260 con 0x7f6d4015ec10 2024-05-09T06:36:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800aab0 con 0x7f6d4015ec10 2024-05-09T06:36:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.172+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401f2820 con 0x7f6d4015ec10 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.173+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d40075320 con 0x7f6d4015ec10 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.174+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d3801b070 con 0x7f6d4015ec10 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.174+0000 7f6d35ffb640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 0x7f6d2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.174+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f6d3808a1b0 con 0x7f6d4015ec10 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.174+0000 7f6d454d7640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 0x7f6d2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.175+0000 7f6d454d7640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 0x7f6d2406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c000f60 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.178+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d38058760 con 0x7f6d4015ec10 2024-05-09T06:36:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.365+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d400cee60 con 0x7f6d4015ec10 2024-05-09T06:36:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.699+0000 7f6d35ffb640 1 -- 172.21.15.1:0/3322021780 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) ==== 138+0+0 (secure 0 0 0) 0x7f6d3805ca60 con 0x7f6d4015ec10 2024-05-09T06:36:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 msgr2=0x7f6d2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 0x7f6d2406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c000f60 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 msgr2=0x7f6d40156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800c7a0 tx=0x7f6d3800cc60 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 shutdown_connections 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d2406be30 0x7f6d2406e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d4015ec10 0x7f6d40156230 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d40157fa0 0x7f6d40159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 --2- 172.21.15.1:0/3322021780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d40072580 0x7f6d40154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 >> 172.21.15.1:0/3322021780 conn(0x7f6d4006b3c0 msgr2=0x7f6d400cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 shutdown_connections 2024-05-09T06:36:33.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.702+0000 7f6d4773b640 1 -- 172.21.15.1:0/3322021780 wait complete. 2024-05-09T06:36:33.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:36:33.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.829+0000 7f4fe79a5640 1 Processor -- start 2024-05-09T06:36:33.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.829+0000 7f4fe79a5640 1 -- start start 2024-05-09T06:36:33.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe79a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 0x7f4fe00cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe79a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe0161c90 con 0x7f4fe00cfa20 2024-05-09T06:36:33.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe79a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe0161f00 con 0x7f4fe00b18f0 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe79a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe015e550 con 0x7f4fe015e170 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe4f40640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4fe00cfa20 0x7f4fe00b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:46992/0 (socket says 172.21.15.1:46992) 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe4f40640 1 -- 172.21.15.1:0/3677477631 learned_addr learned my addr 172.21.15.1:0/3677477631 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe5741640 1 --2- 172.21.15.1:0/3677477631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 0x7f4fe00cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626725268 0 0) 0x7f4fe0161c90 con 0x7f4fe00cfa20 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fcc003520 con 0x7f4fe00cfa20 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364833812 0 0) 0x7f4fe0161f00 con 0x7f4fe00b18f0 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fe0161c90 con 0x7f4fe00b18f0 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe5741640 1 -- 172.21.15.1:0/3677477631 >> v1:172.21.15.6:6789/0 conn(0x7f4fe00b18f0 legacy=0x7f4fe00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe5741640 1 -- 172.21.15.1:0/3677477631 >> v1:172.21.15.1:6789/0 conn(0x7f4fe00cfa20 legacy=0x7f4fe00b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.830+0000 7f4fe5741640 1 -- 172.21.15.1:0/3677477631 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe01621e0 con 0x7f4fe015e170 2024-05-09T06:36:33.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.831+0000 7f4fe5741640 1 --2- 172.21.15.1:0/3677477631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 0x7f4fe00cf4e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0097d0 tx=0x7f4fdc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c44dc95e8f0e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.831+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc037070 con 0x7f4fe015e170 2024-05-09T06:36:33.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.831+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fdc032440 con 0x7f4fe015e170 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.831+0000 7f4fd7fff640 1 -- 172.21.15.1:0/3677477631 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc032810 con 0x7f4fe015e170 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 -- 172.21.15.1:0/3677477631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 msgr2=0x7f4fe00cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/3677477631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 0x7f4fe00cf4e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0097d0 tx=0x7f4fdc02a210 comp rx=0 tx=0).stop 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 -- 172.21.15.1:0/3677477631 shutdown_connections 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/3677477631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe015e170 0x7f4fe00cf4e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 -- 172.21.15.1:0/3677477631 >> 172.21.15.1:0/3677477631 conn(0x7f4fe00b1da0 msgr2=0x7f4fe00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 -- 172.21.15.1:0/3677477631 shutdown_connections 2024-05-09T06:36:33.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.832+0000 7f4fe79a5640 1 -- 172.21.15.1:0/3677477631 wait complete. 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.833+0000 7f4fe79a5640 1 Processor -- start 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.833+0000 7f4fe79a5640 1 -- start start 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe00b18f0 0x7f4fe015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe015e170 0x7f4fe0160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01629e0 con 0x7f4fe00cfa20 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe00cffc0 con 0x7f4fe015e170 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe79a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe00b3050 con 0x7f4fe00b18f0 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42414/0 (socket says 172.21.15.1:42414) 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 -- 172.21.15.1:0/1085520624 learned_addr learned my addr 172.21.15.1:0/1085520624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe5f42640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe015e170 0x7f4fe0160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe5741640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe00b18f0 0x7f4fe015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 -- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe00b18f0 msgr2=0x7f4fe015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe00b18f0 0x7f4fe015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 -- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe015e170 msgr2=0x7f4fe0160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.834+0000 7f4fe4f40640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe015e170 0x7f4fe0160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fe4f40640 1 -- 172.21.15.1:0/1085520624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe01610e0 con 0x7f4fe00cfa20 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fe4f40640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c530 tx=0x7f4fd000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0017070 con 0x7f4fe00cfa20 2024-05-09T06:36:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0005ce0 con 0x7f4fe00cfa20 2024-05-09T06:36:33.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe00c3cd0 con 0x7f4fe00cfa20 2024-05-09T06:36:33.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0007270 con 0x7f4fe00cfa20 2024-05-09T06:36:33.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.835+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe00c4290 con 0x7f4fe00cfa20 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.836+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe00752b0 con 0x7f4fe00cfa20 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.837+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fd001a070 con 0x7f4fe00cfa20 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.837+0000 7f4fd5ffb640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.838+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5619+0+0 (secure 0 0 0) 0x7f4fd008a250 con 0x7f4fe00cfa20 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.838+0000 7f4fe5741640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.838+0000 7f4fe5741640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 0x7f4fc406e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034000 tx=0x7f4fdc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:33.842+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4fd00584e0 con 0x7f4fe00cfa20 2024-05-09T06:36:34.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.026+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4fe00b2b00 con 0x7f4fe00cfa20 2024-05-09T06:36:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.701+0000 7f4fd5ffb640 1 -- 172.21.15.1:0/1085520624 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) ==== 170+0+0 (secure 0 0 0) 0x7f4fd005c7e0 con 0x7f4fe00cfa20 2024-05-09T06:36:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:36:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 msgr2=0x7f4fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 0x7f4fc406e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034000 tx=0x7f4fdc02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 msgr2=0x7f4fe0160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c530 tx=0x7f4fd000c9f0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 shutdown_connections 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fe015e170 0x7f4fe0160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fe00cfa20 0x7f4fe0160500 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 --2- 172.21.15.1:0/1085520624 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4fe00b18f0 0x7f4fe015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 >> 172.21.15.1:0/1085520624 conn(0x7f4fe00b1da0 msgr2=0x7f4fe00b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.704+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 shutdown_connections 2024-05-09T06:36:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.705+0000 7f4fe79a5640 1 -- 172.21.15.1:0/1085520624 wait complete. 2024-05-09T06:36:34.709 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:36:34.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.841+0000 7f58baf8b640 1 Processor -- start 2024-05-09T06:36:34.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.841+0000 7f58baf8b640 1 -- start start 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58baf8b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58b40d0eb0 0x7f58b40d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58baf8b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40d2e20 con 0x7f58b40cf690 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58baf8b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40cfc00 con 0x7f58b415e3d0 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58baf8b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40d04a0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58b40d0eb0 0x7f58b40d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58b40d0eb0 0x7f58b40d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42428/0 (socket says 172.21.15.1:42428) 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b9528640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f58b40cf690 0x7f58b40b39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47002/0 (socket says 172.21.15.1:47002) 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 -- 172.21.15.1:0/2563372336 learned_addr learned my addr 172.21.15.1:0/2563372336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 -- 172.21.15.1:0/2563372336 >> v1:172.21.15.6:6789/0 conn(0x7f58b415e3d0 legacy=0x7f58b40d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 -- 172.21.15.1:0/2563372336 >> v1:172.21.15.1:6789/0 conn(0x7f58b40cf690 legacy=0x7f58b40b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b37fe640 1 -- 172.21.15.1:0/2563372336 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720700203 0 0) 0x7f58b40d2e20 con 0x7f58b40cf690 2024-05-09T06:36:34.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.842+0000 7f58b3fff640 1 -- 172.21.15.1:0/2563372336 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40cfee0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58b3fff640 1 --2- 172.21.15.1:0/2563372336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b40d48b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009e50 tx=0x7f58a402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b509cc2c5858773 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58b37fe640 1 -- 172.21.15.1:0/2563372336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4036070 con 0x7f58b40d0eb0 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58b37fe640 1 -- 172.21.15.1:0/2563372336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4031600 con 0x7f58b40d0eb0 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58b37fe640 1 -- 172.21.15.1:0/2563372336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40319f0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58baf8b640 1 -- 172.21.15.1:0/2563372336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 msgr2=0x7f58b40d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.843+0000 7f58baf8b640 1 --2- 172.21.15.1:0/2563372336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b40d48b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009e50 tx=0x7f58a402a7c0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.844+0000 7f58baf8b640 1 -- 172.21.15.1:0/2563372336 shutdown_connections 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.844+0000 7f58baf8b640 1 --2- 172.21.15.1:0/2563372336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b40d48b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.844+0000 7f58baf8b640 1 -- 172.21.15.1:0/2563372336 >> 172.21.15.1:0/2563372336 conn(0x7f58b40d3d90 msgr2=0x7f58b40d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.844+0000 7f58baf8b640 1 -- 172.21.15.1:0/2563372336 shutdown_connections 2024-05-09T06:36:34.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.844+0000 7f58baf8b640 1 -- 172.21.15.1:0/2563372336 wait complete. 2024-05-09T06:36:34.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.845+0000 7f58baf8b640 1 Processor -- start 2024-05-09T06:36:34.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.845+0000 7f58baf8b640 1 -- start start 2024-05-09T06:36:34.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b40cf690 0x7f58b4153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58b415e3d0 0x7f58b415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40b40d0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40b34a0 con 0x7f58b40cf690 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58baf8b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40b4250 con 0x7f58b415e3d0 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42436/0 (socket says 172.21.15.1:42436) 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 -- 172.21.15.1:0/1106029552 learned_addr learned my addr 172.21.15.1:0/1106029552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 -- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58b415e3d0 msgr2=0x7f58b415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b9528640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58b415e3d0 0x7f58b415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58b415e3d0 0x7f58b415bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 -- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b40cf690 msgr2=0x7f58b4153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b40cf690 0x7f58b4153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 -- 172.21.15.1:0/1106029552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40b7ad0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b3fff640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f58a402adb0 tx=0x7f58a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4036050 con 0x7f58b40d0eb0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.846+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4042070 con 0x7f58b40d0eb0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.847+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b4154dd0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.847+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4009510 con 0x7f58b40d0eb0 2024-05-09T06:36:34.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.847+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b4155390 con 0x7f58b40d0eb0 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.848+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b40752b0 con 0x7f58b40d0eb0 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.848+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58a4004470 con 0x7f58b40d0eb0 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.849+0000 7f58b17fa640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 0x7f589406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.849+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6001+0+0 (secure 0 0 0) 0x7f58a40abc80 con 0x7f58b40d0eb0 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.849+0000 7f58b8d27640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 0x7f589406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.849+0000 7f58b8d27640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 0x7f589406e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f58ac000f60 tx=0x7f58ac0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:34.854+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58a404c020 con 0x7f58b40d0eb0 2024-05-09T06:36:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.080+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f58b40d0c10 con 0x7f58b40d0eb0 2024-05-09T06:36:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.716+0000 7f58b17fa640 1 -- 172.21.15.1:0/1106029552 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v79) ==== 174+0+0 (secure 0 0 0) 0x7f58a407a590 con 0x7f58b40d0eb0 2024-05-09T06:36:35.697 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 15 and data pool 16 2024-05-09T06:36:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.725+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 msgr2=0x7f589406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.725+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 0x7f589406e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f58ac000f60 tx=0x7f58ac0079b0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.725+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 msgr2=0x7f58b415b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.725+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f58a402adb0 tx=0x7f58a4004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 shutdown_connections 2024-05-09T06:36:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f589406be30 0x7f589406e2f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f58b415e3d0 0x7f58b415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58b40d0eb0 0x7f58b415b830 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 --2- 172.21.15.1:0/1106029552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58b40cf690 0x7f58b4153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.727+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 >> 172.21.15.1:0/1106029552 conn(0x7f58b40d3d90 msgr2=0x7f58b406c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.728+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 shutdown_connections 2024-05-09T06:36:35.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.732+0000 7f58baf8b640 1 -- 172.21.15.1:0/1106029552 wait complete. 2024-05-09T06:36:35.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 Processor -- start 2024-05-09T06:36:35.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 -- start start 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77e8161950 0x7f77e80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 0x7f77e80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f77e80d0f80 0x7f77e80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f77e80d4dd0 con 0x7f77e8161950 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f77e80b33c0 con 0x7f77e80d0f80 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.865+0000 7f77eeaf5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80b3660 con 0x7f77e80d4940 2024-05-09T06:36:35.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77e8161950 0x7f77e80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77e8161950 0x7f77e80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42438/0 (socket says 172.21.15.1:42438) 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 -- 172.21.15.1:0/1664779626 learned_addr learned my addr 172.21.15.1:0/1664779626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ed092640 1 --2- 172.21.15.1:0/1664779626 >> v2:172.21.15.6:3300/0 conn(0x7f77e80d0f80 0x7f77e80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77e7fff640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 0x7f77e80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 -- 172.21.15.1:0/1664779626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 msgr2=0x7f77e80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 0x7f77e80d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 -- 172.21.15.1:0/1664779626 >> v2:172.21.15.6:3300/0 conn(0x7f77e80d0f80 msgr2=0x7f77e80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 --2- 172.21.15.1:0/1664779626 >> v2:172.21.15.6:3300/0 conn(0x7f77e80d0f80 0x7f77e80cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77ec891640 1 -- 172.21.15.1:0/1664779626 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80b3940 con 0x7f77e8161950 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.866+0000 7f77e7fff640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 0x7f77e80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77ec891640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e80d2400 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f77e808c710 tx=0x7f77dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3180f1824adc4d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77e77fe640 1 -- 172.21.15.1:0/1664779626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc037070 con 0x7f77e8161950 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77e77fe640 1 -- 172.21.15.1:0/1664779626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc032440 con 0x7f77e8161950 2024-05-09T06:36:35.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77e77fe640 1 -- 172.21.15.1:0/1664779626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc032810 con 0x7f77e8161950 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 -- 172.21.15.1:0/1664779626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 msgr2=0x7f77e80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e80d2400 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f77e808c710 tx=0x7f77dc02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 -- 172.21.15.1:0/1664779626 shutdown_connections 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/1664779626 >> v2:172.21.15.6:3300/0 conn(0x7f77e80d0f80 0x7f77e80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d4940 0x7f77e80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/1664779626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e80d2400 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 -- 172.21.15.1:0/1664779626 >> 172.21.15.1:0/1664779626 conn(0x7f77e80d3e20 msgr2=0x7f77e80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 -- 172.21.15.1:0/1664779626 shutdown_connections 2024-05-09T06:36:35.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.867+0000 7f77eeaf5640 1 -- 172.21.15.1:0/1664779626 wait complete. 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.869+0000 7f77eeaf5640 1 Processor -- start 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.869+0000 7f77eeaf5640 1 -- start start 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 0x7f77e809fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77e80cd370 0x7f77e80a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80d1d80 con 0x7f77e8161950 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80d1c00 con 0x7f77e80cd370 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80d1f00 con 0x7f77e80d0f80 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42446/0 (socket says 172.21.15.1:42446) 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 -- 172.21.15.1:0/3833994820 learned_addr learned my addr 172.21.15.1:0/3833994820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:35.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77ed092640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77e80cd370 0x7f77e80a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77ec891640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 0x7f77e809fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 -- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 msgr2=0x7f77e809fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 0x7f77e809fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 -- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77e80cd370 msgr2=0x7f77e80a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77e80cd370 0x7f77e80a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 -- 172.21.15.1:0/3833994820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81517d0 con 0x7f77e8161950 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77ec891640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 0x7f77e809fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e7fff640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f77d400e730 tx=0x7f77d400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d400f040 con 0x7f77e8161950 2024-05-09T06:36:35.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d4004230 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e80a0150 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d400ee40 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.870+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e80a0710 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.871+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80752b0 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.872+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77d400c6d0 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.872+0000 7f77e57fa640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 0x7f77cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.872+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f77d408ab30 con 0x7f77e8161950 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.872+0000 7f77ec891640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 0x7f77cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.873+0000 7f77ec891640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 0x7f77cc06e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f77dc034000 tx=0x7f77dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:35.877+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77e81517d0 con 0x7f77e8161950 2024-05-09T06:36:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.059+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77e80a0dd0 con 0x7f77e8161950 2024-05-09T06:36:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.060+0000 7f77e57fa640 1 -- 172.21.15.1:0/3833994820 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16765 (secure 0 0 0) 0x7f77d4017020 con 0x7f77e8161950 2024-05-09T06:36:36.041 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:36.041 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":105,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:35.082218+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 msgr2=0x7f77cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 0x7f77cc06e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f77dc034000 tx=0x7f77dc02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 msgr2=0x7f77e8151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f77d400e730 tx=0x7f77d400ebf0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 shutdown_connections 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f77cc06be30 0x7f77cc06e2f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77e80cd370 0x7f77e80a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77e8161950 0x7f77e8151290 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 --2- 172.21.15.1:0/3833994820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77e80d0f80 0x7f77e809fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 >> 172.21.15.1:0/3833994820 conn(0x7f77e80d3e20 msgr2=0x7f77e80b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 shutdown_connections 2024-05-09T06:36:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.062+0000 7f77eeaf5640 1 -- 172.21.15.1:0/3833994820 wait complete. 2024-05-09T06:36:36.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 Processor -- start 2024-05-09T06:36:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 -- start start 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc0ac0d4940 0x7fc0ac0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ac0b33c0 con 0x7fc0ac0d2b10 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ac0b3660 con 0x7fc0ac161950 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.178+0000 7fc0b23e7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac0d4dd0 con 0x7fc0ac0d4940 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0b0984640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc0ac0d2b10 0x7fc0ac0b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47006/0 (socket says 172.21.15.1:47006) 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0b0984640 1 -- 172.21.15.1:0/40491772 learned_addr learned my addr 172.21.15.1:0/40491772 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0ab7fe640 1 --2- 172.21.15.1:0/40491772 >> v2:172.21.15.6:3300/0 conn(0x7fc0ac0d4940 0x7fc0ac0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0aaffd640 1 -- 172.21.15.1:0/40491772 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093581664 0 0) 0x7fc0ac0b33c0 con 0x7fc0ac0d2b10 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0aaffd640 1 -- 172.21.15.1:0/40491772 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc094003520 con 0x7fc0ac0d2b10 2024-05-09T06:36:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/40491772 >> v1:172.21.15.6:6789/0 conn(0x7fc0ac161950 legacy=0x7fc0ac0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/40491772 >> v1:172.21.15.1:6789/0 conn(0x7fc0ac0d2b10 legacy=0x7fc0ac0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.179+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/40491772 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac0b3940 con 0x7fc0ac0d4940 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.180+0000 7fc0ab7fe640 1 --2- 172.21.15.1:0/40491772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d4940 0x7fc0ac0b4140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f0f1aca3548d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.180+0000 7fc0aaffd640 1 -- 172.21.15.1:0/40491772 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0037070 con 0x7fc0ac0d4940 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.180+0000 7fc0aaffd640 1 -- 172.21.15.1:0/40491772 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a0032440 con 0x7fc0ac0d4940 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.180+0000 7fc0aaffd640 1 -- 172.21.15.1:0/40491772 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0032810 con 0x7fc0ac0d4940 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 -- 172.21.15.1:0/40491772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d4940 msgr2=0x7fc0ac0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/40491772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d4940 0x7fc0ac0b4140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 -- 172.21.15.1:0/40491772 shutdown_connections 2024-05-09T06:36:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/40491772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d4940 0x7fc0ac0b4140 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 -- 172.21.15.1:0/40491772 >> 172.21.15.1:0/40491772 conn(0x7fc0ac0d3e20 msgr2=0x7fc0ac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 -- 172.21.15.1:0/40491772 shutdown_connections 2024-05-09T06:36:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.181+0000 7fc0b23e7640 1 -- 172.21.15.1:0/40491772 wait complete. 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.182+0000 7fc0b23e7640 1 Processor -- start 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.182+0000 7fc0b23e7640 1 -- start start 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d2b10 0x7fc0ac0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0ac0b6f50 0x7fc0ac15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0d1100 con 0x7fc0ac161950 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0d0f80 con 0x7fc0ac0d2b10 2024-05-09T06:36:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b23e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac0d1c00 con 0x7fc0ac0b6f50 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42462/0 (socket says 172.21.15.1:42462) 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/3447653221 learned_addr learned my addr 172.21.15.1:0/3447653221 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0abfff640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d2b10 0x7fc0ac0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0b0984640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0ac0b6f50 0x7fc0ac15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0ac0b6f50 msgr2=0x7fc0ac15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0ac0b6f50 0x7fc0ac15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d2b10 msgr2=0x7fc0ac0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d2b10 0x7fc0ac0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.183+0000 7fc0ab7fe640 1 -- 172.21.15.1:0/3447653221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac0b73f0 con 0x7fc0ac161950 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0ab7fe640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0035040 tx=0x7fc0a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0037040 con 0x7fc0ac161950 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a0030500 con 0x7fc0ac161950 2024-05-09T06:36:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac0b76d0 con 0x7fc0ac161950 2024-05-09T06:36:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0030910 con 0x7fc0ac161950 2024-05-09T06:36:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.184+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac0b7c90 con 0x7fc0ac161950 2024-05-09T06:36:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.185+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0752b0 con 0x7fc0ac161950 2024-05-09T06:36:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.185+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0a0044020 con 0x7fc0ac161950 2024-05-09T06:36:36.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.186+0000 7fc0a8ff9640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 0x7fc09806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.186+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fc0a00ab1d0 con 0x7fc0ac161950 2024-05-09T06:36:36.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.186+0000 7fc0abfff640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 0x7fc09806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.186+0000 7fc0abfff640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 0x7fc09806e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d2470 tx=0x7fc09c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.190+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0a0079a80 con 0x7fc0ac161950 2024-05-09T06:36:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.373+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0ac0b8130 con 0x7fc0ac161950 2024-05-09T06:36:36.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.374+0000 7fc0a8ff9640 1 -- 172.21.15.1:0/3447653221 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16765 (secure 0 0 0) 0x7fc0a0079380 con 0x7fc0ac161950 2024-05-09T06:36:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:36.359 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":105,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:35.082218+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 msgr2=0x7fc09806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 0x7fc09806e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d2470 tx=0x7fc09c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 msgr2=0x7fc0ac0b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0035040 tx=0x7fc0a00046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 shutdown_connections 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc09806be30 0x7fc09806e2f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc0ac0b6f50 0x7fc0ac15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0ac161950 0x7fc0ac0b8490 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 --2- 172.21.15.1:0/3447653221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0ac0d2b10 0x7fc0ac0b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 >> 172.21.15.1:0/3447653221 conn(0x7fc0ac0d3e20 msgr2=0x7fc0ac0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 shutdown_connections 2024-05-09T06:36:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.377+0000 7fc0b23e7640 1 -- 172.21.15.1:0/3447653221 wait complete. 2024-05-09T06:36:36.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:36:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.495+0000 7f2557458640 1 Processor -- start 2024-05-09T06:36:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.495+0000 7f2557458640 1 -- start start 2024-05-09T06:36:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f2557458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f2557458640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2550072050 con 0x7f25500d06f0 2024-05-09T06:36:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f2557458640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2550072330 con 0x7f25500c41e0 2024-05-09T06:36:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f2557458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2550072610 con 0x7f255006b1e0 2024-05-09T06:36:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f25551f4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f25500d06f0 0x7f25500cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47018/0 (socket says 172.21.15.1:47018) 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f25551f4640 1 -- 172.21.15.1:0/3735756276 learned_addr learned my addr 172.21.15.1:0/3735756276 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f25559f5640 1 --2- 172.21.15.1:0/3735756276 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.496+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756788136 0 0) 0x7f2550072050 con 0x7f25500d06f0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f253c003520 con 0x7f25500d06f0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132353924 0 0) 0x7f2550072330 con 0x7f25500c41e0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2550072050 con 0x7f25500c41e0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f25559f5640 1 -- 172.21.15.1:0/3735756276 >> v1:172.21.15.6:6789/0 conn(0x7f25500c41e0 legacy=0x7f25500ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f25559f5640 1 -- 172.21.15.1:0/3735756276 >> v1:172.21.15.1:6789/0 conn(0x7f25500d06f0 legacy=0x7f25500cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f25559f5640 1 -- 172.21.15.1:0/3735756276 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25500728f0 con 0x7f255006b1e0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.497+0000 7f25559f5640 1 --2- 172.21.15.1:0/3735756276 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500cefe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2548009dc0 tx=0x7f254802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f857529878be26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548007740 con 0x7f255006b1e0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25480078e0 con 0x7f255006b1e0 2024-05-09T06:36:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2547fff640 1 -- 172.21.15.1:0/3735756276 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548007c90 con 0x7f255006b1e0 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 -- 172.21.15.1:0/3735756276 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 msgr2=0x7f25500cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 --2- 172.21.15.1:0/3735756276 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500cefe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2548009dc0 tx=0x7f254802a6d0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 -- 172.21.15.1:0/3735756276 shutdown_connections 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 --2- 172.21.15.1:0/3735756276 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500cefe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 -- 172.21.15.1:0/3735756276 >> 172.21.15.1:0/3735756276 conn(0x7f25500d0b50 msgr2=0x7f25500d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 -- 172.21.15.1:0/3735756276 shutdown_connections 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.498+0000 7f2557458640 1 -- 172.21.15.1:0/3735756276 wait complete. 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.499+0000 7f2557458640 1 Processor -- start 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.499+0000 7f2557458640 1 -- start start 2024-05-09T06:36:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25500d06f0 0x7f25500c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25500ccf70 con 0x7f25500c41e0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25500ccdf0 con 0x7f25500d06f0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f2557458640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f25500cd0f0 con 0x7f255006b1e0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f25549f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f25549f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42468/0 (socket says 172.21.15.1:42468) 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.500+0000 7f25549f3640 1 -- 172.21.15.1:0/709695561 learned_addr learned my addr 172.21.15.1:0/709695561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25559f5640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25500d06f0 0x7f25500c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25551f4640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 -- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 msgr2=0x7f25500b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 -- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25500d06f0 msgr2=0x7f25500c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25500d06f0 0x7f25500c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 -- 172.21.15.1:0/709695561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25500c76b0 con 0x7f25500c41e0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25551f4640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f25549f3640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f254000c490 tx=0x7f254000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400052d0 con 0x7f25500c41e0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2540005470 con 0x7f25500c41e0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540012850 con 0x7f25500c41e0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501f48a0 con 0x7f25500c41e0 2024-05-09T06:36:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.501+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25500c68f0 con 0x7f25500c41e0 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.502+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25500752b0 con 0x7f25500c41e0 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.503+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25400129f0 con 0x7f25500c41e0 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.503+0000 7f2545ffb640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.503+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f254008ac30 con 0x7f25500c41e0 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.503+0000 7f25551f4640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.504+0000 7f25551f4640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 0x7f253406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f254c007fd0 tx=0x7f254c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.507+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25501f48a0 con 0x7f25500c41e0 2024-05-09T06:36:36.681 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:36:36.700+0000 7f3d7de73640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:36:36.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:36.732+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2550072f30 con 0x7f25500c41e0 2024-05-09T06:36:37.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.726+0000 7f2545ffb640 1 -- 172.21.15.1:0/709695561 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7f25500c76b0 con 0x7f25500c41e0 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 msgr2=0x7f253406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 0x7f253406e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f254c007fd0 tx=0x7f254c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 msgr2=0x7f25501f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f254000c490 tx=0x7f254000c950 comp rx=0 tx=0).stop 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 shutdown_connections 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f253406be30 0x7f253406e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25500d06f0 0x7f25500c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25500c41e0 0x7f25501f4360 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 --2- 172.21.15.1:0/709695561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f255006b1e0 0x7f25500b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 >> 172.21.15.1:0/709695561 conn(0x7f25500d0b50 msgr2=0x7f255015ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 shutdown_connections 2024-05-09T06:36:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.729+0000 7f2557458640 1 -- 172.21.15.1:0/709695561 wait complete. 2024-05-09T06:36:37.724 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:36:37.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:36:37.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.857+0000 7f8304329640 1 Processor -- start 2024-05-09T06:36:37.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.857+0000 7f8304329640 1 -- start start 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f8304329640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc15e390 0x7f82fc0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f8304329640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82fc0b1900 0x7f82fc0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f8304329640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82fc15e900 con 0x7f82fc0cf990 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f8304329640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f82fc15ebe0 con 0x7f82fc0b1900 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f8304329640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc15eec0 con 0x7f82fc15e390 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82fc0b1900 0x7f82fc0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f82fc0b1900 0x7f82fc0cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55352/0 (socket says 172.21.15.1:55352) 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 -- 172.21.15.1:0/915071938 learned_addr learned my addr 172.21.15.1:0/915071938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83020c5640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc15e390 0x7f82fc0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 -- 172.21.15.1:0/915071938 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc15e390 msgr2=0x7f82fc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc15e390 0x7f82fc0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 -- 172.21.15.1:0/915071938 >> v1:172.21.15.6:6789/0 conn(0x7f82fc0cf990 legacy=0x7f82fc0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.858+0000 7f83028c6640 1 -- 172.21.15.1:0/915071938 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc15f1a0 con 0x7f82fc0b1900 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f83028c6640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0b1900 0x7f82fc0cc8e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009e50 tx=0x7f82f402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e591a81a6523a321 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f83010c3640 1 -- 172.21.15.1:0/915071938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4036070 con 0x7f82fc0b1900 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f83010c3640 1 -- 172.21.15.1:0/915071938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f4031600 con 0x7f82fc0b1900 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f83010c3640 1 -- 172.21.15.1:0/915071938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40319d0 con 0x7f82fc0b1900 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f8304329640 1 -- 172.21.15.1:0/915071938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0b1900 msgr2=0x7f82fc0cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.859+0000 7f8304329640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0b1900 0x7f82fc0cc8e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009e50 tx=0x7f82f402a630 comp rx=0 tx=0).stop 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 -- 172.21.15.1:0/915071938 shutdown_connections 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0b1900 0x7f82fc0cc8e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 --2- 172.21.15.1:0/915071938 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc15e390 0x7f82fc0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 -- 172.21.15.1:0/915071938 >> 172.21.15.1:0/915071938 conn(0x7f82fc0b1da0 msgr2=0x7f82fc0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 -- 172.21.15.1:0/915071938 shutdown_connections 2024-05-09T06:36:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.860+0000 7f8304329640 1 -- 172.21.15.1:0/915071938 wait complete. 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.861+0000 7f8304329640 1 Processor -- start 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.861+0000 7f8304329640 1 -- start start 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 0x7f82fc1f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 0x7f82fc1f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0cffb0 con 0x7f82fc0cf990 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc162ba0 con 0x7f82fc15e390 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc0cce20 con 0x7f82fc0b1900 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55364/0 (socket says 172.21.15.1:55364) 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 -- 172.21.15.1:0/70679759 learned_addr learned my addr 172.21.15.1:0/70679759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83028c6640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 0x7f82fc1f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83020c5640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 0x7f82fc1f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 -- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 msgr2=0x7f82fc1f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 0x7f82fc1f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 -- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 msgr2=0x7f82fc1f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 0x7f82fc1f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 -- 172.21.15.1:0/70679759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc0c5ce0 con 0x7f82fc0cf990 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83028c6640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 0x7f82fc1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83020c5640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 0x7f82fc1f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f83018c4640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c460 tx=0x7f82ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec016020 con 0x7f82fc0cf990 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec012870 con 0x7f82fc0cf990 2024-05-09T06:36:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1f4210 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.862+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec012c60 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.863+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1f47d0 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.863+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc0752b0 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.864+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ec019020 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.864+0000 7f82f2ffd640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.865+0000 7f83020c5640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.865+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f82ec08aad0 con 0x7f82fc0cf990 2024-05-09T06:36:37.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.865+0000 7f83020c5640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 0x7f82e006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ad450 tx=0x7f82f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:37.869+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82ec020070 con 0x7f82fc0cf990 2024-05-09T06:36:38.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.090+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f82fc0b2ae0 con 0x7f82fc0cf990 2024-05-09T06:36:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.734+0000 7f82f2ffd640 1 -- 172.21.15.1:0/70679759 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f82ec0592f0 con 0x7f82fc0cf990 2024-05-09T06:36:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 msgr2=0x7f82e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 0x7f82e006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ad450 tx=0x7f82f8008040 comp rx=0 tx=0).stop 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 msgr2=0x7f82fc15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c460 tx=0x7f82ec00c920 comp rx=0 tx=0).stop 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 shutdown_connections 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82fc15e390 0x7f82fc1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc0cf990 0x7f82fc15bf00 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 --2- 172.21.15.1:0/70679759 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82fc0b1900 0x7f82fc1f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 >> 172.21.15.1:0/70679759 conn(0x7f82fc0b1da0 msgr2=0x7f82fc15d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 shutdown_connections 2024-05-09T06:36:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.738+0000 7f8304329640 1 -- 172.21.15.1:0/70679759 wait complete. 2024-05-09T06:36:38.732 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:36:38.733 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:38.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.862+0000 7f2a310d6640 1 Processor -- start 2024-05-09T06:36:38.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.862+0000 7f2a310d6640 1 -- start start 2024-05-09T06:36:38.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a310d6640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a2c164ce0 0x7f2a2c1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a310d6640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c071c20 con 0x7f2a2c1454b0 2024-05-09T06:36:38.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a310d6640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c06eb40 con 0x7f2a2c0b2a60 2024-05-09T06:36:38.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a310d6640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2c06ee20 con 0x7f2a2c164ce0 2024-05-09T06:36:38.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a2ad9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2a2c1454b0 0x7f2a2c0b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57800/0 (socket says 172.21.15.1:57800) 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a2ad9d640 1 -- 172.21.15.1:0/1806484491 learned_addr learned my addr 172.21.15.1:0/1806484491 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a2b59e640 1 --2- 172.21.15.1:0/1806484491 >> v2:172.21.15.6:3300/0 conn(0x7f2a2c164ce0 0x7f2a2c1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301671586 0 0) 0x7f2a2c071c20 con 0x7f2a2c1454b0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.863+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a18003520 con 0x7f2a2c1454b0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798697155 0 0) 0x7f2a2c06eb40 con 0x7f2a2c0b2a60 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a2c071c20 con 0x7f2a2c0b2a60 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2634450533 0 0) 0x7f2a18003520 con 0x7f2a2c1454b0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a2b59e640 1 -- 172.21.15.1:0/1806484491 >> v1:172.21.15.6:6789/0 conn(0x7f2a2c0b2a60 legacy=0x7f2a2c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a2b59e640 1 -- 172.21.15.1:0/1806484491 >> v1:172.21.15.1:6789/0 conn(0x7f2a2c1454b0 legacy=0x7f2a2c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a2b59e640 1 -- 172.21.15.1:0/1806484491 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c06d8e0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.864+0000 7f2a2b59e640 1 --2- 172.21.15.1:0/1806484491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1670d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a2402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91cb11c50ecd5309 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c001670 con 0x7f2a2c164ce0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24007540 con 0x7f2a2c164ce0 2024-05-09T06:36:38.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a29d9b640 1 -- 172.21.15.1:0/1806484491 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24007910 con 0x7f2a2c164ce0 2024-05-09T06:36:38.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 -- 172.21.15.1:0/1806484491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c164ce0 msgr2=0x7f2a2c1670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 --2- 172.21.15.1:0/1806484491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1670d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a2402a680 comp rx=0 tx=0).stop 2024-05-09T06:36:38.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 -- 172.21.15.1:0/1806484491 shutdown_connections 2024-05-09T06:36:38.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 --2- 172.21.15.1:0/1806484491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1670d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 -- 172.21.15.1:0/1806484491 >> 172.21.15.1:0/1806484491 conn(0x7f2a2c0cc4b0 msgr2=0x7f2a2c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 -- 172.21.15.1:0/1806484491 shutdown_connections 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.865+0000 7f2a310d6640 1 -- 172.21.15.1:0/1806484491 wait complete. 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.867+0000 7f2a310d6640 1 Processor -- start 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.867+0000 7f2a310d6640 1 -- start start 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a2c0b2a60 0x7f2a2c1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c1454b0 0x7f2a2c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0b5140 con 0x7f2a2c164ce0 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c145890 con 0x7f2a2c1454b0 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a310d6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c06e0e0 con 0x7f2a2c0b2a60 2024-05-09T06:36:38.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55370/0 (socket says 172.21.15.1:55370) 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 -- 172.21.15.1:0/795269681 learned_addr learned my addr 172.21.15.1:0/795269681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2a59c640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c1454b0 0x7f2a2c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2ad9d640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a2c0b2a60 0x7f2a2c1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 -- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a2c0b2a60 msgr2=0x7f2a2c1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a2c0b2a60 0x7f2a2c1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 -- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c1454b0 msgr2=0x7f2a2c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c1454b0 0x7f2a2c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 -- 172.21.15.1:0/795269681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c0c5510 con 0x7f2a2c164ce0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.868+0000 7f2a2b59e640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.869+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2403f070 con 0x7f2a2c164ce0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.869+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24031bc0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.869+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c0c57f0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.869+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24007e90 con 0x7f2a2c164ce0 2024-05-09T06:36:38.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.869+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c0ad1c0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.870+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c067ac0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.870+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a2403b410 con 0x7f2a2c164ce0 2024-05-09T06:36:38.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.871+0000 7f2a237fe640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 0x7f2a0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.871+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f2a240ac1c0 con 0x7f2a2c164ce0 2024-05-09T06:36:38.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.871+0000 7f2a2ad9d640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 0x7f2a0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.871+0000 7f2a2ad9d640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 0x7f2a0c06e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0017e0 tx=0x7f2a1c001610 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:38.875+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a2c067ac0 con 0x7f2a2c164ce0 2024-05-09T06:36:39.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.097+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a2c0752b0 con 0x7f2a2c164ce0 2024-05-09T06:36:39.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.098+0000 7f2a237fe640 1 -- 172.21.15.1:0/795269681 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5171 (secure 0 0 0) 0x7f2a2407a380 con 0x7f2a2c164ce0 2024-05-09T06:36:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:39.079 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:36:38.092253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6835/3796896332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3796896332},{"type":"v1","addr":"172.21.15.1:6835","nonce":3796896332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":7}]} 2024-05-09T06:36:39.079 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 82 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 msgr2=0x7f2a0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 0x7f2a0c06e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0017e0 tx=0x7f2a1c001610 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 msgr2=0x7f2a2c1436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a24004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 shutdown_connections 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2a0c06be30 0x7f2a0c06e2f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a2c164ce0 0x7f2a2c1436e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a2c1454b0 0x7f2a2c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 --2- 172.21.15.1:0/795269681 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a2c0b2a60 0x7f2a2c1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 >> 172.21.15.1:0/795269681 conn(0x7f2a2c0cc4b0 msgr2=0x7f2a2c141740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 shutdown_connections 2024-05-09T06:36:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.100+0000 7f2a310d6640 1 -- 172.21.15.1:0/795269681 wait complete. 2024-05-09T06:36:39.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.215+0000 7f94361f9640 1 Processor -- start 2024-05-09T06:36:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.215+0000 7f94361f9640 1 -- start start 2024-05-09T06:36:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f94361f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94301454b0 0x7f94300b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f94361f9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943006eb40 con 0x7f94300b2a60 2024-05-09T06:36:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f94361f9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943006ee20 con 0x7f94300b4f70 2024-05-09T06:36:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f94361f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f943006d8e0 con 0x7f94301454b0 2024-05-09T06:36:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942effd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f94300b2a60 0x7f94300b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57810/0 (socket says 172.21.15.1:57810) 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94301454b0 0x7f94300b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942effd640 1 -- 172.21.15.1:0/3631478876 learned_addr learned my addr 172.21.15.1:0/3631478876 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942f7fe640 1 -- 172.21.15.1:0/3631478876 >> v1:172.21.15.6:6789/0 conn(0x7f94300b4f70 legacy=0x7f9430070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942f7fe640 1 -- 172.21.15.1:0/3631478876 >> v1:172.21.15.1:6789/0 conn(0x7f94300b2a60 legacy=0x7f94300b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942f7fe640 1 -- 172.21.15.1:0/3631478876 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943006dbc0 con 0x7f94301454b0 2024-05-09T06:36:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.216+0000 7f942e7fc640 1 -- 172.21.15.1:0/3631478876 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380625465 0 0) 0x7f943006eb40 con 0x7f94300b2a60 2024-05-09T06:36:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.217+0000 7f942f7fe640 1 --2- 172.21.15.1:0/3631478876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94301454b0 0x7f94300b4320 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2368b5dd51555080 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.217+0000 7f942e7fc640 1 -- 172.21.15.1:0/3631478876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420037070 con 0x7f94301454b0 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.217+0000 7f942e7fc640 1 -- 172.21.15.1:0/3631478876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420032440 con 0x7f94301454b0 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.217+0000 7f942e7fc640 1 -- 172.21.15.1:0/3631478876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420032810 con 0x7f94301454b0 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 -- 172.21.15.1:0/3631478876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94301454b0 msgr2=0x7f94300b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 --2- 172.21.15.1:0/3631478876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94301454b0 0x7f94300b4320 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 -- 172.21.15.1:0/3631478876 shutdown_connections 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 --2- 172.21.15.1:0/3631478876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94301454b0 0x7f94300b4320 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 -- 172.21.15.1:0/3631478876 >> 172.21.15.1:0/3631478876 conn(0x7f94300cc4b0 msgr2=0x7f94300cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 -- 172.21.15.1:0/3631478876 shutdown_connections 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.218+0000 7f94361f9640 1 -- 172.21.15.1:0/3631478876 wait complete. 2024-05-09T06:36:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.219+0000 7f94361f9640 1 Processor -- start 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.219+0000 7f94361f9640 1 -- start start 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 0x7f943014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94301454b0 0x7f943013f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f943006e260 con 0x7f94300b2a60 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f943006e0e0 con 0x7f94300b4f70 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f94361f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9430071320 con 0x7f94301454b0 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55380/0 (socket says 172.21.15.1:55380) 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942f7fe640 1 -- 172.21.15.1:0/2265062596 learned_addr learned my addr 172.21.15.1:0/2265062596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942effd640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 0x7f943014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942ffff640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94301454b0 0x7f943013f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942f7fe640 1 -- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94301454b0 msgr2=0x7f943013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.220+0000 7f942f7fe640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94301454b0 0x7f943013f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f942f7fe640 1 -- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 msgr2=0x7f943014e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f942f7fe640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 0x7f943014e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f942f7fe640 1 -- 172.21.15.1:0/2265062596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300ad320 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f942effd640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 0x7f943014e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f942f7fe640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f94200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420037040 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420042070 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94300ad600 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420030a70 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.221+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94300adbc0 con 0x7f94300b2a60 2024-05-09T06:36:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.222+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94300752b0 con 0x7f94300b2a60 2024-05-09T06:36:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.223+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9420030c10 con 0x7f94300b2a60 2024-05-09T06:36:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.223+0000 7f9417fff640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 0x7f941c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.223+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f94200abac0 con 0x7f94300b2a60 2024-05-09T06:36:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.223+0000 7f942effd640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 0x7f941c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.224+0000 7f942effd640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 0x7f941c06e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f94301f5350 tx=0x7f9424009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.227+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9420045360 con 0x7f94300b2a60 2024-05-09T06:36:39.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.410+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9430072fa0 con 0x7f94300b2a60 2024-05-09T06:36:39.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.411+0000 7f9417fff640 1 -- 172.21.15.1:0/2265062596 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16752 (secure 0 0 0) 0x7f942007a2e0 con 0x7f94300b2a60 2024-05-09T06:36:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:39.392 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":106,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:36.717091+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 msgr2=0x7f941c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 0x7f941c06e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f94301f5350 tx=0x7f9424009040 comp rx=0 tx=0).stop 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 msgr2=0x7f94300ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f94200046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 shutdown_connections 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f941c06be30 0x7f941c06e2f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94301454b0 0x7f943013f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94300b4f70 0x7f943014e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 --2- 172.21.15.1:0/2265062596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94300b2a60 0x7f94300ae3c0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 >> 172.21.15.1:0/2265062596 conn(0x7f94300cc4b0 msgr2=0x7f94300cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 shutdown_connections 2024-05-09T06:36:39.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.413+0000 7f94361f9640 1 -- 172.21.15.1:0/2265062596 wait complete. 2024-05-09T06:36:39.405 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:36:39.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.528+0000 7f548fc6b640 1 Processor -- start 2024-05-09T06:36:39.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.528+0000 7f548fc6b640 1 -- start start 2024-05-09T06:36:39.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548fc6b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5488099b30 0x7f54880b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548fc6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f548815e4b0 0x7f548815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548fc6b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5488099930 con 0x7f54880c6860 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548fc6b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f54880b1e20 con 0x7f5488099b30 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548fc6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f548815ee90 con 0x7f548815e4b0 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548da07640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f54880c6860 0x7f54880c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57814/0 (socket says 172.21.15.1:57814) 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548da07640 1 -- 172.21.15.1:0/914335503 learned_addr learned my addr 172.21.15.1:0/914335503 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.529+0000 7f548d206640 1 --2- 172.21.15.1:0/914335503 >> v2:172.21.15.6:3300/0 conn(0x7f5488099b30 0x7f54880b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548e208640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f548815e4b0 0x7f548815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548ca05640 1 -- 172.21.15.1:0/914335503 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 55779989 0 0) 0x7f5484001110 con 0x7f54880c6860 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548ca05640 1 -- 172.21.15.1:0/914335503 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5474003520 con 0x7f54880c6860 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548d206640 1 -- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f548815e4b0 msgr2=0x7f548815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548d206640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f548815e4b0 0x7f548815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548d206640 1 -- 172.21.15.1:0/914335503 >> v1:172.21.15.1:6789/0 conn(0x7f54880c6860 legacy=0x7f54880c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.530+0000 7f548d206640 1 -- 172.21.15.1:0/914335503 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5488162930 con 0x7f5488099b30 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548d206640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f54880b2aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54780097a0 tx=0x7f547802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8880a4334937e74d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548ca05640 1 -- 172.21.15.1:0/914335503 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478037070 con 0x7f5488099b30 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548ca05640 1 -- 172.21.15.1:0/914335503 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5478032440 con 0x7f5488099b30 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548ca05640 1 -- 172.21.15.1:0/914335503 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478032810 con 0x7f5488099b30 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 -- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 msgr2=0x7f54880b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f54880b2aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54780097a0 tx=0x7f547802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 -- 172.21.15.1:0/914335503 shutdown_connections 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f548815e4b0 0x7f548815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 --2- 172.21.15.1:0/914335503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f54880b2aa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 -- 172.21.15.1:0/914335503 >> 172.21.15.1:0/914335503 conn(0x7f54880b1fb0 msgr2=0x7f54880b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 -- 172.21.15.1:0/914335503 shutdown_connections 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.531+0000 7f548fc6b640 1 -- 172.21.15.1:0/914335503 wait complete. 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 Processor -- start 2024-05-09T06:36:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 -- start start 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f5488156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54880c6860 0x7f54880b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f548815f860 con 0x7f548815e4b0 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f548815eb20 con 0x7f5488099b30 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548fc6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f548815f9e0 con 0x7f54880c6860 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.533+0000 7f548e208640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55394/0 (socket says 172.21.15.1:55394) 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 -- 172.21.15.1:0/3876295687 learned_addr learned my addr 172.21.15.1:0/3876295687 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548da07640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f5488156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548d206640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54880c6860 0x7f54880b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 -- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54880c6860 msgr2=0x7f54880b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54880c6860 0x7f54880b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 -- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 msgr2=0x7f5488156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f5488156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 -- 172.21.15.1:0/3876295687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54880ca560 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548e208640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5480009d30 tx=0x7f548000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480018020 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5480014480 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54880caa50 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480014870 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.534+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54880cb010 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.535+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54880d01a0 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.536+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5480005d10 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.536+0000 7f547e7fc640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.536+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f548008b800 con 0x7f548815e4b0 2024-05-09T06:36:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.536+0000 7f548da07640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.537+0000 7f548da07640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 0x7f546c06e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5484005880 tx=0x7f5484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.541+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54880d01a0 con 0x7f548815e4b0 2024-05-09T06:36:39.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.719+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54880752b0 con 0x7f546c06be30 2024-05-09T06:36:39.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.725+0000 7f547e7fc640 1 -- 172.21.15.1:0/3876295687 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f54880752b0 con 0x7f546c06be30 2024-05-09T06:36:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:36:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 msgr2=0x7f546c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 0x7f546c06e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5484005880 tx=0x7f5484009040 comp rx=0 tx=0).stop 2024-05-09T06:36:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 msgr2=0x7f54880cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5480009d30 tx=0x7f548000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 shutdown_connections 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548815e4b0 0x7f54880cb810 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54880c6860 0x7f54880b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 --2- 172.21.15.1:0/3876295687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5488099b30 0x7f5488156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 >> 172.21.15.1:0/3876295687 conn(0x7f54880b1fb0 msgr2=0x7f548815d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 shutdown_connections 2024-05-09T06:36:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.728+0000 7f548fc6b640 1 -- 172.21.15.1:0/3876295687 wait complete. 2024-05-09T06:36:39.722 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-09T06:36:39.194729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260972,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713748,"statfs":{"total":773094113280,"available":772826877952,"internally_reserved":0,"allocated":11771904,"data_stored":6085716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.493671"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362011+0000","last_change":"2024-05-09T06:36:35.724735+0000","last_active":"2024-05-09T06:36:37.362011+0000","last_peered":"2024-05-09T06:36:37.362011+0000","last_clean":"2024-05-09T06:36:37.362011+0000","last_became_active":"2024-05-09T06:36:35.723937+0000","last_became_peered":"2024-05-09T06:36:35.723937+0000","last_unstale":"2024-05-09T06:36:37.362011+0000","last_undegraded":"2024-05-09T06:36:37.362011+0000","last_fullsized":"2024-05-09T06:36:37.362011+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:04.508786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362868+0000","last_change":"2024-05-09T06:36:35.727084+0000","last_active":"2024-05-09T06:36:37.362868+0000","last_peered":"2024-05-09T06:36:37.362868+0000","last_clean":"2024-05-09T06:36:37.362868+0000","last_became_active":"2024-05-09T06:36:35.726770+0000","last_became_peered":"2024-05-09T06:36:35.726770+0000","last_unstale":"2024-05-09T06:36:37.362868+0000","last_undegraded":"2024-05-09T06:36:37.362868+0000","last_fullsized":"2024-05-09T06:36:37.362868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:43.516265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361738+0000","last_change":"2024-05-09T06:36:35.733544+0000","last_active":"2024-05-09T06:36:37.361738+0000","last_peered":"2024-05-09T06:36:37.361738+0000","last_clean":"2024-05-09T06:36:37.361738+0000","last_became_active":"2024-05-09T06:36:35.732638+0000","last_became_peered":"2024-05-09T06:36:35.732638+0000","last_unstale":"2024-05-09T06:36:37.361738+0000","last_undegraded":"2024-05-09T06:36:37.361738+0000","last_fullsized":"2024-05-09T06:36:37.361738+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:53.648280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727802+0000","last_change":"2024-05-09T06:36:35.738005+0000","last_active":"2024-05-09T06:36:36.727802+0000","last_peered":"2024-05-09T06:36:36.727802+0000","last_clean":"2024-05-09T06:36:36.727802+0000","last_became_active":"2024-05-09T06:36:35.737695+0000","last_became_peered":"2024-05-09T06:36:35.737695+0000","last_unstale":"2024-05-09T06:36:36.727802+0000","last_undegraded":"2024-05-09T06:36:36.727802+0000","last_fullsized":"2024-05-09T06:36:36.727802+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:18.486814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171632+0000","last_change":"2024-05-09T06:36:35.726555+0000","last_active":"2024-05-09T06:36:37.171632+0000","last_peered":"2024-05-09T06:36:37.171632+0000","last_clean":"2024-05-09T06:36:37.171632+0000","last_became_active":"2024-05-09T06:36:35.726257+0000","last_became_peered":"2024-05-09T06:36:35.726257+0000","last_unstale":"2024-05-09T06:36:37.171632+0000","last_undegraded":"2024-05-09T06:36:37.171632+0000","last_fullsized":"2024-05-09T06:36:37.171632+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361921+0000","last_change":"2024-05-09T06:36:35.731952+0000","last_active":"2024-05-09T06:36:37.361921+0000","last_peered":"2024-05-09T06:36:37.361921+0000","last_clean":"2024-05-09T06:36:37.361921+0000","last_became_active":"2024-05-09T06:36:35.731626+0000","last_became_peered":"2024-05-09T06:36:35.731626+0000","last_unstale":"2024-05-09T06:36:37.361921+0000","last_undegraded":"2024-05-09T06:36:37.361921+0000","last_fullsized":"2024-05-09T06:36:37.361921+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361800+0000","last_change":"2024-05-09T06:36:35.737819+0000","last_active":"2024-05-09T06:36:37.361800+0000","last_peered":"2024-05-09T06:36:37.361800+0000","last_clean":"2024-05-09T06:36:37.361800+0000","last_became_active":"2024-05-09T06:36:35.737499+0000","last_became_peered":"2024-05-09T06:36:35.737499+0000","last_unstale":"2024-05-09T06:36:37.361800+0000","last_undegraded":"2024-05-09T06:36:37.361800+0000","last_fullsized":"2024-05-09T06:36:37.361800+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860713+0000","last_change":"2024-05-09T06:36:35.734691+0000","last_active":"2024-05-09T06:36:36.860713+0000","last_peered":"2024-05-09T06:36:36.860713+0000","last_clean":"2024-05-09T06:36:36.860713+0000","last_became_active":"2024-05-09T06:36:35.733909+0000","last_became_peered":"2024-05-09T06:36:35.733909+0000","last_unstale":"2024-05-09T06:36:36.860713+0000","last_undegraded":"2024-05-09T06:36:36.860713+0000","last_fullsized":"2024-05-09T06:36:36.860713+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:57.761138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362769+0000","last_change":"2024-05-09T06:36:35.725078+0000","last_active":"2024-05-09T06:36:37.362769+0000","last_peered":"2024-05-09T06:36:37.362769+0000","last_clean":"2024-05-09T06:36:37.362769+0000","last_became_active":"2024-05-09T06:36:35.724253+0000","last_became_peered":"2024-05-09T06:36:35.724253+0000","last_unstale":"2024-05-09T06:36:37.362769+0000","last_undegraded":"2024-05-09T06:36:37.362769+0000","last_fullsized":"2024-05-09T06:36:37.362769+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:43.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171910+0000","last_change":"2024-05-09T06:36:35.738687+0000","last_active":"2024-05-09T06:36:37.171910+0000","last_peered":"2024-05-09T06:36:37.171910+0000","last_clean":"2024-05-09T06:36:37.171910+0000","last_became_active":"2024-05-09T06:36:35.738088+0000","last_became_peered":"2024-05-09T06:36:35.738088+0000","last_unstale":"2024-05-09T06:36:37.171910+0000","last_undegraded":"2024-05-09T06:36:37.171910+0000","last_fullsized":"2024-05-09T06:36:37.171910+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:43.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362661+0000","last_change":"2024-05-09T06:36:35.738177+0000","last_active":"2024-05-09T06:36:37.362661+0000","last_peered":"2024-05-09T06:36:37.362661+0000","last_clean":"2024-05-09T06:36:37.362661+0000","last_became_active":"2024-05-09T06:36:35.737877+0000","last_became_peered":"2024-05-09T06:36:35.737877+0000","last_unstale":"2024-05-09T06:36:37.362661+0000","last_undegraded":"2024-05-09T06:36:37.362661+0000","last_fullsized":"2024-05-09T06:36:37.362661+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733442+0000","last_change":"2024-05-09T06:36:35.728824+0000","last_active":"2024-05-09T06:36:36.733442+0000","last_peered":"2024-05-09T06:36:36.733442+0000","last_clean":"2024-05-09T06:36:36.733442+0000","last_became_active":"2024-05-09T06:36:35.728460+0000","last_became_peered":"2024-05-09T06:36:35.728460+0000","last_unstale":"2024-05-09T06:36:36.733442+0000","last_undegraded":"2024-05-09T06:36:36.733442+0000","last_fullsized":"2024-05-09T06:36:36.733442+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727815+0000","last_change":"2024-05-09T06:36:35.730746+0000","last_active":"2024-05-09T06:36:36.727815+0000","last_peered":"2024-05-09T06:36:36.727815+0000","last_clean":"2024-05-09T06:36:36.727815+0000","last_became_active":"2024-05-09T06:36:35.730477+0000","last_became_peered":"2024-05-09T06:36:35.730477+0000","last_unstale":"2024-05-09T06:36:36.727815+0000","last_undegraded":"2024-05-09T06:36:36.727815+0000","last_fullsized":"2024-05-09T06:36:36.727815+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:30.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884342+0000","last_change":"2024-05-09T06:36:35.728860+0000","last_active":"2024-05-09T06:36:36.884342+0000","last_peered":"2024-05-09T06:36:36.884342+0000","last_clean":"2024-05-09T06:36:36.884342+0000","last_became_active":"2024-05-09T06:36:35.728568+0000","last_became_peered":"2024-05-09T06:36:35.728568+0000","last_unstale":"2024-05-09T06:36:36.884342+0000","last_undegraded":"2024-05-09T06:36:36.884342+0000","last_fullsized":"2024-05-09T06:36:36.884342+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884318+0000","last_change":"2024-05-09T06:36:35.732870+0000","last_active":"2024-05-09T06:36:36.884318+0000","last_peered":"2024-05-09T06:36:36.884318+0000","last_clean":"2024-05-09T06:36:36.884318+0000","last_became_active":"2024-05-09T06:36:35.732578+0000","last_became_peered":"2024-05-09T06:36:35.732578+0000","last_unstale":"2024-05-09T06:36:36.884318+0000","last_undegraded":"2024-05-09T06:36:36.884318+0000","last_fullsized":"2024-05-09T06:36:36.884318+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884277+0000","last_change":"2024-05-09T06:36:35.736396+0000","last_active":"2024-05-09T06:36:36.884277+0000","last_peered":"2024-05-09T06:36:36.884277+0000","last_clean":"2024-05-09T06:36:36.884277+0000","last_became_active":"2024-05-09T06:36:35.736094+0000","last_became_peered":"2024-05-09T06:36:35.736094+0000","last_unstale":"2024-05-09T06:36:36.884277+0000","last_undegraded":"2024-05-09T06:36:36.884277+0000","last_fullsized":"2024-05-09T06:36:36.884277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362031+0000","last_change":"2024-05-09T06:36:35.729017+0000","last_active":"2024-05-09T06:36:37.362031+0000","last_peered":"2024-05-09T06:36:37.362031+0000","last_clean":"2024-05-09T06:36:37.362031+0000","last_became_active":"2024-05-09T06:36:35.728741+0000","last_became_peered":"2024-05-09T06:36:35.728741+0000","last_unstale":"2024-05-09T06:36:37.362031+0000","last_undegraded":"2024-05-09T06:36:37.362031+0000","last_fullsized":"2024-05-09T06:36:37.362031+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:54:21.333941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362876+0000","last_change":"2024-05-09T06:36:35.730249+0000","last_active":"2024-05-09T06:36:37.362876+0000","last_peered":"2024-05-09T06:36:37.362876+0000","last_clean":"2024-05-09T06:36:37.362876+0000","last_became_active":"2024-05-09T06:36:35.729959+0000","last_became_peered":"2024-05-09T06:36:35.729959+0000","last_unstale":"2024-05-09T06:36:37.362876+0000","last_undegraded":"2024-05-09T06:36:37.362876+0000","last_fullsized":"2024-05-09T06:36:37.362876+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:57.577919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361864+0000","last_change":"2024-05-09T06:36:35.737416+0000","last_active":"2024-05-09T06:36:37.361864+0000","last_peered":"2024-05-09T06:36:37.361864+0000","last_clean":"2024-05-09T06:36:37.361864+0000","last_became_active":"2024-05-09T06:36:35.736971+0000","last_became_peered":"2024-05-09T06:36:35.736971+0000","last_unstale":"2024-05-09T06:36:37.361864+0000","last_undegraded":"2024-05-09T06:36:37.361864+0000","last_fullsized":"2024-05-09T06:36:37.361864+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:08.805142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733616+0000","last_change":"2024-05-09T06:36:35.728135+0000","last_active":"2024-05-09T06:36:36.733616+0000","last_peered":"2024-05-09T06:36:36.733616+0000","last_clean":"2024-05-09T06:36:36.733616+0000","last_became_active":"2024-05-09T06:36:35.727832+0000","last_became_peered":"2024-05-09T06:36:35.727832+0000","last_unstale":"2024-05-09T06:36:36.733616+0000","last_undegraded":"2024-05-09T06:36:36.733616+0000","last_fullsized":"2024-05-09T06:36:36.733616+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861218+0000","last_change":"2024-05-09T06:36:35.743735+0000","last_active":"2024-05-09T06:36:36.861218+0000","last_peered":"2024-05-09T06:36:36.861218+0000","last_clean":"2024-05-09T06:36:36.861218+0000","last_became_active":"2024-05-09T06:36:35.743443+0000","last_became_peered":"2024-05-09T06:36:35.743443+0000","last_unstale":"2024-05-09T06:36:36.861218+0000","last_undegraded":"2024-05-09T06:36:36.861218+0000","last_fullsized":"2024-05-09T06:36:36.861218+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:57.299598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171618+0000","last_change":"2024-05-09T06:36:35.735060+0000","last_active":"2024-05-09T06:36:37.171618+0000","last_peered":"2024-05-09T06:36:37.171618+0000","last_clean":"2024-05-09T06:36:37.171618+0000","last_became_active":"2024-05-09T06:36:35.734781+0000","last_became_peered":"2024-05-09T06:36:35.734781+0000","last_unstale":"2024-05-09T06:36:37.171618+0000","last_undegraded":"2024-05-09T06:36:37.171618+0000","last_fullsized":"2024-05-09T06:36:37.171618+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361639+0000","last_change":"2024-05-09T06:36:35.728446+0000","last_active":"2024-05-09T06:36:37.361639+0000","last_peered":"2024-05-09T06:36:37.361639+0000","last_clean":"2024-05-09T06:36:37.361639+0000","last_became_active":"2024-05-09T06:36:35.728041+0000","last_became_peered":"2024-05-09T06:36:35.728041+0000","last_unstale":"2024-05-09T06:36:37.361639+0000","last_undegraded":"2024-05-09T06:36:37.361639+0000","last_fullsized":"2024-05-09T06:36:37.361639+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:20.155376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171430+0000","last_change":"2024-05-09T06:36:35.718304+0000","last_active":"2024-05-09T06:36:37.171430+0000","last_peered":"2024-05-09T06:36:37.171430+0000","last_clean":"2024-05-09T06:36:37.171430+0000","last_became_active":"2024-05-09T06:36:35.717974+0000","last_became_peered":"2024-05-09T06:36:35.717974+0000","last_unstale":"2024-05-09T06:36:37.171430+0000","last_undegraded":"2024-05-09T06:36:37.171430+0000","last_fullsized":"2024-05-09T06:36:37.171430+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733536+0000","last_change":"2024-05-09T06:36:35.735966+0000","last_active":"2024-05-09T06:36:36.733536+0000","last_peered":"2024-05-09T06:36:36.733536+0000","last_clean":"2024-05-09T06:36:36.733536+0000","last_became_active":"2024-05-09T06:36:35.735641+0000","last_became_peered":"2024-05-09T06:36:35.735641+0000","last_unstale":"2024-05-09T06:36:36.733536+0000","last_undegraded":"2024-05-09T06:36:36.733536+0000","last_fullsized":"2024-05-09T06:36:36.733536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733424+0000","last_change":"2024-05-09T06:36:35.738894+0000","last_active":"2024-05-09T06:36:36.733424+0000","last_peered":"2024-05-09T06:36:36.733424+0000","last_clean":"2024-05-09T06:36:36.733424+0000","last_became_active":"2024-05-09T06:36:35.738562+0000","last_became_peered":"2024-05-09T06:36:35.738562+0000","last_unstale":"2024-05-09T06:36:36.733424+0000","last_undegraded":"2024-05-09T06:36:36.733424+0000","last_fullsized":"2024-05-09T06:36:36.733424+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363129+0000","last_change":"2024-05-09T06:36:35.730775+0000","last_active":"2024-05-09T06:36:37.363129+0000","last_peered":"2024-05-09T06:36:37.363129+0000","last_clean":"2024-05-09T06:36:37.363129+0000","last_became_active":"2024-05-09T06:36:35.728173+0000","last_became_peered":"2024-05-09T06:36:35.728173+0000","last_unstale":"2024-05-09T06:36:37.363129+0000","last_undegraded":"2024-05-09T06:36:37.363129+0000","last_fullsized":"2024-05-09T06:36:37.363129+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362292+0000","last_change":"2024-05-09T06:36:35.731737+0000","last_active":"2024-05-09T06:36:37.362292+0000","last_peered":"2024-05-09T06:36:37.362292+0000","last_clean":"2024-05-09T06:36:37.362292+0000","last_became_active":"2024-05-09T06:36:35.731409+0000","last_became_peered":"2024-05-09T06:36:35.731409+0000","last_unstale":"2024-05-09T06:36:37.362292+0000","last_undegraded":"2024-05-09T06:36:37.362292+0000","last_fullsized":"2024-05-09T06:36:37.362292+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861277+0000","last_change":"2024-05-09T06:36:35.734977+0000","last_active":"2024-05-09T06:36:36.861277+0000","last_peered":"2024-05-09T06:36:36.861277+0000","last_clean":"2024-05-09T06:36:36.861277+0000","last_became_active":"2024-05-09T06:36:35.734651+0000","last_became_peered":"2024-05-09T06:36:35.734651+0000","last_unstale":"2024-05-09T06:36:36.861277+0000","last_undegraded":"2024-05-09T06:36:36.861277+0000","last_fullsized":"2024-05-09T06:36:36.861277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:27.486929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732999+0000","last_change":"2024-05-09T06:36:35.731456+0000","last_active":"2024-05-09T06:36:36.732999+0000","last_peered":"2024-05-09T06:36:36.732999+0000","last_clean":"2024-05-09T06:36:36.732999+0000","last_became_active":"2024-05-09T06:36:35.731133+0000","last_became_peered":"2024-05-09T06:36:35.731133+0000","last_unstale":"2024-05-09T06:36:36.732999+0000","last_undegraded":"2024-05-09T06:36:36.732999+0000","last_fullsized":"2024-05-09T06:36:36.732999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361943+0000","last_change":"2024-05-09T06:36:35.737479+0000","last_active":"2024-05-09T06:36:37.361943+0000","last_peered":"2024-05-09T06:36:37.361943+0000","last_clean":"2024-05-09T06:36:37.361943+0000","last_became_active":"2024-05-09T06:36:35.736956+0000","last_became_peered":"2024-05-09T06:36:35.736956+0000","last_unstale":"2024-05-09T06:36:37.361943+0000","last_undegraded":"2024-05-09T06:36:37.361943+0000","last_fullsized":"2024-05-09T06:36:37.361943+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860767+0000","last_change":"2024-05-09T06:36:35.739397+0000","last_active":"2024-05-09T06:36:36.860767+0000","last_peered":"2024-05-09T06:36:36.860767+0000","last_clean":"2024-05-09T06:36:36.860767+0000","last_became_active":"2024-05-09T06:36:35.739043+0000","last_became_peered":"2024-05-09T06:36:35.739043+0000","last_unstale":"2024-05-09T06:36:36.860767+0000","last_undegraded":"2024-05-09T06:36:36.860767+0000","last_fullsized":"2024-05-09T06:36:36.860767+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:26.873134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361775+0000","last_change":"2024-05-09T06:36:35.736463+0000","last_active":"2024-05-09T06:36:37.361775+0000","last_peered":"2024-05-09T06:36:37.361775+0000","last_clean":"2024-05-09T06:36:37.361775+0000","last_became_active":"2024-05-09T06:36:35.736195+0000","last_became_peered":"2024-05-09T06:36:35.736195+0000","last_unstale":"2024-05-09T06:36:37.361775+0000","last_undegraded":"2024-05-09T06:36:37.361775+0000","last_fullsized":"2024-05-09T06:36:37.361775+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:22.442411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728260+0000","last_change":"2024-05-09T06:36:34.726379+0000","last_active":"2024-05-09T06:36:36.728260+0000","last_peered":"2024-05-09T06:36:36.728260+0000","last_clean":"2024-05-09T06:36:36.728260+0000","last_became_active":"2024-05-09T06:36:34.726052+0000","last_became_peered":"2024-05-09T06:36:34.726052+0000","last_unstale":"2024-05-09T06:36:36.728260+0000","last_undegraded":"2024-05-09T06:36:36.728260+0000","last_fullsized":"2024-05-09T06:36:36.728260+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:45.597809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860744+0000","last_change":"2024-05-09T06:36:35.735129+0000","last_active":"2024-05-09T06:36:36.860744+0000","last_peered":"2024-05-09T06:36:36.860744+0000","last_clean":"2024-05-09T06:36:36.860744+0000","last_became_active":"2024-05-09T06:36:35.734800+0000","last_became_peered":"2024-05-09T06:36:35.734800+0000","last_unstale":"2024-05-09T06:36:36.860744+0000","last_undegraded":"2024-05-09T06:36:36.860744+0000","last_fullsized":"2024-05-09T06:36:36.860744+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:17.925190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860977+0000","last_change":"2024-05-09T06:36:34.720647+0000","last_active":"2024-05-09T06:36:36.860977+0000","last_peered":"2024-05-09T06:36:36.860977+0000","last_clean":"2024-05-09T06:36:36.860977+0000","last_became_active":"2024-05-09T06:36:34.720291+0000","last_became_peered":"2024-05-09T06:36:34.720291+0000","last_unstale":"2024-05-09T06:36:36.860977+0000","last_undegraded":"2024-05-09T06:36:36.860977+0000","last_fullsized":"2024-05-09T06:36:36.860977+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:21.990698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733065+0000","last_change":"2024-05-09T06:36:35.727850+0000","last_active":"2024-05-09T06:36:36.733065+0000","last_peered":"2024-05-09T06:36:36.733065+0000","last_clean":"2024-05-09T06:36:36.733065+0000","last_became_active":"2024-05-09T06:36:35.727537+0000","last_became_peered":"2024-05-09T06:36:35.727537+0000","last_unstale":"2024-05-09T06:36:36.733065+0000","last_undegraded":"2024-05-09T06:36:36.733065+0000","last_fullsized":"2024-05-09T06:36:36.733065+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733323+0000","last_change":"2024-05-09T06:36:34.722771+0000","last_active":"2024-05-09T06:36:36.733323+0000","last_peered":"2024-05-09T06:36:36.733323+0000","last_clean":"2024-05-09T06:36:36.733323+0000","last_became_active":"2024-05-09T06:36:34.722433+0000","last_became_peered":"2024-05-09T06:36:34.722433+0000","last_unstale":"2024-05-09T06:36:36.733323+0000","last_undegraded":"2024-05-09T06:36:36.733323+0000","last_fullsized":"2024-05-09T06:36:36.733323+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:12.993813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362672+0000","last_change":"2024-05-09T06:36:35.735357+0000","last_active":"2024-05-09T06:36:37.362672+0000","last_peered":"2024-05-09T06:36:37.362672+0000","last_clean":"2024-05-09T06:36:37.362672+0000","last_became_active":"2024-05-09T06:36:35.735062+0000","last_became_peered":"2024-05-09T06:36:35.735062+0000","last_unstale":"2024-05-09T06:36:37.362672+0000","last_undegraded":"2024-05-09T06:36:37.362672+0000","last_fullsized":"2024-05-09T06:36:37.362672+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171838+0000","last_change":"2024-05-09T06:36:34.727648+0000","last_active":"2024-05-09T06:36:37.171838+0000","last_peered":"2024-05-09T06:36:37.171838+0000","last_clean":"2024-05-09T06:36:37.171838+0000","last_became_active":"2024-05-09T06:36:34.727344+0000","last_became_peered":"2024-05-09T06:36:34.727344+0000","last_unstale":"2024-05-09T06:36:37.171838+0000","last_undegraded":"2024-05-09T06:36:37.171838+0000","last_fullsized":"2024-05-09T06:36:37.171838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727933+0000","last_change":"2024-05-09T06:36:35.729403+0000","last_active":"2024-05-09T06:36:36.727933+0000","last_peered":"2024-05-09T06:36:36.727933+0000","last_clean":"2024-05-09T06:36:36.727933+0000","last_became_active":"2024-05-09T06:36:35.729145+0000","last_became_peered":"2024-05-09T06:36:35.729145+0000","last_unstale":"2024-05-09T06:36:36.727933+0000","last_undegraded":"2024-05-09T06:36:36.727933+0000","last_fullsized":"2024-05-09T06:36:36.727933+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:22.137711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884438+0000","last_change":"2024-05-09T06:36:34.720781+0000","last_active":"2024-05-09T06:36:36.884438+0000","last_peered":"2024-05-09T06:36:36.884438+0000","last_clean":"2024-05-09T06:36:36.884438+0000","last_became_active":"2024-05-09T06:36:34.720513+0000","last_became_peered":"2024-05-09T06:36:34.720513+0000","last_unstale":"2024-05-09T06:36:36.884438+0000","last_undegraded":"2024-05-09T06:36:36.884438+0000","last_fullsized":"2024-05-09T06:36:36.884438+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362005+0000","last_change":"2024-05-09T06:36:35.733052+0000","last_active":"2024-05-09T06:36:37.362005+0000","last_peered":"2024-05-09T06:36:37.362005+0000","last_clean":"2024-05-09T06:36:37.362005+0000","last_became_active":"2024-05-09T06:36:35.732795+0000","last_became_peered":"2024-05-09T06:36:35.732795+0000","last_unstale":"2024-05-09T06:36:37.362005+0000","last_undegraded":"2024-05-09T06:36:37.362005+0000","last_fullsized":"2024-05-09T06:36:37.362005+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:18:06.228612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362137+0000","last_change":"2024-05-09T06:36:34.725360+0000","last_active":"2024-05-09T06:36:37.362137+0000","last_peered":"2024-05-09T06:36:37.362137+0000","last_clean":"2024-05-09T06:36:37.362137+0000","last_became_active":"2024-05-09T06:36:34.725063+0000","last_became_peered":"2024-05-09T06:36:34.725063+0000","last_unstale":"2024-05-09T06:36:37.362137+0000","last_undegraded":"2024-05-09T06:36:37.362137+0000","last_fullsized":"2024-05-09T06:36:37.362137+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363057+0000","last_change":"2024-05-09T06:36:35.736266+0000","last_active":"2024-05-09T06:36:37.363057+0000","last_peered":"2024-05-09T06:36:37.363057+0000","last_clean":"2024-05-09T06:36:37.363057+0000","last_became_active":"2024-05-09T06:36:35.735960+0000","last_became_peered":"2024-05-09T06:36:35.735960+0000","last_unstale":"2024-05-09T06:36:37.363057+0000","last_undegraded":"2024-05-09T06:36:37.363057+0000","last_fullsized":"2024-05-09T06:36:37.363057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:40.906476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732788+0000","last_change":"2024-05-09T06:36:34.722763+0000","last_active":"2024-05-09T06:36:36.732788+0000","last_peered":"2024-05-09T06:36:36.732788+0000","last_clean":"2024-05-09T06:36:36.732788+0000","last_became_active":"2024-05-09T06:36:34.722478+0000","last_became_peered":"2024-05-09T06:36:34.722478+0000","last_unstale":"2024-05-09T06:36:36.732788+0000","last_undegraded":"2024-05-09T06:36:36.732788+0000","last_fullsized":"2024-05-09T06:36:36.732788+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728229+0000","last_change":"2024-05-09T06:36:35.732081+0000","last_active":"2024-05-09T06:36:36.728229+0000","last_peered":"2024-05-09T06:36:36.728229+0000","last_clean":"2024-05-09T06:36:36.728229+0000","last_became_active":"2024-05-09T06:36:35.731786+0000","last_became_peered":"2024-05-09T06:36:35.731786+0000","last_unstale":"2024-05-09T06:36:36.728229+0000","last_undegraded":"2024-05-09T06:36:36.728229+0000","last_fullsized":"2024-05-09T06:36:36.728229+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:03.123560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727630+0000","last_change":"2024-05-09T06:36:34.725217+0000","last_active":"2024-05-09T06:36:36.727630+0000","last_peered":"2024-05-09T06:36:36.727630+0000","last_clean":"2024-05-09T06:36:36.727630+0000","last_became_active":"2024-05-09T06:36:34.724807+0000","last_became_peered":"2024-05-09T06:36:34.724807+0000","last_unstale":"2024-05-09T06:36:36.727630+0000","last_undegraded":"2024-05-09T06:36:36.727630+0000","last_fullsized":"2024-05-09T06:36:36.727630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:42.511981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361517+0000","last_change":"2024-05-09T06:36:35.731453+0000","last_active":"2024-05-09T06:36:37.361517+0000","last_peered":"2024-05-09T06:36:37.361517+0000","last_clean":"2024-05-09T06:36:37.361517+0000","last_became_active":"2024-05-09T06:36:35.731153+0000","last_became_peered":"2024-05-09T06:36:35.731153+0000","last_unstale":"2024-05-09T06:36:37.361517+0000","last_undegraded":"2024-05-09T06:36:37.361517+0000","last_fullsized":"2024-05-09T06:36:37.361517+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:33.854570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733483+0000","last_change":"2024-05-09T06:36:34.726176+0000","last_active":"2024-05-09T06:36:36.733483+0000","last_peered":"2024-05-09T06:36:36.733483+0000","last_clean":"2024-05-09T06:36:36.733483+0000","last_became_active":"2024-05-09T06:36:34.725866+0000","last_became_peered":"2024-05-09T06:36:34.725866+0000","last_unstale":"2024-05-09T06:36:36.733483+0000","last_undegraded":"2024-05-09T06:36:36.733483+0000","last_fullsized":"2024-05-09T06:36:36.733483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171904+0000","last_change":"2024-05-09T06:36:35.744931+0000","last_active":"2024-05-09T06:36:37.171904+0000","last_peered":"2024-05-09T06:36:37.171904+0000","last_clean":"2024-05-09T06:36:37.171904+0000","last_became_active":"2024-05-09T06:36:35.744642+0000","last_became_peered":"2024-05-09T06:36:35.744642+0000","last_unstale":"2024-05-09T06:36:37.171904+0000","last_undegraded":"2024-05-09T06:36:37.171904+0000","last_fullsized":"2024-05-09T06:36:37.171904+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361849+0000","last_change":"2024-05-09T06:36:34.713554+0000","last_active":"2024-05-09T06:36:37.361849+0000","last_peered":"2024-05-09T06:36:37.361849+0000","last_clean":"2024-05-09T06:36:37.361849+0000","last_became_active":"2024-05-09T06:36:34.713251+0000","last_became_peered":"2024-05-09T06:36:34.713251+0000","last_unstale":"2024-05-09T06:36:37.361849+0000","last_undegraded":"2024-05-09T06:36:37.361849+0000","last_fullsized":"2024-05-09T06:36:37.361849+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:38.536452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860958+0000","last_change":"2024-05-09T06:36:35.736939+0000","last_active":"2024-05-09T06:36:36.860958+0000","last_peered":"2024-05-09T06:36:36.860958+0000","last_clean":"2024-05-09T06:36:36.860958+0000","last_became_active":"2024-05-09T06:36:35.736669+0000","last_became_peered":"2024-05-09T06:36:35.736669+0000","last_unstale":"2024-05-09T06:36:36.860958+0000","last_undegraded":"2024-05-09T06:36:36.860958+0000","last_fullsized":"2024-05-09T06:36:36.860958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:23.631610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884507+0000","last_change":"2024-05-09T06:36:34.723605+0000","last_active":"2024-05-09T06:36:36.884507+0000","last_peered":"2024-05-09T06:36:36.884507+0000","last_clean":"2024-05-09T06:36:36.884507+0000","last_became_active":"2024-05-09T06:36:34.723298+0000","last_became_peered":"2024-05-09T06:36:34.723298+0000","last_unstale":"2024-05-09T06:36:36.884507+0000","last_undegraded":"2024-05-09T06:36:36.884507+0000","last_fullsized":"2024-05-09T06:36:36.884507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171805+0000","last_change":"2024-05-09T06:36:35.746081+0000","last_active":"2024-05-09T06:36:37.171805+0000","last_peered":"2024-05-09T06:36:37.171805+0000","last_clean":"2024-05-09T06:36:37.171805+0000","last_became_active":"2024-05-09T06:36:35.745744+0000","last_became_peered":"2024-05-09T06:36:35.745744+0000","last_unstale":"2024-05-09T06:36:37.171805+0000","last_undegraded":"2024-05-09T06:36:37.171805+0000","last_fullsized":"2024-05-09T06:36:37.171805+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:07.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362413+0000","last_change":"2024-05-09T06:36:34.720057+0000","last_active":"2024-05-09T06:36:37.362413+0000","last_peered":"2024-05-09T06:36:37.362413+0000","last_clean":"2024-05-09T06:36:37.362413+0000","last_became_active":"2024-05-09T06:36:34.719725+0000","last_became_peered":"2024-05-09T06:36:34.719725+0000","last_unstale":"2024-05-09T06:36:37.362413+0000","last_undegraded":"2024-05-09T06:36:37.362413+0000","last_fullsized":"2024-05-09T06:36:37.362413+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362337+0000","last_change":"2024-05-09T06:36:35.734423+0000","last_active":"2024-05-09T06:36:37.362337+0000","last_peered":"2024-05-09T06:36:37.362337+0000","last_clean":"2024-05-09T06:36:37.362337+0000","last_became_active":"2024-05-09T06:36:35.733912+0000","last_became_peered":"2024-05-09T06:36:35.733912+0000","last_unstale":"2024-05-09T06:36:37.362337+0000","last_undegraded":"2024-05-09T06:36:37.362337+0000","last_fullsized":"2024-05-09T06:36:37.362337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171840+0000","last_change":"2024-05-09T06:36:34.731400+0000","last_active":"2024-05-09T06:36:37.171840+0000","last_peered":"2024-05-09T06:36:37.171840+0000","last_clean":"2024-05-09T06:36:37.171840+0000","last_became_active":"2024-05-09T06:36:34.731097+0000","last_became_peered":"2024-05-09T06:36:34.731097+0000","last_unstale":"2024-05-09T06:36:37.171840+0000","last_undegraded":"2024-05-09T06:36:37.171840+0000","last_fullsized":"2024-05-09T06:36:37.171840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362862+0000","last_change":"2024-05-09T06:36:35.731381+0000","last_active":"2024-05-09T06:36:37.362862+0000","last_peered":"2024-05-09T06:36:37.362862+0000","last_clean":"2024-05-09T06:36:37.362862+0000","last_became_active":"2024-05-09T06:36:35.731083+0000","last_became_peered":"2024-05-09T06:36:35.731083+0000","last_unstale":"2024-05-09T06:36:37.362862+0000","last_undegraded":"2024-05-09T06:36:37.362862+0000","last_fullsized":"2024-05-09T06:36:37.362862+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:38.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727873+0000","last_change":"2024-05-09T06:36:34.722286+0000","last_active":"2024-05-09T06:36:36.727873+0000","last_peered":"2024-05-09T06:36:36.727873+0000","last_clean":"2024-05-09T06:36:36.727873+0000","last_became_active":"2024-05-09T06:36:34.721994+0000","last_became_peered":"2024-05-09T06:36:34.721994+0000","last_unstale":"2024-05-09T06:36:36.727873+0000","last_undegraded":"2024-05-09T06:36:36.727873+0000","last_fullsized":"2024-05-09T06:36:36.727873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:42.279096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860860+0000","last_change":"2024-05-09T06:36:35.741059+0000","last_active":"2024-05-09T06:36:36.860860+0000","last_peered":"2024-05-09T06:36:36.860860+0000","last_clean":"2024-05-09T06:36:36.860860+0000","last_became_active":"2024-05-09T06:36:35.739832+0000","last_became_peered":"2024-05-09T06:36:35.739832+0000","last_unstale":"2024-05-09T06:36:36.860860+0000","last_undegraded":"2024-05-09T06:36:36.860860+0000","last_fullsized":"2024-05-09T06:36:36.860860+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:13.676589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860765+0000","last_change":"2024-05-09T06:36:34.717389+0000","last_active":"2024-05-09T06:36:36.860765+0000","last_peered":"2024-05-09T06:36:36.860765+0000","last_clean":"2024-05-09T06:36:36.860765+0000","last_became_active":"2024-05-09T06:36:34.717129+0000","last_became_peered":"2024-05-09T06:36:34.717129+0000","last_unstale":"2024-05-09T06:36:36.860765+0000","last_undegraded":"2024-05-09T06:36:36.860765+0000","last_fullsized":"2024-05-09T06:36:36.860765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:00.252278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362192+0000","last_change":"2024-05-09T06:36:35.729816+0000","last_active":"2024-05-09T06:36:37.362192+0000","last_peered":"2024-05-09T06:36:37.362192+0000","last_clean":"2024-05-09T06:36:37.362192+0000","last_became_active":"2024-05-09T06:36:35.729478+0000","last_became_peered":"2024-05-09T06:36:35.729478+0000","last_unstale":"2024-05-09T06:36:37.362192+0000","last_undegraded":"2024-05-09T06:36:37.362192+0000","last_fullsized":"2024-05-09T06:36:37.362192+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:14.738457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732784+0000","last_change":"2024-05-09T06:36:34.722070+0000","last_active":"2024-05-09T06:36:36.732784+0000","last_peered":"2024-05-09T06:36:36.732784+0000","last_clean":"2024-05-09T06:36:36.732784+0000","last_became_active":"2024-05-09T06:36:34.721798+0000","last_became_peered":"2024-05-09T06:36:34.721798+0000","last_unstale":"2024-05-09T06:36:36.732784+0000","last_undegraded":"2024-05-09T06:36:36.732784+0000","last_fullsized":"2024-05-09T06:36:36.732784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:14.243131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860899+0000","last_change":"2024-05-09T06:36:35.732608+0000","last_active":"2024-05-09T06:36:36.860899+0000","last_peered":"2024-05-09T06:36:36.860899+0000","last_clean":"2024-05-09T06:36:36.860899+0000","last_became_active":"2024-05-09T06:36:35.732303+0000","last_became_peered":"2024-05-09T06:36:35.732303+0000","last_unstale":"2024-05-09T06:36:36.860899+0000","last_undegraded":"2024-05-09T06:36:36.860899+0000","last_fullsized":"2024-05-09T06:36:36.860899+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:20.080800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362751+0000","last_change":"2024-05-09T06:36:34.720971+0000","last_active":"2024-05-09T06:36:37.362751+0000","last_peered":"2024-05-09T06:36:37.362751+0000","last_clean":"2024-05-09T06:36:37.362751+0000","last_became_active":"2024-05-09T06:36:34.719853+0000","last_became_peered":"2024-05-09T06:36:34.719853+0000","last_unstale":"2024-05-09T06:36:37.362751+0000","last_undegraded":"2024-05-09T06:36:37.362751+0000","last_fullsized":"2024-05-09T06:36:37.362751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362173+0000","last_change":"2024-05-09T06:36:35.731420+0000","last_active":"2024-05-09T06:36:37.362173+0000","last_peered":"2024-05-09T06:36:37.362173+0000","last_clean":"2024-05-09T06:36:37.362173+0000","last_became_active":"2024-05-09T06:36:35.731137+0000","last_became_peered":"2024-05-09T06:36:35.731137+0000","last_unstale":"2024-05-09T06:36:37.362173+0000","last_undegraded":"2024-05-09T06:36:37.362173+0000","last_fullsized":"2024-05-09T06:36:37.362173+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:07.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884572+0000","last_change":"2024-05-09T06:36:34.725998+0000","last_active":"2024-05-09T06:36:36.884572+0000","last_peered":"2024-05-09T06:36:36.884572+0000","last_clean":"2024-05-09T06:36:36.884572+0000","last_became_active":"2024-05-09T06:36:34.725716+0000","last_became_peered":"2024-05-09T06:36:34.725716+0000","last_unstale":"2024-05-09T06:36:36.884572+0000","last_undegraded":"2024-05-09T06:36:36.884572+0000","last_fullsized":"2024-05-09T06:36:36.884572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171699+0000","last_change":"2024-05-09T06:36:35.733132+0000","last_active":"2024-05-09T06:36:37.171699+0000","last_peered":"2024-05-09T06:36:37.171699+0000","last_clean":"2024-05-09T06:36:37.171699+0000","last_became_active":"2024-05-09T06:36:35.732800+0000","last_became_peered":"2024-05-09T06:36:35.732800+0000","last_unstale":"2024-05-09T06:36:37.171699+0000","last_undegraded":"2024-05-09T06:36:37.171699+0000","last_fullsized":"2024-05-09T06:36:37.171699+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362998+0000","last_change":"2024-05-09T06:36:34.725109+0000","last_active":"2024-05-09T06:36:37.362998+0000","last_peered":"2024-05-09T06:36:37.362998+0000","last_clean":"2024-05-09T06:36:37.362998+0000","last_became_active":"2024-05-09T06:36:34.724684+0000","last_became_peered":"2024-05-09T06:36:34.724684+0000","last_unstale":"2024-05-09T06:36:37.362998+0000","last_undegraded":"2024-05-09T06:36:37.362998+0000","last_fullsized":"2024-05-09T06:36:37.362998+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361762+0000","last_change":"2024-05-09T06:36:35.723304+0000","last_active":"2024-05-09T06:36:37.361762+0000","last_peered":"2024-05-09T06:36:37.361762+0000","last_clean":"2024-05-09T06:36:37.361762+0000","last_became_active":"2024-05-09T06:36:35.723007+0000","last_became_peered":"2024-05-09T06:36:35.723007+0000","last_unstale":"2024-05-09T06:36:37.361762+0000","last_undegraded":"2024-05-09T06:36:37.361762+0000","last_fullsized":"2024-05-09T06:36:37.361762+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:25.757227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861312+0000","last_change":"2024-05-09T06:36:34.729608+0000","last_active":"2024-05-09T06:36:36.861312+0000","last_peered":"2024-05-09T06:36:36.861312+0000","last_clean":"2024-05-09T06:36:36.861312+0000","last_became_active":"2024-05-09T06:36:34.729292+0000","last_became_peered":"2024-05-09T06:36:34.729292+0000","last_unstale":"2024-05-09T06:36:36.861312+0000","last_undegraded":"2024-05-09T06:36:36.861312+0000","last_fullsized":"2024-05-09T06:36:36.861312+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:54.540738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361952+0000","last_change":"2024-05-09T06:36:35.719670+0000","last_active":"2024-05-09T06:36:37.361952+0000","last_peered":"2024-05-09T06:36:37.361952+0000","last_clean":"2024-05-09T06:36:37.361952+0000","last_became_active":"2024-05-09T06:36:35.719375+0000","last_became_peered":"2024-05-09T06:36:35.719375+0000","last_unstale":"2024-05-09T06:36:37.361952+0000","last_undegraded":"2024-05-09T06:36:37.361952+0000","last_fullsized":"2024-05-09T06:36:37.361952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:33.896862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860695+0000","last_change":"2024-05-09T06:36:34.725264+0000","last_active":"2024-05-09T06:36:36.860695+0000","last_peered":"2024-05-09T06:36:36.860695+0000","last_clean":"2024-05-09T06:36:36.860695+0000","last_became_active":"2024-05-09T06:36:34.724991+0000","last_became_peered":"2024-05-09T06:36:34.724991+0000","last_unstale":"2024-05-09T06:36:36.860695+0000","last_undegraded":"2024-05-09T06:36:36.860695+0000","last_fullsized":"2024-05-09T06:36:36.860695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:57.900999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728143+0000","last_change":"2024-05-09T06:36:35.738237+0000","last_active":"2024-05-09T06:36:36.728143+0000","last_peered":"2024-05-09T06:36:36.728143+0000","last_clean":"2024-05-09T06:36:36.728143+0000","last_became_active":"2024-05-09T06:36:35.737494+0000","last_became_peered":"2024-05-09T06:36:35.737494+0000","last_unstale":"2024-05-09T06:36:36.728143+0000","last_undegraded":"2024-05-09T06:36:36.728143+0000","last_fullsized":"2024-05-09T06:36:36.728143+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:43.210498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361934+0000","last_change":"2024-05-09T06:36:34.725808+0000","last_active":"2024-05-09T06:36:37.361934+0000","last_peered":"2024-05-09T06:36:37.361934+0000","last_clean":"2024-05-09T06:36:37.361934+0000","last_became_active":"2024-05-09T06:36:34.725524+0000","last_became_peered":"2024-05-09T06:36:34.725524+0000","last_unstale":"2024-05-09T06:36:37.361934+0000","last_undegraded":"2024-05-09T06:36:37.361934+0000","last_fullsized":"2024-05-09T06:36:37.361934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:12.993813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861054+0000","last_change":"2024-05-09T06:36:35.736077+0000","last_active":"2024-05-09T06:36:36.861054+0000","last_peered":"2024-05-09T06:36:36.861054+0000","last_clean":"2024-05-09T06:36:36.861054+0000","last_became_active":"2024-05-09T06:36:35.735802+0000","last_became_peered":"2024-05-09T06:36:35.735802+0000","last_unstale":"2024-05-09T06:36:36.861054+0000","last_undegraded":"2024-05-09T06:36:36.861054+0000","last_fullsized":"2024-05-09T06:36:36.861054+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:17.420596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860867+0000","last_change":"2024-05-09T06:36:34.724792+0000","last_active":"2024-05-09T06:36:36.860867+0000","last_peered":"2024-05-09T06:36:36.860867+0000","last_clean":"2024-05-09T06:36:36.860867+0000","last_became_active":"2024-05-09T06:36:34.724482+0000","last_became_peered":"2024-05-09T06:36:34.724482+0000","last_unstale":"2024-05-09T06:36:36.860867+0000","last_undegraded":"2024-05-09T06:36:36.860867+0000","last_fullsized":"2024-05-09T06:36:36.860867+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:37.551668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361503+0000","last_change":"2024-05-09T06:36:35.732071+0000","last_active":"2024-05-09T06:36:37.361503+0000","last_peered":"2024-05-09T06:36:37.361503+0000","last_clean":"2024-05-09T06:36:37.361503+0000","last_became_active":"2024-05-09T06:36:35.731781+0000","last_became_peered":"2024-05-09T06:36:35.731781+0000","last_unstale":"2024-05-09T06:36:37.361503+0000","last_undegraded":"2024-05-09T06:36:37.361503+0000","last_fullsized":"2024-05-09T06:36:37.361503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:47.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171964+0000","last_change":"2024-05-09T06:36:34.729856+0000","last_active":"2024-05-09T06:36:37.171964+0000","last_peered":"2024-05-09T06:36:37.171964+0000","last_clean":"2024-05-09T06:36:37.171964+0000","last_became_active":"2024-05-09T06:36:34.729560+0000","last_became_peered":"2024-05-09T06:36:34.729560+0000","last_unstale":"2024-05-09T06:36:37.171964+0000","last_undegraded":"2024-05-09T06:36:37.171964+0000","last_fullsized":"2024-05-09T06:36:37.171964+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733047+0000","last_change":"2024-05-09T06:36:35.730375+0000","last_active":"2024-05-09T06:36:36.733047+0000","last_peered":"2024-05-09T06:36:36.733047+0000","last_clean":"2024-05-09T06:36:36.733047+0000","last_became_active":"2024-05-09T06:36:35.730096+0000","last_became_peered":"2024-05-09T06:36:35.730096+0000","last_unstale":"2024-05-09T06:36:36.733047+0000","last_undegraded":"2024-05-09T06:36:36.733047+0000","last_fullsized":"2024-05-09T06:36:36.733047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361893+0000","last_change":"2024-05-09T06:36:34.725892+0000","last_active":"2024-05-09T06:36:37.361893+0000","last_peered":"2024-05-09T06:36:37.361893+0000","last_clean":"2024-05-09T06:36:37.361893+0000","last_became_active":"2024-05-09T06:36:34.725587+0000","last_became_peered":"2024-05-09T06:36:34.725587+0000","last_unstale":"2024-05-09T06:36:37.361893+0000","last_undegraded":"2024-05-09T06:36:37.361893+0000","last_fullsized":"2024-05-09T06:36:37.361893+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:21.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727794+0000","last_change":"2024-05-09T06:36:35.742341+0000","last_active":"2024-05-09T06:36:36.727794+0000","last_peered":"2024-05-09T06:36:36.727794+0000","last_clean":"2024-05-09T06:36:36.727794+0000","last_became_active":"2024-05-09T06:36:35.742047+0000","last_became_peered":"2024-05-09T06:36:35.742047+0000","last_unstale":"2024-05-09T06:36:36.727794+0000","last_undegraded":"2024-05-09T06:36:36.727794+0000","last_fullsized":"2024-05-09T06:36:36.727794+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:42.840248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362689+0000","last_change":"2024-05-09T06:36:34.725138+0000","last_active":"2024-05-09T06:36:37.362689+0000","last_peered":"2024-05-09T06:36:37.362689+0000","last_clean":"2024-05-09T06:36:37.362689+0000","last_became_active":"2024-05-09T06:36:34.724821+0000","last_became_peered":"2024-05-09T06:36:34.724821+0000","last_unstale":"2024-05-09T06:36:37.362689+0000","last_undegraded":"2024-05-09T06:36:37.362689+0000","last_fullsized":"2024-05-09T06:36:37.362689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362291+0000","last_change":"2024-05-09T06:36:35.733528+0000","last_active":"2024-05-09T06:36:37.362291+0000","last_peered":"2024-05-09T06:36:37.362291+0000","last_clean":"2024-05-09T06:36:37.362291+0000","last_became_active":"2024-05-09T06:36:35.733210+0000","last_became_peered":"2024-05-09T06:36:35.733210+0000","last_unstale":"2024-05-09T06:36:37.362291+0000","last_undegraded":"2024-05-09T06:36:37.362291+0000","last_fullsized":"2024-05-09T06:36:37.362291+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171902+0000","last_change":"2024-05-09T06:36:34.728780+0000","last_active":"2024-05-09T06:36:37.171902+0000","last_peered":"2024-05-09T06:36:37.171902+0000","last_clean":"2024-05-09T06:36:37.171902+0000","last_became_active":"2024-05-09T06:36:34.728489+0000","last_became_peered":"2024-05-09T06:36:34.728489+0000","last_unstale":"2024-05-09T06:36:37.171902+0000","last_undegraded":"2024-05-09T06:36:37.171902+0000","last_fullsized":"2024-05-09T06:36:37.171902+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860852+0000","last_change":"2024-05-09T06:36:35.739237+0000","last_active":"2024-05-09T06:36:36.860852+0000","last_peered":"2024-05-09T06:36:36.860852+0000","last_clean":"2024-05-09T06:36:36.860852+0000","last_became_active":"2024-05-09T06:36:35.738912+0000","last_became_peered":"2024-05-09T06:36:35.738912+0000","last_unstale":"2024-05-09T06:36:36.860852+0000","last_undegraded":"2024-05-09T06:36:36.860852+0000","last_fullsized":"2024-05-09T06:36:36.860852+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:00:43.136465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733190+0000","last_change":"2024-05-09T06:36:34.720898+0000","last_active":"2024-05-09T06:36:36.733190+0000","last_peered":"2024-05-09T06:36:36.733190+0000","last_clean":"2024-05-09T06:36:36.733190+0000","last_became_active":"2024-05-09T06:36:34.720626+0000","last_became_peered":"2024-05-09T06:36:34.720626+0000","last_unstale":"2024-05-09T06:36:36.733190+0000","last_undegraded":"2024-05-09T06:36:36.733190+0000","last_fullsized":"2024-05-09T06:36:36.733190+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860911+0000","last_change":"2024-05-09T06:36:35.739319+0000","last_active":"2024-05-09T06:36:36.860911+0000","last_peered":"2024-05-09T06:36:36.860911+0000","last_clean":"2024-05-09T06:36:36.860911+0000","last_became_active":"2024-05-09T06:36:35.738697+0000","last_became_peered":"2024-05-09T06:36:35.738697+0000","last_unstale":"2024-05-09T06:36:36.860911+0000","last_undegraded":"2024-05-09T06:36:36.860911+0000","last_fullsized":"2024-05-09T06:36:36.860911+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:14.451235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860934+0000","last_change":"2024-05-09T06:36:34.717861+0000","last_active":"2024-05-09T06:36:36.860934+0000","last_peered":"2024-05-09T06:36:36.860934+0000","last_clean":"2024-05-09T06:36:36.860934+0000","last_became_active":"2024-05-09T06:36:34.717576+0000","last_became_peered":"2024-05-09T06:36:34.717576+0000","last_unstale":"2024-05-09T06:36:36.860934+0000","last_undegraded":"2024-05-09T06:36:36.860934+0000","last_fullsized":"2024-05-09T06:36:36.860934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:25.427088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884682+0000","last_change":"2024-05-09T06:36:35.732696+0000","last_active":"2024-05-09T06:36:36.884682+0000","last_peered":"2024-05-09T06:36:36.884682+0000","last_clean":"2024-05-09T06:36:36.884682+0000","last_became_active":"2024-05-09T06:36:35.732342+0000","last_became_peered":"2024-05-09T06:36:35.732342+0000","last_unstale":"2024-05-09T06:36:36.884682+0000","last_undegraded":"2024-05-09T06:36:36.884682+0000","last_fullsized":"2024-05-09T06:36:36.884682+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362069+0000","last_change":"2024-05-09T06:36:34.724569+0000","last_active":"2024-05-09T06:36:37.362069+0000","last_peered":"2024-05-09T06:36:37.362069+0000","last_clean":"2024-05-09T06:36:37.362069+0000","last_became_active":"2024-05-09T06:36:34.724223+0000","last_became_peered":"2024-05-09T06:36:34.724223+0000","last_unstale":"2024-05-09T06:36:37.362069+0000","last_undegraded":"2024-05-09T06:36:37.362069+0000","last_fullsized":"2024-05-09T06:36:37.362069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363148+0000","last_change":"2024-05-09T06:36:35.728379+0000","last_active":"2024-05-09T06:36:37.363148+0000","last_peered":"2024-05-09T06:36:37.363148+0000","last_clean":"2024-05-09T06:36:37.363148+0000","last_became_active":"2024-05-09T06:36:35.728035+0000","last_became_peered":"2024-05-09T06:36:35.728035+0000","last_unstale":"2024-05-09T06:36:37.363148+0000","last_undegraded":"2024-05-09T06:36:37.363148+0000","last_fullsized":"2024-05-09T06:36:37.363148+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860972+0000","last_change":"2024-05-09T06:36:34.719103+0000","last_active":"2024-05-09T06:36:36.860972+0000","last_peered":"2024-05-09T06:36:36.860972+0000","last_clean":"2024-05-09T06:36:36.860972+0000","last_became_active":"2024-05-09T06:36:34.718806+0000","last_became_peered":"2024-05-09T06:36:34.718806+0000","last_unstale":"2024-05-09T06:36:36.860972+0000","last_undegraded":"2024-05-09T06:36:36.860972+0000","last_fullsized":"2024-05-09T06:36:36.860972+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:14.284730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361889+0000","last_change":"2024-05-09T06:36:35.732848+0000","last_active":"2024-05-09T06:36:37.361889+0000","last_peered":"2024-05-09T06:36:37.361889+0000","last_clean":"2024-05-09T06:36:37.361889+0000","last_became_active":"2024-05-09T06:36:35.732542+0000","last_became_peered":"2024-05-09T06:36:35.732542+0000","last_unstale":"2024-05-09T06:36:37.361889+0000","last_undegraded":"2024-05-09T06:36:37.361889+0000","last_fullsized":"2024-05-09T06:36:37.361889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:20.117527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861044+0000","last_change":"2024-05-09T06:36:34.724039+0000","last_active":"2024-05-09T06:36:36.861044+0000","last_peered":"2024-05-09T06:36:36.861044+0000","last_clean":"2024-05-09T06:36:36.861044+0000","last_became_active":"2024-05-09T06:36:34.723772+0000","last_became_peered":"2024-05-09T06:36:34.723772+0000","last_unstale":"2024-05-09T06:36:36.861044+0000","last_undegraded":"2024-05-09T06:36:36.861044+0000","last_fullsized":"2024-05-09T06:36:36.861044+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:58.238882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":328,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884146+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:36:36.884146+0000","last_peered":"2024-05-09T06:36:36.884146+0000","last_clean":"2024-05-09T06:36:36.884146+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:36:36.884146+0000","last_undegraded":"2024-05-09T06:36:36.884146+0000","last_fullsized":"2024-05-09T06:36:36.884146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1203048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":614736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":612520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":612612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":613102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1204590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:39.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:39.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.886+0000 7f06ecba9640 1 Processor -- start 2024-05-09T06:36:39.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.886+0000 7f06ecba9640 1 -- start start 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06ecba9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06e815db10 0x7f06e80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06ecba9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06e80715d0 0x7f06e815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06ecba9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06e8070f40 con 0x7f06e815e500 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06ecba9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f06e815e950 con 0x7f06e80715d0 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06ecba9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f06e80cf170 con 0x7f06e815db10 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06e5d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06e80715d0 0x7f06e815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06e5d9b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f06e80715d0 0x7f06e815dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55400/0 (socket says 172.21.15.1:55400) 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06e5d9b640 1 -- 172.21.15.1:0/52697807 learned_addr learned my addr 172.21.15.1:0/52697807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06e6d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f06e815e500 0x7f06e806e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57820/0 (socket says 172.21.15.1:57820) 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.887+0000 7f06e659c640 1 --2- 172.21.15.1:0/52697807 >> v2:172.21.15.6:3300/0 conn(0x7f06e815db10 0x7f06e80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e5d9b640 1 -- 172.21.15.1:0/52697807 >> v2:172.21.15.6:3300/0 conn(0x7f06e815db10 msgr2=0x7f06e80b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e5d9b640 1 --2- 172.21.15.1:0/52697807 >> v2:172.21.15.6:3300/0 conn(0x7f06e815db10 0x7f06e80b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e5d9b640 1 -- 172.21.15.1:0/52697807 >> v1:172.21.15.1:6789/0 conn(0x7f06e815e500 legacy=0x7f06e806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e5d9b640 1 -- 172.21.15.1:0/52697807 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e80cf450 con 0x7f06e80715d0 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e559a640 1 -- 172.21.15.1:0/52697807 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826004052 0 0) 0x7f06e8070f40 con 0x7f06e815e500 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e5d9b640 1 --2- 172.21.15.1:0/52697807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e80715d0 0x7f06e815dfc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f06e80744b0 tx=0x7f06d002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637a1d8a3e6bd62c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.888+0000 7f06e559a640 1 -- 172.21.15.1:0/52697807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0036020 con 0x7f06e80715d0 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06e559a640 1 -- 172.21.15.1:0/52697807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d0032640 con 0x7f06e80715d0 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06e559a640 1 -- 172.21.15.1:0/52697807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0032a10 con 0x7f06e80715d0 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 -- 172.21.15.1:0/52697807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e80715d0 msgr2=0x7f06e815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 --2- 172.21.15.1:0/52697807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e80715d0 0x7f06e815dfc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f06e80744b0 tx=0x7f06d002c940 comp rx=0 tx=0).stop 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 -- 172.21.15.1:0/52697807 shutdown_connections 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 --2- 172.21.15.1:0/52697807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e80715d0 0x7f06e815dfc0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 --2- 172.21.15.1:0/52697807 >> v2:172.21.15.6:3300/0 conn(0x7f06e815db10 0x7f06e80b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 -- 172.21.15.1:0/52697807 >> 172.21.15.1:0/52697807 conn(0x7f06e80cc4b0 msgr2=0x7f06e80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 -- 172.21.15.1:0/52697807 shutdown_connections 2024-05-09T06:36:39.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.889+0000 7f06ecba9640 1 -- 172.21.15.1:0/52697807 wait complete. 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 Processor -- start 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 -- start start 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06e80715d0 0x7f06e81f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06e815db10 0x7f06e80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8071bd0 con 0x7f06e815e500 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8071a50 con 0x7f06e80715d0 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06ecba9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f06e815ec40 con 0x7f06e815db10 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55416/0 (socket says 172.21.15.1:55416) 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.891+0000 7f06e6d9d640 1 -- 172.21.15.1:0/4039746290 learned_addr learned my addr 172.21.15.1:0/4039746290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e659c640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06e80715d0 0x7f06e81f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e5d9b640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06e815db10 0x7f06e80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 -- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06e815db10 msgr2=0x7f06e80b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06e815db10 0x7f06e80b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 -- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06e80715d0 msgr2=0x7f06e81f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06e80715d0 0x7f06e81f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 -- 172.21.15.1:0/4039746290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e80b16e0 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06e6d9d640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c410 tx=0x7f06d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80074e0 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d80047e0 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e80b19c0 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8004bd0 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.892+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e80b1f80 con 0x7f06e815e500 2024-05-09T06:36:39.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.893+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e80752b0 con 0x7f06e815e500 2024-05-09T06:36:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.894+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06d8011070 con 0x7f06e815e500 2024-05-09T06:36:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.894+0000 7f06deffd640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 0x7f06c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.894+0000 7f06e659c640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 0x7f06c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.894+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f06d808ab30 con 0x7f06e815e500 2024-05-09T06:36:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.895+0000 7f06e659c640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 0x7f06c806e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f06e80b0020 tx=0x7f06e00098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:39.899+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f06d8059350 con 0x7f06e815e500 2024-05-09T06:36:40.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.080+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06e80b4fa0 con 0x7f06c806be30 2024-05-09T06:36:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.086+0000 7f06deffd640 1 -- 172.21.15.1:0/4039746290 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f06e80b4fa0 con 0x7f06c806be30 2024-05-09T06:36:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:40.071 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:36:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 msgr2=0x7f06c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 0x7f06c806e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f06e80b0020 tx=0x7f06e00098d0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 msgr2=0x7f06e8158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c410 tx=0x7f06d800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 shutdown_connections 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f06c806be30 0x7f06c806e2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06e815e500 0x7f06e8158ac0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f06e815db10 0x7f06e80b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 --2- 172.21.15.1:0/4039746290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06e80715d0 0x7f06e81f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 >> 172.21.15.1:0/4039746290 conn(0x7f06e80cc4b0 msgr2=0x7f06e80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 shutdown_connections 2024-05-09T06:36:40.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.089+0000 7f06ecba9640 1 -- 172.21.15.1:0/4039746290 wait complete. 2024-05-09T06:36:40.081 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-09T06:36:39.194729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260972,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713748,"statfs":{"total":773094113280,"available":772826877952,"internally_reserved":0,"allocated":11771904,"data_stored":6085716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.493671"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362011+0000","last_change":"2024-05-09T06:36:35.724735+0000","last_active":"2024-05-09T06:36:37.362011+0000","last_peered":"2024-05-09T06:36:37.362011+0000","last_clean":"2024-05-09T06:36:37.362011+0000","last_became_active":"2024-05-09T06:36:35.723937+0000","last_became_peered":"2024-05-09T06:36:35.723937+0000","last_unstale":"2024-05-09T06:36:37.362011+0000","last_undegraded":"2024-05-09T06:36:37.362011+0000","last_fullsized":"2024-05-09T06:36:37.362011+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:04.508786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362868+0000","last_change":"2024-05-09T06:36:35.727084+0000","last_active":"2024-05-09T06:36:37.362868+0000","last_peered":"2024-05-09T06:36:37.362868+0000","last_clean":"2024-05-09T06:36:37.362868+0000","last_became_active":"2024-05-09T06:36:35.726770+0000","last_became_peered":"2024-05-09T06:36:35.726770+0000","last_unstale":"2024-05-09T06:36:37.362868+0000","last_undegraded":"2024-05-09T06:36:37.362868+0000","last_fullsized":"2024-05-09T06:36:37.362868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:43.516265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361738+0000","last_change":"2024-05-09T06:36:35.733544+0000","last_active":"2024-05-09T06:36:37.361738+0000","last_peered":"2024-05-09T06:36:37.361738+0000","last_clean":"2024-05-09T06:36:37.361738+0000","last_became_active":"2024-05-09T06:36:35.732638+0000","last_became_peered":"2024-05-09T06:36:35.732638+0000","last_unstale":"2024-05-09T06:36:37.361738+0000","last_undegraded":"2024-05-09T06:36:37.361738+0000","last_fullsized":"2024-05-09T06:36:37.361738+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:53.648280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727802+0000","last_change":"2024-05-09T06:36:35.738005+0000","last_active":"2024-05-09T06:36:36.727802+0000","last_peered":"2024-05-09T06:36:36.727802+0000","last_clean":"2024-05-09T06:36:36.727802+0000","last_became_active":"2024-05-09T06:36:35.737695+0000","last_became_peered":"2024-05-09T06:36:35.737695+0000","last_unstale":"2024-05-09T06:36:36.727802+0000","last_undegraded":"2024-05-09T06:36:36.727802+0000","last_fullsized":"2024-05-09T06:36:36.727802+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:18.486814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171632+0000","last_change":"2024-05-09T06:36:35.726555+0000","last_active":"2024-05-09T06:36:37.171632+0000","last_peered":"2024-05-09T06:36:37.171632+0000","last_clean":"2024-05-09T06:36:37.171632+0000","last_became_active":"2024-05-09T06:36:35.726257+0000","last_became_peered":"2024-05-09T06:36:35.726257+0000","last_unstale":"2024-05-09T06:36:37.171632+0000","last_undegraded":"2024-05-09T06:36:37.171632+0000","last_fullsized":"2024-05-09T06:36:37.171632+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361921+0000","last_change":"2024-05-09T06:36:35.731952+0000","last_active":"2024-05-09T06:36:37.361921+0000","last_peered":"2024-05-09T06:36:37.361921+0000","last_clean":"2024-05-09T06:36:37.361921+0000","last_became_active":"2024-05-09T06:36:35.731626+0000","last_became_peered":"2024-05-09T06:36:35.731626+0000","last_unstale":"2024-05-09T06:36:37.361921+0000","last_undegraded":"2024-05-09T06:36:37.361921+0000","last_fullsized":"2024-05-09T06:36:37.361921+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361800+0000","last_change":"2024-05-09T06:36:35.737819+0000","last_active":"2024-05-09T06:36:37.361800+0000","last_peered":"2024-05-09T06:36:37.361800+0000","last_clean":"2024-05-09T06:36:37.361800+0000","last_became_active":"2024-05-09T06:36:35.737499+0000","last_became_peered":"2024-05-09T06:36:35.737499+0000","last_unstale":"2024-05-09T06:36:37.361800+0000","last_undegraded":"2024-05-09T06:36:37.361800+0000","last_fullsized":"2024-05-09T06:36:37.361800+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860713+0000","last_change":"2024-05-09T06:36:35.734691+0000","last_active":"2024-05-09T06:36:36.860713+0000","last_peered":"2024-05-09T06:36:36.860713+0000","last_clean":"2024-05-09T06:36:36.860713+0000","last_became_active":"2024-05-09T06:36:35.733909+0000","last_became_peered":"2024-05-09T06:36:35.733909+0000","last_unstale":"2024-05-09T06:36:36.860713+0000","last_undegraded":"2024-05-09T06:36:36.860713+0000","last_fullsized":"2024-05-09T06:36:36.860713+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:57.761138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362769+0000","last_change":"2024-05-09T06:36:35.725078+0000","last_active":"2024-05-09T06:36:37.362769+0000","last_peered":"2024-05-09T06:36:37.362769+0000","last_clean":"2024-05-09T06:36:37.362769+0000","last_became_active":"2024-05-09T06:36:35.724253+0000","last_became_peered":"2024-05-09T06:36:35.724253+0000","last_unstale":"2024-05-09T06:36:37.362769+0000","last_undegraded":"2024-05-09T06:36:37.362769+0000","last_fullsized":"2024-05-09T06:36:37.362769+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:43.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171910+0000","last_change":"2024-05-09T06:36:35.738687+0000","last_active":"2024-05-09T06:36:37.171910+0000","last_peered":"2024-05-09T06:36:37.171910+0000","last_clean":"2024-05-09T06:36:37.171910+0000","last_became_active":"2024-05-09T06:36:35.738088+0000","last_became_peered":"2024-05-09T06:36:35.738088+0000","last_unstale":"2024-05-09T06:36:37.171910+0000","last_undegraded":"2024-05-09T06:36:37.171910+0000","last_fullsized":"2024-05-09T06:36:37.171910+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:43.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362661+0000","last_change":"2024-05-09T06:36:35.738177+0000","last_active":"2024-05-09T06:36:37.362661+0000","last_peered":"2024-05-09T06:36:37.362661+0000","last_clean":"2024-05-09T06:36:37.362661+0000","last_became_active":"2024-05-09T06:36:35.737877+0000","last_became_peered":"2024-05-09T06:36:35.737877+0000","last_unstale":"2024-05-09T06:36:37.362661+0000","last_undegraded":"2024-05-09T06:36:37.362661+0000","last_fullsized":"2024-05-09T06:36:37.362661+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733442+0000","last_change":"2024-05-09T06:36:35.728824+0000","last_active":"2024-05-09T06:36:36.733442+0000","last_peered":"2024-05-09T06:36:36.733442+0000","last_clean":"2024-05-09T06:36:36.733442+0000","last_became_active":"2024-05-09T06:36:35.728460+0000","last_became_peered":"2024-05-09T06:36:35.728460+0000","last_unstale":"2024-05-09T06:36:36.733442+0000","last_undegraded":"2024-05-09T06:36:36.733442+0000","last_fullsized":"2024-05-09T06:36:36.733442+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727815+0000","last_change":"2024-05-09T06:36:35.730746+0000","last_active":"2024-05-09T06:36:36.727815+0000","last_peered":"2024-05-09T06:36:36.727815+0000","last_clean":"2024-05-09T06:36:36.727815+0000","last_became_active":"2024-05-09T06:36:35.730477+0000","last_became_peered":"2024-05-09T06:36:35.730477+0000","last_unstale":"2024-05-09T06:36:36.727815+0000","last_undegraded":"2024-05-09T06:36:36.727815+0000","last_fullsized":"2024-05-09T06:36:36.727815+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:30.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884342+0000","last_change":"2024-05-09T06:36:35.728860+0000","last_active":"2024-05-09T06:36:36.884342+0000","last_peered":"2024-05-09T06:36:36.884342+0000","last_clean":"2024-05-09T06:36:36.884342+0000","last_became_active":"2024-05-09T06:36:35.728568+0000","last_became_peered":"2024-05-09T06:36:35.728568+0000","last_unstale":"2024-05-09T06:36:36.884342+0000","last_undegraded":"2024-05-09T06:36:36.884342+0000","last_fullsized":"2024-05-09T06:36:36.884342+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884318+0000","last_change":"2024-05-09T06:36:35.732870+0000","last_active":"2024-05-09T06:36:36.884318+0000","last_peered":"2024-05-09T06:36:36.884318+0000","last_clean":"2024-05-09T06:36:36.884318+0000","last_became_active":"2024-05-09T06:36:35.732578+0000","last_became_peered":"2024-05-09T06:36:35.732578+0000","last_unstale":"2024-05-09T06:36:36.884318+0000","last_undegraded":"2024-05-09T06:36:36.884318+0000","last_fullsized":"2024-05-09T06:36:36.884318+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884277+0000","last_change":"2024-05-09T06:36:35.736396+0000","last_active":"2024-05-09T06:36:36.884277+0000","last_peered":"2024-05-09T06:36:36.884277+0000","last_clean":"2024-05-09T06:36:36.884277+0000","last_became_active":"2024-05-09T06:36:35.736094+0000","last_became_peered":"2024-05-09T06:36:35.736094+0000","last_unstale":"2024-05-09T06:36:36.884277+0000","last_undegraded":"2024-05-09T06:36:36.884277+0000","last_fullsized":"2024-05-09T06:36:36.884277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362031+0000","last_change":"2024-05-09T06:36:35.729017+0000","last_active":"2024-05-09T06:36:37.362031+0000","last_peered":"2024-05-09T06:36:37.362031+0000","last_clean":"2024-05-09T06:36:37.362031+0000","last_became_active":"2024-05-09T06:36:35.728741+0000","last_became_peered":"2024-05-09T06:36:35.728741+0000","last_unstale":"2024-05-09T06:36:37.362031+0000","last_undegraded":"2024-05-09T06:36:37.362031+0000","last_fullsized":"2024-05-09T06:36:37.362031+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:54:21.333941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362876+0000","last_change":"2024-05-09T06:36:35.730249+0000","last_active":"2024-05-09T06:36:37.362876+0000","last_peered":"2024-05-09T06:36:37.362876+0000","last_clean":"2024-05-09T06:36:37.362876+0000","last_became_active":"2024-05-09T06:36:35.729959+0000","last_became_peered":"2024-05-09T06:36:35.729959+0000","last_unstale":"2024-05-09T06:36:37.362876+0000","last_undegraded":"2024-05-09T06:36:37.362876+0000","last_fullsized":"2024-05-09T06:36:37.362876+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:57.577919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361864+0000","last_change":"2024-05-09T06:36:35.737416+0000","last_active":"2024-05-09T06:36:37.361864+0000","last_peered":"2024-05-09T06:36:37.361864+0000","last_clean":"2024-05-09T06:36:37.361864+0000","last_became_active":"2024-05-09T06:36:35.736971+0000","last_became_peered":"2024-05-09T06:36:35.736971+0000","last_unstale":"2024-05-09T06:36:37.361864+0000","last_undegraded":"2024-05-09T06:36:37.361864+0000","last_fullsized":"2024-05-09T06:36:37.361864+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:08.805142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733616+0000","last_change":"2024-05-09T06:36:35.728135+0000","last_active":"2024-05-09T06:36:36.733616+0000","last_peered":"2024-05-09T06:36:36.733616+0000","last_clean":"2024-05-09T06:36:36.733616+0000","last_became_active":"2024-05-09T06:36:35.727832+0000","last_became_peered":"2024-05-09T06:36:35.727832+0000","last_unstale":"2024-05-09T06:36:36.733616+0000","last_undegraded":"2024-05-09T06:36:36.733616+0000","last_fullsized":"2024-05-09T06:36:36.733616+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861218+0000","last_change":"2024-05-09T06:36:35.743735+0000","last_active":"2024-05-09T06:36:36.861218+0000","last_peered":"2024-05-09T06:36:36.861218+0000","last_clean":"2024-05-09T06:36:36.861218+0000","last_became_active":"2024-05-09T06:36:35.743443+0000","last_became_peered":"2024-05-09T06:36:35.743443+0000","last_unstale":"2024-05-09T06:36:36.861218+0000","last_undegraded":"2024-05-09T06:36:36.861218+0000","last_fullsized":"2024-05-09T06:36:36.861218+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:57.299598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171618+0000","last_change":"2024-05-09T06:36:35.735060+0000","last_active":"2024-05-09T06:36:37.171618+0000","last_peered":"2024-05-09T06:36:37.171618+0000","last_clean":"2024-05-09T06:36:37.171618+0000","last_became_active":"2024-05-09T06:36:35.734781+0000","last_became_peered":"2024-05-09T06:36:35.734781+0000","last_unstale":"2024-05-09T06:36:37.171618+0000","last_undegraded":"2024-05-09T06:36:37.171618+0000","last_fullsized":"2024-05-09T06:36:37.171618+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361639+0000","last_change":"2024-05-09T06:36:35.728446+0000","last_active":"2024-05-09T06:36:37.361639+0000","last_peered":"2024-05-09T06:36:37.361639+0000","last_clean":"2024-05-09T06:36:37.361639+0000","last_became_active":"2024-05-09T06:36:35.728041+0000","last_became_peered":"2024-05-09T06:36:35.728041+0000","last_unstale":"2024-05-09T06:36:37.361639+0000","last_undegraded":"2024-05-09T06:36:37.361639+0000","last_fullsized":"2024-05-09T06:36:37.361639+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:20.155376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171430+0000","last_change":"2024-05-09T06:36:35.718304+0000","last_active":"2024-05-09T06:36:37.171430+0000","last_peered":"2024-05-09T06:36:37.171430+0000","last_clean":"2024-05-09T06:36:37.171430+0000","last_became_active":"2024-05-09T06:36:35.717974+0000","last_became_peered":"2024-05-09T06:36:35.717974+0000","last_unstale":"2024-05-09T06:36:37.171430+0000","last_undegraded":"2024-05-09T06:36:37.171430+0000","last_fullsized":"2024-05-09T06:36:37.171430+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733536+0000","last_change":"2024-05-09T06:36:35.735966+0000","last_active":"2024-05-09T06:36:36.733536+0000","last_peered":"2024-05-09T06:36:36.733536+0000","last_clean":"2024-05-09T06:36:36.733536+0000","last_became_active":"2024-05-09T06:36:35.735641+0000","last_became_peered":"2024-05-09T06:36:35.735641+0000","last_unstale":"2024-05-09T06:36:36.733536+0000","last_undegraded":"2024-05-09T06:36:36.733536+0000","last_fullsized":"2024-05-09T06:36:36.733536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733424+0000","last_change":"2024-05-09T06:36:35.738894+0000","last_active":"2024-05-09T06:36:36.733424+0000","last_peered":"2024-05-09T06:36:36.733424+0000","last_clean":"2024-05-09T06:36:36.733424+0000","last_became_active":"2024-05-09T06:36:35.738562+0000","last_became_peered":"2024-05-09T06:36:35.738562+0000","last_unstale":"2024-05-09T06:36:36.733424+0000","last_undegraded":"2024-05-09T06:36:36.733424+0000","last_fullsized":"2024-05-09T06:36:36.733424+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363129+0000","last_change":"2024-05-09T06:36:35.730775+0000","last_active":"2024-05-09T06:36:37.363129+0000","last_peered":"2024-05-09T06:36:37.363129+0000","last_clean":"2024-05-09T06:36:37.363129+0000","last_became_active":"2024-05-09T06:36:35.728173+0000","last_became_peered":"2024-05-09T06:36:35.728173+0000","last_unstale":"2024-05-09T06:36:37.363129+0000","last_undegraded":"2024-05-09T06:36:37.363129+0000","last_fullsized":"2024-05-09T06:36:37.363129+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:39.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362292+0000","last_change":"2024-05-09T06:36:35.731737+0000","last_active":"2024-05-09T06:36:37.362292+0000","last_peered":"2024-05-09T06:36:37.362292+0000","last_clean":"2024-05-09T06:36:37.362292+0000","last_became_active":"2024-05-09T06:36:35.731409+0000","last_became_peered":"2024-05-09T06:36:35.731409+0000","last_unstale":"2024-05-09T06:36:37.362292+0000","last_undegraded":"2024-05-09T06:36:37.362292+0000","last_fullsized":"2024-05-09T06:36:37.362292+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861277+0000","last_change":"2024-05-09T06:36:35.734977+0000","last_active":"2024-05-09T06:36:36.861277+0000","last_peered":"2024-05-09T06:36:36.861277+0000","last_clean":"2024-05-09T06:36:36.861277+0000","last_became_active":"2024-05-09T06:36:35.734651+0000","last_became_peered":"2024-05-09T06:36:35.734651+0000","last_unstale":"2024-05-09T06:36:36.861277+0000","last_undegraded":"2024-05-09T06:36:36.861277+0000","last_fullsized":"2024-05-09T06:36:36.861277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:27.486929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732999+0000","last_change":"2024-05-09T06:36:35.731456+0000","last_active":"2024-05-09T06:36:36.732999+0000","last_peered":"2024-05-09T06:36:36.732999+0000","last_clean":"2024-05-09T06:36:36.732999+0000","last_became_active":"2024-05-09T06:36:35.731133+0000","last_became_peered":"2024-05-09T06:36:35.731133+0000","last_unstale":"2024-05-09T06:36:36.732999+0000","last_undegraded":"2024-05-09T06:36:36.732999+0000","last_fullsized":"2024-05-09T06:36:36.732999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361943+0000","last_change":"2024-05-09T06:36:35.737479+0000","last_active":"2024-05-09T06:36:37.361943+0000","last_peered":"2024-05-09T06:36:37.361943+0000","last_clean":"2024-05-09T06:36:37.361943+0000","last_became_active":"2024-05-09T06:36:35.736956+0000","last_became_peered":"2024-05-09T06:36:35.736956+0000","last_unstale":"2024-05-09T06:36:37.361943+0000","last_undegraded":"2024-05-09T06:36:37.361943+0000","last_fullsized":"2024-05-09T06:36:37.361943+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860767+0000","last_change":"2024-05-09T06:36:35.739397+0000","last_active":"2024-05-09T06:36:36.860767+0000","last_peered":"2024-05-09T06:36:36.860767+0000","last_clean":"2024-05-09T06:36:36.860767+0000","last_became_active":"2024-05-09T06:36:35.739043+0000","last_became_peered":"2024-05-09T06:36:35.739043+0000","last_unstale":"2024-05-09T06:36:36.860767+0000","last_undegraded":"2024-05-09T06:36:36.860767+0000","last_fullsized":"2024-05-09T06:36:36.860767+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:42:26.873134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361775+0000","last_change":"2024-05-09T06:36:35.736463+0000","last_active":"2024-05-09T06:36:37.361775+0000","last_peered":"2024-05-09T06:36:37.361775+0000","last_clean":"2024-05-09T06:36:37.361775+0000","last_became_active":"2024-05-09T06:36:35.736195+0000","last_became_peered":"2024-05-09T06:36:35.736195+0000","last_unstale":"2024-05-09T06:36:37.361775+0000","last_undegraded":"2024-05-09T06:36:37.361775+0000","last_fullsized":"2024-05-09T06:36:37.361775+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:22.442411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728260+0000","last_change":"2024-05-09T06:36:34.726379+0000","last_active":"2024-05-09T06:36:36.728260+0000","last_peered":"2024-05-09T06:36:36.728260+0000","last_clean":"2024-05-09T06:36:36.728260+0000","last_became_active":"2024-05-09T06:36:34.726052+0000","last_became_peered":"2024-05-09T06:36:34.726052+0000","last_unstale":"2024-05-09T06:36:36.728260+0000","last_undegraded":"2024-05-09T06:36:36.728260+0000","last_fullsized":"2024-05-09T06:36:36.728260+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:45.597809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860744+0000","last_change":"2024-05-09T06:36:35.735129+0000","last_active":"2024-05-09T06:36:36.860744+0000","last_peered":"2024-05-09T06:36:36.860744+0000","last_clean":"2024-05-09T06:36:36.860744+0000","last_became_active":"2024-05-09T06:36:35.734800+0000","last_became_peered":"2024-05-09T06:36:35.734800+0000","last_unstale":"2024-05-09T06:36:36.860744+0000","last_undegraded":"2024-05-09T06:36:36.860744+0000","last_fullsized":"2024-05-09T06:36:36.860744+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:17.925190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860977+0000","last_change":"2024-05-09T06:36:34.720647+0000","last_active":"2024-05-09T06:36:36.860977+0000","last_peered":"2024-05-09T06:36:36.860977+0000","last_clean":"2024-05-09T06:36:36.860977+0000","last_became_active":"2024-05-09T06:36:34.720291+0000","last_became_peered":"2024-05-09T06:36:34.720291+0000","last_unstale":"2024-05-09T06:36:36.860977+0000","last_undegraded":"2024-05-09T06:36:36.860977+0000","last_fullsized":"2024-05-09T06:36:36.860977+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:21.990698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733065+0000","last_change":"2024-05-09T06:36:35.727850+0000","last_active":"2024-05-09T06:36:36.733065+0000","last_peered":"2024-05-09T06:36:36.733065+0000","last_clean":"2024-05-09T06:36:36.733065+0000","last_became_active":"2024-05-09T06:36:35.727537+0000","last_became_peered":"2024-05-09T06:36:35.727537+0000","last_unstale":"2024-05-09T06:36:36.733065+0000","last_undegraded":"2024-05-09T06:36:36.733065+0000","last_fullsized":"2024-05-09T06:36:36.733065+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:39.459731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733323+0000","last_change":"2024-05-09T06:36:34.722771+0000","last_active":"2024-05-09T06:36:36.733323+0000","last_peered":"2024-05-09T06:36:36.733323+0000","last_clean":"2024-05-09T06:36:36.733323+0000","last_became_active":"2024-05-09T06:36:34.722433+0000","last_became_peered":"2024-05-09T06:36:34.722433+0000","last_unstale":"2024-05-09T06:36:36.733323+0000","last_undegraded":"2024-05-09T06:36:36.733323+0000","last_fullsized":"2024-05-09T06:36:36.733323+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:12.993813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362672+0000","last_change":"2024-05-09T06:36:35.735357+0000","last_active":"2024-05-09T06:36:37.362672+0000","last_peered":"2024-05-09T06:36:37.362672+0000","last_clean":"2024-05-09T06:36:37.362672+0000","last_became_active":"2024-05-09T06:36:35.735062+0000","last_became_peered":"2024-05-09T06:36:35.735062+0000","last_unstale":"2024-05-09T06:36:37.362672+0000","last_undegraded":"2024-05-09T06:36:37.362672+0000","last_fullsized":"2024-05-09T06:36:37.362672+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171838+0000","last_change":"2024-05-09T06:36:34.727648+0000","last_active":"2024-05-09T06:36:37.171838+0000","last_peered":"2024-05-09T06:36:37.171838+0000","last_clean":"2024-05-09T06:36:37.171838+0000","last_became_active":"2024-05-09T06:36:34.727344+0000","last_became_peered":"2024-05-09T06:36:34.727344+0000","last_unstale":"2024-05-09T06:36:37.171838+0000","last_undegraded":"2024-05-09T06:36:37.171838+0000","last_fullsized":"2024-05-09T06:36:37.171838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727933+0000","last_change":"2024-05-09T06:36:35.729403+0000","last_active":"2024-05-09T06:36:36.727933+0000","last_peered":"2024-05-09T06:36:36.727933+0000","last_clean":"2024-05-09T06:36:36.727933+0000","last_became_active":"2024-05-09T06:36:35.729145+0000","last_became_peered":"2024-05-09T06:36:35.729145+0000","last_unstale":"2024-05-09T06:36:36.727933+0000","last_undegraded":"2024-05-09T06:36:36.727933+0000","last_fullsized":"2024-05-09T06:36:36.727933+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:22.137711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884438+0000","last_change":"2024-05-09T06:36:34.720781+0000","last_active":"2024-05-09T06:36:36.884438+0000","last_peered":"2024-05-09T06:36:36.884438+0000","last_clean":"2024-05-09T06:36:36.884438+0000","last_became_active":"2024-05-09T06:36:34.720513+0000","last_became_peered":"2024-05-09T06:36:34.720513+0000","last_unstale":"2024-05-09T06:36:36.884438+0000","last_undegraded":"2024-05-09T06:36:36.884438+0000","last_fullsized":"2024-05-09T06:36:36.884438+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362005+0000","last_change":"2024-05-09T06:36:35.733052+0000","last_active":"2024-05-09T06:36:37.362005+0000","last_peered":"2024-05-09T06:36:37.362005+0000","last_clean":"2024-05-09T06:36:37.362005+0000","last_became_active":"2024-05-09T06:36:35.732795+0000","last_became_peered":"2024-05-09T06:36:35.732795+0000","last_unstale":"2024-05-09T06:36:37.362005+0000","last_undegraded":"2024-05-09T06:36:37.362005+0000","last_fullsized":"2024-05-09T06:36:37.362005+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:18:06.228612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362137+0000","last_change":"2024-05-09T06:36:34.725360+0000","last_active":"2024-05-09T06:36:37.362137+0000","last_peered":"2024-05-09T06:36:37.362137+0000","last_clean":"2024-05-09T06:36:37.362137+0000","last_became_active":"2024-05-09T06:36:34.725063+0000","last_became_peered":"2024-05-09T06:36:34.725063+0000","last_unstale":"2024-05-09T06:36:37.362137+0000","last_undegraded":"2024-05-09T06:36:37.362137+0000","last_fullsized":"2024-05-09T06:36:37.362137+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363057+0000","last_change":"2024-05-09T06:36:35.736266+0000","last_active":"2024-05-09T06:36:37.363057+0000","last_peered":"2024-05-09T06:36:37.363057+0000","last_clean":"2024-05-09T06:36:37.363057+0000","last_became_active":"2024-05-09T06:36:35.735960+0000","last_became_peered":"2024-05-09T06:36:35.735960+0000","last_unstale":"2024-05-09T06:36:37.363057+0000","last_undegraded":"2024-05-09T06:36:37.363057+0000","last_fullsized":"2024-05-09T06:36:37.363057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:40.906476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732788+0000","last_change":"2024-05-09T06:36:34.722763+0000","last_active":"2024-05-09T06:36:36.732788+0000","last_peered":"2024-05-09T06:36:36.732788+0000","last_clean":"2024-05-09T06:36:36.732788+0000","last_became_active":"2024-05-09T06:36:34.722478+0000","last_became_peered":"2024-05-09T06:36:34.722478+0000","last_unstale":"2024-05-09T06:36:36.732788+0000","last_undegraded":"2024-05-09T06:36:36.732788+0000","last_fullsized":"2024-05-09T06:36:36.732788+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728229+0000","last_change":"2024-05-09T06:36:35.732081+0000","last_active":"2024-05-09T06:36:36.728229+0000","last_peered":"2024-05-09T06:36:36.728229+0000","last_clean":"2024-05-09T06:36:36.728229+0000","last_became_active":"2024-05-09T06:36:35.731786+0000","last_became_peered":"2024-05-09T06:36:35.731786+0000","last_unstale":"2024-05-09T06:36:36.728229+0000","last_undegraded":"2024-05-09T06:36:36.728229+0000","last_fullsized":"2024-05-09T06:36:36.728229+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:03.123560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727630+0000","last_change":"2024-05-09T06:36:34.725217+0000","last_active":"2024-05-09T06:36:36.727630+0000","last_peered":"2024-05-09T06:36:36.727630+0000","last_clean":"2024-05-09T06:36:36.727630+0000","last_became_active":"2024-05-09T06:36:34.724807+0000","last_became_peered":"2024-05-09T06:36:34.724807+0000","last_unstale":"2024-05-09T06:36:36.727630+0000","last_undegraded":"2024-05-09T06:36:36.727630+0000","last_fullsized":"2024-05-09T06:36:36.727630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:42.511981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361517+0000","last_change":"2024-05-09T06:36:35.731453+0000","last_active":"2024-05-09T06:36:37.361517+0000","last_peered":"2024-05-09T06:36:37.361517+0000","last_clean":"2024-05-09T06:36:37.361517+0000","last_became_active":"2024-05-09T06:36:35.731153+0000","last_became_peered":"2024-05-09T06:36:35.731153+0000","last_unstale":"2024-05-09T06:36:37.361517+0000","last_undegraded":"2024-05-09T06:36:37.361517+0000","last_fullsized":"2024-05-09T06:36:37.361517+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:33.854570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733483+0000","last_change":"2024-05-09T06:36:34.726176+0000","last_active":"2024-05-09T06:36:36.733483+0000","last_peered":"2024-05-09T06:36:36.733483+0000","last_clean":"2024-05-09T06:36:36.733483+0000","last_became_active":"2024-05-09T06:36:34.725866+0000","last_became_peered":"2024-05-09T06:36:34.725866+0000","last_unstale":"2024-05-09T06:36:36.733483+0000","last_undegraded":"2024-05-09T06:36:36.733483+0000","last_fullsized":"2024-05-09T06:36:36.733483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171904+0000","last_change":"2024-05-09T06:36:35.744931+0000","last_active":"2024-05-09T06:36:37.171904+0000","last_peered":"2024-05-09T06:36:37.171904+0000","last_clean":"2024-05-09T06:36:37.171904+0000","last_became_active":"2024-05-09T06:36:35.744642+0000","last_became_peered":"2024-05-09T06:36:35.744642+0000","last_unstale":"2024-05-09T06:36:37.171904+0000","last_undegraded":"2024-05-09T06:36:37.171904+0000","last_fullsized":"2024-05-09T06:36:37.171904+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:46.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361849+0000","last_change":"2024-05-09T06:36:34.713554+0000","last_active":"2024-05-09T06:36:37.361849+0000","last_peered":"2024-05-09T06:36:37.361849+0000","last_clean":"2024-05-09T06:36:37.361849+0000","last_became_active":"2024-05-09T06:36:34.713251+0000","last_became_peered":"2024-05-09T06:36:34.713251+0000","last_unstale":"2024-05-09T06:36:37.361849+0000","last_undegraded":"2024-05-09T06:36:37.361849+0000","last_fullsized":"2024-05-09T06:36:37.361849+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:38.536452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860958+0000","last_change":"2024-05-09T06:36:35.736939+0000","last_active":"2024-05-09T06:36:36.860958+0000","last_peered":"2024-05-09T06:36:36.860958+0000","last_clean":"2024-05-09T06:36:36.860958+0000","last_became_active":"2024-05-09T06:36:35.736669+0000","last_became_peered":"2024-05-09T06:36:35.736669+0000","last_unstale":"2024-05-09T06:36:36.860958+0000","last_undegraded":"2024-05-09T06:36:36.860958+0000","last_fullsized":"2024-05-09T06:36:36.860958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:23.631610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884507+0000","last_change":"2024-05-09T06:36:34.723605+0000","last_active":"2024-05-09T06:36:36.884507+0000","last_peered":"2024-05-09T06:36:36.884507+0000","last_clean":"2024-05-09T06:36:36.884507+0000","last_became_active":"2024-05-09T06:36:34.723298+0000","last_became_peered":"2024-05-09T06:36:34.723298+0000","last_unstale":"2024-05-09T06:36:36.884507+0000","last_undegraded":"2024-05-09T06:36:36.884507+0000","last_fullsized":"2024-05-09T06:36:36.884507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171805+0000","last_change":"2024-05-09T06:36:35.746081+0000","last_active":"2024-05-09T06:36:37.171805+0000","last_peered":"2024-05-09T06:36:37.171805+0000","last_clean":"2024-05-09T06:36:37.171805+0000","last_became_active":"2024-05-09T06:36:35.745744+0000","last_became_peered":"2024-05-09T06:36:35.745744+0000","last_unstale":"2024-05-09T06:36:37.171805+0000","last_undegraded":"2024-05-09T06:36:37.171805+0000","last_fullsized":"2024-05-09T06:36:37.171805+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:07.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362413+0000","last_change":"2024-05-09T06:36:34.720057+0000","last_active":"2024-05-09T06:36:37.362413+0000","last_peered":"2024-05-09T06:36:37.362413+0000","last_clean":"2024-05-09T06:36:37.362413+0000","last_became_active":"2024-05-09T06:36:34.719725+0000","last_became_peered":"2024-05-09T06:36:34.719725+0000","last_unstale":"2024-05-09T06:36:37.362413+0000","last_undegraded":"2024-05-09T06:36:37.362413+0000","last_fullsized":"2024-05-09T06:36:37.362413+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362337+0000","last_change":"2024-05-09T06:36:35.734423+0000","last_active":"2024-05-09T06:36:37.362337+0000","last_peered":"2024-05-09T06:36:37.362337+0000","last_clean":"2024-05-09T06:36:37.362337+0000","last_became_active":"2024-05-09T06:36:35.733912+0000","last_became_peered":"2024-05-09T06:36:35.733912+0000","last_unstale":"2024-05-09T06:36:37.362337+0000","last_undegraded":"2024-05-09T06:36:37.362337+0000","last_fullsized":"2024-05-09T06:36:37.362337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171840+0000","last_change":"2024-05-09T06:36:34.731400+0000","last_active":"2024-05-09T06:36:37.171840+0000","last_peered":"2024-05-09T06:36:37.171840+0000","last_clean":"2024-05-09T06:36:37.171840+0000","last_became_active":"2024-05-09T06:36:34.731097+0000","last_became_peered":"2024-05-09T06:36:34.731097+0000","last_unstale":"2024-05-09T06:36:37.171840+0000","last_undegraded":"2024-05-09T06:36:37.171840+0000","last_fullsized":"2024-05-09T06:36:37.171840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362862+0000","last_change":"2024-05-09T06:36:35.731381+0000","last_active":"2024-05-09T06:36:37.362862+0000","last_peered":"2024-05-09T06:36:37.362862+0000","last_clean":"2024-05-09T06:36:37.362862+0000","last_became_active":"2024-05-09T06:36:35.731083+0000","last_became_peered":"2024-05-09T06:36:35.731083+0000","last_unstale":"2024-05-09T06:36:37.362862+0000","last_undegraded":"2024-05-09T06:36:37.362862+0000","last_fullsized":"2024-05-09T06:36:37.362862+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:38.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727873+0000","last_change":"2024-05-09T06:36:34.722286+0000","last_active":"2024-05-09T06:36:36.727873+0000","last_peered":"2024-05-09T06:36:36.727873+0000","last_clean":"2024-05-09T06:36:36.727873+0000","last_became_active":"2024-05-09T06:36:34.721994+0000","last_became_peered":"2024-05-09T06:36:34.721994+0000","last_unstale":"2024-05-09T06:36:36.727873+0000","last_undegraded":"2024-05-09T06:36:36.727873+0000","last_fullsized":"2024-05-09T06:36:36.727873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:42.279096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860860+0000","last_change":"2024-05-09T06:36:35.741059+0000","last_active":"2024-05-09T06:36:36.860860+0000","last_peered":"2024-05-09T06:36:36.860860+0000","last_clean":"2024-05-09T06:36:36.860860+0000","last_became_active":"2024-05-09T06:36:35.739832+0000","last_became_peered":"2024-05-09T06:36:35.739832+0000","last_unstale":"2024-05-09T06:36:36.860860+0000","last_undegraded":"2024-05-09T06:36:36.860860+0000","last_fullsized":"2024-05-09T06:36:36.860860+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:13.676589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860765+0000","last_change":"2024-05-09T06:36:34.717389+0000","last_active":"2024-05-09T06:36:36.860765+0000","last_peered":"2024-05-09T06:36:36.860765+0000","last_clean":"2024-05-09T06:36:36.860765+0000","last_became_active":"2024-05-09T06:36:34.717129+0000","last_became_peered":"2024-05-09T06:36:34.717129+0000","last_unstale":"2024-05-09T06:36:36.860765+0000","last_undegraded":"2024-05-09T06:36:36.860765+0000","last_fullsized":"2024-05-09T06:36:36.860765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:00.252278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362192+0000","last_change":"2024-05-09T06:36:35.729816+0000","last_active":"2024-05-09T06:36:37.362192+0000","last_peered":"2024-05-09T06:36:37.362192+0000","last_clean":"2024-05-09T06:36:37.362192+0000","last_became_active":"2024-05-09T06:36:35.729478+0000","last_became_peered":"2024-05-09T06:36:35.729478+0000","last_unstale":"2024-05-09T06:36:37.362192+0000","last_undegraded":"2024-05-09T06:36:37.362192+0000","last_fullsized":"2024-05-09T06:36:37.362192+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:14.738457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.732784+0000","last_change":"2024-05-09T06:36:34.722070+0000","last_active":"2024-05-09T06:36:36.732784+0000","last_peered":"2024-05-09T06:36:36.732784+0000","last_clean":"2024-05-09T06:36:36.732784+0000","last_became_active":"2024-05-09T06:36:34.721798+0000","last_became_peered":"2024-05-09T06:36:34.721798+0000","last_unstale":"2024-05-09T06:36:36.732784+0000","last_undegraded":"2024-05-09T06:36:36.732784+0000","last_fullsized":"2024-05-09T06:36:36.732784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:14.243131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860899+0000","last_change":"2024-05-09T06:36:35.732608+0000","last_active":"2024-05-09T06:36:36.860899+0000","last_peered":"2024-05-09T06:36:36.860899+0000","last_clean":"2024-05-09T06:36:36.860899+0000","last_became_active":"2024-05-09T06:36:35.732303+0000","last_became_peered":"2024-05-09T06:36:35.732303+0000","last_unstale":"2024-05-09T06:36:36.860899+0000","last_undegraded":"2024-05-09T06:36:36.860899+0000","last_fullsized":"2024-05-09T06:36:36.860899+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:20.080800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362751+0000","last_change":"2024-05-09T06:36:34.720971+0000","last_active":"2024-05-09T06:36:37.362751+0000","last_peered":"2024-05-09T06:36:37.362751+0000","last_clean":"2024-05-09T06:36:37.362751+0000","last_became_active":"2024-05-09T06:36:34.719853+0000","last_became_peered":"2024-05-09T06:36:34.719853+0000","last_unstale":"2024-05-09T06:36:37.362751+0000","last_undegraded":"2024-05-09T06:36:37.362751+0000","last_fullsized":"2024-05-09T06:36:37.362751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362173+0000","last_change":"2024-05-09T06:36:35.731420+0000","last_active":"2024-05-09T06:36:37.362173+0000","last_peered":"2024-05-09T06:36:37.362173+0000","last_clean":"2024-05-09T06:36:37.362173+0000","last_became_active":"2024-05-09T06:36:35.731137+0000","last_became_peered":"2024-05-09T06:36:35.731137+0000","last_unstale":"2024-05-09T06:36:37.362173+0000","last_undegraded":"2024-05-09T06:36:37.362173+0000","last_fullsized":"2024-05-09T06:36:37.362173+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:07.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884572+0000","last_change":"2024-05-09T06:36:34.725998+0000","last_active":"2024-05-09T06:36:36.884572+0000","last_peered":"2024-05-09T06:36:36.884572+0000","last_clean":"2024-05-09T06:36:36.884572+0000","last_became_active":"2024-05-09T06:36:34.725716+0000","last_became_peered":"2024-05-09T06:36:34.725716+0000","last_unstale":"2024-05-09T06:36:36.884572+0000","last_undegraded":"2024-05-09T06:36:36.884572+0000","last_fullsized":"2024-05-09T06:36:36.884572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:37.395989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171699+0000","last_change":"2024-05-09T06:36:35.733132+0000","last_active":"2024-05-09T06:36:37.171699+0000","last_peered":"2024-05-09T06:36:37.171699+0000","last_clean":"2024-05-09T06:36:37.171699+0000","last_became_active":"2024-05-09T06:36:35.732800+0000","last_became_peered":"2024-05-09T06:36:35.732800+0000","last_unstale":"2024-05-09T06:36:37.171699+0000","last_undegraded":"2024-05-09T06:36:37.171699+0000","last_fullsized":"2024-05-09T06:36:37.171699+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:25.585098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362998+0000","last_change":"2024-05-09T06:36:34.725109+0000","last_active":"2024-05-09T06:36:37.362998+0000","last_peered":"2024-05-09T06:36:37.362998+0000","last_clean":"2024-05-09T06:36:37.362998+0000","last_became_active":"2024-05-09T06:36:34.724684+0000","last_became_peered":"2024-05-09T06:36:34.724684+0000","last_unstale":"2024-05-09T06:36:37.362998+0000","last_undegraded":"2024-05-09T06:36:37.362998+0000","last_fullsized":"2024-05-09T06:36:37.362998+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361762+0000","last_change":"2024-05-09T06:36:35.723304+0000","last_active":"2024-05-09T06:36:37.361762+0000","last_peered":"2024-05-09T06:36:37.361762+0000","last_clean":"2024-05-09T06:36:37.361762+0000","last_became_active":"2024-05-09T06:36:35.723007+0000","last_became_peered":"2024-05-09T06:36:35.723007+0000","last_unstale":"2024-05-09T06:36:37.361762+0000","last_undegraded":"2024-05-09T06:36:37.361762+0000","last_fullsized":"2024-05-09T06:36:37.361762+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:25.757227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861312+0000","last_change":"2024-05-09T06:36:34.729608+0000","last_active":"2024-05-09T06:36:36.861312+0000","last_peered":"2024-05-09T06:36:36.861312+0000","last_clean":"2024-05-09T06:36:36.861312+0000","last_became_active":"2024-05-09T06:36:34.729292+0000","last_became_peered":"2024-05-09T06:36:34.729292+0000","last_unstale":"2024-05-09T06:36:36.861312+0000","last_undegraded":"2024-05-09T06:36:36.861312+0000","last_fullsized":"2024-05-09T06:36:36.861312+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:54.540738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361952+0000","last_change":"2024-05-09T06:36:35.719670+0000","last_active":"2024-05-09T06:36:37.361952+0000","last_peered":"2024-05-09T06:36:37.361952+0000","last_clean":"2024-05-09T06:36:37.361952+0000","last_became_active":"2024-05-09T06:36:35.719375+0000","last_became_peered":"2024-05-09T06:36:35.719375+0000","last_unstale":"2024-05-09T06:36:37.361952+0000","last_undegraded":"2024-05-09T06:36:37.361952+0000","last_fullsized":"2024-05-09T06:36:37.361952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:33.896862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860695+0000","last_change":"2024-05-09T06:36:34.725264+0000","last_active":"2024-05-09T06:36:36.860695+0000","last_peered":"2024-05-09T06:36:36.860695+0000","last_clean":"2024-05-09T06:36:36.860695+0000","last_became_active":"2024-05-09T06:36:34.724991+0000","last_became_peered":"2024-05-09T06:36:34.724991+0000","last_unstale":"2024-05-09T06:36:36.860695+0000","last_undegraded":"2024-05-09T06:36:36.860695+0000","last_fullsized":"2024-05-09T06:36:36.860695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:57.900999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.728143+0000","last_change":"2024-05-09T06:36:35.738237+0000","last_active":"2024-05-09T06:36:36.728143+0000","last_peered":"2024-05-09T06:36:36.728143+0000","last_clean":"2024-05-09T06:36:36.728143+0000","last_became_active":"2024-05-09T06:36:35.737494+0000","last_became_peered":"2024-05-09T06:36:35.737494+0000","last_unstale":"2024-05-09T06:36:36.728143+0000","last_undegraded":"2024-05-09T06:36:36.728143+0000","last_fullsized":"2024-05-09T06:36:36.728143+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:43.210498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361934+0000","last_change":"2024-05-09T06:36:34.725808+0000","last_active":"2024-05-09T06:36:37.361934+0000","last_peered":"2024-05-09T06:36:37.361934+0000","last_clean":"2024-05-09T06:36:37.361934+0000","last_became_active":"2024-05-09T06:36:34.725524+0000","last_became_peered":"2024-05-09T06:36:34.725524+0000","last_unstale":"2024-05-09T06:36:37.361934+0000","last_undegraded":"2024-05-09T06:36:37.361934+0000","last_fullsized":"2024-05-09T06:36:37.361934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:12.993813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861054+0000","last_change":"2024-05-09T06:36:35.736077+0000","last_active":"2024-05-09T06:36:36.861054+0000","last_peered":"2024-05-09T06:36:36.861054+0000","last_clean":"2024-05-09T06:36:36.861054+0000","last_became_active":"2024-05-09T06:36:35.735802+0000","last_became_peered":"2024-05-09T06:36:35.735802+0000","last_unstale":"2024-05-09T06:36:36.861054+0000","last_undegraded":"2024-05-09T06:36:36.861054+0000","last_fullsized":"2024-05-09T06:36:36.861054+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:17.420596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860867+0000","last_change":"2024-05-09T06:36:34.724792+0000","last_active":"2024-05-09T06:36:36.860867+0000","last_peered":"2024-05-09T06:36:36.860867+0000","last_clean":"2024-05-09T06:36:36.860867+0000","last_became_active":"2024-05-09T06:36:34.724482+0000","last_became_peered":"2024-05-09T06:36:34.724482+0000","last_unstale":"2024-05-09T06:36:36.860867+0000","last_undegraded":"2024-05-09T06:36:36.860867+0000","last_fullsized":"2024-05-09T06:36:36.860867+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:07:37.551668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361503+0000","last_change":"2024-05-09T06:36:35.732071+0000","last_active":"2024-05-09T06:36:37.361503+0000","last_peered":"2024-05-09T06:36:37.361503+0000","last_clean":"2024-05-09T06:36:37.361503+0000","last_became_active":"2024-05-09T06:36:35.731781+0000","last_became_peered":"2024-05-09T06:36:35.731781+0000","last_unstale":"2024-05-09T06:36:37.361503+0000","last_undegraded":"2024-05-09T06:36:37.361503+0000","last_fullsized":"2024-05-09T06:36:37.361503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:47.442436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171964+0000","last_change":"2024-05-09T06:36:34.729856+0000","last_active":"2024-05-09T06:36:37.171964+0000","last_peered":"2024-05-09T06:36:37.171964+0000","last_clean":"2024-05-09T06:36:37.171964+0000","last_became_active":"2024-05-09T06:36:34.729560+0000","last_became_peered":"2024-05-09T06:36:34.729560+0000","last_unstale":"2024-05-09T06:36:37.171964+0000","last_undegraded":"2024-05-09T06:36:37.171964+0000","last_fullsized":"2024-05-09T06:36:37.171964+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:21.429625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733047+0000","last_change":"2024-05-09T06:36:35.730375+0000","last_active":"2024-05-09T06:36:36.733047+0000","last_peered":"2024-05-09T06:36:36.733047+0000","last_clean":"2024-05-09T06:36:36.733047+0000","last_became_active":"2024-05-09T06:36:35.730096+0000","last_became_peered":"2024-05-09T06:36:35.730096+0000","last_unstale":"2024-05-09T06:36:36.733047+0000","last_undegraded":"2024-05-09T06:36:36.733047+0000","last_fullsized":"2024-05-09T06:36:36.733047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:08.226830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361893+0000","last_change":"2024-05-09T06:36:34.725892+0000","last_active":"2024-05-09T06:36:37.361893+0000","last_peered":"2024-05-09T06:36:37.361893+0000","last_clean":"2024-05-09T06:36:37.361893+0000","last_became_active":"2024-05-09T06:36:34.725587+0000","last_became_peered":"2024-05-09T06:36:34.725587+0000","last_unstale":"2024-05-09T06:36:37.361893+0000","last_undegraded":"2024-05-09T06:36:37.361893+0000","last_fullsized":"2024-05-09T06:36:37.361893+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:21.038142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.727794+0000","last_change":"2024-05-09T06:36:35.742341+0000","last_active":"2024-05-09T06:36:36.727794+0000","last_peered":"2024-05-09T06:36:36.727794+0000","last_clean":"2024-05-09T06:36:36.727794+0000","last_became_active":"2024-05-09T06:36:35.742047+0000","last_became_peered":"2024-05-09T06:36:35.742047+0000","last_unstale":"2024-05-09T06:36:36.727794+0000","last_undegraded":"2024-05-09T06:36:36.727794+0000","last_fullsized":"2024-05-09T06:36:36.727794+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:42.840248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362689+0000","last_change":"2024-05-09T06:36:34.725138+0000","last_active":"2024-05-09T06:36:37.362689+0000","last_peered":"2024-05-09T06:36:37.362689+0000","last_clean":"2024-05-09T06:36:37.362689+0000","last_became_active":"2024-05-09T06:36:34.724821+0000","last_became_peered":"2024-05-09T06:36:34.724821+0000","last_unstale":"2024-05-09T06:36:37.362689+0000","last_undegraded":"2024-05-09T06:36:37.362689+0000","last_fullsized":"2024-05-09T06:36:37.362689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362291+0000","last_change":"2024-05-09T06:36:35.733528+0000","last_active":"2024-05-09T06:36:37.362291+0000","last_peered":"2024-05-09T06:36:37.362291+0000","last_clean":"2024-05-09T06:36:37.362291+0000","last_became_active":"2024-05-09T06:36:35.733210+0000","last_became_peered":"2024-05-09T06:36:35.733210+0000","last_unstale":"2024-05-09T06:36:37.362291+0000","last_undegraded":"2024-05-09T06:36:37.362291+0000","last_fullsized":"2024-05-09T06:36:37.362291+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:58.828391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.171902+0000","last_change":"2024-05-09T06:36:34.728780+0000","last_active":"2024-05-09T06:36:37.171902+0000","last_peered":"2024-05-09T06:36:37.171902+0000","last_clean":"2024-05-09T06:36:37.171902+0000","last_became_active":"2024-05-09T06:36:34.728489+0000","last_became_peered":"2024-05-09T06:36:34.728489+0000","last_unstale":"2024-05-09T06:36:37.171902+0000","last_undegraded":"2024-05-09T06:36:37.171902+0000","last_fullsized":"2024-05-09T06:36:37.171902+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:06.415632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860852+0000","last_change":"2024-05-09T06:36:35.739237+0000","last_active":"2024-05-09T06:36:36.860852+0000","last_peered":"2024-05-09T06:36:36.860852+0000","last_clean":"2024-05-09T06:36:36.860852+0000","last_became_active":"2024-05-09T06:36:35.738912+0000","last_became_peered":"2024-05-09T06:36:35.738912+0000","last_unstale":"2024-05-09T06:36:36.860852+0000","last_undegraded":"2024-05-09T06:36:36.860852+0000","last_fullsized":"2024-05-09T06:36:36.860852+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:00:43.136465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.733190+0000","last_change":"2024-05-09T06:36:34.720898+0000","last_active":"2024-05-09T06:36:36.733190+0000","last_peered":"2024-05-09T06:36:36.733190+0000","last_clean":"2024-05-09T06:36:36.733190+0000","last_became_active":"2024-05-09T06:36:34.720626+0000","last_became_peered":"2024-05-09T06:36:34.720626+0000","last_unstale":"2024-05-09T06:36:36.733190+0000","last_undegraded":"2024-05-09T06:36:36.733190+0000","last_fullsized":"2024-05-09T06:36:36.733190+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:11:58.420053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860911+0000","last_change":"2024-05-09T06:36:35.739319+0000","last_active":"2024-05-09T06:36:36.860911+0000","last_peered":"2024-05-09T06:36:36.860911+0000","last_clean":"2024-05-09T06:36:36.860911+0000","last_became_active":"2024-05-09T06:36:35.738697+0000","last_became_peered":"2024-05-09T06:36:35.738697+0000","last_unstale":"2024-05-09T06:36:36.860911+0000","last_undegraded":"2024-05-09T06:36:36.860911+0000","last_fullsized":"2024-05-09T06:36:36.860911+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:14.451235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860934+0000","last_change":"2024-05-09T06:36:34.717861+0000","last_active":"2024-05-09T06:36:36.860934+0000","last_peered":"2024-05-09T06:36:36.860934+0000","last_clean":"2024-05-09T06:36:36.860934+0000","last_became_active":"2024-05-09T06:36:34.717576+0000","last_became_peered":"2024-05-09T06:36:34.717576+0000","last_unstale":"2024-05-09T06:36:36.860934+0000","last_undegraded":"2024-05-09T06:36:36.860934+0000","last_fullsized":"2024-05-09T06:36:36.860934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:25.427088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884682+0000","last_change":"2024-05-09T06:36:35.732696+0000","last_active":"2024-05-09T06:36:36.884682+0000","last_peered":"2024-05-09T06:36:36.884682+0000","last_clean":"2024-05-09T06:36:36.884682+0000","last_became_active":"2024-05-09T06:36:35.732342+0000","last_became_peered":"2024-05-09T06:36:35.732342+0000","last_unstale":"2024-05-09T06:36:36.884682+0000","last_undegraded":"2024-05-09T06:36:36.884682+0000","last_fullsized":"2024-05-09T06:36:36.884682+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:46.341366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.362069+0000","last_change":"2024-05-09T06:36:34.724569+0000","last_active":"2024-05-09T06:36:37.362069+0000","last_peered":"2024-05-09T06:36:37.362069+0000","last_clean":"2024-05-09T06:36:37.362069+0000","last_became_active":"2024-05-09T06:36:34.724223+0000","last_became_peered":"2024-05-09T06:36:34.724223+0000","last_unstale":"2024-05-09T06:36:37.362069+0000","last_undegraded":"2024-05-09T06:36:37.362069+0000","last_fullsized":"2024-05-09T06:36:37.362069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:51.795800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.363148+0000","last_change":"2024-05-09T06:36:35.728379+0000","last_active":"2024-05-09T06:36:37.363148+0000","last_peered":"2024-05-09T06:36:37.363148+0000","last_clean":"2024-05-09T06:36:37.363148+0000","last_became_active":"2024-05-09T06:36:35.728035+0000","last_became_peered":"2024-05-09T06:36:35.728035+0000","last_unstale":"2024-05-09T06:36:37.363148+0000","last_undegraded":"2024-05-09T06:36:37.363148+0000","last_fullsized":"2024-05-09T06:36:37.363148+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:23.344489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.860972+0000","last_change":"2024-05-09T06:36:34.719103+0000","last_active":"2024-05-09T06:36:36.860972+0000","last_peered":"2024-05-09T06:36:36.860972+0000","last_clean":"2024-05-09T06:36:36.860972+0000","last_became_active":"2024-05-09T06:36:34.718806+0000","last_became_peered":"2024-05-09T06:36:34.718806+0000","last_unstale":"2024-05-09T06:36:36.860972+0000","last_undegraded":"2024-05-09T06:36:36.860972+0000","last_fullsized":"2024-05-09T06:36:36.860972+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:14.284730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:37.361889+0000","last_change":"2024-05-09T06:36:35.732848+0000","last_active":"2024-05-09T06:36:37.361889+0000","last_peered":"2024-05-09T06:36:37.361889+0000","last_clean":"2024-05-09T06:36:37.361889+0000","last_became_active":"2024-05-09T06:36:35.732542+0000","last_became_peered":"2024-05-09T06:36:35.732542+0000","last_unstale":"2024-05-09T06:36:37.361889+0000","last_undegraded":"2024-05-09T06:36:37.361889+0000","last_fullsized":"2024-05-09T06:36:37.361889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:34.700633+0000","last_clean_scrub_stamp":"2024-05-09T06:36:34.700633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:20.117527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.861044+0000","last_change":"2024-05-09T06:36:34.724039+0000","last_active":"2024-05-09T06:36:36.861044+0000","last_peered":"2024-05-09T06:36:36.861044+0000","last_clean":"2024-05-09T06:36:36.861044+0000","last_became_active":"2024-05-09T06:36:34.723772+0000","last_became_peered":"2024-05-09T06:36:34.723772+0000","last_unstale":"2024-05-09T06:36:36.861044+0000","last_undegraded":"2024-05-09T06:36:36.861044+0000","last_fullsized":"2024-05-09T06:36:36.861044+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:33.696349+0000","last_clean_scrub_stamp":"2024-05-09T06:36:33.696349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:10:58.238882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":328,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-05-09T06:36:36.884146+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:36:36.884146+0000","last_peered":"2024-05-09T06:36:36.884146+0000","last_clean":"2024-05-09T06:36:36.884146+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:36:36.884146+0000","last_undegraded":"2024-05-09T06:36:36.884146+0000","last_fullsized":"2024-05-09T06:36:36.884146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1203048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":614736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":612520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":612612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":613102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1204590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:40.085 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:36:40.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:36:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.252+0000 7f691c777640 1 Processor -- start 2024-05-09T06:36:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.252+0000 7f691c777640 1 -- start start 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f691c777640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f691406b1e0 0x7f69140cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f691c777640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6914072050 con 0x7f69140c41e0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f691c777640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6914072330 con 0x7f69140d06f0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f691c777640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6914072610 con 0x7f691406b1e0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919d12640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f69140c41e0 0x7f69140ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57822/0 (socket says 172.21.15.1:57822) 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919d12640 1 -- 172.21.15.1:0/3662107854 learned_addr learned my addr 172.21.15.1:0/3662107854 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f691ad14640 1 --2- 172.21.15.1:0/3662107854 >> v2:172.21.15.6:3300/0 conn(0x7f691406b1e0 0x7f69140cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704455203 0 0) 0x7f6914072050 con 0x7f69140c41e0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6900003520 con 0x7f69140c41e0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2877521600 0 0) 0x7f6900003520 con 0x7f69140c41e0 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 >> v2:172.21.15.6:3300/0 conn(0x7f691406b1e0 msgr2=0x7f69140cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 --2- 172.21.15.1:0/3662107854 >> v2:172.21.15.6:3300/0 conn(0x7f691406b1e0 0x7f69140cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 >> v1:172.21.15.6:6789/0 conn(0x7f69140d06f0 legacy=0x7f69140cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69140728f0 con 0x7f69140c41e0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.253+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6904001670 con 0x7f69140c41e0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 >> v1:172.21.15.1:6789/0 conn(0x7f69140c41e0 legacy=0x7f69140ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6914072610 con 0x7f69000039b0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f690000c060 con 0x7f6900004800 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f690000c260 con 0x7f69000051c0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f6919d12640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691ad14640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 -- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69000051c0 msgr2=0x7f690000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 -- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6900004800 msgr2=0x7f6900004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691a513640 1 -- 172.21.15.1:0/3662107854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6900003520 con 0x7f69000039b0 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.254+0000 7f691ad14640 1 -- 172.21.15.1:0/3662107854 reap_dead start 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f691a513640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6910001750 tx=0x7f6900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66183510e628793c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910001370 con 0x7f69000039b0 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69100052d0 con 0x7f69000039b0 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f6919511640 1 -- 172.21.15.1:0/3662107854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691002abe0 con 0x7f69000039b0 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f691c777640 1 -- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 msgr2=0x7f6900003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.255+0000 7f691c777640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6910001750 tx=0x7f6900005830 comp rx=0 tx=0).stop 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.256+0000 7f691c777640 1 -- 172.21.15.1:0/3662107854 shutdown_connections 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.256+0000 7f691c777640 1 --2- 172.21.15.1:0/3662107854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.256+0000 7f691c777640 1 -- 172.21.15.1:0/3662107854 >> 172.21.15.1:0/3662107854 conn(0x7f69140d0b50 msgr2=0x7f69140d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.256+0000 7f691c777640 1 -- 172.21.15.1:0/3662107854 shutdown_connections 2024-05-09T06:36:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.256+0000 7f691c777640 1 -- 172.21.15.1:0/3662107854 wait complete. 2024-05-09T06:36:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.257+0000 7f691c777640 1 Processor -- start 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.257+0000 7f691c777640 1 -- start start 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.257+0000 7f691c777640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69000039b0 0x7f69141e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 0x7f691414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69140cc860 con 0x7f69140c4610 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f69140cc6e0 con 0x7f69000039b0 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f69140cc9e0 con 0x7f691414ff30 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55438/0 (socket says 172.21.15.1:55438) 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 -- 172.21.15.1:0/1289675318 learned_addr learned my addr 172.21.15.1:0/1289675318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691a513640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69000039b0 0x7f69141e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f6919d12640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 0x7f691414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 -- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 msgr2=0x7f691414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 0x7f691414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 -- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69000039b0 msgr2=0x7f69141e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69000039b0 0x7f69141e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 -- 172.21.15.1:0/1289675318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69140c4f80 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f6919d12640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 0x7f691414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691ad14640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f69140b2690 tx=0x7f690c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00fce0 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c01d070 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c015670 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69140c5260 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.258+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69140c5820 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.259+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69140752b0 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.260+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f690c015810 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.260+0000 7f690affd640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.260+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f690c08b8e0 con 0x7f69140c4610 2024-05-09T06:36:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.261+0000 7f691a513640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.261+0000 7f691a513640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6910032000 tx=0x7f691002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.266+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f690c010040 con 0x7f69140c4610 2024-05-09T06:36:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.516+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6914072f30 con 0x7f69140c4610 2024-05-09T06:36:40.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.518+0000 7f690affd640 1 -- 172.21.15.1:0/1289675318 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f690c05a060 con 0x7f69140c4610 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:36:40.499 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:36:40.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 msgr2=0x7f68f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6910032000 tx=0x7f691002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:40.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 msgr2=0x7f69140c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f69140b2690 tx=0x7f690c00f1b0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 shutdown_connections 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f68f806be30 0x7f68f806e2f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69140c4610 0x7f69140c6020 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f691414ff30 0x7f691414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 --2- 172.21.15.1:0/1289675318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69000039b0 0x7f69141e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 >> 172.21.15.1:0/1289675318 conn(0x7f69140d0b50 msgr2=0x7f69140b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 shutdown_connections 2024-05-09T06:36:40.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.520+0000 7f691c777640 1 -- 172.21.15.1:0/1289675318 wait complete. 2024-05-09T06:36:40.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:40.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 Processor -- start 2024-05-09T06:36:40.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 -- start start 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2304161950 0x7f23040d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23040d4940 0x7f23040d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040b33c0 con 0x7f23040d0f80 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f23040d4dd0 con 0x7f2304161950 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.635+0000 7f230b7db640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f23040b3660 con 0x7f23040d4940 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2309d78640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f23040d0f80 0x7f23040cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57830/0 (socket says 172.21.15.1:57830) 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2309d78640 1 -- 172.21.15.1:0/606704179 learned_addr learned my addr 172.21.15.1:0/606704179 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2309577640 1 --2- 172.21.15.1:0/606704179 >> v2:172.21.15.6:3300/0 conn(0x7f2304161950 0x7f23040d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994247131 0 0) 0x7f23040b33c0 con 0x7f23040d0f80 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2308d76640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23040d4940 0x7f23040d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22ec003520 con 0x7f23040d0f80 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1439669284 0 0) 0x7f22ec003520 con 0x7f23040d0f80 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23040d4940 msgr2=0x7f23040d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23040d4940 0x7f23040d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 >> v2:172.21.15.6:3300/0 conn(0x7f2304161950 msgr2=0x7f23040d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 --2- 172.21.15.1:0/606704179 >> v2:172.21.15.6:3300/0 conn(0x7f2304161950 0x7f23040d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040b3940 con 0x7f23040d0f80 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f22fc001670 con 0x7f23040d0f80 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 >> v1:172.21.15.1:6789/0 conn(0x7f23040d0f80 legacy=0x7f23040cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22ec0039b0 0x7f22ec003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f22ec004800 0x7f22ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ec005180 0x7f22ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23040d4dd0 con 0x7f22ec005180 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f23040b3660 con 0x7f22ec0039b0 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec00bf00 con 0x7f22ec004800 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2309d78640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ec005180 0x7f22ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2309577640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22ec0039b0 0x7f22ec003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.636+0000 7f2308d76640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f22ec004800 0x7f22ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 -- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f22ec004800 msgr2=0x7f22ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f22ec004800 0x7f22ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 -- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22ec0039b0 msgr2=0x7f22ec003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f22ec0039b0 0x7f22ec003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 -- 172.21.15.1:0/606704179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec003520 con 0x7f22ec005180 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.637+0000 7f2309d78640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ec005180 0x7f22ec00b9c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f230408d0b0 tx=0x7f22ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66a8e42c19c30ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc007c80 con 0x7f22ec005180 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc005af0 con 0x7f22ec005180 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f2303fff640 1 -- 172.21.15.1:0/606704179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc007070 con 0x7f22ec005180 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 -- 172.21.15.1:0/606704179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ec005180 msgr2=0x7f22ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 --2- 172.21.15.1:0/606704179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22ec005180 0x7f22ec00b9c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f230408d0b0 tx=0x7f22ec005830 comp rx=0 tx=0).stop 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f2309d78640 1 -- 172.21.15.1:0/606704179 reap_dead start 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 -- 172.21.15.1:0/606704179 shutdown_connections 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 -- 172.21.15.1:0/606704179 >> 172.21.15.1:0/606704179 conn(0x7f23040d3e20 msgr2=0x7f23040d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 -- 172.21.15.1:0/606704179 shutdown_connections 2024-05-09T06:36:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.638+0000 7f230b7db640 1 -- 172.21.15.1:0/606704179 wait complete. 2024-05-09T06:36:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.639+0000 7f230b7db640 1 Processor -- start 2024-05-09T06:36:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.639+0000 7f230b7db640 1 -- start start 2024-05-09T06:36:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 0x7f23040cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2304160250 0x7f23041606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23040b52d0 con 0x7f23040b84c0 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f23040b5150 con 0x7f230415c430 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f230b7db640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f23040b5450 con 0x7f2304160250 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f2308d76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f2308d76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55456/0 (socket says 172.21.15.1:55456) 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f2308d76640 1 -- 172.21.15.1:0/1961741110 learned_addr learned my addr 172.21.15.1:0/1961741110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.640+0000 7f2309d78640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2304160250 0x7f23041606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 -- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2304160250 msgr2=0x7f23041606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2304160250 0x7f23041606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2309577640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 0x7f23040cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 -- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 msgr2=0x7f23040cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 0x7f23040cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 -- 172.21.15.1:0/1961741110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040b7420 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2309577640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 0x7f23040cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2308d76640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f22f80119d0 tx=0x7f22f8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800fb50 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f800fcf0 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23040b7700 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800fea0 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.641+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23040b7cc0 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.642+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2304160c10 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.643+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22f8012040 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.643+0000 7f2301ffb640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 0x7f22f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.643+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f22f808b6e0 con 0x7f23040b84c0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.643+0000 7f2309577640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 0x7f22f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.644+0000 7f2309577640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 0x7f22f006e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f22f4007440 tx=0x7f22f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.648+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22f8019030 con 0x7f23040b84c0 2024-05-09T06:36:40.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.869+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23040752b0 con 0x7f23040b84c0 2024-05-09T06:36:40.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.870+0000 7f2301ffb640 1 -- 172.21.15.1:0/1961741110 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5171 (secure 0 0 0) 0x7f22f80598b0 con 0x7f23040b84c0 2024-05-09T06:36:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:40.851 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:36:38.092253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6835/3796896332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3796896332},{"type":"v1","addr":"172.21.15.1:6835","nonce":3796896332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":7}]} 2024-05-09T06:36:40.851 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 82 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 msgr2=0x7f22f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 0x7f22f006e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f22f4007440 tx=0x7f22f400b040 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 msgr2=0x7f23040b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f22f80119d0 tx=0x7f22f8011e90 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 shutdown_connections 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f22f006be30 0x7f22f006e2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2304160250 0x7f23041606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23040b84c0 0x7f23040b6c80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 --2- 172.21.15.1:0/1961741110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f230415c430 0x7f23040cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 >> 172.21.15.1:0/1961741110 conn(0x7f23040d3e20 msgr2=0x7f230406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 shutdown_connections 2024-05-09T06:36:40.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:40.872+0000 7f230b7db640 1 -- 172.21.15.1:0/1961741110 wait complete. 2024-05-09T06:36:41.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:35.082530+0000', 'modified': '2024-05-09T06:36:38.092253+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8853}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8853': {'gid': 8853, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.1:6835/3796896332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 3796896332}, {'type': 'v1', 'addr': '172.21.15.1:6835', 'nonce': 3796896332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8853, 'qdb_cluster': [8853]} 2024-05-09T06:36:41.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:36:41.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.397+0000 7f4573b42640 1 Processor -- start 2024-05-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.397+0000 7f4573b42640 1 -- start start 2024-05-09T06:36:41.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f4573b42640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c161950 0x7f456c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f4573b42640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f456c0d0f80 0x7f456c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f4573b42640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c0b33c0 con 0x7f456c0d4940 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f4573b42640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f456c0d4dd0 con 0x7f456c161950 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f4573b42640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f456c0b3660 con 0x7f456c0d0f80 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c161950 0x7f456c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c161950 0x7f456c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55468/0 (socket says 172.21.15.1:55468) 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 -- 172.21.15.1:0/2956592764 learned_addr learned my addr 172.21.15.1:0/2956592764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 -- 172.21.15.1:0/2956592764 >> v2:172.21.15.6:3300/0 conn(0x7f456c0d0f80 msgr2=0x7f456c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 --2- 172.21.15.1:0/2956592764 >> v2:172.21.15.6:3300/0 conn(0x7f456c0d0f80 0x7f456c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 -- 172.21.15.1:0/2956592764 >> v1:172.21.15.6:6789/0 conn(0x7f456c0d4940 legacy=0x7f456c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.398+0000 7f45718de640 1 -- 172.21.15.1:0/2956592764 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c0b3940 con 0x7f456c161950 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.399+0000 7f45718de640 1 --2- 172.21.15.1:0/2956592764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c161950 0x7f456c0d2400 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f456c06b7f0 tx=0x7f456802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7eef1ad30c112a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.399+0000 7f45708dc640 1 -- 172.21.15.1:0/2956592764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568037070 con 0x7f456c161950 2024-05-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.399+0000 7f45708dc640 1 -- 172.21.15.1:0/2956592764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4568032440 con 0x7f456c161950 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.399+0000 7f45708dc640 1 -- 172.21.15.1:0/2956592764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568032810 con 0x7f456c161950 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 -- 172.21.15.1:0/2956592764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c161950 msgr2=0x7f456c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 --2- 172.21.15.1:0/2956592764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c161950 0x7f456c0d2400 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f456c06b7f0 tx=0x7f456802a210 comp rx=0 tx=0).stop 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 -- 172.21.15.1:0/2956592764 shutdown_connections 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 --2- 172.21.15.1:0/2956592764 >> v2:172.21.15.6:3300/0 conn(0x7f456c0d0f80 0x7f456c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 --2- 172.21.15.1:0/2956592764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c161950 0x7f456c0d2400 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 -- 172.21.15.1:0/2956592764 >> 172.21.15.1:0/2956592764 conn(0x7f456c0d3e20 msgr2=0x7f456c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 -- 172.21.15.1:0/2956592764 shutdown_connections 2024-05-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.400+0000 7f4573b42640 1 -- 172.21.15.1:0/2956592764 wait complete. 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.401+0000 7f4573b42640 1 Processor -- start 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.401+0000 7f4573b42640 1 -- start start 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c161950 0x7f456c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 0x7f456c1f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f456c0b52d0 con 0x7f456c0d0f80 2024-05-09T06:36:41.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f456c0b5150 con 0x7f456c06cfb0 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f4573b42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f456c0b5450 con 0x7f456c161950 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55474/0 (socket says 172.21.15.1:55474) 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 -- 172.21.15.1:0/1567803285 learned_addr learned my addr 172.21.15.1:0/1567803285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45720df640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 0x7f456c1f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45710dd640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c161950 0x7f456c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 -- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c161950 msgr2=0x7f456c06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c161950 0x7f456c06ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 -- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 msgr2=0x7f456c1f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 0x7f456c1f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45718de640 1 -- 172.21.15.1:0/1567803285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c0b6b90 con 0x7f456c0d0f80 2024-05-09T06:36:41.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.402+0000 7f45720df640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 0x7f456c1f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f45718de640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c00 tx=0x7f45680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f45627fc640 1 -- 172.21.15.1:0/1567803285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568041070 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f45627fc640 1 -- 172.21.15.1:0/1567803285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456803c890 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1f47d0 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f45627fc640 1 -- 172.21.15.1:0/1567803285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568037040 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.403+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1f3f70 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.404+0000 7f45627fc640 1 -- 172.21.15.1:0/1567803285 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4568044070 con 0x7f456c0d0f80 2024-05-09T06:36:41.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.405+0000 7f45627fc640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 0x7f455006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.405+0000 7f45627fc640 1 -- 172.21.15.1:0/1567803285 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f45680abda0 con 0x7f456c0d0f80 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.405+0000 7f45710dd640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 0x7f455006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.405+0000 7f45710dd640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 0x7f455006e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.406+0000 7f4576853740 1 Processor -- start 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.406+0000 7f4576853740 1 -- start start 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622f2f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622ee7350 0x556622f2d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556622e4a4d0 con 0x556622f11670 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556622df23e0 con 0x556622ee7350 2024-05-09T06:36:41.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556622e46490 con 0x556622f2ef10 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55482/0 (socket says 172.21.15.1:55482) 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 -- 172.21.15.1:0/750597394 learned_addr learned my addr 172.21.15.1:0/750597394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45720df640 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622ee7350 0x556622f2d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45710dd640 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622f2f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 -- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 msgr2=0x556622f2f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622f2f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 -- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622ee7350 msgr2=0x556622f2d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622ee7350 0x556622f2d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 -- 172.21.15.1:0/750597394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556622b5e2d0 con 0x556622f11670 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f45718de640 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4568079f60 tx=0x7f4568079ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1808150015156c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f454f7fe640 1 -- 172.21.15.1:0/750597394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456804b040 con 0x556622f11670 2024-05-09T06:36:41.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f454f7fe640 1 -- 172.21.15.1:0/750597394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456807e380 con 0x556622f11670 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.407+0000 7f454f7fe640 1 -- 172.21.15.1:0/750597394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568082bb0 con 0x556622f11670 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 -- 172.21.15.1:0/750597394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 msgr2=0x556622f11a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4568079f60 tx=0x7f4568079ed0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 -- 172.21.15.1:0/750597394 shutdown_connections 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622ee7350 0x556622f2d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622f2f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 --2- 172.21.15.1:0/750597394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f11670 0x556622f11a50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 -- 172.21.15.1:0/750597394 >> 172.21.15.1:0/750597394 conn(0x556622ee3c80 msgr2=0x556622f2c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 -- 172.21.15.1:0/750597394 shutdown_connections 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.408+0000 7f4576853740 1 -- 172.21.15.1:0/750597394 wait complete. 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.409+0000 7f4576853740 1 -- start start 2024-05-09T06:36:41.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.409+0000 7f4576853740 1 Processor -- start 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622efefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622efde70 0x556622f14880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556622e4a4d0 con 0x556622f2ef10 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556622e058e0 con 0x556622f11670 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556622ee80f0 con 0x556622efde70 2024-05-09T06:36:41.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 10 client.0 _ll_register_callbacks cb 0x556622dd29c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55490/0 (socket says 172.21.15.1:55490) 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 -- 172.21.15.1:0/4244537900 learned_addr learned my addr 172.21.15.1:0/4244537900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45720df640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622efde70 0x556622f14880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45718de640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622efefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 -- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622efde70 msgr2=0x556622f14880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622efde70 0x556622f14880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 -- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 msgr2=0x556622efefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622efefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 -- 172.21.15.1:0/4244537900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556622c286b0 con 0x556622f2ef10 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f45710dd640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e810 tx=0x7f455c000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db82c993610858b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f454cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c027060 con 0x556622f2ef10 2024-05-09T06:36:41.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f455c010040 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.410+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456807e380 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f455c003b70 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 10 client.9000 fetch_fsmap learned FSMap version 82 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x556622f01ac0 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456807e380 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f455c013420 con 0x556622f2ef10 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 10 client.9000 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 10 client.9000 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 20 client.9000 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622f007f0 0x556622f02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 4 client.9000 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556622e57c00 con 0x556622f007f0 2024-05-09T06:36:41.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f45718de640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622f007f0 0x556622f02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f45718de640 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622f007f0 0x556622f02940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f978e368f9544962 server_cookie=180f70e05b39dcba in_seq=0 out_seq=0 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.411+0000 7f454cff9640 10 client.9000 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f454cff9640 1 -- 172.21.15.1:0/4244537900 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556622e57c00 con 0x556622f007f0 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f454cff9640 10 client.9000 handle_command_reply: tid=0 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 client.9000 shutdown 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 msgr2=0x556622f14340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e810 tx=0x7f455c000fb0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 shutdown_connections 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622f007f0 0x556622f02940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622efde70 0x556622f14880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622f2ef10 0x556622f14340 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f454cff9640 0 client.9000 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 --2- 172.21.15.1:0/4244537900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622efefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 >> 172.21.15.1:0/4244537900 conn(0x556622ee7350 msgr2=0x556622ee65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 shutdown_connections 2024-05-09T06:36:41.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.412+0000 7f4576853740 1 -- 172.21.15.1:0/4244537900 wait complete. 2024-05-09T06:36:41.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.413+0000 7f4576853740 20 client.9000 trim_cache size 0 max 16384 2024-05-09T06:36:41.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.431+0000 7f4576853740 1 Processor -- start 2024-05-09T06:36:41.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.431+0000 7f4576853740 1 -- start start 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622ee7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622ee7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1ffc40 con 0x556622efde70 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f456c0b8850 con 0x556622f11670 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f456c1ff5f0 con 0x556622f2ef10 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55496/0 (socket says 172.21.15.1:55496) 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 -- 172.21.15.1:0/3073620031 learned_addr learned my addr 172.21.15.1:0/3073620031 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 -- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 msgr2=0x556622ee7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45710dd640 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622ee7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622ee7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 -- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 msgr2=0x556622ee7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622ee7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 -- 172.21.15.1:0/3073620031 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556622f14c70 con 0x556622efde70 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f45718de640 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4568079f60 tx=0x7f456807c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395495255ac60b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f454effd640 1 -- 172.21.15.1:0/3073620031 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456804b040 con 0x556622efde70 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f454effd640 1 -- 172.21.15.1:0/3073620031 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456807e380 con 0x556622efde70 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.432+0000 7f454effd640 1 -- 172.21.15.1:0/3073620031 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568082bb0 con 0x556622efde70 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- 172.21.15.1:0/3073620031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 msgr2=0x556622ee6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4568079f60 tx=0x7f456807c000 comp rx=0 tx=0).stop 2024-05-09T06:36:41.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- 172.21.15.1:0/3073620031 shutdown_connections 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f2ef10 0x556622ee7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f11670 0x556622ee7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 --2- 172.21.15.1:0/3073620031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622ee6650 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- 172.21.15.1:0/3073620031 >> 172.21.15.1:0/3073620031 conn(0x556622ee7350 msgr2=0x556622f14e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- 172.21.15.1:0/3073620031 shutdown_connections 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- 172.21.15.1:0/3073620031 wait complete. 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.433+0000 7f4576853740 1 -- start start 2024-05-09T06:36:41.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 Processor -- start 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f11670 0x556622f3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f2ef10 0x556622fa57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556622f00670 con 0x556622efde70 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x556622f033c0 con 0x556622f2ef10 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.434+0000 7f4576853740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x556622f00cc0 con 0x556622f11670 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f4576853740 10 client.0 _ll_register_callbacks cb 0x7f455c005300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55502/0 (socket says 172.21.15.1:55502) 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 -- 172.21.15.1:0/634003927 learned_addr learned my addr 172.21.15.1:0/634003927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45720df640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f2ef10 0x556622fa57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45710dd640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f11670 0x556622f3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 -- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f11670 msgr2=0x556622f3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f11670 0x556622f3bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 -- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f2ef10 msgr2=0x556622fa57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f2ef10 0x556622fa57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 -- 172.21.15.1:0/634003927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556622f2f9e0 con 0x556622efde70 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f45718de640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4568076430 tx=0x7f4568007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8640b0f1ebb243 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f454cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456804b070 con 0x556622efde70 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f45680782d0 con 0x556622efde70 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456807e380 con 0x556622efde70 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4568082bb0 con 0x556622efde70 2024-05-09T06:36:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f4576853740 10 client.9006 fetch_fsmap learned FSMap version 82 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x556622f22bf0 con 0x556622efde70 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.435+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456807e380 con 0x556622efde70 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f4568076640 con 0x556622efde70 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 10 client.9006 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 10 client.9006 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 20 client.9006 populate_metadata read hostname 'smithi001' 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622ee7730 0x556622ee7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 4 client.9006 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "status"}) -- 0x556622e51f50 con 0x556622ee7730 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f45710dd640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622ee7730 0x556622ee7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f454cff9640 10 client.9006 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:36:41.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f45710dd640 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622ee7730 0x556622ee7b10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b78452fbc2fe75 server_cookie=67f2f5efe0165899 in_seq=0 out_seq=0 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f454cff9640 1 -- 172.21.15.1:0/634003927 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556622e51f50 con 0x556622ee7730 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f454cff9640 10 client.9006 handle_command_reply: tid=0 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 client.9006 shutdown 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 msgr2=0x556622f3b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4568076430 tx=0x7f4568007000 comp rx=0 tx=0).stop 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 shutdown_connections 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x556622ee7730 0x556622ee7b10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556622f2ef10 0x556622fa57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x556622f11670 0x556622f3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f454cff9640 0 client.9006 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 --2- 172.21.15.1:0/634003927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556622efde70 0x556622f3b270 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 >> 172.21.15.1:0/634003927 conn(0x556622ee7350 msgr2=0x556622f14e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.436+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 shutdown_connections 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4576853740 1 -- 172.21.15.1:0/634003927 wait complete. 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4576853740 20 client.9006 trim_cache size 0 max 16384 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 8853, 2024-05-09T06:36:41.427 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 5.717827507, 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 82, 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 106, 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 106, 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 10.716851438999999 2024-05-09T06:36:41.428 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 msgr2=0x7f455006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 0x7f455006e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 msgr2=0x7f456c1593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c00 tx=0x7f45680046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 shutdown_connections 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455006be30 0x7f455006e2f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c06cfb0 0x7f456c1f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c161950 0x7f456c06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 --2- 172.21.15.1:0/1567803285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c0d0f80 0x7f456c1593a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 >> 172.21.15.1:0/1567803285 conn(0x7f456c0d3e20 msgr2=0x7f456c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 shutdown_connections 2024-05-09T06:36:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.437+0000 7f4573b42640 1 -- 172.21.15.1:0/1567803285 wait complete. 2024-05-09T06:36:41.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:41.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.555+0000 7efe9f3f9640 1 Processor -- start 2024-05-09T06:36:41.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.555+0000 7efe9f3f9640 1 -- start start 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe9806e6d0 0x7efe9806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe980cf180 0x7efe980cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe9815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efe980ce0e0 con 0x7efe9806e6d0 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efe980ce3c0 con 0x7efe980cf180 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9f3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe980ce6a0 con 0x7efe980cdab0 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe9806e6d0 0x7efe9806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe9806e6d0 0x7efe9806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55508/0 (socket says 172.21.15.1:55508) 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 -- 172.21.15.1:0/2932329296 learned_addr learned my addr 172.21.15.1:0/2932329296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9c994640 1 --2- 172.21.15.1:0/2932329296 >> v2:172.21.15.6:3300/0 conn(0x7efe980cf180 0x7efe980cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 -- 172.21.15.1:0/2932329296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 msgr2=0x7efe9815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 --2- 172.21.15.1:0/2932329296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe9815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 -- 172.21.15.1:0/2932329296 >> v2:172.21.15.6:3300/0 conn(0x7efe980cf180 msgr2=0x7efe980cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 --2- 172.21.15.1:0/2932329296 >> v2:172.21.15.6:3300/0 conn(0x7efe980cf180 0x7efe980cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.556+0000 7efe9d195640 1 -- 172.21.15.1:0/2932329296 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe980ce980 con 0x7efe9806e6d0 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.557+0000 7efe9d195640 1 --2- 172.21.15.1:0/2932329296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9806eab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7efe980b27c0 tx=0x7efe9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fd4a7db0570b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.557+0000 7efe8ffff640 1 -- 172.21.15.1:0/2932329296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94037070 con 0x7efe9806e6d0 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.557+0000 7efe8ffff640 1 -- 172.21.15.1:0/2932329296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe94032440 con 0x7efe9806e6d0 2024-05-09T06:36:41.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.557+0000 7efe8ffff640 1 -- 172.21.15.1:0/2932329296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94032810 con 0x7efe9806e6d0 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2932329296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 msgr2=0x7efe9806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2932329296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9806eab0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7efe980b27c0 tx=0x7efe9402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2932329296 shutdown_connections 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2932329296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe9815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2932329296 >> v2:172.21.15.6:3300/0 conn(0x7efe980cf180 0x7efe980cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2932329296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9806eab0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2932329296 >> 172.21.15.1:0/2932329296 conn(0x7efe980cc4b0 msgr2=0x7efe980cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2932329296 shutdown_connections 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.558+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2932329296 wait complete. 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.559+0000 7efe9f3f9640 1 Processor -- start 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.559+0000 7efe9f3f9640 1 -- start start 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe981514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe980cf180 0x7efe9815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe98070ed0 con 0x7efe9806e6d0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efe9806eff0 con 0x7efe980cf180 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9f3f9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe981447c0 con 0x7efe980cdab0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55514/0 (socket says 172.21.15.1:55514) 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 -- 172.21.15.1:0/2521911200 learned_addr learned my addr 172.21.15.1:0/2521911200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9c994640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe981514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d996640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe980cf180 0x7efe9815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 -- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 msgr2=0x7efe981514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe981514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 -- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe980cf180 msgr2=0x7efe9815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe980cf180 0x7efe9815c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 -- 172.21.15.1:0/2521911200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9815c740 con 0x7efe9806e6d0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.560+0000 7efe9d195640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7efe9402a220 tx=0x7efe940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.561+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94037040 con 0x7efe9806e6d0 2024-05-09T06:36:41.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.561+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe94042070 con 0x7efe9806e6d0 2024-05-09T06:36:41.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.561+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9815c9e0 con 0x7efe9806e6d0 2024-05-09T06:36:41.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.561+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94030bb0 con 0x7efe9806e6d0 2024-05-09T06:36:41.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.561+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9815cfa0 con 0x7efe9806e6d0 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.562+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe980b2ee0 con 0x7efe9806e6d0 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.562+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe94047460 con 0x7efe9806e6d0 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.563+0000 7efe8dffb640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 0x7efe7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.563+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7efe940aba90 con 0x7efe9806e6d0 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.563+0000 7efe9c994640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 0x7efe7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.563+0000 7efe9c994640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 0x7efe7c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe9815e8d0 tx=0x7efe88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.567+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe980b2ee0 con 0x7efe9806e6d0 2024-05-09T06:36:41.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.787+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe980752b0 con 0x7efe9806e6d0 2024-05-09T06:36:41.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.787+0000 7efe8dffb640 1 -- 172.21.15.1:0/2521911200 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7efe94079cf0 con 0x7efe9806e6d0 2024-05-09T06:36:41.767 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:41.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 msgr2=0x7efe7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 0x7efe7c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe9815e8d0 tx=0x7efe88008040 comp rx=0 tx=0).stop 2024-05-09T06:36:41.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 msgr2=0x7efe9815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7efe9402a220 tx=0x7efe940046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 shutdown_connections 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe7c06be30 0x7efe7c06e2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe980cf180 0x7efe9815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe980cdab0 0x7efe981514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 --2- 172.21.15.1:0/2521911200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe9806e6d0 0x7efe9815f470 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 >> 172.21.15.1:0/2521911200 conn(0x7efe980cc4b0 msgr2=0x7efe981408e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 shutdown_connections 2024-05-09T06:36:41.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:41.789+0000 7efe9f3f9640 1 -- 172.21.15.1:0/2521911200 wait complete. 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:41.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:41.781 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:41.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:41.781 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42140sec preferred_lft 42140sec 2024-05-09T06:36:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:41.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:41.801 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:41.863 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:36:41.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:41.865 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:36:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:36:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:36:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:36:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:41.922 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:41.923 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:36:41.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:36:41.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:36:41.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:36:41.923 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:36:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:42.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.052 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:36:42.069 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:36:42.069 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:36:42.069 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:42.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:36:42.135 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:36:42.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:42.136 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:42.191 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:42.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:42.191 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:42.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:36:42.314 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:36:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:42.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.375 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:36:42.428 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:36:42.429 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:36:42.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:36:42.497 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:36:42.554 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:36:42.555 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:42.556 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:42.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.558 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:36:42.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:36:42.577+0000 7fce0c64a480 -1 init, newargv = 0x55efac2561f0 newargc=15 2024-05-09T06:36:42.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[74870]: starting ceph client 2024-05-09T06:36:42.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[74870]: starting fuse 2024-05-09T06:36:42.577 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:36:42.577 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:36:42.577 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:36:42.578 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:36:42.579 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:42.699 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.74870.asok 2024-05-09T06:36:42.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:42.710 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74870.asok 2024-05-09T06:36:42.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74870.asok status 2024-05-09T06:36:42.849 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:36:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "74870", 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9018, 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 9018 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3640580001 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3640580001 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.9018 192.168.0.1:0/3640580001", 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/3640580001", 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 82, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 106, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:36:42.852 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:36:42.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.858 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:36:42.858 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:42.875 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:36:42.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:42.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:42.942 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:42.943 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.999 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-09T06:36:43.104 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records in 2024-05-09T06:36:43.104 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records out 2024-05-09T06:36:43.104 INFO:teuthology.orchestra.run.smithi001.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0433143 s, 145 MB/s 2024-05-09T06:36:43.104 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:43.104 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:43.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:43.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.285+0000 7f3cc5cda640 1 Processor -- start 2024-05-09T06:36:43.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.285+0000 7f3cc5cda640 1 -- start start 2024-05-09T06:36:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cc5cda640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3cc006e6d0 0x7f3cc006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cc5cda640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc00ce0e0 con 0x7f3cc00b3300 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cc5cda640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc00ce3c0 con 0x7f3cc00cf180 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cc5cda640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc00ce6a0 con 0x7f3cc006e6d0 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cbffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3cc00b3300 0x7f3cc00b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57844/0 (socket says 172.21.15.1:57844) 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cbffff640 1 -- 172.21.15.1:0/1797543157 learned_addr learned my addr 172.21.15.1:0/1797543157 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.286+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621174328 0 0) 0x7f3cc00ce0e0 con 0x7f3cc00b3300 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8003520 con 0x7f3cc00b3300 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1797543157 >> v2:172.21.15.6:3300/0 conn(0x7f3cc006e6d0 0x7f3cc006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3474507362 0 0) 0x7f3ca8003520 con 0x7f3cc00b3300 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 >> v2:172.21.15.6:3300/0 conn(0x7f3cc006e6d0 msgr2=0x7f3cc006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1797543157 >> v2:172.21.15.6:3300/0 conn(0x7f3cc006e6d0 0x7f3cc006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 >> v1:172.21.15.6:6789/0 conn(0x7f3cc00cf180 legacy=0x7f3cc00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc00ce980 con 0x7f3cc00b3300 2024-05-09T06:36:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3cb80019f0 con 0x7f3cc00b3300 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 >> v1:172.21.15.1:6789/0 conn(0x7f3cc00b3300 legacy=0x7f3cc00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00ce6a0 con 0x7f3ca80039b0 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca800c060 con 0x7f3ca8004800 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca800c260 con 0x7f3ca80051c0 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbffff640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80051c0 msgr2=0x7f3ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca8004800 msgr2=0x7f3ca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbf7fe640 1 -- 172.21.15.1:0/1797543157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca8003520 con 0x7f3ca80039b0 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.287+0000 7f3cbffff640 1 -- 172.21.15.1:0/1797543157 reap_dead start 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.288+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009ac0 tx=0x7f3ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd43421a4047cc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.288+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0036070 con 0x7f3ca80039b0 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.288+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb0031400 con 0x7f3ca80039b0 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.288+0000 7f3cbe7fc640 1 -- 172.21.15.1:0/1797543157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009690 con 0x7f3ca80039b0 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 -- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 msgr2=0x7f3ca8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009ac0 tx=0x7f3ca8005830 comp rx=0 tx=0).stop 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 -- 172.21.15.1:0/1797543157 shutdown_connections 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/1797543157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 -- 172.21.15.1:0/1797543157 >> 172.21.15.1:0/1797543157 conn(0x7f3cc00cc4b0 msgr2=0x7f3cc00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 -- 172.21.15.1:0/1797543157 shutdown_connections 2024-05-09T06:36:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.289+0000 7f3cc5cda640 1 -- 172.21.15.1:0/1797543157 wait complete. 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.290+0000 7f3cc5cda640 1 Processor -- start 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.290+0000 7f3cc5cda640 1 -- start start 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca80039b0 0x7f3cc01515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc013d0d0 0x7f3cc015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01447c0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0070ed0 con 0x7f3ca80039b0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cc5cda640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc0144940 con 0x7f3cc013d0d0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55528/0 (socket says 172.21.15.1:55528) 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 -- 172.21.15.1:0/80844679 learned_addr learned my addr 172.21.15.1:0/80844679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca80039b0 0x7f3cc01515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbffff640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc013d0d0 0x7f3cc015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 -- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc013d0d0 msgr2=0x7f3cc015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc013d0d0 0x7f3cc015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 -- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca80039b0 msgr2=0x7f3cc01515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca80039b0 0x7f3cc01515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.291+0000 7f3cbeffd640 1 -- 172.21.15.1:0/80844679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc01374e0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3cbeffd640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400f7f0 tx=0x7f3cb400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40050d0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb4018070 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc013c030 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4005490 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.292+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc013c5f0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.293+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc00ceea0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.293+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cb400d6d0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.294+0000 7f3ca7fff640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.294+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f3cb408ab80 con 0x7f3cc015a7a0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.294+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.294+0000 7f3cbf7fe640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 0x7f3cac06e320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0033000 tx=0x7f3cb0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.298+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cc00ceea0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.481+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cc00752b0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.482+0000 7f3ca7fff640 1 -- 172.21.15.1:0/80844679 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16752 (secure 0 0 0) 0x7f3cb4058de0 con 0x7f3cc015a7a0 2024-05-09T06:36:43.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:43.465 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":106,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:36.717091+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:43.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.484+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 msgr2=0x7f3cac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.484+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 0x7f3cac06e320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0033000 tx=0x7f3cb0008030 comp rx=0 tx=0).stop 2024-05-09T06:36:43.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.484+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 msgr2=0x7f3cc0136fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.484+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400f7f0 tx=0x7f3cb400fcb0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 shutdown_connections 2024-05-09T06:36:43.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3cc013d0d0 0x7f3cc015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3cc015a7a0 0x7f3cc0136fa0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 --2- 172.21.15.1:0/80844679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ca80039b0 0x7f3cc01515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 >> 172.21.15.1:0/80844679 conn(0x7f3cc00cc4b0 msgr2=0x7f3cc0071a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 shutdown_connections 2024-05-09T06:36:43.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.485+0000 7f3cc5cda640 1 -- 172.21.15.1:0/80844679 wait complete. 2024-05-09T06:36:43.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:33.397054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:34.058197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:43.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:43.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.605+0000 7ff36b763640 1 Processor -- start 2024-05-09T06:36:43.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.605+0000 7ff36b763640 1 -- start start 2024-05-09T06:36:43.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff36b763640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 0x7ff36415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff36b763640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff3640d1c70 0x7ff3640d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff36b763640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff364161fe0 con 0x7ff3640b56d0 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff36b763640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff36415f580 con 0x7ff3640d1c70 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff36b763640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff3641622c0 con 0x7ff36415e190 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff368cfe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff3640d1c70 0x7ff3640d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff368cfe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff3640d1c70 0x7ff3640d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55542/0 (socket says 172.21.15.1:55542) 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.606+0000 7ff368cfe640 1 -- 172.21.15.1:0/236473936 learned_addr learned my addr 172.21.15.1:0/236473936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff368cfe640 1 -- 172.21.15.1:0/236473936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 msgr2=0x7ff36415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff3694ff640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 0x7ff36415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff368cfe640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 0x7ff36415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff368cfe640 1 -- 172.21.15.1:0/236473936 >> v1:172.21.15.6:6789/0 conn(0x7ff3640b56d0 legacy=0x7ff3640b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff368cfe640 1 -- 172.21.15.1:0/236473936 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3641625a0 con 0x7ff3640d1c70 2024-05-09T06:36:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff3694ff640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 0x7ff36415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff368cfe640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640d3ee0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff354009870 tx=0x7ff35402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b483d94e265280e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff35bfff640 1 -- 172.21.15.1:0/236473936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354037070 con 0x7ff3640d1c70 2024-05-09T06:36:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.607+0000 7ff35bfff640 1 -- 172.21.15.1:0/236473936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff354032440 con 0x7ff3640d1c70 2024-05-09T06:36:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff35bfff640 1 -- 172.21.15.1:0/236473936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354032810 con 0x7ff3640d1c70 2024-05-09T06:36:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 -- 172.21.15.1:0/236473936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 msgr2=0x7ff3640d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640d3ee0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff354009870 tx=0x7ff35402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 -- 172.21.15.1:0/236473936 shutdown_connections 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640d3ee0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 --2- 172.21.15.1:0/236473936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff36415e190 0x7ff36415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 -- 172.21.15.1:0/236473936 >> 172.21.15.1:0/236473936 conn(0x7ff364073340 msgr2=0x7ff36409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 -- 172.21.15.1:0/236473936 shutdown_connections 2024-05-09T06:36:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.608+0000 7ff36b763640 1 -- 172.21.15.1:0/236473936 wait complete. 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.609+0000 7ff36b763640 1 Processor -- start 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.609+0000 7ff36b763640 1 -- start start 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 0x7ff3640b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 0x7ff3641523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff3640d4420 con 0x7ff3640d1c70 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff364161c60 con 0x7ff36415e190 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff36b763640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff3640d45a0 con 0x7ff3640b56d0 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff368cfe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff368cfe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55546/0 (socket says 172.21.15.1:55546) 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.610+0000 7ff368cfe640 1 -- 172.21.15.1:0/2198020726 learned_addr learned my addr 172.21.15.1:0/2198020726 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff3694ff640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 0x7ff3640b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff369d00640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 0x7ff3641523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 -- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 msgr2=0x7ff3640b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 0x7ff3640b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 -- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 msgr2=0x7ff3641523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 0x7ff3641523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 -- 172.21.15.1:0/2198020726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640cba90 con 0x7ff3640d1c70 2024-05-09T06:36:43.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff3694ff640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 0x7ff3640b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff369d00640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 0x7ff3641523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff368cfe640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff354009870 tx=0x7ff3540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354041020 con 0x7ff3640d1c70 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff354030970 con 0x7ff3640d1c70 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354037040 con 0x7ff3640d1c70 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3640cb4d0 con 0x7ff3640d1c70 2024-05-09T06:36:43.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.611+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3640b0d10 con 0x7ff3640d1c70 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.612+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3640d3550 con 0x7ff3640d1c70 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.613+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff354044020 con 0x7ff3640d1c70 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.613+0000 7ff359ffb640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 0x7ff34806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.613+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7ff3540aba00 con 0x7ff3640d1c70 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.613+0000 7ff3694ff640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 0x7ff34806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.614+0000 7ff3694ff640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 0x7ff34806e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.618+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3640d3550 con 0x7ff3640d1c70 2024-05-09T06:36:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.803+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3640752b0 con 0x7ff3640d1c70 2024-05-09T06:36:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.804+0000 7ff359ffb640 1 -- 172.21.15.1:0/2198020726 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7ff35404b030 con 0x7ff3640d1c70 2024-05-09T06:36:43.784 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 msgr2=0x7ff34806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 0x7ff34806e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360008040 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 msgr2=0x7ff3640b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff354009870 tx=0x7ff3540046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 shutdown_connections 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff34806be30 0x7ff34806e2f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff36415e190 0x7ff3641523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3640d1c70 0x7ff3640b6ed0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 --2- 172.21.15.1:0/2198020726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff3640b56d0 0x7ff3640b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 >> 172.21.15.1:0/2198020726 conn(0x7ff364073340 msgr2=0x7ff36406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 shutdown_connections 2024-05-09T06:36:43.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.806+0000 7ff36b763640 1 -- 172.21.15.1:0/2198020726 wait complete. 2024-05-09T06:36:43.797 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:43.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.930+0000 7f4e4e47d640 1 Processor -- start 2024-05-09T06:36:43.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.930+0000 7f4e4e47d640 1 -- start start 2024-05-09T06:36:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4e47d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e480b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4e47d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4815e900 con 0x7f4e480cc8e0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4e47d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4815ebe0 con 0x7f4e4815e390 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4e47d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e480b1b90 con 0x7f4e480b11f0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4ca1a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4e480cc8e0 0x7f4e480ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57850/0 (socket says 172.21.15.1:57850) 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e4ca1a640 1 -- 172.21.15.1:0/1540310002 learned_addr learned my addr 172.21.15.1:0/1540310002 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e477fe640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e480b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.931+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78052786 0 0) 0x7f4e4815e900 con 0x7f4e480cc8e0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e30003520 con 0x7f4e480cc8e0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89053120 0 0) 0x7f4e4815ebe0 con 0x7f4e4815e390 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e4815e900 con 0x7f4e4815e390 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3507402379 0 0) 0x7f4e30003520 con 0x7f4e480cc8e0 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 msgr2=0x7f4e480b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e480b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 >> v1:172.21.15.6:6789/0 conn(0x7f4e4815e390 legacy=0x7f4e480cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4815eec0 con 0x7f4e480cc8e0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f4e400019f0 con 0x7f4e480cc8e0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 >> v1:172.21.15.1:6789/0 conn(0x7f4e480cc8e0 legacy=0x7f4e480ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e30003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004120 0x7f4e30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e30004d40 0x7f4e300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3000bad0 con 0x7f4e30004d40 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3000bc80 con 0x7f4e30004120 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3000be80 con 0x7f4e480b11f0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e30004d40 0x7f4e300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e477fe640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004120 0x7f4e30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e47fff640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e30003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 -- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 msgr2=0x7f4e30003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b11f0 0x7f4e30003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 -- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004120 msgr2=0x7f4e30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004120 0x7f4e30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.932+0000 7f4e4ca1a640 1 -- 172.21.15.1:0/1540310002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e30003520 con 0x7f4e30004d40 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.933+0000 7f4e4ca1a640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e30004d40 0x7f4e300051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000a730 tx=0x7f4e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eec821f16f533fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.933+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000a350 con 0x7f4e30004d40 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.933+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4002f070 con 0x7f4e30004d40 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.933+0000 7f4e46ffd640 1 -- 172.21.15.1:0/1540310002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4002bbd0 con 0x7f4e30004d40 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1540310002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e30004d40 msgr2=0x7f4e300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1540310002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e30004d40 0x7f4e300051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000a730 tx=0x7f4e30005830 comp rx=0 tx=0).stop 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4ca1a640 1 -- 172.21.15.1:0/1540310002 reap_dead start 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1540310002 shutdown_connections 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1540310002 >> 172.21.15.1:0/1540310002 conn(0x7f4e480b1da0 msgr2=0x7f4e480b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1540310002 shutdown_connections 2024-05-09T06:36:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.934+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1540310002 wait complete. 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.935+0000 7f4e4e47d640 1 Processor -- start 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.935+0000 7f4e4e47d640 1 -- start start 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004d40 0x7f4e48160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b5750 0x7f4e48149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48162ba0 con 0x7f4e480b52d0 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4815f6c0 con 0x7f4e30004d40 2024-05-09T06:36:43.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4e47d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e480cffb0 con 0x7f4e480b5750 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55564/0 (socket says 172.21.15.1:55564) 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 -- 172.21.15.1:0/1686610157 learned_addr learned my addr 172.21.15.1:0/1686610157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e47fff640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004d40 0x7f4e48160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e4ca1a640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b5750 0x7f4e48149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 -- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b5750 msgr2=0x7f4e48149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b5750 0x7f4e48149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 -- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004d40 msgr2=0x7f4e48160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004d40 0x7f4e48160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.936+0000 7f4e477fe640 1 -- 172.21.15.1:0/1686610157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e480ae6e0 con 0x7f4e480b52d0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e477fe640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0119a0 tx=0x7f4e3c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00faf0 con 0x7f4e480b52d0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00fc90 con 0x7f4e480b52d0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e480ae9c0 con 0x7f4e480b52d0 2024-05-09T06:36:43.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c014440 con 0x7f4e480b52d0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.937+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e480aef80 con 0x7f4e480b52d0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.938+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e480752b0 con 0x7f4e480b52d0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.938+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e3c012040 con 0x7f4e480b52d0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.939+0000 7f4e44ff9640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 0x7f4e3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.939+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f4e3c08b550 con 0x7f4e480b52d0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.939+0000 7f4e47fff640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 0x7f4e3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.939+0000 7f4e47fff640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 0x7f4e3406e320 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:43.943+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e3c019030 con 0x7f4e480b52d0 2024-05-09T06:36:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.124+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e4815d6d0 con 0x7f4e480b52d0 2024-05-09T06:36:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.125+0000 7f4e44ff9640 1 -- 172.21.15.1:0/1686610157 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f4e3c059e00 con 0x7f4e480b52d0 2024-05-09T06:36:44.105 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 msgr2=0x7f4e3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 0x7f4e3406e320 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e38009040 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 msgr2=0x7f4e48149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0119a0 tx=0x7f4e3c011e60 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 shutdown_connections 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e3406be60 0x7f4e3406e320 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e480b5750 0x7f4e48149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e480b52d0 0x7f4e48149a90 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 --2- 172.21.15.1:0/1686610157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e30004d40 0x7f4e48160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 >> 172.21.15.1:0/1686610157 conn(0x7f4e480b1da0 msgr2=0x7f4e480cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 shutdown_connections 2024-05-09T06:36:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.127+0000 7f4e4e47d640 1 -- 172.21.15.1:0/1686610157 wait complete. 2024-05-09T06:36:44.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:44.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.244+0000 7f51b51a9640 1 Processor -- start 2024-05-09T06:36:44.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.244+0000 7f51b51a9640 1 -- start start 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51b51a9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f51b00b1680 0x7f51b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51b51a9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b015e900 con 0x7f51b00b11f0 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51b51a9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b015ebe0 con 0x7f51b015e390 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51b51a9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f51b015eec0 con 0x7f51b00b1680 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51af59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f51b00b1680 0x7f51b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51ae59c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f51b00b11f0 0x7f51b00cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57860/0 (socket says 172.21.15.1:57860) 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51ae59c640 1 -- 172.21.15.1:0/4206542995 learned_addr learned my addr 172.21.15.1:0/4206542995 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:44.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51af59e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f51b00b1680 0x7f51b00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55574/0 (socket says 172.21.15.1:55574) 2024-05-09T06:36:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51af59e640 1 -- 172.21.15.1:0/4206542995 >> v1:172.21.15.6:6789/0 conn(0x7f51b015e390 legacy=0x7f51b00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51af59e640 1 -- 172.21.15.1:0/4206542995 >> v1:172.21.15.1:6789/0 conn(0x7f51b00b11f0 legacy=0x7f51b00cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51add9b640 1 -- 172.21.15.1:0/4206542995 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1821709967 0 0) 0x7f51b015e900 con 0x7f51b00b11f0 2024-05-09T06:36:44.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.245+0000 7f51af59e640 1 -- 172.21.15.1:0/4206542995 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b015f1a0 con 0x7f51b00b1680 2024-05-09T06:36:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.246+0000 7f51af59e640 1 --2- 172.21.15.1:0/4206542995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b00b1680 0x7f51b00ccff0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009d90 tx=0x7f51a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e526bc3a8f881599 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.246+0000 7f51add9b640 1 -- 172.21.15.1:0/4206542995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007740 con 0x7f51b00b1680 2024-05-09T06:36:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.246+0000 7f51add9b640 1 -- 172.21.15.1:0/4206542995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a80078e0 con 0x7f51b00b1680 2024-05-09T06:36:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.246+0000 7f51add9b640 1 -- 172.21.15.1:0/4206542995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8007cb0 con 0x7f51b00b1680 2024-05-09T06:36:44.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 -- 172.21.15.1:0/4206542995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b00b1680 msgr2=0x7f51b00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 --2- 172.21.15.1:0/4206542995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b00b1680 0x7f51b00ccff0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009d90 tx=0x7f51a802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 -- 172.21.15.1:0/4206542995 shutdown_connections 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 --2- 172.21.15.1:0/4206542995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b00b1680 0x7f51b00ccff0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 -- 172.21.15.1:0/4206542995 >> 172.21.15.1:0/4206542995 conn(0x7f51b00b1da0 msgr2=0x7f51b00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 -- 172.21.15.1:0/4206542995 shutdown_connections 2024-05-09T06:36:44.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.247+0000 7f51b51a9640 1 -- 172.21.15.1:0/4206542995 wait complete. 2024-05-09T06:36:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.248+0000 7f51b51a9640 1 Processor -- start 2024-05-09T06:36:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.248+0000 7f51b51a9640 1 -- start start 2024-05-09T06:36:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51b00b11f0 0x7f51b0157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51b00b1680 0x7f51b0156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f51b0162ba0 con 0x7f51b015e390 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f51b00b1a80 con 0x7f51b00b11f0 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51b51a9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f51b00cffb0 con 0x7f51b00b1680 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51af59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51af59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55586/0 (socket says 172.21.15.1:55586) 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.249+0000 7f51af59e640 1 -- 172.21.15.1:0/3715480042 learned_addr learned my addr 172.21.15.1:0/3715480042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51aed9d640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51b00b11f0 0x7f51b0157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51ae59c640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51b00b1680 0x7f51b0156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 -- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51b00b1680 msgr2=0x7f51b0156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51b00b1680 0x7f51b0156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 -- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51b00b11f0 msgr2=0x7f51b0157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51b00b11f0 0x7f51b0157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 -- 172.21.15.1:0/3715480042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01e8820 con 0x7f51b015e390 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51af59e640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009eb0 tx=0x7f51a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8040070 con 0x7f51b015e390 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a8031c40 con 0x7f51b015e390 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b01e8b00 con 0x7f51b015e390 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8003990 con 0x7f51b015e390 2024-05-09T06:36:44.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.250+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01e93a0 con 0x7f51b015e390 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.251+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b01e8cd0 con 0x7f51b015e390 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.252+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51a8039040 con 0x7f51b015e390 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.252+0000 7f51a77fe640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 0x7f519006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.252+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f51a80ac240 con 0x7f51b015e390 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.252+0000 7f51aed9d640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 0x7f519006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.253+0000 7f51aed9d640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 0x7f519006e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f60 tx=0x7f51a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.257+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51a807a410 con 0x7f51b015e390 2024-05-09T06:36:44.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.439+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51b01e95a0 con 0x7f51b015e390 2024-05-09T06:36:44.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.439+0000 7f51a77fe640 1 -- 172.21.15.1:0/3715480042 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f51b01e95a0 con 0x7f51b015e390 2024-05-09T06:36:44.419 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 msgr2=0x7f519006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 0x7f519006e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f60 tx=0x7f51a0009040 comp rx=0 tx=0).stop 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 msgr2=0x7f51b00c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009eb0 tx=0x7f51a8004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 shutdown_connections 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f519006be30 0x7f519006e2f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b015e390 0x7f51b00c3f60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51b00b1680 0x7f51b0156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 --2- 172.21.15.1:0/3715480042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51b00b11f0 0x7f51b0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 >> 172.21.15.1:0/3715480042 conn(0x7f51b00b1da0 msgr2=0x7f51b006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 shutdown_connections 2024-05-09T06:36:44.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.441+0000 7f51b51a9640 1 -- 172.21.15.1:0/3715480042 wait complete. 2024-05-09T06:36:44.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.557+0000 7f7e9d53b640 1 Processor -- start 2024-05-09T06:36:44.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.557+0000 7f7e9d53b640 1 -- start start 2024-05-09T06:36:44.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e9d53b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7e980d4940 0x7f7e980b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e9d53b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e980b33c0 con 0x7f7e98161950 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e9d53b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e980b3660 con 0x7f7e980d2b10 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e9d53b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7e980d4dd0 con 0x7f7e980d4940 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e96ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7e98161950 0x7f7e980d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57876/0 (socket says 172.21.15.1:57876) 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e8ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7e980d4940 0x7f7e980b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e96ffd640 1 -- 172.21.15.1:0/3553235855 learned_addr learned my addr 172.21.15.1:0/3553235855 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e8ffff640 1 -- 172.21.15.1:0/3553235855 >> v1:172.21.15.6:6789/0 conn(0x7f7e980d2b10 legacy=0x7f7e980b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e8ffff640 1 -- 172.21.15.1:0/3553235855 >> v1:172.21.15.1:6789/0 conn(0x7f7e98161950 legacy=0x7f7e980d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e8ffff640 1 -- 172.21.15.1:0/3553235855 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e980b3940 con 0x7f7e980d4940 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.558+0000 7f7e967fc640 1 -- 172.21.15.1:0/3553235855 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3338353580 0 0) 0x7f7e980b33c0 con 0x7f7e98161950 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.559+0000 7f7e8ffff640 1 --2- 172.21.15.1:0/3553235855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e980d4940 0x7f7e980b4140 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097a0 tx=0x7f7e8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a462de3e1fde87 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.559+0000 7f7e967fc640 1 -- 172.21.15.1:0/3553235855 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037070 con 0x7f7e980d4940 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.559+0000 7f7e967fc640 1 -- 172.21.15.1:0/3553235855 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80032440 con 0x7f7e980d4940 2024-05-09T06:36:44.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.559+0000 7f7e967fc640 1 -- 172.21.15.1:0/3553235855 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80032810 con 0x7f7e980d4940 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 -- 172.21.15.1:0/3553235855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e980d4940 msgr2=0x7f7e980b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/3553235855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e980d4940 0x7f7e980b4140 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097a0 tx=0x7f7e8002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 -- 172.21.15.1:0/3553235855 shutdown_connections 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/3553235855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e980d4940 0x7f7e980b4140 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 -- 172.21.15.1:0/3553235855 >> 172.21.15.1:0/3553235855 conn(0x7f7e980d3e20 msgr2=0x7f7e980d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 -- 172.21.15.1:0/3553235855 shutdown_connections 2024-05-09T06:36:44.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.560+0000 7f7e9d53b640 1 -- 172.21.15.1:0/3553235855 wait complete. 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.561+0000 7f7e9d53b640 1 Processor -- start 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.561+0000 7f7e9d53b640 1 -- start start 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e980d2b10 0x7f7e9814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9809dfe0 0x7f7e980cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980d1100 con 0x7f7e98161950 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980d0f80 con 0x7f7e9809dfe0 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e9d53b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7e980d1c00 con 0x7f7e980d2b10 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55600/0 (socket says 172.21.15.1:55600) 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 -- 172.21.15.1:0/2067026479 learned_addr learned my addr 172.21.15.1:0/2067026479 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e977fe640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9809dfe0 0x7f7e980cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e96ffd640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e980d2b10 0x7f7e9814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 -- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e980d2b10 msgr2=0x7f7e9814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e980d2b10 0x7f7e9814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 -- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9809dfe0 msgr2=0x7f7e980cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9809dfe0 0x7f7e980cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 -- 172.21.15.1:0/2067026479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9809f970 con 0x7f7e98161950 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.562+0000 7f7e8ffff640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.563+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037040 con 0x7f7e98161950 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.563+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80042070 con 0x7f7e98161950 2024-05-09T06:36:44.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.563+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80030a70 con 0x7f7e98161950 2024-05-09T06:36:44.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.563+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e980cd640 con 0x7f7e98161950 2024-05-09T06:36:44.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.563+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9814f150 con 0x7f7e98161950 2024-05-09T06:36:44.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.564+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9809e3f0 con 0x7f7e98161950 2024-05-09T06:36:44.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.564+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e80030c10 con 0x7f7e98161950 2024-05-09T06:36:44.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.565+0000 7f7e8f7fe640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.565+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f7e800aba60 con 0x7f7e98161950 2024-05-09T06:36:44.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.565+0000 7f7e96ffd640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.565+0000 7f7e96ffd640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 0x7f7e7806e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000f60 tx=0x7f7e88007d00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.569+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e9809e3f0 con 0x7f7e98161950 2024-05-09T06:36:44.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.765+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7e980752b0 con 0x7f7e98161950 2024-05-09T06:36:44.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.765+0000 7f7e8f7fe640 1 -- 172.21.15.1:0/2067026479 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f7e9809f970 con 0x7f7e98161950 2024-05-09T06:36:44.747 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 msgr2=0x7f7e7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 0x7f7e7806e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000f60 tx=0x7f7e88007d00 comp rx=0 tx=0).stop 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 msgr2=0x7f7e9809daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e800046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 shutdown_connections 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9809dfe0 0x7f7e980cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e98161950 0x7f7e9809daa0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 --2- 172.21.15.1:0/2067026479 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7e980d2b10 0x7f7e9814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 >> 172.21.15.1:0/2067026479 conn(0x7f7e980d3e20 msgr2=0x7f7e980b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 shutdown_connections 2024-05-09T06:36:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.768+0000 7f7e9d53b640 1 -- 172.21.15.1:0/2067026479 wait complete. 2024-05-09T06:36:44.758 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:44.758 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:44.775 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.939+0000 7fd025f42640 1 Processor -- start 2024-05-09T06:36:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 -- start start 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0200b2a60 0x7fd0200b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0200b5230 con 0x7fd0200b4da0 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd020071c20 con 0x7fd0200b2a60 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.940+0000 7fd025f42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd02006eb40 con 0x7fd0201454b0 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0200b2a60 0x7fd0200b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01ffff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd0200b4da0 0x7fd02015f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:58806/0 (socket says 172.21.15.1:58806) 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01ffff640 1 -- 172.21.15.1:0/764570787 learned_addr learned my addr 172.21.15.1:0/764570787 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0200b2a60 0x7fd0200b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:54754/0 (socket says 172.21.15.1:54754) 2024-05-09T06:36:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:44868/0 (socket says 172.21.15.1:44868) 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 -- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 msgr2=0x7fd0200b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 -- 172.21.15.1:0/764570787 >> v1:172.21.15.6:6789/0 conn(0x7fd0200b4da0 legacy=0x7fd02015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01e7fc640 1 -- 172.21.15.1:0/764570787 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2192274640 0 0) 0x7fd0200b5230 con 0x7fd0200b4da0 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01effd640 1 -- 172.21.15.1:0/764570787 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02006ee20 con 0x7fd0200b2a60 2024-05-09T06:36:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.941+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.942+0000 7fd01effd640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd0200b4860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0080097a0 tx=0x7fd00802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50218d905528084 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.942+0000 7fd01e7fc640 1 -- 172.21.15.1:0/764570787 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008037070 con 0x7fd0200b2a60 2024-05-09T06:36:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.942+0000 7fd01e7fc640 1 -- 172.21.15.1:0/764570787 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008032440 con 0x7fd0200b2a60 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.942+0000 7fd01e7fc640 1 -- 172.21.15.1:0/764570787 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008032810 con 0x7fd0200b2a60 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 -- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 msgr2=0x7fd0200b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd0200b4860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0080097a0 tx=0x7fd00802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 -- 172.21.15.1:0/764570787 shutdown_connections 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd0200b4860 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 --2- 172.21.15.1:0/764570787 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0201454b0 0x7fd0200b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 -- 172.21.15.1:0/764570787 >> 172.21.15.1:0/764570787 conn(0x7fd0200cc4b0 msgr2=0x7fd0200cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 -- 172.21.15.1:0/764570787 shutdown_connections 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.943+0000 7fd025f42640 1 -- 172.21.15.1:0/764570787 wait complete. 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.944+0000 7fd025f42640 1 Processor -- start 2024-05-09T06:36:44.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.944+0000 7fd025f42640 1 -- start start 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd02015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0200b4da0 0x7fd0201e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd020145890 con 0x7fd0201454b0 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd02006dbc0 con 0x7fd0200b2a60 2024-05-09T06:36:44.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd025f42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd02006e0e0 con 0x7fd0200b4da0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55602/0 (socket says 172.21.15.1:55602) 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 -- 172.21.15.1:0/635637771 learned_addr learned my addr 172.21.15.1:0/635637771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd02015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01effd640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0200b4da0 0x7fd0201e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 -- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0200b4da0 msgr2=0x7fd0201e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0200b4da0 0x7fd0201e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 -- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 msgr2=0x7fd02015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd02015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 -- 172.21.15.1:0/635637771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201e4110 con 0x7fd0201454b0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd02015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.945+0000 7fd01ffff640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c700 tx=0x7fd01800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.946+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018018070 con 0x7fd0201454b0 2024-05-09T06:36:44.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.946+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01800ce10 con 0x7fd0201454b0 2024-05-09T06:36:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.946+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201e43f0 con 0x7fd0201454b0 2024-05-09T06:36:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.946+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800a860 con 0x7fd0201454b0 2024-05-09T06:36:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.946+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201e4c90 con 0x7fd0201454b0 2024-05-09T06:36:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.947+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020071e40 con 0x7fd0201454b0 2024-05-09T06:36:44.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.947+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd01801b070 con 0x7fd0201454b0 2024-05-09T06:36:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.948+0000 7fd017fff640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 0x7fd00006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.948+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd01808a600 con 0x7fd0201454b0 2024-05-09T06:36:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.948+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 0x7fd00006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.948+0000 7fd01f7fe640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 0x7fd00006e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0200cf8c0 tx=0x7fd010008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:44.952+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd020071e40 con 0x7fd0201454b0 2024-05-09T06:36:45.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.135+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0201e4ed0 con 0x7fd0201454b0 2024-05-09T06:36:45.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.136+0000 7fd017fff640 1 -- 172.21.15.1:0/635637771 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16752 (secure 0 0 0) 0x7fd0201e4ed0 con 0x7fd0201454b0 2024-05-09T06:36:45.116 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:45.116 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":106,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:36.717091+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:45.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 msgr2=0x7fd00006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 0x7fd00006e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0200cf8c0 tx=0x7fd010008040 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 msgr2=0x7fd0201e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c700 tx=0x7fd01800cbc0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 shutdown_connections 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd00006be30 0x7fd00006e2f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0201454b0 0x7fd0201e3b30 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0200b4da0 0x7fd0201e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 --2- 172.21.15.1:0/635637771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0200b2a60 0x7fd02015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 >> 172.21.15.1:0/635637771 conn(0x7fd0200cc4b0 msgr2=0x7fd0200af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 shutdown_connections 2024-05-09T06:36:45.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.138+0000 7fd025f42640 1 -- 172.21.15.1:0/635637771 wait complete. 2024-05-09T06:36:45.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:33.397054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:34.058197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:45.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.256+0000 7ff089189640 1 Processor -- start 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.256+0000 7ff089189640 1 -- start start 2024-05-09T06:36:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff089189640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff084158a50 0x7ff0840718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff089189640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff084071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff089189640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08415b420 con 0x7ff08415c4a0 2024-05-09T06:36:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff089189640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff084158e30 con 0x7ff084158a50 2024-05-09T06:36:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff089189640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff084072480 con 0x7ff08406ede0 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff082d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff084158a50 0x7ff0840718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff08259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff08415c4a0 0x7ff08406e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57888/0 (socket says 172.21.15.1:57888) 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff08259c640 1 -- 172.21.15.1:0/2519925836 learned_addr learned my addr 172.21.15.1:0/2519925836 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff082d9d640 1 -- 172.21.15.1:0/2519925836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 msgr2=0x7ff084071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff082d9d640 1 --2- 172.21.15.1:0/2519925836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff084071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.257+0000 7ff082d9d640 1 -- 172.21.15.1:0/2519925836 >> v1:172.21.15.1:6789/0 conn(0x7ff08415c4a0 legacy=0x7ff08406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.258+0000 7ff082d9d640 1 -- 172.21.15.1:0/2519925836 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08415b6e0 con 0x7ff084158a50 2024-05-09T06:36:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.258+0000 7ff082d9d640 1 --2- 172.21.15.1:0/2519925836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff084158a50 0x7ff0840718c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff0740097a0 tx=0x7ff07402c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a736852b3f38fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.258+0000 7ff081d9b640 1 -- 172.21.15.1:0/2519925836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074039070 con 0x7ff084158a50 2024-05-09T06:36:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.258+0000 7ff081d9b640 1 -- 172.21.15.1:0/2519925836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff074034640 con 0x7ff084158a50 2024-05-09T06:36:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.258+0000 7ff081d9b640 1 -- 172.21.15.1:0/2519925836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074034a10 con 0x7ff084158a50 2024-05-09T06:36:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 -- 172.21.15.1:0/2519925836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff084158a50 msgr2=0x7ff0840718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 --2- 172.21.15.1:0/2519925836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff084158a50 0x7ff0840718c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff0740097a0 tx=0x7ff07402c3b0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 -- 172.21.15.1:0/2519925836 shutdown_connections 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 --2- 172.21.15.1:0/2519925836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff084071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 --2- 172.21.15.1:0/2519925836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff084158a50 0x7ff0840718c0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 -- 172.21.15.1:0/2519925836 >> 172.21.15.1:0/2519925836 conn(0x7ff0840cc4d0 msgr2=0x7ff0840cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 -- 172.21.15.1:0/2519925836 shutdown_connections 2024-05-09T06:36:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.259+0000 7ff089189640 1 -- 172.21.15.1:0/2519925836 wait complete. 2024-05-09T06:36:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.260+0000 7ff089189640 1 Processor -- start 2024-05-09T06:36:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 -- start start 2024-05-09T06:36:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff0841f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff084158a50 0x7ff08414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff08415bf80 con 0x7ff08415c4a0 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff08415bca0 con 0x7ff084158a50 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff089189640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff084070ea0 con 0x7ff08406ede0 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff08359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff08359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55622/0 (socket says 172.21.15.1:55622) 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.261+0000 7ff08359e640 1 -- 172.21.15.1:0/4094628372 learned_addr learned my addr 172.21.15.1:0/4094628372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08259c640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff084158a50 0x7ff08414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff082d9d640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff0841f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 -- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 msgr2=0x7ff0841f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff0841f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 -- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff084158a50 msgr2=0x7ff08414d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff084158a50 0x7ff08414d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 -- 172.21.15.1:0/4094628372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0840af720 con 0x7ff08415c4a0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff082d9d640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff0841f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff08359e640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00b9e0 tx=0x7ff07c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c019070 con 0x7ff08415c4a0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c00cb20 con 0x7ff08415c4a0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0840af990 con 0x7ff08415c4a0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c014440 con 0x7ff08415c4a0 2024-05-09T06:36:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.262+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0840aff50 con 0x7ff08415c4a0 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.263+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0840752b0 con 0x7ff08415c4a0 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.264+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff07c01c070 con 0x7ff08415c4a0 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.264+0000 7ff07b7fe640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 0x7ff06406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.264+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7ff07c08ac70 con 0x7ff08415c4a0 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.264+0000 7ff082d9d640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 0x7ff06406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.265+0000 7ff082d9d640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 0x7ff06406e320 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff074036000 tx=0x7ff074030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.269+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff07c059490 con 0x7ff08415c4a0 2024-05-09T06:36:45.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.452+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0840ccc30 con 0x7ff08415c4a0 2024-05-09T06:36:45.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.452+0000 7ff07b7fe640 1 -- 172.21.15.1:0/4094628372 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7ff07c058d90 con 0x7ff08415c4a0 2024-05-09T06:36:45.432 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 msgr2=0x7ff06406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 0x7ff06406e320 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff074036000 tx=0x7ff074030000 comp rx=0 tx=0).stop 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 msgr2=0x7ff08406dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00b9e0 tx=0x7ff07c00bea0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 shutdown_connections 2024-05-09T06:36:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff06406be60 0x7ff06406e320 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08415c4a0 0x7ff08406dce0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff084158a50 0x7ff08414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 --2- 172.21.15.1:0/4094628372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff08406ede0 0x7ff0841f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 >> 172.21.15.1:0/4094628372 conn(0x7ff0840cc4d0 msgr2=0x7ff08415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 shutdown_connections 2024-05-09T06:36:45.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.454+0000 7ff089189640 1 -- 172.21.15.1:0/4094628372 wait complete. 2024-05-09T06:36:45.445 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.572+0000 7f60df748640 1 Processor -- start 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.572+0000 7f60df748640 1 -- start start 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60df748640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60d80d06f0 0x7f60d80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60df748640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f60d80cf8e0 0x7f60d80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60df748640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d8072050 con 0x7f60d80c41e0 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60df748640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f60d80cd160 con 0x7f60d80d06f0 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60df748640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f60d8072330 con 0x7f60d80cf8e0 2024-05-09T06:36:45.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60d80d06f0 0x7f60d80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60d80d06f0 0x7f60d80cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55636/0 (socket says 172.21.15.1:55636) 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 -- 172.21.15.1:0/401100777 learned_addr learned my addr 172.21.15.1:0/401100777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60ddce5640 1 --2- 172.21.15.1:0/401100777 >> v2:172.21.15.6:3300/0 conn(0x7f60d80cf8e0 0x7f60d80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 -- 172.21.15.1:0/401100777 >> v2:172.21.15.6:3300/0 conn(0x7f60d80cf8e0 msgr2=0x7f60d80ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/401100777 >> v2:172.21.15.6:3300/0 conn(0x7f60d80cf8e0 0x7f60d80ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 -- 172.21.15.1:0/401100777 >> v1:172.21.15.6:6789/0 conn(0x7f60d80c41e0 legacy=0x7f60d8161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.573+0000 7f60dd4e4640 1 -- 172.21.15.1:0/401100777 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d8072610 con 0x7f60d80d06f0 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.574+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/401100777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80d06f0 0x7f60d80cc6e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f60d80b3780 tx=0x7f60d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb2b5b274a441c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.574+0000 7f60cffff640 1 -- 172.21.15.1:0/401100777 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60d80d06f0 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.574+0000 7f60cffff640 1 -- 172.21.15.1:0/401100777 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d4032440 con 0x7f60d80d06f0 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.574+0000 7f60cffff640 1 -- 172.21.15.1:0/401100777 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032810 con 0x7f60d80d06f0 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 -- 172.21.15.1:0/401100777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80d06f0 msgr2=0x7f60d80cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 --2- 172.21.15.1:0/401100777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80d06f0 0x7f60d80cc6e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f60d80b3780 tx=0x7f60d402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 -- 172.21.15.1:0/401100777 shutdown_connections 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 --2- 172.21.15.1:0/401100777 >> v2:172.21.15.6:3300/0 conn(0x7f60d80cf8e0 0x7f60d80ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 --2- 172.21.15.1:0/401100777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80d06f0 0x7f60d80cc6e0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 -- 172.21.15.1:0/401100777 >> 172.21.15.1:0/401100777 conn(0x7f60d80d0b50 msgr2=0x7f60d80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 -- 172.21.15.1:0/401100777 shutdown_connections 2024-05-09T06:36:45.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.575+0000 7f60df748640 1 -- 172.21.15.1:0/401100777 wait complete. 2024-05-09T06:36:45.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.576+0000 7f60df748640 1 Processor -- start 2024-05-09T06:36:45.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.576+0000 7f60df748640 1 -- start start 2024-05-09T06:36:45.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 0x7f60d80c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 0x7f60d80a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60d806b1e0 con 0x7f60d80cf8e0 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8071ce0 con 0x7f60d80c41e0 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60df748640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f60d806b360 con 0x7f60d80d06f0 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55638/0 (socket says 172.21.15.1:55638) 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 -- 172.21.15.1:0/2800600518 learned_addr learned my addr 172.21.15.1:0/2800600518 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:45.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60ddce5640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 0x7f60d80a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 0x7f60d80c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 -- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 msgr2=0x7f60d80a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 0x7f60d80a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 -- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 msgr2=0x7f60d80c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 0x7f60d80c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 -- 172.21.15.1:0/2800600518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81f5040 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 0x7f60d80c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60ddce5640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 0x7f60d80a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.577+0000 7f60dcce3640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c400 tx=0x7f60c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.578+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80052d0 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.578+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c8005490 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.578+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81f4d60 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.578+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8007260 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.578+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d80c4a00 con 0x7f60d80cf8e0 2024-05-09T06:36:45.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.579+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d80752b0 con 0x7f60d80cf8e0 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.579+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60c8007400 con 0x7f60d80cf8e0 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.580+0000 7f60cdffb640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 0x7f60bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.580+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f60c808a530 con 0x7f60d80cf8e0 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.580+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 0x7f60bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.580+0000 7f60dd4e4640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 0x7f60bc06e320 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.584+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60c8058d50 con 0x7f60d80cf8e0 2024-05-09T06:36:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.769+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60d8072f30 con 0x7f60d80cf8e0 2024-05-09T06:36:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.769+0000 7f60cdffb640 1 -- 172.21.15.1:0/2800600518 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f60c8058650 con 0x7f60d80cf8e0 2024-05-09T06:36:45.749 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 msgr2=0x7f60bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 0x7f60bc06e320 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 msgr2=0x7f60d80a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c400 tx=0x7f60c800c8c0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 shutdown_connections 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60bc06be60 0x7f60bc06e320 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f60d80d06f0 0x7f60d80a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d80cf8e0 0x7f60d80a0c30 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 --2- 172.21.15.1:0/2800600518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60d80c41e0 0x7f60d80c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 >> 172.21.15.1:0/2800600518 conn(0x7f60d80d0b50 msgr2=0x7f60d806ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 shutdown_connections 2024-05-09T06:36:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.772+0000 7f60df748640 1 -- 172.21.15.1:0/2800600518 wait complete. 2024-05-09T06:36:45.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:45.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.890+0000 7ff9fdff9640 1 Processor -- start 2024-05-09T06:36:45.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.890+0000 7ff9fdff9640 1 -- start start 2024-05-09T06:36:45.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9fdff9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9f815c4c0 0x7ff9f806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9fdff9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 0x7ff9f815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9fdff9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f806cf40 con 0x7ff9f80b5260 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9fdff9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f8156980 con 0x7ff9f815c4c0 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9fdff9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f806d870 con 0x7ff9f80b4de0 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9f77fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff9f815c4c0 0x7ff9f806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9f7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff9f80b5260 0x7ff9f8070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57892/0 (socket says 172.21.15.1:57892) 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.891+0000 7ff9f7fff640 1 -- 172.21.15.1:0/4219549822 learned_addr learned my addr 172.21.15.1:0/4219549822 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f6ffd640 1 --2- 172.21.15.1:0/4219549822 >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 0x7ff9f815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f77fe640 1 -- 172.21.15.1:0/4219549822 >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 msgr2=0x7ff9f815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f77fe640 1 --2- 172.21.15.1:0/4219549822 >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 0x7ff9f815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f77fe640 1 -- 172.21.15.1:0/4219549822 >> v1:172.21.15.1:6789/0 conn(0x7ff9f80b5260 legacy=0x7ff9f8070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f77fe640 1 -- 172.21.15.1:0/4219549822 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f806d220 con 0x7ff9f815c4c0 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f67fc640 1 -- 172.21.15.1:0/4219549822 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725522652 0 0) 0x7ff9f806cf40 con 0x7ff9f80b5260 2024-05-09T06:36:45.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f6ffd640 1 --2- 172.21.15.1:0/4219549822 >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 0x7ff9f815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.892+0000 7ff9f77fe640 1 --2- 172.21.15.1:0/4219549822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f815c4c0 0x7ff9f806dd00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80cd190 tx=0x7ff9e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec676a1e4135cb4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9f67fc640 1 -- 172.21.15.1:0/4219549822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8037070 con 0x7ff9f815c4c0 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9f67fc640 1 -- 172.21.15.1:0/4219549822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8032440 con 0x7ff9f815c4c0 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9f67fc640 1 -- 172.21.15.1:0/4219549822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8032810 con 0x7ff9f815c4c0 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 -- 172.21.15.1:0/4219549822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f815c4c0 msgr2=0x7ff9f806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/4219549822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f815c4c0 0x7ff9f806dd00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80cd190 tx=0x7ff9e802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 -- 172.21.15.1:0/4219549822 shutdown_connections 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/4219549822 >> v2:172.21.15.6:3300/0 conn(0x7ff9f80b4de0 0x7ff9f815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/4219549822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f815c4c0 0x7ff9f806dd00 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 -- 172.21.15.1:0/4219549822 >> 172.21.15.1:0/4219549822 conn(0x7ff9f80cc4d0 msgr2=0x7ff9f80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 -- 172.21.15.1:0/4219549822 shutdown_connections 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.893+0000 7ff9fdff9640 1 -- 172.21.15.1:0/4219549822 wait complete. 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.894+0000 7ff9fdff9640 1 Processor -- start 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 -- start start 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9f80b4de0 0x7ff9f813b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9f815c4c0 0x7ff9f80ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80b2dc0 con 0x7ff9f80b5260 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80b2c40 con 0x7ff9f815c4c0 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9fdff9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f815cde0 con 0x7ff9f80b4de0 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9f6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55666/0 (socket says 172.21.15.1:55666) 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.895+0000 7ff9f6ffd640 1 -- 172.21.15.1:0/984935991 learned_addr learned my addr 172.21.15.1:0/984935991 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f7fff640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9f815c4c0 0x7ff9f80ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f77fe640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9f80b4de0 0x7ff9f813b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 -- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9f80b4de0 msgr2=0x7ff9f813b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9f80b4de0 0x7ff9f813b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 -- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9f815c4c0 msgr2=0x7ff9f80ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9f815c4c0 0x7ff9f80ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 -- 172.21.15.1:0/984935991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f813b970 con 0x7ff9f80b5260 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9f6ffd640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c7f0 tx=0x7ff9ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec015070 con 0x7ff9f80b5260 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec007a30 con 0x7ff9f80b5260 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec007e00 con 0x7ff9f80b5260 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f80ad880 con 0x7ff9f80b5260 2024-05-09T06:36:45.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.896+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f80aa9c0 con 0x7ff9f80b5260 2024-05-09T06:36:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.897+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80752b0 con 0x7ff9f80b5260 2024-05-09T06:36:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.898+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9ec018070 con 0x7ff9f80b5260 2024-05-09T06:36:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.898+0000 7ff9dffff640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.898+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7ff9ec08aee0 con 0x7ff9f80b5260 2024-05-09T06:36:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.898+0000 7ff9f77fe640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.899+0000 7ff9f77fe640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 0x7ff9e406e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034000 tx=0x7ff9e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:45.903+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9f813b970 con 0x7ff9f80b5260 2024-05-09T06:36:46.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.085+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9f8155760 con 0x7ff9f80b5260 2024-05-09T06:36:46.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.086+0000 7ff9dffff640 1 -- 172.21.15.1:0/984935991 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7ff9ec059790 con 0x7ff9f80b5260 2024-05-09T06:36:46.066 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 msgr2=0x7ff9e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 0x7ff9e406e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034000 tx=0x7ff9e802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 msgr2=0x7ff9f80a9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c7f0 tx=0x7ff9ec00cbf0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 shutdown_connections 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9f815c4c0 0x7ff9f80ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9f80b5260 0x7ff9f80a9e60 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 --2- 172.21.15.1:0/984935991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9f80b4de0 0x7ff9f813b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 >> 172.21.15.1:0/984935991 conn(0x7ff9f80cc4d0 msgr2=0x7ff9f80cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 shutdown_connections 2024-05-09T06:36:46.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.088+0000 7ff9fdff9640 1 -- 172.21.15.1:0/984935991 wait complete. 2024-05-09T06:36:46.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:46.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.204+0000 7fbf064df640 1 Processor -- start 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.204+0000 7fbf064df640 1 -- start start 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf064df640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbf001454b0 0x7fbf000b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf064df640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf0006eb40 con 0x7fbf000b4f70 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf064df640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf0006ee20 con 0x7fbf000b2a60 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf064df640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbf0006d8e0 con 0x7fbf001454b0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbefffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbf001454b0 0x7fbf000b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf04a7c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbf000b4f70 0x7fbf00070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57904/0 (socket says 172.21.15.1:57904) 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbf04a7c640 1 -- 172.21.15.1:0/942187782 learned_addr learned my addr 172.21.15.1:0/942187782 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbefffff640 1 -- 172.21.15.1:0/942187782 >> v1:172.21.15.6:6789/0 conn(0x7fbf000b2a60 legacy=0x7fbf000b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbefffff640 1 -- 172.21.15.1:0/942187782 >> v1:172.21.15.1:6789/0 conn(0x7fbf000b4f70 legacy=0x7fbf00070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbefffff640 1 -- 172.21.15.1:0/942187782 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0006dbc0 con 0x7fbf001454b0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.205+0000 7fbefeffd640 1 -- 172.21.15.1:0/942187782 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125454941 0 0) 0x7fbf0006eb40 con 0x7fbf000b4f70 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.206+0000 7fbefffff640 1 --2- 172.21.15.1:0/942187782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf001454b0 0x7fbf000b4320 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbef00097a0 tx=0x7fbef002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81d5e3f7fbb2b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.206+0000 7fbefeffd640 1 -- 172.21.15.1:0/942187782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0037070 con 0x7fbf001454b0 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.206+0000 7fbefeffd640 1 -- 172.21.15.1:0/942187782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0032440 con 0x7fbf001454b0 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.206+0000 7fbefeffd640 1 -- 172.21.15.1:0/942187782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0032810 con 0x7fbf001454b0 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 -- 172.21.15.1:0/942187782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf001454b0 msgr2=0x7fbf000b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 --2- 172.21.15.1:0/942187782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf001454b0 0x7fbf000b4320 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbef00097a0 tx=0x7fbef002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 -- 172.21.15.1:0/942187782 shutdown_connections 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 --2- 172.21.15.1:0/942187782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf001454b0 0x7fbf000b4320 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 -- 172.21.15.1:0/942187782 >> 172.21.15.1:0/942187782 conn(0x7fbf000cc4b0 msgr2=0x7fbf000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 -- 172.21.15.1:0/942187782 shutdown_connections 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.207+0000 7fbf064df640 1 -- 172.21.15.1:0/942187782 wait complete. 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.208+0000 7fbf064df640 1 Processor -- start 2024-05-09T06:36:46.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.208+0000 7fbf064df640 1 -- start start 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf000b4f70 0x7fbf001e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 0x7fbf0012eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0006e260 con 0x7fbf000b2a60 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0006e0e0 con 0x7fbf001454b0 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf064df640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbf00071320 con 0x7fbf000b4f70 2024-05-09T06:36:46.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55686/0 (socket says 172.21.15.1:55686) 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 -- 172.21.15.1:0/162785610 learned_addr learned my addr 172.21.15.1:0/162785610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf04a7c640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 0x7fbf0012eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 -- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf000b4f70 msgr2=0x7fbf001e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf000b4f70 0x7fbf001e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 -- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 msgr2=0x7fbf0012eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 0x7fbf0012eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 -- 172.21.15.1:0/162785610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0012f800 con 0x7fbf000b2a60 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbf04a7c640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 0x7fbf0012eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.209+0000 7fbefffff640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000c00 tx=0x7fbef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.210+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0037040 con 0x7fbf000b2a60 2024-05-09T06:36:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.210+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0042070 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.210+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0030a70 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.210+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0012fae0 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.210+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001300a0 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.211+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0004fe40 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.211+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbef0030c10 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.212+0000 7fbefcff9640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 0x7fbeec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.212+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fbef00abb40 con 0x7fbf000b2a60 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.212+0000 7fbeff7fe640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 0x7fbeec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.212+0000 7fbeff7fe640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 0x7fbeec06e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000f60 tx=0x7fbef40079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.216+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf0004fe40 con 0x7fbf000b2a60 2024-05-09T06:36:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.414+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf000752b0 con 0x7fbf000b2a60 2024-05-09T06:36:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.414+0000 7fbefcff9640 1 -- 172.21.15.1:0/162785610 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7fbef007a360 con 0x7fbf000b2a60 2024-05-09T06:36:46.395 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 msgr2=0x7fbeec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 0x7fbeec06e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000f60 tx=0x7fbef40079b0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 msgr2=0x7fbf001308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000c00 tx=0x7fbef00046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 shutdown_connections 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbeec06be30 0x7fbeec06e2f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf001454b0 0x7fbf0012eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf000b4f70 0x7fbf001e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 --2- 172.21.15.1:0/162785610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf000b2a60 0x7fbf001308a0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 >> 172.21.15.1:0/162785610 conn(0x7fbf000cc4b0 msgr2=0x7fbf000ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 shutdown_connections 2024-05-09T06:36:46.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:46.417+0000 7fbf064df640 1 -- 172.21.15.1:0/162785610 wait complete. 2024-05-09T06:36:46.407 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:36:46.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:46.408 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:46.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[74870]: fuse finished with error 0 and tester_r 0 2024-05-09T06:36:46.435 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:48.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:48.850 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:52.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:36:52.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:52.537 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:36:52.558 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.733+0000 7fb24835a640 1 Processor -- start 2024-05-09T06:36:52.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.733+0000 7fb24835a640 1 -- start start 2024-05-09T06:36:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb24835a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb240161950 0x7fb2400d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb24835a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb2400d0f80 0x7fb2400d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb24835a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2400b33c0 con 0x7fb2400d4940 2024-05-09T06:36:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb24835a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb2400d4dd0 con 0x7fb240161950 2024-05-09T06:36:52.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb24835a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb2400b3660 con 0x7fb2400d0f80 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb240161950 0x7fb2400d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb240161950 0x7fb2400d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38152/0 (socket says 172.21.15.1:38152) 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 -- 172.21.15.1:0/3186050206 learned_addr learned my addr 172.21.15.1:0/3186050206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2468f7640 1 --2- 172.21.15.1:0/3186050206 >> v2:172.21.15.6:3300/0 conn(0x7fb2400d0f80 0x7fb2400d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 -- 172.21.15.1:0/3186050206 >> v2:172.21.15.6:3300/0 conn(0x7fb2400d0f80 msgr2=0x7fb2400d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 --2- 172.21.15.1:0/3186050206 >> v2:172.21.15.6:3300/0 conn(0x7fb2400d0f80 0x7fb2400d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 -- 172.21.15.1:0/3186050206 >> v1:172.21.15.6:6789/0 conn(0x7fb2400d4940 legacy=0x7fb2400cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.735+0000 7fb2460f6640 1 -- 172.21.15.1:0/3186050206 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2400b3940 con 0x7fb240161950 2024-05-09T06:36:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.736+0000 7fb2460f6640 1 --2- 172.21.15.1:0/3186050206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400d2400 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb24008e540 tx=0x7fb23c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f4055655458c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.736+0000 7fb2450f4640 1 -- 172.21.15.1:0/3186050206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c037070 con 0x7fb240161950 2024-05-09T06:36:52.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.736+0000 7fb2450f4640 1 -- 172.21.15.1:0/3186050206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c032440 con 0x7fb240161950 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.736+0000 7fb2450f4640 1 -- 172.21.15.1:0/3186050206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c032810 con 0x7fb240161950 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 -- 172.21.15.1:0/3186050206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 msgr2=0x7fb2400d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 --2- 172.21.15.1:0/3186050206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400d2400 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb24008e540 tx=0x7fb23c02a270 comp rx=0 tx=0).stop 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 -- 172.21.15.1:0/3186050206 shutdown_connections 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 --2- 172.21.15.1:0/3186050206 >> v2:172.21.15.6:3300/0 conn(0x7fb2400d0f80 0x7fb2400d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 --2- 172.21.15.1:0/3186050206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400d2400 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 -- 172.21.15.1:0/3186050206 >> 172.21.15.1:0/3186050206 conn(0x7fb2400d3e20 msgr2=0x7fb2400d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 -- 172.21.15.1:0/3186050206 shutdown_connections 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.737+0000 7fb24835a640 1 -- 172.21.15.1:0/3186050206 wait complete. 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.738+0000 7fb24835a640 1 Processor -- start 2024-05-09T06:36:52.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.738+0000 7fb24835a640 1 -- start start 2024-05-09T06:36:52.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 0x7fb2400c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2400cb2d0 0x7fb2400b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb2400b52d0 con 0x7fb240161950 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb2400b5150 con 0x7fb2400d0f80 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb24835a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb2400b5450 con 0x7fb2400cb2d0 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38162/0 (socket says 172.21.15.1:38162) 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 -- 172.21.15.1:0/1837617823 learned_addr learned my addr 172.21.15.1:0/1837617823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2460f6640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 0x7fb2400c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2468f7640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2400cb2d0 0x7fb2400b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 -- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2400cb2d0 msgr2=0x7fb2400b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2400cb2d0 0x7fb2400b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 -- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 msgr2=0x7fb2400c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 0x7fb2400c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 -- 172.21.15.1:0/1837617823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2400b72d0 con 0x7fb240161950 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2460f6640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 0x7fb2400c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:52.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.739+0000 7fb2458f5640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c460 tx=0x7fb23400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.740+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2340052d0 con 0x7fb240161950 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.740+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234012630 con 0x7fb240161950 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.740+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2400b75b0 con 0x7fb240161950 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.740+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2340054b0 con 0x7fb240161950 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.740+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2400b7b70 con 0x7fb240161950 2024-05-09T06:36:52.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.741+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2400752b0 con 0x7fb240161950 2024-05-09T06:36:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.741+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb234012a20 con 0x7fb240161950 2024-05-09T06:36:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.742+0000 7fb232ffd640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 0x7fb22406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.742+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fb23408ab50 con 0x7fb240161950 2024-05-09T06:36:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.742+0000 7fb2460f6640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 0x7fb22406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.742+0000 7fb2460f6640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 0x7fb22406e320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb23c034000 tx=0x7fb23c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.746+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb234059370 con 0x7fb240161950 2024-05-09T06:36:52.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.967+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2400b31a0 con 0x7fb240161950 2024-05-09T06:36:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.967+0000 7fb232ffd640 1 -- 172.21.15.1:0/1837617823 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5171 (secure 0 0 0) 0x7fb23405f020 con 0x7fb240161950 2024-05-09T06:36:52.948 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:52.948 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:36:38.092253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6835/3796896332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3796896332},{"type":"v1","addr":"172.21.15.1:6835","nonce":3796896332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":7}]} 2024-05-09T06:36:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 82 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 msgr2=0x7fb22406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 0x7fb22406e320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb23c034000 tx=0x7fb23c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 msgr2=0x7fb2400b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c460 tx=0x7fb23400c920 comp rx=0 tx=0).stop 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 shutdown_connections 2024-05-09T06:36:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb22406be60 0x7fb22406e320 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2400cb2d0 0x7fb2400b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb240161950 0x7fb2400b6a10 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 --2- 172.21.15.1:0/1837617823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2400d0f80 0x7fb2400c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 >> 172.21.15.1:0/1837617823 conn(0x7fb2400d3e20 msgr2=0x7fb24015ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 shutdown_connections 2024-05-09T06:36:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:52.969+0000 7fb24835a640 1 -- 172.21.15.1:0/1837617823 wait complete. 2024-05-09T06:36:52.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-09T06:36:53.091 INFO:teuthology.orchestra.run.smithi001.stdout:{"message":"","return_code":0} 2024-05-09T06:36:53.091 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:36:53.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.215+0000 7f228fc1c640 1 Processor -- start 2024-05-09T06:36:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.215+0000 7f228fc1c640 1 -- start start 2024-05-09T06:36:53.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.216+0000 7f228fc1c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.216+0000 7f228fc1c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 0x7f22880cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.216+0000 7f228fc1c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2288071c70 con 0x7f2288165ff0 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.216+0000 7f228fc1c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f228806f0c0 con 0x7f2288166b40 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.216+0000 7f228fc1c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f228814c210 con 0x7f228814be30 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228e1b9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 0x7f22880cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:53816/0 (socket says 172.21.15.1:53816) 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 -- 172.21.15.1:0/1445615122 learned_addr learned my addr 172.21.15.1:0/1445615122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d1b7640 1 --1- 172.21.15.1:0/1445615122 >> v1:172.21.15.6:6789/0 conn(0x7f2288165ff0 0x7f2288166430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:56350/0 (socket says 172.21.15.1:56350) 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 -- 172.21.15.1:0/1445615122 >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 msgr2=0x7f22880cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 --2- 172.21.15.1:0/1445615122 >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 0x7f22880cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 -- 172.21.15.1:0/1445615122 >> v1:172.21.15.6:6789/0 conn(0x7f2288165ff0 legacy=0x7f2288166430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228d9b8640 1 -- 172.21.15.1:0/1445615122 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288071f50 con 0x7f228814be30 2024-05-09T06:36:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.217+0000 7f228e1b9640 1 --2- 172.21.15.1:0/1445615122 >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 0x7f22880cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.218+0000 7f228d9b8640 1 --2- 172.21.15.1:0/1445615122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19303d08f72d9b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.218+0000 7f228c9b6640 1 -- 172.21.15.1:0/1445615122 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280037070 con 0x7f228814be30 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.218+0000 7f228c9b6640 1 -- 172.21.15.1:0/1445615122 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280032440 con 0x7f228814be30 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.218+0000 7f228c9b6640 1 -- 172.21.15.1:0/1445615122 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280032810 con 0x7f228814be30 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 -- 172.21.15.1:0/1445615122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 msgr2=0x7f228806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 --2- 172.21.15.1:0/1445615122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228002a250 comp rx=0 tx=0).stop 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 -- 172.21.15.1:0/1445615122 shutdown_connections 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 --2- 172.21.15.1:0/1445615122 >> v2:172.21.15.6:3300/0 conn(0x7f2288166b40 0x7f22880cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 --2- 172.21.15.1:0/1445615122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f228806cd30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 -- 172.21.15.1:0/1445615122 >> 172.21.15.1:0/1445615122 conn(0x7f22880cc4b0 msgr2=0x7f22880cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 -- 172.21.15.1:0/1445615122 shutdown_connections 2024-05-09T06:36:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.219+0000 7f228fc1c640 1 -- 172.21.15.1:0/1445615122 wait complete. 2024-05-09T06:36:53.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.220+0000 7f228fc1c640 1 Processor -- start 2024-05-09T06:36:53.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.220+0000 7f228fc1c640 1 -- start start 2024-05-09T06:36:53.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f22880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 0x7f22880aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f228806e7e0 con 0x7f2288165ff0 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f228806d270 con 0x7f2288166b40 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228fc1c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f228806e960 con 0x7f228814be30 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38172/0 (socket says 172.21.15.1:38172) 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 -- 172.21.15.1:0/586728925 learned_addr learned my addr 172.21.15.1:0/586728925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228e1b9640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 0x7f22880aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d9b8640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f22880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 -- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 msgr2=0x7f22880b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f22880b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 -- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 msgr2=0x7f22880aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 0x7f22880aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d1b7640 1 -- 172.21.15.1:0/586728925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288146f30 con 0x7f2288165ff0 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228e1b9640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 0x7f22880aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.221+0000 7f228d9b8640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f22880b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f228d1b7640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c430 tx=0x7f227c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c0074e0 con 0x7f2288165ff0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c007680 con 0x7f2288165ff0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2288147210 con 0x7f2288165ff0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c004a00 con 0x7f2288165ff0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.222+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22880a9f30 con 0x7f2288165ff0 2024-05-09T06:36:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.223+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880752b0 con 0x7f2288165ff0 2024-05-09T06:36:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.223+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f227c01a020 con 0x7f2288165ff0 2024-05-09T06:36:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.224+0000 7f22867fc640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 0x7f227406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.224+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f227c08a9c0 con 0x7f2288165ff0 2024-05-09T06:36:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.224+0000 7f228d9b8640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 0x7f227406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.224+0000 7f228d9b8640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 0x7f227406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2280034000 tx=0x7f228002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.228+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f227c059030 con 0x7f2288165ff0 2024-05-09T06:36:53.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.449+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22880cf2d0 con 0x7f2288165ff0 2024-05-09T06:36:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.449+0000 7f22867fc640 1 -- 172.21.15.1:0/586728925 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5171 (secure 0 0 0) 0x7f227c058930 con 0x7f2288165ff0 2024-05-09T06:36:53.430 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:53.430 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:36:38.092253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6835/3796896332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3796896332},{"type":"v1","addr":"172.21.15.1:6835","nonce":3796896332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":7}]} 2024-05-09T06:36:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 82 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 msgr2=0x7f227406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 0x7f227406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2280034000 tx=0x7f228002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 msgr2=0x7f2288161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c430 tx=0x7f227c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 shutdown_connections 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f227406be30 0x7f227406e2f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2288166b40 0x7f22880aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2288165ff0 0x7f2288161e90 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 --2- 172.21.15.1:0/586728925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f228814be30 0x7f22880b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 >> 172.21.15.1:0/586728925 conn(0x7f22880cc4b0 msgr2=0x7f2288166430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 shutdown_connections 2024-05-09T06:36:53.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.451+0000 7f228fc1c640 1 -- 172.21.15.1:0/586728925 wait complete. 2024-05-09T06:36:53.443 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-09T06:36:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.507+0000 7f710b692f40 1 Processor -- start 2024-05-09T06:36:53.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.507+0000 7f710b692f40 1 -- start start 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a5931290 0x55b5a58e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a59a4f40 0x55b5a5927fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b5a59110b0 con 0x55b5a592d840 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b5a588bc80 con 0x55b5a5931290 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f710b692f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b5a592c920 con 0x55b5a59a4f40 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38174/0 (socket says 172.21.15.1:38174) 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 -- 172.21.15.1:0/2357904730 learned_addr learned my addr 172.21.15.1:0/2357904730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7108992640 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a5931290 0x55b5a58e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109994640 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a59a4f40 0x55b5a5927fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 -- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a59a4f40 msgr2=0x55b5a5927fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a59a4f40 0x55b5a5927fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 -- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a5931290 msgr2=0x55b5a58e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a5931290 0x55b5a58e42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.508+0000 7f7109193640 1 -- 172.21.15.1:0/2357904730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a5810cb0 con 0x55b5a592d840 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f7109193640 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f70fc055040 tx=0x7f70fc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988e3698df47aae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f70fb7fe640 1 -- 172.21.15.1:0/2357904730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc053800 con 0x55b5a592d840 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f70fb7fe640 1 -- 172.21.15.1:0/2357904730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc0539c0 con 0x55b5a592d840 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f70fb7fe640 1 -- 172.21.15.1:0/2357904730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc053db0 con 0x55b5a592d840 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 -- 172.21.15.1:0/2357904730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 msgr2=0x55b5a59bb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f70fc055040 tx=0x7f70fc051000 comp rx=0 tx=0).stop 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 -- 172.21.15.1:0/2357904730 shutdown_connections 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a59a4f40 0x55b5a5927fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a5931290 0x55b5a58e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 --2- 172.21.15.1:0/2357904730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a592d840 0x55b5a59bb290 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 -- 172.21.15.1:0/2357904730 >> 172.21.15.1:0/2357904730 conn(0x55b5a5931e90 msgr2=0x55b5a5913c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 -- 172.21.15.1:0/2357904730 shutdown_connections 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.509+0000 7f710b692f40 1 -- 172.21.15.1:0/2357904730 wait complete. 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.510+0000 7f710b692f40 1 Processor -- start 2024-05-09T06:36:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.510+0000 7f710b692f40 1 -- start start 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a592d840 0x55b5a5928630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 0x55b5a58fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a59a4f40 0x55b5a58fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b5a5914bd0 con 0x55b5a5931290 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b5a59baf80 con 0x55b5a59a4f40 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f710b692f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b5a5915750 con 0x55b5a592d840 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 0x55b5a58fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 0x55b5a58fba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38180/0 (socket says 172.21.15.1:38180) 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 -- 172.21.15.1:0/3518369178 learned_addr learned my addr 172.21.15.1:0/3518369178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7109994640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a59a4f40 0x55b5a58fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7109193640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a592d840 0x55b5a5928630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 -- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a592d840 msgr2=0x55b5a5928630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b5a592d840 0x55b5a5928630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 -- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a59a4f40 msgr2=0x55b5a58fbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a59a4f40 0x55b5a58fbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 -- 172.21.15.1:0/3518369178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a5928b70 con 0x55b5a5931290 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7109994640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5a59a4f40 0x55b5a58fbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f7108992640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 0x55b5a58fba40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7100000f60 tx=0x7f710000a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.511+0000 7f70fa7fc640 1 -- 172.21.15.1:0/3518369178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710000f420 con 0x55b5a5931290 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.512+0000 7f70fa7fc640 1 -- 172.21.15.1:0/3518369178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7100014020 con 0x55b5a5931290 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.512+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5a58ff160 con 0x55b5a5931290 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.512+0000 7f70fa7fc640 1 -- 172.21.15.1:0/3518369178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710000f800 con 0x55b5a5931290 2024-05-09T06:36:53.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.512+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a58feba0 con 0x55b5a5931290 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.513+0000 7f70fa7fc640 1 -- 172.21.15.1:0/3518369178 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f710000f9a0 con 0x55b5a5931290 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.513+0000 7f70fa7fc640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f70ec06a810 0x7f70ec06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.513+0000 7f70fa7fc640 1 -- 172.21.15.1:0/3518369178 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f710008a0b0 con 0x55b5a5931290 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.513+0000 7f7109193640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f70ec06a810 0x7f70ec06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55b5a59ba1f0 0x55b5a58fad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55b5a5929310 con 0x55b5a59ba1f0 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f7109994640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55b5a59ba1f0 0x55b5a58fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f7109193640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f70ec06a810 0x7f70ec06ccd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f70fc067000 tx=0x7f70fc053000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f7109994640 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55b5a59ba1f0 0x55b5a58fad00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f7109994640 1 -- 172.21.15.1:0/3518369178 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f7104011040 con 0x55b5a59ba1f0 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55b5a59ba1f0 msgr2=0x55b5a58fad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55b5a59ba1f0 0x55b5a58fad00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f70ec06a810 msgr2=0x7f70ec06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f70ec06a810 0x7f70ec06ccd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f70fc067000 tx=0x7f70fc053000 comp rx=0 tx=0).stop 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 msgr2=0x55b5a58fba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.514+0000 7f710b692f40 1 --2- 172.21.15.1:0/3518369178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b5a5931290 0x55b5a58fba40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7100000f60 tx=0x7f710000a5f0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.515+0000 7f7109994640 1 -- 172.21.15.1:0/3518369178 reap_dead start 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.515+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 shutdown_connections 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.515+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 >> 172.21.15.1:0/3518369178 conn(0x55b5a5931e90 msgr2=0x55b5a58e42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.515+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 shutdown_connections 2024-05-09T06:36:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.515+0000 7f710b692f40 1 -- 172.21.15.1:0/3518369178 wait complete. 2024-05-09T06:36:53.501 INFO:teuthology.orchestra.run.smithi001.stdout:����� 2024-05-09T06:36:53.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.624+0000 7f0c11f3b640 1 Processor -- start 2024-05-09T06:36:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 -- start start 2024-05-09T06:36:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c161950 0x7f0c0c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0c0b33c0 con 0x7f0c0c0d4940 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c0d4dd0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c11f3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c0b3660 con 0x7f0c0c161950 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.625+0000 7f0c0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38194/0 (socket says 172.21.15.1:38194) 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 -- 172.21.15.1:0/45766252 learned_addr learned my addr 172.21.15.1:0/45766252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0b7fe640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c161950 0x7f0c0c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 -- 172.21.15.1:0/45766252 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c161950 msgr2=0x7f0c0c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c161950 0x7f0c0c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 -- 172.21.15.1:0/45766252 >> v1:172.21.15.6:6789/0 conn(0x7f0c0c0d4940 legacy=0x7f0c0c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.626+0000 7f0c0bfff640 1 -- 172.21.15.1:0/45766252 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c0b3940 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c0bfff640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b3590 tx=0x7f0c0402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8139e2e1f9ffce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c0a7fc640 1 -- 172.21.15.1:0/45766252 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04037070 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c0a7fc640 1 -- 172.21.15.1:0/45766252 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04032440 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c0a7fc640 1 -- 172.21.15.1:0/45766252 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04032810 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 -- 172.21.15.1:0/45766252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 msgr2=0x7f0c0c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b3590 tx=0x7f0c0402a250 comp rx=0 tx=0).stop 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 -- 172.21.15.1:0/45766252 shutdown_connections 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c0d2940 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/45766252 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c161950 0x7f0c0c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 -- 172.21.15.1:0/45766252 >> 172.21.15.1:0/45766252 conn(0x7f0c0c0d3e20 msgr2=0x7f0c0c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 -- 172.21.15.1:0/45766252 shutdown_connections 2024-05-09T06:36:53.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.627+0000 7f0c11f3b640 1 -- 172.21.15.1:0/45766252 wait complete. 2024-05-09T06:36:53.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.629+0000 7f0c11f3b640 1 Processor -- start 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.629+0000 7f0c11f3b640 1 -- start start 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 0x7f0c0c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c15d4c0 0x7f0c0c0caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0b52d0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0b5150 con 0x7f0c0c161950 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c0b5450 con 0x7f0c0c15d4c0 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38202/0 (socket says 172.21.15.1:38202) 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 -- 172.21.15.1:0/2387963011 learned_addr learned my addr 172.21.15.1:0/2387963011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0affd640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 0x7f0c0c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 -- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c15d4c0 msgr2=0x7f0c0c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0bfff640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c15d4c0 0x7f0c0c0caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c15d4c0 0x7f0c0c0caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 -- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 msgr2=0x7f0c0c15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 0x7f0c0c15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 -- 172.21.15.1:0/2387963011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c0b9720 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0affd640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 0x7f0c0c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c0b7fe640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00efc0 tx=0x7f0bfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc018020 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc014480 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc014850 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c0cbad0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.630+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c0cc090 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.632+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c15d8d0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.632+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bfc005d10 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.632+0000 7f0bf3fff640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 0x7f0bf806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.632+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f0bfc08bda0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.633+0000 7f0c0affd640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 0x7f0bf806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.633+0000 7f0c0affd640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 0x7f0bf806e320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b99b0 tx=0x7f0c00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.637+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c0c15d8d0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.859+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c0c0752b0 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.859+0000 7f0bf3fff640 1 -- 172.21.15.1:0/2387963011 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f0bfc022030 con 0x7f0c0c0d0f80 2024-05-09T06:36:53.840 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 msgr2=0x7f0bf806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 0x7f0bf806e320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b99b0 tx=0x7f0c00009040 comp rx=0 tx=0).stop 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 msgr2=0x7f0c0c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00efc0 tx=0x7f0bfc00c490 comp rx=0 tx=0).stop 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 shutdown_connections 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0bf806be60 0x7f0bf806e320 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c0c15d4c0 0x7f0c0c0caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c0c161950 0x7f0c0c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 --2- 172.21.15.1:0/2387963011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c0c0d0f80 0x7f0c0c15b410 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 >> 172.21.15.1:0/2387963011 conn(0x7f0c0c0d3e20 msgr2=0x7f0c0c15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 shutdown_connections 2024-05-09T06:36:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:53.861+0000 7f0c11f3b640 1 -- 172.21.15.1:0/2387963011 wait complete. 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:53.853 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:53.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:53.853 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:53.870 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:36:53.872 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:53.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:53.873 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:36:53.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:53.936 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:36:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:36:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:36:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:36:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:36:53.994 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:36:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:54.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:54.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.124 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:36:54.140 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:36:54.205 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:36:54.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:54.206 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:54.261 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:54.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:54.262 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:54.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:36:54.384 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:36:54.441 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:36:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:36:54.443 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:54.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.444 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:36:54.497 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:36:54.498 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:36:54.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:36:54.566 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:36:54.624 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:36:54.625 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:54.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.627 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:36:54.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:36:54.645+0000 7f1daca0c480 -1 init, newargv = 0x56372cd0c190 newargc=15 2024-05-09T06:36:54.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[75967]: starting ceph client 2024-05-09T06:36:54.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[75967]: starting fuse 2024-05-09T06:36:54.644 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:36:54.644 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:36:54.644 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:36:54.645 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:36:54.646 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:36:54.646 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:36:54.646 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:54.646 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:36:54.646 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:54.766 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.75967.asok 2024-05-09T06:36:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:54.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75967.asok 2024-05-09T06:36:54.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75967.asok status 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "75967", 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9117, 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 9117 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 817724608 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 817724608 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.9117 192.168.0.1:0/817724608", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/817724608", 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 82, 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 106, 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:36:54.921 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:36:54.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.927 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:36:54.927 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:36:54.943 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:54.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:54.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:55.010 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:55.010 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:55.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:55.067 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-09T06:36:55.165 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records in 2024-05-09T06:36:55.166 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records out 2024-05-09T06:36:55.166 INFO:teuthology.orchestra.run.smithi001.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0379014 s, 166 MB/s 2024-05-09T06:36:55.166 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:55.166 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:55.183 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-09T06:36:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records in 2024-05-09T06:36:55.283 INFO:teuthology.orchestra.run.smithi001.stderr:6+0 records out 2024-05-09T06:36:55.284 INFO:teuthology.orchestra.run.smithi001.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0411208 s, 153 MB/s 2024-05-09T06:36:55.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:55.284 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi001.stdout:./ 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi001.stdout:./file2_sixmegs 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi001.stdout:./file3_sixmegs 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi001.stdout:./file1_sixmegs 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-09T06:36:55.304 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:55.411 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:36:55.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-09T06:36:55.420 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:55.529 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:36:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-09T06:36:55.538 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627778 2024-05-09T06:36:55.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-09T06:36:55.655 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:36:55.763 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:36:55.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:55.773 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:55.773 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:55.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 Processor -- start 2024-05-09T06:36:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 -- start start 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fde90161950 0x7fde900d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde900b33c0 con 0x7fde900d4940 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde900b3660 con 0x7fde900d0f80 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.993+0000 7fde96c15640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fde900d4dd0 con 0x7fde90161950 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fde90161950 0x7fde900d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fde90161950 0x7fde900d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38210/0 (socket says 172.21.15.1:38210) 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 -- 172.21.15.1:0/3156858721 learned_addr learned my addr 172.21.15.1:0/3156858721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde8ffff640 1 --1- 172.21.15.1:0/3156858721 >> v1:172.21.15.1:6789/0 conn(0x7fde900d4940 0x7fde900cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:34842/0 (socket says 172.21.15.1:34842) 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 -- 172.21.15.1:0/3156858721 >> v1:172.21.15.6:6789/0 conn(0x7fde900d0f80 legacy=0x7fde900cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 -- 172.21.15.1:0/3156858721 >> v1:172.21.15.1:6789/0 conn(0x7fde900d4940 legacy=0x7fde900cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde8f7fe640 1 -- 172.21.15.1:0/3156858721 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 411442549 0 0) 0x7fde900b33c0 con 0x7fde900d4940 2024-05-09T06:36:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.994+0000 7fde949b1640 1 -- 172.21.15.1:0/3156858721 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900b3940 con 0x7fde90161950 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde949b1640 1 --2- 172.21.15.1:0/3156858721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900d2400 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fde88009870 tx=0x7fde8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5829346091b7e66c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde8f7fe640 1 -- 172.21.15.1:0/3156858721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88037070 con 0x7fde90161950 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde8f7fe640 1 -- 172.21.15.1:0/3156858721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8802ae50 con 0x7fde90161950 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde8f7fe640 1 -- 172.21.15.1:0/3156858721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88032860 con 0x7fde90161950 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 -- 172.21.15.1:0/3156858721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 msgr2=0x7fde900d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 --2- 172.21.15.1:0/3156858721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900d2400 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fde88009870 tx=0x7fde8802a300 comp rx=0 tx=0).stop 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 -- 172.21.15.1:0/3156858721 shutdown_connections 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 --2- 172.21.15.1:0/3156858721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900d2400 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 -- 172.21.15.1:0/3156858721 >> 172.21.15.1:0/3156858721 conn(0x7fde900d3e20 msgr2=0x7fde900d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.995+0000 7fde96c15640 1 -- 172.21.15.1:0/3156858721 shutdown_connections 2024-05-09T06:36:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.996+0000 7fde96c15640 1 -- 172.21.15.1:0/3156858721 wait complete. 2024-05-09T06:36:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.997+0000 7fde96c15640 1 Processor -- start 2024-05-09T06:36:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.997+0000 7fde96c15640 1 -- start start 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.997+0000 7fde96c15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde900d0f80 0x7fde90156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.997+0000 7fde96c15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 0x7fde9006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde900b5150 con 0x7fde90161950 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fde900cf850 con 0x7fde900d0f80 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fde900b52d0 con 0x7fde900caef0 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38224/0 (socket says 172.21.15.1:38224) 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 -- 172.21.15.1:0/3637293911 learned_addr learned my addr 172.21.15.1:0/3637293911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde949b1640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde900d0f80 0x7fde90156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde951b2640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 0x7fde9006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 -- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 msgr2=0x7fde9006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 0x7fde9006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 -- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde900d0f80 msgr2=0x7fde90156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde900d0f80 0x7fde90156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 -- 172.21.15.1:0/3637293911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900cce60 con 0x7fde90161950 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde951b2640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 0x7fde9006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8ffff640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c7e0 tx=0x7fde8000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8000a640 con 0x7fde90161950 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8000a7e0 con 0x7fde90161950 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde900cd100 con 0x7fde90161950 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8000abf0 con 0x7fde90161950 2024-05-09T06:36:55.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:55.998+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde900cd6c0 con 0x7fde90161950 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.000+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde900752b0 con 0x7fde90161950 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.000+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde80007070 con 0x7fde90161950 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.000+0000 7fde8d7fa640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 0x7fde7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.001+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fde8008aef0 con 0x7fde90161950 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.001+0000 7fde949b1640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 0x7fde7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.001+0000 7fde949b1640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 0x7fde7006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fde88034000 tx=0x7fde8802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.005+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde800597a0 con 0x7fde90161950 2024-05-09T06:36:56.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.187+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde900d21e0 con 0x7fde90161950 2024-05-09T06:36:56.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.188+0000 7fde8d7fa640 1 -- 172.21.15.1:0/3637293911 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16752 (secure 0 0 0) 0x7fde800590a0 con 0x7fde90161950 2024-05-09T06:36:56.169 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:56.169 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":106,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:36.717091+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:56.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.189+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 msgr2=0x7fde7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.189+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 0x7fde7006e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fde88034000 tx=0x7fde8802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 msgr2=0x7fde900cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c7e0 tx=0x7fde8000cca0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 shutdown_connections 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde7006be30 0x7fde7006e2f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde900caef0 0x7fde9006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde90161950 0x7fde900cc920 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 --2- 172.21.15.1:0/3637293911 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde900d0f80 0x7fde90156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 >> 172.21.15.1:0/3637293911 conn(0x7fde900d3e20 msgr2=0x7fde900cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 shutdown_connections 2024-05-09T06:36:56.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.190+0000 7fde96c15640 1 -- 172.21.15.1:0/3637293911 wait complete. 2024-05-09T06:36:56.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:33.397054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:34.058197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:56.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.308+0000 7fd546a5d640 1 Processor -- start 2024-05-09T06:36:56.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.308+0000 7fd546a5d640 1 -- start start 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd546a5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd540158a50 0x7fd5400718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd546a5d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd54006ede0 0x7fd540071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd546a5d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd54015b420 con 0x7fd54015c4a0 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd546a5d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd540158e30 con 0x7fd54006ede0 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd546a5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd540072480 con 0x7fd540158a50 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd54006ede0 0x7fd540071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd54006ede0 0x7fd540071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38234/0 (socket says 172.21.15.1:38234) 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 -- 172.21.15.1:0/1332426234 learned_addr learned my addr 172.21.15.1:0/1332426234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd53ffff640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd540158a50 0x7fd5400718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 -- 172.21.15.1:0/1332426234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd540158a50 msgr2=0x7fd5400718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd540158a50 0x7fd5400718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 -- 172.21.15.1:0/1332426234 >> v1:172.21.15.6:6789/0 conn(0x7fd54015c4a0 legacy=0x7fd54006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.309+0000 7fd544ffa640 1 -- 172.21.15.1:0/1332426234 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54015b6e0 con 0x7fd54006ede0 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.310+0000 7fd544ffa640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd54006ede0 0x7fd540071e00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd538009e50 tx=0x7fd53802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddfee1359929246 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.310+0000 7fd53effd640 1 -- 172.21.15.1:0/1332426234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538036070 con 0x7fd54006ede0 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.310+0000 7fd53effd640 1 -- 172.21.15.1:0/1332426234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd538031600 con 0x7fd54006ede0 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.310+0000 7fd53effd640 1 -- 172.21.15.1:0/1332426234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5380319d0 con 0x7fd54006ede0 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 -- 172.21.15.1:0/1332426234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd54006ede0 msgr2=0x7fd540071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd54006ede0 0x7fd540071e00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd538009e50 tx=0x7fd53802a710 comp rx=0 tx=0).stop 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 -- 172.21.15.1:0/1332426234 shutdown_connections 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd54006ede0 0x7fd540071e00 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 --2- 172.21.15.1:0/1332426234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd540158a50 0x7fd5400718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 -- 172.21.15.1:0/1332426234 >> 172.21.15.1:0/1332426234 conn(0x7fd5400cc4d0 msgr2=0x7fd5400cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 -- 172.21.15.1:0/1332426234 shutdown_connections 2024-05-09T06:36:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.311+0000 7fd546a5d640 1 -- 172.21.15.1:0/1332426234 wait complete. 2024-05-09T06:36:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.312+0000 7fd546a5d640 1 Processor -- start 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.312+0000 7fd546a5d640 1 -- start start 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd54006ede0 0x7fd540144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd54015c4a0 0x7fd5400b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd54015bf80 con 0x7fd540158a50 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd54015bca0 con 0x7fd54006ede0 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd546a5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd540070ea0 con 0x7fd54015c4a0 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38248/0 (socket says 172.21.15.1:38248) 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 -- 172.21.15.1:0/2714870792 learned_addr learned my addr 172.21.15.1:0/2714870792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53ffff640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd54006ede0 0x7fd540144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 -- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd54015c4a0 msgr2=0x7fd5400b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd54015c4a0 0x7fd5400b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 -- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd54006ede0 msgr2=0x7fd540144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd54006ede0 0x7fd540144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 -- 172.21.15.1:0/2714870792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd540144c90 con 0x7fd540158a50 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.313+0000 7fd53f7fe640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd52800c440 tx=0x7fd52800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.314+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5280052d0 con 0x7fd540158a50 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.314+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd528005470 con 0x7fd540158a50 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.314+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5400cf380 con 0x7fd540158a50 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.314+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528007260 con 0x7fd540158a50 2024-05-09T06:36:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.314+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54014d2e0 con 0x7fd540158a50 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.315+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5400752b0 con 0x7fd540158a50 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.315+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd528007400 con 0x7fd540158a50 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.315+0000 7fd53cff9640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.316+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd52808a460 con 0x7fd540158a50 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.316+0000 7fd53ffff640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.316+0000 7fd53ffff640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 0x7fd52006e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd530006fd0 tx=0x7fd530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.320+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd528058c80 con 0x7fd540158a50 2024-05-09T06:36:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.501+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd54006ebc0 con 0x7fd540158a50 2024-05-09T06:36:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.502+0000 7fd53cff9640 1 -- 172.21.15.1:0/2714870792 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fd528058580 con 0x7fd540158a50 2024-05-09T06:36:56.482 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:56.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 msgr2=0x7fd52006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 0x7fd52006e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd530006fd0 tx=0x7fd530008040 comp rx=0 tx=0).stop 2024-05-09T06:36:56.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 msgr2=0x7fd5400b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd52800c440 tx=0x7fd52800c900 comp rx=0 tx=0).stop 2024-05-09T06:36:56.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 shutdown_connections 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd54015c4a0 0x7fd5400b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd540158a50 0x7fd5400b43e0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 --2- 172.21.15.1:0/2714870792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd54006ede0 0x7fd540144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.503+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 >> 172.21.15.1:0/2714870792 conn(0x7fd5400cc4d0 msgr2=0x7fd54006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.504+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 shutdown_connections 2024-05-09T06:36:56.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.504+0000 7fd546a5d640 1 -- 172.21.15.1:0/2714870792 wait complete. 2024-05-09T06:36:56.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:56.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.621+0000 7f738086d640 1 Processor -- start 2024-05-09T06:36:56.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.621+0000 7f738086d640 1 -- start start 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f738086d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f737c15c4c0 0x7f737c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f738086d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f737c0b4de0 0x7f737c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f738086d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f737c06cf40 con 0x7f737c0b5260 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f738086d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f737c156980 con 0x7f737c15c4c0 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f738086d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f737c06d870 con 0x7f737c0b4de0 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f737c15c4c0 0x7f737c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f737c15c4c0 0x7f737c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38250/0 (socket says 172.21.15.1:38250) 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 -- 172.21.15.1:0/848023636 learned_addr learned my addr 172.21.15.1:0/848023636 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:56.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f7379d9b640 1 --2- 172.21.15.1:0/848023636 >> v2:172.21.15.6:3300/0 conn(0x7f737c0b4de0 0x7f737c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 -- 172.21.15.1:0/848023636 >> v2:172.21.15.6:3300/0 conn(0x7f737c0b4de0 msgr2=0x7f737c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 --2- 172.21.15.1:0/848023636 >> v2:172.21.15.6:3300/0 conn(0x7f737c0b4de0 0x7f737c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 -- 172.21.15.1:0/848023636 >> v1:172.21.15.6:6789/0 conn(0x7f737c0b5260 legacy=0x7f737c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.622+0000 7f737a59c640 1 -- 172.21.15.1:0/848023636 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c06d220 con 0x7f737c15c4c0 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.623+0000 7f737a59c640 1 --2- 172.21.15.1:0/848023636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c15c4c0 0x7f737c06dd00 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f737c0cd190 tx=0x7f736c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c56cd2fdc22665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.623+0000 7f737959a640 1 -- 172.21.15.1:0/848023636 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c037070 con 0x7f737c15c4c0 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.623+0000 7f737959a640 1 -- 172.21.15.1:0/848023636 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c032440 con 0x7f737c15c4c0 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.623+0000 7f737959a640 1 -- 172.21.15.1:0/848023636 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c032810 con 0x7f737c15c4c0 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 -- 172.21.15.1:0/848023636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c15c4c0 msgr2=0x7f737c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 --2- 172.21.15.1:0/848023636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c15c4c0 0x7f737c06dd00 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f737c0cd190 tx=0x7f736c02a230 comp rx=0 tx=0).stop 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 -- 172.21.15.1:0/848023636 shutdown_connections 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 --2- 172.21.15.1:0/848023636 >> v2:172.21.15.6:3300/0 conn(0x7f737c0b4de0 0x7f737c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 --2- 172.21.15.1:0/848023636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c15c4c0 0x7f737c06dd00 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 -- 172.21.15.1:0/848023636 >> 172.21.15.1:0/848023636 conn(0x7f737c0cc4d0 msgr2=0x7f737c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 -- 172.21.15.1:0/848023636 shutdown_connections 2024-05-09T06:36:56.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.624+0000 7f738086d640 1 -- 172.21.15.1:0/848023636 wait complete. 2024-05-09T06:36:56.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.625+0000 7f738086d640 1 Processor -- start 2024-05-09T06:36:56.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.625+0000 7f738086d640 1 -- start start 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f737c0b5260 0x7f737c1f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f737c15c4c0 0x7f737c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f737c0b2dc0 con 0x7f737c0b4de0 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f737c0b2c40 con 0x7f737c15c4c0 2024-05-09T06:36:56.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f738086d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f737c15cde0 con 0x7f737c0b5260 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f737a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f737a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38254/0 (socket says 172.21.15.1:38254) 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.626+0000 7f737a59c640 1 -- 172.21.15.1:0/1998785373 learned_addr learned my addr 172.21.15.1:0/1998785373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 -- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f737c0b5260 msgr2=0x7f737c1f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f737c0b5260 0x7f737c1f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 -- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f737c15c4c0 msgr2=0x7f737c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737ad9d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f737c15c4c0 0x7f737c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f737c15c4c0 0x7f737c15af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 -- 172.21.15.1:0/1998785373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c142750 con 0x7f737c0b4de0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f737a59c640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f736c000c00 tx=0x7f736c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c041020 con 0x7f737c0b4de0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c0306c0 con 0x7f737c0b4de0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c1f7610 con 0x7f737c0b4de0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c037040 con 0x7f737c0b4de0 2024-05-09T06:36:56.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.627+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c13f260 con 0x7f737c0b4de0 2024-05-09T06:36:56.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.628+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c0752b0 con 0x7f737c0b4de0 2024-05-09T06:36:56.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.629+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f736c044020 con 0x7f737c0b4de0 2024-05-09T06:36:56.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.629+0000 7f736affd640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.629+0000 7f7379d9b640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.629+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f736c0ab7c0 con 0x7f737c0b4de0 2024-05-09T06:36:56.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.630+0000 7f7379d9b640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 0x7f735c06e2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f737c1362a0 tx=0x7f7370008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.634+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f736c04b030 con 0x7f737c0b4de0 2024-05-09T06:36:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.819+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f737c0cc8b0 con 0x7f737c0b4de0 2024-05-09T06:36:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.819+0000 7f736affd640 1 -- 172.21.15.1:0/1998785373 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f736c07ee20 con 0x7f737c0b4de0 2024-05-09T06:36:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 msgr2=0x7f735c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 0x7f735c06e2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f737c1362a0 tx=0x7f7370008040 comp rx=0 tx=0).stop 2024-05-09T06:36:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 msgr2=0x7f737c13fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f736c000c00 tx=0x7f736c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 shutdown_connections 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f737c15c4c0 0x7f737c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f737c0b5260 0x7f737c1f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 --2- 172.21.15.1:0/1998785373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f737c0b4de0 0x7f737c13fd40 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 >> 172.21.15.1:0/1998785373 conn(0x7f737c0cc4d0 msgr2=0x7f737c0ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.821+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 shutdown_connections 2024-05-09T06:36:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.822+0000 7f738086d640 1 -- 172.21.15.1:0/1998785373 wait complete. 2024-05-09T06:36:56.813 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:56.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.939+0000 7f6cefb4f640 1 Processor -- start 2024-05-09T06:36:56.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.939+0000 7f6cefb4f640 1 -- start start 2024-05-09T06:36:56.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cefb4f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6ce8071fd0 0x7f6ce806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cefb4f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce815b420 con 0x7f6ce8158a50 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cefb4f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce815b6e0 con 0x7f6ce815c4a0 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cefb4f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce8158e30 con 0x7f6ce8071fd0 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6ced8eb640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6ce8158a50 0x7f6ce80718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:34858/0 (socket says 172.21.15.1:34858) 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6ced8eb640 1 -- 172.21.15.1:0/2642466096 learned_addr learned my addr 172.21.15.1:0/2642466096 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6ced0ea640 1 --2- 172.21.15.1:0/2642466096 >> v2:172.21.15.6:3300/0 conn(0x7f6ce8071fd0 0x7f6ce806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136209897 0 0) 0x7f6ce815b420 con 0x7f6ce8158a50 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.940+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cd0003520 con 0x7f6ce8158a50 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.941+0000 7f6ced0ea640 1 -- 172.21.15.1:0/2642466096 >> v1:172.21.15.6:6789/0 conn(0x7f6ce815c4a0 legacy=0x7f6ce815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.941+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796593586 0 0) 0x7f6ce815b6e0 con 0x7f6ce815c4a0 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.941+0000 7f6ced0ea640 1 -- 172.21.15.1:0/2642466096 >> v1:172.21.15.1:6789/0 conn(0x7f6ce8158a50 legacy=0x7f6ce80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.941+0000 7f6ced0ea640 1 -- 172.21.15.1:0/2642466096 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce815b9c0 con 0x7f6ce8071fd0 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.941+0000 7f6ced0ea640 1 --2- 172.21.15.1:0/2642466096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8071fd0 0x7f6ce806e6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13634a51e9bbc73e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc037070 con 0x7f6ce8071fd0 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc02ac30 con 0x7f6ce8071fd0 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cec8e9640 1 -- 172.21.15.1:0/2642466096 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0001670 con 0x7f6ce8071fd0 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2642466096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8071fd0 msgr2=0x7f6ce806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2642466096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8071fd0 0x7f6ce806e6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2642466096 shutdown_connections 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2642466096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8071fd0 0x7f6ce806e6d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2642466096 >> 172.21.15.1:0/2642466096 conn(0x7f6ce80cc4d0 msgr2=0x7f6ce80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2642466096 shutdown_connections 2024-05-09T06:36:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.942+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2642466096 wait complete. 2024-05-09T06:36:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.943+0000 7f6cefb4f640 1 Processor -- start 2024-05-09T06:36:56.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 -- start start 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8158a50 0x7f6ce81f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce815c4a0 0x7f6ce81f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce815bf80 con 0x7f6ce8071fd0 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8072480 con 0x7f6ce8158a50 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6cefb4f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce8070ea0 con 0x7f6ce815c4a0 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6ced8eb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6ced8eb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38256/0 (socket says 172.21.15.1:38256) 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.944+0000 7f6ced8eb640 1 -- 172.21.15.1:0/2794903554 learned_addr learned my addr 172.21.15.1:0/2794903554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced0ea640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8158a50 0x7f6ce81f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6cee0ec640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce815c4a0 0x7f6ce81f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 -- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce815c4a0 msgr2=0x7f6ce81f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce815c4a0 0x7f6ce81f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 -- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8158a50 msgr2=0x7f6ce81f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8158a50 0x7f6ce81f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 -- 172.21.15.1:0/2794903554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce815d550 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ced8eb640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c790 tx=0x7f6ce000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0018070 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce00074e0 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce81f7dd0 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0004a00 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.945+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce80b3c20 con 0x7f6ce8071fd0 2024-05-09T06:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.946+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce80752b0 con 0x7f6ce8071fd0 2024-05-09T06:36:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.947+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ce001b070 con 0x7f6ce8071fd0 2024-05-09T06:36:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.947+0000 7f6ce67fc640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 0x7f6cd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.947+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f6ce008a5c0 con 0x7f6ce8071fd0 2024-05-09T06:36:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.947+0000 7f6ced0ea640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 0x7f6cd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.948+0000 7f6ced0ea640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 0x7f6cd406e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc034000 tx=0x7f6cdc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:56.952+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ce81f7dd0 con 0x7f6ce8071fd0 2024-05-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.135+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ce8153b50 con 0x7f6ce8071fd0 2024-05-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.135+0000 7f6ce67fc640 1 -- 172.21.15.1:0/2794903554 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f6ce0058de0 con 0x7f6ce8071fd0 2024-05-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:57.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 msgr2=0x7f6cd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 0x7f6cd406e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc034000 tx=0x7f6cdc02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:57.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 msgr2=0x7f6ce81556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c790 tx=0x7f6ce000cc50 comp rx=0 tx=0).stop 2024-05-09T06:36:57.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 shutdown_connections 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6cd406be30 0x7f6cd406e2f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ce815c4a0 0x7f6ce81f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ce8158a50 0x7f6ce81f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 --2- 172.21.15.1:0/2794903554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ce8071fd0 0x7f6ce81556c0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 >> 172.21.15.1:0/2794903554 conn(0x7f6ce80cc4d0 msgr2=0x7f6ce8159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 shutdown_connections 2024-05-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.137+0000 7f6cefb4f640 1 -- 172.21.15.1:0/2794903554 wait complete. 2024-05-09T06:36:57.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:57.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.254+0000 7f36547ee640 1 Processor -- start 2024-05-09T06:36:57.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.254+0000 7f36547ee640 1 -- start start 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f36547ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f364c15e1d0 0x7f364c06f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f36547ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 0x7f364c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f36547ee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364c06c2f0 con 0x7f364c15e5b0 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f36547ee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f364c0cc3c0 con 0x7f364c15e1d0 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f36547ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f364c0ccc60 con 0x7f364c0d2890 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f364c15e1d0 0x7f364c06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f364c15e1d0 0x7f364c06f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38270/0 (socket says 172.21.15.1:38270) 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 -- 172.21.15.1:0/1667452963 learned_addr learned my addr 172.21.15.1:0/1667452963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f3652d8b640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 0x7f364c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 -- 172.21.15.1:0/1667452963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 msgr2=0x7f364c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 0x7f364c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 -- 172.21.15.1:0/1667452963 >> v1:172.21.15.1:6789/0 conn(0x7f364c15e5b0 legacy=0x7f364c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f3651588640 1 -- 172.21.15.1:0/1667452963 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943700758 0 0) 0x7f363c001110 con 0x7f364c15e5b0 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f365258a640 1 -- 172.21.15.1:0/1667452963 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c06f500 con 0x7f364c15e1d0 2024-05-09T06:36:57.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.255+0000 7f3652d8b640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 0x7f364c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.256+0000 7f365258a640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e1d0 0x7f364c06f890 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3648009870 tx=0x7f364802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe280f29775a7555 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.256+0000 7f3651588640 1 -- 172.21.15.1:0/1667452963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648037070 con 0x7f364c15e1d0 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.256+0000 7f3651588640 1 -- 172.21.15.1:0/1667452963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3648032440 con 0x7f364c15e1d0 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.256+0000 7f3651588640 1 -- 172.21.15.1:0/1667452963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648032810 con 0x7f364c15e1d0 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 -- 172.21.15.1:0/1667452963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e1d0 msgr2=0x7f364c06f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e1d0 0x7f364c06f890 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3648009870 tx=0x7f364802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 -- 172.21.15.1:0/1667452963 shutdown_connections 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c0d2890 0x7f364c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 --2- 172.21.15.1:0/1667452963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e1d0 0x7f364c06f890 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 -- 172.21.15.1:0/1667452963 >> 172.21.15.1:0/1667452963 conn(0x7f364c0d1d70 msgr2=0x7f364c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 -- 172.21.15.1:0/1667452963 shutdown_connections 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.257+0000 7f36547ee640 1 -- 172.21.15.1:0/1667452963 wait complete. 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.258+0000 7f36547ee640 1 Processor -- start 2024-05-09T06:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.258+0000 7f36547ee640 1 -- start start 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f364c0d2890 0x7f364c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 0x7f364c0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f364c06f1b0 con 0x7f364c15e5b0 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f364c0cc8f0 con 0x7f364c0d2890 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f36547ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f364c06fdd0 con 0x7f364c15e1d0 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38280/0 (socket says 172.21.15.1:38280) 2024-05-09T06:36:57.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 -- 172.21.15.1:0/88088235 learned_addr learned my addr 172.21.15.1:0/88088235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f365258a640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f364c0d2890 0x7f364c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3651d89640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 0x7f364c0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 -- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 msgr2=0x7f364c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 0x7f364c0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 -- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f364c0d2890 msgr2=0x7f364c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f364c0d2890 0x7f364c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 -- 172.21.15.1:0/88088235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c1578e0 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3651d89640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 0x7f364c0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.259+0000 7f3652d8b640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f364400cea0 tx=0x7f364400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.260+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644003b90 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.260+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644003d50 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.260+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c157f90 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.260+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644004260 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.260+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c158550 con 0x7f364c15e5b0 2024-05-09T06:36:57.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.261+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364c0752b0 con 0x7f364c15e5b0 2024-05-09T06:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.261+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3644004400 con 0x7f364c15e5b0 2024-05-09T06:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.262+0000 7f3642ffd640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 0x7f363006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.262+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f364408ba70 con 0x7f364c15e5b0 2024-05-09T06:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.262+0000 7f365258a640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 0x7f363006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.262+0000 7f365258a640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 0x7f363006e2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3648034000 tx=0x7f364802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.266+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f364c1578e0 con 0x7f364c15e5b0 2024-05-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.460+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f364c0d2280 con 0x7f364c15e5b0 2024-05-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.460+0000 7f3642ffd640 1 -- 172.21.15.1:0/88088235 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f364405a200 con 0x7f364c15e5b0 2024-05-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:36:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 msgr2=0x7f363006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 0x7f363006e2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3648034000 tx=0x7f364802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 msgr2=0x7f364c0cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f364400cea0 tx=0x7f364400cf90 comp rx=0 tx=0).stop 2024-05-09T06:36:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 shutdown_connections 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f363006be30 0x7f363006e2f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f364c15e5b0 0x7f364c0cb720 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f364c15e1d0 0x7f364c0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 --2- 172.21.15.1:0/88088235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f364c0d2890 0x7f364c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 >> 172.21.15.1:0/88088235 conn(0x7f364c0d1d70 msgr2=0x7f364c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 shutdown_connections 2024-05-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.462+0000 7f36547ee640 1 -- 172.21.15.1:0/88088235 wait complete. 2024-05-09T06:36:57.454 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:57.454 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:57.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.633+0000 7f75272ae640 1 Processor -- start 2024-05-09T06:36:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.634+0000 7f75272ae640 1 -- start start 2024-05-09T06:36:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.634+0000 7f75272ae640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f75200d2b10 0x7f75200b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.634+0000 7f75272ae640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75200b33c0 con 0x7f7520161950 2024-05-09T06:36:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.634+0000 7f75272ae640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75200b3660 con 0x7f75200d4940 2024-05-09T06:36:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.634+0000 7f75272ae640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f75200d4dd0 con 0x7f75200d2b10 2024-05-09T06:36:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752584b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f75200d2b10 0x7f75200b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752504a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7520161950 0x7f75200d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:34876/0 (socket says 172.21.15.1:34876) 2024-05-09T06:36:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752504a640 1 -- 172.21.15.1:0/2195021171 learned_addr learned my addr 172.21.15.1:0/2195021171 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752584b640 1 --2- 172.21.15.1:0/2195021171 >> v2:172.21.15.1:3300/0 conn(0x7f75200d2b10 0x7f75200b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38290/0 (socket says 172.21.15.1:38290) 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752584b640 1 -- 172.21.15.1:0/2195021171 >> v1:172.21.15.6:6789/0 conn(0x7f75200d4940 legacy=0x7f75200b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752584b640 1 -- 172.21.15.1:0/2195021171 >> v1:172.21.15.1:6789/0 conn(0x7f7520161950 legacy=0x7f75200d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.635+0000 7f752584b640 1 -- 172.21.15.1:0/2195021171 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200b3940 con 0x7f75200d2b10 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f752584b640 1 --2- 172.21.15.1:0/2195021171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75200d2b10 0x7f75200b4850 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f752008e540 tx=0x7f751802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d5f757b96bdd70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f7517fff640 1 -- 172.21.15.1:0/2195021171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751802ae60 con 0x7f75200d2b10 2024-05-09T06:36:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f7517fff640 1 -- 172.21.15.1:0/2195021171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7518007740 con 0x7f75200d2b10 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f7517fff640 1 -- 172.21.15.1:0/2195021171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518007b10 con 0x7f75200d2b10 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 -- 172.21.15.1:0/2195021171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75200d2b10 msgr2=0x7f75200b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 --2- 172.21.15.1:0/2195021171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75200d2b10 0x7f75200b4850 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f752008e540 tx=0x7f751802a680 comp rx=0 tx=0).stop 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 -- 172.21.15.1:0/2195021171 shutdown_connections 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 --2- 172.21.15.1:0/2195021171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75200d2b10 0x7f75200b4850 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 -- 172.21.15.1:0/2195021171 >> 172.21.15.1:0/2195021171 conn(0x7f75200d3e20 msgr2=0x7f75200d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 -- 172.21.15.1:0/2195021171 shutdown_connections 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.636+0000 7f75272ae640 1 -- 172.21.15.1:0/2195021171 wait complete. 2024-05-09T06:36:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 Processor -- start 2024-05-09T06:36:57.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 -- start start 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75200d2b10 0x7f75200cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75200cb430 0x7f75200b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75200d1100 con 0x7f7520161950 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f75200d0f80 con 0x7f75200d2b10 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f75272ae640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75200d1c00 con 0x7f75200cb430 2024-05-09T06:36:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f7524849640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.638+0000 7f7524849640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38306/0 (socket says 172.21.15.1:38306) 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 -- 172.21.15.1:0/2770467006 learned_addr learned my addr 172.21.15.1:0/2770467006 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f752504a640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75200d2b10 0x7f75200cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f752584b640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75200cb430 0x7f75200b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 -- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75200cb430 msgr2=0x7f75200b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75200cb430 0x7f75200b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 -- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75200d2b10 msgr2=0x7f75200cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75200d2b10 0x7f75200cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 -- 172.21.15.1:0/2770467006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200b7d50 con 0x7f7520161950 2024-05-09T06:36:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7524849640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f751000c430 tx=0x7f751000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510011790 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7510017070 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510011bb0 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75200cbad0 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.639+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75200cc090 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.640+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7520073fe0 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.641+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f751001c460 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.641+0000 7f7515ffb640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.641+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f751008ae80 con 0x7f7520161950 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.641+0000 7f752504a640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.642+0000 7f752504a640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 0x7f750406e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f751c007fd0 tx=0x7f751c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.646+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7520073fe0 con 0x7f7520161950 2024-05-09T06:36:57.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.834+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7520160050 con 0x7f7520161950 2024-05-09T06:36:57.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.835+0000 7f7515ffb640 1 -- 172.21.15.1:0/2770467006 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16752 (secure 0 0 0) 0x7f7520160050 con 0x7f7520161950 2024-05-09T06:36:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":106,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:36:36.717091+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 msgr2=0x7f750406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 0x7f750406e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f751c007fd0 tx=0x7f751c007e80 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 msgr2=0x7f75200caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f751000c430 tx=0x7f751000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 shutdown_connections 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75200cb430 0x7f75200b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7520161950 0x7f75200caef0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 --2- 172.21.15.1:0/2770467006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75200d2b10 0x7f75200cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 >> 172.21.15.1:0/2770467006 conn(0x7f75200d3e20 msgr2=0x7f75200d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 shutdown_connections 2024-05-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.837+0000 7f75272ae640 1 -- 172.21.15.1:0/2770467006 wait complete. 2024-05-09T06:36:57.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:33.397054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:34.058197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:57.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.955+0000 7fd542c76640 1 Processor -- start 2024-05-09T06:36:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.955+0000 7fd542c76640 1 -- start start 2024-05-09T06:36:57.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd542c76640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 0x7fd53c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd542c76640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd53c0ccbf0 0x7fd53c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd542c76640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd53c0cd5c0 con 0x7fd53c0ce600 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd542c76640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd53c0d2ba0 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd542c76640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd53c0ae9a0 con 0x7fd53c15c000 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd53bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd53c0ccbf0 0x7fd53c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd53bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd53c0ccbf0 0x7fd53c0cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53286/0 (socket says 172.21.15.1:53286) 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.956+0000 7fd53bfff640 1 -- 172.21.15.1:0/209757807 learned_addr learned my addr 172.21.15.1:0/209757807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd540a12640 1 --2- 172.21.15.1:0/209757807 >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 0x7fd53c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd53bfff640 1 -- 172.21.15.1:0/209757807 >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 msgr2=0x7fd53c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd53bfff640 1 --2- 172.21.15.1:0/209757807 >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 0x7fd53c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd53bfff640 1 -- 172.21.15.1:0/209757807 >> v1:172.21.15.6:6789/0 conn(0x7fd53c0ce600 legacy=0x7fd53c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd53bfff640 1 -- 172.21.15.1:0/209757807 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c0d0390 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd540a12640 1 --2- 172.21.15.1:0/209757807 >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 0x7fd53c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.957+0000 7fd53bfff640 1 --2- 172.21.15.1:0/209757807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c0cd080 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd53c073320 tx=0x7fd52402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d19960d013b38b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd53b7fe640 1 -- 172.21.15.1:0/209757807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037070 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd53b7fe640 1 -- 172.21.15.1:0/209757807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524032440 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd53b7fe640 1 -- 172.21.15.1:0/209757807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524032810 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 -- 172.21.15.1:0/209757807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c0ccbf0 msgr2=0x7fd53c0cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 --2- 172.21.15.1:0/209757807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c0cd080 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd53c073320 tx=0x7fd52402a200 comp rx=0 tx=0).stop 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 -- 172.21.15.1:0/209757807 shutdown_connections 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 --2- 172.21.15.1:0/209757807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c0cd080 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 --2- 172.21.15.1:0/209757807 >> v2:172.21.15.6:3300/0 conn(0x7fd53c15c000 0x7fd53c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 -- 172.21.15.1:0/209757807 >> 172.21.15.1:0/209757807 conn(0x7fd53c0d1c10 msgr2=0x7fd53c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 -- 172.21.15.1:0/209757807 shutdown_connections 2024-05-09T06:36:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.958+0000 7fd542c76640 1 -- 172.21.15.1:0/209757807 wait complete. 2024-05-09T06:36:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.959+0000 7fd542c76640 1 Processor -- start 2024-05-09T06:36:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.959+0000 7fd542c76640 1 -- start start 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c06f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c0ce600 0x7fd53c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c15c3e0 con 0x7fd53c15c000 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c0d0950 con 0x7fd53c0ccbf0 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd542c76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd53c0b0de0 con 0x7fd53c0ce600 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd541213640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.960+0000 7fd541213640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53296/0 (socket says 172.21.15.1:53296) 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 -- 172.21.15.1:0/1734150305 learned_addr learned my addr 172.21.15.1:0/1734150305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd540a12640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c06f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd53bfff640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c0ce600 0x7fd53c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 -- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c0ce600 msgr2=0x7fd53c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c0ce600 0x7fd53c06f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 -- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c0ccbf0 msgr2=0x7fd53c06f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c06f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 -- 172.21.15.1:0/1734150305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c070c60 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd541213640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b9a0 tx=0x7fd53400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534019070 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53400cba0 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534014440 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c070050 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.961+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c070610 con 0x7fd53c15c000 2024-05-09T06:36:57.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.962+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c0752b0 con 0x7fd53c15c000 2024-05-09T06:36:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.963+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd53401c070 con 0x7fd53c15c000 2024-05-09T06:36:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.963+0000 7fd5397fa640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 0x7fd51c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.963+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd53408ab50 con 0x7fd53c15c000 2024-05-09T06:36:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.963+0000 7fd540a12640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 0x7fd51c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.964+0000 7fd540a12640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 0x7fd51c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00aac0 tx=0x7fd52c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:57.968+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd534058d20 con 0x7fd53c15c000 2024-05-09T06:36:58.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.148+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd53c1f2c40 con 0x7fd53c15c000 2024-05-09T06:36:58.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.149+0000 7fd5397fa640 1 -- 172.21.15.1:0/1734150305 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fd53405f070 con 0x7fd53c15c000 2024-05-09T06:36:58.130 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 msgr2=0x7fd51c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 0x7fd51c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00aac0 tx=0x7fd52c008040 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 msgr2=0x7fd53c09df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b9a0 tx=0x7fd53400be60 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 shutdown_connections 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd51c06be30 0x7fd51c06e2f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd53c15c000 0x7fd53c09df90 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c0ce600 0x7fd53c06f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 --2- 172.21.15.1:0/1734150305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c0ccbf0 0x7fd53c06f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 >> 172.21.15.1:0/1734150305 conn(0x7fd53c0d1c10 msgr2=0x7fd53c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 shutdown_connections 2024-05-09T06:36:58.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.151+0000 7fd542c76640 1 -- 172.21.15.1:0/1734150305 wait complete. 2024-05-09T06:36:58.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.269+0000 7f902e264640 1 Processor -- start 2024-05-09T06:36:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.269+0000 7f902e264640 1 -- start start 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902e264640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90280b11f0 0x7f90280b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902e264640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90280cc8e0 0x7f90280ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902e264640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f902815e900 con 0x7f902815e390 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902e264640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f902815ebe0 con 0x7f90280b11f0 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902e264640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f902815eec0 con 0x7f90280cc8e0 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f90277fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90280b11f0 0x7f90280b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f9027fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f902815e390 0x7f90280cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54912/0 (socket says 172.21.15.1:54912) 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f9027fff640 1 -- 172.21.15.1:0/1196254421 learned_addr learned my addr 172.21.15.1:0/1196254421 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f902c801640 1 --2- 172.21.15.1:0/1196254421 >> v2:172.21.15.6:3300/0 conn(0x7f90280cc8e0 0x7f90280ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f90277fe640 1 -- 172.21.15.1:0/1196254421 >> v2:172.21.15.6:3300/0 conn(0x7f90280cc8e0 msgr2=0x7f90280ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f90277fe640 1 --2- 172.21.15.1:0/1196254421 >> v2:172.21.15.6:3300/0 conn(0x7f90280cc8e0 0x7f90280ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f90277fe640 1 -- 172.21.15.1:0/1196254421 >> v1:172.21.15.1:6789/0 conn(0x7f902815e390 legacy=0x7f90280cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f9026ffd640 1 -- 172.21.15.1:0/1196254421 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226387112 0 0) 0x7f902815e900 con 0x7f902815e390 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.270+0000 7f90277fe640 1 -- 172.21.15.1:0/1196254421 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902815f1a0 con 0x7f90280b11f0 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.271+0000 7f90277fe640 1 --2- 172.21.15.1:0/1196254421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280b11f0 0x7f90280b1680 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f90280721e0 tx=0x7f901002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc7f1c5c0870725 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.271+0000 7f9026ffd640 1 -- 172.21.15.1:0/1196254421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010037070 con 0x7f90280b11f0 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.271+0000 7f9026ffd640 1 -- 172.21.15.1:0/1196254421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9010032440 con 0x7f90280b11f0 2024-05-09T06:36:58.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.271+0000 7f9026ffd640 1 -- 172.21.15.1:0/1196254421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010032810 con 0x7f90280b11f0 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 -- 172.21.15.1:0/1196254421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280b11f0 msgr2=0x7f90280b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 --2- 172.21.15.1:0/1196254421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280b11f0 0x7f90280b1680 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f90280721e0 tx=0x7f901002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 -- 172.21.15.1:0/1196254421 shutdown_connections 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 --2- 172.21.15.1:0/1196254421 >> v2:172.21.15.6:3300/0 conn(0x7f90280cc8e0 0x7f90280ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 --2- 172.21.15.1:0/1196254421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280b11f0 0x7f90280b1680 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 -- 172.21.15.1:0/1196254421 >> 172.21.15.1:0/1196254421 conn(0x7f90280b1da0 msgr2=0x7f90280b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 -- 172.21.15.1:0/1196254421 shutdown_connections 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.272+0000 7f902e264640 1 -- 172.21.15.1:0/1196254421 wait complete. 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.273+0000 7f902e264640 1 Processor -- start 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.273+0000 7f902e264640 1 -- start start 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 0x7f902806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 0x7f90280b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90280cffb0 con 0x7f90280cc8e0 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9028162ba0 con 0x7f902815e390 2024-05-09T06:36:58.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902e264640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90280cfb60 con 0x7f90280b11f0 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f90277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f90277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53316/0 (socket says 172.21.15.1:53316) 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f90277fe640 1 -- 172.21.15.1:0/3641546605 learned_addr learned my addr 172.21.15.1:0/3641546605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.274+0000 7f902c801640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 0x7f90280b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f9027fff640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 0x7f902806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 -- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 msgr2=0x7f902806eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 0x7f902806eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 -- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 msgr2=0x7f90280b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 0x7f90280b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 -- 172.21.15.1:0/3641546605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90280aec80 con 0x7f90280cc8e0 2024-05-09T06:36:58.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f902c801640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 0x7f90280b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f9027fff640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 0x7f902806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f90277fe640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9010009890 tx=0x7f90100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010037040 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9010042070 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90280aef60 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010030bd0 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.275+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90280af520 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.276+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90280752b0 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.277+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9010047460 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.277+0000 7f9024ff9640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 0x7f900806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.277+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f90100abcc0 con 0x7f90280cc8e0 2024-05-09T06:36:58.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.277+0000 7f9027fff640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 0x7f900806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.278+0000 7f9027fff640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 0x7f900806e2f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9018000f60 tx=0x7f9018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.281+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9010032e20 con 0x7f90280cc8e0 2024-05-09T06:36:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.465+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90280b61f0 con 0x7f90280cc8e0 2024-05-09T06:36:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.466+0000 7f9024ff9640 1 -- 172.21.15.1:0/3641546605 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f901007a450 con 0x7f90280cc8e0 2024-05-09T06:36:58.447 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:36:58.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 msgr2=0x7f900806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 0x7f900806e2f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9018000f60 tx=0x7f9018009040 comp rx=0 tx=0).stop 2024-05-09T06:36:58.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 msgr2=0x7f90281f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9010009890 tx=0x7f90100046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 shutdown_connections 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f900806be30 0x7f900806e2f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f902815e390 0x7f90280b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90280cc8e0 0x7f90281f3920 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 --2- 172.21.15.1:0/3641546605 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90280b11f0 0x7f902806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 >> 172.21.15.1:0/3641546605 conn(0x7f90280b1da0 msgr2=0x7f9028160080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 shutdown_connections 2024-05-09T06:36:58.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.468+0000 7f902e264640 1 -- 172.21.15.1:0/3641546605 wait complete. 2024-05-09T06:36:58.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:58.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.586+0000 7f2192970640 1 Processor -- start 2024-05-09T06:36:58.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.586+0000 7f2192970640 1 -- start start 2024-05-09T06:36:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2192970640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f218c15e390 0x7f218c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2192970640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0cf990 0x7f218c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2192970640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c15e900 con 0x7f218c0b1730 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2192970640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f218c0b1ba0 con 0x7f218c15e390 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2192970640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f218c15ebe0 con 0x7f218c0cf990 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f218bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f218c15e390 0x7f218c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2190f0d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f218c0b1730 0x7f218c072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54916/0 (socket says 172.21.15.1:54916) 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2190f0d640 1 -- 172.21.15.1:0/554900627 learned_addr learned my addr 172.21.15.1:0/554900627 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f218bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f218c15e390 0x7f218c0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53320/0 (socket says 172.21.15.1:53320) 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.587+0000 7f2183fff640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0cf990 0x7f218c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218bfff640 1 -- 172.21.15.1:0/554900627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0cf990 msgr2=0x7f218c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218bfff640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0cf990 0x7f218c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218bfff640 1 -- 172.21.15.1:0/554900627 >> v1:172.21.15.1:6789/0 conn(0x7f218c0b1730 legacy=0x7f218c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218b7fe640 1 -- 172.21.15.1:0/554900627 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561617617 0 0) 0x7f218c15e900 con 0x7f218c0b1730 2024-05-09T06:36:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218bfff640 1 -- 172.21.15.1:0/554900627 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c15eec0 con 0x7f218c15e390 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.588+0000 7f218bfff640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c15e390 0x7f218c0cf450 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f217c0097a0 tx=0x7f217c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b5420f27368318 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f218b7fe640 1 -- 172.21.15.1:0/554900627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c037070 con 0x7f218c15e390 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f218b7fe640 1 -- 172.21.15.1:0/554900627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c032440 con 0x7f218c15e390 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f218b7fe640 1 -- 172.21.15.1:0/554900627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c032810 con 0x7f218c15e390 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 -- 172.21.15.1:0/554900627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c15e390 msgr2=0x7f218c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c15e390 0x7f218c0cf450 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f217c0097a0 tx=0x7f217c02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 -- 172.21.15.1:0/554900627 shutdown_connections 2024-05-09T06:36:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0cf990 0x7f218c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 --2- 172.21.15.1:0/554900627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c15e390 0x7f218c0cf450 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 -- 172.21.15.1:0/554900627 >> 172.21.15.1:0/554900627 conn(0x7f218c0b1da0 msgr2=0x7f218c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 -- 172.21.15.1:0/554900627 shutdown_connections 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.589+0000 7f2192970640 1 -- 172.21.15.1:0/554900627 wait complete. 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.590+0000 7f2192970640 1 Processor -- start 2024-05-09T06:36:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.590+0000 7f2192970640 1 -- start start 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0b1730 0x7f218c0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f218c15e390 0x7f218c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f218c0cd8f0 con 0x7f218c0cf990 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f218c15f6c0 con 0x7f218c15e390 2024-05-09T06:36:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2192970640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f218c162ba0 con 0x7f218c0b1730 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2183fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2183fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53326/0 (socket says 172.21.15.1:53326) 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2183fff640 1 -- 172.21.15.1:0/3161281579 learned_addr learned my addr 172.21.15.1:0/3161281579 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f2190f0d640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f218c15e390 0x7f218c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.591+0000 7f218bfff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0b1730 0x7f218c0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 -- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0b1730 msgr2=0x7f218c0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0b1730 0x7f218c0ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 -- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f218c15e390 msgr2=0x7f218c0b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f218c15e390 0x7f218c0b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 -- 172.21.15.1:0/3161281579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c0c4e00 con 0x7f218c0cf990 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2183fff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f217400e9b0 tx=0x7f217400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174017070 con 0x7f218c0cf990 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217400c9a0 con 0x7f218c0cf990 2024-05-09T06:36:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217400cd90 con 0x7f218c0cf990 2024-05-09T06:36:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c0c50a0 con 0x7f218c0cf990 2024-05-09T06:36:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.592+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c0c5660 con 0x7f218c0cf990 2024-05-09T06:36:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.593+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218c0752b0 con 0x7f218c0cf990 2024-05-09T06:36:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.594+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f217401a070 con 0x7f218c0cf990 2024-05-09T06:36:58.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.594+0000 7f21897fa640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.594+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f217408aea0 con 0x7f218c0cf990 2024-05-09T06:36:58.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.594+0000 7f218bfff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.595+0000 7f218bfff640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 0x7f216c06e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f217c034000 tx=0x7f217c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.599+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2174059100 con 0x7f218c0cf990 2024-05-09T06:36:58.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.783+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f218c0b2ae0 con 0x7f218c0cf990 2024-05-09T06:36:58.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.783+0000 7f21897fa640 1 -- 172.21.15.1:0/3161281579 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f217405d400 con 0x7f218c0cf990 2024-05-09T06:36:58.764 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:36:58.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 msgr2=0x7f216c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 0x7f216c06e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f217c034000 tx=0x7f217c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:58.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 msgr2=0x7f218c1f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f217400e9b0 tx=0x7f217400ee70 comp rx=0 tx=0).stop 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 shutdown_connections 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f218c15e390 0x7f218c0b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f218c0cf990 0x7f218c1f68f0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 --2- 172.21.15.1:0/3161281579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f218c0b1730 0x7f218c0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 >> 172.21.15.1:0/3161281579 conn(0x7f218c0b1da0 msgr2=0x7f218c06eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 shutdown_connections 2024-05-09T06:36:58.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.785+0000 7f2192970640 1 -- 172.21.15.1:0/3161281579 wait complete. 2024-05-09T06:36:58.777 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:58.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.902+0000 7f096e9b7640 1 Processor -- start 2024-05-09T06:36:58.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.902+0000 7f096e9b7640 1 -- start start 2024-05-09T06:36:58.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f096e9b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 0x7f0968074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f096e9b7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0968074a70 con 0x7f096815c680 2024-05-09T06:36:58.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f096e9b7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096815cc40 con 0x7f09681601c0 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f096e9b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f096815fd30 con 0x7f09681565e0 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f09677fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f096815c680 0x7f09680afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54922/0 (socket says 172.21.15.1:54922) 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f09677fe640 1 -- 172.21.15.1:0/4167214072 learned_addr learned my addr 172.21.15.1:0/4167214072 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0967fff640 1 --2- 172.21.15.1:0/4167214072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 0x7f0968074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1686348485 0 0) 0x7f0968074a70 con 0x7f096815c680 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0950003520 con 0x7f096815c680 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954338520 0 0) 0x7f096815cc40 con 0x7f09681601c0 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0967fff640 1 -- 172.21.15.1:0/4167214072 >> v1:172.21.15.6:6789/0 conn(0x7f09681601c0 legacy=0x7f09680b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0967fff640 1 -- 172.21.15.1:0/4167214072 >> v1:172.21.15.1:6789/0 conn(0x7f096815c680 legacy=0x7f09680afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.903+0000 7f0967fff640 1 -- 172.21.15.1:0/4167214072 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096815cf20 con 0x7f09681565e0 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.904+0000 7f0967fff640 1 --2- 172.21.15.1:0/4167214072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 0x7f0968074530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09580097d0 tx=0x7f095802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417e72f4a361a1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.904+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958037070 con 0x7f09681565e0 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.904+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0958032440 con 0x7f09681565e0 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.904+0000 7f0966ffd640 1 -- 172.21.15.1:0/4167214072 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958032810 con 0x7f09681565e0 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 -- 172.21.15.1:0/4167214072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 msgr2=0x7f0968074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 --2- 172.21.15.1:0/4167214072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 0x7f0968074530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09580097d0 tx=0x7f095802a210 comp rx=0 tx=0).stop 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 -- 172.21.15.1:0/4167214072 shutdown_connections 2024-05-09T06:36:58.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 --2- 172.21.15.1:0/4167214072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681565e0 0x7f0968074530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 -- 172.21.15.1:0/4167214072 >> 172.21.15.1:0/4167214072 conn(0x7f09680b1fb0 msgr2=0x7f09680b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 -- 172.21.15.1:0/4167214072 shutdown_connections 2024-05-09T06:36:58.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.905+0000 7f096e9b7640 1 -- 172.21.15.1:0/4167214072 wait complete. 2024-05-09T06:36:58.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.906+0000 7f096e9b7640 1 Processor -- start 2024-05-09T06:36:58.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.906+0000 7f096e9b7640 1 -- start start 2024-05-09T06:36:58.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f096815c680 0x7f09680aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681601c0 0x7f09680aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f09681569c0 con 0x7f09681565e0 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f096815d4e0 con 0x7f096815c680 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096e9b7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f09680b0340 con 0x7f09681601c0 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f0967fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f0967fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53334/0 (socket says 172.21.15.1:53334) 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f0967fff640 1 -- 172.21.15.1:0/419095526 learned_addr learned my addr 172.21.15.1:0/419095526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f09677fe640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f096815c680 0x7f09680aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f096cf54640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681601c0 0x7f09680aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f0967fff640 1 -- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681601c0 msgr2=0x7f09680aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.907+0000 7f0967fff640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681601c0 0x7f09680aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0967fff640 1 -- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f096815c680 msgr2=0x7f09680aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0967fff640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f096815c680 0x7f09680aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0967fff640 1 -- 172.21.15.1:0/419095526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09680ad2c0 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0967fff640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f09580098f0 tx=0x7f09580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958041020 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095803d6e0 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09680ad560 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958037040 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.908+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09680adb20 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.909+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09680752b0 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.909+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0958044020 con 0x7f09681565e0 2024-05-09T06:36:58.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.910+0000 7f0964ff9640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 0x7f095406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.910+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f09580ab8a0 con 0x7f09681565e0 2024-05-09T06:36:58.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.910+0000 7f09677fe640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 0x7f095406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.910+0000 7f09677fe640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 0x7f095406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0968155b80 tx=0x7f095c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:58.914+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f095804b030 con 0x7f09681565e0 2024-05-09T06:36:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.110+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09680aff80 con 0x7f09681565e0 2024-05-09T06:36:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.111+0000 7f0964ff9640 1 -- 172.21.15.1:0/419095526 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f095807a030 con 0x7f09681565e0 2024-05-09T06:36:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:36:59.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 msgr2=0x7f095406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 0x7f095406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0968155b80 tx=0x7f095c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 msgr2=0x7f0968154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f09580098f0 tx=0x7f09580046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 shutdown_connections 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f095406be30 0x7f095406e2f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09681601c0 0x7f09680aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f096815c680 0x7f09680aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 --2- 172.21.15.1:0/419095526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09681565e0 0x7f0968154a20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 >> 172.21.15.1:0/419095526 conn(0x7f09680b1fb0 msgr2=0x7f096806d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 shutdown_connections 2024-05-09T06:36:59.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.113+0000 7f096e9b7640 1 -- 172.21.15.1:0/419095526 wait complete. 2024-05-09T06:36:59.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:36:59.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:59.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:59.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[75967]: fuse finished with error 0 and tester_r 0 2024-05-09T06:36:59.133 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:59.218 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:36:59.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:59.218 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:36:59.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 Processor -- start 2024-05-09T06:36:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 -- start start 2024-05-09T06:36:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8818161950 0x7f88180d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88180d0f80 0x7f88180d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88180b33c0 con 0x7f88180d4940 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f88180d4dd0 con 0x7f8818161950 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.398+0000 7f881dfa5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88180b3660 con 0x7f88180d0f80 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8818161950 0x7f88180d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8818161950 0x7f88180d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53350/0 (socket says 172.21.15.1:53350) 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f8816ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f88180d4940 0x7f88180cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54936/0 (socket says 172.21.15.1:54936) 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 -- 172.21.15.1:0/1431587805 learned_addr learned my addr 172.21.15.1:0/1431587805 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 -- 172.21.15.1:0/1431587805 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88180d0f80 msgr2=0x7f88180d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 --2- 172.21.15.1:0/1431587805 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88180d0f80 0x7f88180d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 -- 172.21.15.1:0/1431587805 >> v1:172.21.15.1:6789/0 conn(0x7f88180d4940 legacy=0x7f88180cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.399+0000 7f88177fe640 1 -- 172.21.15.1:0/1431587805 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88180b3940 con 0x7f8818161950 2024-05-09T06:36:59.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f88177fe640 1 --2- 172.21.15.1:0/1431587805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8818161950 0x7f88180d2400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98493c577f39313 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f88167fc640 1 -- 172.21.15.1:0/1431587805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808037070 con 0x7f8818161950 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f88167fc640 1 -- 172.21.15.1:0/1431587805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808032440 con 0x7f8818161950 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f88167fc640 1 -- 172.21.15.1:0/1431587805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808032810 con 0x7f8818161950 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 -- 172.21.15.1:0/1431587805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8818161950 msgr2=0x7f88180d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 --2- 172.21.15.1:0/1431587805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8818161950 0x7f88180d2400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 -- 172.21.15.1:0/1431587805 shutdown_connections 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 --2- 172.21.15.1:0/1431587805 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88180d0f80 0x7f88180d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 --2- 172.21.15.1:0/1431587805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8818161950 0x7f88180d2400 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 -- 172.21.15.1:0/1431587805 >> 172.21.15.1:0/1431587805 conn(0x7f88180d3e20 msgr2=0x7f88180d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 -- 172.21.15.1:0/1431587805 shutdown_connections 2024-05-09T06:36:59.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.400+0000 7f881dfa5640 1 -- 172.21.15.1:0/1431587805 wait complete. 2024-05-09T06:36:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.402+0000 7f881dfa5640 1 Processor -- start 2024-05-09T06:36:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.402+0000 7f881dfa5640 1 -- start start 2024-05-09T06:36:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8818161950 0x7f88181e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 0x7f88181f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88180b52d0 con 0x7f88180d0f80 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88180b5150 con 0x7f881814b9e0 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88180b5450 con 0x7f8818161950 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53358/0 (socket says 172.21.15.1:53358) 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 -- 172.21.15.1:0/3835830552 learned_addr learned my addr 172.21.15.1:0/3835830552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f8817fff640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 0x7f88181f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f8816ffd640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8818161950 0x7f88181e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 -- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8818161950 msgr2=0x7f88181e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8818161950 0x7f88181e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 -- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 msgr2=0x7f88181f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 0x7f88181f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 -- 172.21.15.1:0/3835830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88180cb500 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f8817fff640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 0x7f88181f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f88177fe640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f88080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808041070 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880803c870 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808037040 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.403+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88180cb7a0 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.404+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88180cbd60 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.405+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88180752b0 con 0x7f88180d0f80 2024-05-09T06:36:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.405+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8808044070 con 0x7f88180d0f80 2024-05-09T06:36:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.405+0000 7f87fffff640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 0x7f880406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.406+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f88080abdf0 con 0x7f88180d0f80 2024-05-09T06:36:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.406+0000 7f8816ffd640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 0x7f880406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.406+0000 7f8816ffd640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 0x7f880406e320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f88181f7040 tx=0x7f880c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.410+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f880807a720 con 0x7f88180d0f80 2024-05-09T06:36:59.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.629+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88180cbf30 con 0x7f88180d0f80 2024-05-09T06:36:59.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.630+0000 7f87fffff640 1 -- 172.21.15.1:0/3835830552 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5171 (secure 0 0 0) 0x7f880807a020 con 0x7f88180d0f80 2024-05-09T06:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:36:38.092253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6835/3796896332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":3796896332},{"type":"v1","addr":"172.21.15.1:6835","nonce":3796896332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":7}]} 2024-05-09T06:36:59.611 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 82 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 msgr2=0x7f880406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 0x7f880406e320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f88181f7040 tx=0x7f880c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 msgr2=0x7f88181f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f88080046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 shutdown_connections 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f880406be60 0x7f880406e320 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f881814b9e0 0x7f88181f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8818161950 0x7f88181e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 --2- 172.21.15.1:0/3835830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180d0f80 0x7f88181f7c40 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.631+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 >> 172.21.15.1:0/3835830552 conn(0x7f88180d3e20 msgr2=0x7f88180b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.632+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 shutdown_connections 2024-05-09T06:36:59.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:36:59.632+0000 7f881dfa5640 1 -- 172.21.15.1:0/3835830552 wait complete. 2024-05-09T06:36:59.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-09T06:36:59.758 INFO:teuthology.orchestra.run.smithi001.stdout:{"message":"","return_code":0} 2024-05-09T06:36:59.758 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T06:36:59.758 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:36:59.758 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:59.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:36:59.775 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:59.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:36:59.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:59.776 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:36:59.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:59.832 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T06:36:59.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-09T06:36:59.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:59.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-09T06:36:59.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.003+0000 7f082254f640 1 Processor -- start 2024-05-09T06:36:59.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.003+0000 7f082254f640 1 -- start start 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f082254f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f081c0b2ed0 0x7f081c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f082254f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 0x7f081c0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f082254f640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c0d1cc0 con 0x7f081c0d2630 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f082254f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f081c0d4d10 con 0x7f081c0b2ed0 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f082254f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f081c0d1f80 con 0x7f081c0b2110 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f081c0b2ed0 0x7f081c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f081c0d2630 0x7f081c15ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54942/0 (socket says 172.21.15.1:54942) 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081bfff640 1 -- 172.21.15.1:0/2058827544 learned_addr learned my addr 172.21.15.1:0/2058827544 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f0820aec640 1 --2- 172.21.15.1:0/2058827544 >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 0x7f081c0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081b7fe640 1 -- 172.21.15.1:0/2058827544 >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 msgr2=0x7f081c0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081b7fe640 1 --2- 172.21.15.1:0/2058827544 >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 0x7f081c0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081b7fe640 1 -- 172.21.15.1:0/2058827544 >> v1:172.21.15.1:6789/0 conn(0x7f081c0d2630 legacy=0x7f081c15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081b7fe640 1 -- 172.21.15.1:0/2058827544 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0b0c00 con 0x7f081c0b2ed0 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f081affd640 1 -- 172.21.15.1:0/2058827544 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702183318 0 0) 0x7f081c0d1cc0 con 0x7f081c0d2630 2024-05-09T06:36:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.004+0000 7f0820aec640 1 --2- 172.21.15.1:0/2058827544 >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 0x7f081c0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.005+0000 7f081b7fe640 1 --2- 172.21.15.1:0/2058827544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0b2ed0 0x7f081c0ceba0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f081c08e0e0 tx=0x7f081002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b7a0cb0c85957 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.005+0000 7f081affd640 1 -- 172.21.15.1:0/2058827544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810037070 con 0x7f081c0b2ed0 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.005+0000 7f081affd640 1 -- 172.21.15.1:0/2058827544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810032440 con 0x7f081c0b2ed0 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.005+0000 7f081affd640 1 -- 172.21.15.1:0/2058827544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810032810 con 0x7f081c0b2ed0 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 -- 172.21.15.1:0/2058827544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0b2ed0 msgr2=0x7f081c0ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 --2- 172.21.15.1:0/2058827544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0b2ed0 0x7f081c0ceba0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f081c08e0e0 tx=0x7f081002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 -- 172.21.15.1:0/2058827544 shutdown_connections 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 --2- 172.21.15.1:0/2058827544 >> v2:172.21.15.6:3300/0 conn(0x7f081c0b2110 0x7f081c0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 --2- 172.21.15.1:0/2058827544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0b2ed0 0x7f081c0ceba0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 -- 172.21.15.1:0/2058827544 >> 172.21.15.1:0/2058827544 conn(0x7f081c0d3ee0 msgr2=0x7f081c0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 -- 172.21.15.1:0/2058827544 shutdown_connections 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.006+0000 7f082254f640 1 -- 172.21.15.1:0/2058827544 wait complete. 2024-05-09T06:36:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.007+0000 7f082254f640 1 Processor -- start 2024-05-09T06:36:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.007+0000 7f082254f640 1 -- start start 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 0x7f081c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 0x7f081c0b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0d2b90 con 0x7f081c0d2630 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0d2a10 con 0x7f081c0b2ed0 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f082254f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f081c0d2d10 con 0x7f081c0b2110 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f0820aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f0820aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53376/0 (socket says 172.21.15.1:53376) 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f0820aec640 1 -- 172.21.15.1:0/894363383 learned_addr learned my addr 172.21.15.1:0/894363383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f081b7fe640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 0x7f081c0b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.008+0000 7f081bfff640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 0x7f081c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 -- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 msgr2=0x7f081c155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 0x7f081c155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 -- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 msgr2=0x7f081c0b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 0x7f081c0b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 -- 172.21.15.1:0/894363383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0b4180 con 0x7f081c0d2630 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f081bfff640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 0x7f081c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f081b7fe640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 0x7f081c0b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0820aec640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f081400cd50 tx=0x7f081400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0818ff9640 1 -- 172.21.15.1:0/894363383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400f680 con 0x7f081c0d2630 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0818ff9640 1 -- 172.21.15.1:0/894363383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081400f840 con 0x7f081c0d2630 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f0818ff9640 1 -- 172.21.15.1:0/894363383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400fc30 con 0x7f081c0d2630 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c0cbca0 con 0x7f081c0d2630 2024-05-09T06:36:59.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.009+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c0b46d0 con 0x7f081c0d2630 2024-05-09T06:36:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.011+0000 7f0818ff9640 1 -- 172.21.15.1:0/894363383 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08140213f0 con 0x7f081c0d2630 2024-05-09T06:36:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.011+0000 7f0818ff9640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 0x7f080806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.011+0000 7f0818ff9640 1 -- 172.21.15.1:0/894363383 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f081408bd20 con 0x7f081c0d2630 2024-05-09T06:36:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.011+0000 7f081bfff640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 0x7f080806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.012+0000 7f081bfff640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 0x7f080806e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f080c000f60 tx=0x7f080c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.013+0000 7f0825256740 1 Processor -- start 2024-05-09T06:36:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.013+0000 7f0825256740 1 -- start start 2024-05-09T06:36:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91590b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e914f2990 con 0x559e915cb080 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e9149a5b0 con 0x559e9158e9b0 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e914ee950 con 0x559e915b9470 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53386/0 (socket says 172.21.15.1:53386) 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 -- 172.21.15.1:0/869579758 learned_addr learned my addr 172.21.15.1:0/869579758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081bfff640 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91590b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f0820aec640 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 -- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 msgr2=0x559e915b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 -- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 msgr2=0x559e91590b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91590b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 -- 172.21.15.1:0/869579758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e912ce7e0 con 0x559e915cb080 2024-05-09T06:36:59.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f081b7fe640 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0810035040 tx=0x7f081002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f59505b03f0d13b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.015+0000 7f08027fc640 1 -- 172.21.15.1:0/869579758 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810037070 con 0x559e915cb080 2024-05-09T06:36:59.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f08027fc640 1 -- 172.21.15.1:0/869579758 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810043070 con 0x559e915cb080 2024-05-09T06:36:59.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f08027fc640 1 -- 172.21.15.1:0/869579758 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810003610 con 0x559e915cb080 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 -- 172.21.15.1:0/869579758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 msgr2=0x559e9158ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0810035040 tx=0x7f081002c000 comp rx=0 tx=0).stop 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 -- 172.21.15.1:0/869579758 shutdown_connections 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e9158ed90 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 --2- 172.21.15.1:0/869579758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91590b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.016+0000 7f0825256740 1 -- 172.21.15.1:0/869579758 >> 172.21.15.1:0/869579758 conn(0x559e915560a0 msgr2=0x559e915d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.017+0000 7f0825256740 1 -- 172.21.15.1:0/869579758 shutdown_connections 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.017+0000 7f0825256740 1 -- 172.21.15.1:0/869579758 wait complete. 2024-05-09T06:37:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.017+0000 7f0825256740 1 -- start start 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.018+0000 7f0825256740 1 Processor -- start 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 0x559e915cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e914adb40 con 0x559e9158e9b0 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x559e915b6fc0 con 0x559e915cb080 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e915bb100 con 0x559e915b9470 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0825256740 10 client.0 _ll_register_callbacks cb 0x559e914a20f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53392/0 (socket says 172.21.15.1:53392) 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 -- 172.21.15.1:0/4045406482 learned_addr learned my addr 172.21.15.1:0/4045406482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 -- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 msgr2=0x559e915ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0820aec640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 0x559e915cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 -- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 msgr2=0x559e915cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 0x559e915cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 -- 172.21.15.1:0/4045406482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9132ef70 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f0820aec640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 0x559e915cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f081bfff640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f080c001050 tx=0x7f080c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb1bb87528e0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.019+0000 7f07faffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c014e40 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f080c010040 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810043070 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f080c02e050 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 10 client.9216 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x559e911c6170 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810043070 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f080c031020 con 0x559e9158e9b0 2024-05-09T06:37:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 10 client.9216 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 10 client.9216 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 20 client.9216 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e915eee20 0x559e915f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 4 client.9216 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e915000c0 con 0x559e915eee20 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e915eee20 0x559e915f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.020+0000 7f07faffd640 10 client.9216 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.021+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e915eee20 0x559e915f1210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b0a7e654fd902f server_cookie=171087351ff78da1 in_seq=0 out_seq=0 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.021+0000 7f07faffd640 1 -- 172.21.15.1:0/4045406482 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e915000c0 con 0x559e915eee20 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.021+0000 7f07faffd640 10 client.9216 handle_command_reply: tid=0 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.021+0000 7f0825256740 1 client.9216 shutdown 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 msgr2=0x559e915aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f080c001050 tx=0x7f080c000ed0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 shutdown_connections 2024-05-09T06:37:00.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e915eee20 0x559e915f1210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915cb080 0x559e915cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 --2- 172.21.15.1:0/4045406482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915aba40 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f07faffd640 0 client.9216 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 >> 172.21.15.1:0/4045406482 conn(0x559e915560a0 msgr2=0x559e915d79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 shutdown_connections 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 1 -- 172.21.15.1:0/4045406482 wait complete. 2024-05-09T06:37:00.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.022+0000 7f0825256740 20 client.9216 trim_cache size 0 max 16384 2024-05-09T06:37:00.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.027+0000 7fcbda67d640 1 Processor -- start 2024-05-09T06:37:00.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 -- start start 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcbd4161cb0 0x7fcbd40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 0x7fcbd415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd40d2110 con 0x7fcbd4162090 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd40d2d20 con 0x7fcbd4161cb0 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbda67d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd40d23f0 con 0x7fcbd415db00 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.028+0000 7fcbd37fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcbd4162090 0x7fcbd40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54950/0 (socket says 172.21.15.1:54950) 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd37fe640 1 -- 172.21.15.1:0/2955685329 learned_addr learned my addr 172.21.15.1:0/2955685329 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/2955685329 >> v2:172.21.15.6:3300/0 conn(0x7fcbd4161cb0 0x7fcbd40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd8c1a640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 0x7fcbd415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd2ffd640 1 -- 172.21.15.1:0/2955685329 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3742496870 0 0) 0x7fcbd40d2110 con 0x7fcbd4162090 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd2ffd640 1 -- 172.21.15.1:0/2955685329 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbbc003520 con 0x7fcbd4162090 2024-05-09T06:37:00.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd3fff640 1 -- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 msgr2=0x7fcbd415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 0x7fcbd415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd3fff640 1 -- 172.21.15.1:0/2955685329 >> v1:172.21.15.1:6789/0 conn(0x7fcbd4162090 legacy=0x7fcbd40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd3fff640 1 -- 172.21.15.1:0/2955685329 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd40d26d0 con 0x7fcbd4161cb0 2024-05-09T06:37:00.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.029+0000 7fcbd8c1a640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 0x7fcbd415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:00.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.030+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4161cb0 0x7fcbd40d1a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c834b932dc3f4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.030+0000 7fcbd2ffd640 1 -- 172.21.15.1:0/2955685329 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4037070 con 0x7fcbd4161cb0 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.030+0000 7fcbd2ffd640 1 -- 172.21.15.1:0/2955685329 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc4032440 con 0x7fcbd4161cb0 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.030+0000 7fcbd2ffd640 1 -- 172.21.15.1:0/2955685329 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4032810 con 0x7fcbd4161cb0 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 -- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4161cb0 msgr2=0x7fcbd40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4161cb0 0x7fcbd40d1a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402a180 comp rx=0 tx=0).stop 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 -- 172.21.15.1:0/2955685329 shutdown_connections 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd415db00 0x7fcbd415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 --2- 172.21.15.1:0/2955685329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4161cb0 0x7fcbd40d1a80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 -- 172.21.15.1:0/2955685329 >> 172.21.15.1:0/2955685329 conn(0x7fcbd40d3eb0 msgr2=0x7fcbd40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 -- 172.21.15.1:0/2955685329 shutdown_connections 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.031+0000 7fcbda67d640 1 -- 172.21.15.1:0/2955685329 wait complete. 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.032+0000 7fcbda67d640 1 Processor -- start 2024-05-09T06:37:00.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.032+0000 7fcbda67d640 1 -- start start 2024-05-09T06:37:00.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbd415db00 0x7fcbd4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd4161cb0 0x7fcbd415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4162090 0x7fcbd415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40cfd50 con 0x7fcbd415db00 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40d29b0 con 0x7fcbd4162090 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbda67d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd415e4d0 con 0x7fcbd4161cb0 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbd415db00 0x7fcbd4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbd415db00 0x7fcbd4150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53408/0 (socket says 172.21.15.1:53408) 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 -- 172.21.15.1:0/3892224596 learned_addr learned my addr 172.21.15.1:0/3892224596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd8c1a640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4162090 0x7fcbd415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 -- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd4161cb0 msgr2=0x7fcbd415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd37fe640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd4161cb0 0x7fcbd415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcbd4161cb0 0x7fcbd415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 -- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4162090 msgr2=0x7fcbd415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4162090 0x7fcbd415fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd3fff640 1 -- 172.21.15.1:0/3892224596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd40cb160 con 0x7fcbd415db00 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.033+0000 7fcbd8c1a640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd4162090 0x7fcbd415fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbd3fff640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbd415db00 0x7fcbd4150210 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4035040 tx=0x7fcbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4041030 con 0x7fcbd415db00 2024-05-09T06:37:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc403d4e0 con 0x7fcbd415db00 2024-05-09T06:37:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbda67d640 1 -- 172.21.15.1:0/3892224596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd40cb440 con 0x7fcbd415db00 2024-05-09T06:37:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4037040 con 0x7fcbd415db00 2024-05-09T06:37:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbda67d640 1 -- 172.21.15.1:0/3892224596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd40cba00 con 0x7fcbd415db00 2024-05-09T06:37:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.034+0000 7fcbda67d640 1 -- 172.21.15.1:0/3892224596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fcbd41f3a10 con 0x7fcbd415db00 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.035+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbc4044070 con 0x7fcbd415db00 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbd0ff9640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcbc006be30 0x7fcbc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fcbc40abfc0 con 0x7fcbd415db00 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbd37fe640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcbc006be30 0x7fcbc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+513 (secure 0 0 0) 0x7fcbc40ac470 con 0x7fcbd415db00 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbda67d640 1 -- 172.21.15.1:0/3892224596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fcbd40752b0 con 0x7fcbd415db00 2024-05-09T06:37:00.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.036+0000 7fcbd37fe640 1 --2- 172.21.15.1:0/3892224596 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcbc006be30 0x7fcbc006e2f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40d1c80 tx=0x7fcbc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.041+0000 7f0825256740 1 Processor -- start 2024-05-09T06:37:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.041+0000 7f0825256740 1 -- start start 2024-05-09T06:37:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 0x559e915f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915cb080 0x559e915f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f081c2007d0 con 0x559e9158e9b0 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1ff7d0 con 0x559e915b9470 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f081c0b59d0 con 0x559e915cb080 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53418/0 (socket says 172.21.15.1:53418) 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 -- 172.21.15.1:0/4240571375 learned_addr learned my addr 172.21.15.1:0/4240571375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 0x559e915f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f0820aec640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915cb080 0x559e915f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 -- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915cb080 msgr2=0x559e915f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915cb080 0x559e915f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 -- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 msgr2=0x559e915f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 0x559e915f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 -- 172.21.15.1:0/4240571375 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e915a2580 con 0x559e9158e9b0 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081b7fe640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 0x559e915f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.042+0000 7f081bfff640 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f080c014c50 tx=0x7f080c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6555b6d4277ec50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0801ffb640 1 -- 172.21.15.1:0/4240571375 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c0126e0 con 0x559e9158e9b0 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0801ffb640 1 -- 172.21.15.1:0/4240571375 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c026070 con 0x559e9158e9b0 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0801ffb640 1 -- 172.21.15.1:0/4240571375 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c012a80 con 0x559e9158e9b0 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 -- 172.21.15.1:0/4240571375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 msgr2=0x559e915ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f080c014c50 tx=0x7f080c020000 comp rx=0 tx=0).stop 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 -- 172.21.15.1:0/4240571375 shutdown_connections 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915cb080 0x559e915f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e915b9470 0x559e915f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 --2- 172.21.15.1:0/4240571375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e9158e9b0 0x559e915ab4a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 -- 172.21.15.1:0/4240571375 >> 172.21.15.1:0/4240571375 conn(0x559e915560a0 msgr2=0x559e915887f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 -- 172.21.15.1:0/4240571375 shutdown_connections 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.043+0000 7f0825256740 1 -- 172.21.15.1:0/4240571375 wait complete. 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.044+0000 7f0825256740 1 -- start start 2024-05-09T06:37:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.044+0000 7f0825256740 1 Processor -- start 2024-05-09T06:37:00.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91644820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559e915f1660 con 0x559e915cb080 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1ff280 con 0x559e9158e9b0 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x559e914afd90 con 0x559e915b9470 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0825256740 10 client.0 _ll_register_callbacks cb 0x7f080c005c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53432/0 (socket says 172.21.15.1:53432) 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 -- 172.21.15.1:0/17662877 learned_addr learned my addr 172.21.15.1:0/17662877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f081bfff640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91644820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f081b7fe640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 -- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 msgr2=0x559e915a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 -- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 msgr2=0x559e91644820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91644820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 -- 172.21.15.1:0/17662877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e915ba230 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f0820aec640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0814059bd0 tx=0x7f0814059ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640b7cd0cc3274a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f07faffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.045+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814087070 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f0814062620 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080c026070 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0814062820 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 10 client.9228 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x559e915d5bd0 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c026070 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f0814059320 con 0x559e915cb080 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 10 client.9228 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 10 client.9228 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:00.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 20 client.9228 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e91644e30 0x559e915d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 4 client.9228 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x559e914fa410 con 0x559e91644e30 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f081bfff640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e91644e30 0x559e915d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f07faffd640 10 client.9228 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.046+0000 7f081bfff640 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e91644e30 0x559e915d3670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479f3e94310f22d8 server_cookie=70868c84e0ce691d in_seq=0 out_seq=0 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f07faffd640 1 -- 172.21.15.1:0/17662877 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e914fa410 con 0x559e91644e30 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f07faffd640 10 client.9228 handle_command_reply: tid=0 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 client.9228 shutdown 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 msgr2=0x559e915a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0814059bd0 tx=0x7f0814059ed0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 shutdown_connections 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x559e91644e30 0x559e915d3670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e915cb080 0x559e915a1800 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x559e915b9470 0x559e915a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 --2- 172.21.15.1:0/17662877 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e9158e9b0 0x559e91644820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 >> 172.21.15.1:0/17662877 conn(0x559e915560a0 msgr2=0x559e915e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f07faffd640 0 client.9228 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 shutdown_connections 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.047+0000 7f0825256740 1 -- 172.21.15.1:0/17662877 wait complete. 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f0825256740 20 client.9228 trim_cache size 0 max 16384 2024-05-09T06:37:00.038 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:00.039 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:37:00.039 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "bd31eddc-c554-4738-8f54-032e358682c2", 2024-05-09T06:37:00.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:37:00.039 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:00.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 msgr2=0x7f080806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 0x7f080806e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f080c000f60 tx=0x7f080c007c10 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 msgr2=0x7f081c0b8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f081400cd50 tx=0x7f081400a650 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 shutdown_connections 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f080806be30 0x7f080806e2f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f081c0d2630 0x7f081c0b8db0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f081c0b2ed0 0x7f081c0b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 --2- 172.21.15.1:0/894363383 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f081c0b2110 0x7f081c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 >> 172.21.15.1:0/894363383 conn(0x7f081c0d3ee0 msgr2=0x7f081c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 shutdown_connections 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.048+0000 7f082254f640 1 -- 172.21.15.1:0/894363383 wait complete. 2024-05-09T06:37:00.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:00.042 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-09T06:37:00.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.166+0000 7f9bea702640 1 Processor -- start 2024-05-09T06:37:00.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.166+0000 7f9bea702640 1 -- start start 2024-05-09T06:37:00.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bea702640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9be406e7e0 0x7f9be40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bea702640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9be4072270 0x7f9be40addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bea702640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be4147d40 con 0x7f9be4147960 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bea702640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9be406d2f0 con 0x7f9be406e7e0 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bea702640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9be4072820 con 0x7f9be4072270 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9bdbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9be406e7e0 0x7f9be40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9be3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9be4147960 0x7f9be4071b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:54960/0 (socket says 172.21.15.1:54960) 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.167+0000 7f9be3fff640 1 -- 172.21.15.1:0/2510450405 learned_addr learned my addr 172.21.15.1:0/2510450405 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9be8c9f640 1 --2- 172.21.15.1:0/2510450405 >> v2:172.21.15.6:3300/0 conn(0x7f9be4072270 0x7f9be40addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2510450405 >> v2:172.21.15.6:3300/0 conn(0x7f9be4072270 msgr2=0x7f9be40addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/2510450405 >> v2:172.21.15.6:3300/0 conn(0x7f9be4072270 0x7f9be40addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2510450405 >> v1:172.21.15.1:6789/0 conn(0x7f9be4147960 legacy=0x7f9be4071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9be37fe640 1 -- 172.21.15.1:0/2510450405 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548726453 0 0) 0x7f9be4147d40 con 0x7f9be4147960 2024-05-09T06:37:00.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2510450405 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be4072b00 con 0x7f9be406e7e0 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.168+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/2510450405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be40ad890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0097a0 tx=0x7f9bcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a05cc659d9bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9be37fe640 1 -- 172.21.15.1:0/2510450405 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc037070 con 0x7f9be406e7e0 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9be37fe640 1 -- 172.21.15.1:0/2510450405 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc032440 con 0x7f9be406e7e0 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9be37fe640 1 -- 172.21.15.1:0/2510450405 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc032810 con 0x7f9be406e7e0 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 -- 172.21.15.1:0/2510450405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 msgr2=0x7f9be40ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 --2- 172.21.15.1:0/2510450405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be40ad890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0097a0 tx=0x7f9bcc02a270 comp rx=0 tx=0).stop 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 -- 172.21.15.1:0/2510450405 shutdown_connections 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 --2- 172.21.15.1:0/2510450405 >> v2:172.21.15.6:3300/0 conn(0x7f9be4072270 0x7f9be40addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 --2- 172.21.15.1:0/2510450405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be40ad890 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 -- 172.21.15.1:0/2510450405 >> 172.21.15.1:0/2510450405 conn(0x7f9be40b1ba0 msgr2=0x7f9be40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 -- 172.21.15.1:0/2510450405 shutdown_connections 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.169+0000 7f9bea702640 1 -- 172.21.15.1:0/2510450405 wait complete. 2024-05-09T06:37:00.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.170+0000 7f9bea702640 1 Processor -- start 2024-05-09T06:37:00.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.170+0000 7f9bea702640 1 -- start start 2024-05-09T06:37:00.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be4072270 0x7f9be4145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be4147960 0x7f9be4145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9be40ae310 con 0x7f9be406e7e0 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9be40730c0 con 0x7f9be4072270 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9bea702640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9be40ae490 con 0x7f9be4147960 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53462/0 (socket says 172.21.15.1:53462) 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.171+0000 7f9be3fff640 1 -- 172.21.15.1:0/3893426751 learned_addr learned my addr 172.21.15.1:0/3893426751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 -- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be4147960 msgr2=0x7f9be4145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be4147960 0x7f9be4145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 -- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be4072270 msgr2=0x7f9be4145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be4072270 0x7f9be4145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 -- 172.21.15.1:0/3893426751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be4141700 con 0x7f9be406e7e0 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c460 tx=0x7f9bd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be17fa640 1 -- 172.21.15.1:0/3893426751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4018070 con 0x7f9be406e7e0 2024-05-09T06:37:00.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be17fa640 1 -- 172.21.15.1:0/3893426751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4007500 con 0x7f9be406e7e0 2024-05-09T06:37:00.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be4142730 con 0x7f9be406e7e0 2024-05-09T06:37:00.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9be17fa640 1 -- 172.21.15.1:0/3893426751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4004a20 con 0x7f9be406e7e0 2024-05-09T06:37:00.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.172+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be4141c50 con 0x7f9be406e7e0 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.174+0000 7f9be17fa640 1 -- 172.21.15.1:0/3893426751 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bd401b070 con 0x7f9be406e7e0 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.174+0000 7f9be17fa640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 0x7f9bc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.174+0000 7f9be17fa640 1 -- 172.21.15.1:0/3893426751 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f9bd408a210 con 0x7f9be406e7e0 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.174+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 0x7f9bc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.175+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 0x7f9bc406e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc034000 tx=0x7f9bcc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.175+0000 7f9bed405740 1 Processor -- start 2024-05-09T06:37:00.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.175+0000 7f9bed405740 1 -- start start 2024-05-09T06:37:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975030 0x564004949e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5640048aed00 con 0x564004975770 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5640048564c0 con 0x564004992650 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5640048aacc0 con 0x564004975030 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53472/0 (socket says 172.21.15.1:53472) 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2176994985 learned_addr learned my addr 172.21.15.1:0/2176994985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9be3fff640 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9be8c9f640 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975030 0x564004949e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975030 msgr2=0x564004949e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.176+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975030 0x564004949e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 msgr2=0x564004944fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bdbfff640 1 -- 172.21.15.1:0/2176994985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400468c7d0 con 0x564004975770 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc032670 tx=0x7f9bcc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a432b10c67430715 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bd9ffb640 1 -- 172.21.15.1:0/2176994985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc04c070 con 0x564004975770 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bd9ffb640 1 -- 172.21.15.1:0/2176994985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc004230 con 0x564004975770 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bd9ffb640 1 -- 172.21.15.1:0/2176994985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc037040 con 0x564004975770 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bed405740 1 -- 172.21.15.1:0/2176994985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 msgr2=0x564004974af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.177+0000 7f9bed405740 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc032670 tx=0x7f9bcc008030 comp rx=0 tx=0).stop 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 -- 172.21.15.1:0/2176994985 shutdown_connections 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975030 0x564004949e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004974af0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 --2- 172.21.15.1:0/2176994985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 -- 172.21.15.1:0/2176994985 >> 172.21.15.1:0/2176994985 conn(0x564004947250 msgr2=0x56400498faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 -- 172.21.15.1:0/2176994985 shutdown_connections 2024-05-09T06:37:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 -- 172.21.15.1:0/2176994985 wait complete. 2024-05-09T06:37:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 -- start start 2024-05-09T06:37:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.178+0000 7f9bed405740 1 Processor -- start 2024-05-09T06:37:00.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975770 0x56400498c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x56400496fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564004949400 con 0x564004992650 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x564004869b90 con 0x564004977a90 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5640049487f0 con 0x564004975770 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bed405740 10 client.0 _ll_register_callbacks cb 0x5640048a5da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53482/0 (socket says 172.21.15.1:53482) 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9bdbfff640 1 -- 172.21.15.1:0/3177034596 learned_addr learned my addr 172.21.15.1:0/3177034596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9be8c9f640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x56400496fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.179+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975770 0x56400498c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 -- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975770 msgr2=0x56400498c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975770 0x56400498c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 -- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 msgr2=0x56400496fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x56400496fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 -- 172.21.15.1:0/3177034596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640045c23f0 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc04a040 tx=0x7f9bcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb5d63d1934350a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc04c070 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f9bcc003610 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc004230 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc05c050 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 10 client.9243 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5640045820f0 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc004230 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f9bcc05f020 con 0x564004992650 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 10 client.9243 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 10 client.9243 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 20 client.9243 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x56400498faf0 0x56400498ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 4 client.9243 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.180+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640048bc430 con 0x56400498faf0 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x56400498faf0 0x56400498ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9bc2ffd640 10 client.9243 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9be3fff640 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x56400498faf0 0x56400498ff00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1ca17cf2f04a0 server_cookie=e25b0a8b04b9e780 in_seq=0 out_seq=0 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/3177034596 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640048bc430 con 0x56400498faf0 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9bc2ffd640 10 client.9243 handle_command_reply: tid=0 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.181+0000 7f9bed405740 1 client.9243 shutdown 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 msgr2=0x5640049a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc04a040 tx=0x7f9bcc004730 comp rx=0 tx=0).stop 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 shutdown_connections 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x56400498faf0 0x56400498ff00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x56400496fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004992650 0x5640049a02b0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bc2ffd640 0 client.9243 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 --2- 172.21.15.1:0/3177034596 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004975770 0x56400498c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 >> 172.21.15.1:0/3177034596 conn(0x564004975030 msgr2=0x564004949800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 shutdown_connections 2024-05-09T06:37:00.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 1 -- 172.21.15.1:0/3177034596 wait complete. 2024-05-09T06:37:00.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.182+0000 7f9bed405740 20 client.9243 trim_cache size 0 max 16384 2024-05-09T06:37:00.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 Processor -- start 2024-05-09T06:37:00.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 -- start start 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004992650 0x564004944490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9be42008f0 con 0x564004975770 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41ff950 con 0x564004977a90 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9be40ce420 con 0x564004992650 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53484/0 (socket says 172.21.15.1:53484) 2024-05-09T06:37:00.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.200+0000 7f9be3fff640 1 -- 172.21.15.1:0/1313549671 learned_addr learned my addr 172.21.15.1:0/1313549671 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be8c9f640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004992650 0x564004944490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 -- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004992650 msgr2=0x564004944490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004992650 0x564004944490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 -- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 msgr2=0x564004943f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 -- 172.21.15.1:0/1313549671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564004970890 con 0x564004975770 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9be3fff640 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4054820 tx=0x7f9bd4088510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7dfaa3bfadd33d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9bd97fa640 1 -- 172.21.15.1:0/1313549671 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40566c0 con 0x564004975770 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9bd97fa640 1 -- 172.21.15.1:0/1313549671 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4054a30 con 0x564004975770 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.201+0000 7f9bd97fa640 1 -- 172.21.15.1:0/1313549671 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4070410 con 0x564004975770 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- 172.21.15.1:0/1313549671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 msgr2=0x564004977090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4054820 tx=0x7f9bd4088510 comp rx=0 tx=0).stop 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- 172.21.15.1:0/1313549671 shutdown_connections 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004992650 0x564004944490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 --2- 172.21.15.1:0/1313549671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004977090 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- 172.21.15.1:0/1313549671 >> 172.21.15.1:0/1313549671 conn(0x564004975030 msgr2=0x564004986c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- 172.21.15.1:0/1313549671 shutdown_connections 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- 172.21.15.1:0/1313549671 wait complete. 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 -- start start 2024-05-09T06:37:00.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.202+0000 7f9bed405740 1 Processor -- start 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5640049778f0 con 0x564004975770 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41ff300 con 0x564004992650 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x564004993a20 con 0x564004977a90 2024-05-09T06:37:00.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bed405740 10 client.0 _ll_register_callbacks cb 0x7f9bcc04a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53490/0 (socket says 172.21.15.1:53490) 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9be3fff640 1 -- 172.21.15.1:0/136314423 learned_addr learned my addr 172.21.15.1:0/136314423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9be8c9f640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.203+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 -- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004977a90 msgr2=0x564004943f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 -- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 msgr2=0x564004944490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 -- 172.21.15.1:0/136314423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400498f510 con 0x564004975770 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9be3fff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4054820 tx=0x7f9bd405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d912706a2ede6d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4086060 con 0x564004975770 2024-05-09T06:37:00.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f9bd4004bc0 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd4054a30 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9bd4075050 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 10 client.9249 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x56400493fbf0 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4054a30 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f9bd40772e0 con 0x564004975770 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 10 client.9249 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 10 client.9249 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 20 client.9249 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x5640049a8ae0 0x564004a09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 4 client.9249 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.204+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640048b6780 con 0x5640049a8ae0 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x5640049a8ae0 0x564004a09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bc2ffd640 10 client.9249 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bdbfff640 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x5640049a8ae0 0x564004a09ed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5e0c1831627799 server_cookie=f299a235be6e5098 in_seq=0 out_seq=0 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bc2ffd640 1 -- 172.21.15.1:0/136314423 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5640048b6780 con 0x5640049a8ae0 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bc2ffd640 10 client.9249 handle_command_reply: tid=0 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 client.9249 shutdown 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 msgr2=0x564004a00580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4054820 tx=0x7f9bd405b040 comp rx=0 tx=0).stop 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 shutdown_connections 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x5640049a8ae0 0x564004a09ed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564004992650 0x564004944490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x564004977a90 0x564004943f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 --2- 172.21.15.1:0/136314423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564004975770 0x564004a00580 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bc2ffd640 0 client.9249 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 >> 172.21.15.1:0/136314423 conn(0x564004975030 msgr2=0x564004944c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 shutdown_connections 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 1 -- 172.21.15.1:0/136314423 wait complete. 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.205+0000 7f9bed405740 20 client.9249 trim_cache size 0 max 16384 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bd31eddc-c554-4738-8f54-032e358682c2": { 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "bd31eddc-c554-4738-8f54-032e358682c2", 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:37:00.193 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 msgr2=0x7f9bc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 0x7f9bc406e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc034000 tx=0x7f9bcc02c000 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 msgr2=0x7f9be413ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c460 tx=0x7f9bd400c920 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 shutdown_connections 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9bc406be30 0x7f9bc406e2f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9be4147960 0x7f9be4145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be4072270 0x7f9be4145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 --2- 172.21.15.1:0/3893426751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9be406e7e0 0x7f9be413ec60 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 >> 172.21.15.1:0/3893426751 conn(0x7f9be40b1ba0 msgr2=0x7f9be40b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 shutdown_connections 2024-05-09T06:37:00.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.206+0000 7f9bea702640 1 -- 172.21.15.1:0/3893426751 wait complete. 2024-05-09T06:37:00.197 INFO:tasks.cephfs.filesystem:scrub status for tag:bd31eddc-c554-4738-8f54-032e358682c2 - {'path': '/', 'tag': 'bd31eddc-c554-4738-8f54-032e358682c2', 'options': 'recursive,repair'} 2024-05-09T06:37:00.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:00.788+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 7 ==== log(5 entries from seq 1 at 2024-05-09T06:37:00.048275+0000) ==== 846+0+0 (secure 0 0 0) 0x7fcbc40aa470 con 0x7fcbd415db00 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: cluster: 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: id: 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: health: HEALTH_OK 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: services: 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: mgr: y(active, since 9m), standbys: x 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: mds: 1/1 daemons up, 3 standby 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: data: 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: volumes: 1/1 healthy 2024-05-09T06:37:00.770 INFO:teuthology.orchestra.run.smithi001.stdout: pools: 3 pools, 97 pgs 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: objects: 30 objects, 19 MiB 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: pgs: 97 active+clean 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: io: 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:00.048275+0000 mds.a [INF] scrub queued for path: / 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:00.048279+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:00.048284+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:00.049626+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:37:00.771 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:00.726214+0000 mds.a [INF] scrub summary: idle 2024-05-09T06:37:03.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:03.793+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 8 ==== log(1 entries from seq 1238 at 2024-05-09T06:37:02.834178+0000) ==== 321+0+0 (secure 0 0 0) 0x7fcbc4078190 con 0x7fcbd415db00 2024-05-09T06:37:04.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:04.799+0000 7fcbd0ff9640 1 -- 172.21.15.1:0/3892224596 <== mon.0 v2:172.21.15.1:3300/0 9 ==== log(1 entries from seq 1239 at 2024-05-09T06:37:03.798804+0000) ==== 323+0+0 (secure 0 0 0) 0x7fcbc40783f0 con 0x7fcbd415db00 2024-05-09T06:37:18.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:18.914 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:30.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:30.199 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-09T06:37:30.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.338+0000 7f54c625d640 1 Processor -- start 2024-05-09T06:37:30.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.338+0000 7f54c625d640 1 -- start start 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 0x7f54c00718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54c015c4a0 0x7f54c0071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 0x7f54c006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f54c0158e30 con 0x7f54c015c4a0 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f54c0072480 con 0x7f54c0158a50 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54c625d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54c015b420 con 0x7f54c006e6d0 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54c015c4a0 0x7f54c0071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f54c015c4a0 0x7f54c0071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59950/0 (socket says 172.21.15.1:59950) 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 -- 172.21.15.1:0/572572031 learned_addr learned my addr 172.21.15.1:0/572572031 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54bffff640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 0x7f54c006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/572572031 >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 0x7f54c00718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 -- 172.21.15.1:0/572572031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 msgr2=0x7f54c006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 0x7f54c006eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 -- 172.21.15.1:0/572572031 >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 msgr2=0x7f54c00718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 --2- 172.21.15.1:0/572572031 >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 0x7f54c00718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54beffd640 1 -- 172.21.15.1:0/572572031 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c015b6e0 con 0x7f54c015c4a0 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54bffff640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 0x7f54c006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.339+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/572572031 >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 0x7f54c00718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.340+0000 7f54beffd640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0071e00 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f54c0073950 tx=0x7f54b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777d0cb807a997ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.340+0000 7f54be7fc640 1 -- 172.21.15.1:0/572572031 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4037070 con 0x7f54c015c4a0 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.340+0000 7f54be7fc640 1 -- 172.21.15.1:0/572572031 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4032440 con 0x7f54c015c4a0 2024-05-09T06:37:30.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.340+0000 7f54be7fc640 1 -- 172.21.15.1:0/572572031 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4032810 con 0x7f54c015c4a0 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 -- 172.21.15.1:0/572572031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 msgr2=0x7f54c0071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0071e00 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f54c0073950 tx=0x7f54b402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 -- 172.21.15.1:0/572572031 shutdown_connections 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c006e6d0 0x7f54c006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 --2- 172.21.15.1:0/572572031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0071e00 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 --2- 172.21.15.1:0/572572031 >> v2:172.21.15.6:3300/0 conn(0x7f54c0158a50 0x7f54c00718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 -- 172.21.15.1:0/572572031 >> 172.21.15.1:0/572572031 conn(0x7f54c00cc4d0 msgr2=0x7f54c00cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 -- 172.21.15.1:0/572572031 shutdown_connections 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.341+0000 7f54c625d640 1 -- 172.21.15.1:0/572572031 wait complete. 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.342+0000 7f54c625d640 1 Processor -- start 2024-05-09T06:37:30.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.342+0000 7f54c625d640 1 -- start start 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 0x7f54c00b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c0158a50 0x7f54c01f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c015bf80 con 0x7f54c015c4a0 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f54c015bca0 con 0x7f54c006e6d0 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54c625d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54c0070ea0 con 0x7f54c0158a50 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59962/0 (socket says 172.21.15.1:59962) 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54bffff640 1 -- 172.21.15.1:0/1098456642 learned_addr learned my addr 172.21.15.1:0/1098456642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 0x7f54c00b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54beffd640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c0158a50 0x7f54c01f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.343+0000 7f54bffff640 1 -- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c0158a50 msgr2=0x7f54c01f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bffff640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c0158a50 0x7f54c01f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bffff640 1 -- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 msgr2=0x7f54c00b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bffff640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 0x7f54c00b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bffff640 1 -- 172.21.15.1:0/1098456642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01f6360 con 0x7f54c015c4a0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 0x7f54c00b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54bffff640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f54b800cd10 tx=0x7f54b800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54a7fff640 1 -- 172.21.15.1:0/1098456642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800f680 con 0x7f54c015c4a0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54a7fff640 1 -- 172.21.15.1:0/1098456642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b800f840 con 0x7f54c015c4a0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c00b48e0 con 0x7f54c015c4a0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54a7fff640 1 -- 172.21.15.1:0/1098456642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800fc30 con 0x7f54c015c4a0 2024-05-09T06:37:30.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.344+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c00b4ea0 con 0x7f54c015c4a0 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.346+0000 7f54a7fff640 1 -- 172.21.15.1:0/1098456642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54b80213f0 con 0x7f54c015c4a0 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.346+0000 7f54a7fff640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 0x7f54ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.346+0000 7f54a7fff640 1 -- 172.21.15.1:0/1098456642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6130+0+0 (secure 0 0 0) 0x7f54b808bcf0 con 0x7f54c015c4a0 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.346+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 0x7f54ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.347+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 0x7f54ac06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f54b0006fd0 tx=0x7f54b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.347+0000 7f54c8f6b740 1 Processor -- start 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.347+0000 7f54c8f6b740 1 -- start start 2024-05-09T06:37:30.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51030 0x555e14e25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555e14d8ad00 con 0x555e14e6e650 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555e14d324c0 con 0x555e14e51770 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555e14d86cc0 con 0x555e14e51030 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59964/0 (socket says 172.21.15.1:59964) 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 -- 172.21.15.1:0/284236112 learned_addr learned my addr 172.21.15.1:0/284236112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 -- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51030 msgr2=0x555e14e25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51030 0x555e14e25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 -- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 msgr2=0x555e14e50af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.348+0000 7f54bf7fe640 1 -- 172.21.15.1:0/284236112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e14b687d0 con 0x555e14e6e650 2024-05-09T06:37:30.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f54b000db30 tx=0x7f54b0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d80982095059eed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54a57fa640 1 -- 172.21.15.1:0/284236112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0019050 con 0x555e14e6e650 2024-05-09T06:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54a57fa640 1 -- 172.21.15.1:0/284236112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b0023070 con 0x555e14e6e650 2024-05-09T06:37:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54a57fa640 1 -- 172.21.15.1:0/284236112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0012c90 con 0x555e14e6e650 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 -- 172.21.15.1:0/284236112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 msgr2=0x555e14e20fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f54b000db30 tx=0x7f54b0010000 comp rx=0 tx=0).stop 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 -- 172.21.15.1:0/284236112 shutdown_connections 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51030 0x555e14e25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e50af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/284236112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e20fe0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.349+0000 7f54c8f6b740 1 -- 172.21.15.1:0/284236112 >> 172.21.15.1:0/284236112 conn(0x555e14e23250 msgr2=0x555e14e6baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.350+0000 7f54c8f6b740 1 -- 172.21.15.1:0/284236112 shutdown_connections 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.350+0000 7f54c8f6b740 1 -- 172.21.15.1:0/284236112 wait complete. 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.350+0000 7f54c8f6b740 1 -- start start 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.350+0000 7f54c8f6b740 1 Processor -- start 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e6e650 0x555e14e7f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e7cad0 0x555e14e4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555e14e25400 con 0x555e14e51770 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555e14d45b90 con 0x555e14e6e650 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555e14e247f0 con 0x555e14e7cad0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54c8f6b740 10 client.0 _ll_register_callbacks cb 0x555e14d81da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59970/0 (socket says 172.21.15.1:59970) 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 -- 172.21.15.1:0/3121564234 learned_addr learned my addr 172.21.15.1:0/3121564234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54beffd640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e6e650 0x555e14e7f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bffff640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e7cad0 0x555e14e4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 -- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e7cad0 msgr2=0x555e14e4bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e7cad0 0x555e14e4bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 -- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e6e650 msgr2=0x555e14e7f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e6e650 0x555e14e7f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.351+0000 7f54bf7fe640 1 -- 172.21.15.1:0/3121564234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e14a9e3f0 con 0x555e14e51770 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f54b000dff0 tx=0x7f54b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4473af66fc6eb0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0035060 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6130+0+0 (secure 0 0 0) 0x7f54b001e790 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b0023070 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f54b0019020 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 10 client.9261 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x555e14a5e0f0 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b0023070 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f54b001e990 con 0x555e14e51770 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 10 client.9261 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 10 client.9261 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 20 client.9261 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e53d40 0x555e14e54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 4 client.9261 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:30.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.352+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e14d98430 con 0x555e14e53d40 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f54beffd640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e53d40 0x555e14e54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f54beffd640 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e53d40 0x555e14e54140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2311839419656c0d server_cookie=dd1257564a01f5c2 in_seq=0 out_seq=0 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f549e7fc640 10 client.9261 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f549e7fc640 1 -- 172.21.15.1:0/3121564234 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e14d98430 con 0x555e14e53d40 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f549e7fc640 10 client.9261 handle_command_reply: tid=0 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.353+0000 7f54c8f6b740 1 client.9261 shutdown 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 msgr2=0x555e14e6d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f54b000dff0 tx=0x7f54b0004730 comp rx=0 tx=0).stop 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 shutdown_connections 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e53d40 0x555e14e54140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e7cad0 0x555e14e4bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e6e650 0x555e14e7f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/3121564234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e51770 0x555e14e6d390 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 >> 172.21.15.1:0/3121564234 conn(0x555e14e51030 msgr2=0x555e14e6dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f549e7fc640 0 client.9261 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:30.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 shutdown_connections 2024-05-09T06:37:30.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 1 -- 172.21.15.1:0/3121564234 wait complete. 2024-05-09T06:37:30.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.354+0000 7f54c8f6b740 20 client.9261 trim_cache size 0 max 16384 2024-05-09T06:37:30.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 Processor -- start 2024-05-09T06:37:30.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 -- start start 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 0x555e14e4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e7cad0 0x555e14e64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01fea00 con 0x555e14e6e650 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01fda20 con 0x555e14e7cad0 2024-05-09T06:37:30.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.372+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f54c014e5d0 con 0x555e14e51770 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59972/0 (socket says 172.21.15.1:59972) 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 -- 172.21.15.1:0/1062628264 learned_addr learned my addr 172.21.15.1:0/1062628264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54bffff640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e7cad0 0x555e14e64410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 0x555e14e4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 -- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 msgr2=0x555e14e4be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 0x555e14e4be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 -- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e7cad0 msgr2=0x555e14e64410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e7cad0 0x555e14e64410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 -- 172.21.15.1:0/1062628264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e14e65db0 con 0x555e14e6e650 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 0x555e14e4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54beffd640 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a7d0 tx=0x7f54b402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e763ebdb0877ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54a4ff9640 1 -- 172.21.15.1:0/1062628264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b404e070 con 0x555e14e6e650 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54a4ff9640 1 -- 172.21.15.1:0/1062628264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4032620 con 0x555e14e6e650 2024-05-09T06:37:30.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.373+0000 7f54a4ff9640 1 -- 172.21.15.1:0/1062628264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4037040 con 0x555e14e6e650 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- 172.21.15.1:0/1062628264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 msgr2=0x555e14e63ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a7d0 tx=0x7f54b402c070 comp rx=0 tx=0).stop 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- 172.21.15.1:0/1062628264 shutdown_connections 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e7cad0 0x555e14e64410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e6e650 0x555e14e63ed0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/1062628264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e51770 0x555e14e4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- 172.21.15.1:0/1062628264 >> 172.21.15.1:0/1062628264 conn(0x555e14e51030 msgr2=0x555e14e54120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- 172.21.15.1:0/1062628264 shutdown_connections 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- 172.21.15.1:0/1062628264 wait complete. 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.374+0000 7f54c8f6b740 1 -- start start 2024-05-09T06:37:30.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 Processor -- start 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e52790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e6e650 0x555e14e7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c014ec20 con 0x555e14e7cad0 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555e14e7b410 con 0x555e14e51770 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555e14e6c800 con 0x555e14e6e650 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54c8f6b740 10 client.0 _ll_register_callbacks cb 0x7f54b000d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:30.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.375+0000 7f54bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59988/0 (socket says 172.21.15.1:59988) 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 -- 172.21.15.1:0/2466659655 learned_addr learned my addr 172.21.15.1:0/2466659655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e52790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54beffd640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e6e650 0x555e14e7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 -- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e6e650 msgr2=0x555e14e7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e6e650 0x555e14e7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 -- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 msgr2=0x555e14e52790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e52790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 -- 172.21.15.1:0/2466659655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e14e286a0 con 0x555e14e7cad0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54bffff640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f54b808a060 tx=0x7f54b8062aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ad2db5d074e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f549e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8077060 con 0x555e14e7cad0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6130+0+0 (secure 0 0 0) 0x7f54b80592e0 con 0x555e14e7cad0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b4032620 con 0x555e14e7cad0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f54b8087070 con 0x555e14e7cad0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54c8f6b740 10 client.9267 fetch_fsmap learned FSMap version 82 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x555e14e7b150 con 0x555e14e7cad0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.376+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4032620 con 0x555e14e7cad0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f54b8071040 con 0x555e14e7cad0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 10 client.9267 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 10 client.9267 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 20 client.9267 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e7acd0 0x555e14e64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 4 client.9267 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 --> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e14d92780 con 0x555e14e7acd0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e7acd0 0x555e14e64410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54bf7fe640 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e7acd0 0x555e14e64410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49276990d06ac6e7 server_cookie=a0b102d0879e33ca in_seq=0 out_seq=0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f549e7fc640 10 client.9267 ms_handle_connect on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f549e7fc640 1 -- 172.21.15.1:0/2466659655 <== mds.0 v2:172.21.15.1:6834/3796896332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e14d92780 con 0x555e14e7acd0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f549e7fc640 10 client.9267 handle_command_reply: tid=0 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 client.9267 shutdown 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 msgr2=0x555e14e63ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f54b808a060 tx=0x7f54b8062aa0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 shutdown_connections 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:6834/3796896332,v1:172.21.15.1:6835/3796896332] conn(0x555e14e7acd0 0x555e14e64410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555e14e7cad0 0x555e14e63ed0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555e14e6e650 0x555e14e7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f549e7fc640 0 client.9267 ms_handle_reset on v2:172.21.15.1:6834/3796896332 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 --2- 172.21.15.1:0/2466659655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555e14e51770 0x555e14e52790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 >> 172.21.15.1:0/2466659655 conn(0x555e14e51030 msgr2=0x555e14e507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 shutdown_connections 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.377+0000 7f54c8f6b740 1 -- 172.21.15.1:0/2466659655 wait complete. 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c8f6b740 20 client.9267 trim_cache size 0 max 16384 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 msgr2=0x7f54ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 0x7f54ac06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f54b0006fd0 tx=0x7f54b0008040 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 msgr2=0x7f54c0143440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f54b800cd10 tx=0x7f54b800a650 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 shutdown_connections 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f54ac06be30 0x7f54ac06e2f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c015c4a0 0x7f54c0143440 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f54c0158a50 0x7f54c01f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 --2- 172.21.15.1:0/1098456642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54c006e6d0 0x7f54c00b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 >> 172.21.15.1:0/1098456642 conn(0x7f54c00cc4d0 msgr2=0x7f54c0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 shutdown_connections 2024-05-09T06:37:30.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.378+0000 7f54c625d640 1 -- 172.21.15.1:0/1098456642 wait complete. 2024-05-09T06:37:30.370 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:37:30.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:37:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.597+0000 7ff116968640 1 Processor -- start 2024-05-09T06:37:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.597+0000 7ff116968640 1 -- start start 2024-05-09T06:37:30.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff116968640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 0x7ff11006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff116968640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1100b3300 0x7ff1100b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff116968640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1100ce0e0 con 0x7ff1100cf180 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff116968640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff1100ce3c0 con 0x7ff1100b3300 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff116968640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff1100ce6a0 con 0x7ff11006e6d0 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff114f05640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1100b3300 0x7ff1100b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff114f05640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff1100b3300 0x7ff1100b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59994/0 (socket says 172.21.15.1:59994) 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.598+0000 7ff114f05640 1 -- 172.21.15.1:0/3194744838 learned_addr learned my addr 172.21.15.1:0/3194744838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff10ffff640 1 --2- 172.21.15.1:0/3194744838 >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 0x7ff11006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff114f05640 1 -- 172.21.15.1:0/3194744838 >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 msgr2=0x7ff11006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff114f05640 1 --2- 172.21.15.1:0/3194744838 >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 0x7ff11006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff114f05640 1 -- 172.21.15.1:0/3194744838 >> v1:172.21.15.6:6789/0 conn(0x7ff1100cf180 legacy=0x7ff1100cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff114f05640 1 -- 172.21.15.1:0/3194744838 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100ce980 con 0x7ff1100b3300 2024-05-09T06:37:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff10ffff640 1 --2- 172.21.15.1:0/3194744838 >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 0x7ff11006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.599+0000 7ff114f05640 1 --2- 172.21.15.1:0/3194744838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1100b3300 0x7ff1100b3710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff110074650 tx=0x7ff10802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca99b9123a271582 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff10f7fe640 1 -- 172.21.15.1:0/3194744838 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108036070 con 0x7ff1100b3300 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff10f7fe640 1 -- 172.21.15.1:0/3194744838 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff108031600 con 0x7ff1100b3300 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff10f7fe640 1 -- 172.21.15.1:0/3194744838 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1080319d0 con 0x7ff1100b3300 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 -- 172.21.15.1:0/3194744838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1100b3300 msgr2=0x7ff1100b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 --2- 172.21.15.1:0/3194744838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1100b3300 0x7ff1100b3710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff110074650 tx=0x7ff10802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 -- 172.21.15.1:0/3194744838 shutdown_connections 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 --2- 172.21.15.1:0/3194744838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1100b3300 0x7ff1100b3710 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 --2- 172.21.15.1:0/3194744838 >> v2:172.21.15.6:3300/0 conn(0x7ff11006e6d0 0x7ff11006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 -- 172.21.15.1:0/3194744838 >> 172.21.15.1:0/3194744838 conn(0x7ff1100cc4b0 msgr2=0x7ff1100cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 -- 172.21.15.1:0/3194744838 shutdown_connections 2024-05-09T06:37:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.600+0000 7ff116968640 1 -- 172.21.15.1:0/3194744838 wait complete. 2024-05-09T06:37:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.601+0000 7ff116968640 1 Processor -- start 2024-05-09T06:37:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 -- start start 2024-05-09T06:37:30.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1100b3300 0x7ff1100b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 0x7ff11015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff110070ed0 con 0x7ff11006e6d0 2024-05-09T06:37:30.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff11006eff0 con 0x7ff1100b3300 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff116968640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff1101447c0 con 0x7ff1100cf180 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff10ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff10ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60008/0 (socket says 172.21.15.1:60008) 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.602+0000 7ff10ffff640 1 -- 172.21.15.1:0/2392892291 learned_addr learned my addr 172.21.15.1:0/2392892291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff114f05640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 0x7ff11015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff107fff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1100b3300 0x7ff1100b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 -- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 msgr2=0x7ff11015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 0x7ff11015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 -- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1100b3300 msgr2=0x7ff1100b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1100b3300 0x7ff1100b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 -- 172.21.15.1:0/2392892291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100afab0 con 0x7ff11006e6d0 2024-05-09T06:37:30.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff114f05640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 0x7ff11015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:30.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10ffff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c7f0 tx=0x7ff10000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100015070 con 0x7ff11006e6d0 2024-05-09T06:37:30.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff100007820 con 0x7ff11006e6d0 2024-05-09T06:37:30.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1100afd90 con 0x7ff11006e6d0 2024-05-09T06:37:30.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100007bd0 con 0x7ff11006e6d0 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.603+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1100b0350 con 0x7ff11006e6d0 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.604+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1100b2c60 con 0x7ff11006e6d0 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.605+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff100018070 con 0x7ff11006e6d0 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.605+0000 7ff10d7fa640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 0x7ff0f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.605+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6130+0+0 (secure 0 0 0) 0x7ff10008b060 con 0x7ff11006e6d0 2024-05-09T06:37:30.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.605+0000 7ff107fff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 0x7ff0f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.606+0000 7ff107fff640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 0x7ff0f006e320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff11015e820 tx=0x7ff0f8007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.610+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1100b2c60 con 0x7ff11006e6d0 2024-05-09T06:37:30.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.835+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1100752b0 con 0x7ff11006e6d0 2024-05-09T06:37:30.817 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:37:30.836+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:37:30.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.838+0000 7ff10d7fa640 1 -- 172.21.15.1:0/2392892291 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7ff100059210 con 0x7ff11006e6d0 2024-05-09T06:37:30.818 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:37:30.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.840+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 msgr2=0x7ff0f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.840+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 0x7ff0f006e320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff11015e820 tx=0x7ff0f8007e30 comp rx=0 tx=0).stop 2024-05-09T06:37:30.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 msgr2=0x7ff1101567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c7f0 tx=0x7ff10000ccb0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 shutdown_connections 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff0f006be60 0x7ff0f006e320 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff1100cf180 0x7ff11015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1100b3300 0x7ff1100b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 --2- 172.21.15.1:0/2392892291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff11006e6d0 0x7ff1101567c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 >> 172.21.15.1:0/2392892291 conn(0x7ff1100cc4b0 msgr2=0x7ff1100b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 shutdown_connections 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.841+0000 7ff116968640 1 -- 172.21.15.1:0/2392892291 wait complete. 2024-05-09T06:37:30.824 INFO:tasks.ceph.mds.a.smithi001.stderr: -6263> 2024-05-09T06:36:36.700+0000 7f3d7de73640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:37:30.833 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-09T06:37:30.858 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:37:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.897+0000 7f2219d92280 1 -- start start 2024-05-09T06:37:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.897+0000 7f2219d92280 1 Processor -- start 2024-05-09T06:37:30.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 0x555b553f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 0x555b553f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b5615a000 con 0x555b5614fc00 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555b5615a180 con 0x555b5614e800 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555b5615a300 con 0x555b5614f400 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2217d3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2217d3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60062/0 (socket says 172.21.15.1:60062) 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2217d3d640 1 -- 172.21.15.1:0/3917481881 learned_addr learned my addr 172.21.15.1:0/3917481881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f221853e640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 0x555b553f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2218d3f640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 0x555b553f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2217d3d640 1 -- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 msgr2=0x555b553f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.898+0000 7f2217d3d640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 0x555b553f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2217d3d640 1 -- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 msgr2=0x555b553f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2217d3d640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 0x555b553f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2217d3d640 1 -- 172.21.15.1:0/3917481881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b553bc780 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2218d3f640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 0x555b553f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f221853e640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 0x555b553f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2217d3d640 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x555b561b1380 tx=0x555b5611a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62888b365ae682 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2215d39640 1 -- 172.21.15.1:0/3917481881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553ac9c0 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2215d39640 1 -- 172.21.15.1:0/3917481881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b553b2a80 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2215d39640 1 -- 172.21.15.1:0/3917481881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553acb60 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b553bc3c0 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555b553bd860 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 4 waiting for MDS map... 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2215d39640 1 -- 172.21.15.1:0/3917481881 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x555b56190400 con 0x555b5614fc00 2024-05-09T06:37:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2215d39640 1 -- 172.21.15.1:0/3917481881 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x555b553e1180 con 0x555b5614fc00 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 4 Got MDS map 83 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 10 main: JournalTool::main 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.899+0000 7f2219d92280 4 main: JournalTool: connecting to RADOS... 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 Processor -- start 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 -- start start 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b5615a480 con 0x555b56253000 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555b5615a600 con 0x555b56253400 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.900+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555b5615a780 con 0x555b56253800 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60074/0 (socket says 172.21.15.1:60074) 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 -- 172.21.15.1:0/1330138172 learned_addr learned my addr 172.21.15.1:0/1330138172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f221853e640 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2217d3d640 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 -- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 msgr2=0x555b553f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 -- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253400 msgr2=0x555b553f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 -- 172.21.15.1:0/1330138172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b553bda40 con 0x555b56253000 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2218d3f640 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x555b553b9fb0 tx=0x555b5611a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d828c71b59c0369 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2213d35640 1 -- 172.21.15.1:0/1330138172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553ac680 con 0x555b56253000 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2213d35640 1 -- 172.21.15.1:0/1330138172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b553b2c40 con 0x555b56253000 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2213d35640 1 -- 172.21.15.1:0/1330138172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553ac9c0 con 0x555b56253000 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2219d92280 1 -- 172.21.15.1:0/1330138172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 msgr2=0x555b553f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.901+0000 7f2219d92280 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x555b553b9fb0 tx=0x555b5611a480 comp rx=0 tx=0).stop 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 -- 172.21.15.1:0/1330138172 shutdown_connections 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 --2- 172.21.15.1:0/1330138172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 -- 172.21.15.1:0/1330138172 >> 172.21.15.1:0/1330138172 conn(0x555b5614f800 msgr2=0x555b55487800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 -- 172.21.15.1:0/1330138172 shutdown_connections 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 -- 172.21.15.1:0/1330138172 wait complete. 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 Processor -- start 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.902+0000 7f2219d92280 1 -- start start 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555b5615bc80 con 0x555b56253400 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555b5615bb00 con 0x555b56253000 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2219d92280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x555b5615b980 con 0x555b56253800 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 0x555b553f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60086/0 (socket says 172.21.15.1:60086) 2024-05-09T06:37:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 learned_addr learned my addr 172.21.15.1:0/991964161 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2217d3d640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2218d3f640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 msgr2=0x555b553f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253000 msgr2=0x555b553f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b56253000 0x555b553f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b553bd4a0 con 0x555b56253400 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2217d3d640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b56253800 0x555b553f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f221853e640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 0x555b553f8100 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x555b5626f440 tx=0x555b56271800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.903+0000 7f2212532640 1 -- 172.21.15.1:0/991964161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553ac680 con 0x555b56253400 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.904+0000 7f2212532640 1 -- 172.21.15.1:0/991964161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b553b28c0 con 0x555b56253400 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.904+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b553bc960 con 0x555b56253400 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.904+0000 7f2212532640 1 -- 172.21.15.1:0/991964161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b553ac9c0 con 0x555b56253400 2024-05-09T06:37:30.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.904+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b553bc780 con 0x555b56253400 2024-05-09T06:37:30.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.904+0000 7f2219d92280 4 main: JournalTool: resolving pool 15 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2212532640 1 -- 172.21.15.1:0/991964161 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x555b562c4c00 con 0x555b56253400 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2212532640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x555b562c8c00 0x555b553f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2212532640 1 -- 172.21.15.1:0/991964161 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x555b562c2c00 con 0x555b56253400 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2219d92280 4 main: JournalTool: creating IoCtx.. 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2219d92280 4 main: Executing for rank 0 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2219d92280 4 Filtering by inode '1099511627777' 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x555b56253c00 0x555b553f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2218d3f640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x555b562c8c00 0x555b553f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x555b562c9000 con 0x555b56253c00 2024-05-09T06:37:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.905+0000 7f2217d3d640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x555b56253c00 0x555b553f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.906+0000 7f2218d3f640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x555b562c8c00 0x555b553f9700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x555b553b9fb0 tx=0x555b5611bce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.906+0000 7f2217d3d640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x555b56253c00 0x555b553f8c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2217d3d640 1 -- 172.21.15.1:0/991964161 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x555b554f6000 con 0x555b56253c00 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 4 Pointer 400.00000000 is readable 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x555b562c9400 con 0x555b56253c00 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2217d3d640 1 -- 172.21.15.1:0/991964161 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x555b554f6000 con 0x555b56253c00 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 10 Header 0x400000 0x405f0e 0x40628a 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 10 Starting journal scan from offset 0x405f0e 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.909+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x555b562c9800 con 0x555b56253c00 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2217d3d640 1 -- 172.21.15.1:0/991964161 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x555b554f6000 con 0x555b56253c00 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 10 Parsing data, 0x37c bytes available 2024-05-09T06:37:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 10 Attempting decode at 0x405f0e 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 10 Consumed 0x37c bytes 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 10 Valid entry at 0x405f0e 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x555b562c9c00 0x555b553f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x555b55704000 con 0x555b562c9c00 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.910+0000 7f2218d3f640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x555b562c9c00 0x555b553f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.911+0000 7f2218d3f640 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x555b562c9c00 0x555b553f9180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stdout:Events by type: 2024-05-09T06:37:30.895 INFO:teuthology.orchestra.run.smithi001.stdout:Errors: 0 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2218d3f640 1 -- 172.21.15.1:0/991964161 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555b554f6500 con 0x555b562c9c00 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 Reached end of journal objects 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 Scanned objects, 0 missing, 1 valid 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 Events scanned, 0 gaps 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 Found 1 valid events 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 Selected 0 events events for processing 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 4 0 events 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x555b56253c00 msgr2=0x555b553f8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x555b56253c00 0x555b553f8c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x555b562c9c00 msgr2=0x555b553f9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x555b562c9c00 0x555b553f9180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x555b562c8c00 msgr2=0x555b553f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x555b562c8c00 0x555b553f9700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x555b553b9fb0 tx=0x555b5611bce0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 msgr2=0x555b553f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/991964161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b56253400 0x555b553f8100 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x555b5626f440 tx=0x555b56271800 comp rx=0 tx=0).stop 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 reap_dead start 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f221853e640 1 -- 172.21.15.1:0/991964161 reap_dead start 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 shutdown_connections 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 >> 172.21.15.1:0/991964161 conn(0x555b5614f800 msgr2=0x555b55487800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 shutdown_connections 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/991964161 wait complete. 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 msgr2=0x555b553f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x555b561b1380 tx=0x555b5611a420 comp rx=0 tx=0).stop 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 shutdown_connections 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x555b5614f400 0x555b553f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b5614fc00 0x555b553f7b80 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 --2- 172.21.15.1:0/3917481881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b5614e800 0x555b553f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 >> 172.21.15.1:0/3917481881 conn(0x555b55387000 msgr2=0x555b55486000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 shutdown_connections 2024-05-09T06:37:30.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:30.913+0000 7f2219d92280 1 -- 172.21.15.1:0/3917481881 wait complete. 2024-05-09T06:37:30.900 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.067010, result: Events by type: Errors: 0 2024-05-09T06:37:30.900 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-09T06:37:30.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.001+0000 7ff5b6122280 1 -- start start 2024-05-09T06:37:30.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.001+0000 7ff5b6122280 1 Processor -- start 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fa984c00 0x55e5fa9f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fa985c00 0x55e5fa9f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae180 con 0x55e5fa985800 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae300 con 0x55e5fa984c00 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae480 con 0x55e5fa985c00 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60104/0 (socket says 172.21.15.1:60104) 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 -- 172.21.15.1:0/1771152979 learned_addr learned my addr 172.21.15.1:0/1771152979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b4655640 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fa985c00 0x55e5fa9f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fa984c00 0x55e5fa9f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 -- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fa985c00 msgr2=0x55e5fa9f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fa985c00 0x55e5fa9f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 -- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fa984c00 msgr2=0x55e5fa9f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fa984c00 0x55e5fa9f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.002+0000 7ff5b3653640 1 -- 172.21.15.1:0/1771152979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fa9ba780 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b3653640 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x55e5fb799380 tx=0x55e5fb718480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d398bb2fd1f31569 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b164f640 1 -- 172.21.15.1:0/1771152979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aa680 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b164f640 1 -- 172.21.15.1:0/1771152979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5fa9b0a80 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b164f640 1 -- 172.21.15.1:0/1771152979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aab60 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fa9bb4a0 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e5fa9ba000 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 4 waiting for MDS map... 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b164f640 1 -- 172.21.15.1:0/1771152979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x55e5fb854a00 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b164f640 1 -- 172.21.15.1:0/1771152979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x55e5fa9df180 con 0x55e5fa985800 2024-05-09T06:37:30.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 4 Got MDS map 83 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 10 main: JournalTool::main 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.003+0000 7ff5b6122280 4 main: JournalTool: connecting to RADOS... 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 Processor -- start 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 -- start start 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 0x55e5fa9f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857c00 0x55e5fa9f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae600 con 0x55e5fb857800 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae780 con 0x55e5fb857400 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e5fb7ae900 con 0x55e5fb857c00 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b3e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b3e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60116/0 (socket says 172.21.15.1:60116) 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.004+0000 7ff5b3e54640 1 -- 172.21.15.1:0/1285016852 learned_addr learned my addr 172.21.15.1:0/1285016852 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b4655640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 0x55e5fa9f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3653640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857c00 0x55e5fa9f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 -- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857c00 msgr2=0x55e5fa9f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857c00 0x55e5fa9f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 -- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 msgr2=0x55e5fa9f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 0x55e5fa9f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 -- 172.21.15.1:0/1285016852 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fa9ba3c0 con 0x55e5fb857800 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b4655640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 0x55e5fa9f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x55e5fb86e660 tx=0x55e5fb8718c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa54a5a7ad598d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5af64b640 1 -- 172.21.15.1:0/1285016852 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aa680 con 0x55e5fb857800 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5af64b640 1 -- 172.21.15.1:0/1285016852 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5fa9b0fc0 con 0x55e5fb857800 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5af64b640 1 -- 172.21.15.1:0/1285016852 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aab60 con 0x55e5fb857800 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 -- 172.21.15.1:0/1285016852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 msgr2=0x55e5fa9f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x55e5fb86e660 tx=0x55e5fb8718c0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 -- 172.21.15.1:0/1285016852 shutdown_connections 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857c00 0x55e5fa9f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f5b80 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1285016852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857400 0x55e5fa9f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 -- 172.21.15.1:0/1285016852 >> 172.21.15.1:0/1285016852 conn(0x55e5fb857000 msgr2=0x55e5faa85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.005+0000 7ff5b6122280 1 -- 172.21.15.1:0/1285016852 shutdown_connections 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.006+0000 7ff5b6122280 1 -- 172.21.15.1:0/1285016852 wait complete. 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.006+0000 7ff5b6122280 1 Processor -- start 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.006+0000 7ff5b6122280 1 -- start start 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857400 0x55e5fa9f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857c00 0x55e5fa9f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7afc80 con 0x55e5fb857800 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e5fb7afb00 con 0x55e5fb857c00 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55e5fb7af980 con 0x55e5fb857400 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60132/0 (socket says 172.21.15.1:60132) 2024-05-09T06:37:30.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 learned_addr learned my addr 172.21.15.1:0/4134881208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3653640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857c00 0x55e5fa9f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857400 msgr2=0x55e5fa9f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b4655640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857400 0x55e5fa9f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fb857400 0x55e5fa9f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857c00 msgr2=0x55e5fa9f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857c00 0x55e5fa9f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fa9ba780 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3653640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fb857c00 0x55e5fa9f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b3e54640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f6c00 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x55e5fb86f320 tx=0x55e5fb871800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5ade48640 1 -- 172.21.15.1:0/4134881208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aa4e0 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5ade48640 1 -- 172.21.15.1:0/4134881208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5fa9b0e00 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5fa9bb4a0 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5ade48640 1 -- 172.21.15.1:0/4134881208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5fa9aa680 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fa9ba960 con 0x55e5fb857800 2024-05-09T06:37:30.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.007+0000 7ff5b6122280 4 main: JournalTool: resolving pool 15 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5ade48640 1 -- 172.21.15.1:0/4134881208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55e5fb8cec00 con 0x55e5fb857800 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5ade48640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55e5fb8c6c00 0x55e5fa9f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5ade48640 1 -- 172.21.15.1:0/4134881208 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x55e5fb8c8a00 con 0x55e5fb857800 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b6122280 4 main: JournalTool: creating IoCtx.. 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b6122280 4 main: Executing for rank 0 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b6122280 4 Filtering by inode '1099511627778' 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b4655640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55e5fb8c6c00 0x55e5fa9f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e5fb8c7000 0x55e5fa9f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x55e5fb8c7400 con 0x55e5fb8c7000 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b3653640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e5fb8c7000 0x55e5fa9f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.009+0000 7ff5b4655640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55e5fb8c6c00 0x55e5fa9f7700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55e5fa9b7f80 tx=0x55e5fb719f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b3653640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e5fb8c7000 0x55e5fa9f6100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b3653640 1 -- 172.21.15.1:0/4134881208 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55e5faaf4000 con 0x55e5fb8c7000 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 4 Pointer 400.00000000 is readable 2024-05-09T06:37:30.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-09T06:37:30.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x55e5fb8c7800 con 0x55e5fb8c7000 2024-05-09T06:37:30.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b3653640 1 -- 172.21.15.1:0/4134881208 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55e5faaf4000 con 0x55e5fb8c7000 2024-05-09T06:37:30.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 10 Header 0x400000 0x405f0e 0x40628a 2024-05-09T06:37:30.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 10 Starting journal scan from offset 0x405f0e 2024-05-09T06:37:30.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.010+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x55e5fb8c7c00 con 0x55e5fb8c7000 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b3653640 1 -- 172.21.15.1:0/4134881208 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55e5faaf4000 con 0x55e5fb8c7000 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 10 Parsing data, 0x37c bytes available 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 10 Attempting decode at 0x405f0e 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 10 Consumed 0x37c bytes 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 10 Valid entry at 0x405f0e 2024-05-09T06:37:30.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e5fad04000 0x55e5fa9f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x55e5fad04400 con 0x55e5fad04000 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b4655640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e5fad04000 0x55e5fa9f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.011+0000 7ff5b4655640 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e5fad04000 0x55e5fa9f5b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b4655640 1 -- 172.21.15.1:0/4134881208 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e5faaf4500 con 0x55e5fad04000 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stdout:Events by type: 2024-05-09T06:37:31.000 INFO:teuthology.orchestra.run.smithi001.stdout:Errors: 0 2024-05-09T06:37:31.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 Reached end of journal objects 2024-05-09T06:37:31.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 Scanned objects, 0 missing, 1 valid 2024-05-09T06:37:31.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 Events scanned, 0 gaps 2024-05-09T06:37:31.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 Found 1 valid events 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 Selected 0 events events for processing 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 4 0 events 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e5fb8c7000 msgr2=0x55e5fa9f6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55e5fb8c7000 0x55e5fa9f6100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e5fad04000 msgr2=0x55e5fa9f5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55e5fad04000 0x55e5fa9f5b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55e5fb8c6c00 msgr2=0x55e5fa9f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55e5fb8c6c00 0x55e5fa9f7700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55e5fa9b7f80 tx=0x55e5fb719f80 comp rx=0 tx=0).stop 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 msgr2=0x55e5fa9f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/4134881208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fb857800 0x55e5fa9f6c00 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x55e5fb86f320 tx=0x55e5fb871800 comp rx=0 tx=0).stop 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 reap_dead start 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b3e54640 1 -- 172.21.15.1:0/4134881208 reap_dead start 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 shutdown_connections 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 >> 172.21.15.1:0/4134881208 conn(0x55e5fb857000 msgr2=0x55e5faa85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 shutdown_connections 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/4134881208 wait complete. 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 msgr2=0x55e5fa9f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x55e5fb799380 tx=0x55e5fb718480 comp rx=0 tx=0).stop 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 shutdown_connections 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55e5fa985c00 0x55e5fa9f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5fa985800 0x55e5fa9f4b00 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 --2- 172.21.15.1:0/1771152979 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e5fa984c00 0x55e5fa9f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 >> 172.21.15.1:0/1771152979 conn(0x55e5fa985000 msgr2=0x55e5faa84800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 shutdown_connections 2024-05-09T06:37:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.012+0000 7ff5b6122280 1 -- 172.21.15.1:0/1771152979 wait complete. 2024-05-09T06:37:31.004 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.104384, result: Events by type: Errors: 0 2024-05-09T06:37:31.005 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-09T06:37:31.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 Processor -- start 2024-05-09T06:37:31.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 -- start start 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4bd250 0x5652fa435060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa56f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5652fa3adc80 con 0x5652fa4056d0 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5652fa3b1cc0 con 0x5652fa4bf120 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5652fa44e750 con 0x5652fa4bd250 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.107+0000 7f80f1859640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60150/0 (socket says 172.21.15.1:60150) 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 -- 172.21.15.1:0/2470728555 learned_addr learned my addr 172.21.15.1:0/2470728555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f205a640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa56f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1058640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4bd250 0x5652fa435060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 -- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4bd250 msgr2=0x5652fa435060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4bd250 0x5652fa435060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 -- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 msgr2=0x5652fa56f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa56f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 -- 172.21.15.1:0/2470728555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652fa4b8570 con 0x5652fa4056d0 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f205a640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa56f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f1859640 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2c0 tx=0x7f80e400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1877e078ea8528ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80e3fff640 1 -- 172.21.15.1:0/2470728555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4008070 con 0x5652fa4056d0 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80e3fff640 1 -- 172.21.15.1:0/2470728555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e400d500 con 0x5652fa4056d0 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80e3fff640 1 -- 172.21.15.1:0/2470728555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e400d930 con 0x5652fa4056d0 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f3d58f40 1 -- 172.21.15.1:0/2470728555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 msgr2=0x5652fa434b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.108+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2c0 tx=0x7f80e400b780 comp rx=0 tx=0).stop 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 -- 172.21.15.1:0/2470728555 shutdown_connections 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa56f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4bd250 0x5652fa435060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/2470728555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4056d0 0x5652fa434b20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 -- 172.21.15.1:0/2470728555 >> 172.21.15.1:0/2470728555 conn(0x5652fa406d10 msgr2=0x5652fa4dd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 -- 172.21.15.1:0/2470728555 shutdown_connections 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 -- 172.21.15.1:0/2470728555 wait complete. 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 Processor -- start 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.109+0000 7f80f3d58f40 1 -- start start 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4056d0 0x5652fa4d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 0x5652fa4d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa4b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5652fa4bca20 con 0x5652fa4bd250 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5652fa4d4c30 con 0x5652fa4bf120 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f3d58f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5652fa4bc780 con 0x5652fa4056d0 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 0x5652fa4d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 0x5652fa4d26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60162/0 (socket says 172.21.15.1:60162) 2024-05-09T06:37:31.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 -- 172.21.15.1:0/3270969876 learned_addr learned my addr 172.21.15.1:0/3270969876 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1859640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4056d0 0x5652fa4d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f205a640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa4b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 -- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4056d0 msgr2=0x5652fa4d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5652fa4056d0 0x5652fa4d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 -- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 msgr2=0x5652fa4b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652fa4bf120 0x5652fa4b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.110+0000 7f80f1058640 1 -- 172.21.15.1:0/3270969876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652fa4b7cc0 con 0x5652fa4bd250 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80f1058640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 0x5652fa4d26b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f80e800df70 tx=0x7f80e800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80e2ffd640 1 -- 172.21.15.1:0/3270969876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80030d0 con 0x5652fa4bd250 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80e2ffd640 1 -- 172.21.15.1:0/3270969876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e8016070 con 0x5652fa4bd250 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5652fa4052a0 con 0x5652fa4bd250 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80e2ffd640 1 -- 172.21.15.1:0/3270969876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80014d0 con 0x5652fa4bd250 2024-05-09T06:37:31.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.111+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652fa4d0dc0 con 0x5652fa4bd250 2024-05-09T06:37:31.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.112+0000 7f80e2ffd640 1 -- 172.21.15.1:0/3270969876 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80e8001670 con 0x5652fa4bd250 2024-05-09T06:37:31.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80e2ffd640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f80d406a810 0x7f80d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80e2ffd640 1 -- 172.21.15.1:0/3270969876 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f80e8089a80 con 0x5652fa4bd250 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f1859640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f80d406a810 0x7f80d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5652fa4d9c40 0x5652fa4043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5652fa4b6f10 con 0x5652fa4d9c40 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f205a640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5652fa4d9c40 0x5652fa4043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f1859640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f80d406a810 0x7f80d406ccd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2c0 tx=0x7f80e4009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.113+0000 7f80f205a640 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5652fa4d9c40 0x5652fa4043f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f205a640 1 -- 172.21.15.1:0/3270969876 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v109'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f80ec04f970 con 0x5652fa4d9c40 2024-05-09T06:37:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5652fa4d9c40 msgr2=0x5652fa4043f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5652fa4d9c40 0x5652fa4043f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f80d406a810 msgr2=0x7f80d406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f80d406a810 0x7f80d406ccd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b2c0 tx=0x7f80e4009000 comp rx=0 tx=0).stop 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 msgr2=0x5652fa4d26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 --2- 172.21.15.1:0/3270969876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652fa4bd250 0x5652fa4d26b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f80e800df70 tx=0x7f80e800b5b0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f205a640 1 -- 172.21.15.1:0/3270969876 reap_dead start 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 shutdown_connections 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 >> 172.21.15.1:0/3270969876 conn(0x5652fa406d10 msgr2=0x5652fa4d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 shutdown_connections 2024-05-09T06:37:31.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.117+0000 7f80f3d58f40 1 -- 172.21.15.1:0/3270969876 wait complete. 2024-05-09T06:37:31.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:37:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.267+0000 7fb689316640 1 Processor -- start 2024-05-09T06:37:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.267+0000 7fb689316640 1 -- start start 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 0x7fb6840718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb68415c4a0 0x7fb684071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb68406e6d0 0x7fb68406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb684158e30 con 0x7fb68406e6d0 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb684072480 con 0x7fb68415c4a0 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb689316640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb68415b420 con 0x7fb684158a50 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb68406e6d0 0x7fb68406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb68406e6d0 0x7fb68406eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60170/0 (socket says 172.21.15.1:60170) 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 -- 172.21.15.1:0/1231050630 learned_addr learned my addr 172.21.15.1:0/1231050630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6827fc640 1 --2- 172.21.15.1:0/1231050630 >> v2:172.21.15.6:3300/0 conn(0x7fb68415c4a0 0x7fb684071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb682ffd640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 0x7fb6840718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 -- 172.21.15.1:0/1231050630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 msgr2=0x7fb6840718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 0x7fb6840718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 -- 172.21.15.1:0/1231050630 >> v2:172.21.15.6:3300/0 conn(0x7fb68415c4a0 msgr2=0x7fb684071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 --2- 172.21.15.1:0/1231050630 >> v2:172.21.15.6:3300/0 conn(0x7fb68415c4a0 0x7fb684071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb6837fe640 1 -- 172.21.15.1:0/1231050630 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68415b6e0 con 0x7fb68406e6d0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.268+0000 7fb682ffd640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 0x7fb6840718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.269+0000 7fb6837fe640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb68406e6d0 0x7fb68406eb80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb6840b2770 tx=0x7fb67402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2537cc964fd3123 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.269+0000 7fb681ffb640 1 -- 172.21.15.1:0/1231050630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67400b2e0 con 0x7fb68406e6d0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.269+0000 7fb681ffb640 1 -- 172.21.15.1:0/1231050630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674009600 con 0x7fb68406e6d0 2024-05-09T06:37:31.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.269+0000 7fb681ffb640 1 -- 172.21.15.1:0/1231050630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67400b480 con 0x7fb68406e6d0 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 -- 172.21.15.1:0/1231050630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb68406e6d0 msgr2=0x7fb68406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb68406e6d0 0x7fb68406eb80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb6840b2770 tx=0x7fb67402c8a0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 -- 172.21.15.1:0/1231050630 shutdown_connections 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb68406e6d0 0x7fb68406eb80 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 --2- 172.21.15.1:0/1231050630 >> v2:172.21.15.6:3300/0 conn(0x7fb68415c4a0 0x7fb684071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 --2- 172.21.15.1:0/1231050630 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb684158a50 0x7fb6840718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 -- 172.21.15.1:0/1231050630 >> 172.21.15.1:0/1231050630 conn(0x7fb6840cc4d0 msgr2=0x7fb6840cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 -- 172.21.15.1:0/1231050630 shutdown_connections 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.270+0000 7fb689316640 1 -- 172.21.15.1:0/1231050630 wait complete. 2024-05-09T06:37:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.271+0000 7fb689316640 1 Processor -- start 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.271+0000 7fb689316640 1 -- start start 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb68406e6d0 0x7fb68414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb68415c4a0 0x7fb68415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb68415bf80 con 0x7fb684158a50 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb68415bca0 con 0x7fb68406e6d0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb689316640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb684070ea0 con 0x7fb68415c4a0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60184/0 (socket says 172.21.15.1:60184) 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 -- 172.21.15.1:0/3404768345 learned_addr learned my addr 172.21.15.1:0/3404768345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb682ffd640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb68406e6d0 0x7fb68414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6837fe640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb68415c4a0 0x7fb68415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 -- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb68415c4a0 msgr2=0x7fb68415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb68415c4a0 0x7fb68415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 -- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb68406e6d0 msgr2=0x7fb68414a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb68406e6d0 0x7fb68414a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.272+0000 7fb6827fc640 1 -- 172.21.15.1:0/3404768345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6840acae0 con 0x7fb684158a50 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb6827fc640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00e980 tx=0x7fb66c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c017070 con 0x7fb684158a50 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c00ca00 con 0x7fb684158a50 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00cdd0 con 0x7fb684158a50 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6840a9d80 con 0x7fb684158a50 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.273+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6840aa340 con 0x7fb684158a50 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.274+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6840752b0 con 0x7fb684158a50 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.274+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb66c01a070 con 0x7fb684158a50 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.275+0000 7fb67b7fe640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 0x7fb66406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.275+0000 7fb682ffd640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 0x7fb66406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.275+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fb66c08b080 con 0x7fb684158a50 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.275+0000 7fb682ffd640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 0x7fb66406e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006fd0 tx=0x7fb67c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.279+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb66c059830 con 0x7fb684158a50 2024-05-09T06:37:31.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.504+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb6840aa6e0 con 0x7fb684158a50 2024-05-09T06:37:31.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.586+0000 7fb67b7fe640 1 -- 172.21.15.1:0/3404768345 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) ==== 162+0+0 (secure 0 0 0) 0x7fb66c059130 con 0x7fb684158a50 2024-05-09T06:37:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:37:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.588+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 msgr2=0x7fb66406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.588+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 0x7fb66406e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006fd0 tx=0x7fb67c008040 comp rx=0 tx=0).stop 2024-05-09T06:37:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.588+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 msgr2=0x7fb6840a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.588+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00e980 tx=0x7fb66c00ee40 comp rx=0 tx=0).stop 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 shutdown_connections 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb66406be30 0x7fb66406e2f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb68415c4a0 0x7fb68415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb684158a50 0x7fb6840a9480 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 --2- 172.21.15.1:0/3404768345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb68406e6d0 0x7fb68414a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 >> 172.21.15.1:0/3404768345 conn(0x7fb6840cc4d0 msgr2=0x7fb684157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 shutdown_connections 2024-05-09T06:37:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.589+0000 7fb689316640 1 -- 172.21.15.1:0/3404768345 wait complete. 2024-05-09T06:37:31.583 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:31.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.705+0000 7f78b3730640 1 Processor -- start 2024-05-09T06:37:31.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 -- start start 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78ac161950 0x7f78ac0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ac0b33c0 con 0x7f78ac0d4940 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f78ac0d4dd0 con 0x7f78ac161950 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.706+0000 7f78b3730640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac0b3660 con 0x7f78ac0d0f80 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b14cc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78ac161950 0x7f78ac0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:47326/0 (socket says 172.21.15.1:47326) 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 -- 172.21.15.1:0/27773597 learned_addr learned my addr 172.21.15.1:0/27773597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 -- 172.21.15.1:0/27773597 >> v2:172.21.15.6:3300/0 conn(0x7f78ac161950 msgr2=0x7f78ac0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 --2- 172.21.15.1:0/27773597 >> v2:172.21.15.6:3300/0 conn(0x7f78ac161950 0x7f78ac0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 -- 172.21.15.1:0/27773597 >> v1:172.21.15.6:6789/0 conn(0x7f78ac0d4940 legacy=0x7f78ac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.707+0000 7f78b1ccd640 1 -- 172.21.15.1:0/27773597 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac0b3940 con 0x7f78ac0d0f80 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.708+0000 7f78b1ccd640 1 --2- 172.21.15.1:0/27773597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7ea8c778fd4a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.708+0000 7f78abfff640 1 -- 172.21.15.1:0/27773597 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c036070 con 0x7f78ac0d0f80 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.708+0000 7f78abfff640 1 -- 172.21.15.1:0/27773597 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c031600 con 0x7f78ac0d0f80 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.708+0000 7f78abfff640 1 -- 172.21.15.1:0/27773597 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0319d0 con 0x7f78ac0d0f80 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 -- 172.21.15.1:0/27773597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 msgr2=0x7f78ac0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 --2- 172.21.15.1:0/27773597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c02a710 comp rx=0 tx=0).stop 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 -- 172.21.15.1:0/27773597 shutdown_connections 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 --2- 172.21.15.1:0/27773597 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac0d0f80 0x7f78ac0d2940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 --2- 172.21.15.1:0/27773597 >> v2:172.21.15.6:3300/0 conn(0x7f78ac161950 0x7f78ac0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 -- 172.21.15.1:0/27773597 >> 172.21.15.1:0/27773597 conn(0x7f78ac0d3e20 msgr2=0x7f78ac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 -- 172.21.15.1:0/27773597 shutdown_connections 2024-05-09T06:37:31.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.709+0000 7f78b3730640 1 -- 172.21.15.1:0/27773597 wait complete. 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.710+0000 7f78b3730640 1 Processor -- start 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.710+0000 7f78b3730640 1 -- start start 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78ac0d0f80 0x7f78ac157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac161950 0x7f78ac0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac0b52d0 con 0x7f78ac0b1b10 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac0b5150 con 0x7f78ac0d0f80 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b3730640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac0b5450 con 0x7f78ac161950 2024-05-09T06:37:31.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60186/0 (socket says 172.21.15.1:60186) 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 -- 172.21.15.1:0/2048902743 learned_addr learned my addr 172.21.15.1:0/2048902743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b14cc640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78ac0d0f80 0x7f78ac157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b0ccb640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac161950 0x7f78ac0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 -- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac161950 msgr2=0x7f78ac0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac161950 0x7f78ac0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 -- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78ac0d0f80 msgr2=0x7f78ac157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78ac0d0f80 0x7f78ac157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 -- 172.21.15.1:0/2048902743 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac1583d0 con 0x7f78ac0b1b10 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.711+0000 7f78b1ccd640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.712+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c036050 con 0x7f78ac0b1b10 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.712+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac0ccd50 con 0x7f78ac0b1b10 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.712+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c042070 con 0x7f78ac0b1b10 2024-05-09T06:37:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.712+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c009510 con 0x7f78ac0b1b10 2024-05-09T06:37:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.712+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac0cd310 con 0x7f78ac0b1b10 2024-05-09T06:37:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.713+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac0b3190 con 0x7f78ac0b1b10 2024-05-09T06:37:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.713+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f789c007bf0 con 0x7f78ac0b1b10 2024-05-09T06:37:31.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.714+0000 7f78a9ffb640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 0x7f789806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.714+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f789c0abd80 con 0x7f78ac0b1b10 2024-05-09T06:37:31.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.714+0000 7f78b14cc640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 0x7f789806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.715+0000 7f78b14cc640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 0x7f789806e320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0b6e20 tx=0x7f78a4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.719+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78ac0b3190 con 0x7f78ac0b1b10 2024-05-09T06:37:31.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.942+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78ac0752b0 con 0x7f78ac0b1b10 2024-05-09T06:37:31.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.942+0000 7f78a9ffb640 1 -- 172.21.15.1:0/2048902743 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5163 (secure 0 0 0) 0x7f789c079e60 con 0x7f78ac0b1b10 2024-05-09T06:37:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:31.923 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:37:31.586143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"a","rank":0,"incarnation":84,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6835/356239460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":356239460},{"type":"v1","addr":"172.21.15.1:6835","nonce":356239460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:37:31.924 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 84 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 msgr2=0x7f789806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 0x7f789806e320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0b6e20 tx=0x7f78a4009190 comp rx=0 tx=0).stop 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 msgr2=0x7f78ac0b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 shutdown_connections 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f789806be60 0x7f789806e320 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78ac0b1b10 0x7f78ac0b1fa0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78ac161950 0x7f78ac0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 --2- 172.21.15.1:0/2048902743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78ac0d0f80 0x7f78ac157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 >> 172.21.15.1:0/2048902743 conn(0x7f78ac0d3e20 msgr2=0x7f78ac0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 shutdown_connections 2024-05-09T06:37:31.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:31.944+0000 7f78b3730640 1 -- 172.21.15.1:0/2048902743 wait complete. 2024-05-09T06:37:32.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:35.082530+0000', 'modified': '2024-05-09T06:37:31.586143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9282': {'gid': 9282, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6835/356239460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 356239460}, {'type': 'v1', 'addr': '172.21.15.1:6835', 'nonce': 356239460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:37:32.339 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9282:up:replay 2024-05-09T06:37:33.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.476+0000 7fc4bdf77640 1 Processor -- start 2024-05-09T06:37:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.476+0000 7fc4bdf77640 1 -- start start 2024-05-09T06:37:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4bdf77640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc4b8099b30 0x7fc4b80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4bdf77640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4b815e4b0 0x7fc4b815f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4bdf77640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b8099930 con 0x7fc4b80c6860 2024-05-09T06:37:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4bdf77640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80b1e20 con 0x7fc4b8099b30 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4bdf77640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b815ee90 con 0x7fc4b815e4b0 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4b6ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc4b8099b30 0x7fc4b80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4b6ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc4b8099b30 0x7fc4b80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60198/0 (socket says 172.21.15.1:60198) 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4b6ffd640 1 -- 172.21.15.1:0/842707544 learned_addr learned my addr 172.21.15.1:0/842707544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.477+0000 7fc4b77fe640 1 --1- 172.21.15.1:0/842707544 >> v1:172.21.15.1:6789/0 conn(0x7fc4b80c6860 0x7fc4b80c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55918/0 (socket says 172.21.15.1:55918) 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b7fff640 1 --2- 172.21.15.1:0/842707544 >> v2:172.21.15.6:3300/0 conn(0x7fc4b815e4b0 0x7fc4b815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b6ffd640 1 -- 172.21.15.1:0/842707544 >> v2:172.21.15.6:3300/0 conn(0x7fc4b815e4b0 msgr2=0x7fc4b815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/842707544 >> v2:172.21.15.6:3300/0 conn(0x7fc4b815e4b0 0x7fc4b815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b6ffd640 1 -- 172.21.15.1:0/842707544 >> v1:172.21.15.1:6789/0 conn(0x7fc4b80c6860 legacy=0x7fc4b80c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b6ffd640 1 -- 172.21.15.1:0/842707544 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b8162930 con 0x7fc4b8099b30 2024-05-09T06:37:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.478+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/842707544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b80b2aa0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80ca960 tx=0x7fc4ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfd6fdfc784624d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4b67fc640 1 -- 172.21.15.1:0/842707544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac037070 con 0x7fc4b8099b30 2024-05-09T06:37:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4b67fc640 1 -- 172.21.15.1:0/842707544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ac032440 con 0x7fc4b8099b30 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4b67fc640 1 -- 172.21.15.1:0/842707544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac032810 con 0x7fc4b8099b30 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 -- 172.21.15.1:0/842707544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 msgr2=0x7fc4b80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/842707544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b80b2aa0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80ca960 tx=0x7fc4ac02a270 comp rx=0 tx=0).stop 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 -- 172.21.15.1:0/842707544 shutdown_connections 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/842707544 >> v2:172.21.15.6:3300/0 conn(0x7fc4b815e4b0 0x7fc4b815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/842707544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b80b2aa0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 -- 172.21.15.1:0/842707544 >> 172.21.15.1:0/842707544 conn(0x7fc4b80b1fb0 msgr2=0x7fc4b80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 -- 172.21.15.1:0/842707544 shutdown_connections 2024-05-09T06:37:33.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.479+0000 7fc4bdf77640 1 -- 172.21.15.1:0/842707544 wait complete. 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.480+0000 7fc4bdf77640 1 Processor -- start 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 -- start start 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 0x7fc4b80b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4b815e4b0 0x7fc4b80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b815f860 con 0x7fc4b8099b30 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b815eb20 con 0x7fc4b80c6860 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4bdf77640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b815f9e0 con 0x7fc4b815e4b0 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.481+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60200/0 (socket says 172.21.15.1:60200) 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 -- 172.21.15.1:0/1169733965 learned_addr learned my addr 172.21.15.1:0/1169733965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 0x7fc4b80b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b7fff640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4b815e4b0 0x7fc4b80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 -- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4b815e4b0 msgr2=0x7fc4b80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4b815e4b0 0x7fc4b80cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 -- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 msgr2=0x7fc4b80b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 0x7fc4b80b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 -- 172.21.15.1:0/1169733965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b80c82c0 con 0x7fc4b8099b30 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 0x7fc4b80b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4b77fe640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c430 tx=0x7fc4a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80074e0 con 0x7fc4b8099b30 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a8007680 con 0x7fc4b8099b30 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8004a00 con 0x7fc4b8099b30 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b80cbba0 con 0x7fc4b8099b30 2024-05-09T06:37:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.482+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b80cc160 con 0x7fc4b8099b30 2024-05-09T06:37:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.483+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b80752b0 con 0x7fc4b8099b30 2024-05-09T06:37:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.484+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4a801a020 con 0x7fc4b8099b30 2024-05-09T06:37:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.484+0000 7fc49ffff640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 0x7fc4a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.484+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fc4a808a790 con 0x7fc4b8099b30 2024-05-09T06:37:33.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.484+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 0x7fc4a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:33.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.485+0000 7fc4b6ffd640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 0x7fc4a406e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac034000 tx=0x7fc4ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:33.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.489+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4b8099930 con 0x7fc4b8099b30 2024-05-09T06:37:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.716+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4b806ce30 con 0x7fc4b8099b30 2024-05-09T06:37:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.716+0000 7fc49ffff640 1 -- 172.21.15.1:0/1169733965 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5163 (secure 0 0 0) 0x7fc4a8058fd0 con 0x7fc4b8099b30 2024-05-09T06:37:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:33.697 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:37:32.591135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"a","rank":0,"incarnation":84,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.1:6835/356239460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":356239460},{"type":"v1","addr":"172.21.15.1:6835","nonce":356239460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:37:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 86 2024-05-09T06:37:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.718+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 msgr2=0x7fc4a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.718+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 0x7fc4a406e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac034000 tx=0x7fc4ac02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.718+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 msgr2=0x7fc4b8156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.718+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c430 tx=0x7fc4a800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 shutdown_connections 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc4a406be30 0x7fc4a406e2f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc4b815e4b0 0x7fc4b80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4b80c6860 0x7fc4b80b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 --2- 172.21.15.1:0/1169733965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b8099b30 0x7fc4b8156a90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 >> 172.21.15.1:0/1169733965 conn(0x7fc4b80b1fb0 msgr2=0x7fc4b806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 shutdown_connections 2024-05-09T06:37:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:33.719+0000 7fc4bdf77640 1 -- 172.21.15.1:0/1169733965 wait complete. 2024-05-09T06:37:34.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:35.082530+0000', 'modified': '2024-05-09T06:37:32.591135+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9282': {'gid': 9282, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.1:6835/356239460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 356239460}, {'type': 'v1', 'addr': '172.21.15.1:6835', 'nonce': 356239460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:37:34.113 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9282:up:rejoin 2024-05-09T06:37:35.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.250+0000 7f7bad631640 1 Processor -- start 2024-05-09T06:37:35.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 -- start start 2024-05-09T06:37:35.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7ba80b2a60 0x7f7ba815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 0x7f7ba806e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba8070f40 con 0x7f7ba815db10 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba80cf170 con 0x7f7ba80b2a60 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.251+0000 7f7bad631640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba80cf450 con 0x7f7ba815e500 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba67fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7ba80b2a60 0x7f7ba815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba6ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7ba815db10 0x7f7ba80715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55926/0 (socket says 172.21.15.1:55926) 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba6ffd640 1 -- 172.21.15.1:0/813609824 learned_addr learned my addr 172.21.15.1:0/813609824 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba77fe640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 0x7f7ba806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba67fc640 1 -- 172.21.15.1:0/813609824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 msgr2=0x7f7ba806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba67fc640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 0x7f7ba806e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba67fc640 1 -- 172.21.15.1:0/813609824 >> v1:172.21.15.1:6789/0 conn(0x7f7ba815db10 legacy=0x7f7ba80715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba67fc640 1 -- 172.21.15.1:0/813609824 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba8070c10 con 0x7f7ba80b2a60 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba5ffb640 1 -- 172.21.15.1:0/813609824 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661549708 0 0) 0x7f7ba8070f40 con 0x7f7ba815db10 2024-05-09T06:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.252+0000 7f7ba77fe640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 0x7f7ba806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7ba67fc640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba80b2a60 0x7f7ba815dfc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7b900097a0 tx=0x7f7b9002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836524893cf0fcec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7ba5ffb640 1 -- 172.21.15.1:0/813609824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90037070 con 0x7f7ba80b2a60 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7ba5ffb640 1 -- 172.21.15.1:0/813609824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9002ae50 con 0x7f7ba80b2a60 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7ba5ffb640 1 -- 172.21.15.1:0/813609824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90032660 con 0x7f7ba80b2a60 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 -- 172.21.15.1:0/813609824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba80b2a60 msgr2=0x7f7ba815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba80b2a60 0x7f7ba815dfc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7b900097a0 tx=0x7f7b9002a100 comp rx=0 tx=0).stop 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 -- 172.21.15.1:0/813609824 shutdown_connections 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba815e500 0x7f7ba806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 --2- 172.21.15.1:0/813609824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba80b2a60 0x7f7ba815dfc0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 -- 172.21.15.1:0/813609824 >> 172.21.15.1:0/813609824 conn(0x7f7ba80cc4b0 msgr2=0x7f7ba80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 -- 172.21.15.1:0/813609824 shutdown_connections 2024-05-09T06:37:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.253+0000 7f7bad631640 1 -- 172.21.15.1:0/813609824 wait complete. 2024-05-09T06:37:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.255+0000 7f7bad631640 1 Processor -- start 2024-05-09T06:37:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.255+0000 7f7bad631640 1 -- start start 2024-05-09T06:37:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba80b2a60 0x7f7ba80b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ba815e500 0x7f7ba80addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba806cd30 con 0x7f7ba815db10 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba815e910 con 0x7f7ba815e500 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7bad631640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba806ceb0 con 0x7f7ba80b2a60 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60216/0 (socket says 172.21.15.1:60216) 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 -- 172.21.15.1:0/3492001190 learned_addr learned my addr 172.21.15.1:0/3492001190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba6ffd640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba80b2a60 0x7f7ba80b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba77fe640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ba815e500 0x7f7ba80addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 -- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba80b2a60 msgr2=0x7f7ba80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba80b2a60 0x7f7ba80b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 -- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ba815e500 msgr2=0x7f7ba80addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ba815e500 0x7f7ba80addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 -- 172.21.15.1:0/3492001190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba80b0d60 con 0x7f7ba815db10 2024-05-09T06:37:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7ba67fc640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7b900098c0 tx=0x7f7b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.256+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90042070 con 0x7f7ba815db10 2024-05-09T06:37:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.257+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9003e670 con 0x7f7ba815db10 2024-05-09T06:37:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.257+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba80b04c0 con 0x7f7ba815db10 2024-05-09T06:37:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.257+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90037040 con 0x7f7ba815db10 2024-05-09T06:37:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.257+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba80b0a80 con 0x7f7ba815db10 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.258+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba80752b0 con 0x7f7ba815db10 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.258+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b90045070 con 0x7f7ba815db10 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.259+0000 7f7b9f7fe640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 0x7f7b8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.259+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f7b900abb00 con 0x7f7ba815db10 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.259+0000 7f7ba6ffd640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 0x7f7b8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.259+0000 7f7ba6ffd640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 0x7f7b8806e320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7b980012e0 tx=0x7f7b98007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.263+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b9007a330 con 0x7f7ba815db10 2024-05-09T06:37:35.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.488+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ba806d910 con 0x7f7ba815db10 2024-05-09T06:37:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.488+0000 7f7b9f7fe640 1 -- 172.21.15.1:0/3492001190 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5170 (secure 0 0 0) 0x7f7b900ab020 con 0x7f7ba815db10 2024-05-09T06:37:35.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:35.469 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:37:34.590315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6835/356239460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":356239460},{"type":"v1","addr":"172.21.15.1:6835","nonce":356239460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9282,"qdb_cluster":[9282]},"id":7}]} 2024-05-09T06:37:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 87 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 msgr2=0x7f7b8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 0x7f7b8806e320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7b980012e0 tx=0x7f7b98007ed0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 msgr2=0x7f7ba80ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7b900098c0 tx=0x7f7b900046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 shutdown_connections 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b8806be60 0x7f7b8806e320 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ba815e500 0x7f7ba80addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ba815db10 0x7f7ba80ad890 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 --2- 172.21.15.1:0/3492001190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ba80b2a60 0x7f7ba80b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 >> 172.21.15.1:0/3492001190 conn(0x7f7ba80cc4b0 msgr2=0x7f7ba8069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 shutdown_connections 2024-05-09T06:37:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:35.490+0000 7f7bad631640 1 -- 172.21.15.1:0/3492001190 wait complete. 2024-05-09T06:37:35.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:35.082530+0000', 'modified': '2024-05-09T06:37:34.590315+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9282': {'gid': 9282, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.1:6835/356239460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 356239460}, {'type': 'v1', 'addr': '172.21.15.1:6835', 'nonce': 356239460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9282, 'qdb_cluster': [9282]} 2024-05-09T06:37:35.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:37:35.884 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:37:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.015+0000 7fa8eb82a640 1 Processor -- start 2024-05-09T06:37:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.015+0000 7fa8eb82a640 1 -- start start 2024-05-09T06:37:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8eb82a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8eb82a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8eb82a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e415cc40 con 0x7fa8e41565e0 2024-05-09T06:37:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8eb82a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e415fd30 con 0x7fa8e40b0170 2024-05-09T06:37:35.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8eb82a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e415cf20 con 0x7fa8e415c680 2024-05-09T06:37:35.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e9dc7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:35.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e9dc7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:54198/0 (socket says 172.21.15.1:54198) 2024-05-09T06:37:35.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:47364/0 (socket says 172.21.15.1:47364) 2024-05-09T06:37:35.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e9dc7640 1 -- 172.21.15.1:0/624708279 learned_addr learned my addr 172.21.15.1:0/624708279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.016+0000 7fa8e95c6640 1 --1- 172.21.15.1:0/624708279 >> v1:172.21.15.6:6789/0 conn(0x7fa8e41565e0 0x7fa8e4074530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:53940/0 (socket says 172.21.15.1:53940) 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/624708279 >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 msgr2=0x7fa8e41601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/624708279 >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/624708279 >> v1:172.21.15.6:6789/0 conn(0x7fa8e41565e0 legacy=0x7fa8e4074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/624708279 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e415d200 con 0x7fa8e415c680 2024-05-09T06:37:35.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/624708279 >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.017+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/624708279 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f7fafa491f84c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8dbfff640 1 -- 172.21.15.1:0/624708279 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037070 con 0x7fa8e415c680 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8dbfff640 1 -- 172.21.15.1:0/624708279 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4032440 con 0x7fa8e415c680 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8dbfff640 1 -- 172.21.15.1:0/624708279 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4032810 con 0x7fa8e415c680 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 -- 172.21.15.1:0/624708279 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 msgr2=0x7fa8e40afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:35.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/624708279 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402a180 comp rx=0 tx=0).stop 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 -- 172.21.15.1:0/624708279 shutdown_connections 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/624708279 >> v2:172.21.15.6:3300/0 conn(0x7fa8e40b0170 0x7fa8e41601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/624708279 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e40afc30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 -- 172.21.15.1:0/624708279 >> 172.21.15.1:0/624708279 conn(0x7fa8e40b1fb0 msgr2=0x7fa8e40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 -- 172.21.15.1:0/624708279 shutdown_connections 2024-05-09T06:37:36.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.018+0000 7fa8eb82a640 1 -- 172.21.15.1:0/624708279 wait complete. 2024-05-09T06:37:36.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.019+0000 7fa8eb82a640 1 Processor -- start 2024-05-09T06:37:36.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.019+0000 7fa8eb82a640 1 -- start start 2024-05-09T06:37:36.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 0x7fa8e4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4160700 con 0x7fa8e40b0170 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e41569c0 con 0x7fa8e41565e0 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8eb82a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e4160880 con 0x7fa8e415c680 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60218/0 (socket says 172.21.15.1:60218) 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.020+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3979770820 learned_addr learned my addr 172.21.15.1:0/3979770820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 0x7fa8e4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 msgr2=0x7fa8e415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 msgr2=0x7fa8e4152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 0x7fa8e4152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3979770820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e4154610 con 0x7fa8e40b0170 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 0x7fa8e4152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000c790 tx=0x7fa8e000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8d9ffb640 1 -- 172.21.15.1:0/3979770820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x7fa8e40b0170 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8d9ffb640 1 -- 172.21.15.1:0/3979770820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e000a640 con 0x7fa8e40b0170 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e4152fc0 con 0x7fa8e40b0170 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8d9ffb640 1 -- 172.21.15.1:0/3979770820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000a9f0 con 0x7fa8e40b0170 2024-05-09T06:37:36.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.021+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e4154b90 con 0x7fa8e40b0170 2024-05-09T06:37:36.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.023+0000 7fa8d9ffb640 1 -- 172.21.15.1:0/3979770820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8e001a070 con 0x7fa8e40b0170 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.023+0000 7fa8d9ffb640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.023+0000 7fa8d9ffb640 1 -- 172.21.15.1:0/3979770820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fa8e008a5b0 con 0x7fa8e40b0170 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.023+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.024+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 0x7fa8c806e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4034000 tx=0x7fa8d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.024+0000 7fa8ee613740 1 Processor -- start 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.024+0000 7fa8ee613740 1 -- start start 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e22350 0x55b263e685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b263d854d0 con 0x55b263e4c670 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b263d2d3e0 con 0x55b263e69f10 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b263d81490 con 0x55b263e22350 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60224/0 (socket says 172.21.15.1:60224) 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 -- 172.21.15.1:0/2666007727 learned_addr learned my addr 172.21.15.1:0/2666007727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e22350 0x55b263e685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 -- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e22350 msgr2=0x55b263e685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e22350 0x55b263e685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 -- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 msgr2=0x55b263e6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.025+0000 7fa8e95c6640 1 -- 172.21.15.1:0/2666007727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b263a992d0 con 0x55b263e4c670 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0060d10 tx=0x7fa8e0058820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4c56f8c89457bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ceffd640 1 -- 172.21.15.1:0/2666007727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0054d80 con 0x55b263e4c670 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ceffd640 1 -- 172.21.15.1:0/2666007727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e0060db0 con 0x55b263e4c670 2024-05-09T06:37:36.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ceffd640 1 -- 172.21.15.1:0/2666007727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e005dba0 con 0x55b263e4c670 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 -- 172.21.15.1:0/2666007727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 msgr2=0x55b263e4ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0060d10 tx=0x7fa8e0058820 comp rx=0 tx=0).stop 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 -- 172.21.15.1:0/2666007727 shutdown_connections 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e22350 0x55b263e685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2666007727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e4ca50 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 -- 172.21.15.1:0/2666007727 >> 172.21.15.1:0/2666007727 conn(0x55b263e1ec80 msgr2=0x55b263e67480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 -- 172.21.15.1:0/2666007727 shutdown_connections 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.026+0000 7fa8ee613740 1 -- 172.21.15.1:0/2666007727 wait complete. 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.027+0000 7fa8ee613740 1 -- start start 2024-05-09T06:37:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.027+0000 7fa8ee613740 1 Processor -- start 2024-05-09T06:37:36.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b263e230f0 con 0x55b263e69f10 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b263d408e0 con 0x55b263e4c670 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b263e1cbb0 con 0x55b263e48220 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8ee613740 10 client.0 _ll_register_callbacks cb 0x55b263d0d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60240/0 (socket says 172.21.15.1:60240) 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/2197031565 learned_addr learned my addr 172.21.15.1:0/2197031565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 msgr2=0x55b263e780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 msgr2=0x55b263e5fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e5fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.028+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/2197031565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b263b636b0 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4035040 tx=0x7fa8d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eece5de9a1b96916 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037040 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fa8d403ed40 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e0060db0 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa8d4056050 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 10 client.9357 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55b263e5f730 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e0060db0 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fa8d4059020 con 0x55b263e69f10 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 10 client.9357 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 10 client.9357 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 20 client.9357 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4d710 0x55b263e4daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 4 client.9357 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b263d92c00 con 0x55b263e4d710 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.029+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4d710 0x55b263e4daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.030+0000 7fa8c3fff640 10 client.9357 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.030+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4d710 0x55b263e4daf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d623cf5c3d009f server_cookie=9d98e89c4d3dbff0 in_seq=0 out_seq=0 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.030+0000 7fa8c3fff640 1 -- 172.21.15.1:0/2197031565 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b263d92c00 con 0x55b263e4d710 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.030+0000 7fa8c3fff640 10 client.9357 handle_command_reply: tid=0 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.030+0000 7fa8ee613740 1 client.9357 shutdown 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 msgr2=0x55b263e77ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4035040 tx=0x7fa8d4004730 comp rx=0 tx=0).stop 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 shutdown_connections 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4d710 0x55b263e4daf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e69f10 0x55b263e77ba0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8c3fff640 0 client.9357 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 --2- 172.21.15.1:0/2197031565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e5fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 >> 172.21.15.1:0/2197031565 conn(0x55b263e22350 msgr2=0x55b263e66c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 shutdown_connections 2024-05-09T06:37:36.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 1 -- 172.21.15.1:0/2197031565 wait complete. 2024-05-09T06:37:36.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.031+0000 7fa8ee613740 20 client.9357 trim_cache size 0 max 16384 2024-05-09T06:37:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.049+0000 7fa8ee613740 1 Processor -- start 2024-05-09T06:37:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.049+0000 7fa8ee613740 1 -- start start 2024-05-09T06:37:36.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e66ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e78070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e41ffc40 con 0x55b263e4c670 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e40cc9f0 con 0x55b263e69f10 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e40cd040 con 0x55b263e48220 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60252/0 (socket says 172.21.15.1:60252) 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/451837548 learned_addr learned my addr 172.21.15.1:0/451837548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e66ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e78070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 msgr2=0x55b263e66ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e66ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 msgr2=0x55b263e78070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e78070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.050+0000 7fa8e8dc5640 1 -- 172.21.15.1:0/451837548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b263e1d890 con 0x55b263e4c670 2024-05-09T06:37:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402a6f0 tx=0x7fa8d4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f791115ecf5ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ce7fc640 1 -- 172.21.15.1:0/451837548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037040 con 0x55b263e4c670 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ce7fc640 1 -- 172.21.15.1:0/451837548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d404e070 con 0x55b263e4c670 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ce7fc640 1 -- 172.21.15.1:0/451837548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d403ed40 con 0x55b263e4c670 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 -- 172.21.15.1:0/451837548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 msgr2=0x55b263e77b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402a6f0 tx=0x7fa8d4047000 comp rx=0 tx=0).stop 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 -- 172.21.15.1:0/451837548 shutdown_connections 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e69f10 0x55b263e78070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e4c670 0x55b263e77b30 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 --2- 172.21.15.1:0/451837548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e48220 0x55b263e66ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 -- 172.21.15.1:0/451837548 >> 172.21.15.1:0/451837548 conn(0x55b263e22350 msgr2=0x55b263e4a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 -- 172.21.15.1:0/451837548 shutdown_connections 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.051+0000 7fa8ee613740 1 -- 172.21.15.1:0/451837548 wait complete. 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.052+0000 7fa8ee613740 1 -- start start 2024-05-09T06:37:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.052+0000 7fa8ee613740 1 Processor -- start 2024-05-09T06:37:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e69f10 0x55b263e4daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b263e4f9d0 con 0x55b263e48220 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b263e4fdc0 con 0x55b263e4c670 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55b263e4aea0 con 0x55b263e69f10 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8ee613740 10 client.0 _ll_register_callbacks cb 0x7fa8dc0090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60268/0 (socket says 172.21.15.1:60268) 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3275219970 learned_addr learned my addr 172.21.15.1:0/3275219970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e9dc7640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e69f10 0x55b263e4daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e69f10 msgr2=0x55b263e4daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e69f10 0x55b263e4daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 msgr2=0x55b263e6abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e6abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 -- 172.21.15.1:0/3275219970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b263ee1d80 con 0x55b263e48220 2024-05-09T06:37:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8e95c6640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0088930 tx=0x7fa8e008a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b0902954d25327 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.053+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0088440 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fa8e0060db0 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d404e070 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa8e00885e0 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 10 client.9363 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55b263ee2380 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d404e070 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fa8e005dba0 con 0x55b263e48220 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 10 client.9363 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 10 client.9363 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 20 client.9363 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4e1d0 0x55b263e4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 4 client.9363 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:37:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "status"}) -- 0x55b263d8cf50 con 0x55b263e4e1d0 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4e1d0 0x55b263e4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8c3fff640 10 client.9363 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.054+0000 7fa8e8dc5640 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4e1d0 0x55b263e4e5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54c4911b174c07c server_cookie=145d83853d55aba2 in_seq=0 out_seq=0 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8c3fff640 1 -- 172.21.15.1:0/3275219970 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b263d8cf50 con 0x55b263e4e1d0 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8c3fff640 10 client.9363 handle_command_reply: tid=0 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 client.9363 shutdown 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 msgr2=0x55b263e6a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0088930 tx=0x7fa8e008a050 comp rx=0 tx=0).stop 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 shutdown_connections 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55b263e4e1d0 0x55b263e4e5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55b263e69f10 0x55b263e4daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8c3fff640 0 client.9363 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b263e4c670 0x55b263e6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 --2- 172.21.15.1:0/3275219970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b263e48220 0x55b263e6a6a0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 >> 172.21.15.1:0/3275219970 conn(0x55b263e22350 msgr2=0x55b263e5fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 shutdown_connections 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 1 -- 172.21.15.1:0/3275219970 wait complete. 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.055+0000 7fa8ee613740 20 client.9363 trim_cache size 0 max 16384 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9282, 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:37:36.042 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4681120559999998, 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 87, 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 109, 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 109, 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 5.1774836049999999 2024-05-09T06:37:36.043 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 msgr2=0x7fa8c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 0x7fa8c806e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4034000 tx=0x7fa8d402e000 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 msgr2=0x7fa8e40c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000c790 tx=0x7fa8e000cc50 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 shutdown_connections 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8e415c680 0x7fa8e415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8e41565e0 0x7fa8e4152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 --2- 172.21.15.1:0/3979770820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8e40b0170 0x7fa8e40c6e50 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 >> 172.21.15.1:0/3979770820 conn(0x7fa8e40b1fb0 msgr2=0x7fa8e40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 shutdown_connections 2024-05-09T06:37:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.056+0000 7fa8eb82a640 1 -- 172.21.15.1:0/3979770820 wait complete. 2024-05-09T06:37:36.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-09T06:37:36.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:36.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-09T06:37:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.176+0000 7f925069e640 1 Processor -- start 2024-05-09T06:37:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.176+0000 7f925069e640 1 -- start start 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.177+0000 7f925069e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92480b2ed0 0x7f92480ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.177+0000 7f925069e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92480cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.177+0000 7f925069e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92480d1cc0 con 0x7f92480d2630 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.177+0000 7f925069e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f92480d4d10 con 0x7f92480b2ed0 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.177+0000 7f925069e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f92480d1f80 con 0x7f92480b2110 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92480b2ed0 0x7f92480ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92480b2ed0 0x7f92480ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60282/0 (socket says 172.21.15.1:60282) 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 -- 172.21.15.1:0/4099449545 learned_addr learned my addr 172.21.15.1:0/4099449545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924e43a640 1 --1- 172.21.15.1:0/4099449545 >> v1:172.21.15.1:6789/0 conn(0x7f92480d2630 0x7f924815ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55940/0 (socket says 172.21.15.1:55940) 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924ec3b640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92480cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 -- 172.21.15.1:0/4099449545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 msgr2=0x7f92480cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92480cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 -- 172.21.15.1:0/4099449545 >> v1:172.21.15.1:6789/0 conn(0x7f92480d2630 legacy=0x7f924815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924dc39640 1 -- 172.21.15.1:0/4099449545 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480b0c00 con 0x7f92480b2ed0 2024-05-09T06:37:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.178+0000 7f924d438640 1 -- 172.21.15.1:0/4099449545 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539038572 0 0) 0x7f92480d1cc0 con 0x7f92480d2630 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f924dc39640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92480ceba0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9238009870 tx=0x7f923802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1a2a108d179de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f924d438640 1 -- 172.21.15.1:0/4099449545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238037070 con 0x7f92480b2ed0 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f924d438640 1 -- 172.21.15.1:0/4099449545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238032440 con 0x7f92480b2ed0 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f924d438640 1 -- 172.21.15.1:0/4099449545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238032810 con 0x7f92480b2ed0 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f925069e640 1 -- 172.21.15.1:0/4099449545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 msgr2=0x7f92480ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.179+0000 7f925069e640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92480ceba0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9238009870 tx=0x7f923802a270 comp rx=0 tx=0).stop 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 -- 172.21.15.1:0/4099449545 shutdown_connections 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92480cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 --2- 172.21.15.1:0/4099449545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92480ceba0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 -- 172.21.15.1:0/4099449545 >> 172.21.15.1:0/4099449545 conn(0x7f92480d3ee0 msgr2=0x7f92480d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 -- 172.21.15.1:0/4099449545 shutdown_connections 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.180+0000 7f925069e640 1 -- 172.21.15.1:0/4099449545 wait complete. 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.181+0000 7f925069e640 1 Processor -- start 2024-05-09T06:37:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.181+0000 7f925069e640 1 -- start start 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92481f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92480d2630 0x7f924815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f92480d2b90 con 0x7f92480b2ed0 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f92480d2a10 con 0x7f92480d2630 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f925069e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f92480d2d10 con 0x7f92480b2110 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60296/0 (socket says 172.21.15.1:60296) 2024-05-09T06:37:36.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 -- 172.21.15.1:0/2539017621 learned_addr learned my addr 172.21.15.1:0/2539017621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924e43a640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92481f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924ec3b640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92480d2630 0x7f924815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 -- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 msgr2=0x7f92481f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92481f10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 -- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92480d2630 msgr2=0x7f924815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92480d2630 0x7f924815bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 -- 172.21.15.1:0/2539017621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9248160c20 con 0x7f92480b2ed0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.182+0000 7f924dc39640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f92380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.183+0000 7f923effd640 1 -- 172.21.15.1:0/2539017621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238037040 con 0x7f92480b2ed0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.183+0000 7f923effd640 1 -- 172.21.15.1:0/2539017621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238042070 con 0x7f92480b2ed0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.183+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92481f4a30 con 0x7f92480b2ed0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.183+0000 7f923effd640 1 -- 172.21.15.1:0/2539017621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238030880 con 0x7f92480b2ed0 2024-05-09T06:37:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.183+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481f43e0 con 0x7f92480b2ed0 2024-05-09T06:37:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.184+0000 7f923effd640 1 -- 172.21.15.1:0/2539017621 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9238030a20 con 0x7f92480b2ed0 2024-05-09T06:37:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.185+0000 7f923effd640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 0x7f922c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.185+0000 7f923effd640 1 -- 172.21.15.1:0/2539017621 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f92380ab820 con 0x7f92480b2ed0 2024-05-09T06:37:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.185+0000 7f924e43a640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 0x7f922c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.186+0000 7f924e43a640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 0x7f922c06e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f92440012e0 tx=0x7f9244001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.186+0000 7f92533ad740 1 Processor -- start 2024-05-09T06:37:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.186+0000 7f92533ad740 1 -- start start 2024-05-09T06:37:36.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e1ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e2148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d50e14d990 con 0x55d50e226080 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d50e0f55b0 con 0x55d50e214470 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d50e149950 con 0x55d50e1e99b0 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60304/0 (socket says 172.21.15.1:60304) 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 -- 172.21.15.1:0/1167608950 learned_addr learned my addr 172.21.15.1:0/1167608950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924ec3b640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e2148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924e43a640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e1ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 -- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 msgr2=0x55d50e1ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e1ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 -- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 msgr2=0x55d50e2148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e2148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 -- 172.21.15.1:0/1167608950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50df297e0 con 0x55d50e226080 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924e43a640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e1ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924ec3b640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e2148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.187+0000 7f924dc39640 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f9238075d50 tx=0x7f9238079310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b029fb3067af2f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f922bfff640 1 -- 172.21.15.1:0/1167608950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238082420 con 0x55d50e226080 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f922bfff640 1 -- 172.21.15.1:0/1167608950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238077990 con 0x55d50e226080 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f922bfff640 1 -- 172.21.15.1:0/1167608950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238030d10 con 0x55d50e226080 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 -- 172.21.15.1:0/1167608950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 msgr2=0x55d50e1e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f9238075d50 tx=0x7f9238079310 comp rx=0 tx=0).stop 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 -- 172.21.15.1:0/1167608950 shutdown_connections 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e2148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e226080 0x55d50e1e9d90 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 --2- 172.21.15.1:0/1167608950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e1ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 -- 172.21.15.1:0/1167608950 >> 172.21.15.1:0/1167608950 conn(0x55d50e1b10a0 msgr2=0x55d50e22f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 -- 172.21.15.1:0/1167608950 shutdown_connections 2024-05-09T06:37:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.188+0000 7f92533ad740 1 -- 172.21.15.1:0/1167608950 wait complete. 2024-05-09T06:37:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.189+0000 7f92533ad740 1 -- start start 2024-05-09T06:37:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.189+0000 7f92533ad740 1 Processor -- start 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e22bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e226080 0x55d50e22c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d50e14d990 con 0x55d50e1e99b0 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d50e108b40 con 0x55d50e214470 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d50e22a590 con 0x55d50e226080 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f92533ad740 10 client.0 _ll_register_callbacks cb 0x55d50e0fd0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60320/0 (socket says 172.21.15.1:60320) 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 -- 172.21.15.1:0/706355714 learned_addr learned my addr 172.21.15.1:0/706355714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924dc39640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e22bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924ec3b640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e226080 0x55d50e22c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 -- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e226080 msgr2=0x55d50e22c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e226080 0x55d50e22c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 -- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 msgr2=0x55d50e22bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e22bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.190+0000 7f924e43a640 1 -- 172.21.15.1:0/706355714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50df89f70 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f924e43a640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9244013980 tx=0x7f9244001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c642250f283ca60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244026060 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f9244021ca0 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9238077990 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9244019070 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 10 client.9378 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55d50de21170 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238077990 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f924401f070 con 0x55d50e1e99b0 2024-05-09T06:37:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 10 client.9378 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 10 client.9378 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 20 client.9378 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e213930 0x55d50e2428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 4 client.9378 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50e15b0c0 con 0x55d50e213930 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.191+0000 7f924dc39640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e213930 0x55d50e2428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.192+0000 7f924dc39640 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e213930 0x55d50e2428b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=540a04386b0f7ea2 server_cookie=ebcfe06589ee91dc in_seq=0 out_seq=0 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.192+0000 7f92297fa640 10 client.9378 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.192+0000 7f92297fa640 1 -- 172.21.15.1:0/706355714 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d50e15b0c0 con 0x55d50e213930 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.192+0000 7f92297fa640 10 client.9378 handle_command_reply: tid=0 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.192+0000 7f92533ad740 1 client.9378 shutdown 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 msgr2=0x55d50e2292c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9244013980 tx=0x7f9244001250 comp rx=0 tx=0).stop 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 shutdown_connections 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e213930 0x55d50e2428b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e226080 0x55d50e22c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e214470 0x55d50e22bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92297fa640 0 client.9378 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 --2- 172.21.15.1:0/706355714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e2292c0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 >> 172.21.15.1:0/706355714 conn(0x55d50e1b10a0 msgr2=0x55d50e2305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 shutdown_connections 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 1 -- 172.21.15.1:0/706355714 wait complete. 2024-05-09T06:37:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.193+0000 7f92533ad740 20 client.9378 trim_cache size 0 max 16384 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 Processor -- start 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 -- start start 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 0x7f2c040cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c0415db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c040d2110 con 0x7f2c04161cb0 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2c040d2d20 con 0x7f2c040d1a80 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.197+0000 7f2c0a342640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c040d1ee0 con 0x7f2c040cfb80 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c03fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2c04161cb0 0x7f2c04162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55944/0 (socket says 172.21.15.1:55944) 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c03fff640 1 -- 172.21.15.1:0/2576867711 learned_addr learned my addr 172.21.15.1:0/2576867711 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:37:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c088df640 1 --2- 172.21.15.1:0/2576867711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c0415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c037fe640 1 --2- 172.21.15.1:0/2576867711 >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 0x7f2c040cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c02ffd640 1 -- 172.21.15.1:0/2576867711 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498371929 0 0) 0x7f2c040d2110 con 0x7f2c04161cb0 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c02ffd640 1 -- 172.21.15.1:0/2576867711 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bec003520 con 0x7f2c04161cb0 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c088df640 1 -- 172.21.15.1:0/2576867711 >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 msgr2=0x7f2c040cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c088df640 1 --2- 172.21.15.1:0/2576867711 >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 0x7f2c040cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c088df640 1 -- 172.21.15.1:0/2576867711 >> v1:172.21.15.1:6789/0 conn(0x7f2c04161cb0 legacy=0x7f2c04162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c088df640 1 -- 172.21.15.1:0/2576867711 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c040d23f0 con 0x7f2c040cfb80 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.198+0000 7f2c037fe640 1 --2- 172.21.15.1:0/2576867711 >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 0x7f2c040cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.199+0000 7f2c088df640 1 --2- 172.21.15.1:0/2576867711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c0415db00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6800ff8c940e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.199+0000 7f2c02ffd640 1 -- 172.21.15.1:0/2576867711 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc007740 con 0x7f2c040cfb80 2024-05-09T06:37:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.199+0000 7f2c02ffd640 1 -- 172.21.15.1:0/2576867711 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0078e0 con 0x7f2c040cfb80 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.199+0000 7f2c02ffd640 1 -- 172.21.15.1:0/2576867711 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc007cb0 con 0x7f2c040cfb80 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 -- 172.21.15.1:0/2576867711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 msgr2=0x7f2c0415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 --2- 172.21.15.1:0/2576867711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c0415db00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc02a660 comp rx=0 tx=0).stop 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 -- 172.21.15.1:0/2576867711 shutdown_connections 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 --2- 172.21.15.1:0/2576867711 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c0415db00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 --2- 172.21.15.1:0/2576867711 >> v2:172.21.15.6:3300/0 conn(0x7f2c040d1a80 0x7f2c040cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 -- 172.21.15.1:0/2576867711 >> 172.21.15.1:0/2576867711 conn(0x7f2c040d3eb0 msgr2=0x7f2c040d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 -- 172.21.15.1:0/2576867711 shutdown_connections 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.200+0000 7f2c0a342640 1 -- 172.21.15.1:0/2576867711 wait complete. 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.201+0000 7f2c0a342640 1 Processor -- start 2024-05-09T06:37:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.201+0000 7f2c0a342640 1 -- start start 2024-05-09T06:37:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c040c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c040d1a80 0x7f2c041f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c04161cb0 0x7f2c04153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040d29b0 con 0x7f2c04161cb0 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040d26d0 con 0x7f2c040d1a80 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c0a342640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0415e040 con 0x7f2c040cfb80 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c04161cb0 0x7f2c04153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c04161cb0 0x7f2c04153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60326/0 (socket says 172.21.15.1:60326) 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 -- 172.21.15.1:0/1578176104 learned_addr learned my addr 172.21.15.1:0/1578176104 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c037fe640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c040d1a80 0x7f2c041f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c03fff640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c040c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 -- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 msgr2=0x7f2c040c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c040cfb80 0x7f2c040c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 -- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c040d1a80 msgr2=0x7f2c041f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c040d1a80 0x7f2c041f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c088df640 1 -- 172.21.15.1:0/1578176104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c04155e80 con 0x7f2c04161cb0 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.202+0000 7f2c037fe640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c040d1a80 0x7f2c041f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c088df640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c04161cb0 0x7f2c04153af0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc02ac50 tx=0x7f2bfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc040070 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc031bc0 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c0a342640 1 -- 172.21.15.1:0/1578176104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c04156160 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc009450 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c0a342640 1 -- 172.21.15.1:0/1578176104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c04156720 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.203+0000 7f2c0a342640 1 -- 172.21.15.1:0/1578176104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2c040752b0 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.204+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bfc039040 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c00ff9640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2bf006be30 0x7f2bf006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f2bfc0ad040 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c03fff640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2bf006be30 0x7f2bf006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7f2bfc0ad4f0 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c0a342640 1 -- 172.21.15.1:0/1578176104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f2c040b7190 con 0x7f2c04161cb0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c03fff640 1 --2- 172.21.15.1:0/1578176104 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2bf006be30 0x7f2bf006e2f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4000f60 tx=0x7f2bf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.205+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 7 ==== log(2 entries from seq 1262 at 2024-05-09T06:37:34.590383+0000) ==== 389+0+0 (secure 0 0 0) 0x7f2bfc0ab480 con 0x7f2c04161cb0 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: cluster: 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: id: 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: health: HEALTH_OK 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: services: 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: mgr: y(active, since 10m), standbys: x 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: mds: 1/1 daemons up, 3 standby 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: data: 2024-05-09T06:37:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: volumes: 1/1 healthy 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: pools: 3 pools, 97 pgs 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: objects: 30 objects, 19 MiB 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: pgs: 97 active+clean 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: io: 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:34.590383+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:37:36.193 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:34.590405+0000 mon.a [INF] Cluster is now healthy 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.212+0000 7f92533ad740 1 Processor -- start 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.212+0000 7f92533ad740 1 -- start start 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e214470 0x55d50e22c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e251870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9248200be0 con 0x55d50e1e99b0 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f92481ffbf0 con 0x55d50e226080 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f92481f6060 con 0x55d50e214470 2024-05-09T06:37:36.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60334/0 (socket says 172.21.15.1:60334) 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 -- 172.21.15.1:0/2851086930 learned_addr learned my addr 172.21.15.1:0/2851086930 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924ec3b640 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e251870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924dc39640 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e214470 0x55d50e22c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 -- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e214470 msgr2=0x55d50e22c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e214470 0x55d50e22c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 -- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 msgr2=0x55d50e251870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e251870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 -- 172.21.15.1:0/2851086930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50e2303a0 con 0x55d50e1e99b0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.213+0000 7f924e43a640 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f9244001a80 tx=0x7f924401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f350a6bc1aa6320f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f922b7fe640 1 -- 172.21.15.1:0/2851086930 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244026020 con 0x55d50e1e99b0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f922b7fe640 1 -- 172.21.15.1:0/2851086930 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244012860 con 0x55d50e1e99b0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f922b7fe640 1 -- 172.21.15.1:0/2851086930 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244012a10 con 0x55d50e1e99b0 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 -- 172.21.15.1:0/2851086930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 msgr2=0x55d50e22bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f9244001a80 tx=0x7f924401f000 comp rx=0 tx=0).stop 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 -- 172.21.15.1:0/2851086930 shutdown_connections 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e251870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e214470 0x55d50e22c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 --2- 172.21.15.1:0/2851086930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e1e99b0 0x55d50e22bc80 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 -- 172.21.15.1:0/2851086930 >> 172.21.15.1:0/2851086930 conn(0x55d50e1b10a0 msgr2=0x55d50e212370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 -- 172.21.15.1:0/2851086930 shutdown_connections 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.214+0000 7f92533ad740 1 -- 172.21.15.1:0/2851086930 wait complete. 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.215+0000 7f92533ad740 1 -- start start 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.215+0000 7f92533ad740 1 Processor -- start 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e2305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e224e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d50e22b2c0 con 0x55d50e214470 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f92481ff5a0 con 0x55d50e226080 2024-05-09T06:37:36.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d50e2aa450 con 0x55d50e1e99b0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f92533ad740 10 client.0 _ll_register_callbacks cb 0x7f9244007db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60340/0 (socket says 172.21.15.1:60340) 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 -- 172.21.15.1:0/2323324296 learned_addr learned my addr 172.21.15.1:0/2323324296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924ec3b640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e224e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924e43a640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e2305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 -- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 msgr2=0x55d50e2305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e2305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 -- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 msgr2=0x55d50e224e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e224e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.216+0000 7f924dc39640 1 -- 172.21.15.1:0/2323324296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50e249050 con 0x55d50e214470 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f924dc39640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f923802a2b0 tx=0x7f92380ab070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8132cfb958fe068b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238075b90 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7f9238077bf0 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9244012860 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f923809d050 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 10 client.9390 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55d50e2aa190 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244012860 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f923803a070 con 0x55d50e214470 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 10 client.9390 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 10 client.9390 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 20 client.9390 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e22b770 0x55d50e225340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 4 client.9390 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55d50e155410 con 0x55d50e22b770 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.217+0000 7f924e43a640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e22b770 0x55d50e225340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f924e43a640 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e22b770 0x55d50e225340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd2ba4f8a40d849 server_cookie=ca868a5d63b809f in_seq=0 out_seq=0 2024-05-09T06:37:36.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92297fa640 10 client.9390 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92297fa640 1 -- 172.21.15.1:0/2323324296 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d50e155410 con 0x55d50e22b770 2024-05-09T06:37:36.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92297fa640 10 client.9390 handle_command_reply: tid=0 2024-05-09T06:37:36.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92533ad740 1 client.9390 shutdown 2024-05-09T06:37:36.204 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-09T06:37:36.204 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 msgr2=0x55d50e230ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f923802a2b0 tx=0x7f92380ab070 comp rx=0 tx=0).stop 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 shutdown_connections 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.218+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55d50e22b770 0x55d50e225340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92297fa640 0 client.9390 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d50e226080 0x55d50e224e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e214470 0x55d50e230ae0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 --2- 172.21.15.1:0/2323324296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d50e1e99b0 0x55d50e2305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 >> 172.21.15.1:0/2323324296 conn(0x55d50e1b10a0 msgr2=0x55d50e217900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.206 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-09T06:37:36.206 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-09T06:37:36.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 shutdown_connections 2024-05-09T06:37:36.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 1 -- 172.21.15.1:0/2323324296 wait complete. 2024-05-09T06:37:36.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.219+0000 7f92533ad740 20 client.9390 trim_cache size 0 max 16384 2024-05-09T06:37:36.207 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-05-09T06:37:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "4490adae-da7a-4eca-8aba-5d80b41e5acc", 2024-05-09T06:37:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-05-09T06:37:36.207 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 msgr2=0x7f922c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 0x7f922c06e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f92440012e0 tx=0x7f9244001700 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 msgr2=0x7f92481606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f92380046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 shutdown_connections 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f922c06be30 0x7f922c06e2f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92480d2630 0x7f924815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92480b2ed0 0x7f92481606e0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 --2- 172.21.15.1:0/2539017621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f92480b2110 0x7f92481f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 >> 172.21.15.1:0/2539017621 conn(0x7f92480d3ee0 msgr2=0x7f92480ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 shutdown_connections 2024-05-09T06:37:36.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.220+0000 7f925069e640 1 -- 172.21.15.1:0/2539017621 wait complete. 2024-05-09T06:37:36.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:36.212 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-09T06:37:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.338+0000 7fbc62415640 1 Processor -- start 2024-05-09T06:37:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.338+0000 7fbc62415640 1 -- start start 2024-05-09T06:37:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc62415640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 0x7fbc5c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc62415640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbc5c06b1e0 0x7fbc5c0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc62415640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5c072050 con 0x7fbc5c0d06f0 2024-05-09T06:37:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc62415640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5c072330 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc62415640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c072610 con 0x7fbc5c0c41e0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc609b2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbc5c06b1e0 0x7fbc5c0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc5bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbc5c0d06f0 0x7fbc5c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55950/0 (socket says 172.21.15.1:55950) 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc5bfff640 1 -- 172.21.15.1:0/4003610072 learned_addr learned my addr 172.21.15.1:0/4003610072 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc537fe640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 0x7fbc5c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc609b2640 1 -- 172.21.15.1:0/4003610072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 msgr2=0x7fbc5c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc609b2640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 0x7fbc5c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc609b2640 1 -- 172.21.15.1:0/4003610072 >> v1:172.21.15.1:6789/0 conn(0x7fbc5c0d06f0 legacy=0x7fbc5c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc609b2640 1 -- 172.21.15.1:0/4003610072 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0728f0 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.339+0000 7fbc537fe640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 0x7fbc5c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc609b2640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c0cee10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009d90 tx=0x7fbc5402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c9b8e5fbefeb1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc5b7fe640 1 -- 172.21.15.1:0/4003610072 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007740 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc5b7fe640 1 -- 172.21.15.1:0/4003610072 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc540078e0 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc5b7fe640 1 -- 172.21.15.1:0/4003610072 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007cb0 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 -- 172.21.15.1:0/4003610072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 msgr2=0x7fbc5c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c0cee10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009d90 tx=0x7fbc5402a660 comp rx=0 tx=0).stop 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 -- 172.21.15.1:0/4003610072 shutdown_connections 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0c41e0 0x7fbc5c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 --2- 172.21.15.1:0/4003610072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c0cee10 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.340+0000 7fbc62415640 1 -- 172.21.15.1:0/4003610072 >> 172.21.15.1:0/4003610072 conn(0x7fbc5c0d0b50 msgr2=0x7fbc5c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.341+0000 7fbc62415640 1 -- 172.21.15.1:0/4003610072 shutdown_connections 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.341+0000 7fbc62415640 1 -- 172.21.15.1:0/4003610072 wait complete. 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.342+0000 7fbc62415640 1 Processor -- start 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.342+0000 7fbc62415640 1 -- start start 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc5c0c41e0 0x7fbc5c0c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0d06f0 0x7fbc5c0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0ccf70 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0ccdf0 con 0x7fbc5c0c41e0 2024-05-09T06:37:36.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0cd0f0 con 0x7fbc5c0d06f0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60358/0 (socket says 172.21.15.1:60358) 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 -- 172.21.15.1:0/2330409821 learned_addr learned my addr 172.21.15.1:0/2330409821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc537fe640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc5c0c41e0 0x7fbc5c0c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc609b2640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0d06f0 0x7fbc5c0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 -- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0d06f0 msgr2=0x7fbc5c0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0d06f0 0x7fbc5c0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 -- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc5c0c41e0 msgr2=0x7fbc5c0c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc5c0c41e0 0x7fbc5c0c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 -- 172.21.15.1:0/2330409821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0c5d30 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c3f0 tx=0x7fbc4c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc597fa640 1 -- 172.21.15.1:0/2330409821 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c017070 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc597fa640 1 -- 172.21.15.1:0/2330409821 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c005ce0 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc597fa640 1 -- 172.21.15.1:0/2330409821 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0072f0 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.343+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c0c6010 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.344+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c0c5a20 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.345+0000 7fbc597fa640 1 -- 172.21.15.1:0/2330409821 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc4c01a070 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.345+0000 7fbc597fa640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 0x7fbc3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.346+0000 7fbc597fa640 1 -- 172.21.15.1:0/2330409821 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fbc4c08a440 con 0x7fbc5c06b1e0 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.346+0000 7fbc537fe640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 0x7fbc3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.346+0000 7fbc537fe640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 0x7fbc3c06e320 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0d1310 tx=0x7fbc44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.346+0000 7fbc65200740 1 Processor -- start 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.346+0000 7fbc65200740 1 -- start start 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e47675fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 0x562e476a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e475dfd00 con 0x562e476a6030 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562e475874c0 con 0x562e476a6770 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562e475dbcc0 con 0x562e476c3650 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60364/0 (socket says 172.21.15.1:60364) 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc609b2640 1 -- 172.21.15.1:0/1535546285 learned_addr learned my addr 172.21.15.1:0/1535546285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.347+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 0x562e476a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 -- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 msgr2=0x562e47675fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e47675fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 -- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 msgr2=0x562e476a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 0x562e476a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 -- 172.21.15.1:0/1535546285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e473bd7d0 con 0x562e476a6030 2024-05-09T06:37:36.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 0x562e476a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e80 tx=0x7fbc5402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2189b3372c18a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc51ffb640 1 -- 172.21.15.1:0/1535546285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007a30 con 0x562e476a6030 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc51ffb640 1 -- 172.21.15.1:0/1535546285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54041070 con 0x562e476a6030 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc51ffb640 1 -- 172.21.15.1:0/1535546285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007dd0 con 0x562e476a6030 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc65200740 1 -- 172.21.15.1:0/1535546285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 msgr2=0x562e4767ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.348+0000 7fbc65200740 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e80 tx=0x7fbc5402e000 comp rx=0 tx=0).stop 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 -- 172.21.15.1:0/1535546285 shutdown_connections 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6030 0x562e4767ae40 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e476a6770 0x562e476a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 --2- 172.21.15.1:0/1535546285 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e47675fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 -- 172.21.15.1:0/1535546285 >> 172.21.15.1:0/1535546285 conn(0x562e47678250 msgr2=0x562e476c0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 -- 172.21.15.1:0/1535546285 shutdown_connections 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 -- 172.21.15.1:0/1535546285 wait complete. 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 -- start start 2024-05-09T06:37:36.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.349+0000 7fbc65200740 1 Processor -- start 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e476c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e4767a400 con 0x562e476a6770 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562e4759ab90 con 0x562e47690560 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562e476797f0 con 0x562e476c3650 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc65200740 10 client.0 _ll_register_callbacks cb 0x562e475d6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60378/0 (socket says 172.21.15.1:60378) 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 -- 172.21.15.1:0/1813199492 learned_addr learned my addr 172.21.15.1:0/1813199492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e476c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 -- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 msgr2=0x562e476c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e476c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 -- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 msgr2=0x562e476c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.350+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc5bfff640 1 -- 172.21.15.1:0/1813199492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e472f33f0 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0548c0 tx=0x7fbc4c058460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821b0ebdf45f8c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c061040 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fbc4c007490 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc54041070 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c0076b0 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 10 client.9405 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x562e472b30f0 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54041070 con 0x562e476a6770 2024-05-09T06:37:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fbc4c0578b0 con 0x562e476a6770 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 10 client.9405 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 10 client.9405 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 20 client.9405 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476bf7a0 0x562e476bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 4 client.9405 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e475ed430 con 0x562e476bf7a0 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.351+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476bf7a0 0x562e476bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476bf7a0 0x562e476bfbc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94724a2d65011d77 server_cookie=48eea7c434d71c10 in_seq=0 out_seq=0 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc3affd640 10 client.9405 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc3affd640 1 -- 172.21.15.1:0/1813199492 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e475ed430 con 0x562e476bf7a0 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc3affd640 10 client.9405 handle_command_reply: tid=0 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 client.9405 shutdown 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 msgr2=0x562e476be800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0548c0 tx=0x7fbc4c058460 comp rx=0 tx=0).stop 2024-05-09T06:37:36.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 shutdown_connections 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476bf7a0 0x562e476bfbc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476c3650 0x562e476c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc3affd640 0 client.9405 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 --2- 172.21.15.1:0/1813199492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476a6770 0x562e476be800 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.352+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 >> 172.21.15.1:0/1813199492 conn(0x562e476a6030 msgr2=0x562e47676d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.353+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 shutdown_connections 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.353+0000 7fbc65200740 1 -- 172.21.15.1:0/1813199492 wait complete. 2024-05-09T06:37:36.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.353+0000 7fbc65200740 20 client.9405 trim_cache size 0 max 16384 2024-05-09T06:37:36.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.370+0000 7fbc65200740 1 Processor -- start 2024-05-09T06:37:36.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.370+0000 7fbc65200740 1 -- start start 2024-05-09T06:37:36.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e47676d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c200cb0 con 0x562e476c3650 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1ffcd0 con 0x562e47690560 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0c8fe0 con 0x562e476a6770 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60394/0 (socket says 172.21.15.1:60394) 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 -- 172.21.15.1:0/2291994956 learned_addr learned my addr 172.21.15.1:0/2291994956 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc537fe640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e47676d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 -- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 msgr2=0x562e47676d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e47676d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 -- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 msgr2=0x562e476c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 -- 172.21.15.1:0/2291994956 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e476a7500 con 0x562e476c3650 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.371+0000 7fbc609b2640 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fbc5402ac20 tx=0x7fbc54030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f2becea5d43af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc517fa640 1 -- 172.21.15.1:0/2291994956 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007830 con 0x562e476c3650 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc517fa640 1 -- 172.21.15.1:0/2291994956 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5404c040 con 0x562e476c3650 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc517fa640 1 -- 172.21.15.1:0/2291994956 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007c10 con 0x562e476c3650 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 -- 172.21.15.1:0/2291994956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 msgr2=0x562e47677260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fbc5402ac20 tx=0x7fbc54030000 comp rx=0 tx=0).stop 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 -- 172.21.15.1:0/2291994956 shutdown_connections 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e47677260 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e47676d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 --2- 172.21.15.1:0/2291994956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e476c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 -- 172.21.15.1:0/2291994956 >> 172.21.15.1:0/2291994956 conn(0x562e476a6030 msgr2=0x562e4767b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 -- 172.21.15.1:0/2291994956 shutdown_connections 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.372+0000 7fbc65200740 1 -- 172.21.15.1:0/2291994956 wait complete. 2024-05-09T06:37:36.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.373+0000 7fbc65200740 1 -- start start 2024-05-09T06:37:36.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.373+0000 7fbc65200740 1 Processor -- start 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e47739ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e4767b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562e47679380 con 0x562e476c3650 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c1ff7d0 con 0x562e47690560 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562e476c1520 con 0x562e476a6770 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc65200740 10 client.0 _ll_register_callbacks cb 0x7fbc4c005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60406/0 (socket says 172.21.15.1:60406) 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 -- 172.21.15.1:0/1902912030 learned_addr learned my addr 172.21.15.1:0/1902912030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e47739ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc537fe640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e4767b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 -- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 msgr2=0x562e4767b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e4767b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 -- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 msgr2=0x562e47739ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e47739ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 -- 172.21.15.1:0/1902912030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e476d1a70 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc609b2640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009d90 tx=0x7fbc54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca5540e81cf6f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.374+0000 7fbc3affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54046880 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fbc54046a20 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5404c040 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbc54066050 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 10 client.9411 fetch_fsmap learned FSMap version 87 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x562e476bfff0 con 0x562e476c3650 2024-05-09T06:37:36.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5404c040 con 0x562e476c3650 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fbc54069020 con 0x562e476c3650 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 10 client.9411 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 10 client.9411 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 20 client.9411 populate_metadata read hostname 'smithi001' 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476a4f50 0x562e47676d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 4 client.9411 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e476d1260 con 0x562e476a4f50 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476a4f50 0x562e47676d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc5bfff640 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476a4f50 0x562e47676d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c3a83af3c8f2 server_cookie=90aae4564440a471 in_seq=0 out_seq=0 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.375+0000 7fbc3affd640 10 client.9411 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc3affd640 1 -- 172.21.15.1:0/1902912030 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x562e476d1260 con 0x562e476a4f50 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc3affd640 10 client.9411 handle_command_reply: tid=0 2024-05-09T06:37:36.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 client.9411 shutdown 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 msgr2=0x562e476c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009d90 tx=0x7fbc54004ab0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 shutdown_connections 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x562e476a4f50 0x562e47676d20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562e476c3650 0x562e476c20c0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562e476a6770 0x562e4767b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 --2- 172.21.15.1:0/1902912030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e47690560 0x562e47739ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc3affd640 0 client.9411 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 >> 172.21.15.1:0/1902912030 conn(0x562e476a6030 msgr2=0x562e47671750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 shutdown_connections 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 1 -- 172.21.15.1:0/1902912030 wait complete. 2024-05-09T06:37:36.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc65200740 20 client.9411 trim_cache size 0 max 16384 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "4490adae-da7a-4eca-8aba-5d80b41e5acc": { 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4490adae-da7a-4eca-8aba-5d80b41e5acc", 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,repair" 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:37:36.364 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:37:36.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 msgr2=0x7fbc3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 0x7fbc3c06e320 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0d1310 tx=0x7fbc44008040 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 msgr2=0x7fbc5c14f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.376+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c3f0 tx=0x7fbc4c00c8b0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 shutdown_connections 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbc3c06be60 0x7fbc3c06e320 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc5c0d06f0 0x7fbc5c0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc5c0c41e0 0x7fbc5c0c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 --2- 172.21.15.1:0/2330409821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5c06b1e0 0x7fbc5c14f840 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 >> 172.21.15.1:0/2330409821 conn(0x7fbc5c0d0b50 msgr2=0x7fbc5c15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 shutdown_connections 2024-05-09T06:37:36.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:36.377+0000 7fbc62415640 1 -- 172.21.15.1:0/2330409821 wait complete. 2024-05-09T06:37:36.368 INFO:tasks.cephfs.filesystem:scrub status for tag:4490adae-da7a-4eca-8aba-5d80b41e5acc - {'path': '/', 'tag': '4490adae-da7a-4eca-8aba-5d80b41e5acc', 'options': 'recursive,repair'} 2024-05-09T06:37:36.573 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:37:36.592+0000 7f4bbd591640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:37.601+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-09T06:37:36.219412+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f2bfc043e10 con 0x7f2c04161cb0 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.219412+0000 mds.a [INF] scrub queued for path: / 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.219415+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.219419+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.219917+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.219923+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-05-09T06:37:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.220210+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-09T06:37:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.220217+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-05-09T06:37:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:36.220556+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-09T06:37:42.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:37:42.608+0000 7f2c00ff9640 1 -- 172.21.15.1:0/1578176104 <== mon.0 v2:172.21.15.1:3300/0 9 ==== log(1 entries from seq 9 at 2024-05-09T06:37:40.880767+0000) ==== 189+0+0 (secure 0 0 0) 0x7f2bfc079340 con 0x7f2c04161cb0 2024-05-09T06:37:42.589 INFO:teuthology.orchestra.run.smithi001.stdout:2024-05-09T06:37:40.880767+0000 mds.a [INF] scrub summary: idle 2024-05-09T06:37:48.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:48.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:38:06.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:06.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-09T06:38:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.508+0000 7fc782210640 1 Processor -- start 2024-05-09T06:38:06.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.508+0000 7fc782210640 1 -- start start 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.509+0000 7fc782210640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc77c15e390 0x7fc77c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.509+0000 7fc782210640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.509+0000 7fc782210640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c15e900 con 0x7fc77c0b1730 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.509+0000 7fc782210640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc77c0b1ba0 con 0x7fc77c15e390 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.509+0000 7fc782210640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c15ebe0 con 0x7fc77c0cf990 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc77c0b1730 0x7fc77c072800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48476/0 (socket says 172.21.15.1:48476) 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77bfff640 1 -- 172.21.15.1:0/4157009000 learned_addr learned my addr 172.21.15.1:0/4157009000 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/4157009000 >> v2:172.21.15.6:3300/0 conn(0x7fc77c15e390 0x7fc77c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77affd640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671753405 0 0) 0x7fc77c15e900 con 0x7fc77c0b1730 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc764003520 con 0x7fc77c0b1730 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1210165144 0 0) 0x7fc764003520 con 0x7fc77c0b1730 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77b7fe640 1 -- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 msgr2=0x7fc77c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77b7fe640 1 -- 172.21.15.1:0/4157009000 >> v1:172.21.15.1:6789/0 conn(0x7fc77c0b1730 legacy=0x7fc77c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.510+0000 7fc77b7fe640 1 -- 172.21.15.1:0/4157009000 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c15eec0 con 0x7fc77c15e390 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.511+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c15e390 0x7fc77c0cf450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0097a0 tx=0x7fc76c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b57ac7fa4d928c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.511+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c037070 con 0x7fc77c15e390 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.511+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c02aba0 con 0x7fc77c15e390 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.511+0000 7fc77a7fc640 1 -- 172.21.15.1:0/4157009000 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740019f0 con 0x7fc77c15e390 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 -- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c15e390 msgr2=0x7fc77c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c15e390 0x7fc77c0cf450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0097a0 tx=0x7fc76c02a110 comp rx=0 tx=0).stop 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 -- 172.21.15.1:0/4157009000 shutdown_connections 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 --2- 172.21.15.1:0/4157009000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c15e390 0x7fc77c0cf450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 -- 172.21.15.1:0/4157009000 >> 172.21.15.1:0/4157009000 conn(0x7fc77c0b1da0 msgr2=0x7fc77c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 -- 172.21.15.1:0/4157009000 shutdown_connections 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.512+0000 7fc782210640 1 -- 172.21.15.1:0/4157009000 wait complete. 2024-05-09T06:38:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.513+0000 7fc782210640 1 Processor -- start 2024-05-09T06:38:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.513+0000 7fc782210640 1 -- start start 2024-05-09T06:38:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 0x7fc77c08f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c0cd8f0 con 0x7fc77c15e390 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c15f6c0 con 0x7fc77c0b1730 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc782210640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c162ba0 con 0x7fc77c0cf990 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37828/0 (socket says 172.21.15.1:37828) 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc77bfff640 1 -- 172.21.15.1:0/1600710722 learned_addr learned my addr 172.21.15.1:0/1600710722 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 0x7fc77c08f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.514+0000 7fc77affd640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 -- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 msgr2=0x7fc77c0c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 -- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 msgr2=0x7fc77c08f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 0x7fc77c08f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 -- 172.21.15.1:0/1600710722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c1604a0 con 0x7fc77c15e390 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77affd640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 0x7fc77c08f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:06.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc77bfff640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc774001b60 tx=0x7fc77400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc763fff640 1 -- 172.21.15.1:0/1600710722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774017070 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc763fff640 1 -- 172.21.15.1:0/1600710722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc774012400 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c0999c0 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc763fff640 1 -- 172.21.15.1:0/1600710722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740127b0 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.515+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c06ee90 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.517+0000 7fc763fff640 1 -- 172.21.15.1:0/1600710722 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc77401a070 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.517+0000 7fc763fff640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 0x7fc76806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.517+0000 7fc763fff640 1 -- 172.21.15.1:0/1600710722 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fc77408af10 con 0x7fc77c15e390 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.517+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 0x7fc76806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.518+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 0x7fc76806e2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc76c034000 tx=0x7fc76c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.518+0000 7fc784ff3740 1 Processor -- start 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.518+0000 7fc784ff3740 1 -- start start 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 0x55ba04467fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 0x55ba0446ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ba043d1d00 con 0x55ba04498770 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ba043794c0 con 0x55ba04498030 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ba043cdcc0 con 0x55ba044b5650 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37832/0 (socket says 172.21.15.1:37832) 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 -- 172.21.15.1:0/200434103 learned_addr learned my addr 172.21.15.1:0/200434103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 0x55ba04467fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77bfff640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 0x55ba0446ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 -- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 msgr2=0x55ba04467fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 0x55ba04467fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 -- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 msgr2=0x55ba0446ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 0x55ba0446ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 -- 172.21.15.1:0/200434103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba041af7d0 con 0x55ba04498770 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 0x55ba04467fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:06.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77bfff640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 0x55ba0446ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.519+0000 7fc77affd640 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc770009c40 tx=0x7fc770009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceab96550b8a580d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc7617fa640 1 -- 172.21.15.1:0/200434103 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770018070 con 0x55ba04498770 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc7617fa640 1 -- 172.21.15.1:0/200434103 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770009e50 con 0x55ba04498770 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc7617fa640 1 -- 172.21.15.1:0/200434103 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f040 con 0x55ba04498770 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 -- 172.21.15.1:0/200434103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 msgr2=0x55ba04497af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc770009c40 tx=0x7fc770009190 comp rx=0 tx=0).stop 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 -- 172.21.15.1:0/200434103 shutdown_connections 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498030 0x55ba0446ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba04498770 0x55ba04497af0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 --2- 172.21.15.1:0/200434103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba044b5650 0x55ba04467fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 -- 172.21.15.1:0/200434103 >> 172.21.15.1:0/200434103 conn(0x55ba0446a250 msgr2=0x55ba044b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 -- 172.21.15.1:0/200434103 shutdown_connections 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.520+0000 7fc784ff3740 1 -- 172.21.15.1:0/200434103 wait complete. 2024-05-09T06:38:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.521+0000 7fc784ff3740 1 -- start start 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.521+0000 7fc784ff3740 1 Processor -- start 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04488030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ba0446c400 con 0x55ba044b5650 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ba0438cb90 con 0x55ba04498770 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ba0446b7f0 con 0x55ba0449afc0 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc784ff3740 10 client.0 _ll_register_callbacks cb 0x55ba043c8da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37836/0 (socket says 172.21.15.1:37836) 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 -- 172.21.15.1:0/425547223 learned_addr learned my addr 172.21.15.1:0/425547223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77bfff640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04488030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 -- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 msgr2=0x55ba04488030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04488030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 -- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 msgr2=0x55ba044b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77affd640 1 -- 172.21.15.1:0/425547223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba040e53f0 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.522+0000 7fc77bfff640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04488030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc77affd640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fc770017000 tx=0x7fc770004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d576f69b94bdf94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770018050 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fc770028cd0 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc770009e50 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc77002f050 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 10 client.9423 fetch_fsmap learned FSMap version 87 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55ba040a50f0 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770009e50 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fc770032020 con 0x55ba044b5650 2024-05-09T06:38:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 10 client.9423 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 10 client.9423 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 20 client.9423 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba044c6500 0x55ba044c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 4 client.9423 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba043df430 con 0x55ba044c6500 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.523+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba044c6500 0x55ba044c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba044c6500 0x55ba044c6910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53fe1cf35078da7 server_cookie=4a5145d4923b3fc1 in_seq=0 out_seq=0 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc75a7fc640 10 client.9423 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc75a7fc640 1 -- 172.21.15.1:0/425547223 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba043df430 con 0x55ba044c6500 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc75a7fc640 10 client.9423 handle_command_reply: tid=0 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 client.9423 shutdown 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 msgr2=0x55ba0449aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fc770017000 tx=0x7fc770004730 comp rx=0 tx=0).stop 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 shutdown_connections 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba044c6500 0x55ba044c6910 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04488030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba0449aa80 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 --2- 172.21.15.1:0/425547223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.524+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 >> 172.21.15.1:0/425547223 conn(0x55ba04498030 msgr2=0x55ba044b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.525+0000 7fc75a7fc640 0 client.9423 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.525+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 shutdown_connections 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.525+0000 7fc784ff3740 1 -- 172.21.15.1:0/425547223 wait complete. 2024-05-09T06:38:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.525+0000 7fc784ff3740 20 client.9423 trim_cache size 0 max 16384 2024-05-09T06:38:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.542+0000 7fc784ff3740 1 Processor -- start 2024-05-09T06:38:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.542+0000 7fc784ff3740 1 -- start start 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba04498770 0x55ba044b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba044b5650 0x55ba04487980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c200b20 con 0x55ba0449afc0 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c1ffb80 con 0x55ba044b5650 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c0b4b50 con 0x55ba04498770 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc77affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37846/0 (socket says 172.21.15.1:37846) 2024-05-09T06:38:06.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc77affd640 1 -- 172.21.15.1:0/22515769 learned_addr learned my addr 172.21.15.1:0/22515769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.543+0000 7fc77bfff640 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba044b5650 0x55ba04487980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 -- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba04498770 msgr2=0x55ba044b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba04498770 0x55ba044b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba04498770 0x55ba044b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 -- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba044b5650 msgr2=0x55ba04487980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba044b5650 0x55ba04487980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 -- 172.21.15.1:0/22515769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba0446b280 con 0x55ba0449afc0 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc77affd640 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc7700038a0 tx=0x7fc770022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a36d9ef2562f776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc760ff9640 1 -- 172.21.15.1:0/22515769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770018040 con 0x55ba0449afc0 2024-05-09T06:38:06.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc760ff9640 1 -- 172.21.15.1:0/22515769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770004350 con 0x55ba0449afc0 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.544+0000 7fc760ff9640 1 -- 172.21.15.1:0/22515769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770005bb0 con 0x55ba0449afc0 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- 172.21.15.1:0/22515769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 msgr2=0x55ba044cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc7700038a0 tx=0x7fc770022000 comp rx=0 tx=0).stop 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- 172.21.15.1:0/22515769 shutdown_connections 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba044b5650 0x55ba04487980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba0449afc0 0x55ba044cc7f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 --2- 172.21.15.1:0/22515769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba04498770 0x55ba044b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- 172.21.15.1:0/22515769 >> 172.21.15.1:0/22515769 conn(0x55ba04498030 msgr2=0x55ba044b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- 172.21.15.1:0/22515769 shutdown_connections 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- 172.21.15.1:0/22515769 wait complete. 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 -- start start 2024-05-09T06:38:06.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.545+0000 7fc784ff3740 1 Processor -- start 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04493b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ba044a9f40 con 0x55ba044b5650 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c0b51a0 con 0x55ba04498770 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ba044957b0 con 0x55ba0449afc0 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc784ff3740 10 client.0 _ll_register_callbacks cb 0x7fc770009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37858/0 (socket says 172.21.15.1:37858) 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc77bfff640 1 -- 172.21.15.1:0/812829641 learned_addr learned my addr 172.21.15.1:0/812829641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.546+0000 7fc77affd640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04493b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 -- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 msgr2=0x55ba04493b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04493b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 -- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 msgr2=0x55ba044c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 -- 172.21.15.1:0/812829641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba043574d0 con 0x55ba044b5650 2024-05-09T06:38:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77bfff640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc774055450 tx=0x7fc774059130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7583855c45b3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774059450 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7fc774061690 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc770004350 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc774061890 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 10 client.9429 fetch_fsmap learned FSMap version 87 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55ba044c1d50 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770004350 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fc774082ba0 con 0x55ba044b5650 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 10 client.9429 fetch_fsmap finished waiting for FSMap version 87 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 10 client.9429 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 20 client.9429 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba04526190 0x55ba044c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 4 client.9429 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 --> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba043d9780 con 0x55ba04526190 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.547+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba04526190 0x55ba044c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc77b7fe640 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba04526190 0x55ba044c5080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265f7f28f5ad6783 server_cookie=95d8572b052698ad in_seq=0 out_seq=0 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc75a7fc640 10 client.9429 ms_handle_connect on v2:172.21.15.1:6834/356239460 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc75a7fc640 1 -- 172.21.15.1:0/812829641 <== mds.0 v2:172.21.15.1:6834/356239460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba043d9780 con 0x55ba04526190 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc75a7fc640 10 client.9429 handle_command_reply: tid=0 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 client.9429 shutdown 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 msgr2=0x55ba044c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc774055450 tx=0x7fc774059130 comp rx=0 tx=0).stop 2024-05-09T06:38:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 shutdown_connections 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:6834/356239460,v1:172.21.15.1:6835/356239460] conn(0x55ba04526190 0x55ba044c5080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba044b5650 0x55ba044c4b40 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ba0449afc0 0x55ba04493b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc75a7fc640 0 client.9429 ms_handle_reset on v2:172.21.15.1:6834/356239460 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 --2- 172.21.15.1:0/812829641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ba04498770 0x55ba044c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 >> 172.21.15.1:0/812829641 conn(0x55ba04498030 msgr2=0x55ba044b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 shutdown_connections 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 1 -- 172.21.15.1:0/812829641 wait complete. 2024-05-09T06:38:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.548+0000 7fc784ff3740 20 client.9429 trim_cache size 0 max 16384 2024-05-09T06:38:06.536 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-05-09T06:38:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-05-09T06:38:06.536 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:06.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 msgr2=0x7fc76806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 0x7fc76806e2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc76c034000 tx=0x7fc76c02e000 comp rx=0 tx=0).stop 2024-05-09T06:38:06.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 msgr2=0x7fc77c0c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc774001b60 tx=0x7fc77400a430 comp rx=0 tx=0).stop 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 shutdown_connections 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76806be30 0x7fc76806e2f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc77c15e390 0x7fc77c0c42a0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc77c0cf990 0x7fc77c0c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 --2- 172.21.15.1:0/1600710722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc77c0b1730 0x7fc77c08f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 >> 172.21.15.1:0/1600710722 conn(0x7fc77c0b1da0 msgr2=0x7fc77c15fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 shutdown_connections 2024-05-09T06:38:06.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.549+0000 7fc782210640 1 -- 172.21.15.1:0/1600710722 wait complete. 2024-05-09T06:38:06.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T06:38:06.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:38:06.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.767+0000 7ff5271e9640 1 Processor -- start 2024-05-09T06:38:06.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.767+0000 7ff5271e9640 1 -- start start 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff5271e9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff52015e1d0 0x7ff52006f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff5271e9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52006c2f0 con 0x7ff52015e5b0 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff5271e9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52006f500 con 0x7ff5200d2890 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff5271e9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff5200cc3c0 con 0x7ff52015e1d0 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff51ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff52015e5b0 0x7ff5200cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48488/0 (socket says 172.21.15.1:48488) 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff51ffff640 1 -- 172.21.15.1:0/4059745254 learned_addr learned my addr 172.21.15.1:0/4059745254 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297380750 0 0) 0x7ff50c001110 con 0x7ff52015e5b0 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff510003520 con 0x7ff52015e5b0 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.768+0000 7ff524f85640 1 --2- 172.21.15.1:0/4059745254 >> v2:172.21.15.6:3300/0 conn(0x7ff52015e1d0 0x7ff52006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2554201317 0 0) 0x7ff510003520 con 0x7ff52015e5b0 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 >> v2:172.21.15.6:3300/0 conn(0x7ff52015e1d0 msgr2=0x7ff52006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 --2- 172.21.15.1:0/4059745254 >> v2:172.21.15.6:3300/0 conn(0x7ff52015e1d0 0x7ff52006f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 >> v1:172.21.15.6:6789/0 conn(0x7ff5200d2890 legacy=0x7ff5200cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5200cc610 con 0x7ff52015e5b0 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7ff50c001840 con 0x7ff52015e5b0 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 >> v1:172.21.15.1:6789/0 conn(0x7ff52015e5b0 legacy=0x7ff5200cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5100039b0 0x7ff510003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff510004800 0x7ff510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff51000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200cc3c0 con 0x7ff5100051c0 2024-05-09T06:38:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff51000c160 con 0x7ff510004800 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff51000c360 con 0x7ff5100039b0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff51000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff524f85640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5100039b0 0x7ff510003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff51ffff640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff510004800 0x7ff510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 -- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5100039b0 msgr2=0x7ff510003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5100039b0 0x7ff510003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 -- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff510004800 msgr2=0x7ff510004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff510004800 0x7ff510004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.769+0000 7ff525786640 1 -- 172.21.15.1:0/4059745254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff510003520 con 0x7ff5100051c0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.770+0000 7ff525786640 1 -- 172.21.15.1:0/4059745254 reap_dead start 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.770+0000 7ff525786640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff51000b9c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff518001e20 tx=0x7ff510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8ab95d28f4ca68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.770+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518001a70 con 0x7ff5100051c0 2024-05-09T06:38:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.770+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51802ae40 con 0x7ff5100051c0 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.770+0000 7ff51f7fe640 1 -- 172.21.15.1:0/4059745254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518003bb0 con 0x7ff5100051c0 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 -- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 msgr2=0x7ff51000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff51000b9c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff518001e20 tx=0x7ff510005830 comp rx=0 tx=0).stop 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 -- 172.21.15.1:0/4059745254 shutdown_connections 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 --2- 172.21.15.1:0/4059745254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff51000b9c0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 -- 172.21.15.1:0/4059745254 >> 172.21.15.1:0/4059745254 conn(0x7ff5200d1d70 msgr2=0x7ff5200d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 -- 172.21.15.1:0/4059745254 shutdown_connections 2024-05-09T06:38:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.771+0000 7ff5271e9640 1 -- 172.21.15.1:0/4059745254 wait complete. 2024-05-09T06:38:06.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.772+0000 7ff5271e9640 1 Processor -- start 2024-05-09T06:38:06.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.772+0000 7ff5271e9640 1 -- start start 2024-05-09T06:38:06.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 0x7ff5201e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5201e7bf0 0x7ff5201f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff52006f1b0 con 0x7ff5100051c0 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200ccc60 con 0x7ff5200c6270 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff5271e9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff52006f890 con 0x7ff5201e7bf0 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37880/0 (socket says 172.21.15.1:37880) 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 -- 172.21.15.1:0/739479933 learned_addr learned my addr 172.21.15.1:0/739479933 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff51ffff640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 0x7ff5201e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 -- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5201e7bf0 msgr2=0x7ff5201f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff525786640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5201e7bf0 0x7ff5201f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5201e7bf0 0x7ff5201f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 -- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 msgr2=0x7ff5201e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 0x7ff5201e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff524f85640 1 -- 172.21.15.1:0/739479933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5200c7590 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.773+0000 7ff51ffff640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 0x7ff5201e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff524f85640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7ff514011910 tx=0x7ff514011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51400faf0 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51400fc90 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5200c7870 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514014440 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.774+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5200c7e30 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.775+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5200752b0 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.775+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff514012040 con 0x7ff5100051c0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.776+0000 7ff51d7fa640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 0x7ff50406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.776+0000 7ff51ffff640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 0x7ff50406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.776+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6216+0+0 (secure 0 0 0) 0x7ff51408b680 con 0x7ff5100051c0 2024-05-09T06:38:06.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.776+0000 7ff51ffff640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 0x7ff50406e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0012b0 tx=0x7ff50c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:06.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:06.781+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff514019030 con 0x7ff5100051c0 2024-05-09T06:38:06.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.006+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff5201e7fd0 con 0x7ff5100051c0 2024-05-09T06:38:06.986 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:38:07.006+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:38:06.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.007+0000 7ff51d7fa640 1 -- 172.21.15.1:0/739479933 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7ff514059e30 con 0x7ff5100051c0 2024-05-09T06:38:06.987 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:38:06.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 msgr2=0x7ff50406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 0x7ff50406e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0012b0 tx=0x7ff50c009040 comp rx=0 tx=0).stop 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 msgr2=0x7ff5201f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7ff514011910 tx=0x7ff514011dd0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 shutdown_connections 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff50406be30 0x7ff50406e2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5201e7bf0 0x7ff5201f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5200c6270 0x7ff5201e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 --2- 172.21.15.1:0/739479933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5100051c0 0x7ff5201f36b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:06.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 >> 172.21.15.1:0/739479933 conn(0x7ff5200d1d70 msgr2=0x7ff5200d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:06.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 shutdown_connections 2024-05-09T06:38:06.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.011+0000 7ff5271e9640 1 -- 172.21.15.1:0/739479933 wait complete. 2024-05-09T06:38:06.993 INFO:tasks.ceph.mds.a.smithi001.stderr: -1154> 2024-05-09T06:37:36.218+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-09T06:38:06.994 INFO:tasks.ceph.mds.a.smithi001.stderr: -1151> 2024-05-09T06:37:36.218+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-09T06:38:06.994 INFO:tasks.ceph.mds.a.smithi001.stderr: -1133> 2024-05-09T06:37:36.219+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-09T06:38:06.994 INFO:tasks.ceph.mds.a.smithi001.stderr: -1130> 2024-05-09T06:37:36.219+0000 7f4bb9589640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-09T06:38:06.994 INFO:tasks.ceph.mds.a.smithi001.stderr: -987> 2024-05-09T06:37:36.592+0000 7f4bbd591640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:38:07.002 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-09T06:38:07.018 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:38:07.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.066+0000 7f9f7b87b280 1 -- start start 2024-05-09T06:38:07.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.066+0000 7f9f7b87b280 1 Processor -- start 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c6000 0x561345867080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 0x561345867b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5613465d6780 con 0x5613465c7000 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5613465d6600 con 0x5613465c6000 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5613465d6180 con 0x5613465c7400 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37904/0 (socket says 172.21.15.1:37904) 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 -- 172.21.15.1:0/1129625768 learned_addr learned my addr 172.21.15.1:0/1129625768 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7a027640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c6000 0x561345867080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7a828640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 0x561345867b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 -- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 msgr2=0x561345867b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 0x561345867b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 -- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c6000 msgr2=0x561345867080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c6000 0x561345867080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 -- 172.21.15.1:0/1129625768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134582c780 con 0x5613465c7000 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7a828640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 0x561345867b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f79826640 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x5613466254a0 tx=0x56134658c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4f3d9e9b714030 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f77822640 1 -- 172.21.15.1:0/1129625768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581cd00 con 0x5613465c7000 2024-05-09T06:38:07.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f77822640 1 -- 172.21.15.1:0/1129625768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561345822e00 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f77822640 1 -- 172.21.15.1:0/1129625768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581c680 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134582d860 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56134582da40 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.067+0000 7f9f7b87b280 4 waiting for MDS map... 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f77822640 1 -- 172.21.15.1:0/1129625768 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x5613466c2600 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f77822640 1 -- 172.21.15.1:0/1129625768 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 88) ==== 2885+0+0 (secure 0 0 0) 0x561345851180 con 0x5613465c7000 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f7b87b280 4 Got MDS map 88 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f7b87b280 10 main: main 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f7b87b280 4 main: connecting to RADOS... 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f7b87b280 1 Processor -- start 2024-05-09T06:38:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.068+0000 7f9f7b87b280 1 -- start start 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465fdc00 0x561345868100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346736000 0x561345868680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561345858480 con 0x5613465c7c00 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561345858900 con 0x5613465fdc00 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5613465d7200 con 0x561346736000 2024-05-09T06:38:07.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37912/0 (socket says 172.21.15.1:37912) 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 -- 172.21.15.1:0/3985342942 learned_addr learned my addr 172.21.15.1:0/3985342942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a027640 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465fdc00 0x561345868100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f79826640 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346736000 0x561345868680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 -- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346736000 msgr2=0x561345868680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346736000 0x561345868680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 -- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465fdc00 msgr2=0x561345868100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465fdc00 0x561345868100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 -- 172.21.15.1:0/3985342942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134582dc20 con 0x5613465c7c00 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.069+0000 7f9f7a828640 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x5613466da240 tx=0x5613466e5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969c2d9d747eaad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7581e640 1 -- 172.21.15.1:0/3985342942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581cd00 con 0x5613465c7c00 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7581e640 1 -- 172.21.15.1:0/3985342942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561345822c40 con 0x5613465c7c00 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7581e640 1 -- 172.21.15.1:0/3985342942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581c680 con 0x5613465c7c00 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 -- 172.21.15.1:0/3985342942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 msgr2=0x561345867600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x5613466da240 tx=0x5613466e5800 comp rx=0 tx=0).stop 2024-05-09T06:38:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 -- 172.21.15.1:0/3985342942 shutdown_connections 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346736000 0x561345868680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465fdc00 0x561345868100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/3985342942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7c00 0x561345867600 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 -- 172.21.15.1:0/3985342942 >> 172.21.15.1:0/3985342942 conn(0x5613465c7800 msgr2=0x5613458e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 -- 172.21.15.1:0/3985342942 shutdown_connections 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.070+0000 7f9f7b87b280 1 -- 172.21.15.1:0/3985342942 wait complete. 2024-05-09T06:38:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 Processor -- start 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 -- start start 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c7c00 0x561345869700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 0x561345869180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346737400 0x561345868c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5613465d7500 con 0x5613465fdc00 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5613465d7380 con 0x5613465c7c00 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7b87b280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5613465d7680 con 0x561346737400 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7a027640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 0x561345869180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7a027640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 0x561345869180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37918/0 (socket says 172.21.15.1:37918) 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7a027640 1 -- 172.21.15.1:0/1509185027 learned_addr learned my addr 172.21.15.1:0/1509185027 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f7a828640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c7c00 0x561345869700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.071+0000 7f9f79826640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346737400 0x561345868c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 -- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346737400 msgr2=0x561345868c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561346737400 0x561345868c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 -- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c7c00 msgr2=0x561345869700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c7c00 0x561345869700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 -- 172.21.15.1:0/1509185027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134582c780 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7a027640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 0x561345869180 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x5613466249c0 tx=0x56134658aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7401b640 1 -- 172.21.15.1:0/1509185027 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581cb60 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7401b640 1 -- 172.21.15.1:0/1509185027 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561345822a80 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56134582d860 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7401b640 1 -- 172.21.15.1:0/1509185027 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56134581cd00 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134582de00 con 0x5613465fdc00 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.072+0000 7f9f7b87b280 4 main: resolving pool 15 2024-05-09T06:38:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.073+0000 7f9f7401b640 1 -- 172.21.15.1:0/1509185027 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x561346742c00 con 0x5613465fdc00 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7401b640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x561345b76000 0x561345b7c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7401b640 1 -- 172.21.15.1:0/1509185027 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x561346740a00 con 0x5613465fdc00 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7b87b280 4 main: creating IoCtx.. 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7a828640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x561345b76000 0x561345b7c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x561346737000 0x561345868680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x561346736c00 con 0x561346737000 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f79826640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x561346737000 0x561345868680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f7a828640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x561345b76000 0x561345b7c000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5613466da240 tx=0x56134658bec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.074+0000 7f9f79826640 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x561346737000 0x561345868680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.076+0000 7f9f79826640 1 -- 172.21.15.1:0/1509185027 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x5613458f6000 con 0x561346737000 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "0": { 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "data": { 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 5, 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "inotable": { 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "projected_free": [ 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "start": 1099511627779, 2024-05-09T06:38:07.066 INFO:teuthology.orchestra.run.smithi001.stdout: "len": 1099511627773 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: "free": [ 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: "start": 1099511627779, 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: "len": 1099511627773 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: "result": 0 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:07.067 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:07.068 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.076+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x561346737000 msgr2=0x561345868680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.076+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x561346737000 0x561345868680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.076+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x561345b76000 msgr2=0x561345b7c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x561345b76000 0x561345b7c000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5613466da240 tx=0x56134658bec0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 msgr2=0x561345869180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1509185027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465fdc00 0x561345869180 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x5613466249c0 tx=0x56134658aa80 comp rx=0 tx=0).stop 2024-05-09T06:38:07.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7a027640 1 -- 172.21.15.1:0/1509185027 reap_dead start 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 shutdown_connections 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 >> 172.21.15.1:0/1509185027 conn(0x5613465c7800 msgr2=0x5613458e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 shutdown_connections 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1509185027 wait complete. 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 msgr2=0x561345866b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x5613466254a0 tx=0x56134658c120 comp rx=0 tx=0).stop 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 shutdown_connections 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5613465c7400 0x561345867b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5613465c7000 0x561345866b00 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 --2- 172.21.15.1:0/1129625768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5613465c6000 0x561345867080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 >> 172.21.15.1:0/1129625768 conn(0x5613457f7000 msgr2=0x5613458e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 shutdown_connections 2024-05-09T06:38:07.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.077+0000 7f9f7b87b280 1 -- 172.21.15.1:0/1129625768 wait complete. 2024-05-09T06:38:07.072 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.069763, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-09T06:38:07.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:38:07.072 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:38:07.147 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:38:07.195 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:38:07.195 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:38:07.196 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:07.196 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:07.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:38:07.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:38:07.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:07.212 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:38:07.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:38:07.268 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T06:38:07.269 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:38:07.269 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:38:07.269 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:07.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:38:07.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:38:07.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:07.289 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:38:07.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:38:07.348 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:38:07.348 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:38:07.348 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:07.349 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:07.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:38:07.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:38:07.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:07.366 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:38:07.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:38:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:38:07.422 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:38:07.422 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:38:07.422 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:07.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:38:07.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:38:07.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:07.442 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:38:07.500 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:38:07.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:38:07.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.626+0000 7f0cc42ee640 1 Processor -- start 2024-05-09T06:38:07.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.626+0000 7f0cc42ee640 1 -- start start 2024-05-09T06:38:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc42ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc42ee640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cbc0b33c0 con 0x7f0cbc0d4940 2024-05-09T06:38:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc42ee640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cbc0b3660 con 0x7f0cbc161950 2024-05-09T06:38:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc42ee640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0cbc0d4dd0 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc288b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc288b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37932/0 (socket says 172.21.15.1:37932) 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc1889640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0cbc0d4940 0x7f0cbc0b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48510/0 (socket says 172.21.15.1:48510) 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.627+0000 7f0cc288b640 1 -- 172.21.15.1:0/3610171804 learned_addr learned my addr 172.21.15.1:0/3610171804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.628+0000 7f0cc288b640 1 -- 172.21.15.1:0/3610171804 >> v1:172.21.15.6:6789/0 conn(0x7f0cbc161950 legacy=0x7f0cbc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.628+0000 7f0cc288b640 1 -- 172.21.15.1:0/3610171804 >> v1:172.21.15.1:6789/0 conn(0x7f0cbc0d4940 legacy=0x7f0cbc0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.628+0000 7f0cc1088640 1 -- 172.21.15.1:0/3610171804 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930314744 0 0) 0x7f0cbc0b33c0 con 0x7f0cbc0d4940 2024-05-09T06:38:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.628+0000 7f0cc288b640 1 -- 172.21.15.1:0/3610171804 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc0b3940 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.628+0000 7f0cc288b640 1 --2- 172.21.15.1:0/3610171804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009e50 tx=0x7f0cac02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63e934fb37c53d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc1088640 1 -- 172.21.15.1:0/3610171804 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac036070 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc1088640 1 -- 172.21.15.1:0/3610171804 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac031600 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc1088640 1 -- 172.21.15.1:0/3610171804 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0319f0 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 -- 172.21.15.1:0/3610171804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc0d2b10 msgr2=0x7f0cbc0b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/3610171804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009e50 tx=0x7f0cac02a7c0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 -- 172.21.15.1:0/3610171804 shutdown_connections 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/3610171804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc0b4850 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 -- 172.21.15.1:0/3610171804 >> 172.21.15.1:0/3610171804 conn(0x7f0cbc0d3e20 msgr2=0x7f0cbc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 -- 172.21.15.1:0/3610171804 shutdown_connections 2024-05-09T06:38:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.629+0000 7f0cc42ee640 1 -- 172.21.15.1:0/3610171804 wait complete. 2024-05-09T06:38:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.630+0000 7f0cc42ee640 1 Processor -- start 2024-05-09T06:38:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.630+0000 7f0cc42ee640 1 -- start start 2024-05-09T06:38:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cbc161950 0x7f0cbc153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc0d1100 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc0d0f80 con 0x7f0cbc0d2b10 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc42ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0cbc0d1c00 con 0x7f0cbc161950 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc288b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc288b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37946/0 (socket says 172.21.15.1:37946) 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.631+0000 7f0cc288b640 1 -- 172.21.15.1:0/1780662959 learned_addr learned my addr 172.21.15.1:0/1780662959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc208a640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 -- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cbc161950 msgr2=0x7f0cbc153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cbc161950 0x7f0cbc153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 -- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 msgr2=0x7f0cbc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 -- 172.21.15.1:0/1780662959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc153860 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc208a640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc288b640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02a800 tx=0x7f0cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac036050 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac042070 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac009510 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc0cccd0 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.632+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc0b6e20 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.633+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc15fd50 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.634+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cac004490 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.634+0000 7f0cbaffd640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.634+0000 7f0cc208a640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.634+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f0cac0abfd0 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.635+0000 7f0cc208a640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 0x7f0ca806e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4007fd0 tx=0x7f0cb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.639+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0cbc15fd50 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.861+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cbc0d21e0 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.861+0000 7f0cbaffd640 1 -- 172.21.15.1:0/1780662959 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+4336 (secure 0 0 0) 0x7f0cac07a8e0 con 0x7f0cbc15c9a0 2024-05-09T06:38:07.841 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:07.841 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:38:07.006701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:38:07.842 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 88 2024-05-09T06:38:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 msgr2=0x7f0ca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 0x7f0ca806e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4007fd0 tx=0x7f0cb4009040 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 msgr2=0x7f0cbc0b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0cac02a800 tx=0x7f0cac004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 shutdown_connections 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0cbc15c9a0 0x7f0cbc0b7620 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0cbc161950 0x7f0cbc153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 --2- 172.21.15.1:0/1780662959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cbc0d2b10 0x7f0cbc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.863+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 >> 172.21.15.1:0/1780662959 conn(0x7f0cbc0d3e20 msgr2=0x7f0cbc0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.864+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 shutdown_connections 2024-05-09T06:38:07.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.864+0000 7f0cc42ee640 1 -- 172.21.15.1:0/1780662959 wait complete. 2024-05-09T06:38:07.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:07.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.979+0000 7f909c2e7640 1 Processor -- start 2024-05-09T06:38:07.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.979+0000 7f909c2e7640 1 -- start start 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f909406e6d0 0x7f909406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f90940cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 0x7f909415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f90940ce0e0 con 0x7f909406e6d0 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f90940ce3c0 con 0x7f90940cdab0 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909c2e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90940ce6a0 con 0x7f90940cf180 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f909406e6d0 0x7f909406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f909406e6d0 0x7f909406eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37436/0 (socket says 172.21.15.1:37436) 2024-05-09T06:38:07.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 -- 172.21.15.1:0/3690611707 learned_addr learned my addr 172.21.15.1:0/3690611707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a884640 1 --2- 172.21.15.1:0/3690611707 >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 0x7f909415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f9099882640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f90940cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 -- 172.21.15.1:0/3690611707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 msgr2=0x7f90940cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f90940cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 -- 172.21.15.1:0/3690611707 >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 msgr2=0x7f909415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 --2- 172.21.15.1:0/3690611707 >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 0x7f909415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a083640 1 -- 172.21.15.1:0/3690611707 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940ce980 con 0x7f909406e6d0 2024-05-09T06:38:07.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.980+0000 7f909a884640 1 --2- 172.21.15.1:0/3690611707 >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 0x7f909415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.981+0000 7f909a083640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f909406eab0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f90940b16f0 tx=0x7f909002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1701ee70a6561a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.981+0000 7f9099081640 1 -- 172.21.15.1:0/3690611707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090037070 con 0x7f909406e6d0 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.981+0000 7f9099081640 1 -- 172.21.15.1:0/3690611707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090032440 con 0x7f909406e6d0 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.981+0000 7f9099081640 1 -- 172.21.15.1:0/3690611707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090032810 con 0x7f909406e6d0 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 -- 172.21.15.1:0/3690611707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 msgr2=0x7f909406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f909406eab0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f90940b16f0 tx=0x7f909002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 -- 172.21.15.1:0/3690611707 shutdown_connections 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 --2- 172.21.15.1:0/3690611707 >> v2:172.21.15.6:3300/0 conn(0x7f90940cdab0 0x7f909415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f90940cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 --2- 172.21.15.1:0/3690611707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f909406eab0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 -- 172.21.15.1:0/3690611707 >> 172.21.15.1:0/3690611707 conn(0x7f90940cc4b0 msgr2=0x7f90940cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:07.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 -- 172.21.15.1:0/3690611707 shutdown_connections 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.982+0000 7f909c2e7640 1 -- 172.21.15.1:0/3690611707 wait complete. 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.983+0000 7f909c2e7640 1 Processor -- start 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.983+0000 7f909c2e7640 1 -- start start 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90940cdab0 0x7f909413a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f9094159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9094070ed0 con 0x7f909406e6d0 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f909406eff0 con 0x7f90940cdab0 2024-05-09T06:38:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90941447c0 con 0x7f90940cf180 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37438/0 (socket says 172.21.15.1:37438) 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 -- 172.21.15.1:0/2023091261 learned_addr learned my addr 172.21.15.1:0/2023091261 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f9099882640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90940cdab0 0x7f909413a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a884640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f9094159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 -- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 msgr2=0x7f9094159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f9094159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 -- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90940cdab0 msgr2=0x7f909413a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90940cdab0 0x7f909413a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 -- 172.21.15.1:0/2023091261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909413ad20 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909a083640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f909002a220 tx=0x7f90900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090041070 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909003c870 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909413b580 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.984+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090037040 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.985+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909413b2a0 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.986+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940752b0 con 0x7f909406e6d0 2024-05-09T06:38:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.986+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9090044070 con 0x7f909406e6d0 2024-05-09T06:38:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.987+0000 7f908affd640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 0x7f907806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.987+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f90900abe30 con 0x7f909406e6d0 2024-05-09T06:38:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.987+0000 7f9099882640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 0x7f907806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:07.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.987+0000 7f9099882640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 0x7f907806e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f9084008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:07.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:07.991+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f909007a610 con 0x7f909406e6d0 2024-05-09T06:38:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.176+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f909406e4b0 con 0x7f909406e6d0 2024-05-09T06:38:08.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.177+0000 7f908affd640 1 -- 172.21.15.1:0/2023091261 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+17055 (secure 0 0 0) 0x7f9090079f10 con 0x7f909406e6d0 2024-05-09T06:38:08.157 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:08.157 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":110,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:07.006528+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6835/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6834/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6835/3796896332":"2024-05-10T06:37:30.837395+0000","172.21.15.1:6834/3796896332":"2024-05-10T06:37:30.837395+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 msgr2=0x7f907806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 0x7f907806e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f9084008040 comp rx=0 tx=0).stop 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 msgr2=0x7f9094143ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f909002a220 tx=0x7f90900046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 shutdown_connections 2024-05-09T06:38:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f907806be30 0x7f907806e2f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90940cf180 0x7f9094159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90940cdab0 0x7f909413a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 --2- 172.21.15.1:0/2023091261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f909406e6d0 0x7f9094143ca0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 >> 172.21.15.1:0/2023091261 conn(0x7f90940cc4b0 msgr2=0x7f90940b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 shutdown_connections 2024-05-09T06:38:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.179+0000 7f909c2e7640 1 -- 172.21.15.1:0/2023091261 wait complete. 2024-05-09T06:38:08.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:33.397054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:34.058197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:08.171 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:08.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 Processor -- start 2024-05-09T06:38:08.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 -- start start 2024-05-09T06:38:08.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7b80d45f0 0x7fd7b815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b8161fe0 con 0x7fd7b80d1c70 2024-05-09T06:38:08.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b81622c0 con 0x7fd7b815e190 2024-05-09T06:38:08.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.299+0000 7fd7bed5a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b815f580 con 0x7fd7b80d45f0 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7b7fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd7b80d1c70 0x7fd7b80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47438/0 (socket says 172.21.15.1:47438) 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7b7fff640 1 -- 172.21.15.1:0/2156323536 learned_addr learned my addr 172.21.15.1:0/2156323536 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7bd2f7640 1 --2- 172.21.15.1:0/2156323536 >> v2:172.21.15.1:3300/0 conn(0x7fd7b80d45f0 0x7fd7b815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/2156323536 >> v1:172.21.15.6:6789/0 conn(0x7fd7b815e190 legacy=0x7fd7b80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/2156323536 >> v1:172.21.15.1:6789/0 conn(0x7fd7b80d1c70 legacy=0x7fd7b80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7b77fe640 1 -- 172.21.15.1:0/2156323536 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434077786 0 0) 0x7fd7b8161fe0 con 0x7fd7b80d1c70 2024-05-09T06:38:08.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.300+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/2156323536 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b81625a0 con 0x7fd7b80d45f0 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bd2f7640 1 --2- 172.21.15.1:0/2156323536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b80d45f0 0x7fd7b815f780 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009d90 tx=0x7fd7a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf2a7f42a19271f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7b77fe640 1 -- 172.21.15.1:0/2156323536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8007740 con 0x7fd7b80d45f0 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7b77fe640 1 -- 172.21.15.1:0/2156323536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a80078e0 con 0x7fd7b80d45f0 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7b77fe640 1 -- 172.21.15.1:0/2156323536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8007cb0 con 0x7fd7b80d45f0 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 -- 172.21.15.1:0/2156323536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b80d45f0 msgr2=0x7fd7b815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/2156323536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b80d45f0 0x7fd7b815f780 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009d90 tx=0x7fd7a802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 -- 172.21.15.1:0/2156323536 shutdown_connections 2024-05-09T06:38:08.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/2156323536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b80d45f0 0x7fd7b815f780 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 -- 172.21.15.1:0/2156323536 >> 172.21.15.1:0/2156323536 conn(0x7fd7b8073340 msgr2=0x7fd7b809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 -- 172.21.15.1:0/2156323536 shutdown_connections 2024-05-09T06:38:08.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.301+0000 7fd7bed5a640 1 -- 172.21.15.1:0/2156323536 wait complete. 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 Processor -- start 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 -- start start 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 0x7fd7b8159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7b80d45f0 0x7fd7b80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b80b56d0 con 0x7fd7b815e190 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8161c60 con 0x7fd7b80d1c70 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.303+0000 7fd7bed5a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b80b5850 con 0x7fd7b80d45f0 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37468/0 (socket says 172.21.15.1:37468) 2024-05-09T06:38:08.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/3575136558 learned_addr learned my addr 172.21.15.1:0/3575136558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bcaf6640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 0x7fd7b8159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7b7fff640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7b80d45f0 0x7fd7b80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7b80d45f0 msgr2=0x7fd7b80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7b80d45f0 0x7fd7b80cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 msgr2=0x7fd7b8159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 0x7fd7b8159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 -- 172.21.15.1:0/3575136558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b80b7c80 con 0x7fd7b815e190 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bcaf6640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 0x7fd7b8159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bd2f7640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009eb0 tx=0x7fd7a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8040070 con 0x7fd7b815e190 2024-05-09T06:38:08.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a8031c90 con 0x7fd7b815e190 2024-05-09T06:38:08.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b81f3dc0 con 0x7fd7b815e190 2024-05-09T06:38:08.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8045680 con 0x7fd7b815e190 2024-05-09T06:38:08.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.304+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b81f4380 con 0x7fd7b815e190 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.305+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b81f5ef0 con 0x7fd7b815e190 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.306+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7a8039040 con 0x7fd7b815e190 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.306+0000 7fd7b57fa640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 0x7fd79806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.306+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7fd7a80aca30 con 0x7fd7b815e190 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.306+0000 7fd7bcaf6640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 0x7fd79806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.307+0000 7fd7bcaf6640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 0x7fd79806e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.311+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7b80b7c80 con 0x7fd7b815e190 2024-05-09T06:38:08.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.496+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7b80752b0 con 0x7fd7b815e190 2024-05-09T06:38:08.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.496+0000 7fd7b57fa640 1 -- 172.21.15.1:0/3575136558 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fd7a807b2e0 con 0x7fd7b815e190 2024-05-09T06:38:08.476 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 msgr2=0x7fd79806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 0x7fd79806e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 msgr2=0x7fd7b80b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009eb0 tx=0x7fd7a8004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 shutdown_connections 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd79806be30 0x7fd79806e2f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7b815e190 0x7fd7b80b7740 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd7b80d45f0 0x7fd7b80cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 --2- 172.21.15.1:0/3575136558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80d1c70 0x7fd7b8159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 >> 172.21.15.1:0/3575136558 conn(0x7fd7b8073340 msgr2=0x7fd7b80d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 shutdown_connections 2024-05-09T06:38:08.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.499+0000 7fd7bed5a640 1 -- 172.21.15.1:0/3575136558 wait complete. 2024-05-09T06:38:08.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:38:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.617+0000 7fbe421c8640 1 Processor -- start 2024-05-09T06:38:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.617+0000 7fbe421c8640 1 -- start start 2024-05-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe421c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbe3c0d06f0 0x7fbe3c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe421c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe3c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe421c8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe3c072050 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe421c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3c0cd160 con 0x7fbe3c0d06f0 2024-05-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe421c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c072330 con 0x7fbe3c0cf8e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe3affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbe3c0c41e0 0x7fbe3c161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47444/0 (socket says 172.21.15.1:47444) 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe3affd640 1 -- 172.21.15.1:0/1740169682 learned_addr learned my addr 172.21.15.1:0/1740169682 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.618+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/1740169682 >> v2:172.21.15.6:3300/0 conn(0x7fbe3c0d06f0 0x7fbe3c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3bfff640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe3c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285970897 0 0) 0x7fbe3c072050 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe24003520 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643517396 0 0) 0x7fbe24003520 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 msgr2=0x7fbe3c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe3c0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 >> v2:172.21.15.6:3300/0 conn(0x7fbe3c0d06f0 msgr2=0x7fbe3c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 --2- 172.21.15.1:0/1740169682 >> v2:172.21.15.6:3300/0 conn(0x7fbe3c0d06f0 0x7fbe3c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c072610 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fbe30001670 con 0x7fbe3c0c41e0 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 >> v1:172.21.15.1:6789/0 conn(0x7fbe3c0c41e0 legacy=0x7fbe3c161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c0cd160 con 0x7fbe240039b0 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c072330 con 0x7fbe24004800 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2400b9c0 con 0x7fbe3c0cf8e0 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3affd640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3bfff640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 -- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 msgr2=0x7fbe240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe3c0cf8e0 0x7fbe240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 -- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe24004800 msgr2=0x7fbe24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3b7fe640 1 -- 172.21.15.1:0/1740169682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe24003520 con 0x7fbe240039b0 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.619+0000 7fbe3affd640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.620+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009ac0 tx=0x7fbe24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a4cbee6c0072de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.620+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c036070 con 0x7fbe240039b0 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.620+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2c031400 con 0x7fbe240039b0 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.620+0000 7fbe3a7fc640 1 -- 172.21.15.1:0/1740169682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c009690 con 0x7fbe240039b0 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 -- 172.21.15.1:0/1740169682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe240039b0 msgr2=0x7fbe24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 --2- 172.21.15.1:0/1740169682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009ac0 tx=0x7fbe24005830 comp rx=0 tx=0).stop 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe3bfff640 1 -- 172.21.15.1:0/1740169682 reap_dead start 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 -- 172.21.15.1:0/1740169682 shutdown_connections 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 -- 172.21.15.1:0/1740169682 >> 172.21.15.1:0/1740169682 conn(0x7fbe3c0d0b50 msgr2=0x7fbe3c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 -- 172.21.15.1:0/1740169682 shutdown_connections 2024-05-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.621+0000 7fbe421c8640 1 -- 172.21.15.1:0/1740169682 wait complete. 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.622+0000 7fbe421c8640 1 Processor -- start 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.622+0000 7fbe421c8640 1 -- start start 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 0x7fbe3c0a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe3c1f3870 0x7fbe3c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c06b1e0 con 0x7fbe3c1f7220 2024-05-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c071ce0 con 0x7fbe3c1f3870 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe421c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c06b360 con 0x7fbe240039b0 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37494/0 (socket says 172.21.15.1:37494) 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 -- 172.21.15.1:0/3716065554 learned_addr learned my addr 172.21.15.1:0/3716065554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3bfff640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe3c1f3870 0x7fbe3c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 0x7fbe3c0a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 -- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 msgr2=0x7fbe3c0a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 0x7fbe3c0a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 -- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe3c1f3870 msgr2=0x7fbe3c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe3c1f3870 0x7fbe3c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 -- 172.21.15.1:0/3716065554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c1f7990 con 0x7fbe3c1f7220 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 0x7fbe3c0a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.623+0000 7fbe3affd640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e7e0 tx=0x7fbe3000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.624+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000ee90 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.624+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe30018070 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.624+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1f7c70 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.624+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000ca90 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.624+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c0b3950 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.625+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c0752b0 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.625+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe3000cc30 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.626+0000 7fbe23fff640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 0x7fbe2806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.626+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7fbe3008ac60 con 0x7fbe3c1f7220 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.626+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 0x7fbe2806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.626+0000 7fbe3b7fe640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 0x7fbe2806e4c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c033000 tx=0x7fbe2c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.630+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe300593c0 con 0x7fbe3c1f7220 2024-05-09T06:38:08.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.811+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe3c072f30 con 0x7fbe3c1f7220 2024-05-09T06:38:08.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.811+0000 7fbe23fff640 1 -- 172.21.15.1:0/3716065554 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fbe30058cc0 con 0x7fbe3c1f7220 2024-05-09T06:38:08.791 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 msgr2=0x7fbe2806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 0x7fbe2806e4c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c033000 tx=0x7fbe2c008030 comp rx=0 tx=0).stop 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 msgr2=0x7fbe3c1f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e7e0 tx=0x7fbe3000eca0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 shutdown_connections 2024-05-09T06:38:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe2806c000 0x7fbe2806e4c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe3c1f3870 0x7fbe3c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe3c1f7220 0x7fbe3c1f3330 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 --2- 172.21.15.1:0/3716065554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe240039b0 0x7fbe3c0a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 >> 172.21.15.1:0/3716065554 conn(0x7fbe3c0d0b50 msgr2=0x7fbe3c15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 shutdown_connections 2024-05-09T06:38:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.813+0000 7fbe421c8640 1 -- 172.21.15.1:0/3716065554 wait complete. 2024-05-09T06:38:08.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:38:08.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.931+0000 7f942ac93640 1 Processor -- start 2024-05-09T06:38:08.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 -- start start 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f942415e190 0x7f942415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94240d45f0 0x7f94240b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9424161fe0 con 0x7f94240d1c70 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f942415f580 con 0x7f94240d45f0 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f942ac93640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f94241622c0 con 0x7f942415e190 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.932+0000 7f9429230640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94240d45f0 0x7f94240b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9423fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f94240d1c70 0x7f94240d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47446/0 (socket says 172.21.15.1:47446) 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9423fff640 1 -- 172.21.15.1:0/1234086995 learned_addr learned my addr 172.21.15.1:0/1234086995 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9429230640 1 --2- 172.21.15.1:0/1234086995 >> v2:172.21.15.1:3300/0 conn(0x7f94240d45f0 0x7f94240b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37502/0 (socket says 172.21.15.1:37502) 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9428a2f640 1 --2- 172.21.15.1:0/1234086995 >> v2:172.21.15.6:3300/0 conn(0x7f942415e190 0x7f942415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9429230640 1 -- 172.21.15.1:0/1234086995 >> v2:172.21.15.6:3300/0 conn(0x7f942415e190 msgr2=0x7f942415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9429230640 1 --2- 172.21.15.1:0/1234086995 >> v2:172.21.15.6:3300/0 conn(0x7f942415e190 0x7f942415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9429230640 1 -- 172.21.15.1:0/1234086995 >> v1:172.21.15.1:6789/0 conn(0x7f94240d1c70 legacy=0x7f94240d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f9429230640 1 -- 172.21.15.1:0/1234086995 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94241625a0 con 0x7f94240d45f0 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.933+0000 7f94237fe640 1 -- 172.21.15.1:0/1234086995 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260582749 0 0) 0x7f9424161fe0 con 0x7f94240d1c70 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f9429230640 1 --2- 172.21.15.1:0/1234086995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b3de0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f94240d1bf0 tx=0x7f940c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6342a173d4bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f94237fe640 1 -- 172.21.15.1:0/1234086995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c037070 con 0x7f94240d45f0 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f94237fe640 1 -- 172.21.15.1:0/1234086995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c032440 con 0x7f94240d45f0 2024-05-09T06:38:08.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f94237fe640 1 -- 172.21.15.1:0/1234086995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c032810 con 0x7f94240d45f0 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 -- 172.21.15.1:0/1234086995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 msgr2=0x7f94240b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 --2- 172.21.15.1:0/1234086995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b3de0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f94240d1bf0 tx=0x7f940c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 -- 172.21.15.1:0/1234086995 shutdown_connections 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 --2- 172.21.15.1:0/1234086995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b3de0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 --2- 172.21.15.1:0/1234086995 >> v2:172.21.15.6:3300/0 conn(0x7f942415e190 0x7f942415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 -- 172.21.15.1:0/1234086995 >> 172.21.15.1:0/1234086995 conn(0x7f9424073340 msgr2=0x7f942409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 -- 172.21.15.1:0/1234086995 shutdown_connections 2024-05-09T06:38:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.934+0000 7f942ac93640 1 -- 172.21.15.1:0/1234086995 wait complete. 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.935+0000 7f942ac93640 1 Processor -- start 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 -- start start 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94240d1c70 0x7f94240ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942415e190 0x7f94240b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94240b56d0 con 0x7f94240d45f0 2024-05-09T06:38:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9424161c60 con 0x7f942415e190 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.936+0000 7f942ac93640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94240b5850 con 0x7f94240d1c70 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37516/0 (socket says 172.21.15.1:37516) 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 -- 172.21.15.1:0/3689527325 learned_addr learned my addr 172.21.15.1:0/3689527325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9429230640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942415e190 0x7f94240b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9428a2f640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94240d1c70 0x7f94240ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 -- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94240d1c70 msgr2=0x7f94240ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94240d1c70 0x7f94240ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 -- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942415e190 msgr2=0x7f94240b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942415e190 0x7f94240b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 -- 172.21.15.1:0/3689527325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240cc130 con 0x7f94240d45f0 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f9423fff640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f941400c820 tx=0x7f941400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414018070 con 0x7f94240d45f0 2024-05-09T06:38:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941400a640 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941400aa30 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94240cc410 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.937+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94240cc9d0 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.938+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94240752b0 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.939+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f941401b070 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.939+0000 7f94217fa640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 0x7f940406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.939+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f941408a650 con 0x7f94240d45f0 2024-05-09T06:38:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.939+0000 7f9428a2f640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 0x7f940406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.940+0000 7f9428a2f640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 0x7f940406e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f941c0079a0 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:08.944+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94140587f0 con 0x7f94240d45f0 2024-05-09T06:38:09.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.126+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f942408e1c0 con 0x7f94240d45f0 2024-05-09T06:38:09.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.127+0000 7f94217fa640 1 -- 172.21.15.1:0/3689527325 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f941405caf0 con 0x7f94240d45f0 2024-05-09T06:38:09.106 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:38:09.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 msgr2=0x7f940406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 0x7f940406e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f941c0079a0 tx=0x7f941c008040 comp rx=0 tx=0).stop 2024-05-09T06:38:09.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 msgr2=0x7f94240b8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f941400c820 tx=0x7f941400cce0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 shutdown_connections 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f940406be30 0x7f940406e2f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942415e190 0x7f94240b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94240d45f0 0x7f94240b8fa0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 --2- 172.21.15.1:0/3689527325 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94240d1c70 0x7f94240ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 >> 172.21.15.1:0/3689527325 conn(0x7f9424073340 msgr2=0x7f942406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 shutdown_connections 2024-05-09T06:38:09.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.129+0000 7f942ac93640 1 -- 172.21.15.1:0/3689527325 wait complete. 2024-05-09T06:38:09.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:09.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.244+0000 7f1875453640 1 Processor -- start 2024-05-09T06:38:09.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.244+0000 7f1875453640 1 -- start start 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f1875453640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f18700d4940 0x7f18700b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f1875453640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18700b33c0 con 0x7f18700d2b10 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f1875453640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18700b3660 con 0x7f1870161950 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f1875453640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f18700d4dd0 con 0x7f18700d4940 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f186e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f18700d4940 0x7f18700b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f186e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f18700d4940 0x7f18700b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37520/0 (socket says 172.21.15.1:37520) 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f186e7fc640 1 -- 172.21.15.1:0/3440375963 learned_addr learned my addr 172.21.15.1:0/3440375963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.245+0000 7f186f7fe640 1 --1- 172.21.15.1:0/3440375963 >> v1:172.21.15.1:6789/0 conn(0x7f18700d2b10 0x7f18700b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47458/0 (socket says 172.21.15.1:47458) 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186e7fc640 1 -- 172.21.15.1:0/3440375963 >> v1:172.21.15.6:6789/0 conn(0x7f1870161950 legacy=0x7f18700d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186e7fc640 1 -- 172.21.15.1:0/3440375963 >> v1:172.21.15.1:6789/0 conn(0x7f18700d2b10 legacy=0x7f18700b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186dffb640 1 -- 172.21.15.1:0/3440375963 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457085204 0 0) 0x7f18700b33c0 con 0x7f18700d2b10 2024-05-09T06:38:09.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186e7fc640 1 -- 172.21.15.1:0/3440375963 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700b3940 con 0x7f18700d4940 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186e7fc640 1 --2- 172.21.15.1:0/3440375963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700d4940 0x7f18700b4140 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e50 tx=0x7f186402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fa36a8464dc464 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186dffb640 1 -- 172.21.15.1:0/3440375963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864036070 con 0x7f18700d4940 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186dffb640 1 -- 172.21.15.1:0/3440375963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864031600 con 0x7f18700d4940 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.246+0000 7f186dffb640 1 -- 172.21.15.1:0/3440375963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640319f0 con 0x7f18700d4940 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 -- 172.21.15.1:0/3440375963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700d4940 msgr2=0x7f18700b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 --2- 172.21.15.1:0/3440375963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700d4940 0x7f18700b4140 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e50 tx=0x7f186402a7c0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 -- 172.21.15.1:0/3440375963 shutdown_connections 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 --2- 172.21.15.1:0/3440375963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700d4940 0x7f18700b4140 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 -- 172.21.15.1:0/3440375963 >> 172.21.15.1:0/3440375963 conn(0x7f18700d3e20 msgr2=0x7f18700d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 -- 172.21.15.1:0/3440375963 shutdown_connections 2024-05-09T06:38:09.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.247+0000 7f1875453640 1 -- 172.21.15.1:0/3440375963 wait complete. 2024-05-09T06:38:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.248+0000 7f1875453640 1 Processor -- start 2024-05-09T06:38:09.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.248+0000 7f1875453640 1 -- start start 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 0x7f18701f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1870161950 0x7f18700ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18700d1100 con 0x7f18700b6a10 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f18700d0f80 con 0x7f18700d2b10 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f1875453640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18700d1c00 con 0x7f1870161950 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f186f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f186f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37536/0 (socket says 172.21.15.1:37536) 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.249+0000 7f186f7fe640 1 -- 172.21.15.1:0/4139057512 learned_addr learned my addr 172.21.15.1:0/4139057512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186effd640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 0x7f18701f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186e7fc640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1870161950 0x7f18700ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 -- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1870161950 msgr2=0x7f18700ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1870161950 0x7f18700ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 -- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 msgr2=0x7f18701f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 0x7f18701f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 -- 172.21.15.1:0/4139057512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700b6fd0 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186effd640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 0x7f18701f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f186f7fe640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f186800c480 tx=0x7f186800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868018070 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868007500 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18700b72b0 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868004a20 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.250+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18700b7870 con 0x7f18700b6a10 2024-05-09T06:38:09.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.251+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18700752b0 con 0x7f18700b6a10 2024-05-09T06:38:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.252+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f186801b070 con 0x7f18700b6a10 2024-05-09T06:38:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.252+0000 7f18577fe640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 0x7f185c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.252+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f186808a390 con 0x7f18700b6a10 2024-05-09T06:38:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.252+0000 7f186effd640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 0x7f185c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.253+0000 7f186effd640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 0x7f185c06e2f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f60 tx=0x7f1860009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.257+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1868058b80 con 0x7f18700b6a10 2024-05-09T06:38:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.474+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18700d21e0 con 0x7f18700b6a10 2024-05-09T06:38:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.475+0000 7f18577fe640 1 -- 172.21.15.1:0/4139057512 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5158 (secure 0 0 0) 0x7f1868058480 con 0x7f18700b6a10 2024-05-09T06:38:09.456 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:09.456 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9441,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1642527228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1642527228},{"type":"v1","addr":"172.21.15.1:6835","nonce":1642527228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:38:07.006701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:38:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 89 2024-05-09T06:38:09.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 msgr2=0x7f185c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 0x7f185c06e2f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f60 tx=0x7f1860009040 comp rx=0 tx=0).stop 2024-05-09T06:38:09.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 msgr2=0x7f18700b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f186800c480 tx=0x7f186800c940 comp rx=0 tx=0).stop 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 shutdown_connections 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f185c06be30 0x7f185c06e2f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18700b6a10 0x7f18700b86f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1870161950 0x7f18700ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 --2- 172.21.15.1:0/4139057512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18700d2b10 0x7f18701f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 >> 172.21.15.1:0/4139057512 conn(0x7f18700d3e20 msgr2=0x7f18700b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 shutdown_connections 2024-05-09T06:38:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.477+0000 7f1875453640 1 -- 172.21.15.1:0/4139057512 wait complete. 2024-05-09T06:38:09.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.593+0000 7f6ba53b0640 1 Processor -- start 2024-05-09T06:38:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.593+0000 7f6ba53b0640 1 -- start start 2024-05-09T06:38:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6ba53b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ba00d2b10 0x7f6ba00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6ba53b0640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba00b33c0 con 0x7f6ba0161950 2024-05-09T06:38:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6ba53b0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba00b3660 con 0x7f6ba00d4940 2024-05-09T06:38:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6ba53b0640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba00d4dd0 con 0x7f6ba00d2b10 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ba00d2b10 0x7f6ba00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9effd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6ba0161950 0x7f6ba00d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47462/0 (socket says 172.21.15.1:47462) 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9f7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6ba00d2b10 0x7f6ba00b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37542/0 (socket says 172.21.15.1:37542) 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9effd640 1 -- 172.21.15.1:0/3387575444 learned_addr learned my addr 172.21.15.1:0/3387575444 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9f7fe640 1 -- 172.21.15.1:0/3387575444 >> v1:172.21.15.6:6789/0 conn(0x7f6ba00d4940 legacy=0x7f6ba00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9f7fe640 1 -- 172.21.15.1:0/3387575444 >> v1:172.21.15.1:6789/0 conn(0x7f6ba0161950 legacy=0x7f6ba00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9f7fe640 1 -- 172.21.15.1:0/3387575444 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba00b3940 con 0x7f6ba00d2b10 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.594+0000 7f6b9e7fc640 1 -- 172.21.15.1:0/3387575444 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625718778 0 0) 0x7f6ba00b33c0 con 0x7f6ba0161950 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.595+0000 7f6b9f7fe640 1 --2- 172.21.15.1:0/3387575444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba00d2b10 0x7f6ba00b4850 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009d90 tx=0x7f6b9802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b48f239772edd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.595+0000 7f6b9e7fc640 1 -- 172.21.15.1:0/3387575444 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98007740 con 0x7f6ba00d2b10 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.595+0000 7f6b9e7fc640 1 -- 172.21.15.1:0/3387575444 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b980078e0 con 0x7f6ba00d2b10 2024-05-09T06:38:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.595+0000 7f6b9e7fc640 1 -- 172.21.15.1:0/3387575444 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98007cb0 con 0x7f6ba00d2b10 2024-05-09T06:38:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3387575444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba00d2b10 msgr2=0x7f6ba00b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3387575444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba00d2b10 0x7f6ba00b4850 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009d90 tx=0x7f6b9802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3387575444 shutdown_connections 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3387575444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba00d2b10 0x7f6ba00b4850 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3387575444 >> 172.21.15.1:0/3387575444 conn(0x7f6ba00d3e20 msgr2=0x7f6ba00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3387575444 shutdown_connections 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.596+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3387575444 wait complete. 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.597+0000 7f6ba53b0640 1 Processor -- start 2024-05-09T06:38:09.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.597+0000 7f6ba53b0640 1 -- start start 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba00d2b10 0x7f6ba0154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 0x7f6ba009f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00d1100 con 0x7f6ba0161950 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00d0f80 con 0x7f6ba00d2b10 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6ba53b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba00d1c00 con 0x7f6ba00b7560 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37552/0 (socket says 172.21.15.1:37552) 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 -- 172.21.15.1:0/3295024792 learned_addr learned my addr 172.21.15.1:0/3295024792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b9effd640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba00d2b10 0x7f6ba0154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b9f7fe640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 0x7f6ba009f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 -- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 msgr2=0x7f6ba009f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 0x7f6ba009f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 -- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba00d2b10 msgr2=0x7f6ba0154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba00d2b10 0x7f6ba0154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 -- 172.21.15.1:0/3295024792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba00b7b00 con 0x7f6ba0161950 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b9f7fe640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 0x7f6ba009f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b97fff640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c4d0 tx=0x7f6b8800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.598+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880052d0 con 0x7f6ba0161950 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.599+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b88005470 con 0x7f6ba0161950 2024-05-09T06:38:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.599+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba00b7de0 con 0x7f6ba0161950 2024-05-09T06:38:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.599+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88012850 con 0x7f6ba0161950 2024-05-09T06:38:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.599+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba00b83a0 con 0x7f6ba0161950 2024-05-09T06:38:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.600+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba00752b0 con 0x7f6ba0161950 2024-05-09T06:38:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.600+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b880129f0 con 0x7f6ba0161950 2024-05-09T06:38:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.601+0000 7f6b977fe640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.601+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f6b8808abf0 con 0x7f6ba0161950 2024-05-09T06:38:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.601+0000 7f6b9effd640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.601+0000 7f6b9effd640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 0x7f6b8006e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000f60 tx=0x7f6b90007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.605+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b88059350 con 0x7f6ba0161950 2024-05-09T06:38:09.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.790+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ba00d21e0 con 0x7f6ba0161950 2024-05-09T06:38:09.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.791+0000 7f6b977fe640 1 -- 172.21.15.1:0/3295024792 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+17055 (secure 0 0 0) 0x7f6b88058c50 con 0x7f6ba0161950 2024-05-09T06:38:09.772 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:09.772 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":110,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:07.006528+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6835/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6834/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6835/3796896332":"2024-05-10T06:37:30.837395+0000","172.21.15.1:6834/3796896332":"2024-05-10T06:37:30.837395+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:09.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 msgr2=0x7f6b8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 0x7f6b8006e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000f60 tx=0x7f6b90007e40 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 msgr2=0x7f6ba00b9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c4d0 tx=0x7f6b8800c990 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 shutdown_connections 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6b8006be30 0x7f6b8006e2f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6ba00b7560 0x7f6ba009f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ba0161950 0x7f6ba00b9300 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 --2- 172.21.15.1:0/3295024792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ba00d2b10 0x7f6ba0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 >> 172.21.15.1:0/3295024792 conn(0x7f6ba00d3e20 msgr2=0x7f6ba00b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 shutdown_connections 2024-05-09T06:38:09.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.794+0000 7f6ba53b0640 1 -- 172.21.15.1:0/3295024792 wait complete. 2024-05-09T06:38:09.786 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:38:09.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.910+0000 7f4571fc8640 1 Processor -- start 2024-05-09T06:38:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.910+0000 7f4571fc8640 1 -- start start 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f4571fc8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c15e6d0 0x7f456c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f4571fc8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c070f40 con 0x7f456c15db10 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f4571fc8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c0cf170 con 0x7f456c0b2a60 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f4571fc8640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f456c0cf450 con 0x7f456c15e6d0 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c15e6d0 0x7f456c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f456c15e6d0 0x7f456c06e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37560/0 (socket says 172.21.15.1:37560) 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f456c15db10 0x7f456c0715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47472/0 (socket says 172.21.15.1:47472) 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 -- 172.21.15.1:0/823503085 learned_addr learned my addr 172.21.15.1:0/823503085 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 -- 172.21.15.1:0/823503085 >> v1:172.21.15.6:6789/0 conn(0x7f456c0b2a60 legacy=0x7f456c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 -- 172.21.15.1:0/823503085 >> v1:172.21.15.1:6789/0 conn(0x7f456c15db10 legacy=0x7f456c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456a7fc640 1 -- 172.21.15.1:0/823503085 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683301079 0 0) 0x7f456c070f40 con 0x7f456c15db10 2024-05-09T06:38:09.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.911+0000 7f456bfff640 1 -- 172.21.15.1:0/823503085 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c070c10 con 0x7f456c15e6d0 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.912+0000 7f456bfff640 1 --2- 172.21.15.1:0/823503085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c06e340 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f456400af60 tx=0x7f456402b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe1e95809b8dd0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.912+0000 7f456a7fc640 1 -- 172.21.15.1:0/823503085 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564038070 con 0x7f456c15e6d0 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.912+0000 7f456a7fc640 1 -- 172.21.15.1:0/823503085 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456400a2d0 con 0x7f456c15e6d0 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.912+0000 7f456a7fc640 1 -- 172.21.15.1:0/823503085 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564008720 con 0x7f456c15e6d0 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 -- 172.21.15.1:0/823503085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 msgr2=0x7f456c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 --2- 172.21.15.1:0/823503085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c06e340 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f456400af60 tx=0x7f456402b830 comp rx=0 tx=0).stop 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 -- 172.21.15.1:0/823503085 shutdown_connections 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 --2- 172.21.15.1:0/823503085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c06e340 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 -- 172.21.15.1:0/823503085 >> 172.21.15.1:0/823503085 conn(0x7f456c0cc4b0 msgr2=0x7f456c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 -- 172.21.15.1:0/823503085 shutdown_connections 2024-05-09T06:38:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.913+0000 7f4571fc8640 1 -- 172.21.15.1:0/823503085 wait complete. 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.914+0000 7f4571fc8640 1 Processor -- start 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.914+0000 7f4571fc8640 1 -- start start 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c0b2a60 0x7f456c144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 0x7f456c1449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f456c06ceb0 con 0x7f456c15e6d0 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f456c06cd30 con 0x7f456c15db10 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f4571fc8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f456c06d030 con 0x7f456c0b2a60 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37570/0 (socket says 172.21.15.1:37570) 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 -- 172.21.15.1:0/770556209 learned_addr learned my addr 172.21.15.1:0/770556209 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456affd640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 0x7f456c1449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456b7fe640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c0b2a60 0x7f456c144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 -- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c0b2a60 msgr2=0x7f456c144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c0b2a60 0x7f456c144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 -- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 msgr2=0x7f456c1449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 0x7f456c1449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456bfff640 1 -- 172.21.15.1:0/770556209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1433c0 con 0x7f456c15e6d0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.915+0000 7f456affd640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 0x7f456c1449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f456bfff640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f456400af60 tx=0x7f4564004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564038040 con 0x7f456c15e6d0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4564043070 con 0x7f456c15e6d0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1436a0 con 0x7f456c15e6d0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564008d50 con 0x7f456c15e6d0 2024-05-09T06:38:09.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.916+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c143c60 con 0x7f456c15e6d0 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.917+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c0752b0 con 0x7f456c15e6d0 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.917+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4564048460 con 0x7f456c15e6d0 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.918+0000 7f4553fff640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 0x7f455806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.918+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f45640ab9e0 con 0x7f456c15e6d0 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.918+0000 7f456b7fe640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 0x7f455806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.918+0000 7f456b7fe640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 0x7f455806e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:09.922+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f456407a140 con 0x7f456c15e6d0 2024-05-09T06:38:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.142+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f456c15af10 con 0x7f456c15e6d0 2024-05-09T06:38:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.142+0000 7f4553fff640 1 -- 172.21.15.1:0/770556209 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5158 (secure 0 0 0) 0x7f4564079a40 con 0x7f456c15e6d0 2024-05-09T06:38:10.123 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:10.123 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9441,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1642527228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1642527228},{"type":"v1","addr":"172.21.15.1:6835","nonce":1642527228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:38:07.006701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:38:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 89 2024-05-09T06:38:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 msgr2=0x7f455806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 0x7f455806e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f455c007fd0 tx=0x7f455c009040 comp rx=0 tx=0).stop 2024-05-09T06:38:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 msgr2=0x7f456c144ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f456400af60 tx=0x7f4564004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 shutdown_connections 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f455806be30 0x7f455806e2f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f456c15e6d0 0x7f456c144ee0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f456c15db10 0x7f456c1449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 --2- 172.21.15.1:0/770556209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f456c0b2a60 0x7f456c144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.144+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 >> 172.21.15.1:0/770556209 conn(0x7f456c0cc4b0 msgr2=0x7f456c15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.145+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 shutdown_connections 2024-05-09T06:38:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.145+0000 7f4571fc8640 1 -- 172.21.15.1:0/770556209 wait complete. 2024-05-09T06:38:10.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.260+0000 7f77b749e640 1 Processor -- start 2024-05-09T06:38:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.260+0000 7f77b749e640 1 -- start start 2024-05-09T06:38:10.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b749e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 0x7f77b00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b749e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77b00d4940 0x7f77b00d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b749e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b00b33c0 con 0x7f77b00d0f80 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b749e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f77b00d4dd0 con 0x7f77b00d4940 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b749e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77b00b3660 con 0x7f77b0161950 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77b00d4940 0x7f77b00d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77b00d4940 0x7f77b00d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37586/0 (socket says 172.21.15.1:37586) 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b5a3b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f77b00d0f80 0x7f77b00cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47478/0 (socket says 172.21.15.1:47478) 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 -- 172.21.15.1:0/2017941808 learned_addr learned my addr 172.21.15.1:0/2017941808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b523a640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 0x7f77b00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 -- 172.21.15.1:0/2017941808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 msgr2=0x7f77b00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 0x7f77b00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 -- 172.21.15.1:0/2017941808 >> v1:172.21.15.1:6789/0 conn(0x7f77b00d0f80 legacy=0x7f77b00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b4a39640 1 -- 172.21.15.1:0/2017941808 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b00b3940 con 0x7f77b00d4940 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.261+0000 7f77b523a640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 0x7f77b00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:10.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.262+0000 7f77b4a39640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b00d4940 0x7f77b00d2940 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f77b00b3590 tx=0x7f77a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc480babeadfac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.262+0000 7f77a7fff640 1 -- 172.21.15.1:0/2017941808 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037070 con 0x7f77b00d4940 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.262+0000 7f77a7fff640 1 -- 172.21.15.1:0/2017941808 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a0032440 con 0x7f77b00d4940 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.262+0000 7f77a7fff640 1 -- 172.21.15.1:0/2017941808 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032810 con 0x7f77b00d4940 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 -- 172.21.15.1:0/2017941808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b00d4940 msgr2=0x7f77b00d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b00d4940 0x7f77b00d2940 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f77b00b3590 tx=0x7f77a002a250 comp rx=0 tx=0).stop 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 -- 172.21.15.1:0/2017941808 shutdown_connections 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b00d4940 0x7f77b00d2940 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 --2- 172.21.15.1:0/2017941808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b0161950 0x7f77b00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 -- 172.21.15.1:0/2017941808 >> 172.21.15.1:0/2017941808 conn(0x7f77b00d3e20 msgr2=0x7f77b00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 -- 172.21.15.1:0/2017941808 shutdown_connections 2024-05-09T06:38:10.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.263+0000 7f77b749e640 1 -- 172.21.15.1:0/2017941808 wait complete. 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.264+0000 7f77b749e640 1 Processor -- start 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.264+0000 7f77b749e640 1 -- start start 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77b00d0f80 0x7f77b0155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b006c8d0 0x7f77b006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77b00b52d0 con 0x7f77b0161950 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77b00b5150 con 0x7f77b00d0f80 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77b00b5450 con 0x7f77b006c8d0 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37588/0 (socket says 172.21.15.1:37588) 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 -- 172.21.15.1:0/3635709867 learned_addr learned my addr 172.21.15.1:0/3635709867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b523a640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77b00d0f80 0x7f77b0155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b5a3b640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b006c8d0 0x7f77b006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 -- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b006c8d0 msgr2=0x7f77b006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b006c8d0 0x7f77b006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 -- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77b00d0f80 msgr2=0x7f77b0155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77b00d0f80 0x7f77b0155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 -- 172.21.15.1:0/3635709867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b00cce50 con 0x7f77b0161950 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b4a39640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f77a002a290 tx=0x7f77a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037040 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a0042070 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b00b6a10 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.265+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0030bb0 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.266+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b015cb50 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.267+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b00752b0 con 0x7f77b0161950 2024-05-09T06:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.267+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77a0047460 con 0x7f77b0161950 2024-05-09T06:38:10.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.267+0000 7f77a5ffb640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 0x7f779406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.268+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f77a00abce0 con 0x7f77b0161950 2024-05-09T06:38:10.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.268+0000 7f77b523a640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 0x7f779406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.268+0000 7f77b523a640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 0x7f779406e320 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00a950 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.272+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77b00cce50 con 0x7f77b0161950 2024-05-09T06:38:10.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.493+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b00d29a0 con 0x7f77b0161950 2024-05-09T06:38:10.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.493+0000 7f77a5ffb640 1 -- 172.21.15.1:0/3635709867 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5158 (secure 0 0 0) 0x7f77a0045360 con 0x7f77b0161950 2024-05-09T06:38:10.474 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:10.474 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9441,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1642527228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1642527228},{"type":"v1","addr":"172.21.15.1:6835","nonce":1642527228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:35.082530+0000","modified":"2024-05-09T06:38:07.006701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-09T06:38:10.474 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 89 2024-05-09T06:38:10.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.495+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 msgr2=0x7f779406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 0x7f779406e320 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00a950 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2024-05-09T06:38:10.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 msgr2=0x7f77b00cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f77a002a290 tx=0x7f77a00046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 shutdown_connections 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be60 0x7f779406e320 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77b006c8d0 0x7f77b006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77b0161950 0x7f77b00cb8b0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 --2- 172.21.15.1:0/3635709867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77b00d0f80 0x7f77b0155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 >> 172.21.15.1:0/3635709867 conn(0x7f77b00d3e20 msgr2=0x7f77b00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 shutdown_connections 2024-05-09T06:38:10.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.496+0000 7f77b749e640 1 -- 172.21.15.1:0/3635709867 wait complete. 2024-05-09T06:38:10.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.610+0000 7ffb3d664640 1 Processor -- start 2024-05-09T06:38:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.610+0000 7ffb3d664640 1 -- start start 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb3d664640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 0x7ffb380b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb3d664640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb380d2b10 0x7ffb380b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb3d664640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb380b33c0 con 0x7ffb38161950 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb3d664640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffb380d4dd0 con 0x7ffb380d2b10 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb3d664640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ffb380b3660 con 0x7ffb380d4940 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb377fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffb380d2b10 0x7ffb380b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb36ffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ffb38161950 0x7ffb380d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47492/0 (socket says 172.21.15.1:47492) 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.611+0000 7ffb36ffd640 1 -- 172.21.15.1:0/4170753760 learned_addr learned my addr 172.21.15.1:0/4170753760 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb367fc640 1 --2- 172.21.15.1:0/4170753760 >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 0x7ffb380b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb377fe640 1 -- 172.21.15.1:0/4170753760 >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 msgr2=0x7ffb380b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb377fe640 1 --2- 172.21.15.1:0/4170753760 >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 0x7ffb380b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb377fe640 1 -- 172.21.15.1:0/4170753760 >> v1:172.21.15.1:6789/0 conn(0x7ffb38161950 legacy=0x7ffb380d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb377fe640 1 -- 172.21.15.1:0/4170753760 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb380b3940 con 0x7ffb380d2b10 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb35ffb640 1 -- 172.21.15.1:0/4170753760 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323959126 0 0) 0x7ffb380b33c0 con 0x7ffb38161950 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb367fc640 1 --2- 172.21.15.1:0/4170753760 >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 0x7ffb380b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.612+0000 7ffb377fe640 1 --2- 172.21.15.1:0/4170753760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb380d2b10 0x7ffb380b4680 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ffb3806e3d0 tx=0x7ffb3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53263b189975c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb35ffb640 1 -- 172.21.15.1:0/4170753760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30037070 con 0x7ffb380d2b10 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb35ffb640 1 -- 172.21.15.1:0/4170753760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb30032440 con 0x7ffb380d2b10 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb35ffb640 1 -- 172.21.15.1:0/4170753760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30032810 con 0x7ffb380d2b10 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 -- 172.21.15.1:0/4170753760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb380d2b10 msgr2=0x7ffb380b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 --2- 172.21.15.1:0/4170753760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb380d2b10 0x7ffb380b4680 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ffb3806e3d0 tx=0x7ffb3002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 -- 172.21.15.1:0/4170753760 shutdown_connections 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 --2- 172.21.15.1:0/4170753760 >> v2:172.21.15.6:3300/0 conn(0x7ffb380d4940 0x7ffb380b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 --2- 172.21.15.1:0/4170753760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb380d2b10 0x7ffb380b4680 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 -- 172.21.15.1:0/4170753760 >> 172.21.15.1:0/4170753760 conn(0x7ffb380d3e20 msgr2=0x7ffb380d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 -- 172.21.15.1:0/4170753760 shutdown_connections 2024-05-09T06:38:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.613+0000 7ffb3d664640 1 -- 172.21.15.1:0/4170753760 wait complete. 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 Processor -- start 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 -- start start 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb380d2b10 0x7ffb3814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 0x7ffb38150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb380d1100 con 0x7ffb38161950 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffb380d0f80 con 0x7ffb380cef70 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb3d664640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ffb380d1c00 con 0x7ffb380d2b10 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37608/0 (socket says 172.21.15.1:37608) 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.615+0000 7ffb367fc640 1 -- 172.21.15.1:0/3534713649 learned_addr learned my addr 172.21.15.1:0/3534713649 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb377fe640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 0x7ffb38150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb36ffd640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb380d2b10 0x7ffb3814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 -- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb380d2b10 msgr2=0x7ffb3814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb380d2b10 0x7ffb3814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 -- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 msgr2=0x7ffb38150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 0x7ffb38150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 -- 172.21.15.1:0/3534713649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb381510b0 con 0x7ffb38161950 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb377fe640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 0x7ffb38150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb367fc640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c7f0 tx=0x7ffb2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c015070 con 0x7ffb38161950 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c007a30 con 0x7ffb38161950 2024-05-09T06:38:10.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3814e490 con 0x7ffb38161950 2024-05-09T06:38:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c007e00 con 0x7ffb38161950 2024-05-09T06:38:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.616+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3814ea50 con 0x7ffb38161950 2024-05-09T06:38:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.617+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb380cf380 con 0x7ffb38161950 2024-05-09T06:38:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.618+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb2c018070 con 0x7ffb38161950 2024-05-09T06:38:10.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.618+0000 7ffb1f7fe640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 0x7ffb2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.618+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7ffb2c08b0a0 con 0x7ffb38161950 2024-05-09T06:38:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.618+0000 7ffb36ffd640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 0x7ffb2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.619+0000 7ffb36ffd640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 0x7ffb2406e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb280012e0 tx=0x7ffb28005eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.622+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb380cf380 con 0x7ffb38161950 2024-05-09T06:38:10.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.810+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb3814e280 con 0x7ffb38161950 2024-05-09T06:38:10.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.811+0000 7ffb1f7fe640 1 -- 172.21.15.1:0/3534713649 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+17055 (secure 0 0 0) 0x7ffb2c059800 con 0x7ffb38161950 2024-05-09T06:38:10.799 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:10.800 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":110,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:07.006528+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:33.397054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:34.058197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6835/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6834/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6835/3796896332":"2024-05-10T06:37:30.837395+0000","172.21.15.1:6834/3796896332":"2024-05-10T06:37:30.837395+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 msgr2=0x7ffb2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 0x7ffb2406e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb280012e0 tx=0x7ffb28005eb0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 msgr2=0x7ffb380cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c7f0 tx=0x7ffb2c00cbf0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 shutdown_connections 2024-05-09T06:38:10.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ffb2406be30 0x7ffb2406e2f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb380cef70 0x7ffb38150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb38161950 0x7ffb380cea30 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 --2- 172.21.15.1:0/3534713649 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffb380d2b10 0x7ffb3814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 >> 172.21.15.1:0/3534713649 conn(0x7ffb380d3e20 msgr2=0x7ffb380b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 shutdown_connections 2024-05-09T06:38:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.813+0000 7ffb3d664640 1 -- 172.21.15.1:0/3534713649 wait complete. 2024-05-09T06:38:10.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:38:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.943+0000 7f579cda9640 1 Processor -- start 2024-05-09T06:38:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.943+0000 7f579cda9640 1 -- start start 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.944+0000 7f579cda9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5798161950 0x7f57980d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.944+0000 7f579cda9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 0x7f57980d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.944+0000 7f579cda9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57980b33c0 con 0x7f57980d4940 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.944+0000 7f579cda9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f57980d4dd0 con 0x7f5798161950 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.944+0000 7f579cda9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f57980b3660 con 0x7f57980d0f80 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5798161950 0x7f57980d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5798161950 0x7f57980d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37624/0 (socket says 172.21.15.1:37624) 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 -- 172.21.15.1:0/3025026450 learned_addr learned my addr 172.21.15.1:0/3025026450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f5796d9d640 1 --2- 172.21.15.1:0/3025026450 >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 0x7f57980d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 -- 172.21.15.1:0/3025026450 >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 msgr2=0x7f57980d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 --2- 172.21.15.1:0/3025026450 >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 0x7f57980d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 -- 172.21.15.1:0/3025026450 >> v1:172.21.15.6:6789/0 conn(0x7f57980d4940 legacy=0x7f57980cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f579659c640 1 -- 172.21.15.1:0/3025026450 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57980b3940 con 0x7f5798161950 2024-05-09T06:38:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.945+0000 7f5796d9d640 1 --2- 172.21.15.1:0/3025026450 >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 0x7f57980d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579659c640 1 --2- 172.21.15.1:0/3025026450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5798161950 0x7f57980d2400 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f579806b7f0 tx=0x7f579002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeed4dffff865337 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579559a640 1 -- 172.21.15.1:0/3025026450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790036070 con 0x7f5798161950 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579559a640 1 -- 172.21.15.1:0/3025026450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790031600 con 0x7f5798161950 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579559a640 1 -- 172.21.15.1:0/3025026450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900319d0 con 0x7f5798161950 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579cda9640 1 -- 172.21.15.1:0/3025026450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5798161950 msgr2=0x7f57980d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579cda9640 1 --2- 172.21.15.1:0/3025026450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5798161950 0x7f57980d2400 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f579806b7f0 tx=0x7f579002a6c0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.946+0000 7f579cda9640 1 -- 172.21.15.1:0/3025026450 shutdown_connections 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.947+0000 7f579cda9640 1 --2- 172.21.15.1:0/3025026450 >> v2:172.21.15.6:3300/0 conn(0x7f57980d0f80 0x7f57980d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.947+0000 7f579cda9640 1 --2- 172.21.15.1:0/3025026450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5798161950 0x7f57980d2400 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.947+0000 7f579cda9640 1 -- 172.21.15.1:0/3025026450 >> 172.21.15.1:0/3025026450 conn(0x7f57980d3e20 msgr2=0x7f57980d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.947+0000 7f579cda9640 1 -- 172.21.15.1:0/3025026450 shutdown_connections 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.947+0000 7f579cda9640 1 -- 172.21.15.1:0/3025026450 wait complete. 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.948+0000 7f579cda9640 1 Processor -- start 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.948+0000 7f579cda9640 1 -- start start 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5798161950 0x7f579809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f579815de20 0x7f57980cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57980b52d0 con 0x7f57980d0f80 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f57980b5150 con 0x7f5798161950 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579cda9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f57980b5450 con 0x7f579815de20 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37632/0 (socket says 172.21.15.1:37632) 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 -- 172.21.15.1:0/668001460 learned_addr learned my addr 172.21.15.1:0/668001460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 -- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f579815de20 msgr2=0x7f57980cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f579815de20 0x7f57980cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 -- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5798161950 msgr2=0x7f579809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5798161950 0x7f579809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 -- 172.21.15.1:0/668001460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57980a07d0 con 0x7f57980d0f80 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f579659c640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5790034040 tx=0x7f5790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790036050 con 0x7f57980d0f80 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790042070 con 0x7f57980d0f80 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.949+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790009510 con 0x7f57980d0f80 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.950+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57980a0ab0 con 0x7f57980d0f80 2024-05-09T06:38:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.950+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57980a1070 con 0x7f57980d0f80 2024-05-09T06:38:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.951+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57980ca7d0 con 0x7f57980d0f80 2024-05-09T06:38:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.951+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5790007c50 con 0x7f57980d0f80 2024-05-09T06:38:10.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.952+0000 7f578effd640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 0x7f577806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:10.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.952+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f57900ac0a0 con 0x7f57980d0f80 2024-05-09T06:38:10.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.952+0000 7f5795d9b640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 0x7f577806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:10.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.952+0000 7f5795d9b640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 0x7f577806e2f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f579806d310 tx=0x7f57800079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:10.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:10.956+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57980ca7d0 con 0x7f57980d0f80 2024-05-09T06:38:11.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.179+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f579815ec10 con 0x7f57980d0f80 2024-05-09T06:38:11.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.658+0000 7f578effd640 1 -- 172.21.15.1:0/668001460 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f579815ec10 con 0x7f57980d0f80 2024-05-09T06:38:11.638 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 msgr2=0x7f577806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 0x7f577806e2f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f579806d310 tx=0x7f57800079b0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 msgr2=0x7f5798151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5790034040 tx=0x7f5790004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 shutdown_connections 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f577806be30 0x7f577806e2f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f579815de20 0x7f57980cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5798161950 0x7f579809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 --2- 172.21.15.1:0/668001460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57980d0f80 0x7f5798151390 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 >> 172.21.15.1:0/668001460 conn(0x7f57980d3e20 msgr2=0x7f579806d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 shutdown_connections 2024-05-09T06:38:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.660+0000 7f579cda9640 1 -- 172.21.15.1:0/668001460 wait complete. 2024-05-09T06:38:11.654 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:38:11.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 Processor -- start 2024-05-09T06:38:11.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 -- start start 2024-05-09T06:38:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d5c15e190 0x7f6d5c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c161fe0 con 0x7f6d5c0b56d0 2024-05-09T06:38:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c15f580 con 0x7f6d5c15e190 2024-05-09T06:38:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.786+0000 7f6d620d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c1622c0 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d5c15e190 0x7f6d5c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6d5c15e190 0x7f6d5c15f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37642/0 (socket says 172.21.15.1:37642) 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 -- 172.21.15.1:0/3608306833 learned_addr learned my addr 172.21.15.1:0/3608306833 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5affd640 1 --2- 172.21.15.1:0/3608306833 >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 -- 172.21.15.1:0/3608306833 >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 msgr2=0x7f6d5c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 --2- 172.21.15.1:0/3608306833 >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 -- 172.21.15.1:0/3608306833 >> v1:172.21.15.6:6789/0 conn(0x7f6d5c0b56d0 legacy=0x7f6d5c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5b7fe640 1 -- 172.21.15.1:0/3608306833 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c1625a0 con 0x7f6d5c15e190 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.787+0000 7f6d5affd640 1 --2- 172.21.15.1:0/3608306833 >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d5b7fe640 1 --2- 172.21.15.1:0/3608306833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c15e190 0x7f6d5c15f780 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0b6c30 tx=0x7f6d4c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b90db39f8106778 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d5a7fc640 1 -- 172.21.15.1:0/3608306833 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c037070 con 0x7f6d5c15e190 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d5a7fc640 1 -- 172.21.15.1:0/3608306833 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c032440 con 0x7f6d5c15e190 2024-05-09T06:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d5a7fc640 1 -- 172.21.15.1:0/3608306833 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c032810 con 0x7f6d5c15e190 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 -- 172.21.15.1:0/3608306833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c15e190 msgr2=0x7f6d5c15f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3608306833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c15e190 0x7f6d5c15f780 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0b6c30 tx=0x7f6d4c02a230 comp rx=0 tx=0).stop 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 -- 172.21.15.1:0/3608306833 shutdown_connections 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3608306833 >> v2:172.21.15.6:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3608306833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c15e190 0x7f6d5c15f780 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 -- 172.21.15.1:0/3608306833 >> 172.21.15.1:0/3608306833 conn(0x7f6d5c073340 msgr2=0x7f6d5c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 -- 172.21.15.1:0/3608306833 shutdown_connections 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.788+0000 7f6d620d4640 1 -- 172.21.15.1:0/3608306833 wait complete. 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.790+0000 7f6d620d4640 1 Processor -- start 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.790+0000 7f6d620d4640 1 -- start start 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.790+0000 7f6d620d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d5c0b56d0 0x7f6d5c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.790+0000 7f6d620d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d5c15e190 0x7f6d5c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d4420 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c161c60 con 0x7f6d5c15e190 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d45a0 con 0x7f6d5c0b56d0 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37654/0 (socket says 172.21.15.1:37654) 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 -- 172.21.15.1:0/3851351128 learned_addr learned my addr 172.21.15.1:0/3851351128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5bfff640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d5c15e190 0x7f6d5c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5b7fe640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d5c0b56d0 0x7f6d5c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 -- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d5c0b56d0 msgr2=0x7f6d5c0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d5c0b56d0 0x7f6d5c0cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 -- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d5c15e190 msgr2=0x7f6d5c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d5c15e190 0x7f6d5c0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 -- 172.21.15.1:0/3851351128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0c8190 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d5affd640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c7f0 tx=0x7f6d4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44004230 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d440043d0 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44007a70 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c0c8470 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.791+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c0cbc00 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.792+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c0752b0 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.793+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d44007c10 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.793+0000 7f6d53fff640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 0x7f6d3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.793+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7f6d4408b180 con 0x7f6d5c0d1c70 2024-05-09T06:38:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.794+0000 7f6d5b7fe640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 0x7f6d3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.794+0000 7f6d5b7fe640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 0x7f6d3c06e2f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c034000 tx=0x7f6d4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:11.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:11.798+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d44015020 con 0x7f6d5c0d1c70 2024-05-09T06:38:12.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.020+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6d5c08e1c0 con 0x7f6d5c0d1c70 2024-05-09T06:38:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.659+0000 7f6d53fff640 1 -- 172.21.15.1:0/3851351128 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7f6d440591f0 con 0x7f6d5c0d1c70 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 msgr2=0x7f6d3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 0x7f6d3c06e2f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c034000 tx=0x7f6d4c02e000 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 msgr2=0x7f6d5c152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c7f0 tx=0x7f6d4400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 shutdown_connections 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6d3c06be30 0x7f6d3c06e2f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6d5c15e190 0x7f6d5c0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c152420 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 --2- 172.21.15.1:0/3851351128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6d5c0b56d0 0x7f6d5c0cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 >> 172.21.15.1:0/3851351128 conn(0x7f6d5c073340 msgr2=0x7f6d5c0d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 shutdown_connections 2024-05-09T06:38:12.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.662+0000 7f6d620d4640 1 -- 172.21.15.1:0/3851351128 wait complete. 2024-05-09T06:38:12.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.790+0000 7fa4fa59d640 1 Processor -- start 2024-05-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.790+0000 7fa4fa59d640 1 -- start start 2024-05-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4fa59d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa4f40d2630 0x7fa4f415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4fa59d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa4f40d2a10 0x7fa4f40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4fa59d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40d1cc0 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4fa59d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f40d4d10 con 0x7fa4f40d2630 2024-05-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4fa59d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f40d1f80 con 0x7fa4f40d2a10 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.791+0000 7fa4f3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa4f40d2630 0x7fa4f415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa4f40d2630 0x7fa4f415ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37660/0 (socket says 172.21.15.1:37660) 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 -- 172.21.15.1:0/95214022 learned_addr learned my addr 172.21.15.1:0/95214022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f8b3a640 1 --2- 172.21.15.1:0/95214022 >> v2:172.21.15.6:3300/0 conn(0x7fa4f40d2a10 0x7fa4f40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 -- 172.21.15.1:0/95214022 >> v2:172.21.15.6:3300/0 conn(0x7fa4f40d2a10 msgr2=0x7fa4f40cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/95214022 >> v2:172.21.15.6:3300/0 conn(0x7fa4f40d2a10 0x7fa4f40cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 -- 172.21.15.1:0/95214022 >> v1:172.21.15.6:6789/0 conn(0x7fa4f40b2ed0 legacy=0x7fa4f40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 -- 172.21.15.1:0/95214022 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40b0c00 con 0x7fa4f40d2630 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.792+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/95214022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40d2630 0x7fa4f415ddf0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fa4f408e0e0 tx=0x7fa4e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f727fa18e6ae7de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4f2ffd640 1 -- 172.21.15.1:0/95214022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4037070 con 0x7fa4f40d2630 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4f2ffd640 1 -- 172.21.15.1:0/95214022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4032440 con 0x7fa4f40d2630 2024-05-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4f2ffd640 1 -- 172.21.15.1:0/95214022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4032810 con 0x7fa4f40d2630 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 -- 172.21.15.1:0/95214022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40d2630 msgr2=0x7fa4f415ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/95214022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40d2630 0x7fa4f415ddf0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fa4f408e0e0 tx=0x7fa4e402a230 comp rx=0 tx=0).stop 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 -- 172.21.15.1:0/95214022 shutdown_connections 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/95214022 >> v2:172.21.15.6:3300/0 conn(0x7fa4f40d2a10 0x7fa4f40cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/95214022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40d2630 0x7fa4f415ddf0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 -- 172.21.15.1:0/95214022 >> 172.21.15.1:0/95214022 conn(0x7fa4f40d3ee0 msgr2=0x7fa4f40d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 -- 172.21.15.1:0/95214022 shutdown_connections 2024-05-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.793+0000 7fa4fa59d640 1 -- 172.21.15.1:0/95214022 wait complete. 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.794+0000 7fa4fa59d640 1 Processor -- start 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 -- start start 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4f40d2630 0x7fa4f40b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4f40d2a10 0x7fa4f40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f415e4b0 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f415e330 con 0x7fa4f40d2a10 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.795+0000 7fa4fa59d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f415e630 con 0x7fa4f40d2630 2024-05-09T06:38:12.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37664/0 (socket says 172.21.15.1:37664) 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 -- 172.21.15.1:0/3549321204 learned_addr learned my addr 172.21.15.1:0/3549321204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f8b3a640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4f40d2a10 0x7fa4f40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f37fe640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4f40d2630 0x7fa4f40b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 -- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4f40d2630 msgr2=0x7fa4f40b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4f40d2630 0x7fa4f40b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 -- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4f40d2a10 msgr2=0x7fa4f40b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4f40d2a10 0x7fa4f40b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 -- 172.21.15.1:0/3549321204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f409e6c0 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f3fff640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fa4e402a270 tx=0x7fa4e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4041020 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e40306c0 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4037040 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f409e9a0 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.796+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f409ef60 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.797+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4075320 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.798+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4e4044020 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.798+0000 7fa4f0ff9640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.798+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6302+0+0 (secure 0 0 0) 0x7fa4e40ab830 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.798+0000 7fa4f37fe640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.799+0000 7fa4f37fe640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 0x7fa4e006e2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40b33f0 tx=0x7fa4e8007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.803+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4e404b030 con 0x7fa4f40b2ed0 2024-05-09T06:38:12.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:12.985+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa4f415d7b0 con 0x7fa4f40b2ed0 2024-05-09T06:38:13.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.668+0000 7fa4f0ff9640 1 -- 172.21.15.1:0/3549321204 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7fa4e40798c0 con 0x7fa4f40b2ed0 2024-05-09T06:38:13.649 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:38:13.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.672+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 msgr2=0x7fa4e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.672+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 0x7fa4e006e2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40b33f0 tx=0x7fa4e8007e30 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.672+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 msgr2=0x7fa4f415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.672+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fa4e402a270 tx=0x7fa4e40046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.672+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 shutdown_connections 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4f40d2a10 0x7fa4f40b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa4f40d2630 0x7fa4f40b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 --2- 172.21.15.1:0/3549321204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f40b2ed0 0x7fa4f415c3e0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 >> 172.21.15.1:0/3549321204 conn(0x7fa4f40d3ee0 msgr2=0x7fa4f415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 shutdown_connections 2024-05-09T06:38:13.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.673+0000 7fa4fa59d640 1 -- 172.21.15.1:0/3549321204 wait complete. 2024-05-09T06:38:13.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:38:13.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.801+0000 7fc03677e640 1 Processor -- start 2024-05-09T06:38:13.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.801+0000 7fc03677e640 1 -- start start 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc03677e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc03677e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0301628f0 0x7fc0300ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc03677e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0300cb0e0 con 0x7fc03015e5f0 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc03677e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc0300cca50 con 0x7fc0301628f0 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc03677e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc0300b17c0 con 0x7fc03015e210 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0301628f0 0x7fc0300ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0301628f0 0x7fc0300ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37680/0 (socket says 172.21.15.1:37680) 2024-05-09T06:38:13.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 -- 172.21.15.1:0/1949138844 learned_addr learned my addr 172.21.15.1:0/1949138844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:13.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 -- 172.21.15.1:0/1949138844 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 msgr2=0x7fc0300b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:13.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 --2- 172.21.15.1:0/1949138844 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 -- 172.21.15.1:0/1949138844 >> v1:172.21.15.6:6789/0 conn(0x7fc03015e5f0 legacy=0x7fc0300ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:13.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.802+0000 7fc034d1b640 1 -- 172.21.15.1:0/1949138844 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0300b1aa0 con 0x7fc0301628f0 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.803+0000 7fc034d1b640 1 --2- 172.21.15.1:0/1949138844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0301628f0 0x7fc0300ccee0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc02802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4378e5045e248af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.803+0000 7fc02effd640 1 -- 172.21.15.1:0/1949138844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028036070 con 0x7fc0301628f0 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.803+0000 7fc02effd640 1 -- 172.21.15.1:0/1949138844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028031600 con 0x7fc0301628f0 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.803+0000 7fc02effd640 1 -- 172.21.15.1:0/1949138844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280319d0 con 0x7fc0301628f0 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 -- 172.21.15.1:0/1949138844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0301628f0 msgr2=0x7fc0300ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 --2- 172.21.15.1:0/1949138844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0301628f0 0x7fc0300ccee0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc02802a6c0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 -- 172.21.15.1:0/1949138844 shutdown_connections 2024-05-09T06:38:13.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 --2- 172.21.15.1:0/1949138844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0301628f0 0x7fc0300ccee0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 --2- 172.21.15.1:0/1949138844 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 -- 172.21.15.1:0/1949138844 >> 172.21.15.1:0/1949138844 conn(0x7fc030071a50 msgr2=0x7fc0300b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:13.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 -- 172.21.15.1:0/1949138844 shutdown_connections 2024-05-09T06:38:13.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.804+0000 7fc03677e640 1 -- 172.21.15.1:0/1949138844 wait complete. 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.805+0000 7fc03677e640 1 Processor -- start 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.805+0000 7fc03677e640 1 -- start start 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 0x7fc030152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0300cd420 con 0x7fc030160bb0 2024-05-09T06:38:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc030162d70 con 0x7fc0301628f0 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc03677e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc0300cd5a0 con 0x7fc03015e210 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc034d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc034d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37684/0 (socket says 172.21.15.1:37684) 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc034d1b640 1 -- 172.21.15.1:0/1385127174 learned_addr learned my addr 172.21.15.1:0/1385127174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc02f7fe640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 0x7fc030152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.806+0000 7fc02ffff640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 -- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 msgr2=0x7fc0300c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 -- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 msgr2=0x7fc030152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 0x7fc030152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 -- 172.21.15.1:0/1385127174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0300c82d0 con 0x7fc030160bb0 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc02f7fe640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 0x7fc030152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc02ffff640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc034d1b640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028040030 con 0x7fc030160bb0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028031400 con 0x7fc030160bb0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0300c8d10 con 0x7fc030160bb0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028036040 con 0x7fc030160bb0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.807+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc030152560 con 0x7fc030160bb0 2024-05-09T06:38:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.808+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030162700 con 0x7fc030160bb0 2024-05-09T06:38:13.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.808+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc028043030 con 0x7fc030160bb0 2024-05-09T06:38:13.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.809+0000 7fc02cff9640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.809+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5864+0+0 (secure 0 0 0) 0x7fc0280ab5b0 con 0x7fc030160bb0 2024-05-09T06:38:13.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.809+0000 7fc02ffff640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.809+0000 7fc02ffff640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc020009930 tx=0x7fc020008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.813+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc028079ec0 con 0x7fc030160bb0 2024-05-09T06:38:13.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:13.998+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc030075320 con 0x7fc030160bb0 2024-05-09T06:38:14.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.670+0000 7fc02cff9640 1 -- 172.21.15.1:0/1385127174 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7fc0280797c0 con 0x7fc030160bb0 2024-05-09T06:38:14.650 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 msgr2=0x7fc01c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc020009930 tx=0x7fc020008040 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 msgr2=0x7fc030161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc028004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 shutdown_connections 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc030160bb0 0x7fc030161040 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0301628f0 0x7fc030152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.673+0000 7fc03677e640 1 --2- 172.21.15.1:0/1385127174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc03015e210 0x7fc0300c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.674+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 >> 172.21.15.1:0/1385127174 conn(0x7fc030071a50 msgr2=0x7fc03006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.674+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 shutdown_connections 2024-05-09T06:38:14.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.674+0000 7fc03677e640 1 -- 172.21.15.1:0/1385127174 wait complete. 2024-05-09T06:38:14.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:38:14.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.802+0000 7fb253fa5640 1 Processor -- start 2024-05-09T06:38:14.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.802+0000 7fb253fa5640 1 -- start start 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb253fa5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb24c15a4d0 0x7fb24c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb253fa5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c071610 0x7fb24c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb253fa5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c0cf1b0 con 0x7fb24c0cdb90 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb253fa5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c15a8b0 con 0x7fb24c15a4d0 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb253fa5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c06dc10 con 0x7fb24c071610 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb24c15a4d0 0x7fb24c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb24c15a4d0 0x7fb24c0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37700/0 (socket says 172.21.15.1:37700) 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 -- 172.21.15.1:0/1950543761 learned_addr learned my addr 172.21.15.1:0/1950543761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb252542640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c071610 0x7fb24c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 -- 172.21.15.1:0/1950543761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c071610 msgr2=0x7fb24c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c071610 0x7fb24c06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.803+0000 7fb251d41640 1 -- 172.21.15.1:0/1950543761 >> v1:172.21.15.6:6789/0 conn(0x7fb24c0cdb90 legacy=0x7fb24c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.804+0000 7fb251d41640 1 -- 172.21.15.1:0/1950543761 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c0cf490 con 0x7fb24c15a4d0 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.804+0000 7fb251d41640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c0cea80 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bb10 tx=0x7fb24402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e2287eec1a137b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.804+0000 7fb250d3f640 1 -- 172.21.15.1:0/1950543761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244032790 con 0x7fb24c15a4d0 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.804+0000 7fb250d3f640 1 -- 172.21.15.1:0/1950543761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb244038070 con 0x7fb24c15a4d0 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.804+0000 7fb250d3f640 1 -- 172.21.15.1:0/1950543761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244032b50 con 0x7fb24c15a4d0 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 -- 172.21.15.1:0/1950543761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 msgr2=0x7fb24c0cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c0cea80 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bb10 tx=0x7fb24402c8d0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 -- 172.21.15.1:0/1950543761 shutdown_connections 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c071610 0x7fb24c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 --2- 172.21.15.1:0/1950543761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c0cea80 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 -- 172.21.15.1:0/1950543761 >> 172.21.15.1:0/1950543761 conn(0x7fb24c0cc4d0 msgr2=0x7fb24c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 -- 172.21.15.1:0/1950543761 shutdown_connections 2024-05-09T06:38:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.805+0000 7fb253fa5640 1 -- 172.21.15.1:0/1950543761 wait complete. 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.806+0000 7fb253fa5640 1 Processor -- start 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.806+0000 7fb253fa5640 1 -- start start 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb24c071610 0x7fb24c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c0cdb90 0x7fb24c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c0b2be0 con 0x7fb24c15a4d0 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c0b2a60 con 0x7fb24c071610 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb253fa5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c06cd30 con 0x7fb24c0cdb90 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb252542640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb252542640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37708/0 (socket says 172.21.15.1:37708) 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb252542640 1 -- 172.21.15.1:0/3022459489 learned_addr learned my addr 172.21.15.1:0/3022459489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb251d41640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb24c071610 0x7fb24c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.807+0000 7fb251540640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c0cdb90 0x7fb24c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 -- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c0cdb90 msgr2=0x7fb24c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c0cdb90 0x7fb24c150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 -- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb24c071610 msgr2=0x7fb24c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb24c071610 0x7fb24c1594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 -- 172.21.15.1:0/3022459489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c141420 con 0x7fb24c15a4d0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb252542640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00cd40 tx=0x7fb23c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c003b90 con 0x7fb24c15a4d0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c003d30 con 0x7fb24c15a4d0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c156a20 con 0x7fb24c15a4d0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c004240 con 0x7fb24c15a4d0 2024-05-09T06:38:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.808+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c13e460 con 0x7fb24c15a4d0 2024-05-09T06:38:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.809+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c0752b0 con 0x7fb24c15a4d0 2024-05-09T06:38:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.810+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb23c0043e0 con 0x7fb24c15a4d0 2024-05-09T06:38:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.810+0000 7fb24a7fc640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 0x7fb23806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.810+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7fb23c08b650 con 0x7fb24c15a4d0 2024-05-09T06:38:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.810+0000 7fb251d41640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 0x7fb23806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.811+0000 7fb251d41640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 0x7fb23806e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb244037000 tx=0x7fb24403e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:14.814+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb24c141420 con 0x7fb24c15a4d0 2024-05-09T06:38:14.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.002+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb24c0cd590 con 0x7fb24c15a4d0 2024-05-09T06:38:14.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.005+0000 7fb24a7fc640 1 -- 172.21.15.1:0/3022459489 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fb23c059b60 con 0x7fb24c15a4d0 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 msgr2=0x7fb23806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 0x7fb23806e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb244037000 tx=0x7fb24403e040 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 msgr2=0x7fb24c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00cd40 tx=0x7fb23c00a660 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 shutdown_connections 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb23806be30 0x7fb23806e2f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb24c15a4d0 0x7fb24c150a60 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb24c0cdb90 0x7fb24c150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 --2- 172.21.15.1:0/3022459489 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb24c071610 0x7fb24c1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 >> 172.21.15.1:0/3022459489 conn(0x7fb24c0cc4d0 msgr2=0x7fb24c1519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 shutdown_connections 2024-05-09T06:38:14.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.007+0000 7fb253fa5640 1 -- 172.21.15.1:0/3022459489 wait complete. 2024-05-09T06:38:14.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-09T06:38:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.123+0000 7fd9e7122640 1 Processor -- start 2024-05-09T06:38:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.123+0000 7fd9e7122640 1 -- start start 2024-05-09T06:38:15.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e7122640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9e00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e7122640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e00ce0e0 con 0x7fd9e006e6d0 2024-05-09T06:38:15.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e7122640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e00ce3c0 con 0x7fd9e00cdab0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e7122640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00ce6a0 con 0x7fd9e00cf180 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e4ebe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd9e006e6d0 0x7fd9e006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47502/0 (socket says 172.21.15.1:47502) 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9e4ebe640 1 -- 172.21.15.1:0/3591880766 learned_addr learned my addr 172.21.15.1:0/3591880766 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9e00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1957998009 0 0) 0x7fd9e00ce0e0 con 0x7fd9e006e6d0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c8003520 con 0x7fd9e006e6d0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156002593 0 0) 0x7fd9e00ce3c0 con 0x7fd9e00cdab0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9e00ce0e0 con 0x7fd9e00cdab0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.124+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1735699905 0 0) 0x7fd9c8003520 con 0x7fd9e006e6d0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 msgr2=0x7fd9e00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9e00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 >> v1:172.21.15.6:6789/0 conn(0x7fd9e00cdab0 legacy=0x7fd9e00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00ce980 con 0x7fd9e006e6d0 2024-05-09T06:38:15.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fd9dc001670 con 0x7fd9e006e6d0 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 >> v1:172.21.15.1:6789/0 conn(0x7fd9e006e6d0 legacy=0x7fd9e006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8003b70 0x7fd9c8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9c8004800 0x7fd9c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e00ce6a0 con 0x7fd9c8004800 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c800baa0 con 0x7fd9c8003b70 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c800bca0 con 0x7fd9e00cf180 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9c8004800 0x7fd9c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9e4ebe640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8003b70 0x7fd9c8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9e56bf640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 -- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 msgr2=0x7fd9c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e00cf180 0x7fd9c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 -- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8003b70 msgr2=0x7fd9c8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8003b70 0x7fd9c8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.125+0000 7fd9d7fff640 1 -- 172.21.15.1:0/3591880766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c8003520 con 0x7fd9c8004800 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9c8004800 0x7fd9c8004c80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009ac0 tx=0x7fd9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad0c03bb2b7b982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0036070 con 0x7fd9c8004800 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d0031400 con 0x7fd9c8004800 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9d77fe640 1 -- 172.21.15.1:0/3591880766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0009690 con 0x7fd9c8004800 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9e7122640 1 -- 172.21.15.1:0/3591880766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9c8004800 msgr2=0x7fd9c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9e7122640 1 --2- 172.21.15.1:0/3591880766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9c8004800 0x7fd9c8004c80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009ac0 tx=0x7fd9c8005830 comp rx=0 tx=0).stop 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.126+0000 7fd9e56bf640 1 -- 172.21.15.1:0/3591880766 reap_dead start 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.127+0000 7fd9e7122640 1 -- 172.21.15.1:0/3591880766 shutdown_connections 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.127+0000 7fd9e7122640 1 -- 172.21.15.1:0/3591880766 >> 172.21.15.1:0/3591880766 conn(0x7fd9e00cc4b0 msgr2=0x7fd9e00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.127+0000 7fd9e7122640 1 -- 172.21.15.1:0/3591880766 shutdown_connections 2024-05-09T06:38:15.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.127+0000 7fd9e7122640 1 -- 172.21.15.1:0/3591880766 wait complete. 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.128+0000 7fd9e7122640 1 Processor -- start 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.128+0000 7fd9e7122640 1 -- start start 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8004800 0x7fd9e014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 0x7fd9e015e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01447c0 con 0x7fd9e013d2e0 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0070ed0 con 0x7fd9c8004800 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e0144940 con 0x7fd9e0141910 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37730/0 (socket says 172.21.15.1:37730) 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 -- 172.21.15.1:0/798757812 learned_addr learned my addr 172.21.15.1:0/798757812 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 0x7fd9e015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e4ebe640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8004800 0x7fd9e014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 -- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 msgr2=0x7fd9e015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 0x7fd9e015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 -- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8004800 msgr2=0x7fd9e014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8004800 0x7fd9e014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 -- 172.21.15.1:0/798757812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e015efe0 con 0x7fd9e013d2e0 2024-05-09T06:38:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9d7fff640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 0x7fd9e015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e56bf640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00714f0 tx=0x7fd9d800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800ee50 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d8003b80 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8003d70 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.129+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e014eee0 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.130+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e014f4a0 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.130+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e00752b0 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.131+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9d801a020 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.131+0000 7fd9d57fa640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 0x7fd9cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.132+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7fd9d808a270 con 0x7fd9e013d2e0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.132+0000 7fd9e4ebe640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 0x7fd9cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.132+0000 7fd9e4ebe640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 0x7fd9cc06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001eb0 tx=0x7fd9dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.136+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9d8021020 con 0x7fd9e013d2e0 2024-05-09T06:38:15.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.389+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fd9e006cf50 con 0x7fd9e013d2e0 2024-05-09T06:38:15.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.680+0000 7fd9d57fa640 1 -- 172.21.15.1:0/798757812 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v508) ==== 146+0+0 (secure 0 0 0) 0x7fd9d8058d40 con 0x7fd9e013d2e0 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 msgr2=0x7fd9cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 0x7fd9cc06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001eb0 tx=0x7fd9dc009040 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 msgr2=0x7fd9e013f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00714f0 tx=0x7fd9d800ec00 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 shutdown_connections 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9cc06be60 0x7fd9cc06e320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e013d2e0 0x7fd9e013f580 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e0141910 0x7fd9e015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 --2- 172.21.15.1:0/798757812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c8004800 0x7fd9e014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 >> 172.21.15.1:0/798757812 conn(0x7fd9e00cc4b0 msgr2=0x7fd9e006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 shutdown_connections 2024-05-09T06:38:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.682+0000 7fd9e7122640 1 -- 172.21.15.1:0/798757812 wait complete. 2024-05-09T06:38:15.674 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-09T06:38:15.675 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-09T06:38:15.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.800+0000 7f9feab22640 1 Processor -- start 2024-05-09T06:38:15.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.800+0000 7f9feab22640 1 -- start start 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ce4d0 0x7f9fe40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9fe415e340 0x7f9fe40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40cf330 con 0x7f9fe40ccb40 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe406d070 con 0x7f9fe415e340 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9feab22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe406d350 con 0x7f9fe40ce4d0 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9fe90bf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9fe90bf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37744/0 (socket says 172.21.15.1:37744) 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9fe90bf640 1 -- 172.21.15.1:0/269260933 learned_addr learned my addr 172.21.15.1:0/269260933 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9fe3fff640 1 --2- 172.21.15.1:0/269260933 >> v2:172.21.15.6:3300/0 conn(0x7f9fe415e340 0x7f9fe40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.801+0000 7f9fe88be640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ce4d0 0x7f9fe40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 -- 172.21.15.1:0/269260933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ce4d0 msgr2=0x7f9fe40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ce4d0 0x7f9fe40ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 -- 172.21.15.1:0/269260933 >> v2:172.21.15.6:3300/0 conn(0x7f9fe415e340 msgr2=0x7f9fe40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/269260933 >> v2:172.21.15.6:3300/0 conn(0x7f9fe415e340 0x7f9fe40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 -- 172.21.15.1:0/269260933 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40cc340 con 0x7f9fe40ccb40 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.802+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40716b0 tx=0x7f9fdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a913b015f37616 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9fe37fe640 1 -- 172.21.15.1:0/269260933 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc037070 con 0x7f9fe40ccb40 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9fe37fe640 1 -- 172.21.15.1:0/269260933 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc032440 con 0x7f9fe40ccb40 2024-05-09T06:38:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9fe37fe640 1 -- 172.21.15.1:0/269260933 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc032810 con 0x7f9fe40ccb40 2024-05-09T06:38:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 -- 172.21.15.1:0/269260933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe40ccb40 msgr2=0x7f9fe40ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40716b0 tx=0x7f9fdc02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 -- 172.21.15.1:0/269260933 shutdown_connections 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe40ccb40 0x7f9fe40ccfd0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 --2- 172.21.15.1:0/269260933 >> v2:172.21.15.6:3300/0 conn(0x7f9fe415e340 0x7f9fe40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 --2- 172.21.15.1:0/269260933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ce4d0 0x7f9fe40ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 -- 172.21.15.1:0/269260933 >> 172.21.15.1:0/269260933 conn(0x7f9fe40d1c50 msgr2=0x7f9fe40d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 -- 172.21.15.1:0/269260933 shutdown_connections 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.803+0000 7f9feab22640 1 -- 172.21.15.1:0/269260933 wait complete. 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.804+0000 7f9feab22640 1 Processor -- start 2024-05-09T06:38:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.804+0000 7f9feab22640 1 -- start start 2024-05-09T06:38:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ccb40 0x7f9fe4158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe40ce4d0 0x7f9fe40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe406cd00 con 0x7f9fe4160cb0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe406d630 con 0x7f9fe40ce4d0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.805+0000 7f9feab22640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe40cd510 con 0x7f9fe40ccb40 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe88be640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ccb40 0x7f9fe4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37748/0 (socket says 172.21.15.1:37748) 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 -- 172.21.15.1:0/2903180399 learned_addr learned my addr 172.21.15.1:0/2903180399 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe3fff640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe40ce4d0 0x7f9fe40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 -- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ccb40 msgr2=0x7f9fe4158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ccb40 0x7f9fe4158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 -- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe40ce4d0 msgr2=0x7f9fe40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe40ce4d0 0x7f9fe40cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 -- 172.21.15.1:0/2903180399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40c8f30 con 0x7f9fe4160cb0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe90bf640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0090b0 tx=0x7f9fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc037040 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc042070 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe40c9210 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc030920 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.806+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe40c97d0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.807+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe40752b0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.808+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fdc030ac0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.808+0000 7f9fe17fa640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 0x7f9fc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.809+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7f9fdc0ab470 con 0x7f9fe4160cb0 2024-05-09T06:38:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.809+0000 7f9fe88be640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 0x7f9fc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:15.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.809+0000 7f9fe88be640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 0x7f9fc406e320 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4003aa0 tx=0x7f9fd4009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:15.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:15.813+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fdc0798f0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.000+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fe40c83e0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.001+0000 7f9fe17fa640 1 -- 172.21.15.1:0/2903180399 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+13295 (secure 0 0 0) 0x7f9fdc07dbf0 con 0x7f9fe4160cb0 2024-05-09T06:38:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:15.981 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:14.669510+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6835/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6834/356239460":"2024-05-10T06:38:07.006506+0000","172.21.15.1:6835/3796896332":"2024-05-10T06:37:30.837395+0000","172.21.15.1:6834/3796896332":"2024-05-10T06:37:30.837395+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:15.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 msgr2=0x7f9fc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 0x7f9fc406e320 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4003aa0 tx=0x7f9fd4009830 comp rx=0 tx=0).stop 2024-05-09T06:38:15.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 msgr2=0x7f9fe4161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0090b0 tx=0x7f9fdc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 shutdown_connections 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9fc406be60 0x7f9fc406e320 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fe4160cb0 0x7f9fe4161160 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fe40ce4d0 0x7f9fe40cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 --2- 172.21.15.1:0/2903180399 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9fe40ccb40 0x7f9fe4158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 >> 172.21.15.1:0/2903180399 conn(0x7f9fe40d1c50 msgr2=0x7f9fe40aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 shutdown_connections 2024-05-09T06:38:15.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.003+0000 7f9feab22640 1 -- 172.21.15.1:0/2903180399 wait complete. 2024-05-09T06:38:15.995 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:15.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.124+0000 7fa153efc640 1 Processor -- start 2024-05-09T06:38:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.124+0000 7fa153efc640 1 -- start start 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa153efc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa153efc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa14c15c4a0 0x7fa14c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa153efc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c15b420 con 0x7fa14c158a50 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa153efc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa14c158e30 con 0x7fa14c15c4a0 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa153efc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa14c072480 con 0x7fa14c071fd0 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa14c15c4a0 0x7fa14c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa14c15c4a0 0x7fa14c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37762/0 (socket says 172.21.15.1:37762) 2024-05-09T06:38:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa151c98640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa14c158a50 0x7fa14c0718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47518/0 (socket says 172.21.15.1:47518) 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 -- 172.21.15.1:0/94824121 learned_addr learned my addr 172.21.15.1:0/94824121 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa151497640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 -- 172.21.15.1:0/94824121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 msgr2=0x7fa14c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 -- 172.21.15.1:0/94824121 >> v1:172.21.15.1:6789/0 conn(0x7fa14c158a50 legacy=0x7fa14c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa150c96640 1 -- 172.21.15.1:0/94824121 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165708464 0 0) 0x7fa14c15b420 con 0x7fa14c158a50 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa152499640 1 -- 172.21.15.1:0/94824121 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c15b6e0 con 0x7fa14c15c4a0 2024-05-09T06:38:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.125+0000 7fa151497640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.126+0000 7fa152499640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c06ec10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa144009870 tx=0x7fa14402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ed8c7b2f8e275f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.126+0000 7fa150c96640 1 -- 172.21.15.1:0/94824121 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037070 con 0x7fa14c15c4a0 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.126+0000 7fa150c96640 1 -- 172.21.15.1:0/94824121 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa144032440 con 0x7fa14c15c4a0 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.126+0000 7fa150c96640 1 -- 172.21.15.1:0/94824121 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144032810 con 0x7fa14c15c4a0 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 -- 172.21.15.1:0/94824121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 msgr2=0x7fa14c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c06ec10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa144009870 tx=0x7fa14402a270 comp rx=0 tx=0).stop 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 -- 172.21.15.1:0/94824121 shutdown_connections 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 --2- 172.21.15.1:0/94824121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c06ec10 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 -- 172.21.15.1:0/94824121 >> 172.21.15.1:0/94824121 conn(0x7fa14c0cc4d0 msgr2=0x7fa14c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 -- 172.21.15.1:0/94824121 shutdown_connections 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.127+0000 7fa153efc640 1 -- 172.21.15.1:0/94824121 wait complete. 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.128+0000 7fa153efc640 1 Processor -- start 2024-05-09T06:38:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.128+0000 7fa153efc640 1 -- start start 2024-05-09T06:38:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa14c158a50 0x7fa14c1607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c15bf80 con 0x7fa14c15c4a0 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c15bca0 con 0x7fa14c158a50 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa153efc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa14c070ea0 con 0x7fa14c071fd0 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37778/0 (socket says 172.21.15.1:37778) 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 -- 172.21.15.1:0/4174735259 learned_addr learned my addr 172.21.15.1:0/4174735259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 -- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 msgr2=0x7fa14c151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 -- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa14c158a50 msgr2=0x7fa14c1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa14c158a50 0x7fa14c1607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.129+0000 7fa152499640 1 -- 172.21.15.1:0/4174735259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c0b3420 con 0x7fa14c15c4a0 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa152499640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa144009870 tx=0x7fa1440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144041020 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1440306c0 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c0b3700 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037040 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.130+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c0b3fa0 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.131+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c0b3af0 con 0x7fa14c15c4a0 2024-05-09T06:38:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.131+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa144044020 con 0x7fa14c15c4a0 2024-05-09T06:38:16.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.132+0000 7fa14a7fc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 0x7fa13806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.132+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7fa1440ab510 con 0x7fa14c15c4a0 2024-05-09T06:38:16.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.132+0000 7fa151c98640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 0x7fa13806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.132+0000 7fa151c98640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 0x7fa13806e2f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001560 tx=0x7fa13c007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.136+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa14404b030 con 0x7fa14c15c4a0 2024-05-09T06:38:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.318+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa14c0752b0 con 0x7fa14c15c4a0 2024-05-09T06:38:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.318+0000 7fa14a7fc640 1 -- 172.21.15.1:0/4174735259 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7fa14407ee20 con 0x7fa14c15c4a0 2024-05-09T06:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 msgr2=0x7fa13806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 0x7fa13806e2f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001560 tx=0x7fa13c007ca0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 msgr2=0x7fa14c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa144009870 tx=0x7fa1440046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 shutdown_connections 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa13806be30 0x7fa13806e2f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa14c15c4a0 0x7fa14c0b0750 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa14c158a50 0x7fa14c1607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 --2- 172.21.15.1:0/4174735259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa14c071fd0 0x7fa14c151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 >> 172.21.15.1:0/4174735259 conn(0x7fa14c0cc4d0 msgr2=0x7fa14c06d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:16.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 shutdown_connections 2024-05-09T06:38:16.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.320+0000 7fa153efc640 1 -- 172.21.15.1:0/4174735259 wait complete. 2024-05-09T06:38:16.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-09T06:38:16.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.442+0000 7f9106990640 1 Processor -- start 2024-05-09T06:38:16.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.442+0000 7f9106990640 1 -- start start 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9106990640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 0x7f910006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9106990640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9100070f40 con 0x7f91000b2a60 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9106990640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91000cf170 con 0x7f910015db10 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9106990640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f91000cf450 con 0x7f910015e6d0 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90ff7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f91000b2a60 0x7f910015dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:47520/0 (socket says 172.21.15.1:47520) 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90ff7fe640 1 -- 172.21.15.1:0/3346112153 learned_addr learned my addr 172.21.15.1:0/3346112153 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9104f2d640 1 --2- 172.21.15.1:0/3346112153 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 0x7f910006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3477209853 0 0) 0x7f9100070f40 con 0x7f91000b2a60 2024-05-09T06:38:16.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90e8003440 con 0x7f91000b2a60 2024-05-09T06:38:16.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1002186326 0 0) 0x7f91000cf170 con 0x7f910015db10 2024-05-09T06:38:16.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9100070f40 con 0x7f910015db10 2024-05-09T06:38:16.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9104f2d640 1 -- 172.21.15.1:0/3346112153 >> v1:172.21.15.6:6789/0 conn(0x7f910015db10 legacy=0x7f91000715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9104f2d640 1 -- 172.21.15.1:0/3346112153 >> v1:172.21.15.1:6789/0 conn(0x7f91000b2a60 legacy=0x7f910015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.443+0000 7f9104f2d640 1 -- 172.21.15.1:0/3346112153 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9100070c10 con 0x7f910015e6d0 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.444+0000 7f9104f2d640 1 --2- 172.21.15.1:0/3346112153 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 0x7f910006e340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009dc0 tx=0x7f90f802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6937a50163872b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.444+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f802ae60 con 0x7f910015e6d0 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.444+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8007740 con 0x7f910015e6d0 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.444+0000 7f90feffd640 1 -- 172.21.15.1:0/3346112153 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007af0 con 0x7f910015e6d0 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 -- 172.21.15.1:0/3346112153 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 msgr2=0x7f910006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 --2- 172.21.15.1:0/3346112153 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 0x7f910006e340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009dc0 tx=0x7f90f802a640 comp rx=0 tx=0).stop 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 -- 172.21.15.1:0/3346112153 shutdown_connections 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 --2- 172.21.15.1:0/3346112153 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015e6d0 0x7f910006e340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 -- 172.21.15.1:0/3346112153 >> 172.21.15.1:0/3346112153 conn(0x7f91000cc4b0 msgr2=0x7f91000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 -- 172.21.15.1:0/3346112153 shutdown_connections 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.445+0000 7f9106990640 1 -- 172.21.15.1:0/3346112153 wait complete. 2024-05-09T06:38:16.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.446+0000 7f9106990640 1 Processor -- start 2024-05-09T06:38:16.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.446+0000 7f9106990640 1 -- start start 2024-05-09T06:38:16.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91000b2a60 0x7f91001f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 0x7f91001f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f910006ceb0 con 0x7f910015e6d0 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f910006cd30 con 0x7f91000b2a60 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9106990640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f910006d030 con 0x7f910015db10 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9104f2d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9104f2d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37780/0 (socket says 172.21.15.1:37780) 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f9104f2d640 1 -- 172.21.15.1:0/1068208469 learned_addr learned my addr 172.21.15.1:0/1068208469 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f90fffff640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91000b2a60 0x7f91001f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.447+0000 7f90ff7fe640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 0x7f91001f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 -- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 msgr2=0x7f91001f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 0x7f91001f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 -- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91000b2a60 msgr2=0x7f91001f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91000b2a60 0x7f91001f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 -- 172.21.15.1:0/1068208469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9100152530 con 0x7f910015e6d0 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f90ff7fe640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 0x7f91001f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:16.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9104f2d640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f90f802ac30 tx=0x7f90f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8009450 con 0x7f910015e6d0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f802ae60 con 0x7f910015e6d0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9100153990 con 0x7f910015e6d0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004470 con 0x7f910015e6d0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.448+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9100153f10 con 0x7f910015e6d0 2024-05-09T06:38:16.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.449+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91000752b0 con 0x7f910015e6d0 2024-05-09T06:38:16.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.449+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90f8004610 con 0x7f910015e6d0 2024-05-09T06:38:16.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.450+0000 7f90fcff9640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 0x7f90ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:16.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.450+0000 7f90fffff640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 0x7f90ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:16.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.450+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7f90f80ac0c0 con 0x7f910015e6d0 2024-05-09T06:38:16.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.450+0000 7f90fffff640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 0x7f90ec06e2f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:16.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.454+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9100152530 con 0x7f910015e6d0 2024-05-09T06:38:16.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:16.708+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f91000b3240 con 0x7f910015e6d0 2024-05-09T06:38:17.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.686+0000 7f90fcff9640 1 -- 172.21.15.1:0/1068208469 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v510) ==== 147+0+0 (secure 0 0 0) 0x7f90f803f030 con 0x7f910015e6d0 2024-05-09T06:38:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 msgr2=0x7f90ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 0x7f90ec06e2f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0009040 comp rx=0 tx=0).stop 2024-05-09T06:38:17.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 msgr2=0x7f9100151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f90f802ac30 tx=0x7f90f8004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 shutdown_connections 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f90ec06be30 0x7f90ec06e2f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f910015e6d0 0x7f9100151db0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f910015db10 0x7f91001f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 --2- 172.21.15.1:0/1068208469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91000b2a60 0x7f91001f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 >> 172.21.15.1:0/1068208469 conn(0x7f91000cc4b0 msgr2=0x7f910015dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 shutdown_connections 2024-05-09T06:38:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.689+0000 7f9106990640 1 -- 172.21.15.1:0/1068208469 wait complete. 2024-05-09T06:38:17.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:38:17.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.816+0000 7fcdfed7e640 1 Processor -- start 2024-05-09T06:38:17.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.816+0000 7fcdfed7e640 1 -- start start 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcdf815c4c0 0x7fcdf806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcdf80b4de0 0x7fcdf815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf8156980 con 0x7fcdf80b4de0 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf806d870 con 0x7fcdf815c4c0 2024-05-09T06:38:17.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfed7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf815d320 con 0x7fcdf80b5260 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdf7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcdf80b4de0 0x7fcdf815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdf7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fcdf80b4de0 0x7fcdf815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36508/0 (socket says 172.21.15.1:36508) 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdf7fff640 1 -- 172.21.15.1:0/3831756851 learned_addr learned my addr 172.21.15.1:0/3831756851 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfcb1a640 1 --2- 172.21.15.1:0/3831756851 >> v2:172.21.15.6:3300/0 conn(0x7fcdf815c4c0 0x7fcdf806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.817+0000 7fcdfd31b640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.818+0000 7fcdf7fff640 1 -- 172.21.15.1:0/3831756851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 msgr2=0x7fcdf815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.818+0000 7fcdf7fff640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.818+0000 7fcdf7fff640 1 -- 172.21.15.1:0/3831756851 >> v2:172.21.15.6:3300/0 conn(0x7fcdf815c4c0 msgr2=0x7fcdf806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.818+0000 7fcdf7fff640 1 --2- 172.21.15.1:0/3831756851 >> v2:172.21.15.6:3300/0 conn(0x7fcdf815c4c0 0x7fcdf806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.818+0000 7fcdf7fff640 1 -- 172.21.15.1:0/3831756851 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf806cf40 con 0x7fcdf80b4de0 2024-05-09T06:38:17.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdf7fff640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf80b4de0 0x7fcdf815c8a0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcdf804fb80 tx=0x7fcdec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14c35bd1f8af8de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:17.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdf77fe640 1 -- 172.21.15.1:0/3831756851 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037070 con 0x7fcdf80b4de0 2024-05-09T06:38:17.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdf77fe640 1 -- 172.21.15.1:0/3831756851 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec032440 con 0x7fcdf80b4de0 2024-05-09T06:38:17.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdf77fe640 1 -- 172.21.15.1:0/3831756851 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec032810 con 0x7fcdf80b4de0 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 -- 172.21.15.1:0/3831756851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf80b4de0 msgr2=0x7fcdf815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf80b4de0 0x7fcdf815c8a0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcdf804fb80 tx=0x7fcdec02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 -- 172.21.15.1:0/3831756851 shutdown_connections 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/3831756851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf80b4de0 0x7fcdf815c8a0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/3831756851 >> v2:172.21.15.6:3300/0 conn(0x7fcdf815c4c0 0x7fcdf806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 -- 172.21.15.1:0/3831756851 >> 172.21.15.1:0/3831756851 conn(0x7fcdf80cc4d0 msgr2=0x7fcdf80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 -- 172.21.15.1:0/3831756851 shutdown_connections 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.819+0000 7fcdfed7e640 1 -- 172.21.15.1:0/3831756851 wait complete. 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 Processor -- start 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 -- start start 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 0x7fcdf81f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf80aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80b2c40 con 0x7fcdf815c4c0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.821+0000 7fcdfed7e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf806d500 con 0x7fcdf80b4de0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfed7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80b2dc0 con 0x7fcdf80b5260 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36516/0 (socket says 172.21.15.1:36516) 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 -- 172.21.15.1:0/1179432548 learned_addr learned my addr 172.21.15.1:0/1179432548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdf7fff640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf80aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfcb1a640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 0x7fcdf81f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 -- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 msgr2=0x7fcdf80aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf80aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 -- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 msgr2=0x7fcdf81f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 0x7fcdf81f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 -- 172.21.15.1:0/1179432548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf81f44e0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfcb1a640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 0x7fcdf81f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfd31b640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e830 tx=0x7fcde800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8017070 con 0x7fcdf815c4c0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde800c8e0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf81f47c0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde800ccd0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.822+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf81f4d80 con 0x7fcdf815c4c0 2024-05-09T06:38:17.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.823+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf80752b0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.824+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcde801a070 con 0x7fcdf815c4c0 2024-05-09T06:38:17.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.824+0000 7fcdf57fa640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 0x7fcde406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:17.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.824+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7fcde808afc0 con 0x7fcdf815c4c0 2024-05-09T06:38:17.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.825+0000 7fcdfcb1a640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 0x7fcde406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:17.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.825+0000 7fcdfcb1a640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 0x7fcde406e320 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007440 tx=0x7fcdf000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:17.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:17.829+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcde8059a90 con 0x7fcdf815c4c0 2024-05-09T06:38:17.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.013+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcdf80b1500 con 0x7fcdf815c4c0 2024-05-09T06:38:17.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.017+0000 7fcdf57fa640 1 -- 172.21.15.1:0/1179432548 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fcde8059390 con 0x7fcdf815c4c0 2024-05-09T06:38:17.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 msgr2=0x7fcde406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:17.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 0x7fcde406e320 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007440 tx=0x7fcdf000b040 comp rx=0 tx=0).stop 2024-05-09T06:38:17.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 msgr2=0x7fcdf815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e830 tx=0x7fcde800ecf0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 shutdown_connections 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcde406be60 0x7fcde406e320 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdf815c4c0 0x7fcdf815b6c0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdf80b5260 0x7fcdf80aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 --2- 172.21.15.1:0/1179432548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdf80b4de0 0x7fcdf81f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 >> 172.21.15.1:0/1179432548 conn(0x7fcdf80cc4d0 msgr2=0x7fcdf815eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 shutdown_connections 2024-05-09T06:38:18.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.019+0000 7fcdfed7e640 1 -- 172.21.15.1:0/1179432548 wait complete. 2024-05-09T06:38:18.010 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:38:18.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:38:18.055 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:38:18.095 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:38:18.095 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:18.095 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:18.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:38:18.111 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:38:18.111 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:18.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:38:18.154 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:18.154 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:18.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:38:18.170 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:38:18.171 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:18.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:38:18.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:38:18.214 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:18.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.337+0000 7f9782261640 1 Processor -- start 2024-05-09T06:38:18.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 -- start start 2024-05-09T06:38:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 0x7f977c1627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f977c06b8f0 0x7f977c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977c0d0360 con 0x7f977c0d17e0 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f977c162d20 con 0x7f977c06b8f0 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.338+0000 7f9782261640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f977c0d05e0 con 0x7f977c15e150 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f977c06b8f0 0x7f977c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f977c06b8f0 0x7f977c0adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36532/0 (socket says 172.21.15.1:36532) 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 -- 172.21.15.1:0/990825303 learned_addr learned my addr 172.21.15.1:0/990825303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977b7fe640 1 --2- 172.21.15.1:0/990825303 >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 0x7f977c1627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 -- 172.21.15.1:0/990825303 >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 msgr2=0x7f977c1627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 --2- 172.21.15.1:0/990825303 >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 0x7f977c1627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 -- 172.21.15.1:0/990825303 >> v1:172.21.15.6:6789/0 conn(0x7f977c0d17e0 legacy=0x7f977c06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977bfff640 1 -- 172.21.15.1:0/990825303 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c0d08c0 con 0x7f977c06b8f0 2024-05-09T06:38:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.339+0000 7f977b7fe640 1 --2- 172.21.15.1:0/990825303 >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 0x7f977c1627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f977bfff640 1 --2- 172.21.15.1:0/990825303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c06b8f0 0x7f977c0adae0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f977c068ba0 tx=0x7f977402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a7598e7b3f9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f977affd640 1 -- 172.21.15.1:0/990825303 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774036070 con 0x7f977c06b8f0 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f977affd640 1 -- 172.21.15.1:0/990825303 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9774031600 con 0x7f977c06b8f0 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f977affd640 1 -- 172.21.15.1:0/990825303 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97740319d0 con 0x7f977c06b8f0 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 -- 172.21.15.1:0/990825303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c06b8f0 msgr2=0x7f977c0adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 --2- 172.21.15.1:0/990825303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c06b8f0 0x7f977c0adae0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f977c068ba0 tx=0x7f977402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 -- 172.21.15.1:0/990825303 shutdown_connections 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 --2- 172.21.15.1:0/990825303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c06b8f0 0x7f977c0adae0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 --2- 172.21.15.1:0/990825303 >> v2:172.21.15.6:3300/0 conn(0x7f977c15e150 0x7f977c1627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 -- 172.21.15.1:0/990825303 >> 172.21.15.1:0/990825303 conn(0x7f977c0d0c50 msgr2=0x7f977c0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 -- 172.21.15.1:0/990825303 shutdown_connections 2024-05-09T06:38:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.340+0000 7f9782261640 1 -- 172.21.15.1:0/990825303 wait complete. 2024-05-09T06:38:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.341+0000 7f9782261640 1 Processor -- start 2024-05-09T06:38:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 -- start start 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c06b8f0 0x7f977c1489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 0x7f977c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f977c15e530 con 0x7f977c0d17e0 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0ced10 con 0x7f977c06b8f0 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.342+0000 7f9782261640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f977c15e6b0 con 0x7f977c15e150 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36542/0 (socket says 172.21.15.1:36542) 2024-05-09T06:38:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 -- 172.21.15.1:0/2148874958 learned_addr learned my addr 172.21.15.1:0/2148874958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c06b8f0 0x7f977c1489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f977bfff640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 0x7f977c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 -- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 msgr2=0x7f977c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 0x7f977c0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 -- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c06b8f0 msgr2=0x7f977c1489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c06b8f0 0x7f977c1489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 -- 172.21.15.1:0/2148874958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c161720 con 0x7f977c0d17e0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f977bfff640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 0x7f977c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9772ffd640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f976400c430 tx=0x7f976400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764011790 con 0x7f977c0d17e0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764017070 con 0x7f977c0d17e0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c0b47b0 con 0x7f977c0d17e0 2024-05-09T06:38:18.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764011bb0 con 0x7f977c0d17e0 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.343+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c0b4d70 con 0x7f977c0d17e0 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.344+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c0752b0 con 0x7f977c0d17e0 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.345+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f976401c460 con 0x7f977c0d17e0 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.345+0000 7f9778ff9640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.345+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5421+0+0 (secure 0 0 0) 0x7f976408ab50 con 0x7f977c0d17e0 2024-05-09T06:38:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.345+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.346+0000 7f977b7fe640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 0x7f975c06e2f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f976c00aac0 tx=0x7f976c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.350+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9764059620 con 0x7f977c0d17e0 2024-05-09T06:38:18.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.568+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f977c162790 con 0x7f977c0d17e0 2024-05-09T06:38:18.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.569+0000 7f9778ff9640 1 -- 172.21.15.1:0/2148874958 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3857 (secure 0 0 0) 0x7f9764058f20 con 0x7f977c0d17e0 2024-05-09T06:38:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:18.550 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":91,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1445643534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1445643534},{"type":"v1","addr":"172.21.15.6:6834","nonce":1445643534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/4202695896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":4202695896},{"type":"v1","addr":"172.21.15.1:6837","nonce":4202695896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/1842783308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":1842783308},{"type":"v1","addr":"172.21.15.6:6835","nonce":1842783308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9441,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1642527228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1642527228},{"type":"v1","addr":"172.21.15.1:6835","nonce":1642527228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-05-09T06:38:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 91 2024-05-09T06:38:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 msgr2=0x7f975c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 0x7f975c06e2f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f976c00aac0 tx=0x7f976c008040 comp rx=0 tx=0).stop 2024-05-09T06:38:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 msgr2=0x7f977c160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f976400c430 tx=0x7f976400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 shutdown_connections 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f975c06be30 0x7f975c06e2f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f977c15e150 0x7f977c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c0d17e0 0x7f977c160da0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 --2- 172.21.15.1:0/2148874958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f977c06b8f0 0x7f977c1489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 >> 172.21.15.1:0/2148874958 conn(0x7f977c0d0c50 msgr2=0x7f977c0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 shutdown_connections 2024-05-09T06:38:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:18.571+0000 7f9782261640 1 -- 172.21.15.1:0/2148874958 wait complete. 2024-05-09T06:38:18.563 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:38:18.563 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:38:18.563 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:38:18.563 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:18.563 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:38:18.563 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:38:18.564 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:38:18.564 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:18.564 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:38:18.564 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:38:18.564 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:38:18.564 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:18.564 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:38:18.564 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:38:18.565 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:38:18.565 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:18.573 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:38:18.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:38:18.576 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:38:18.576 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:38:18.577 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:38:18.577 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:38:18.616 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:38:18.616 DEBUG:teuthology.parallel:result is None 2024-05-09T06:38:18.618 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:38:18.619 DEBUG:teuthology.parallel:result is None 2024-05-09T06:38:18.619 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:38:18.619 DEBUG:teuthology.parallel:result is None 2024-05-09T06:38:18.693 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T06:38:18.705 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2024-05-09T06:38:18.705 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2024-05-09T06:38:19.043 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:38:19.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:38:24.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-05-09T06:38:24.573 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:38:24.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:38:24.577 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:38:24.577 DEBUG:teuthology.parallel:result is None 2024-05-09T06:38:24.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:38:24.663 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:38:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.714+0000 7f8535ad2640 1 Processor -- start 2024-05-09T06:38:24.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.714+0000 7f8535ad2640 1 -- start start 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8530161700 0x7f85300d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85300d4990 0x7f85300d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f85300d2990 0x7f85300d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f85300cfd70 con 0x7f85300d2990 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f85300cf510 con 0x7f85300d4990 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f8535ad2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f85300d2e20 con 0x7f8530161700 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f85300d2990 0x7f85300d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f85300d2990 0x7f85300d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36582/0 (socket says 172.21.15.1:36582) 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 -- 172.21.15.1:0/4290032217 learned_addr learned my addr 172.21.15.1:0/4290032217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852effd640 1 --2- 172.21.15.1:0/4290032217 >> v2:172.21.15.6:3300/0 conn(0x7f85300d4990 0x7f85300d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852f7fe640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8530161700 0x7f85300d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 -- 172.21.15.1:0/4290032217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8530161700 msgr2=0x7f85300d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8530161700 0x7f85300d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 -- 172.21.15.1:0/4290032217 >> v2:172.21.15.6:3300/0 conn(0x7f85300d4990 msgr2=0x7f85300d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 --2- 172.21.15.1:0/4290032217 >> v2:172.21.15.6:3300/0 conn(0x7f85300d4990 0x7f85300d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.715+0000 7f852ffff640 1 -- 172.21.15.1:0/4290032217 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85300cf7b0 con 0x7f85300d2990 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.716+0000 7f852ffff640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d2990 0x7f85300d04f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f853008d7b0 tx=0x7f852802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4509dd42014858d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.716+0000 7f852e7fc640 1 -- 172.21.15.1:0/4290032217 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528036070 con 0x7f85300d2990 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.716+0000 7f852e7fc640 1 -- 172.21.15.1:0/4290032217 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8528031600 con 0x7f85300d2990 2024-05-09T06:38:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.716+0000 7f852e7fc640 1 -- 172.21.15.1:0/4290032217 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280319d0 con 0x7f85300d2990 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 -- 172.21.15.1:0/4290032217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d2990 msgr2=0x7f85300d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d2990 0x7f85300d04f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f853008d7b0 tx=0x7f852802a610 comp rx=0 tx=0).stop 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 -- 172.21.15.1:0/4290032217 shutdown_connections 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d2990 0x7f85300d04f0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 --2- 172.21.15.1:0/4290032217 >> v2:172.21.15.6:3300/0 conn(0x7f85300d4990 0x7f85300d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 --2- 172.21.15.1:0/4290032217 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8530161700 0x7f85300d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 -- 172.21.15.1:0/4290032217 >> 172.21.15.1:0/4290032217 conn(0x7f85300d3e70 msgr2=0x7f85300d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 -- 172.21.15.1:0/4290032217 shutdown_connections 2024-05-09T06:38:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.717+0000 7f8535ad2640 1 -- 172.21.15.1:0/4290032217 wait complete. 2024-05-09T06:38:24.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.719+0000 7f8535ad2640 1 Processor -- start 2024-05-09T06:38:24.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.719+0000 7f8535ad2640 1 -- start start 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85300d2990 0x7f8530150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8530161700 0x7f85300b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85300d2190 con 0x7f85300d4990 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f85300cb070 con 0x7f8530161700 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f8535ad2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8530161ae0 con 0x7f85300d2990 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f852effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f852effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36586/0 (socket says 172.21.15.1:36586) 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f852effd640 1 -- 172.21.15.1:0/3191362218 learned_addr learned my addr 172.21.15.1:0/3191362218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f852f7fe640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85300d2990 0x7f8530150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.720+0000 7f852ffff640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8530161700 0x7f85300b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 -- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85300d2990 msgr2=0x7f8530150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85300d2990 0x7f8530150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 -- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8530161700 msgr2=0x7f85300b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8530161700 0x7f85300b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 -- 172.21.15.1:0/3191362218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85300b2900 con 0x7f85300d4990 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f852effd640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f852400e890 tx=0x7f852400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524017070 con 0x7f85300d4990 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852400c940 con 0x7f85300d4990 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85300b88f0 con 0x7f85300d4990 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.721+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400cd10 con 0x7f85300d4990 2024-05-09T06:38:24.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.722+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85300b8eb0 con 0x7f85300d4990 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.723+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f852401a070 con 0x7f85300d4990 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.723+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85300752b0 con 0x7f85300d4990 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.723+0000 7f8517fff640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 0x7f851c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.723+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5421+0+0 (secure 0 0 0) 0x7f852408ac40 con 0x7f85300d4990 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.723+0000 7f852f7fe640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 0x7f851c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:24.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.724+0000 7f852f7fe640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 0x7f851c06e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f85300c7ec0 tx=0x7f8520009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.728+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f85240590c0 con 0x7f85300d4990 2024-05-09T06:38:24.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:24.925+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f853006d9a0 con 0x7f85300d4990 2024-05-09T06:38:25.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.717+0000 7f8517fff640 1 -- 172.21.15.1:0/3191362218 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) ==== 97+0+0 (secure 0 0 0) 0x7f852405d3c0 con 0x7f85300d4990 2024-05-09T06:38:25.698 INFO:teuthology.orchestra.run.smithi001.stderr: removed all blocklist entries 2024-05-09T06:38:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.720+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 msgr2=0x7f851c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.720+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 0x7f851c06e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f85300c7ec0 tx=0x7f8520009190 comp rx=0 tx=0).stop 2024-05-09T06:38:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.720+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 msgr2=0x7f85300b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.720+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f852400e890 tx=0x7f852400ed50 comp rx=0 tx=0).stop 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 shutdown_connections 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f851c06be30 0x7f851c06e2f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8530161700 0x7f85300b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85300d4990 0x7f85300b0a30 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 --2- 172.21.15.1:0/3191362218 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85300d2990 0x7f8530150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 >> 172.21.15.1:0/3191362218 conn(0x7f85300d3e70 msgr2=0x7f85300cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 shutdown_connections 2024-05-09T06:38:25.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.721+0000 7f8535ad2640 1 -- 172.21.15.1:0/3191362218 wait complete. 2024-05-09T06:38:25.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:38:25.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.844+0000 7f1b85da9640 1 Processor -- start 2024-05-09T06:38:25.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 -- start start 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b800d4940 0x7f1b800b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1b800d2b10 0x7f1b800b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b800b33c0 con 0x7f1b80161950 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1b800d4dd0 con 0x7f1b800d4940 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.845+0000 7f1b85da9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1b800b3660 con 0x7f1b800d2b10 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b800d4940 0x7f1b800b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1b800d4940 0x7f1b800b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36596/0 (socket says 172.21.15.1:36596) 2024-05-09T06:38:25.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 -- 172.21.15.1:0/4214653332 learned_addr learned my addr 172.21.15.1:0/4214653332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7ffff640 1 --2- 172.21.15.1:0/4214653332 >> v2:172.21.15.6:3300/0 conn(0x7f1b800d2b10 0x7f1b800b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 -- 172.21.15.1:0/4214653332 >> v2:172.21.15.6:3300/0 conn(0x7f1b800d2b10 msgr2=0x7f1b800b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 --2- 172.21.15.1:0/4214653332 >> v2:172.21.15.6:3300/0 conn(0x7f1b800d2b10 0x7f1b800b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 -- 172.21.15.1:0/4214653332 >> v1:172.21.15.6:6789/0 conn(0x7f1b80161950 legacy=0x7f1b800d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.846+0000 7f1b7effd640 1 -- 172.21.15.1:0/4214653332 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800b3940 con 0x7f1b800d4940 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b7effd640 1 --2- 172.21.15.1:0/4214653332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d4940 0x7f1b800b4140 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1b8008e540 tx=0x7f1b7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a21ab5ea86f3e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b7e7fc640 1 -- 172.21.15.1:0/4214653332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74037070 con 0x7f1b800d4940 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b7e7fc640 1 -- 172.21.15.1:0/4214653332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b74032440 con 0x7f1b800d4940 2024-05-09T06:38:25.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b7e7fc640 1 -- 172.21.15.1:0/4214653332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74032810 con 0x7f1b800d4940 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 -- 172.21.15.1:0/4214653332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d4940 msgr2=0x7f1b800b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 --2- 172.21.15.1:0/4214653332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d4940 0x7f1b800b4140 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f1b8008e540 tx=0x7f1b7402a270 comp rx=0 tx=0).stop 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 -- 172.21.15.1:0/4214653332 shutdown_connections 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 --2- 172.21.15.1:0/4214653332 >> v2:172.21.15.6:3300/0 conn(0x7f1b800d2b10 0x7f1b800b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 --2- 172.21.15.1:0/4214653332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d4940 0x7f1b800b4140 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 -- 172.21.15.1:0/4214653332 >> 172.21.15.1:0/4214653332 conn(0x7f1b800d3e20 msgr2=0x7f1b800d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:25.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 -- 172.21.15.1:0/4214653332 shutdown_connections 2024-05-09T06:38:25.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.847+0000 7f1b85da9640 1 -- 172.21.15.1:0/4214653332 wait complete. 2024-05-09T06:38:25.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.849+0000 7f1b85da9640 1 Processor -- start 2024-05-09T06:38:25.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.849+0000 7f1b85da9640 1 -- start start 2024-05-09T06:38:25.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b80161950 0x7f1b800ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b80159470 0x7f1b800cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800d1100 con 0x7f1b800d2b10 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800d0f80 con 0x7f1b80159470 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800d1c00 con 0x7f1b80161950 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36608/0 (socket says 172.21.15.1:36608) 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 -- 172.21.15.1:0/1287232585 learned_addr learned my addr 172.21.15.1:0/1287232585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7effd640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b80161950 0x7f1b800ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 -- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b80161950 msgr2=0x7f1b800ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b80161950 0x7f1b800ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 -- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b80159470 msgr2=0x7f1b800cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7ffff640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b80159470 0x7f1b800cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b80159470 0x7f1b800cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 -- 172.21.15.1:0/1287232585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800b6be0 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b7f7fe640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c3f0 tx=0x7f1b7000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b700052d0 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70005490 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70007260 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b800b6e80 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.850+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b800b7440 con 0x7f1b800d2b10 2024-05-09T06:38:25.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.851+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b800752b0 con 0x7f1b800d2b10 2024-05-09T06:38:25.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.852+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b70007400 con 0x7f1b800d2b10 2024-05-09T06:38:25.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.853+0000 7f1b67fff640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:25.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.853+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f1b7008a200 con 0x7f1b800d2b10 2024-05-09T06:38:25.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.853+0000 7f1b7effd640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:25.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.853+0000 7f1b7effd640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1b74034000 tx=0x7f1b7402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:25.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:25.857+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b70058ce0 con 0x7f1b800d2b10 2024-05-09T06:38:26.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.107+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b8006bb00 con 0x7f1b800d2b10 2024-05-09T06:38:26.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.108+0000 7f1b67fff640 1 -- 172.21.15.1:0/1287232585 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f1b700585e0 con 0x7f1b800d2b10 2024-05-09T06:38:26.088 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:26.088 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_dump": [ 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.a", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmJuapNhAAkQJheTcV/qc2+I7nzszWRw==", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.b", 2024-05-09T06:38:26.089 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm5O+XBxAASRkXgsR6jHdbEdohwY/iSw==", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.c", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmsTNLAhAArwrhy+4wSFhteLlgQd5Qiw==", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mds.d", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmD8P8DhAA7Mi3U3x7SGAPOchgepwmKA==", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.0", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCNbDxmOX5TMRAAAicszIEFLL9DIY4HwFyzMw==", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.1", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCQbDxmSkdjLxAAF3f/d+rzMVVFJl3RK9rH5Q==", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.2", 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCTbDxm5KP+LRAAIy+fqpKPlsbSohxKCcawug==", 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.3", 2024-05-09T06:38:26.093 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCWbDxmzoDxKxAA5PfmnDGK0A18jIH/FUccqQ==", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.4", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCcbDxmvd4fMxAA+QT1YqUjLcKCHKeuGMdh7A==", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.5", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCfbDxmjUxHMhAAxnuRsTg94HuySOw3Zmtsyw==", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.6", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCibDxm8NzrMBAA7Q2uLxFGKIkMRRR4kwfSEg==", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.095 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "osd.7", 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQClbDxmjsaILxAAFn5Vw6HCIhWE56rdo/WJ2Q==", 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow profile osd", 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile osd", 2024-05-09T06:38:26.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.0", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ==", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow r", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.1", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmFp1gHBAABPuQjALcymmaWGUp2UUfvQ==", 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.097 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.2", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmgXT6GBAADzBU6+xM251zPxwQ5n0Eqw==", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:38:26.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.3", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCMbDxmCv28HxAARllGAMQ7t49Rtkwl7/jPFw==", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow r", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow rw", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow rwx" 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "client.admin", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmjtoaGhAA9nbDQTED7zbK1+d4m8m+ng==", 2024-05-09T06:38:26.099 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": "allow *", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow *", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.x", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmO7pTMxAA2zHechtl5ahcgLt0hzwqVw==", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "entity": "mgr.y", 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "AQCLbDxmGxpVLxAALLaZG//4YlhCzcKe9px61Q==", 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": { 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": "allow *", 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": "allow profile mgr", 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": "allow *" 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T06:38:26.101 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 msgr2=0x7f1b6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1b74034000 tx=0x7f1b7402e000 comp rx=0 tx=0).stop 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 msgr2=0x7f1b800cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c3f0 tx=0x7f1b7000c8b0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 shutdown_connections 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b80159470 0x7f1b800cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1b80161950 0x7f1b800ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 --2- 172.21.15.1:0/1287232585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b800d2b10 0x7f1b800cd2b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 >> 172.21.15.1:0/1287232585 conn(0x7f1b800d3e20 msgr2=0x7f1b800d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 shutdown_connections 2024-05-09T06:38:26.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.110+0000 7f1b85da9640 1 -- 172.21.15.1:0/1287232585 wait complete. 2024-05-09T06:38:26.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:26.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.228+0000 7f6e77148640 1 Processor -- start 2024-05-09T06:38:26.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.228+0000 7f6e77148640 1 -- start start 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e610 0x7f6e70070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7015e9f0 0x7f6e70161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6e701620b0 0x7f6e70162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6e700ceb50 con 0x7f6e7015e9f0 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6e700cdfd0 con 0x7f6e701620b0 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e77148640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700ce2b0 con 0x7f6e7015e610 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7015e9f0 0x7f6e70161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6e7015e9f0 0x7f6e70161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36622/0 (socket says 172.21.15.1:36622) 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 -- 172.21.15.1:0/634367470 learned_addr learned my addr 172.21.15.1:0/634367470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:26.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e756e5640 1 --2- 172.21.15.1:0/634367470 >> v2:172.21.15.6:3300/0 conn(0x7f6e701620b0 0x7f6e70162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e74ee4640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e610 0x7f6e70070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 -- 172.21.15.1:0/634367470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e610 msgr2=0x7f6e70070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e610 0x7f6e70070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 -- 172.21.15.1:0/634367470 >> v2:172.21.15.6:3300/0 conn(0x7f6e701620b0 msgr2=0x7f6e70162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 --2- 172.21.15.1:0/634367470 >> v2:172.21.15.6:3300/0 conn(0x7f6e701620b0 0x7f6e70162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.229+0000 7f6e67fff640 1 -- 172.21.15.1:0/634367470 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700ce590 con 0x7f6e7015e9f0 2024-05-09T06:38:26.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.230+0000 7f6e67fff640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7015e9f0 0x7f6e70161b70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6e700d0a20 tx=0x7f6e6002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424170d4a3a8a57c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.230+0000 7f6e677fe640 1 -- 172.21.15.1:0/634367470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60037070 con 0x7f6e7015e9f0 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.230+0000 7f6e677fe640 1 -- 172.21.15.1:0/634367470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e60032440 con 0x7f6e7015e9f0 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.230+0000 7f6e677fe640 1 -- 172.21.15.1:0/634367470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60032810 con 0x7f6e7015e9f0 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 -- 172.21.15.1:0/634367470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7015e9f0 msgr2=0x7f6e70161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7015e9f0 0x7f6e70161b70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6e700d0a20 tx=0x7f6e6002a140 comp rx=0 tx=0).stop 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 -- 172.21.15.1:0/634367470 shutdown_connections 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 --2- 172.21.15.1:0/634367470 >> v2:172.21.15.6:3300/0 conn(0x7f6e701620b0 0x7f6e70162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e7015e9f0 0x7f6e70161b70 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 --2- 172.21.15.1:0/634367470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e610 0x7f6e70070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 -- 172.21.15.1:0/634367470 >> 172.21.15.1:0/634367470 conn(0x7f6e700cc4b0 msgr2=0x7f6e700cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 -- 172.21.15.1:0/634367470 shutdown_connections 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.231+0000 7f6e77148640 1 -- 172.21.15.1:0/634367470 wait complete. 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.232+0000 7f6e77148640 1 Processor -- start 2024-05-09T06:38:26.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.232+0000 7f6e77148640 1 -- start start 2024-05-09T06:38:26.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 0x7f6e7015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e9f0 0x7f6e70142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7015ee30 con 0x7f6e701620b0 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700718c0 con 0x7f6e7015e610 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e77148640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700b3790 con 0x7f6e7015e9f0 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36638/0 (socket says 172.21.15.1:36638) 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 -- 172.21.15.1:0/4254000975 learned_addr learned my addr 172.21.15.1:0/4254000975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e67fff640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e9f0 0x7f6e70142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e74ee4640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 0x7f6e7015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 -- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e9f0 msgr2=0x7f6e70142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e9f0 0x7f6e70142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 -- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 msgr2=0x7f6e7015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 0x7f6e7015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e756e5640 1 -- 172.21.15.1:0/4254000975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700b2b80 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.233+0000 7f6e74ee4640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 0x7f6e7015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e756e5640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800d5d0 tx=0x7f6e6800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680125f0 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68019070 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e700b2e60 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680129f0 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.234+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e700b3420 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.235+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e700752b0 con 0x7f6e701620b0 2024-05-09T06:38:26.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.235+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e68012b90 con 0x7f6e701620b0 2024-05-09T06:38:26.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.236+0000 7f6e657fa640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 0x7f6e5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.236+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f6e6808af80 con 0x7f6e701620b0 2024-05-09T06:38:26.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.236+0000 7f6e74ee4640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 0x7f6e5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.236+0000 7f6e74ee4640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 0x7f6e5c06e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6e700b5a70 tx=0x7f6e6c003a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.240+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e68059b00 con 0x7f6e701620b0 2024-05-09T06:38:26.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.425+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e700cce50 con 0x7f6e701620b0 2024-05-09T06:38:26.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.426+0000 7f6e657fa640 1 -- 172.21.15.1:0/4254000975 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+13042 (secure 0 0 0) 0x7f6e68059400 con 0x7f6e701620b0 2024-05-09T06:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":117,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:25.714870+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:26.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 msgr2=0x7f6e5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 0x7f6e5c06e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6e700b5a70 tx=0x7f6e6c003a10 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 msgr2=0x7f6e7013e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800d5d0 tx=0x7f6e6800da90 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 shutdown_connections 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6e5c06be30 0x7f6e5c06e2f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e701620b0 0x7f6e7013e890 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6e7015e9f0 0x7f6e70142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 --2- 172.21.15.1:0/4254000975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e7015e610 0x7f6e7015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 >> 172.21.15.1:0/4254000975 conn(0x7f6e700cc4b0 msgr2=0x7f6e70070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 shutdown_connections 2024-05-09T06:38:26.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.428+0000 7f6e77148640 1 -- 172.21.15.1:0/4254000975 wait complete. 2024-05-09T06:38:26.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:26.420 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.547+0000 7fdb25d89640 1 Processor -- start 2024-05-09T06:38:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.547+0000 7fdb25d89640 1 -- start start 2024-05-09T06:38:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb25d89640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb20158a50 0x7fdb200718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb25d89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb20071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb25d89640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2015b420 con 0x7fdb2006e6d0 2024-05-09T06:38:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb25d89640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdb20158e30 con 0x7fdb20158a50 2024-05-09T06:38:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb25d89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb20072480 con 0x7fdb2015c4a0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb1ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdb2006e6d0 0x7fdb200cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:34142/0 (socket says 172.21.15.1:34142) 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.548+0000 7fdb1ffff640 1 -- 172.21.15.1:0/388853318 learned_addr learned my addr 172.21.15.1:0/388853318 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1effd640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb20071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442108039 0 0) 0x7fdb2015b420 con 0x7fdb2006e6d0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1f7fe640 1 --2- 172.21.15.1:0/388853318 >> v2:172.21.15.6:3300/0 conn(0x7fdb20158a50 0x7fdb200718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb08003520 con 0x7fdb2006e6d0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1546903579 0 0) 0x7fdb08003520 con 0x7fdb2006e6d0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 msgr2=0x7fdb20071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb20071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 >> v2:172.21.15.6:3300/0 conn(0x7fdb20158a50 msgr2=0x7fdb200718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 --2- 172.21.15.1:0/388853318 >> v2:172.21.15.6:3300/0 conn(0x7fdb20158a50 0x7fdb200718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2015b6e0 con 0x7fdb2006e6d0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fdb180019f0 con 0x7fdb2006e6d0 2024-05-09T06:38:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 >> v1:172.21.15.1:6789/0 conn(0x7fdb2006e6d0 legacy=0x7fdb200cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08004d40 0x7fdb080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20158e30 con 0x7fdb08004d40 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20072480 con 0x7fdb080039b0 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0800b9c0 con 0x7fdb2015c4a0 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08004d40 0x7fdb080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1f7fe640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1effd640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 -- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 msgr2=0x7fdb08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb2015c4a0 0x7fdb08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 -- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb080039b0 msgr2=0x7fdb08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb080039b0 0x7fdb08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.549+0000 7fdb1ffff640 1 -- 172.21.15.1:0/388853318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb08003520 con 0x7fdb08004d40 2024-05-09T06:38:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.550+0000 7fdb1ffff640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08004d40 0x7fdb080051c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800a8a0 tx=0x7fdb08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7096610f7f24b8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.550+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800a4c0 con 0x7fdb08004d40 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.550+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1802f070 con 0x7fdb08004d40 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.550+0000 7fdb1e7fc640 1 -- 172.21.15.1:0/388853318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802bd30 con 0x7fdb08004d40 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 -- 172.21.15.1:0/388853318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08004d40 msgr2=0x7fdb080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 --2- 172.21.15.1:0/388853318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08004d40 0x7fdb080051c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800a8a0 tx=0x7fdb08005830 comp rx=0 tx=0).stop 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb1ffff640 1 -- 172.21.15.1:0/388853318 reap_dead start 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 -- 172.21.15.1:0/388853318 shutdown_connections 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 -- 172.21.15.1:0/388853318 >> 172.21.15.1:0/388853318 conn(0x7fdb200cc4d0 msgr2=0x7fdb200cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 -- 172.21.15.1:0/388853318 shutdown_connections 2024-05-09T06:38:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.551+0000 7fdb25d89640 1 -- 172.21.15.1:0/388853318 wait complete. 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.552+0000 7fdb25d89640 1 Processor -- start 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.552+0000 7fdb25d89640 1 -- start start 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb08004d40 0x7fdb20150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb200b0620 0x7fdb200b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2015bf80 con 0x7fdb201f1970 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2015bca0 con 0x7fdb200b0620 2024-05-09T06:38:26.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb25d89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdb20070ea0 con 0x7fdb08004d40 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36648/0 (socket says 172.21.15.1:36648) 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 -- 172.21.15.1:0/1790034601 learned_addr learned my addr 172.21.15.1:0/1790034601 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1ffff640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb200b0620 0x7fdb200b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1f7fe640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb08004d40 0x7fdb20150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 -- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb08004d40 msgr2=0x7fdb20150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb08004d40 0x7fdb20150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 -- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb200b0620 msgr2=0x7fdb200b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb200b0620 0x7fdb200b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.553+0000 7fdb1effd640 1 -- 172.21.15.1:0/1790034601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2015d820 con 0x7fdb201f1970 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb1effd640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fdb14011910 tx=0x7fdb14011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14019070 con 0x7fdb201f1970 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14004240 con 0x7fdb201f1970 2024-05-09T06:38:26.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201f1ee0 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f8e0 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.554+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201f24a0 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.555+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb200752b0 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.555+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb1401c070 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.556+0000 7fdb07fff640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 0x7fdb0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.556+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7fdb1408b1a0 con 0x7fdb201f1970 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.556+0000 7fdb1f7fe640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 0x7fdb0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.556+0000 7fdb1f7fe640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 0x7fdb0c06e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009b00 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.560+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb2015d820 con 0x7fdb201f1970 2024-05-09T06:38:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.741+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb2015e8b0 con 0x7fdb201f1970 2024-05-09T06:38:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.741+0000 7fdb07fff640 1 -- 172.21.15.1:0/1790034601 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) ==== 93+0+10 (secure 0 0 0) 0x7fdb14059c70 con 0x7fdb201f1970 2024-05-09T06:38:26.722 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:26.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 msgr2=0x7fdb0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 0x7fdb0c06e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009b00 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 msgr2=0x7fdb2015d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fdb14011910 tx=0x7fdb14011dd0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 shutdown_connections 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdb0c06be60 0x7fdb0c06e320 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb200b0620 0x7fdb200b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb201f1970 0x7fdb2015d2e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.743+0000 7fdb25d89640 1 --2- 172.21.15.1:0/1790034601 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb08004d40 0x7fdb20150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.744+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 >> 172.21.15.1:0/1790034601 conn(0x7fdb200cc4d0 msgr2=0x7fdb200cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.744+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 shutdown_connections 2024-05-09T06:38:26.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.744+0000 7fdb25d89640 1 -- 172.21.15.1:0/1790034601 wait complete. 2024-05-09T06:38:26.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:38:26.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.858+0000 7f33fba5d640 1 Processor -- start 2024-05-09T06:38:26.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.858+0000 7f33fba5d640 1 -- start start 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33fba5d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f33f4161950 0x7f33f40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33fba5d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33f40d4940 0x7f33f40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33fba5d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f40b33c0 con 0x7f33f40d0f80 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33fba5d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f33f40d4dd0 con 0x7f33f40d4940 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33fba5d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f33f40b3660 con 0x7f33f4161950 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33f40d4940 0x7f33f40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f33f40d4940 0x7f33f40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36658/0 (socket says 172.21.15.1:36658) 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 -- 172.21.15.1:0/4170544464 learned_addr learned my addr 172.21.15.1:0/4170544464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f97f9640 1 --2- 172.21.15.1:0/4170544464 >> v2:172.21.15.6:3300/0 conn(0x7f33f4161950 0x7f33f40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 -- 172.21.15.1:0/4170544464 >> v2:172.21.15.6:3300/0 conn(0x7f33f4161950 msgr2=0x7f33f40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 --2- 172.21.15.1:0/4170544464 >> v2:172.21.15.6:3300/0 conn(0x7f33f4161950 0x7f33f40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 -- 172.21.15.1:0/4170544464 >> v1:172.21.15.1:6789/0 conn(0x7f33f40d0f80 legacy=0x7f33f40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33f8ff8640 1 -- 172.21.15.1:0/4170544464 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f40b3940 con 0x7f33f40d4940 2024-05-09T06:38:26.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.859+0000 7f33ebfff640 1 -- 172.21.15.1:0/4170544464 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623360557 0 0) 0x7f33f40b33c0 con 0x7f33f40d0f80 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.860+0000 7f33f8ff8640 1 --2- 172.21.15.1:0/4170544464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d4940 0x7f33f40d2940 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f33f406b7f0 tx=0x7f33e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f4061165a72633 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.860+0000 7f33ebfff640 1 -- 172.21.15.1:0/4170544464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4037070 con 0x7f33f40d4940 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.860+0000 7f33ebfff640 1 -- 172.21.15.1:0/4170544464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e4032440 con 0x7f33f40d4940 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.860+0000 7f33ebfff640 1 -- 172.21.15.1:0/4170544464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4032810 con 0x7f33f40d4940 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 -- 172.21.15.1:0/4170544464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d4940 msgr2=0x7f33f40d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 --2- 172.21.15.1:0/4170544464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d4940 0x7f33f40d2940 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f33f406b7f0 tx=0x7f33e402a270 comp rx=0 tx=0).stop 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 -- 172.21.15.1:0/4170544464 shutdown_connections 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 --2- 172.21.15.1:0/4170544464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d4940 0x7f33f40d2940 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 --2- 172.21.15.1:0/4170544464 >> v2:172.21.15.6:3300/0 conn(0x7f33f4161950 0x7f33f40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 -- 172.21.15.1:0/4170544464 >> 172.21.15.1:0/4170544464 conn(0x7f33f40d3e20 msgr2=0x7f33f40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:26.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 -- 172.21.15.1:0/4170544464 shutdown_connections 2024-05-09T06:38:26.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.861+0000 7f33fba5d640 1 -- 172.21.15.1:0/4170544464 wait complete. 2024-05-09T06:38:26.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.862+0000 7f33fba5d640 1 Processor -- start 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.862+0000 7f33fba5d640 1 -- start start 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f4161950 0x7f33f40b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33f40b7fc0 0x7f33f415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33f40b52d0 con 0x7f33f40d0f80 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f33f40b5150 con 0x7f33f4161950 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33fba5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f33f40b5450 con 0x7f33f40b7fc0 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36664/0 (socket says 172.21.15.1:36664) 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 -- 172.21.15.1:0/772550799 learned_addr learned my addr 172.21.15.1:0/772550799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 -- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33f40b7fc0 msgr2=0x7f33f415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33f40b7fc0 0x7f33f415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 -- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f4161950 msgr2=0x7f33f40b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f4161950 0x7f33f40b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 -- 172.21.15.1:0/772550799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f41f4b90 con 0x7f33f40d0f80 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.863+0000 7f33f97f9640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d5f0 tx=0x7f33f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.864+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f00125f0 con 0x7f33f40d0f80 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.864+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f0019070 con 0x7f33f40d0f80 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.864+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f40cb020 con 0x7f33f40d0f80 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.864+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f00129d0 con 0x7f33f40d0f80 2024-05-09T06:38:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.864+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f40cb5e0 con 0x7f33f40d0f80 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.865+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f40752b0 con 0x7f33f40d0f80 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.865+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33f0012b70 con 0x7f33f40d0f80 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.866+0000 7f33e9ffb640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 0x7f33d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.866+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f33f008b020 con 0x7f33f40d0f80 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.866+0000 7f33f8ff8640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 0x7f33d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.867+0000 7f33f8ff8640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 0x7f33d806e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33e4034000 tx=0x7f33e402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:26.871+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33f0059c20 con 0x7f33f40d0f80 2024-05-09T06:38:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.056+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f33f40d21e0 con 0x7f33f40d0f80 2024-05-09T06:38:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.056+0000 7f33e9ffb640 1 -- 172.21.15.1:0/772550799 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) ==== 84+0+65 (secure 0 0 0) 0x7f33f0059520 con 0x7f33f40d0f80 2024-05-09T06:38:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 msgr2=0x7f33d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 0x7f33d806e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33e4034000 tx=0x7f33e402c000 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 msgr2=0x7f33f40cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d5f0 tx=0x7f33f000dab0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 shutdown_connections 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f33d806be30 0x7f33d806e2f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33f40b7fc0 0x7f33f415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f4161950 0x7f33f40b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 --2- 172.21.15.1:0/772550799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33f40d0f80 0x7f33f40cd620 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 >> 172.21.15.1:0/772550799 conn(0x7f33f40d3e20 msgr2=0x7f33f406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 shutdown_connections 2024-05-09T06:38:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.058+0000 7f33fba5d640 1 -- 172.21.15.1:0/772550799 wait complete. 2024-05-09T06:38:27.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:38:27.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:38:27.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.177+0000 7f680b97e640 1 Processor -- start 2024-05-09T06:38:27.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.177+0000 7f680b97e640 1 -- start start 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 0x7f68040cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68041446d0 0x7f68040d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f680415e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f68040ce8d0 con 0x7f68041446d0 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f68040cdd50 con 0x7f6804143bb0 2024-05-09T06:38:27.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680b97e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f68040ce030 con 0x7f6804144110 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68041446d0 0x7f68040d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68041446d0 0x7f68040d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36678/0 (socket says 172.21.15.1:36678) 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 -- 172.21.15.1:0/1986923436 learned_addr learned my addr 172.21.15.1:0/1986923436 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680971a640 1 --2- 172.21.15.1:0/1986923436 >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 0x7f68040cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 -- 172.21.15.1:0/1986923436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 msgr2=0x7f680415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6809f1b640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f680415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f680415e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 -- 172.21.15.1:0/1986923436 >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 msgr2=0x7f68040cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 --2- 172.21.15.1:0/1986923436 >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 0x7f68040cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f6808f19640 1 -- 172.21.15.1:0/1986923436 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68040ce310 con 0x7f68041446d0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.178+0000 7f680971a640 1 --2- 172.21.15.1:0/1986923436 >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 0x7f68040cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.179+0000 7f6808f19640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f68040d01f0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f68040cef80 tx=0x7f67f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7064b58306003c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.179+0000 7f6803fff640 1 -- 172.21.15.1:0/1986923436 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8037070 con 0x7f68041446d0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.179+0000 7f6803fff640 1 -- 172.21.15.1:0/1986923436 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f8032440 con 0x7f68041446d0 2024-05-09T06:38:27.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.179+0000 7f6803fff640 1 -- 172.21.15.1:0/1986923436 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8032810 con 0x7f68041446d0 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 -- 172.21.15.1:0/1986923436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 msgr2=0x7f68040d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f68040d01f0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f68040cef80 tx=0x7f67f802a140 comp rx=0 tx=0).stop 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 -- 172.21.15.1:0/1986923436 shutdown_connections 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f680415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 --2- 172.21.15.1:0/1986923436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f68040d01f0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 --2- 172.21.15.1:0/1986923436 >> v2:172.21.15.6:3300/0 conn(0x7f6804143bb0 0x7f68040cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 -- 172.21.15.1:0/1986923436 >> 172.21.15.1:0/1986923436 conn(0x7f680406b3c0 msgr2=0x7f680406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 -- 172.21.15.1:0/1986923436 shutdown_connections 2024-05-09T06:38:27.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.180+0000 7f680b97e640 1 -- 172.21.15.1:0/1986923436 wait complete. 2024-05-09T06:38:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.181+0000 7f680b97e640 1 Processor -- start 2024-05-09T06:38:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.181+0000 7f680b97e640 1 -- start start 2024-05-09T06:38:27.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6804143bb0 0x7f680413cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f6804159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68040cdaf0 con 0x7f68041446d0 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f68040cd970 con 0x7f6804143bb0 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f680415eea0 con 0x7f6804144110 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36690/0 (socket says 172.21.15.1:36690) 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 -- 172.21.15.1:0/3129244404 learned_addr learned my addr 172.21.15.1:0/3129244404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6808f19640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f6804159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680971a640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6804143bb0 0x7f680413cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 -- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 msgr2=0x7f6804159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f6804159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 -- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6804143bb0 msgr2=0x7f680413cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6804143bb0 0x7f680413cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 -- 172.21.15.1:0/3129244404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6804141ce0 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6808f19640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f6804159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6809f1b640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c7f0 tx=0x7f67fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc015070 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc004220 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680415a3a0 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.182+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc007620 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.183+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680413cff0 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.184+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804075320 con 0x7f68041446d0 2024-05-09T06:38:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.184+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67fc0079d0 con 0x7f68041446d0 2024-05-09T06:38:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.185+0000 7f6801ffb640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 0x7f67f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.185+0000 7f680971a640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 0x7f67f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.185+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5249+0+0 (secure 0 0 0) 0x7f67fc08abf0 con 0x7f68041446d0 2024-05-09T06:38:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.185+0000 7f680971a640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 0x7f67f006e2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f67f4008f80 tx=0x7f67f40084d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.189+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67fc0597f0 con 0x7f68041446d0 2024-05-09T06:38:27.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.372+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f68040cf6b0 con 0x7f68041446d0 2024-05-09T06:38:27.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.723+0000 7f6801ffb640 1 -- 172.21.15.1:0/3129244404 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) ==== 138+0+0 (secure 0 0 0) 0x7f67fc0590f0 con 0x7f68041446d0 2024-05-09T06:38:27.704 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T06:38:27.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.726+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 msgr2=0x7f67f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.726+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 0x7f67f006e2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f67f4008f80 tx=0x7f67f40084d0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 msgr2=0x7f6804156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c7f0 tx=0x7f67fc00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 shutdown_connections 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f67f006be30 0x7f67f006e2f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041446d0 0x7f6804156280 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6804144110 0x7f6804159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 --2- 172.21.15.1:0/3129244404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6804143bb0 0x7f680413cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 >> 172.21.15.1:0/3129244404 conn(0x7f680406b3c0 msgr2=0x7f680406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.727+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 shutdown_connections 2024-05-09T06:38:27.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.728+0000 7f680b97e640 1 -- 172.21.15.1:0/3129244404 wait complete. 2024-05-09T06:38:27.723 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:38:27.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.854+0000 7f1bd2efc640 1 Processor -- start 2024-05-09T06:38:27.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.854+0000 7f1bd2efc640 1 -- start start 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd2efc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 0x7f1bcc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd2efc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1bcc06ce70 0x7f1bcc06d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd2efc640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bcc0cf0a0 con 0x7f1bcc142650 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd2efc640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc142a30 con 0x7f1bcc06ce70 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd2efc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc0cee50 con 0x7f1bcc1452e0 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd0c98640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1bcc142650 0x7f1bcc0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60224/0 (socket says 172.21.15.1:60224) 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd0c98640 1 -- 172.21.15.1:0/2499452600 learned_addr learned my addr 172.21.15.1:0/2499452600 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bd1499640 1 --2- 172.21.15.1:0/2499452600 >> v2:172.21.15.1:3300/0 conn(0x7f1bcc06ce70 0x7f1bcc06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bcb7fe640 1 -- 172.21.15.1:0/2499452600 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3776926462 0 0) 0x7f1bcc0cf0a0 con 0x7f1bcc142650 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.855+0000 7f1bcb7fe640 1 -- 172.21.15.1:0/2499452600 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bb8003520 con 0x7f1bcc142650 2024-05-09T06:38:27.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bd1499640 1 -- 172.21.15.1:0/2499452600 >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 msgr2=0x7f1bcc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bcbfff640 1 --2- 172.21.15.1:0/2499452600 >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 0x7f1bcc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bd1499640 1 --2- 172.21.15.1:0/2499452600 >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 0x7f1bcc0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bd1499640 1 -- 172.21.15.1:0/2499452600 >> v1:172.21.15.1:6789/0 conn(0x7f1bcc142650 legacy=0x7f1bcc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bd1499640 1 -- 172.21.15.1:0/2499452600 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc0727d0 con 0x7f1bcc06ce70 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bcbfff640 1 --2- 172.21.15.1:0/2499452600 >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 0x7f1bcc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bd1499640 1 --2- 172.21.15.1:0/2499452600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc06d280 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc06cbe0 tx=0x7f1bb402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f20e36093dd2118 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.856+0000 7f1bcb7fe640 1 -- 172.21.15.1:0/2499452600 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4037070 con 0x7f1bcc06ce70 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bcb7fe640 1 -- 172.21.15.1:0/2499452600 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb402add0 con 0x7f1bcc06ce70 2024-05-09T06:38:27.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bcb7fe640 1 -- 172.21.15.1:0/2499452600 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4032660 con 0x7f1bcc06ce70 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 -- 172.21.15.1:0/2499452600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc06ce70 msgr2=0x7f1bcc06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/2499452600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc06d280 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc06cbe0 tx=0x7f1bb402a080 comp rx=0 tx=0).stop 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 -- 172.21.15.1:0/2499452600 shutdown_connections 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/2499452600 >> v2:172.21.15.6:3300/0 conn(0x7f1bcc1452e0 0x7f1bcc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/2499452600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc06d280 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 -- 172.21.15.1:0/2499452600 >> 172.21.15.1:0/2499452600 conn(0x7f1bcc0b1760 msgr2=0x7f1bcc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 -- 172.21.15.1:0/2499452600 shutdown_connections 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.857+0000 7f1bd2efc640 1 -- 172.21.15.1:0/2499452600 wait complete. 2024-05-09T06:38:27.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.858+0000 7f1bd2efc640 1 Processor -- start 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.858+0000 7f1bd2efc640 1 -- start start 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc13f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bcc1452e0 0x7f1bcc0ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc072d90 con 0x7f1bcc142650 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc072580 con 0x7f1bcc06ce70 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bd2efc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc0ce5c0 con 0x7f1bcc1452e0 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47846/0 (socket says 172.21.15.1:47846) 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.859+0000 7f1bcbfff640 1 -- 172.21.15.1:0/1862722043 learned_addr learned my addr 172.21.15.1:0/1862722043 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bd1499640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bcc1452e0 0x7f1bcc0ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bd0c98640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc13f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 -- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bcc1452e0 msgr2=0x7f1bcc0ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bcc1452e0 0x7f1bcc0ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 -- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 msgr2=0x7f1bcc13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc13f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 -- 172.21.15.1:0/1862722043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc0af0b0 con 0x7f1bcc142650 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bd0c98640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc13f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bcbfff640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00ce00 tx=0x7f1bbc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00f890 con 0x7f1bcc142650 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00fa30 con 0x7f1bcc142650 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc0addc0 con 0x7f1bcc142650 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00fe40 con 0x7f1bcc142650 2024-05-09T06:38:27.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.860+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc0ae380 con 0x7f1bcc142650 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.861+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc0752b0 con 0x7f1bcc142650 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.862+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bbc018400 con 0x7f1bcc142650 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.862+0000 7f1bc97fa640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 0x7f1bac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.862+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5619+0+0 (secure 0 0 0) 0x7f1bbc08bb00 con 0x7f1bcc142650 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.862+0000 7f1bd0c98640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 0x7f1bac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.863+0000 7f1bd0c98640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 0x7f1bac06e320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4007fd0 tx=0x7f1bc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:27.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:27.867+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1bbc014030 con 0x7f1bcc142650 2024-05-09T06:38:28.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.050+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1bcc0b1490 con 0x7f1bcc142650 2024-05-09T06:38:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.726+0000 7f1bc97fa640 1 -- 172.21.15.1:0/1862722043 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) ==== 170+0+0 (secure 0 0 0) 0x7f1bbc00ae20 con 0x7f1bcc142650 2024-05-09T06:38:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T06:38:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 msgr2=0x7f1bac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 0x7f1bac06e320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4007fd0 tx=0x7f1bc4009040 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 msgr2=0x7f1bcc0aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00ce00 tx=0x7f1bbc00a650 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 shutdown_connections 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1bac06be60 0x7f1bac06e320 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bcc1452e0 0x7f1bcc0ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bcc142650 0x7f1bcc0aaf30 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 --2- 172.21.15.1:0/1862722043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bcc06ce70 0x7f1bcc13f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 >> 172.21.15.1:0/1862722043 conn(0x7f1bcc0b1760 msgr2=0x7f1bcc1438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 shutdown_connections 2024-05-09T06:38:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.729+0000 7f1bd2efc640 1 -- 172.21.15.1:0/1862722043 wait complete. 2024-05-09T06:38:28.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:38:28.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.868+0000 7f2f9a91b640 1 Processor -- start 2024-05-09T06:38:28.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.868+0000 7f2f9a91b640 1 -- start start 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f9a91b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2f9406b1e0 0x7f2f940cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f9a91b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f94072050 con 0x7f2f940c41e0 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f9a91b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f94072330 con 0x7f2f940d06f0 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f9a91b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2f94072610 con 0x7f2f9406b1e0 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f98eb8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2f9406b1e0 0x7f2f940cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f8bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2f940c41e0 0x7f2f940ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60234/0 (socket says 172.21.15.1:60234) 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f98eb8640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2f9406b1e0 0x7f2f940cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47862/0 (socket says 172.21.15.1:47862) 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.869+0000 7f2f8bfff640 1 -- 172.21.15.1:0/1006965789 learned_addr learned my addr 172.21.15.1:0/1006965789 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f98eb8640 1 -- 172.21.15.1:0/1006965789 >> v1:172.21.15.6:6789/0 conn(0x7f2f940d06f0 legacy=0x7f2f940cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f98eb8640 1 -- 172.21.15.1:0/1006965789 >> v1:172.21.15.1:6789/0 conn(0x7f2f940c41e0 legacy=0x7f2f940ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f98eb8640 1 -- 172.21.15.1:0/1006965789 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f940728f0 con 0x7f2f9406b1e0 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f98eb8640 1 --2- 172.21.15.1:0/1006965789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f9406b1e0 0x7f2f940cefe0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009d90 tx=0x7f2f8c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3856f649c92bcc7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f937fe640 1 -- 172.21.15.1:0/1006965789 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c007740 con 0x7f2f9406b1e0 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f937fe640 1 -- 172.21.15.1:0/1006965789 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0078e0 con 0x7f2f9406b1e0 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.870+0000 7f2f937fe640 1 -- 172.21.15.1:0/1006965789 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c007cb0 con 0x7f2f9406b1e0 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 -- 172.21.15.1:0/1006965789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f9406b1e0 msgr2=0x7f2f940cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/1006965789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f9406b1e0 0x7f2f940cefe0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009d90 tx=0x7f2f8c02a6f0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 -- 172.21.15.1:0/1006965789 shutdown_connections 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/1006965789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f9406b1e0 0x7f2f940cefe0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 -- 172.21.15.1:0/1006965789 >> 172.21.15.1:0/1006965789 conn(0x7f2f940d0b50 msgr2=0x7f2f940d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 -- 172.21.15.1:0/1006965789 shutdown_connections 2024-05-09T06:38:28.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.871+0000 7f2f9a91b640 1 -- 172.21.15.1:0/1006965789 wait complete. 2024-05-09T06:38:28.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.872+0000 7f2f9a91b640 1 Processor -- start 2024-05-09T06:38:28.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.872+0000 7f2f9a91b640 1 -- start start 2024-05-09T06:38:28.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f9406b1e0 0x7f2f941f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f940c41e0 0x7f2f940c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2f940ccf70 con 0x7f2f940d06f0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2f940ccdf0 con 0x7f2f940c41e0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f9a91b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2f940cd0f0 con 0x7f2f9406b1e0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47866/0 (socket says 172.21.15.1:47866) 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 -- 172.21.15.1:0/3610168103 learned_addr learned my addr 172.21.15.1:0/3610168103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f93fff640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f9406b1e0 0x7f2f941f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f8bfff640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f940c41e0 0x7f2f940c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 -- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f9406b1e0 msgr2=0x7f2f941f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f9406b1e0 0x7f2f941f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 -- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f940c41e0 msgr2=0x7f2f940c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:28.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f940c41e0 0x7f2f940c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.873+0000 7f2f98eb8640 1 -- 172.21.15.1:0/3610168103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f940c5260 con 0x7f2f940d06f0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f98eb8640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009eb0 tx=0x7f2f8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c040070 con 0x7f2f940d06f0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c031c90 con 0x7f2f940d06f0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f940c5540 con 0x7f2f940d06f0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c045680 con 0x7f2f940d06f0 2024-05-09T06:38:28.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.874+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f940c5de0 con 0x7f2f940d06f0 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.875+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f940752b0 con 0x7f2f940d06f0 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.876+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f8c039040 con 0x7f2f940d06f0 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.876+0000 7f2f917fa640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 0x7f2f7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.876+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6001+0+0 (secure 0 0 0) 0x7f2f8c0ac900 con 0x7f2f940d06f0 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.876+0000 7f2f93fff640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 0x7f2f7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:28.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.877+0000 7f2f93fff640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 0x7f2f7406e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2f84007fd0 tx=0x7f2f84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:28.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:28.881+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f8c043360 con 0x7f2f940d06f0 2024-05-09T06:38:29.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.103+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2f94072f30 con 0x7f2f940d06f0 2024-05-09T06:38:29.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.748+0000 7f2f917fa640 1 -- 172.21.15.1:0/3610168103 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v95) ==== 174+0+0 (secure 0 0 0) 0x7f2f8c07b2d0 con 0x7f2f940d06f0 2024-05-09T06:38:29.730 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 17 and data pool 18 2024-05-09T06:38:29.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 msgr2=0x7f2f7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 0x7f2f7406e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2f84007fd0 tx=0x7f2f84009040 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 msgr2=0x7f2f941f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009eb0 tx=0x7f2f8c004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 shutdown_connections 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2f7406be30 0x7f2f7406e2f0 secure :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2f84007fd0 tx=0x7f2f84009040 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f940d06f0 0x7f2f941f7490 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f940c41e0 0x7f2f940c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 --2- 172.21.15.1:0/3610168103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f9406b1e0 0x7f2f941f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.758+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 >> 172.21.15.1:0/3610168103 conn(0x7f2f940d0b50 msgr2=0x7f2f9415d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:29.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.759+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 shutdown_connections 2024-05-09T06:38:29.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.759+0000 7f2f9a91b640 1 -- 172.21.15.1:0/3610168103 wait complete. 2024-05-09T06:38:29.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:29.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.882+0000 7f4e15ea7640 1 Processor -- start 2024-05-09T06:38:29.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 -- start start 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4e100d0f80 0x7f4e100d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e100b33c0 con 0x7f4e100d4940 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4e100d4dd0 con 0x7f4e100d0f80 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.883+0000 7f4e15ea7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e100b3660 con 0x7f4e10161950 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4e100d0f80 0x7f4e100d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0effd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4e100d4940 0x7f4e100cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:44394/0 (socket says 172.21.15.1:44394) 2024-05-09T06:38:29.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0effd640 1 -- 172.21.15.1:0/3071534704 learned_addr learned my addr 172.21.15.1:0/3071534704 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:29.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0ffff640 1 -- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 msgr2=0x7f4e100d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0ffff640 1 --2- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0ffff640 1 -- 172.21.15.1:0/3071534704 >> v1:172.21.15.6:6789/0 conn(0x7f4e100d4940 legacy=0x7f4e100cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.884+0000 7f4e0ffff640 1 -- 172.21.15.1:0/3071534704 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e100b3940 con 0x7f4e100d0f80 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.885+0000 7f4e0ffff640 1 --2- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100d0f80 0x7f4e100d2940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e100b3590 tx=0x7f4e0802a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9dd7c48b0c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.885+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/3071534704 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0802ae60 con 0x7f4e100d0f80 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.885+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/3071534704 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e08007740 con 0x7f4e100d0f80 2024-05-09T06:38:29.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.885+0000 7f4e0e7fc640 1 -- 172.21.15.1:0/3071534704 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08007b10 con 0x7f4e100d0f80 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 -- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100d0f80 msgr2=0x7f4e100d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100d0f80 0x7f4e100d2940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e100b3590 tx=0x7f4e0802a640 comp rx=0 tx=0).stop 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 -- 172.21.15.1:0/3071534704 shutdown_connections 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100d0f80 0x7f4e100d2940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/3071534704 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 -- 172.21.15.1:0/3071534704 >> 172.21.15.1:0/3071534704 conn(0x7f4e100d3e20 msgr2=0x7f4e100d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 -- 172.21.15.1:0/3071534704 shutdown_connections 2024-05-09T06:38:29.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.886+0000 7f4e15ea7640 1 -- 172.21.15.1:0/3071534704 wait complete. 2024-05-09T06:38:29.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.887+0000 7f4e15ea7640 1 Processor -- start 2024-05-09T06:38:29.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.887+0000 7f4e15ea7640 1 -- start start 2024-05-09T06:38:29.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100cd500 0x7f4e100cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e100b52d0 con 0x7f4e100d0f80 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e100b5150 con 0x7f4e100cd500 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e15ea7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e100b5450 con 0x7f4e10161950 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47882/0 (socket says 172.21.15.1:47882) 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 -- 172.21.15.1:0/2299249554 learned_addr learned my addr 172.21.15.1:0/2299249554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0ffff640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100cd500 0x7f4e100cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0effd640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 -- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 msgr2=0x7f4e100b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 -- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100cd500 msgr2=0x7f4e100cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100cd500 0x7f4e100cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 -- 172.21.15.1:0/2299249554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e100b6d80 con 0x7f4e100d0f80 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.888+0000 7f4e0f7fe640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e8a0 tx=0x7f4e0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.889+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00017070 con 0x7f4e100d0f80 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.889+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0000c940 con 0x7f4e100d0f80 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.889+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e100b7060 con 0x7f4e100d0f80 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.889+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000cd30 con 0x7f4e100d0f80 2024-05-09T06:38:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.889+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e100b7620 con 0x7f4e100d0f80 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.890+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e100d4730 con 0x7f4e100d0f80 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.890+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e0001a070 con 0x7f4e100d0f80 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.891+0000 7f4df7fff640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 0x7f4dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.891+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6101+0+0 (secure 0 0 0) 0x7f4e0008ae40 con 0x7f4e100d0f80 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.891+0000 7f4e0effd640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 0x7f4dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:29.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.892+0000 7f4e0effd640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 0x7f4dfc06e2f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4e100cd490 tx=0x7f4e04007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:29.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:29.895+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e100d4730 con 0x7f4e100d0f80 2024-05-09T06:38:30.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.080+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e100cfb40 con 0x7f4e100d0f80 2024-05-09T06:38:30.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.082+0000 7f4df7fff640 1 -- 172.21.15.1:0/2299249554 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16833 (secure 0 0 0) 0x7f4e100cfb40 con 0x7f4e100d0f80 2024-05-09T06:38:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:30.062 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":120,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:29.105412+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 msgr2=0x7f4dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 0x7f4dfc06e2f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4e100cd490 tx=0x7f4e04007e80 comp rx=0 tx=0).stop 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 msgr2=0x7f4e10159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e8a0 tx=0x7f4e0000ed60 comp rx=0 tx=0).stop 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 shutdown_connections 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4dfc06be30 0x7f4dfc06e2f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.083+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e100cd500 0x7f4e100cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.084+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e10161950 0x7f4e100b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.084+0000 7f4e15ea7640 1 --2- 172.21.15.1:0/2299249554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e100d0f80 0x7f4e10159050 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.084+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 >> 172.21.15.1:0/2299249554 conn(0x7f4e100d3e20 msgr2=0x7f4e100b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:30.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.084+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 shutdown_connections 2024-05-09T06:38:30.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.084+0000 7f4e15ea7640 1 -- 172.21.15.1:0/2299249554 wait complete. 2024-05-09T06:38:30.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.199+0000 7f44ae714640 1 Processor -- start 2024-05-09T06:38:30.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.199+0000 7f44ae714640 1 -- start start 2024-05-09T06:38:30.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.199+0000 7f44ae714640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f44a8161950 0x7f44a80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44ae714640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44a80d0f80 0x7f44a80d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44ae714640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a80b33c0 con 0x7f44a80d4940 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44ae714640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f44a80d4dd0 con 0x7f44a8161950 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44ae714640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f44a80b3660 con 0x7f44a80d0f80 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f44a8161950 0x7f44a80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f44a8161950 0x7f44a80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47888/0 (socket says 172.21.15.1:47888) 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 -- 172.21.15.1:0/3174926226 learned_addr learned my addr 172.21.15.1:0/3174926226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44accb1640 1 --2- 172.21.15.1:0/3174926226 >> v2:172.21.15.6:3300/0 conn(0x7f44a80d0f80 0x7f44a80d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 -- 172.21.15.1:0/3174926226 >> v2:172.21.15.6:3300/0 conn(0x7f44a80d0f80 msgr2=0x7f44a80d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 --2- 172.21.15.1:0/3174926226 >> v2:172.21.15.6:3300/0 conn(0x7f44a80d0f80 0x7f44a80d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 -- 172.21.15.1:0/3174926226 >> v1:172.21.15.6:6789/0 conn(0x7f44a80d4940 legacy=0x7f44a80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:30.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.200+0000 7f44a7fff640 1 -- 172.21.15.1:0/3174926226 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80b3940 con 0x7f44a8161950 2024-05-09T06:38:30.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44a7fff640 1 --2- 172.21.15.1:0/3174926226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a8161950 0x7f44a80d2400 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f44a808e540 tx=0x7f449802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f328f121bd3de563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44a77fe640 1 -- 172.21.15.1:0/3174926226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498037070 con 0x7f44a8161950 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44a77fe640 1 -- 172.21.15.1:0/3174926226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498032440 con 0x7f44a8161950 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44a77fe640 1 -- 172.21.15.1:0/3174926226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498032810 con 0x7f44a8161950 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44ae714640 1 -- 172.21.15.1:0/3174926226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a8161950 msgr2=0x7f44a80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44ae714640 1 --2- 172.21.15.1:0/3174926226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a8161950 0x7f44a80d2400 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f44a808e540 tx=0x7f449802a270 comp rx=0 tx=0).stop 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44ae714640 1 -- 172.21.15.1:0/3174926226 shutdown_connections 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44ae714640 1 --2- 172.21.15.1:0/3174926226 >> v2:172.21.15.6:3300/0 conn(0x7f44a80d0f80 0x7f44a80d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.201+0000 7f44ae714640 1 --2- 172.21.15.1:0/3174926226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a8161950 0x7f44a80d2400 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.202+0000 7f44ae714640 1 -- 172.21.15.1:0/3174926226 >> 172.21.15.1:0/3174926226 conn(0x7f44a80d3e20 msgr2=0x7f44a80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.202+0000 7f44ae714640 1 -- 172.21.15.1:0/3174926226 shutdown_connections 2024-05-09T06:38:30.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.202+0000 7f44ae714640 1 -- 172.21.15.1:0/3174926226 wait complete. 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 Processor -- start 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 -- start start 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f44a80d0f80 0x7f44a80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44a8161950 0x7f44a80b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80b52d0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80b5150 con 0x7f44a8161950 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.203+0000 7f44ae714640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f44a80b5450 con 0x7f44a80d0f80 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47896/0 (socket says 172.21.15.1:47896) 2024-05-09T06:38:30.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 -- 172.21.15.1:0/829224312 learned_addr learned my addr 172.21.15.1:0/829224312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44a7fff640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f44a80d0f80 0x7f44a80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f449ffff640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44a8161950 0x7f44a80b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 -- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f44a80d0f80 msgr2=0x7f44a80c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f44a80d0f80 0x7f44a80c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 -- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44a8161950 msgr2=0x7f44a80b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44a8161950 0x7f44a80b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 -- 172.21.15.1:0/829224312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80b72d0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44accb1640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f44a0007e70 tx=0x7f44a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0014620 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0019070 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a80b75b0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0014a20 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.204+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a80b7b70 con 0x7f44a80cb2d0 2024-05-09T06:38:30.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.205+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a80752b0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.206+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44a0014bc0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.207+0000 7f44a57fa640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 0x7f448806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.207+0000 7f44a7fff640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 0x7f448806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.207+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6101+0+0 (secure 0 0 0) 0x7f44a008b750 con 0x7f44a80cb2d0 2024-05-09T06:38:30.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.207+0000 7f44a7fff640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 0x7f448806e320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4498034000 tx=0x7f449802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.211+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44a0059920 con 0x7f44a80cb2d0 2024-05-09T06:38:30.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.399+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44a80b31a0 con 0x7f44a80cb2d0 2024-05-09T06:38:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.400+0000 7f44a57fa640 1 -- 172.21.15.1:0/829224312 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16833 (secure 0 0 0) 0x7f44a005dc20 con 0x7f44a80cb2d0 2024-05-09T06:38:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:30.380 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":120,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:29.105412+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:30.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.401+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 msgr2=0x7f448806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 0x7f448806e320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4498034000 tx=0x7f449802e000 comp rx=0 tx=0).stop 2024-05-09T06:38:30.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 msgr2=0x7f44a80b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f44a0007e70 tx=0x7f44a000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 shutdown_connections 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f448806be60 0x7f448806e320 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a80cb2d0 0x7f44a80b8370 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44a8161950 0x7f44a80b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 --2- 172.21.15.1:0/829224312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f44a80d0f80 0x7f44a80c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 >> 172.21.15.1:0/829224312 conn(0x7f44a80d3e20 msgr2=0x7f44a815ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 shutdown_connections 2024-05-09T06:38:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.402+0000 7f44ae714640 1 -- 172.21.15.1:0/829224312 wait complete. 2024-05-09T06:38:30.393 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:38:30.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.520+0000 7f57afca5640 1 Processor -- start 2024-05-09T06:38:30.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.520+0000 7f57afca5640 1 -- start start 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57afca5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57a815c4c0 0x7f57a806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57afca5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f57a80b4de0 0x7f57a815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57afca5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a806cf40 con 0x7f57a80b5260 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57afca5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f57a8156980 con 0x7f57a815c4c0 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57afca5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f57a806d870 con 0x7f57a80b4de0 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57a815c4c0 0x7f57a806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ae242640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f57a80b5260 0x7f57a8070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60250/0 (socket says 172.21.15.1:60250) 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57a815c4c0 0x7f57a806dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47900/0 (socket says 172.21.15.1:47900) 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ae242640 1 -- 172.21.15.1:0/2992134246 learned_addr learned my addr 172.21.15.1:0/2992134246 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ad240640 1 --2- 172.21.15.1:0/2992134246 >> v2:172.21.15.6:3300/0 conn(0x7f57a80b4de0 0x7f57a815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 -- 172.21.15.1:0/2992134246 >> v2:172.21.15.6:3300/0 conn(0x7f57a80b4de0 msgr2=0x7f57a815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 --2- 172.21.15.1:0/2992134246 >> v2:172.21.15.6:3300/0 conn(0x7f57a80b4de0 0x7f57a815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 -- 172.21.15.1:0/2992134246 >> v1:172.21.15.1:6789/0 conn(0x7f57a80b5260 legacy=0x7f57a8070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.521+0000 7f57ada41640 1 -- 172.21.15.1:0/2992134246 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a806d220 con 0x7f57a815c4c0 2024-05-09T06:38:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.522+0000 7f57ada41640 1 --2- 172.21.15.1:0/2992134246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a815c4c0 0x7f57a806dd00 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd190 tx=0x7f57a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce4bd4d26ff2942 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.522+0000 7f57aca3f640 1 -- 172.21.15.1:0/2992134246 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57a815c4c0 2024-05-09T06:38:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.522+0000 7f57aca3f640 1 -- 172.21.15.1:0/2992134246 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a4032440 con 0x7f57a815c4c0 2024-05-09T06:38:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.522+0000 7f57aca3f640 1 -- 172.21.15.1:0/2992134246 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4032810 con 0x7f57a815c4c0 2024-05-09T06:38:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 -- 172.21.15.1:0/2992134246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a815c4c0 msgr2=0x7f57a806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 --2- 172.21.15.1:0/2992134246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a815c4c0 0x7f57a806dd00 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd190 tx=0x7f57a402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 -- 172.21.15.1:0/2992134246 shutdown_connections 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 --2- 172.21.15.1:0/2992134246 >> v2:172.21.15.6:3300/0 conn(0x7f57a80b4de0 0x7f57a815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 --2- 172.21.15.1:0/2992134246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a815c4c0 0x7f57a806dd00 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 -- 172.21.15.1:0/2992134246 >> 172.21.15.1:0/2992134246 conn(0x7f57a80cc4d0 msgr2=0x7f57a80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 -- 172.21.15.1:0/2992134246 shutdown_connections 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.523+0000 7f57afca5640 1 -- 172.21.15.1:0/2992134246 wait complete. 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.524+0000 7f57afca5640 1 Processor -- start 2024-05-09T06:38:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.524+0000 7f57afca5640 1 -- start start 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 0x7f57a81e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f57a815c4c0 0x7f57a81441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80b2dc0 con 0x7f57a80b5260 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80b2c40 con 0x7f57a80b4de0 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57afca5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f57a815cde0 con 0x7f57a815c4c0 2024-05-09T06:38:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47912/0 (socket says 172.21.15.1:47912) 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 -- 172.21.15.1:0/1000707794 learned_addr learned my addr 172.21.15.1:0/1000707794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ada41640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 0x7f57a81e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 -- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f57a815c4c0 msgr2=0x7f57a81441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ae242640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f57a815c4c0 0x7f57a81441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f57a815c4c0 0x7f57a81441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 -- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 msgr2=0x7f57a81e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 0x7f57a81e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 -- 172.21.15.1:0/1000707794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a81e45f0 con 0x7f57a80b5260 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ada41640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 0x7f57a81e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f57ad240640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.525+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800b6c0 con 0x7f57a80b5260 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.526+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798019070 con 0x7f57a80b5260 2024-05-09T06:38:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.526+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800ba80 con 0x7f57a80b5260 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.526+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a81e4890 con 0x7f57a80b5260 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.526+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a81e4e50 con 0x7f57a80b5260 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.526+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a80752b0 con 0x7f57a80b5260 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.527+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f579800bc20 con 0x7f57a80b5260 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.527+0000 7f579e7fc640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.528+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6101+0+0 (secure 0 0 0) 0x7f579808b490 con 0x7f57a80b5260 2024-05-09T06:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.528+0000 7f57ada41640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.528+0000 7f57ada41640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:30.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.532+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5798059ca0 con 0x7f57a80b5260 2024-05-09T06:38:30.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:30.752+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f57a815c1e0 con 0x7f57a80b5260 2024-05-09T06:38:31.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-05-09T06:38:31.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.756+0000 7f579e7fc640 1 -- 172.21.15.1:0/1000707794 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7f57980595a0 con 0x7f57a80b5260 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 msgr2=0x7f578c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).stop 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 msgr2=0x7f57a8143ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dcc0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 shutdown_connections 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f57a815c4c0 0x7f57a81441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a80b5260 0x7f57a8143ca0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 --2- 172.21.15.1:0/1000707794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57a80b4de0 0x7f57a81e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 >> 172.21.15.1:0/1000707794 conn(0x7f57a80cc4d0 msgr2=0x7f57a80af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:31.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.759+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 shutdown_connections 2024-05-09T06:38:31.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.760+0000 7f57afca5640 1 -- 172.21.15.1:0/1000707794 wait complete. 2024-05-09T06:38:31.753 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:38:31.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:38:31.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.886+0000 7fa6cc57e640 1 Processor -- start 2024-05-09T06:38:31.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.886+0000 7fa6cc57e640 1 -- start start 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.887+0000 7fa6cc57e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa6c40d1c70 0x7fa6c415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.887+0000 7fa6cc57e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c4161fe0 con 0x7fa6c415e190 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.887+0000 7fa6cc57e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c41622c0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.887+0000 7fa6cc57e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c415f580 con 0x7fa6c40d1c70 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.887+0000 7fa6ca31a640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa6c415e190 0x7fa6c40d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60252/0 (socket says 172.21.15.1:60252) 2024-05-09T06:38:31.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6ca31a640 1 -- 172.21.15.1:0/265916918 learned_addr learned my addr 172.21.15.1:0/265916918 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6c9b19640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa6c40d1c70 0x7fa6c415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6c9b19640 1 -- 172.21.15.1:0/265916918 >> v1:172.21.15.6:6789/0 conn(0x7fa6c40d45f0 legacy=0x7fa6c40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6c9b19640 1 -- 172.21.15.1:0/265916918 >> v1:172.21.15.1:6789/0 conn(0x7fa6c415e190 legacy=0x7fa6c40d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6c9b19640 1 -- 172.21.15.1:0/265916918 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c41625a0 con 0x7fa6c40d1c70 2024-05-09T06:38:31.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.888+0000 7fa6c9318640 1 -- 172.21.15.1:0/265916918 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794026485 0 0) 0x7fa6c4161fe0 con 0x7fa6c415e190 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6c9b19640 1 --2- 172.21.15.1:0/265916918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d1c70 0x7fa6c415f780 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513a034faa77fd65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6c9318640 1 -- 172.21.15.1:0/265916918 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037070 con 0x7fa6c40d1c70 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6c9318640 1 -- 172.21.15.1:0/265916918 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8032440 con 0x7fa6c40d1c70 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6c9318640 1 -- 172.21.15.1:0/265916918 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8032810 con 0x7fa6c40d1c70 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 -- 172.21.15.1:0/265916918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d1c70 msgr2=0x7fa6c415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/265916918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d1c70 0x7fa6c415f780 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 -- 172.21.15.1:0/265916918 shutdown_connections 2024-05-09T06:38:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/265916918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d1c70 0x7fa6c415f780 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 -- 172.21.15.1:0/265916918 >> 172.21.15.1:0/265916918 conn(0x7fa6c4073340 msgr2=0x7fa6c409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 -- 172.21.15.1:0/265916918 shutdown_connections 2024-05-09T06:38:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.889+0000 7fa6cc57e640 1 -- 172.21.15.1:0/265916918 wait complete. 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 Processor -- start 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 -- start start 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6c40d1c70 0x7fa6c40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c415e190 0x7fa6c40c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c40b56d0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4161c60 con 0x7fa6c415e190 2024-05-09T06:38:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.891+0000 7fa6cc57e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c40b5850 con 0x7fa6c40d1c70 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47938/0 (socket says 172.21.15.1:47938) 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 -- 172.21.15.1:0/2128778190 learned_addr learned my addr 172.21.15.1:0/2128778190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6ca31a640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6c40d1c70 0x7fa6c40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6cab1b640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c415e190 0x7fa6c40c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 -- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6c40d1c70 msgr2=0x7fa6c40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6c40d1c70 0x7fa6c40ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 -- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c415e190 msgr2=0x7fa6c40c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c415e190 0x7fa6c40c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 -- 172.21.15.1:0/2128778190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4159ae0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c9b19640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037040 con 0x7fa6c40d45f0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8004250 con 0x7fa6c40d45f0 2024-05-09T06:38:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8032e20 con 0x7fa6c40d45f0 2024-05-09T06:38:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c414b3b0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.892+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c414b970 con 0x7fa6c40d45f0 2024-05-09T06:38:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.893+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c40752b0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.894+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6b8030770 con 0x7fa6c40d45f0 2024-05-09T06:38:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.894+0000 7fa6c2ffd640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 0x7fa6b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.894+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fa6b80ac1d0 con 0x7fa6c40d45f0 2024-05-09T06:38:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.895+0000 7fa6ca31a640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 0x7fa6b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.895+0000 7fa6ca31a640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 0x7fa6b006e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40012e0 tx=0x7fa6b4001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:31.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:31.899+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6b807a3a0 con 0x7fa6c40d45f0 2024-05-09T06:38:32.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.123+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa6c408e1c0 con 0x7fa6c40d45f0 2024-05-09T06:38:32.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.757+0000 7fa6c2ffd640 1 -- 172.21.15.1:0/2128778190 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7fa6b807e6a0 con 0x7fa6c40d45f0 2024-05-09T06:38:32.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 msgr2=0x7fa6b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:32.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 0x7fa6b006e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40012e0 tx=0x7fa6b4001630 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 msgr2=0x7fa6c40c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b80046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 shutdown_connections 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa6b006be30 0x7fa6b006e2f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c415e190 0x7fa6c40c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6c40d45f0 0x7fa6c40c7e80 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 --2- 172.21.15.1:0/2128778190 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6c40d1c70 0x7fa6c40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 >> 172.21.15.1:0/2128778190 conn(0x7fa6c4073340 msgr2=0x7fa6c406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 shutdown_connections 2024-05-09T06:38:32.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.760+0000 7fa6cc57e640 1 -- 172.21.15.1:0/2128778190 wait complete. 2024-05-09T06:38:32.753 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:38:32.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:32.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.883+0000 7fd9ecd3b640 1 Processor -- start 2024-05-09T06:38:32.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.883+0000 7fd9ecd3b640 1 -- start start 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd9e806e6d0 0x7fd9e806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd9e80cdab0 0x7fd9e815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e80ce0e0 con 0x7fd9e806e6d0 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e80ce3c0 con 0x7fd9e80cdab0 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9ecd3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e80ce6a0 con 0x7fd9e80cf180 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd9e806e6d0 0x7fd9e806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd9e806e6d0 0x7fd9e806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47942/0 (socket says 172.21.15.1:47942) 2024-05-09T06:38:32.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 -- 172.21.15.1:0/3524377784 learned_addr learned my addr 172.21.15.1:0/3524377784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e6d9d640 1 --2- 172.21.15.1:0/3524377784 >> v2:172.21.15.6:3300/0 conn(0x7fd9e80cdab0 0x7fd9e815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9dffff640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 -- 172.21.15.1:0/3524377784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 msgr2=0x7fd9e80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e80cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 -- 172.21.15.1:0/3524377784 >> v2:172.21.15.6:3300/0 conn(0x7fd9e80cdab0 msgr2=0x7fd9e815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 --2- 172.21.15.1:0/3524377784 >> v2:172.21.15.6:3300/0 conn(0x7fd9e80cdab0 0x7fd9e815e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.884+0000 7fd9e659c640 1 -- 172.21.15.1:0/3524377784 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e80ce980 con 0x7fd9e806e6d0 2024-05-09T06:38:32.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.885+0000 7fd9e659c640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e806e6d0 0x7fd9e806eab0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b16f0 tx=0x7fd9d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4029edeb80df90fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:32.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.885+0000 7fd9e5d9b640 1 -- 172.21.15.1:0/3524377784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8037070 con 0x7fd9e806e6d0 2024-05-09T06:38:32.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.885+0000 7fd9e5d9b640 1 -- 172.21.15.1:0/3524377784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d8032440 con 0x7fd9e806e6d0 2024-05-09T06:38:32.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.885+0000 7fd9e5d9b640 1 -- 172.21.15.1:0/3524377784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8032810 con 0x7fd9e806e6d0 2024-05-09T06:38:32.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/3524377784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e806e6d0 msgr2=0x7fd9e806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e806e6d0 0x7fd9e806eab0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b16f0 tx=0x7fd9d802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/3524377784 shutdown_connections 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/3524377784 >> v2:172.21.15.6:3300/0 conn(0x7fd9e80cdab0 0x7fd9e815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/3524377784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e806e6d0 0x7fd9e806eab0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/3524377784 >> 172.21.15.1:0/3524377784 conn(0x7fd9e80cc4b0 msgr2=0x7fd9e80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/3524377784 shutdown_connections 2024-05-09T06:38:32.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.886+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/3524377784 wait complete. 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.887+0000 7fd9ecd3b640 1 Processor -- start 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.887+0000 7fd9ecd3b640 1 -- start start 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9e806e6d0 0x7fd9e80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e8070ed0 con 0x7fd9e80cdab0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e806eff0 con 0x7fd9e806e6d0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9ecd3b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e81447c0 con 0x7fd9e80cf180 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47956/0 (socket says 172.21.15.1:47956) 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9dffff640 1 -- 172.21.15.1:0/1609584651 learned_addr learned my addr 172.21.15.1:0/1609584651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9e6d9d640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.888+0000 7fd9e659c640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9e806e6d0 0x7fd9e80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 -- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 msgr2=0x7fd9e815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 -- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9e806e6d0 msgr2=0x7fd9e80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9e806e6d0 0x7fd9e80ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 -- 172.21.15.1:0/1609584651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e815c3b0 con 0x7fd9e80cdab0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9dffff640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000d620 tx=0x7fd9d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:32.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000b620 con 0x7fd9e80cdab0 2024-05-09T06:38:32.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d0019070 con 0x7fd9e80cdab0 2024-05-09T06:38:32.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e815c690 con 0x7fd9e80cdab0 2024-05-09T06:38:32.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000ba00 con 0x7fd9e80cdab0 2024-05-09T06:38:32.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.889+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e815cc50 con 0x7fd9e80cdab0 2024-05-09T06:38:32.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.890+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e80752b0 con 0x7fd9e80cdab0 2024-05-09T06:38:32.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.890+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9d000bba0 con 0x7fd9e80cdab0 2024-05-09T06:38:32.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.891+0000 7fd9deffd640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 0x7fd9c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:32.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.891+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fd9d008b430 con 0x7fd9e80cdab0 2024-05-09T06:38:32.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.891+0000 7fd9e659c640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 0x7fd9c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:32.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.892+0000 7fd9e659c640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 0x7fd9c806e2f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8034000 tx=0x7fd9d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:32.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:32.896+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9d0059c50 con 0x7fd9e80cdab0 2024-05-09T06:38:33.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.115+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9e806cf50 con 0x7fd9e80cdab0 2024-05-09T06:38:33.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.116+0000 7fd9deffd640 1 -- 172.21.15.1:0/1609584651 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5171 (secure 0 0 0) 0x7fd9d0059550 con 0x7fd9e80cdab0 2024-05-09T06:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:32.125327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9636":{"gid":9636,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6837/1060593353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1060593353},{"type":"v1","addr":"172.21.15.1:6837","nonce":1060593353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9636,"qdb_cluster":[9636]},"id":8}]} 2024-05-09T06:38:33.097 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 99 2024-05-09T06:38:33.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 msgr2=0x7fd9c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 0x7fd9c806e2f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8034000 tx=0x7fd9d802e000 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 msgr2=0x7fd9e815d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000d620 tx=0x7fd9d000dae0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 shutdown_connections 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd9c806be30 0x7fd9c806e2f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd9e80cf180 0x7fd9e815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9e80cdab0 0x7fd9e815d450 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 --2- 172.21.15.1:0/1609584651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9e806e6d0 0x7fd9e80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 >> 172.21.15.1:0/1609584651 conn(0x7fd9e80cc4b0 msgr2=0x7fd9e8142520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 shutdown_connections 2024-05-09T06:38:33.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.118+0000 7fd9ecd3b640 1 -- 172.21.15.1:0/1609584651 wait complete. 2024-05-09T06:38:33.110 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:33.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.234+0000 7f01028da640 1 Processor -- start 2024-05-09T06:38:33.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.234+0000 7f01028da640 1 -- start start 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f01028da640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 0x7f00fc06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f01028da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f01028da640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc0ce0e0 con 0x7f00fc0cdab0 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f01028da640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc0ce3c0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f01028da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc0ce6a0 con 0x7f00fc0cf180 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fbfff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 0x7f00fc06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:52014/0 (socket says 172.21.15.1:52014) 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 -- 172.21.15.1:0/457897798 learned_addr learned my addr 172.21.15.1:0/457897798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 -- 172.21.15.1:0/457897798 >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 msgr2=0x7f00fc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 --2- 172.21.15.1:0/457897798 >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 0x7f00fc06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 -- 172.21.15.1:0/457897798 >> v1:172.21.15.6:6789/0 conn(0x7f00fc0cdab0 legacy=0x7f00fc15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fb7fe640 1 -- 172.21.15.1:0/457897798 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc0ce980 con 0x7f00fc0cf180 2024-05-09T06:38:33.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.235+0000 7f00fbfff640 1 --2- 172.21.15.1:0/457897798 >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 0x7f00fc06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.236+0000 7f00fb7fe640 1 --2- 172.21.15.1:0/457897798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009870 tx=0x7f00f002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db0547bd0343ff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.236+0000 7f00faffd640 1 -- 172.21.15.1:0/457897798 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0037070 con 0x7f00fc0cf180 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.236+0000 7f00faffd640 1 -- 172.21.15.1:0/457897798 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0032440 con 0x7f00fc0cf180 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.236+0000 7f00faffd640 1 -- 172.21.15.1:0/457897798 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0032810 con 0x7f00fc0cf180 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 -- 172.21.15.1:0/457897798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 msgr2=0x7f00fc0cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 --2- 172.21.15.1:0/457897798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009870 tx=0x7f00f002a270 comp rx=0 tx=0).stop 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 -- 172.21.15.1:0/457897798 shutdown_connections 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 --2- 172.21.15.1:0/457897798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cf180 0x7f00fc0cd570 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 --2- 172.21.15.1:0/457897798 >> v2:172.21.15.6:3300/0 conn(0x7f00fc06e6d0 0x7f00fc06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 -- 172.21.15.1:0/457897798 >> 172.21.15.1:0/457897798 conn(0x7f00fc0cc4b0 msgr2=0x7f00fc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 -- 172.21.15.1:0/457897798 shutdown_connections 2024-05-09T06:38:33.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.237+0000 7f01028da640 1 -- 172.21.15.1:0/457897798 wait complete. 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.238+0000 7f01028da640 1 Processor -- start 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.238+0000 7f01028da640 1 -- start start 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cdab0 0x7f00fc13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00fc0cf180 0x7f00fc13cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc070ed0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc06eff0 con 0x7f00fc0cf180 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f01028da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc1447c0 con 0x7f00fc0cdab0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47962/0 (socket says 172.21.15.1:47962) 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f00fbfff640 1 -- 172.21.15.1:0/1700895174 learned_addr learned my addr 172.21.15.1:0/1700895174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f00fb7fe640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cdab0 0x7f00fc13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.239+0000 7f0100e77640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00fc0cf180 0x7f00fc13cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 -- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cdab0 msgr2=0x7f00fc13ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cdab0 0x7f00fc13ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 -- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00fc0cf180 msgr2=0x7f00fc13cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00fc0cf180 0x7f00fc13cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 -- 172.21.15.1:0/1700895174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc13d600 con 0x7f00fc06e6d0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00fbfff640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00e990 tx=0x7f00ec00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec017070 con 0x7f00fc06e6d0 2024-05-09T06:38:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec00c9e0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc13d8e0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00cdb0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.240+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc13dea0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.241+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc0752b0 con 0x7f00fc06e6d0 2024-05-09T06:38:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.242+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00ec01a070 con 0x7f00fc06e6d0 2024-05-09T06:38:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.242+0000 7f00f8ff9640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 0x7f00e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.242+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f00ec08b190 con 0x7f00fc06e6d0 2024-05-09T06:38:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.242+0000 7f00fb7fe640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 0x7f00e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.242+0000 7f00fb7fe640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 0x7f00e806e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f00f0034000 tx=0x7f00f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.246+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00ec059360 con 0x7f00fc06e6d0 2024-05-09T06:38:33.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.429+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00fc06cf50 con 0x7f00fc06e6d0 2024-05-09T06:38:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.430+0000 7f00f8ff9640 1 -- 172.21.15.1:0/1700895174 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16820 (secure 0 0 0) 0x7f00ec05d660 con 0x7f00fc06e6d0 2024-05-09T06:38:33.411 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:33.411 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":121,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:30.738835+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:33.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.432+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 msgr2=0x7f00e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.432+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 0x7f00e806e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f00f0034000 tx=0x7f00f002e000 comp rx=0 tx=0).stop 2024-05-09T06:38:33.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.432+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 msgr2=0x7f00fc15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.432+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00e990 tx=0x7f00ec00ee50 comp rx=0 tx=0).stop 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 shutdown_connections 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f00e806be30 0x7f00e806e2f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00fc0cf180 0x7f00fc13cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f00fc0cdab0 0x7f00fc13ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 --2- 172.21.15.1:0/1700895174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00fc06e6d0 0x7f00fc15a770 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 >> 172.21.15.1:0/1700895174 conn(0x7f00fc0cc4b0 msgr2=0x7f00fc0b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 shutdown_connections 2024-05-09T06:38:33.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.433+0000 7f01028da640 1 -- 172.21.15.1:0/1700895174 wait complete. 2024-05-09T06:38:33.424 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:38:33.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:38:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.547+0000 7f9acd0da640 1 Processor -- start 2024-05-09T06:38:33.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 -- start start 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9ac806e6d0 0x7f9ac806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 0x7f9ac80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac80ce0e0 con 0x7f9ac80cf180 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac80ce3c0 con 0x7f9ac806e6d0 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.548+0000 7f9acd0da640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac80ce6a0 con 0x7f9ac80b3300 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9ac806e6d0 0x7f9ac806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9ac806e6d0 0x7f9ac806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47972/0 (socket says 172.21.15.1:47972) 2024-05-09T06:38:33.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 -- 172.21.15.1:0/3255359299 learned_addr learned my addr 172.21.15.1:0/3255359299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac759e640 1 --2- 172.21.15.1:0/3255359299 >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 0x7f9ac80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 -- 172.21.15.1:0/3255359299 >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 msgr2=0x7f9ac80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/3255359299 >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 0x7f9ac80b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 -- 172.21.15.1:0/3255359299 >> v1:172.21.15.6:6789/0 conn(0x7f9ac80cf180 legacy=0x7f9ac80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac6d9d640 1 -- 172.21.15.1:0/3255359299 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac80ce980 con 0x7f9ac806e6d0 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.549+0000 7f9ac759e640 1 --2- 172.21.15.1:0/3255359299 >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 0x7f9ac80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/3255359299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac806e6d0 0x7f9ac806eab0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80b16f0 tx=0x7f9ac002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f90d496fca883be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9ac5d9b640 1 -- 172.21.15.1:0/3255359299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0036070 con 0x7f9ac806e6d0 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9ac5d9b640 1 -- 172.21.15.1:0/3255359299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac0031600 con 0x7f9ac806e6d0 2024-05-09T06:38:33.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9ac5d9b640 1 -- 172.21.15.1:0/3255359299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00319d0 con 0x7f9ac806e6d0 2024-05-09T06:38:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 -- 172.21.15.1:0/3255359299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac806e6d0 msgr2=0x7f9ac806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 --2- 172.21.15.1:0/3255359299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac806e6d0 0x7f9ac806eab0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80b16f0 tx=0x7f9ac002a730 comp rx=0 tx=0).stop 2024-05-09T06:38:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 -- 172.21.15.1:0/3255359299 shutdown_connections 2024-05-09T06:38:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 --2- 172.21.15.1:0/3255359299 >> v2:172.21.15.6:3300/0 conn(0x7f9ac80b3300 0x7f9ac80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 --2- 172.21.15.1:0/3255359299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac806e6d0 0x7f9ac806eab0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 -- 172.21.15.1:0/3255359299 >> 172.21.15.1:0/3255359299 conn(0x7f9ac80cc4b0 msgr2=0x7f9ac80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 -- 172.21.15.1:0/3255359299 shutdown_connections 2024-05-09T06:38:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.550+0000 7f9acd0da640 1 -- 172.21.15.1:0/3255359299 wait complete. 2024-05-09T06:38:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.552+0000 7f9acd0da640 1 Processor -- start 2024-05-09T06:38:33.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.552+0000 7f9acd0da640 1 -- start start 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 0x7f9ac813a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac80b3300 0x7f9ac815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8070ed0 con 0x7f9ac80cf180 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac806eff0 con 0x7f9ac80b3300 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9acd0da640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac81447c0 con 0x7f9ac806e6d0 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47986/0 (socket says 172.21.15.1:47986) 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 -- 172.21.15.1:0/1915141717 learned_addr learned my addr 172.21.15.1:0/1915141717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac659c640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac80b3300 0x7f9ac815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 0x7f9ac813a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 -- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 msgr2=0x7f9ac813a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 0x7f9ac813a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 -- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac80b3300 msgr2=0x7f9ac815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac80b3300 0x7f9ac815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 -- 172.21.15.1:0/1915141717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac815ac20 con 0x7f9ac80cf180 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 0x7f9ac813a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.553+0000 7f9ac759e640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800c820 tx=0x7f9ab800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.554+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8015070 con 0x7f9ac80cf180 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.554+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab8007a30 con 0x7f9ac80cf180 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.554+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac8139240 con 0x7f9ac80cf180 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.554+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8007e20 con 0x7f9ac80cf180 2024-05-09T06:38:33.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.554+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac8139800 con 0x7f9ac80cf180 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.555+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac80752b0 con 0x7f9ac80cf180 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.555+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ab8018070 con 0x7f9ac80cf180 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.556+0000 7f9abf7fe640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 0x7f9aa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.556+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 0x7f9aa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.556+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f9ab808b050 con 0x7f9ac80cf180 2024-05-09T06:38:33.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.556+0000 7f9ac6d9d640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 0x7f9aa806e2f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0033000 tx=0x7f9ac0009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.560+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9ac815ac20 con 0x7f9ac80cf180 2024-05-09T06:38:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.740+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ac806cf50 con 0x7f9aa806be30 2024-05-09T06:38:33.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.746+0000 7f9abf7fe640 1 -- 172.21.15.1:0/1915141717 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f9ac806cf50 con 0x7f9aa806be30 2024-05-09T06:38:33.729 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:38:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 msgr2=0x7f9aa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 0x7f9aa806e2f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0033000 tx=0x7f9ac0009000 comp rx=0 tx=0).stop 2024-05-09T06:38:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 msgr2=0x7f9ac813a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800c820 tx=0x7f9ab800cce0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 shutdown_connections 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9aa806be30 0x7f9aa806e2f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ac80cf180 0x7f9ac813a920 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ac80b3300 0x7f9ac815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 --2- 172.21.15.1:0/1915141717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ac806e6d0 0x7f9ac813a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 >> 172.21.15.1:0/1915141717 conn(0x7f9ac80cc4b0 msgr2=0x7f9ac8140c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 shutdown_connections 2024-05-09T06:38:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.749+0000 7f9acd0da640 1 -- 172.21.15.1:0/1915141717 wait complete. 2024-05-09T06:38:33.741 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-05-09T06:38:33.221902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270792,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703928,"statfs":{"total":773094113280,"available":772816822272,"internally_reserved":0,"allocated":13312000,"data_stored":6831036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.497016"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743791+0000","last_change":"2024-05-09T06:38:29.760784+0000","last_active":"2024-05-09T06:38:30.743791+0000","last_peered":"2024-05-09T06:38:30.743791+0000","last_clean":"2024-05-09T06:38:30.743791+0000","last_became_active":"2024-05-09T06:38:29.760456+0000","last_became_peered":"2024-05-09T06:38:29.760456+0000","last_unstale":"2024-05-09T06:38:30.743791+0000","last_undegraded":"2024-05-09T06:38:30.743791+0000","last_fullsized":"2024-05-09T06:38:30.743791+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246661+0000","last_change":"2024-05-09T06:38:29.769260+0000","last_active":"2024-05-09T06:38:31.246661+0000","last_peered":"2024-05-09T06:38:31.246661+0000","last_clean":"2024-05-09T06:38:31.246661+0000","last_became_active":"2024-05-09T06:38:29.768980+0000","last_became_peered":"2024-05-09T06:38:29.768980+0000","last_unstale":"2024-05-09T06:38:31.246661+0000","last_undegraded":"2024-05-09T06:38:31.246661+0000","last_fullsized":"2024-05-09T06:38:31.246661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:06.222662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246633+0000","last_change":"2024-05-09T06:38:29.759714+0000","last_active":"2024-05-09T06:38:31.246633+0000","last_peered":"2024-05-09T06:38:31.246633+0000","last_clean":"2024-05-09T06:38:31.246633+0000","last_became_active":"2024-05-09T06:38:29.759442+0000","last_became_peered":"2024-05-09T06:38:29.759442+0000","last_unstale":"2024-05-09T06:38:31.246633+0000","last_undegraded":"2024-05-09T06:38:31.246633+0000","last_fullsized":"2024-05-09T06:38:31.246633+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:48.150075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901441+0000","last_change":"2024-05-09T06:38:29.765065+0000","last_active":"2024-05-09T06:38:30.901441+0000","last_peered":"2024-05-09T06:38:30.901441+0000","last_clean":"2024-05-09T06:38:30.901441+0000","last_became_active":"2024-05-09T06:38:29.764776+0000","last_became_peered":"2024-05-09T06:38:29.764776+0000","last_unstale":"2024-05-09T06:38:30.901441+0000","last_undegraded":"2024-05-09T06:38:30.901441+0000","last_fullsized":"2024-05-09T06:38:30.901441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743304+0000","last_change":"2024-05-09T06:38:29.758633+0000","last_active":"2024-05-09T06:38:30.743304+0000","last_peered":"2024-05-09T06:38:30.743304+0000","last_clean":"2024-05-09T06:38:30.743304+0000","last_became_active":"2024-05-09T06:38:29.758274+0000","last_became_peered":"2024-05-09T06:38:29.758274+0000","last_unstale":"2024-05-09T06:38:30.743304+0000","last_undegraded":"2024-05-09T06:38:30.743304+0000","last_fullsized":"2024-05-09T06:38:30.743304+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:25.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742077+0000","last_change":"2024-05-09T06:38:29.754566+0000","last_active":"2024-05-09T06:38:30.742077+0000","last_peered":"2024-05-09T06:38:30.742077+0000","last_clean":"2024-05-09T06:38:30.742077+0000","last_became_active":"2024-05-09T06:38:29.754314+0000","last_became_peered":"2024-05-09T06:38:29.754314+0000","last_unstale":"2024-05-09T06:38:30.742077+0000","last_undegraded":"2024-05-09T06:38:30.742077+0000","last_fullsized":"2024-05-09T06:38:30.742077+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:17.620153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741956+0000","last_change":"2024-05-09T06:38:29.758534+0000","last_active":"2024-05-09T06:38:30.741956+0000","last_peered":"2024-05-09T06:38:30.741956+0000","last_clean":"2024-05-09T06:38:30.741956+0000","last_became_active":"2024-05-09T06:38:29.758263+0000","last_became_peered":"2024-05-09T06:38:29.758263+0000","last_unstale":"2024-05-09T06:38:30.741956+0000","last_undegraded":"2024-05-09T06:38:30.741956+0000","last_fullsized":"2024-05-09T06:38:30.741956+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:35.903972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247080+0000","last_change":"2024-05-09T06:38:29.761460+0000","last_active":"2024-05-09T06:38:31.247080+0000","last_peered":"2024-05-09T06:38:31.247080+0000","last_clean":"2024-05-09T06:38:31.247080+0000","last_became_active":"2024-05-09T06:38:29.761210+0000","last_became_peered":"2024-05-09T06:38:29.761210+0000","last_unstale":"2024-05-09T06:38:31.247080+0000","last_undegraded":"2024-05-09T06:38:31.247080+0000","last_fullsized":"2024-05-09T06:38:31.247080+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.802863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901605+0000","last_change":"2024-05-09T06:38:29.772536+0000","last_active":"2024-05-09T06:38:30.901605+0000","last_peered":"2024-05-09T06:38:30.901605+0000","last_clean":"2024-05-09T06:38:30.901605+0000","last_became_active":"2024-05-09T06:38:29.772205+0000","last_became_peered":"2024-05-09T06:38:29.772205+0000","last_unstale":"2024-05-09T06:38:30.901605+0000","last_undegraded":"2024-05-09T06:38:30.901605+0000","last_fullsized":"2024-05-09T06:38:30.901605+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245911+0000","last_change":"2024-05-09T06:38:29.751002+0000","last_active":"2024-05-09T06:38:31.245911+0000","last_peered":"2024-05-09T06:38:31.245911+0000","last_clean":"2024-05-09T06:38:31.245911+0000","last_became_active":"2024-05-09T06:38:29.750625+0000","last_became_peered":"2024-05-09T06:38:29.750625+0000","last_unstale":"2024-05-09T06:38:31.245911+0000","last_undegraded":"2024-05-09T06:38:31.245911+0000","last_fullsized":"2024-05-09T06:38:31.245911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743504+0000","last_change":"2024-05-09T06:38:29.749599+0000","last_active":"2024-05-09T06:38:30.743504+0000","last_peered":"2024-05-09T06:38:30.743504+0000","last_clean":"2024-05-09T06:38:30.743504+0000","last_became_active":"2024-05-09T06:38:29.749265+0000","last_became_peered":"2024-05-09T06:38:29.749265+0000","last_unstale":"2024-05-09T06:38:30.743504+0000","last_undegraded":"2024-05-09T06:38:30.743504+0000","last_fullsized":"2024-05-09T06:38:30.743504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742557+0000","last_change":"2024-05-09T06:38:29.759488+0000","last_active":"2024-05-09T06:38:30.742557+0000","last_peered":"2024-05-09T06:38:30.742557+0000","last_clean":"2024-05-09T06:38:30.742557+0000","last_became_active":"2024-05-09T06:38:29.759030+0000","last_became_peered":"2024-05-09T06:38:29.759030+0000","last_unstale":"2024-05-09T06:38:30.742557+0000","last_undegraded":"2024-05-09T06:38:30.742557+0000","last_fullsized":"2024-05-09T06:38:30.742557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:05.591808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246125+0000","last_change":"2024-05-09T06:38:29.756266+0000","last_active":"2024-05-09T06:38:31.246125+0000","last_peered":"2024-05-09T06:38:31.246125+0000","last_clean":"2024-05-09T06:38:31.246125+0000","last_became_active":"2024-05-09T06:38:29.755981+0000","last_became_peered":"2024-05-09T06:38:29.755981+0000","last_unstale":"2024-05-09T06:38:31.246125+0000","last_undegraded":"2024-05-09T06:38:31.246125+0000","last_fullsized":"2024-05-09T06:38:31.246125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246317+0000","last_change":"2024-05-09T06:38:29.758946+0000","last_active":"2024-05-09T06:38:31.246317+0000","last_peered":"2024-05-09T06:38:31.246317+0000","last_clean":"2024-05-09T06:38:31.246317+0000","last_became_active":"2024-05-09T06:38:29.758701+0000","last_became_peered":"2024-05-09T06:38:29.758701+0000","last_unstale":"2024-05-09T06:38:31.246317+0000","last_undegraded":"2024-05-09T06:38:31.246317+0000","last_fullsized":"2024-05-09T06:38:31.246317+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:56.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743234+0000","last_change":"2024-05-09T06:38:29.756392+0000","last_active":"2024-05-09T06:38:30.743234+0000","last_peered":"2024-05-09T06:38:30.743234+0000","last_clean":"2024-05-09T06:38:30.743234+0000","last_became_active":"2024-05-09T06:38:29.755337+0000","last_became_peered":"2024-05-09T06:38:29.755337+0000","last_unstale":"2024-05-09T06:38:30.743234+0000","last_undegraded":"2024-05-09T06:38:30.743234+0000","last_fullsized":"2024-05-09T06:38:30.743234+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:53.182411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741951+0000","last_change":"2024-05-09T06:38:29.757752+0000","last_active":"2024-05-09T06:38:30.741951+0000","last_peered":"2024-05-09T06:38:30.741951+0000","last_clean":"2024-05-09T06:38:30.741951+0000","last_became_active":"2024-05-09T06:38:29.757391+0000","last_became_peered":"2024-05-09T06:38:29.757391+0000","last_unstale":"2024-05-09T06:38:30.741951+0000","last_undegraded":"2024-05-09T06:38:30.741951+0000","last_fullsized":"2024-05-09T06:38:30.741951+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:01.176437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245939+0000","last_change":"2024-05-09T06:38:29.756836+0000","last_active":"2024-05-09T06:38:31.245939+0000","last_peered":"2024-05-09T06:38:31.245939+0000","last_clean":"2024-05-09T06:38:31.245939+0000","last_became_active":"2024-05-09T06:38:29.756583+0000","last_became_peered":"2024-05-09T06:38:29.756583+0000","last_unstale":"2024-05-09T06:38:31.245939+0000","last_undegraded":"2024-05-09T06:38:31.245939+0000","last_fullsized":"2024-05-09T06:38:31.245939+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246805+0000","last_change":"2024-05-09T06:38:29.769182+0000","last_active":"2024-05-09T06:38:31.246805+0000","last_peered":"2024-05-09T06:38:31.246805+0000","last_clean":"2024-05-09T06:38:31.246805+0000","last_became_active":"2024-05-09T06:38:29.768889+0000","last_became_peered":"2024-05-09T06:38:29.768889+0000","last_unstale":"2024-05-09T06:38:31.246805+0000","last_undegraded":"2024-05-09T06:38:31.246805+0000","last_fullsized":"2024-05-09T06:38:31.246805+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:46.068757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757312+0000","last_change":"2024-05-09T06:38:29.764995+0000","last_active":"2024-05-09T06:38:30.757312+0000","last_peered":"2024-05-09T06:38:30.757312+0000","last_clean":"2024-05-09T06:38:30.757312+0000","last_became_active":"2024-05-09T06:38:29.764704+0000","last_became_peered":"2024-05-09T06:38:29.764704+0000","last_unstale":"2024-05-09T06:38:30.757312+0000","last_undegraded":"2024-05-09T06:38:30.757312+0000","last_fullsized":"2024-05-09T06:38:30.757312+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246668+0000","last_change":"2024-05-09T06:38:29.746800+0000","last_active":"2024-05-09T06:38:31.246668+0000","last_peered":"2024-05-09T06:38:31.246668+0000","last_clean":"2024-05-09T06:38:31.246668+0000","last_became_active":"2024-05-09T06:38:29.746512+0000","last_became_peered":"2024-05-09T06:38:29.746512+0000","last_unstale":"2024-05-09T06:38:31.246668+0000","last_undegraded":"2024-05-09T06:38:31.246668+0000","last_fullsized":"2024-05-09T06:38:31.246668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:34.930379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901528+0000","last_change":"2024-05-09T06:38:29.761604+0000","last_active":"2024-05-09T06:38:30.901528+0000","last_peered":"2024-05-09T06:38:30.901528+0000","last_clean":"2024-05-09T06:38:30.901528+0000","last_became_active":"2024-05-09T06:38:29.761301+0000","last_became_peered":"2024-05-09T06:38:29.761301+0000","last_unstale":"2024-05-09T06:38:30.901528+0000","last_undegraded":"2024-05-09T06:38:30.901528+0000","last_fullsized":"2024-05-09T06:38:30.901528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743735+0000","last_change":"2024-05-09T06:38:29.769275+0000","last_active":"2024-05-09T06:38:30.743735+0000","last_peered":"2024-05-09T06:38:30.743735+0000","last_clean":"2024-05-09T06:38:30.743735+0000","last_became_active":"2024-05-09T06:38:29.769036+0000","last_became_peered":"2024-05-09T06:38:29.769036+0000","last_unstale":"2024-05-09T06:38:30.743735+0000","last_undegraded":"2024-05-09T06:38:30.743735+0000","last_fullsized":"2024-05-09T06:38:30.743735+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901252+0000","last_change":"2024-05-09T06:38:29.771176+0000","last_active":"2024-05-09T06:38:30.901252+0000","last_peered":"2024-05-09T06:38:30.901252+0000","last_clean":"2024-05-09T06:38:30.901252+0000","last_became_active":"2024-05-09T06:38:29.770832+0000","last_became_peered":"2024-05-09T06:38:29.770832+0000","last_unstale":"2024-05-09T06:38:30.901252+0000","last_undegraded":"2024-05-09T06:38:30.901252+0000","last_fullsized":"2024-05-09T06:38:30.901252+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783361+0000","last_change":"2024-05-09T06:38:29.760008+0000","last_active":"2024-05-09T06:38:30.783361+0000","last_peered":"2024-05-09T06:38:30.783361+0000","last_clean":"2024-05-09T06:38:30.783361+0000","last_became_active":"2024-05-09T06:38:29.759673+0000","last_became_peered":"2024-05-09T06:38:29.759673+0000","last_unstale":"2024-05-09T06:38:30.783361+0000","last_undegraded":"2024-05-09T06:38:30.783361+0000","last_fullsized":"2024-05-09T06:38:30.783361+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783832+0000","last_change":"2024-05-09T06:38:29.750626+0000","last_active":"2024-05-09T06:38:30.783832+0000","last_peered":"2024-05-09T06:38:30.783832+0000","last_clean":"2024-05-09T06:38:30.783832+0000","last_became_active":"2024-05-09T06:38:29.748935+0000","last_became_peered":"2024-05-09T06:38:29.748935+0000","last_unstale":"2024-05-09T06:38:30.783832+0000","last_undegraded":"2024-05-09T06:38:30.783832+0000","last_fullsized":"2024-05-09T06:38:30.783832+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.394253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246887+0000","last_change":"2024-05-09T06:38:29.751749+0000","last_active":"2024-05-09T06:38:31.246887+0000","last_peered":"2024-05-09T06:38:31.246887+0000","last_clean":"2024-05-09T06:38:31.246887+0000","last_became_active":"2024-05-09T06:38:29.751470+0000","last_became_peered":"2024-05-09T06:38:29.751470+0000","last_unstale":"2024-05-09T06:38:31.246887+0000","last_undegraded":"2024-05-09T06:38:31.246887+0000","last_fullsized":"2024-05-09T06:38:31.246887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:31.011590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757375+0000","last_change":"2024-05-09T06:38:29.761155+0000","last_active":"2024-05-09T06:38:30.757375+0000","last_peered":"2024-05-09T06:38:30.757375+0000","last_clean":"2024-05-09T06:38:30.757375+0000","last_became_active":"2024-05-09T06:38:29.760770+0000","last_became_peered":"2024-05-09T06:38:29.760770+0000","last_unstale":"2024-05-09T06:38:30.757375+0000","last_undegraded":"2024-05-09T06:38:30.757375+0000","last_fullsized":"2024-05-09T06:38:30.757375+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742629+0000","last_change":"2024-05-09T06:38:29.757022+0000","last_active":"2024-05-09T06:38:30.742629+0000","last_peered":"2024-05-09T06:38:30.742629+0000","last_clean":"2024-05-09T06:38:30.742629+0000","last_became_active":"2024-05-09T06:38:29.756765+0000","last_became_peered":"2024-05-09T06:38:29.756765+0000","last_unstale":"2024-05-09T06:38:30.742629+0000","last_undegraded":"2024-05-09T06:38:30.742629+0000","last_fullsized":"2024-05-09T06:38:30.742629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:44.514436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246699+0000","last_change":"2024-05-09T06:38:29.750100+0000","last_active":"2024-05-09T06:38:31.246699+0000","last_peered":"2024-05-09T06:38:31.246699+0000","last_clean":"2024-05-09T06:38:31.246699+0000","last_became_active":"2024-05-09T06:38:29.749832+0000","last_became_peered":"2024-05-09T06:38:29.749832+0000","last_unstale":"2024-05-09T06:38:31.246699+0000","last_undegraded":"2024-05-09T06:38:31.246699+0000","last_fullsized":"2024-05-09T06:38:31.246699+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:20.627213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246392+0000","last_change":"2024-05-09T06:38:29.769670+0000","last_active":"2024-05-09T06:38:31.246392+0000","last_peered":"2024-05-09T06:38:31.246392+0000","last_clean":"2024-05-09T06:38:31.246392+0000","last_became_active":"2024-05-09T06:38:29.768971+0000","last_became_peered":"2024-05-09T06:38:29.768971+0000","last_unstale":"2024-05-09T06:38:31.246392+0000","last_undegraded":"2024-05-09T06:38:31.246392+0000","last_fullsized":"2024-05-09T06:38:31.246392+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:21.469243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783415+0000","last_change":"2024-05-09T06:38:29.747737+0000","last_active":"2024-05-09T06:38:30.783415+0000","last_peered":"2024-05-09T06:38:30.783415+0000","last_clean":"2024-05-09T06:38:30.783415+0000","last_became_active":"2024-05-09T06:38:29.747419+0000","last_became_peered":"2024-05-09T06:38:29.747419+0000","last_unstale":"2024-05-09T06:38:30.783415+0000","last_undegraded":"2024-05-09T06:38:30.783415+0000","last_fullsized":"2024-05-09T06:38:30.783415+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783365+0000","last_change":"2024-05-09T06:38:29.759239+0000","last_active":"2024-05-09T06:38:30.783365+0000","last_peered":"2024-05-09T06:38:30.783365+0000","last_clean":"2024-05-09T06:38:30.783365+0000","last_became_active":"2024-05-09T06:38:29.758949+0000","last_became_peered":"2024-05-09T06:38:29.758949+0000","last_unstale":"2024-05-09T06:38:30.783365+0000","last_undegraded":"2024-05-09T06:38:30.783365+0000","last_fullsized":"2024-05-09T06:38:30.783365+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246146+0000","last_change":"2024-05-09T06:38:29.765597+0000","last_active":"2024-05-09T06:38:31.246146+0000","last_peered":"2024-05-09T06:38:31.246146+0000","last_clean":"2024-05-09T06:38:31.246146+0000","last_became_active":"2024-05-09T06:38:29.765299+0000","last_became_peered":"2024-05-09T06:38:29.765299+0000","last_unstale":"2024-05-09T06:38:31.246146+0000","last_undegraded":"2024-05-09T06:38:31.246146+0000","last_fullsized":"2024-05-09T06:38:31.246146+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757013+0000","last_change":"2024-05-09T06:38:28.737931+0000","last_active":"2024-05-09T06:38:30.757013+0000","last_peered":"2024-05-09T06:38:30.757013+0000","last_clean":"2024-05-09T06:38:30.757013+0000","last_became_active":"2024-05-09T06:38:28.737614+0000","last_became_peered":"2024-05-09T06:38:28.737614+0000","last_unstale":"2024-05-09T06:38:30.757013+0000","last_undegraded":"2024-05-09T06:38:30.757013+0000","last_fullsized":"2024-05-09T06:38:30.757013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:19.967238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742223+0000","last_change":"2024-05-09T06:38:29.761964+0000","last_active":"2024-05-09T06:38:30.742223+0000","last_peered":"2024-05-09T06:38:30.742223+0000","last_clean":"2024-05-09T06:38:30.742223+0000","last_became_active":"2024-05-09T06:38:29.761386+0000","last_became_peered":"2024-05-09T06:38:29.761386+0000","last_unstale":"2024-05-09T06:38:30.742223+0000","last_undegraded":"2024-05-09T06:38:30.742223+0000","last_fullsized":"2024-05-09T06:38:30.742223+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:46.892560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245972+0000","last_change":"2024-05-09T06:38:28.747690+0000","last_active":"2024-05-09T06:38:31.245972+0000","last_peered":"2024-05-09T06:38:31.245972+0000","last_clean":"2024-05-09T06:38:31.245972+0000","last_became_active":"2024-05-09T06:38:28.747413+0000","last_became_peered":"2024-05-09T06:38:28.747413+0000","last_unstale":"2024-05-09T06:38:31.245972+0000","last_undegraded":"2024-05-09T06:38:31.245972+0000","last_fullsized":"2024-05-09T06:38:31.245972+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783636+0000","last_change":"2024-05-09T06:38:29.754812+0000","last_active":"2024-05-09T06:38:30.783636+0000","last_peered":"2024-05-09T06:38:30.783636+0000","last_clean":"2024-05-09T06:38:30.783636+0000","last_became_active":"2024-05-09T06:38:29.754325+0000","last_became_peered":"2024-05-09T06:38:29.754325+0000","last_unstale":"2024-05-09T06:38:30.783636+0000","last_undegraded":"2024-05-09T06:38:30.783636+0000","last_fullsized":"2024-05-09T06:38:30.783636+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901338+0000","last_change":"2024-05-09T06:38:28.743221+0000","last_active":"2024-05-09T06:38:30.901338+0000","last_peered":"2024-05-09T06:38:30.901338+0000","last_clean":"2024-05-09T06:38:30.901338+0000","last_became_active":"2024-05-09T06:38:28.742941+0000","last_became_peered":"2024-05-09T06:38:28.742941+0000","last_unstale":"2024-05-09T06:38:30.901338+0000","last_undegraded":"2024-05-09T06:38:30.901338+0000","last_fullsized":"2024-05-09T06:38:30.901338+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901727+0000","last_change":"2024-05-09T06:38:29.762323+0000","last_active":"2024-05-09T06:38:30.901727+0000","last_peered":"2024-05-09T06:38:30.901727+0000","last_clean":"2024-05-09T06:38:30.901727+0000","last_became_active":"2024-05-09T06:38:29.761973+0000","last_became_peered":"2024-05-09T06:38:29.761973+0000","last_unstale":"2024-05-09T06:38:30.901727+0000","last_undegraded":"2024-05-09T06:38:30.901727+0000","last_fullsized":"2024-05-09T06:38:30.901727+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246269+0000","last_change":"2024-05-09T06:38:28.749687+0000","last_active":"2024-05-09T06:38:31.246269+0000","last_peered":"2024-05-09T06:38:31.246269+0000","last_clean":"2024-05-09T06:38:31.246269+0000","last_became_active":"2024-05-09T06:38:28.749395+0000","last_became_peered":"2024-05-09T06:38:28.749395+0000","last_unstale":"2024-05-09T06:38:31.246269+0000","last_undegraded":"2024-05-09T06:38:31.246269+0000","last_fullsized":"2024-05-09T06:38:31.246269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:14.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246176+0000","last_change":"2024-05-09T06:38:29.761152+0000","last_active":"2024-05-09T06:38:31.246176+0000","last_peered":"2024-05-09T06:38:31.246176+0000","last_clean":"2024-05-09T06:38:31.246176+0000","last_became_active":"2024-05-09T06:38:29.760869+0000","last_became_peered":"2024-05-09T06:38:29.760869+0000","last_unstale":"2024-05-09T06:38:31.246176+0000","last_undegraded":"2024-05-09T06:38:31.246176+0000","last_fullsized":"2024-05-09T06:38:31.246176+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743178+0000","last_change":"2024-05-09T06:38:28.750110+0000","last_active":"2024-05-09T06:38:30.743178+0000","last_peered":"2024-05-09T06:38:30.743178+0000","last_clean":"2024-05-09T06:38:30.743178+0000","last_became_active":"2024-05-09T06:38:28.749711+0000","last_became_peered":"2024-05-09T06:38:28.749711+0000","last_unstale":"2024-05-09T06:38:30.743178+0000","last_undegraded":"2024-05-09T06:38:30.743178+0000","last_fullsized":"2024-05-09T06:38:30.743178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743391+0000","last_change":"2024-05-09T06:38:29.763944+0000","last_active":"2024-05-09T06:38:30.743391+0000","last_peered":"2024-05-09T06:38:30.743391+0000","last_clean":"2024-05-09T06:38:30.743391+0000","last_became_active":"2024-05-09T06:38:29.763626+0000","last_became_peered":"2024-05-09T06:38:29.763626+0000","last_unstale":"2024-05-09T06:38:30.743391+0000","last_undegraded":"2024-05-09T06:38:30.743391+0000","last_fullsized":"2024-05-09T06:38:30.743391+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:00.819768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742147+0000","last_change":"2024-05-09T06:38:28.750014+0000","last_active":"2024-05-09T06:38:30.742147+0000","last_peered":"2024-05-09T06:38:30.742147+0000","last_clean":"2024-05-09T06:38:30.742147+0000","last_became_active":"2024-05-09T06:38:28.749739+0000","last_became_peered":"2024-05-09T06:38:28.749739+0000","last_unstale":"2024-05-09T06:38:30.742147+0000","last_undegraded":"2024-05-09T06:38:30.742147+0000","last_fullsized":"2024-05-09T06:38:30.742147+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:37.313308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246769+0000","last_change":"2024-05-09T06:38:29.754580+0000","last_active":"2024-05-09T06:38:31.246769+0000","last_peered":"2024-05-09T06:38:31.246769+0000","last_clean":"2024-05-09T06:38:31.246769+0000","last_became_active":"2024-05-09T06:38:29.754015+0000","last_became_peered":"2024-05-09T06:38:29.754015+0000","last_unstale":"2024-05-09T06:38:31.246769+0000","last_undegraded":"2024-05-09T06:38:31.246769+0000","last_fullsized":"2024-05-09T06:38:31.246769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:12.607951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783327+0000","last_change":"2024-05-09T06:38:28.747303+0000","last_active":"2024-05-09T06:38:30.783327+0000","last_peered":"2024-05-09T06:38:30.783327+0000","last_clean":"2024-05-09T06:38:30.783327+0000","last_became_active":"2024-05-09T06:38:28.746858+0000","last_became_peered":"2024-05-09T06:38:28.746858+0000","last_unstale":"2024-05-09T06:38:30.783327+0000","last_undegraded":"2024-05-09T06:38:30.783327+0000","last_fullsized":"2024-05-09T06:38:30.783327+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:15.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246290+0000","last_change":"2024-05-09T06:38:29.752770+0000","last_active":"2024-05-09T06:38:31.246290+0000","last_peered":"2024-05-09T06:38:31.246290+0000","last_clean":"2024-05-09T06:38:31.246290+0000","last_became_active":"2024-05-09T06:38:29.752478+0000","last_became_peered":"2024-05-09T06:38:29.752478+0000","last_unstale":"2024-05-09T06:38:31.246290+0000","last_undegraded":"2024-05-09T06:38:31.246290+0000","last_fullsized":"2024-05-09T06:38:31.246290+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.394253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742167+0000","last_change":"2024-05-09T06:38:28.749180+0000","last_active":"2024-05-09T06:38:30.742167+0000","last_peered":"2024-05-09T06:38:30.742167+0000","last_clean":"2024-05-09T06:38:30.742167+0000","last_became_active":"2024-05-09T06:38:28.748888+0000","last_became_peered":"2024-05-09T06:38:28.748888+0000","last_unstale":"2024-05-09T06:38:30.742167+0000","last_undegraded":"2024-05-09T06:38:30.742167+0000","last_fullsized":"2024-05-09T06:38:30.742167+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.798256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757193+0000","last_change":"2024-05-09T06:38:29.766343+0000","last_active":"2024-05-09T06:38:30.757193+0000","last_peered":"2024-05-09T06:38:30.757193+0000","last_clean":"2024-05-09T06:38:30.757193+0000","last_became_active":"2024-05-09T06:38:29.766008+0000","last_became_peered":"2024-05-09T06:38:29.766008+0000","last_unstale":"2024-05-09T06:38:30.757193+0000","last_undegraded":"2024-05-09T06:38:30.757193+0000","last_fullsized":"2024-05-09T06:38:30.757193+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757169+0000","last_change":"2024-05-09T06:38:28.740728+0000","last_active":"2024-05-09T06:38:30.757169+0000","last_peered":"2024-05-09T06:38:30.757169+0000","last_clean":"2024-05-09T06:38:30.757169+0000","last_became_active":"2024-05-09T06:38:28.740457+0000","last_became_peered":"2024-05-09T06:38:28.740457+0000","last_unstale":"2024-05-09T06:38:30.757169+0000","last_undegraded":"2024-05-09T06:38:30.757169+0000","last_fullsized":"2024-05-09T06:38:30.757169+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:52.177804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783704+0000","last_change":"2024-05-09T06:38:29.753396+0000","last_active":"2024-05-09T06:38:30.783704+0000","last_peered":"2024-05-09T06:38:30.783704+0000","last_clean":"2024-05-09T06:38:30.783704+0000","last_became_active":"2024-05-09T06:38:29.753036+0000","last_became_peered":"2024-05-09T06:38:29.753036+0000","last_unstale":"2024-05-09T06:38:30.783704+0000","last_undegraded":"2024-05-09T06:38:30.783704+0000","last_fullsized":"2024-05-09T06:38:30.783704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783554+0000","last_change":"2024-05-09T06:38:28.751282+0000","last_active":"2024-05-09T06:38:30.783554+0000","last_peered":"2024-05-09T06:38:30.783554+0000","last_clean":"2024-05-09T06:38:30.783554+0000","last_became_active":"2024-05-09T06:38:28.750962+0000","last_became_peered":"2024-05-09T06:38:28.750962+0000","last_unstale":"2024-05-09T06:38:30.783554+0000","last_undegraded":"2024-05-09T06:38:30.783554+0000","last_fullsized":"2024-05-09T06:38:30.783554+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246961+0000","last_change":"2024-05-09T06:38:29.760363+0000","last_active":"2024-05-09T06:38:31.246961+0000","last_peered":"2024-05-09T06:38:31.246961+0000","last_clean":"2024-05-09T06:38:31.246961+0000","last_became_active":"2024-05-09T06:38:29.760090+0000","last_became_peered":"2024-05-09T06:38:29.760090+0000","last_unstale":"2024-05-09T06:38:31.246961+0000","last_undegraded":"2024-05-09T06:38:31.246961+0000","last_fullsized":"2024-05-09T06:38:31.246961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:38.317915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783600+0000","last_change":"2024-05-09T06:38:28.748636+0000","last_active":"2024-05-09T06:38:30.783600+0000","last_peered":"2024-05-09T06:38:30.783600+0000","last_clean":"2024-05-09T06:38:30.783600+0000","last_became_active":"2024-05-09T06:38:28.748336+0000","last_became_peered":"2024-05-09T06:38:28.748336+0000","last_unstale":"2024-05-09T06:38:30.783600+0000","last_undegraded":"2024-05-09T06:38:30.783600+0000","last_fullsized":"2024-05-09T06:38:30.783600+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783801+0000","last_change":"2024-05-09T06:38:29.755192+0000","last_active":"2024-05-09T06:38:30.783801+0000","last_peered":"2024-05-09T06:38:30.783801+0000","last_clean":"2024-05-09T06:38:30.783801+0000","last_became_active":"2024-05-09T06:38:29.754772+0000","last_became_peered":"2024-05-09T06:38:29.754772+0000","last_unstale":"2024-05-09T06:38:30.783801+0000","last_undegraded":"2024-05-09T06:38:30.783801+0000","last_fullsized":"2024-05-09T06:38:30.783801+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246896+0000","last_change":"2024-05-09T06:38:28.745965+0000","last_active":"2024-05-09T06:38:31.246896+0000","last_peered":"2024-05-09T06:38:31.246896+0000","last_clean":"2024-05-09T06:38:31.246896+0000","last_became_active":"2024-05-09T06:38:28.745675+0000","last_became_peered":"2024-05-09T06:38:28.745675+0000","last_unstale":"2024-05-09T06:38:31.246896+0000","last_undegraded":"2024-05-09T06:38:31.246896+0000","last_fullsized":"2024-05-09T06:38:31.246896+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:10.080412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757175+0000","last_change":"2024-05-09T06:38:29.757835+0000","last_active":"2024-05-09T06:38:30.757175+0000","last_peered":"2024-05-09T06:38:30.757175+0000","last_clean":"2024-05-09T06:38:30.757175+0000","last_became_active":"2024-05-09T06:38:29.757188+0000","last_became_peered":"2024-05-09T06:38:29.757188+0000","last_unstale":"2024-05-09T06:38:30.757175+0000","last_undegraded":"2024-05-09T06:38:30.757175+0000","last_fullsized":"2024-05-09T06:38:30.757175+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246638+0000","last_change":"2024-05-09T06:38:28.743054+0000","last_active":"2024-05-09T06:38:31.246638+0000","last_peered":"2024-05-09T06:38:31.246638+0000","last_clean":"2024-05-09T06:38:31.246638+0000","last_became_active":"2024-05-09T06:38:28.742764+0000","last_became_peered":"2024-05-09T06:38:28.742764+0000","last_unstale":"2024-05-09T06:38:31.246638+0000","last_undegraded":"2024-05-09T06:38:31.246638+0000","last_fullsized":"2024-05-09T06:38:31.246638+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:34.394936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245820+0000","last_change":"2024-05-09T06:38:29.766170+0000","last_active":"2024-05-09T06:38:31.245820+0000","last_peered":"2024-05-09T06:38:31.245820+0000","last_clean":"2024-05-09T06:38:31.245820+0000","last_became_active":"2024-05-09T06:38:29.765908+0000","last_became_peered":"2024-05-09T06:38:29.765908+0000","last_unstale":"2024-05-09T06:38:31.245820+0000","last_undegraded":"2024-05-09T06:38:31.245820+0000","last_fullsized":"2024-05-09T06:38:31.245820+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246716+0000","last_change":"2024-05-09T06:38:28.746476+0000","last_active":"2024-05-09T06:38:31.246716+0000","last_peered":"2024-05-09T06:38:31.246716+0000","last_clean":"2024-05-09T06:38:31.246716+0000","last_became_active":"2024-05-09T06:38:28.746204+0000","last_became_peered":"2024-05-09T06:38:28.746204+0000","last_unstale":"2024-05-09T06:38:31.246716+0000","last_undegraded":"2024-05-09T06:38:31.246716+0000","last_fullsized":"2024-05-09T06:38:31.246716+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:48.389646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245765+0000","last_change":"2024-05-09T06:38:29.749306+0000","last_active":"2024-05-09T06:38:31.245765+0000","last_peered":"2024-05-09T06:38:31.245765+0000","last_clean":"2024-05-09T06:38:31.245765+0000","last_became_active":"2024-05-09T06:38:29.749017+0000","last_became_peered":"2024-05-09T06:38:29.749017+0000","last_unstale":"2024-05-09T06:38:31.245765+0000","last_undegraded":"2024-05-09T06:38:31.245765+0000","last_fullsized":"2024-05-09T06:38:31.245765+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:51.601822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247006+0000","last_change":"2024-05-09T06:38:28.745058+0000","last_active":"2024-05-09T06:38:31.247006+0000","last_peered":"2024-05-09T06:38:31.247006+0000","last_clean":"2024-05-09T06:38:31.247006+0000","last_became_active":"2024-05-09T06:38:28.744726+0000","last_became_peered":"2024-05-09T06:38:28.744726+0000","last_unstale":"2024-05-09T06:38:31.247006+0000","last_undegraded":"2024-05-09T06:38:31.247006+0000","last_fullsized":"2024-05-09T06:38:31.247006+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:53.740364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783470+0000","last_change":"2024-05-09T06:38:29.755533+0000","last_active":"2024-05-09T06:38:30.783470+0000","last_peered":"2024-05-09T06:38:30.783470+0000","last_clean":"2024-05-09T06:38:30.783470+0000","last_became_active":"2024-05-09T06:38:29.755144+0000","last_became_peered":"2024-05-09T06:38:29.755144+0000","last_unstale":"2024-05-09T06:38:30.783470+0000","last_undegraded":"2024-05-09T06:38:30.783470+0000","last_fullsized":"2024-05-09T06:38:30.783470+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246001+0000","last_change":"2024-05-09T06:38:28.748335+0000","last_active":"2024-05-09T06:38:31.246001+0000","last_peered":"2024-05-09T06:38:31.246001+0000","last_clean":"2024-05-09T06:38:31.246001+0000","last_became_active":"2024-05-09T06:38:28.747996+0000","last_became_peered":"2024-05-09T06:38:28.747996+0000","last_unstale":"2024-05-09T06:38:31.246001+0000","last_undegraded":"2024-05-09T06:38:31.246001+0000","last_fullsized":"2024-05-09T06:38:31.246001+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:57.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743711+0000","last_change":"2024-05-09T06:38:29.769319+0000","last_active":"2024-05-09T06:38:30.743711+0000","last_peered":"2024-05-09T06:38:30.743711+0000","last_clean":"2024-05-09T06:38:30.743711+0000","last_became_active":"2024-05-09T06:38:29.769020+0000","last_became_peered":"2024-05-09T06:38:29.769020+0000","last_unstale":"2024-05-09T06:38:30.743711+0000","last_undegraded":"2024-05-09T06:38:30.743711+0000","last_fullsized":"2024-05-09T06:38:30.743711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246945+0000","last_change":"2024-05-09T06:38:28.739768+0000","last_active":"2024-05-09T06:38:31.246945+0000","last_peered":"2024-05-09T06:38:31.246945+0000","last_clean":"2024-05-09T06:38:31.246945+0000","last_became_active":"2024-05-09T06:38:28.739460+0000","last_became_peered":"2024-05-09T06:38:28.739460+0000","last_unstale":"2024-05-09T06:38:31.246945+0000","last_undegraded":"2024-05-09T06:38:31.246945+0000","last_fullsized":"2024-05-09T06:38:31.246945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:18.604394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742591+0000","last_change":"2024-05-09T06:38:29.755262+0000","last_active":"2024-05-09T06:38:30.742591+0000","last_peered":"2024-05-09T06:38:30.742591+0000","last_clean":"2024-05-09T06:38:30.742591+0000","last_became_active":"2024-05-09T06:38:29.754999+0000","last_became_peered":"2024-05-09T06:38:29.754999+0000","last_unstale":"2024-05-09T06:38:30.742591+0000","last_undegraded":"2024-05-09T06:38:30.742591+0000","last_fullsized":"2024-05-09T06:38:30.742591+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:28.631222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901357+0000","last_change":"2024-05-09T06:38:28.749325+0000","last_active":"2024-05-09T06:38:30.901357+0000","last_peered":"2024-05-09T06:38:30.901357+0000","last_clean":"2024-05-09T06:38:30.901357+0000","last_became_active":"2024-05-09T06:38:28.748819+0000","last_became_peered":"2024-05-09T06:38:28.748819+0000","last_unstale":"2024-05-09T06:38:30.901357+0000","last_undegraded":"2024-05-09T06:38:30.901357+0000","last_fullsized":"2024-05-09T06:38:30.901357+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783894+0000","last_change":"2024-05-09T06:38:29.757014+0000","last_active":"2024-05-09T06:38:30.783894+0000","last_peered":"2024-05-09T06:38:30.783894+0000","last_clean":"2024-05-09T06:38:30.783894+0000","last_became_active":"2024-05-09T06:38:29.756267+0000","last_became_peered":"2024-05-09T06:38:29.756267+0000","last_unstale":"2024-05-09T06:38:30.783894+0000","last_undegraded":"2024-05-09T06:38:30.783894+0000","last_fullsized":"2024-05-09T06:38:30.783894+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:47.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901363+0000","last_change":"2024-05-09T06:38:28.749323+0000","last_active":"2024-05-09T06:38:30.901363+0000","last_peered":"2024-05-09T06:38:30.901363+0000","last_clean":"2024-05-09T06:38:30.901363+0000","last_became_active":"2024-05-09T06:38:28.748749+0000","last_became_peered":"2024-05-09T06:38:28.748749+0000","last_unstale":"2024-05-09T06:38:30.901363+0000","last_undegraded":"2024-05-09T06:38:30.901363+0000","last_fullsized":"2024-05-09T06:38:30.901363+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741950+0000","last_change":"2024-05-09T06:38:29.758725+0000","last_active":"2024-05-09T06:38:30.741950+0000","last_peered":"2024-05-09T06:38:30.741950+0000","last_clean":"2024-05-09T06:38:30.741950+0000","last_became_active":"2024-05-09T06:38:29.758465+0000","last_became_peered":"2024-05-09T06:38:29.758465+0000","last_unstale":"2024-05-09T06:38:30.741950+0000","last_undegraded":"2024-05-09T06:38:30.741950+0000","last_fullsized":"2024-05-09T06:38:30.741950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:54:09.412143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741964+0000","last_change":"2024-05-09T06:38:28.747146+0000","last_active":"2024-05-09T06:38:30.741964+0000","last_peered":"2024-05-09T06:38:30.741964+0000","last_clean":"2024-05-09T06:38:30.741964+0000","last_became_active":"2024-05-09T06:38:28.746779+0000","last_became_peered":"2024-05-09T06:38:28.746779+0000","last_unstale":"2024-05-09T06:38:30.741964+0000","last_undegraded":"2024-05-09T06:38:30.741964+0000","last_fullsized":"2024-05-09T06:38:30.741964+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:45.064149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246046+0000","last_change":"2024-05-09T06:38:29.749910+0000","last_active":"2024-05-09T06:38:31.246046+0000","last_peered":"2024-05-09T06:38:31.246046+0000","last_clean":"2024-05-09T06:38:31.246046+0000","last_became_active":"2024-05-09T06:38:29.749656+0000","last_became_peered":"2024-05-09T06:38:29.749656+0000","last_unstale":"2024-05-09T06:38:31.246046+0000","last_undegraded":"2024-05-09T06:38:31.246046+0000","last_fullsized":"2024-05-09T06:38:31.246046+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.238142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901019+0000","last_change":"2024-05-09T06:38:28.740359+0000","last_active":"2024-05-09T06:38:30.901019+0000","last_peered":"2024-05-09T06:38:30.901019+0000","last_clean":"2024-05-09T06:38:30.901019+0000","last_became_active":"2024-05-09T06:38:28.740077+0000","last_became_peered":"2024-05-09T06:38:28.740077+0000","last_unstale":"2024-05-09T06:38:30.901019+0000","last_undegraded":"2024-05-09T06:38:30.901019+0000","last_fullsized":"2024-05-09T06:38:30.901019+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:57.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246658+0000","last_change":"2024-05-09T06:38:29.758211+0000","last_active":"2024-05-09T06:38:31.246658+0000","last_peered":"2024-05-09T06:38:31.246658+0000","last_clean":"2024-05-09T06:38:31.246658+0000","last_became_active":"2024-05-09T06:38:29.757719+0000","last_became_peered":"2024-05-09T06:38:29.757719+0000","last_unstale":"2024-05-09T06:38:31.246658+0000","last_undegraded":"2024-05-09T06:38:31.246658+0000","last_fullsized":"2024-05-09T06:38:31.246658+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.981345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247039+0000","last_change":"2024-05-09T06:38:28.747358+0000","last_active":"2024-05-09T06:38:31.247039+0000","last_peered":"2024-05-09T06:38:31.247039+0000","last_clean":"2024-05-09T06:38:31.247039+0000","last_became_active":"2024-05-09T06:38:28.746987+0000","last_became_peered":"2024-05-09T06:38:28.746987+0000","last_unstale":"2024-05-09T06:38:31.247039+0000","last_undegraded":"2024-05-09T06:38:31.247039+0000","last_fullsized":"2024-05-09T06:38:31.247039+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:42.705250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783481+0000","last_change":"2024-05-09T06:38:29.748493+0000","last_active":"2024-05-09T06:38:30.783481+0000","last_peered":"2024-05-09T06:38:30.783481+0000","last_clean":"2024-05-09T06:38:30.783481+0000","last_became_active":"2024-05-09T06:38:29.748190+0000","last_became_peered":"2024-05-09T06:38:29.748190+0000","last_unstale":"2024-05-09T06:38:30.783481+0000","last_undegraded":"2024-05-09T06:38:30.783481+0000","last_fullsized":"2024-05-09T06:38:30.783481+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.900827+0000","last_change":"2024-05-09T06:38:28.742183+0000","last_active":"2024-05-09T06:38:30.900827+0000","last_peered":"2024-05-09T06:38:30.900827+0000","last_clean":"2024-05-09T06:38:30.900827+0000","last_became_active":"2024-05-09T06:38:28.741637+0000","last_became_peered":"2024-05-09T06:38:28.741637+0000","last_unstale":"2024-05-09T06:38:30.900827+0000","last_undegraded":"2024-05-09T06:38:30.900827+0000","last_fullsized":"2024-05-09T06:38:30.900827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:18.776523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742781+0000","last_change":"2024-05-09T06:38:29.757836+0000","last_active":"2024-05-09T06:38:30.742781+0000","last_peered":"2024-05-09T06:38:30.742781+0000","last_clean":"2024-05-09T06:38:30.742781+0000","last_became_active":"2024-05-09T06:38:29.757527+0000","last_became_peered":"2024-05-09T06:38:29.757527+0000","last_unstale":"2024-05-09T06:38:30.742781+0000","last_undegraded":"2024-05-09T06:38:30.742781+0000","last_fullsized":"2024-05-09T06:38:30.742781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:33.601288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742178+0000","last_change":"2024-05-09T06:38:28.751772+0000","last_active":"2024-05-09T06:38:30.742178+0000","last_peered":"2024-05-09T06:38:30.742178+0000","last_clean":"2024-05-09T06:38:30.742178+0000","last_became_active":"2024-05-09T06:38:28.751455+0000","last_became_peered":"2024-05-09T06:38:28.751455+0000","last_unstale":"2024-05-09T06:38:30.742178+0000","last_undegraded":"2024-05-09T06:38:30.742178+0000","last_fullsized":"2024-05-09T06:38:30.742178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:27:20.878031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246703+0000","last_change":"2024-05-09T06:38:29.760060+0000","last_active":"2024-05-09T06:38:31.246703+0000","last_peered":"2024-05-09T06:38:31.246703+0000","last_clean":"2024-05-09T06:38:31.246703+0000","last_became_active":"2024-05-09T06:38:29.759799+0000","last_became_peered":"2024-05-09T06:38:29.759799+0000","last_unstale":"2024-05-09T06:38:31.246703+0000","last_undegraded":"2024-05-09T06:38:31.246703+0000","last_fullsized":"2024-05-09T06:38:31.246703+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:50.306342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757198+0000","last_change":"2024-05-09T06:38:28.739861+0000","last_active":"2024-05-09T06:38:30.757198+0000","last_peered":"2024-05-09T06:38:30.757198+0000","last_clean":"2024-05-09T06:38:30.757198+0000","last_became_active":"2024-05-09T06:38:28.739521+0000","last_became_peered":"2024-05-09T06:38:28.739521+0000","last_unstale":"2024-05-09T06:38:30.757198+0000","last_undegraded":"2024-05-09T06:38:30.757198+0000","last_fullsized":"2024-05-09T06:38:30.757198+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.473368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901590+0000","last_change":"2024-05-09T06:38:29.761001+0000","last_active":"2024-05-09T06:38:30.901590+0000","last_peered":"2024-05-09T06:38:30.901590+0000","last_clean":"2024-05-09T06:38:30.901590+0000","last_became_active":"2024-05-09T06:38:29.760697+0000","last_became_peered":"2024-05-09T06:38:29.760697+0000","last_unstale":"2024-05-09T06:38:30.901590+0000","last_undegraded":"2024-05-09T06:38:30.901590+0000","last_fullsized":"2024-05-09T06:38:30.901590+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757372+0000","last_change":"2024-05-09T06:38:28.739194+0000","last_active":"2024-05-09T06:38:30.757372+0000","last_peered":"2024-05-09T06:38:30.757372+0000","last_clean":"2024-05-09T06:38:30.757372+0000","last_became_active":"2024-05-09T06:38:28.738875+0000","last_became_peered":"2024-05-09T06:38:28.738875+0000","last_unstale":"2024-05-09T06:38:30.757372+0000","last_undegraded":"2024-05-09T06:38:30.757372+0000","last_fullsized":"2024-05-09T06:38:30.757372+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:24.807360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247137+0000","last_change":"2024-05-09T06:38:29.751188+0000","last_active":"2024-05-09T06:38:31.247137+0000","last_peered":"2024-05-09T06:38:31.247137+0000","last_clean":"2024-05-09T06:38:31.247137+0000","last_became_active":"2024-05-09T06:38:29.750897+0000","last_became_peered":"2024-05-09T06:38:29.750897+0000","last_unstale":"2024-05-09T06:38:31.247137+0000","last_undegraded":"2024-05-09T06:38:31.247137+0000","last_fullsized":"2024-05-09T06:38:31.247137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:43.881398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742686+0000","last_change":"2024-05-09T06:38:28.750379+0000","last_active":"2024-05-09T06:38:30.742686+0000","last_peered":"2024-05-09T06:38:30.742686+0000","last_clean":"2024-05-09T06:38:30.742686+0000","last_became_active":"2024-05-09T06:38:28.750040+0000","last_became_peered":"2024-05-09T06:38:28.750040+0000","last_unstale":"2024-05-09T06:38:30.742686+0000","last_undegraded":"2024-05-09T06:38:30.742686+0000","last_fullsized":"2024-05-09T06:38:30.742686+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:12.256422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742109+0000","last_change":"2024-05-09T06:38:29.757931+0000","last_active":"2024-05-09T06:38:30.742109+0000","last_peered":"2024-05-09T06:38:30.742109+0000","last_clean":"2024-05-09T06:38:30.742109+0000","last_became_active":"2024-05-09T06:38:29.756980+0000","last_became_peered":"2024-05-09T06:38:29.756980+0000","last_unstale":"2024-05-09T06:38:30.742109+0000","last_undegraded":"2024-05-09T06:38:30.742109+0000","last_fullsized":"2024-05-09T06:38:30.742109+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:34.117615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743796+0000","last_change":"2024-05-09T06:38:28.742817+0000","last_active":"2024-05-09T06:38:30.743796+0000","last_peered":"2024-05-09T06:38:30.743796+0000","last_clean":"2024-05-09T06:38:30.743796+0000","last_became_active":"2024-05-09T06:38:28.742480+0000","last_became_peered":"2024-05-09T06:38:28.742480+0000","last_unstale":"2024-05-09T06:38:30.743796+0000","last_undegraded":"2024-05-09T06:38:30.743796+0000","last_fullsized":"2024-05-09T06:38:30.743796+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246204+0000","last_change":"2024-05-09T06:38:29.755565+0000","last_active":"2024-05-09T06:38:31.246204+0000","last_peered":"2024-05-09T06:38:31.246204+0000","last_clean":"2024-05-09T06:38:31.246204+0000","last_became_active":"2024-05-09T06:38:29.755291+0000","last_became_peered":"2024-05-09T06:38:29.755291+0000","last_unstale":"2024-05-09T06:38:31.246204+0000","last_undegraded":"2024-05-09T06:38:31.246204+0000","last_fullsized":"2024-05-09T06:38:31.246204+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:26.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743680+0000","last_change":"2024-05-09T06:38:28.745054+0000","last_active":"2024-05-09T06:38:30.743680+0000","last_peered":"2024-05-09T06:38:30.743680+0000","last_clean":"2024-05-09T06:38:30.743680+0000","last_became_active":"2024-05-09T06:38:28.744751+0000","last_became_peered":"2024-05-09T06:38:28.744751+0000","last_unstale":"2024-05-09T06:38:30.743680+0000","last_undegraded":"2024-05-09T06:38:30.743680+0000","last_fullsized":"2024-05-09T06:38:30.743680+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901112+0000","last_change":"2024-05-09T06:38:29.762323+0000","last_active":"2024-05-09T06:38:30.901112+0000","last_peered":"2024-05-09T06:38:30.901112+0000","last_clean":"2024-05-09T06:38:30.901112+0000","last_became_active":"2024-05-09T06:38:29.762012+0000","last_became_peered":"2024-05-09T06:38:29.762012+0000","last_unstale":"2024-05-09T06:38:30.901112+0000","last_undegraded":"2024-05-09T06:38:30.901112+0000","last_fullsized":"2024-05-09T06:38:30.901112+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742076+0000","last_change":"2024-05-09T06:38:28.752996+0000","last_active":"2024-05-09T06:38:30.742076+0000","last_peered":"2024-05-09T06:38:30.742076+0000","last_clean":"2024-05-09T06:38:30.742076+0000","last_became_active":"2024-05-09T06:38:28.752716+0000","last_became_peered":"2024-05-09T06:38:28.752716+0000","last_unstale":"2024-05-09T06:38:30.742076+0000","last_undegraded":"2024-05-09T06:38:30.742076+0000","last_fullsized":"2024-05-09T06:38:30.742076+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:32.408558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246443+0000","last_change":"2024-05-09T06:38:29.764208+0000","last_active":"2024-05-09T06:38:31.246443+0000","last_peered":"2024-05-09T06:38:31.246443+0000","last_clean":"2024-05-09T06:38:31.246443+0000","last_became_active":"2024-05-09T06:38:29.763914+0000","last_became_peered":"2024-05-09T06:38:29.763914+0000","last_unstale":"2024-05-09T06:38:31.246443+0000","last_undegraded":"2024-05-09T06:38:31.246443+0000","last_fullsized":"2024-05-09T06:38:31.246443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:47.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.900853+0000","last_change":"2024-05-09T06:38:28.744382+0000","last_active":"2024-05-09T06:38:30.900853+0000","last_peered":"2024-05-09T06:38:30.900853+0000","last_clean":"2024-05-09T06:38:30.900853+0000","last_became_active":"2024-05-09T06:38:28.743975+0000","last_became_peered":"2024-05-09T06:38:28.743975+0000","last_unstale":"2024-05-09T06:38:30.900853+0000","last_undegraded":"2024-05-09T06:38:30.900853+0000","last_fullsized":"2024-05-09T06:38:30.900853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:15.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901629+0000","last_change":"2024-05-09T06:38:29.763309+0000","last_active":"2024-05-09T06:38:30.901629+0000","last_peered":"2024-05-09T06:38:30.901629+0000","last_clean":"2024-05-09T06:38:30.901629+0000","last_became_active":"2024-05-09T06:38:29.763008+0000","last_became_peered":"2024-05-09T06:38:29.763008+0000","last_unstale":"2024-05-09T06:38:30.901629+0000","last_undegraded":"2024-05-09T06:38:30.901629+0000","last_fullsized":"2024-05-09T06:38:30.901629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245977+0000","last_change":"2024-05-09T06:38:28.743982+0000","last_active":"2024-05-09T06:38:31.245977+0000","last_peered":"2024-05-09T06:38:31.245977+0000","last_clean":"2024-05-09T06:38:31.245977+0000","last_became_active":"2024-05-09T06:38:28.743653+0000","last_became_peered":"2024-05-09T06:38:28.743653+0000","last_unstale":"2024-05-09T06:38:31.245977+0000","last_undegraded":"2024-05-09T06:38:31.245977+0000","last_fullsized":"2024-05-09T06:38:31.245977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:46.631441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":357,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901005+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:38:30.901005+0000","last_peered":"2024-05-09T06:38:30.901005+0000","last_clean":"2024-05-09T06:38:30.901005+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:38:30.901005+0000","last_undegraded":"2024-05-09T06:38:30.901005+0000","last_fullsized":"2024-05-09T06:38:30.901005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2117632,"data_stored":1296829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":706267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":705845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":707297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1523712,"data_stored":707331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":705685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":705707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2105344,"data_stored":1296075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:38:33.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:38:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.906+0000 7f94d57d5640 1 Processor -- start 2024-05-09T06:38:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.906+0000 7f94d57d5640 1 -- start start 2024-05-09T06:38:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.906+0000 7f94d57d5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d00d4940 0x7f94d00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.906+0000 7f94d57d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94d00d2b10 0x7f94d00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.906+0000 7f94d57d5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d00b33c0 con 0x7f94d0161950 2024-05-09T06:38:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94d57d5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f94d00d4dd0 con 0x7f94d00d4940 2024-05-09T06:38:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94d57d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f94d00b3660 con 0x7f94d00d2b10 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d00d4940 0x7f94d00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ceffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f94d0161950 0x7f94d00d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60264/0 (socket says 172.21.15.1:60264) 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ceffd640 1 -- 172.21.15.1:0/1290770109 learned_addr learned my addr 172.21.15.1:0/1290770109 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d00d4940 0x7f94d00b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47990/0 (socket says 172.21.15.1:47990) 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 -- 172.21.15.1:0/1290770109 >> v2:172.21.15.6:3300/0 conn(0x7f94d00d2b10 msgr2=0x7f94d00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94cf7fe640 1 --2- 172.21.15.1:0/1290770109 >> v2:172.21.15.6:3300/0 conn(0x7f94d00d2b10 0x7f94d00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94cdffb640 1 -- 172.21.15.1:0/1290770109 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341792459 0 0) 0x7f94d00b33c0 con 0x7f94d0161950 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 --2- 172.21.15.1:0/1290770109 >> v2:172.21.15.6:3300/0 conn(0x7f94d00d2b10 0x7f94d00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 -- 172.21.15.1:0/1290770109 >> v1:172.21.15.1:6789/0 conn(0x7f94d0161950 legacy=0x7f94d00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.907+0000 7f94ce7fc640 1 -- 172.21.15.1:0/1290770109 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d00b3940 con 0x7f94d00d4940 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94ce7fc640 1 --2- 172.21.15.1:0/1290770109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d00d4940 0x7f94d00b4140 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f94d008e540 tx=0x7f94c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1506318c41eadf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94cdffb640 1 -- 172.21.15.1:0/1290770109 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4037070 con 0x7f94d00d4940 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94cdffb640 1 -- 172.21.15.1:0/1290770109 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4032440 con 0x7f94d00d4940 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94cdffb640 1 -- 172.21.15.1:0/1290770109 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4032810 con 0x7f94d00d4940 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 -- 172.21.15.1:0/1290770109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d00d4940 msgr2=0x7f94d00b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 --2- 172.21.15.1:0/1290770109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d00d4940 0x7f94d00b4140 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f94d008e540 tx=0x7f94c402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 -- 172.21.15.1:0/1290770109 shutdown_connections 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 --2- 172.21.15.1:0/1290770109 >> v2:172.21.15.6:3300/0 conn(0x7f94d00d2b10 0x7f94d00b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 --2- 172.21.15.1:0/1290770109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d00d4940 0x7f94d00b4140 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 -- 172.21.15.1:0/1290770109 >> 172.21.15.1:0/1290770109 conn(0x7f94d00d3e20 msgr2=0x7f94d00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 -- 172.21.15.1:0/1290770109 shutdown_connections 2024-05-09T06:38:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.908+0000 7f94d57d5640 1 -- 172.21.15.1:0/1290770109 wait complete. 2024-05-09T06:38:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.910+0000 7f94d57d5640 1 Processor -- start 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.910+0000 7f94d57d5640 1 -- start start 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d00d2b10 0x7f94d00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 0x7f94d00b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00d1100 con 0x7f94d0161950 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00d0f80 con 0x7f94d015c3e0 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f94d00d1c00 con 0x7f94d00d2b10 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47992/0 (socket says 172.21.15.1:47992) 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 -- 172.21.15.1:0/3295549749 learned_addr learned my addr 172.21.15.1:0/3295549749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ceffd640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d00d2b10 0x7f94d00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94cf7fe640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 0x7f94d00b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 -- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d00d2b10 msgr2=0x7f94d00cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d00d2b10 0x7f94d00cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 -- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 msgr2=0x7f94d00b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 0x7f94d00b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 -- 172.21.15.1:0/3295549749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d015cbf0 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94cf7fe640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 0x7f94d00b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94ce7fc640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000c00 tx=0x7f94c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4037040 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4042070 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d00cd570 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.911+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4030a50 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.912+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d00cc550 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.912+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d00752b0 con 0x7f94d0161950 2024-05-09T06:38:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.913+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94c4030bf0 con 0x7f94d0161950 2024-05-09T06:38:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.913+0000 7f94b77fe640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 0x7f94bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.914+0000 7f94ceffd640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 0x7f94bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.914+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f94c40aba20 con 0x7f94d0161950 2024-05-09T06:38:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.914+0000 7f94ceffd640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 0x7f94bc06e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007fd0 tx=0x7f94c0007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:33.918+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94c407a240 con 0x7f94d0161950 2024-05-09T06:38:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.098+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94d00d21e0 con 0x7f94bc06be30 2024-05-09T06:38:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.104+0000 7f94b77fe640 1 -- 172.21.15.1:0/3295549749 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f94d00d21e0 con 0x7f94bc06be30 2024-05-09T06:38:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:34.088 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 msgr2=0x7f94bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 0x7f94bc06e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007fd0 tx=0x7f94c0007de0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 msgr2=0x7f94d00cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000c00 tx=0x7f94c40046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 shutdown_connections 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f94bc06be30 0x7f94bc06e2f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94d015c3e0 0x7f94d00b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d0161950 0x7f94d00cd030 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 --2- 172.21.15.1:0/3295549749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94d00d2b10 0x7f94d00cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 >> 172.21.15.1:0/3295549749 conn(0x7f94d00d3e20 msgr2=0x7f94d00b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:34.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 shutdown_connections 2024-05-09T06:38:34.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.107+0000 7f94d57d5640 1 -- 172.21.15.1:0/3295549749 wait complete. 2024-05-09T06:38:34.098 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-05-09T06:38:33.221902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270792,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703928,"statfs":{"total":773094113280,"available":772816822272,"internally_reserved":0,"allocated":13312000,"data_stored":6831036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.497016"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743791+0000","last_change":"2024-05-09T06:38:29.760784+0000","last_active":"2024-05-09T06:38:30.743791+0000","last_peered":"2024-05-09T06:38:30.743791+0000","last_clean":"2024-05-09T06:38:30.743791+0000","last_became_active":"2024-05-09T06:38:29.760456+0000","last_became_peered":"2024-05-09T06:38:29.760456+0000","last_unstale":"2024-05-09T06:38:30.743791+0000","last_undegraded":"2024-05-09T06:38:30.743791+0000","last_fullsized":"2024-05-09T06:38:30.743791+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246661+0000","last_change":"2024-05-09T06:38:29.769260+0000","last_active":"2024-05-09T06:38:31.246661+0000","last_peered":"2024-05-09T06:38:31.246661+0000","last_clean":"2024-05-09T06:38:31.246661+0000","last_became_active":"2024-05-09T06:38:29.768980+0000","last_became_peered":"2024-05-09T06:38:29.768980+0000","last_unstale":"2024-05-09T06:38:31.246661+0000","last_undegraded":"2024-05-09T06:38:31.246661+0000","last_fullsized":"2024-05-09T06:38:31.246661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:06.222662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246633+0000","last_change":"2024-05-09T06:38:29.759714+0000","last_active":"2024-05-09T06:38:31.246633+0000","last_peered":"2024-05-09T06:38:31.246633+0000","last_clean":"2024-05-09T06:38:31.246633+0000","last_became_active":"2024-05-09T06:38:29.759442+0000","last_became_peered":"2024-05-09T06:38:29.759442+0000","last_unstale":"2024-05-09T06:38:31.246633+0000","last_undegraded":"2024-05-09T06:38:31.246633+0000","last_fullsized":"2024-05-09T06:38:31.246633+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:48.150075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901441+0000","last_change":"2024-05-09T06:38:29.765065+0000","last_active":"2024-05-09T06:38:30.901441+0000","last_peered":"2024-05-09T06:38:30.901441+0000","last_clean":"2024-05-09T06:38:30.901441+0000","last_became_active":"2024-05-09T06:38:29.764776+0000","last_became_peered":"2024-05-09T06:38:29.764776+0000","last_unstale":"2024-05-09T06:38:30.901441+0000","last_undegraded":"2024-05-09T06:38:30.901441+0000","last_fullsized":"2024-05-09T06:38:30.901441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743304+0000","last_change":"2024-05-09T06:38:29.758633+0000","last_active":"2024-05-09T06:38:30.743304+0000","last_peered":"2024-05-09T06:38:30.743304+0000","last_clean":"2024-05-09T06:38:30.743304+0000","last_became_active":"2024-05-09T06:38:29.758274+0000","last_became_peered":"2024-05-09T06:38:29.758274+0000","last_unstale":"2024-05-09T06:38:30.743304+0000","last_undegraded":"2024-05-09T06:38:30.743304+0000","last_fullsized":"2024-05-09T06:38:30.743304+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:25.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742077+0000","last_change":"2024-05-09T06:38:29.754566+0000","last_active":"2024-05-09T06:38:30.742077+0000","last_peered":"2024-05-09T06:38:30.742077+0000","last_clean":"2024-05-09T06:38:30.742077+0000","last_became_active":"2024-05-09T06:38:29.754314+0000","last_became_peered":"2024-05-09T06:38:29.754314+0000","last_unstale":"2024-05-09T06:38:30.742077+0000","last_undegraded":"2024-05-09T06:38:30.742077+0000","last_fullsized":"2024-05-09T06:38:30.742077+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:17.620153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741956+0000","last_change":"2024-05-09T06:38:29.758534+0000","last_active":"2024-05-09T06:38:30.741956+0000","last_peered":"2024-05-09T06:38:30.741956+0000","last_clean":"2024-05-09T06:38:30.741956+0000","last_became_active":"2024-05-09T06:38:29.758263+0000","last_became_peered":"2024-05-09T06:38:29.758263+0000","last_unstale":"2024-05-09T06:38:30.741956+0000","last_undegraded":"2024-05-09T06:38:30.741956+0000","last_fullsized":"2024-05-09T06:38:30.741956+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:35.903972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247080+0000","last_change":"2024-05-09T06:38:29.761460+0000","last_active":"2024-05-09T06:38:31.247080+0000","last_peered":"2024-05-09T06:38:31.247080+0000","last_clean":"2024-05-09T06:38:31.247080+0000","last_became_active":"2024-05-09T06:38:29.761210+0000","last_became_peered":"2024-05-09T06:38:29.761210+0000","last_unstale":"2024-05-09T06:38:31.247080+0000","last_undegraded":"2024-05-09T06:38:31.247080+0000","last_fullsized":"2024-05-09T06:38:31.247080+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.802863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901605+0000","last_change":"2024-05-09T06:38:29.772536+0000","last_active":"2024-05-09T06:38:30.901605+0000","last_peered":"2024-05-09T06:38:30.901605+0000","last_clean":"2024-05-09T06:38:30.901605+0000","last_became_active":"2024-05-09T06:38:29.772205+0000","last_became_peered":"2024-05-09T06:38:29.772205+0000","last_unstale":"2024-05-09T06:38:30.901605+0000","last_undegraded":"2024-05-09T06:38:30.901605+0000","last_fullsized":"2024-05-09T06:38:30.901605+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245911+0000","last_change":"2024-05-09T06:38:29.751002+0000","last_active":"2024-05-09T06:38:31.245911+0000","last_peered":"2024-05-09T06:38:31.245911+0000","last_clean":"2024-05-09T06:38:31.245911+0000","last_became_active":"2024-05-09T06:38:29.750625+0000","last_became_peered":"2024-05-09T06:38:29.750625+0000","last_unstale":"2024-05-09T06:38:31.245911+0000","last_undegraded":"2024-05-09T06:38:31.245911+0000","last_fullsized":"2024-05-09T06:38:31.245911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743504+0000","last_change":"2024-05-09T06:38:29.749599+0000","last_active":"2024-05-09T06:38:30.743504+0000","last_peered":"2024-05-09T06:38:30.743504+0000","last_clean":"2024-05-09T06:38:30.743504+0000","last_became_active":"2024-05-09T06:38:29.749265+0000","last_became_peered":"2024-05-09T06:38:29.749265+0000","last_unstale":"2024-05-09T06:38:30.743504+0000","last_undegraded":"2024-05-09T06:38:30.743504+0000","last_fullsized":"2024-05-09T06:38:30.743504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742557+0000","last_change":"2024-05-09T06:38:29.759488+0000","last_active":"2024-05-09T06:38:30.742557+0000","last_peered":"2024-05-09T06:38:30.742557+0000","last_clean":"2024-05-09T06:38:30.742557+0000","last_became_active":"2024-05-09T06:38:29.759030+0000","last_became_peered":"2024-05-09T06:38:29.759030+0000","last_unstale":"2024-05-09T06:38:30.742557+0000","last_undegraded":"2024-05-09T06:38:30.742557+0000","last_fullsized":"2024-05-09T06:38:30.742557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:05.591808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246125+0000","last_change":"2024-05-09T06:38:29.756266+0000","last_active":"2024-05-09T06:38:31.246125+0000","last_peered":"2024-05-09T06:38:31.246125+0000","last_clean":"2024-05-09T06:38:31.246125+0000","last_became_active":"2024-05-09T06:38:29.755981+0000","last_became_peered":"2024-05-09T06:38:29.755981+0000","last_unstale":"2024-05-09T06:38:31.246125+0000","last_undegraded":"2024-05-09T06:38:31.246125+0000","last_fullsized":"2024-05-09T06:38:31.246125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246317+0000","last_change":"2024-05-09T06:38:29.758946+0000","last_active":"2024-05-09T06:38:31.246317+0000","last_peered":"2024-05-09T06:38:31.246317+0000","last_clean":"2024-05-09T06:38:31.246317+0000","last_became_active":"2024-05-09T06:38:29.758701+0000","last_became_peered":"2024-05-09T06:38:29.758701+0000","last_unstale":"2024-05-09T06:38:31.246317+0000","last_undegraded":"2024-05-09T06:38:31.246317+0000","last_fullsized":"2024-05-09T06:38:31.246317+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:56.734015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743234+0000","last_change":"2024-05-09T06:38:29.756392+0000","last_active":"2024-05-09T06:38:30.743234+0000","last_peered":"2024-05-09T06:38:30.743234+0000","last_clean":"2024-05-09T06:38:30.743234+0000","last_became_active":"2024-05-09T06:38:29.755337+0000","last_became_peered":"2024-05-09T06:38:29.755337+0000","last_unstale":"2024-05-09T06:38:30.743234+0000","last_undegraded":"2024-05-09T06:38:30.743234+0000","last_fullsized":"2024-05-09T06:38:30.743234+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:53.182411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741951+0000","last_change":"2024-05-09T06:38:29.757752+0000","last_active":"2024-05-09T06:38:30.741951+0000","last_peered":"2024-05-09T06:38:30.741951+0000","last_clean":"2024-05-09T06:38:30.741951+0000","last_became_active":"2024-05-09T06:38:29.757391+0000","last_became_peered":"2024-05-09T06:38:29.757391+0000","last_unstale":"2024-05-09T06:38:30.741951+0000","last_undegraded":"2024-05-09T06:38:30.741951+0000","last_fullsized":"2024-05-09T06:38:30.741951+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:01.176437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245939+0000","last_change":"2024-05-09T06:38:29.756836+0000","last_active":"2024-05-09T06:38:31.245939+0000","last_peered":"2024-05-09T06:38:31.245939+0000","last_clean":"2024-05-09T06:38:31.245939+0000","last_became_active":"2024-05-09T06:38:29.756583+0000","last_became_peered":"2024-05-09T06:38:29.756583+0000","last_unstale":"2024-05-09T06:38:31.245939+0000","last_undegraded":"2024-05-09T06:38:31.245939+0000","last_fullsized":"2024-05-09T06:38:31.245939+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246805+0000","last_change":"2024-05-09T06:38:29.769182+0000","last_active":"2024-05-09T06:38:31.246805+0000","last_peered":"2024-05-09T06:38:31.246805+0000","last_clean":"2024-05-09T06:38:31.246805+0000","last_became_active":"2024-05-09T06:38:29.768889+0000","last_became_peered":"2024-05-09T06:38:29.768889+0000","last_unstale":"2024-05-09T06:38:31.246805+0000","last_undegraded":"2024-05-09T06:38:31.246805+0000","last_fullsized":"2024-05-09T06:38:31.246805+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:46.068757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757312+0000","last_change":"2024-05-09T06:38:29.764995+0000","last_active":"2024-05-09T06:38:30.757312+0000","last_peered":"2024-05-09T06:38:30.757312+0000","last_clean":"2024-05-09T06:38:30.757312+0000","last_became_active":"2024-05-09T06:38:29.764704+0000","last_became_peered":"2024-05-09T06:38:29.764704+0000","last_unstale":"2024-05-09T06:38:30.757312+0000","last_undegraded":"2024-05-09T06:38:30.757312+0000","last_fullsized":"2024-05-09T06:38:30.757312+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246668+0000","last_change":"2024-05-09T06:38:29.746800+0000","last_active":"2024-05-09T06:38:31.246668+0000","last_peered":"2024-05-09T06:38:31.246668+0000","last_clean":"2024-05-09T06:38:31.246668+0000","last_became_active":"2024-05-09T06:38:29.746512+0000","last_became_peered":"2024-05-09T06:38:29.746512+0000","last_unstale":"2024-05-09T06:38:31.246668+0000","last_undegraded":"2024-05-09T06:38:31.246668+0000","last_fullsized":"2024-05-09T06:38:31.246668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:34.930379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901528+0000","last_change":"2024-05-09T06:38:29.761604+0000","last_active":"2024-05-09T06:38:30.901528+0000","last_peered":"2024-05-09T06:38:30.901528+0000","last_clean":"2024-05-09T06:38:30.901528+0000","last_became_active":"2024-05-09T06:38:29.761301+0000","last_became_peered":"2024-05-09T06:38:29.761301+0000","last_unstale":"2024-05-09T06:38:30.901528+0000","last_undegraded":"2024-05-09T06:38:30.901528+0000","last_fullsized":"2024-05-09T06:38:30.901528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743735+0000","last_change":"2024-05-09T06:38:29.769275+0000","last_active":"2024-05-09T06:38:30.743735+0000","last_peered":"2024-05-09T06:38:30.743735+0000","last_clean":"2024-05-09T06:38:30.743735+0000","last_became_active":"2024-05-09T06:38:29.769036+0000","last_became_peered":"2024-05-09T06:38:29.769036+0000","last_unstale":"2024-05-09T06:38:30.743735+0000","last_undegraded":"2024-05-09T06:38:30.743735+0000","last_fullsized":"2024-05-09T06:38:30.743735+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901252+0000","last_change":"2024-05-09T06:38:29.771176+0000","last_active":"2024-05-09T06:38:30.901252+0000","last_peered":"2024-05-09T06:38:30.901252+0000","last_clean":"2024-05-09T06:38:30.901252+0000","last_became_active":"2024-05-09T06:38:29.770832+0000","last_became_peered":"2024-05-09T06:38:29.770832+0000","last_unstale":"2024-05-09T06:38:30.901252+0000","last_undegraded":"2024-05-09T06:38:30.901252+0000","last_fullsized":"2024-05-09T06:38:30.901252+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783361+0000","last_change":"2024-05-09T06:38:29.760008+0000","last_active":"2024-05-09T06:38:30.783361+0000","last_peered":"2024-05-09T06:38:30.783361+0000","last_clean":"2024-05-09T06:38:30.783361+0000","last_became_active":"2024-05-09T06:38:29.759673+0000","last_became_peered":"2024-05-09T06:38:29.759673+0000","last_unstale":"2024-05-09T06:38:30.783361+0000","last_undegraded":"2024-05-09T06:38:30.783361+0000","last_fullsized":"2024-05-09T06:38:30.783361+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783832+0000","last_change":"2024-05-09T06:38:29.750626+0000","last_active":"2024-05-09T06:38:30.783832+0000","last_peered":"2024-05-09T06:38:30.783832+0000","last_clean":"2024-05-09T06:38:30.783832+0000","last_became_active":"2024-05-09T06:38:29.748935+0000","last_became_peered":"2024-05-09T06:38:29.748935+0000","last_unstale":"2024-05-09T06:38:30.783832+0000","last_undegraded":"2024-05-09T06:38:30.783832+0000","last_fullsized":"2024-05-09T06:38:30.783832+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.394253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246887+0000","last_change":"2024-05-09T06:38:29.751749+0000","last_active":"2024-05-09T06:38:31.246887+0000","last_peered":"2024-05-09T06:38:31.246887+0000","last_clean":"2024-05-09T06:38:31.246887+0000","last_became_active":"2024-05-09T06:38:29.751470+0000","last_became_peered":"2024-05-09T06:38:29.751470+0000","last_unstale":"2024-05-09T06:38:31.246887+0000","last_undegraded":"2024-05-09T06:38:31.246887+0000","last_fullsized":"2024-05-09T06:38:31.246887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:35:31.011590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757375+0000","last_change":"2024-05-09T06:38:29.761155+0000","last_active":"2024-05-09T06:38:30.757375+0000","last_peered":"2024-05-09T06:38:30.757375+0000","last_clean":"2024-05-09T06:38:30.757375+0000","last_became_active":"2024-05-09T06:38:29.760770+0000","last_became_peered":"2024-05-09T06:38:29.760770+0000","last_unstale":"2024-05-09T06:38:30.757375+0000","last_undegraded":"2024-05-09T06:38:30.757375+0000","last_fullsized":"2024-05-09T06:38:30.757375+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742629+0000","last_change":"2024-05-09T06:38:29.757022+0000","last_active":"2024-05-09T06:38:30.742629+0000","last_peered":"2024-05-09T06:38:30.742629+0000","last_clean":"2024-05-09T06:38:30.742629+0000","last_became_active":"2024-05-09T06:38:29.756765+0000","last_became_peered":"2024-05-09T06:38:29.756765+0000","last_unstale":"2024-05-09T06:38:30.742629+0000","last_undegraded":"2024-05-09T06:38:30.742629+0000","last_fullsized":"2024-05-09T06:38:30.742629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:44.514436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246699+0000","last_change":"2024-05-09T06:38:29.750100+0000","last_active":"2024-05-09T06:38:31.246699+0000","last_peered":"2024-05-09T06:38:31.246699+0000","last_clean":"2024-05-09T06:38:31.246699+0000","last_became_active":"2024-05-09T06:38:29.749832+0000","last_became_peered":"2024-05-09T06:38:29.749832+0000","last_unstale":"2024-05-09T06:38:31.246699+0000","last_undegraded":"2024-05-09T06:38:31.246699+0000","last_fullsized":"2024-05-09T06:38:31.246699+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:20.627213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246392+0000","last_change":"2024-05-09T06:38:29.769670+0000","last_active":"2024-05-09T06:38:31.246392+0000","last_peered":"2024-05-09T06:38:31.246392+0000","last_clean":"2024-05-09T06:38:31.246392+0000","last_became_active":"2024-05-09T06:38:29.768971+0000","last_became_peered":"2024-05-09T06:38:29.768971+0000","last_unstale":"2024-05-09T06:38:31.246392+0000","last_undegraded":"2024-05-09T06:38:31.246392+0000","last_fullsized":"2024-05-09T06:38:31.246392+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:21.469243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783415+0000","last_change":"2024-05-09T06:38:29.747737+0000","last_active":"2024-05-09T06:38:30.783415+0000","last_peered":"2024-05-09T06:38:30.783415+0000","last_clean":"2024-05-09T06:38:30.783415+0000","last_became_active":"2024-05-09T06:38:29.747419+0000","last_became_peered":"2024-05-09T06:38:29.747419+0000","last_unstale":"2024-05-09T06:38:30.783415+0000","last_undegraded":"2024-05-09T06:38:30.783415+0000","last_fullsized":"2024-05-09T06:38:30.783415+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783365+0000","last_change":"2024-05-09T06:38:29.759239+0000","last_active":"2024-05-09T06:38:30.783365+0000","last_peered":"2024-05-09T06:38:30.783365+0000","last_clean":"2024-05-09T06:38:30.783365+0000","last_became_active":"2024-05-09T06:38:29.758949+0000","last_became_peered":"2024-05-09T06:38:29.758949+0000","last_unstale":"2024-05-09T06:38:30.783365+0000","last_undegraded":"2024-05-09T06:38:30.783365+0000","last_fullsized":"2024-05-09T06:38:30.783365+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246146+0000","last_change":"2024-05-09T06:38:29.765597+0000","last_active":"2024-05-09T06:38:31.246146+0000","last_peered":"2024-05-09T06:38:31.246146+0000","last_clean":"2024-05-09T06:38:31.246146+0000","last_became_active":"2024-05-09T06:38:29.765299+0000","last_became_peered":"2024-05-09T06:38:29.765299+0000","last_unstale":"2024-05-09T06:38:31.246146+0000","last_undegraded":"2024-05-09T06:38:31.246146+0000","last_fullsized":"2024-05-09T06:38:31.246146+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757013+0000","last_change":"2024-05-09T06:38:28.737931+0000","last_active":"2024-05-09T06:38:30.757013+0000","last_peered":"2024-05-09T06:38:30.757013+0000","last_clean":"2024-05-09T06:38:30.757013+0000","last_became_active":"2024-05-09T06:38:28.737614+0000","last_became_peered":"2024-05-09T06:38:28.737614+0000","last_unstale":"2024-05-09T06:38:30.757013+0000","last_undegraded":"2024-05-09T06:38:30.757013+0000","last_fullsized":"2024-05-09T06:38:30.757013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:19.967238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742223+0000","last_change":"2024-05-09T06:38:29.761964+0000","last_active":"2024-05-09T06:38:30.742223+0000","last_peered":"2024-05-09T06:38:30.742223+0000","last_clean":"2024-05-09T06:38:30.742223+0000","last_became_active":"2024-05-09T06:38:29.761386+0000","last_became_peered":"2024-05-09T06:38:29.761386+0000","last_unstale":"2024-05-09T06:38:30.742223+0000","last_undegraded":"2024-05-09T06:38:30.742223+0000","last_fullsized":"2024-05-09T06:38:30.742223+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:46.892560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245972+0000","last_change":"2024-05-09T06:38:28.747690+0000","last_active":"2024-05-09T06:38:31.245972+0000","last_peered":"2024-05-09T06:38:31.245972+0000","last_clean":"2024-05-09T06:38:31.245972+0000","last_became_active":"2024-05-09T06:38:28.747413+0000","last_became_peered":"2024-05-09T06:38:28.747413+0000","last_unstale":"2024-05-09T06:38:31.245972+0000","last_undegraded":"2024-05-09T06:38:31.245972+0000","last_fullsized":"2024-05-09T06:38:31.245972+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783636+0000","last_change":"2024-05-09T06:38:29.754812+0000","last_active":"2024-05-09T06:38:30.783636+0000","last_peered":"2024-05-09T06:38:30.783636+0000","last_clean":"2024-05-09T06:38:30.783636+0000","last_became_active":"2024-05-09T06:38:29.754325+0000","last_became_peered":"2024-05-09T06:38:29.754325+0000","last_unstale":"2024-05-09T06:38:30.783636+0000","last_undegraded":"2024-05-09T06:38:30.783636+0000","last_fullsized":"2024-05-09T06:38:30.783636+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:35.399543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901338+0000","last_change":"2024-05-09T06:38:28.743221+0000","last_active":"2024-05-09T06:38:30.901338+0000","last_peered":"2024-05-09T06:38:30.901338+0000","last_clean":"2024-05-09T06:38:30.901338+0000","last_became_active":"2024-05-09T06:38:28.742941+0000","last_became_peered":"2024-05-09T06:38:28.742941+0000","last_unstale":"2024-05-09T06:38:30.901338+0000","last_undegraded":"2024-05-09T06:38:30.901338+0000","last_fullsized":"2024-05-09T06:38:30.901338+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901727+0000","last_change":"2024-05-09T06:38:29.762323+0000","last_active":"2024-05-09T06:38:30.901727+0000","last_peered":"2024-05-09T06:38:30.901727+0000","last_clean":"2024-05-09T06:38:30.901727+0000","last_became_active":"2024-05-09T06:38:29.761973+0000","last_became_peered":"2024-05-09T06:38:29.761973+0000","last_unstale":"2024-05-09T06:38:30.901727+0000","last_undegraded":"2024-05-09T06:38:30.901727+0000","last_fullsized":"2024-05-09T06:38:30.901727+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246269+0000","last_change":"2024-05-09T06:38:28.749687+0000","last_active":"2024-05-09T06:38:31.246269+0000","last_peered":"2024-05-09T06:38:31.246269+0000","last_clean":"2024-05-09T06:38:31.246269+0000","last_became_active":"2024-05-09T06:38:28.749395+0000","last_became_peered":"2024-05-09T06:38:28.749395+0000","last_unstale":"2024-05-09T06:38:31.246269+0000","last_undegraded":"2024-05-09T06:38:31.246269+0000","last_fullsized":"2024-05-09T06:38:31.246269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:14.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246176+0000","last_change":"2024-05-09T06:38:29.761152+0000","last_active":"2024-05-09T06:38:31.246176+0000","last_peered":"2024-05-09T06:38:31.246176+0000","last_clean":"2024-05-09T06:38:31.246176+0000","last_became_active":"2024-05-09T06:38:29.760869+0000","last_became_peered":"2024-05-09T06:38:29.760869+0000","last_unstale":"2024-05-09T06:38:31.246176+0000","last_undegraded":"2024-05-09T06:38:31.246176+0000","last_fullsized":"2024-05-09T06:38:31.246176+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743178+0000","last_change":"2024-05-09T06:38:28.750110+0000","last_active":"2024-05-09T06:38:30.743178+0000","last_peered":"2024-05-09T06:38:30.743178+0000","last_clean":"2024-05-09T06:38:30.743178+0000","last_became_active":"2024-05-09T06:38:28.749711+0000","last_became_peered":"2024-05-09T06:38:28.749711+0000","last_unstale":"2024-05-09T06:38:30.743178+0000","last_undegraded":"2024-05-09T06:38:30.743178+0000","last_fullsized":"2024-05-09T06:38:30.743178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743391+0000","last_change":"2024-05-09T06:38:29.763944+0000","last_active":"2024-05-09T06:38:30.743391+0000","last_peered":"2024-05-09T06:38:30.743391+0000","last_clean":"2024-05-09T06:38:30.743391+0000","last_became_active":"2024-05-09T06:38:29.763626+0000","last_became_peered":"2024-05-09T06:38:29.763626+0000","last_unstale":"2024-05-09T06:38:30.743391+0000","last_undegraded":"2024-05-09T06:38:30.743391+0000","last_fullsized":"2024-05-09T06:38:30.743391+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:00.819768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742147+0000","last_change":"2024-05-09T06:38:28.750014+0000","last_active":"2024-05-09T06:38:30.742147+0000","last_peered":"2024-05-09T06:38:30.742147+0000","last_clean":"2024-05-09T06:38:30.742147+0000","last_became_active":"2024-05-09T06:38:28.749739+0000","last_became_peered":"2024-05-09T06:38:28.749739+0000","last_unstale":"2024-05-09T06:38:30.742147+0000","last_undegraded":"2024-05-09T06:38:30.742147+0000","last_fullsized":"2024-05-09T06:38:30.742147+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:37.313308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246769+0000","last_change":"2024-05-09T06:38:29.754580+0000","last_active":"2024-05-09T06:38:31.246769+0000","last_peered":"2024-05-09T06:38:31.246769+0000","last_clean":"2024-05-09T06:38:31.246769+0000","last_became_active":"2024-05-09T06:38:29.754015+0000","last_became_peered":"2024-05-09T06:38:29.754015+0000","last_unstale":"2024-05-09T06:38:31.246769+0000","last_undegraded":"2024-05-09T06:38:31.246769+0000","last_fullsized":"2024-05-09T06:38:31.246769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:12.607951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783327+0000","last_change":"2024-05-09T06:38:28.747303+0000","last_active":"2024-05-09T06:38:30.783327+0000","last_peered":"2024-05-09T06:38:30.783327+0000","last_clean":"2024-05-09T06:38:30.783327+0000","last_became_active":"2024-05-09T06:38:28.746858+0000","last_became_peered":"2024-05-09T06:38:28.746858+0000","last_unstale":"2024-05-09T06:38:30.783327+0000","last_undegraded":"2024-05-09T06:38:30.783327+0000","last_fullsized":"2024-05-09T06:38:30.783327+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:15.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246290+0000","last_change":"2024-05-09T06:38:29.752770+0000","last_active":"2024-05-09T06:38:31.246290+0000","last_peered":"2024-05-09T06:38:31.246290+0000","last_clean":"2024-05-09T06:38:31.246290+0000","last_became_active":"2024-05-09T06:38:29.752478+0000","last_became_peered":"2024-05-09T06:38:29.752478+0000","last_unstale":"2024-05-09T06:38:31.246290+0000","last_undegraded":"2024-05-09T06:38:31.246290+0000","last_fullsized":"2024-05-09T06:38:31.246290+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:49.394253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742167+0000","last_change":"2024-05-09T06:38:28.749180+0000","last_active":"2024-05-09T06:38:30.742167+0000","last_peered":"2024-05-09T06:38:30.742167+0000","last_clean":"2024-05-09T06:38:30.742167+0000","last_became_active":"2024-05-09T06:38:28.748888+0000","last_became_peered":"2024-05-09T06:38:28.748888+0000","last_unstale":"2024-05-09T06:38:30.742167+0000","last_undegraded":"2024-05-09T06:38:30.742167+0000","last_fullsized":"2024-05-09T06:38:30.742167+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.798256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757193+0000","last_change":"2024-05-09T06:38:29.766343+0000","last_active":"2024-05-09T06:38:30.757193+0000","last_peered":"2024-05-09T06:38:30.757193+0000","last_clean":"2024-05-09T06:38:30.757193+0000","last_became_active":"2024-05-09T06:38:29.766008+0000","last_became_peered":"2024-05-09T06:38:29.766008+0000","last_unstale":"2024-05-09T06:38:30.757193+0000","last_undegraded":"2024-05-09T06:38:30.757193+0000","last_fullsized":"2024-05-09T06:38:30.757193+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757169+0000","last_change":"2024-05-09T06:38:28.740728+0000","last_active":"2024-05-09T06:38:30.757169+0000","last_peered":"2024-05-09T06:38:30.757169+0000","last_clean":"2024-05-09T06:38:30.757169+0000","last_became_active":"2024-05-09T06:38:28.740457+0000","last_became_peered":"2024-05-09T06:38:28.740457+0000","last_unstale":"2024-05-09T06:38:30.757169+0000","last_undegraded":"2024-05-09T06:38:30.757169+0000","last_fullsized":"2024-05-09T06:38:30.757169+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:52.177804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783704+0000","last_change":"2024-05-09T06:38:29.753396+0000","last_active":"2024-05-09T06:38:30.783704+0000","last_peered":"2024-05-09T06:38:30.783704+0000","last_clean":"2024-05-09T06:38:30.783704+0000","last_became_active":"2024-05-09T06:38:29.753036+0000","last_became_peered":"2024-05-09T06:38:29.753036+0000","last_unstale":"2024-05-09T06:38:30.783704+0000","last_undegraded":"2024-05-09T06:38:30.783704+0000","last_fullsized":"2024-05-09T06:38:30.783704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783554+0000","last_change":"2024-05-09T06:38:28.751282+0000","last_active":"2024-05-09T06:38:30.783554+0000","last_peered":"2024-05-09T06:38:30.783554+0000","last_clean":"2024-05-09T06:38:30.783554+0000","last_became_active":"2024-05-09T06:38:28.750962+0000","last_became_peered":"2024-05-09T06:38:28.750962+0000","last_unstale":"2024-05-09T06:38:30.783554+0000","last_undegraded":"2024-05-09T06:38:30.783554+0000","last_fullsized":"2024-05-09T06:38:30.783554+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246961+0000","last_change":"2024-05-09T06:38:29.760363+0000","last_active":"2024-05-09T06:38:31.246961+0000","last_peered":"2024-05-09T06:38:31.246961+0000","last_clean":"2024-05-09T06:38:31.246961+0000","last_became_active":"2024-05-09T06:38:29.760090+0000","last_became_peered":"2024-05-09T06:38:29.760090+0000","last_unstale":"2024-05-09T06:38:31.246961+0000","last_undegraded":"2024-05-09T06:38:31.246961+0000","last_fullsized":"2024-05-09T06:38:31.246961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:38.317915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783600+0000","last_change":"2024-05-09T06:38:28.748636+0000","last_active":"2024-05-09T06:38:30.783600+0000","last_peered":"2024-05-09T06:38:30.783600+0000","last_clean":"2024-05-09T06:38:30.783600+0000","last_became_active":"2024-05-09T06:38:28.748336+0000","last_became_peered":"2024-05-09T06:38:28.748336+0000","last_unstale":"2024-05-09T06:38:30.783600+0000","last_undegraded":"2024-05-09T06:38:30.783600+0000","last_fullsized":"2024-05-09T06:38:30.783600+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783801+0000","last_change":"2024-05-09T06:38:29.755192+0000","last_active":"2024-05-09T06:38:30.783801+0000","last_peered":"2024-05-09T06:38:30.783801+0000","last_clean":"2024-05-09T06:38:30.783801+0000","last_became_active":"2024-05-09T06:38:29.754772+0000","last_became_peered":"2024-05-09T06:38:29.754772+0000","last_unstale":"2024-05-09T06:38:30.783801+0000","last_undegraded":"2024-05-09T06:38:30.783801+0000","last_fullsized":"2024-05-09T06:38:30.783801+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:54.744971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246896+0000","last_change":"2024-05-09T06:38:28.745965+0000","last_active":"2024-05-09T06:38:31.246896+0000","last_peered":"2024-05-09T06:38:31.246896+0000","last_clean":"2024-05-09T06:38:31.246896+0000","last_became_active":"2024-05-09T06:38:28.745675+0000","last_became_peered":"2024-05-09T06:38:28.745675+0000","last_unstale":"2024-05-09T06:38:31.246896+0000","last_undegraded":"2024-05-09T06:38:31.246896+0000","last_fullsized":"2024-05-09T06:38:31.246896+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:10.080412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757175+0000","last_change":"2024-05-09T06:38:29.757835+0000","last_active":"2024-05-09T06:38:30.757175+0000","last_peered":"2024-05-09T06:38:30.757175+0000","last_clean":"2024-05-09T06:38:30.757175+0000","last_became_active":"2024-05-09T06:38:29.757188+0000","last_became_peered":"2024-05-09T06:38:29.757188+0000","last_unstale":"2024-05-09T06:38:30.757175+0000","last_undegraded":"2024-05-09T06:38:30.757175+0000","last_fullsized":"2024-05-09T06:38:30.757175+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246638+0000","last_change":"2024-05-09T06:38:28.743054+0000","last_active":"2024-05-09T06:38:31.246638+0000","last_peered":"2024-05-09T06:38:31.246638+0000","last_clean":"2024-05-09T06:38:31.246638+0000","last_became_active":"2024-05-09T06:38:28.742764+0000","last_became_peered":"2024-05-09T06:38:28.742764+0000","last_unstale":"2024-05-09T06:38:31.246638+0000","last_undegraded":"2024-05-09T06:38:31.246638+0000","last_fullsized":"2024-05-09T06:38:31.246638+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:34.394936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245820+0000","last_change":"2024-05-09T06:38:29.766170+0000","last_active":"2024-05-09T06:38:31.245820+0000","last_peered":"2024-05-09T06:38:31.245820+0000","last_clean":"2024-05-09T06:38:31.245820+0000","last_became_active":"2024-05-09T06:38:29.765908+0000","last_became_peered":"2024-05-09T06:38:29.765908+0000","last_unstale":"2024-05-09T06:38:31.245820+0000","last_undegraded":"2024-05-09T06:38:31.245820+0000","last_fullsized":"2024-05-09T06:38:31.245820+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:12.289279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246716+0000","last_change":"2024-05-09T06:38:28.746476+0000","last_active":"2024-05-09T06:38:31.246716+0000","last_peered":"2024-05-09T06:38:31.246716+0000","last_clean":"2024-05-09T06:38:31.246716+0000","last_became_active":"2024-05-09T06:38:28.746204+0000","last_became_peered":"2024-05-09T06:38:28.746204+0000","last_unstale":"2024-05-09T06:38:31.246716+0000","last_undegraded":"2024-05-09T06:38:31.246716+0000","last_fullsized":"2024-05-09T06:38:31.246716+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:48.389646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245765+0000","last_change":"2024-05-09T06:38:29.749306+0000","last_active":"2024-05-09T06:38:31.245765+0000","last_peered":"2024-05-09T06:38:31.245765+0000","last_clean":"2024-05-09T06:38:31.245765+0000","last_became_active":"2024-05-09T06:38:29.749017+0000","last_became_peered":"2024-05-09T06:38:29.749017+0000","last_unstale":"2024-05-09T06:38:31.245765+0000","last_undegraded":"2024-05-09T06:38:31.245765+0000","last_fullsized":"2024-05-09T06:38:31.245765+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:51.601822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247006+0000","last_change":"2024-05-09T06:38:28.745058+0000","last_active":"2024-05-09T06:38:31.247006+0000","last_peered":"2024-05-09T06:38:31.247006+0000","last_clean":"2024-05-09T06:38:31.247006+0000","last_became_active":"2024-05-09T06:38:28.744726+0000","last_became_peered":"2024-05-09T06:38:28.744726+0000","last_unstale":"2024-05-09T06:38:31.247006+0000","last_undegraded":"2024-05-09T06:38:31.247006+0000","last_fullsized":"2024-05-09T06:38:31.247006+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:53.740364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783470+0000","last_change":"2024-05-09T06:38:29.755533+0000","last_active":"2024-05-09T06:38:30.783470+0000","last_peered":"2024-05-09T06:38:30.783470+0000","last_clean":"2024-05-09T06:38:30.783470+0000","last_became_active":"2024-05-09T06:38:29.755144+0000","last_became_peered":"2024-05-09T06:38:29.755144+0000","last_unstale":"2024-05-09T06:38:30.783470+0000","last_undegraded":"2024-05-09T06:38:30.783470+0000","last_fullsized":"2024-05-09T06:38:30.783470+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:55:12.182702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246001+0000","last_change":"2024-05-09T06:38:28.748335+0000","last_active":"2024-05-09T06:38:31.246001+0000","last_peered":"2024-05-09T06:38:31.246001+0000","last_clean":"2024-05-09T06:38:31.246001+0000","last_became_active":"2024-05-09T06:38:28.747996+0000","last_became_peered":"2024-05-09T06:38:28.747996+0000","last_unstale":"2024-05-09T06:38:31.246001+0000","last_undegraded":"2024-05-09T06:38:31.246001+0000","last_fullsized":"2024-05-09T06:38:31.246001+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:57.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743711+0000","last_change":"2024-05-09T06:38:29.769319+0000","last_active":"2024-05-09T06:38:30.743711+0000","last_peered":"2024-05-09T06:38:30.743711+0000","last_clean":"2024-05-09T06:38:30.743711+0000","last_became_active":"2024-05-09T06:38:29.769020+0000","last_became_peered":"2024-05-09T06:38:29.769020+0000","last_unstale":"2024-05-09T06:38:30.743711+0000","last_undegraded":"2024-05-09T06:38:30.743711+0000","last_fullsized":"2024-05-09T06:38:30.743711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:16.068944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246945+0000","last_change":"2024-05-09T06:38:28.739768+0000","last_active":"2024-05-09T06:38:31.246945+0000","last_peered":"2024-05-09T06:38:31.246945+0000","last_clean":"2024-05-09T06:38:31.246945+0000","last_became_active":"2024-05-09T06:38:28.739460+0000","last_became_peered":"2024-05-09T06:38:28.739460+0000","last_unstale":"2024-05-09T06:38:31.246945+0000","last_undegraded":"2024-05-09T06:38:31.246945+0000","last_fullsized":"2024-05-09T06:38:31.246945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:18.604394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742591+0000","last_change":"2024-05-09T06:38:29.755262+0000","last_active":"2024-05-09T06:38:30.742591+0000","last_peered":"2024-05-09T06:38:30.742591+0000","last_clean":"2024-05-09T06:38:30.742591+0000","last_became_active":"2024-05-09T06:38:29.754999+0000","last_became_peered":"2024-05-09T06:38:29.754999+0000","last_unstale":"2024-05-09T06:38:30.742591+0000","last_undegraded":"2024-05-09T06:38:30.742591+0000","last_fullsized":"2024-05-09T06:38:30.742591+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:28.631222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901357+0000","last_change":"2024-05-09T06:38:28.749325+0000","last_active":"2024-05-09T06:38:30.901357+0000","last_peered":"2024-05-09T06:38:30.901357+0000","last_clean":"2024-05-09T06:38:30.901357+0000","last_became_active":"2024-05-09T06:38:28.748819+0000","last_became_peered":"2024-05-09T06:38:28.748819+0000","last_unstale":"2024-05-09T06:38:30.901357+0000","last_undegraded":"2024-05-09T06:38:30.901357+0000","last_fullsized":"2024-05-09T06:38:30.901357+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:26.916158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783894+0000","last_change":"2024-05-09T06:38:29.757014+0000","last_active":"2024-05-09T06:38:30.783894+0000","last_peered":"2024-05-09T06:38:30.783894+0000","last_clean":"2024-05-09T06:38:30.783894+0000","last_became_active":"2024-05-09T06:38:29.756267+0000","last_became_peered":"2024-05-09T06:38:29.756267+0000","last_unstale":"2024-05-09T06:38:30.783894+0000","last_undegraded":"2024-05-09T06:38:30.783894+0000","last_fullsized":"2024-05-09T06:38:30.783894+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:47.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901363+0000","last_change":"2024-05-09T06:38:28.749323+0000","last_active":"2024-05-09T06:38:30.901363+0000","last_peered":"2024-05-09T06:38:30.901363+0000","last_clean":"2024-05-09T06:38:30.901363+0000","last_became_active":"2024-05-09T06:38:28.748749+0000","last_became_peered":"2024-05-09T06:38:28.748749+0000","last_unstale":"2024-05-09T06:38:30.901363+0000","last_undegraded":"2024-05-09T06:38:30.901363+0000","last_fullsized":"2024-05-09T06:38:30.901363+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741950+0000","last_change":"2024-05-09T06:38:29.758725+0000","last_active":"2024-05-09T06:38:30.741950+0000","last_peered":"2024-05-09T06:38:30.741950+0000","last_clean":"2024-05-09T06:38:30.741950+0000","last_became_active":"2024-05-09T06:38:29.758465+0000","last_became_peered":"2024-05-09T06:38:29.758465+0000","last_unstale":"2024-05-09T06:38:30.741950+0000","last_undegraded":"2024-05-09T06:38:30.741950+0000","last_fullsized":"2024-05-09T06:38:30.741950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:54:09.412143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.741964+0000","last_change":"2024-05-09T06:38:28.747146+0000","last_active":"2024-05-09T06:38:30.741964+0000","last_peered":"2024-05-09T06:38:30.741964+0000","last_clean":"2024-05-09T06:38:30.741964+0000","last_became_active":"2024-05-09T06:38:28.746779+0000","last_became_peered":"2024-05-09T06:38:28.746779+0000","last_unstale":"2024-05-09T06:38:30.741964+0000","last_undegraded":"2024-05-09T06:38:30.741964+0000","last_fullsized":"2024-05-09T06:38:30.741964+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:45.064149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246046+0000","last_change":"2024-05-09T06:38:29.749910+0000","last_active":"2024-05-09T06:38:31.246046+0000","last_peered":"2024-05-09T06:38:31.246046+0000","last_clean":"2024-05-09T06:38:31.246046+0000","last_became_active":"2024-05-09T06:38:29.749656+0000","last_became_peered":"2024-05-09T06:38:29.749656+0000","last_unstale":"2024-05-09T06:38:31.246046+0000","last_undegraded":"2024-05-09T06:38:31.246046+0000","last_fullsized":"2024-05-09T06:38:31.246046+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.238142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901019+0000","last_change":"2024-05-09T06:38:28.740359+0000","last_active":"2024-05-09T06:38:30.901019+0000","last_peered":"2024-05-09T06:38:30.901019+0000","last_clean":"2024-05-09T06:38:30.901019+0000","last_became_active":"2024-05-09T06:38:28.740077+0000","last_became_peered":"2024-05-09T06:38:28.740077+0000","last_unstale":"2024-05-09T06:38:30.901019+0000","last_undegraded":"2024-05-09T06:38:30.901019+0000","last_fullsized":"2024-05-09T06:38:30.901019+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:57.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246658+0000","last_change":"2024-05-09T06:38:29.758211+0000","last_active":"2024-05-09T06:38:31.246658+0000","last_peered":"2024-05-09T06:38:31.246658+0000","last_clean":"2024-05-09T06:38:31.246658+0000","last_became_active":"2024-05-09T06:38:29.757719+0000","last_became_peered":"2024-05-09T06:38:29.757719+0000","last_unstale":"2024-05-09T06:38:31.246658+0000","last_undegraded":"2024-05-09T06:38:31.246658+0000","last_fullsized":"2024-05-09T06:38:31.246658+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.981345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247039+0000","last_change":"2024-05-09T06:38:28.747358+0000","last_active":"2024-05-09T06:38:31.247039+0000","last_peered":"2024-05-09T06:38:31.247039+0000","last_clean":"2024-05-09T06:38:31.247039+0000","last_became_active":"2024-05-09T06:38:28.746987+0000","last_became_peered":"2024-05-09T06:38:28.746987+0000","last_unstale":"2024-05-09T06:38:31.247039+0000","last_undegraded":"2024-05-09T06:38:31.247039+0000","last_fullsized":"2024-05-09T06:38:31.247039+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:42.705250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.783481+0000","last_change":"2024-05-09T06:38:29.748493+0000","last_active":"2024-05-09T06:38:30.783481+0000","last_peered":"2024-05-09T06:38:30.783481+0000","last_clean":"2024-05-09T06:38:30.783481+0000","last_became_active":"2024-05-09T06:38:29.748190+0000","last_became_peered":"2024-05-09T06:38:29.748190+0000","last_unstale":"2024-05-09T06:38:30.783481+0000","last_undegraded":"2024-05-09T06:38:30.783481+0000","last_fullsized":"2024-05-09T06:38:30.783481+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.900827+0000","last_change":"2024-05-09T06:38:28.742183+0000","last_active":"2024-05-09T06:38:30.900827+0000","last_peered":"2024-05-09T06:38:30.900827+0000","last_clean":"2024-05-09T06:38:30.900827+0000","last_became_active":"2024-05-09T06:38:28.741637+0000","last_became_peered":"2024-05-09T06:38:28.741637+0000","last_unstale":"2024-05-09T06:38:30.900827+0000","last_undegraded":"2024-05-09T06:38:30.900827+0000","last_fullsized":"2024-05-09T06:38:30.900827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:18.776523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742781+0000","last_change":"2024-05-09T06:38:29.757836+0000","last_active":"2024-05-09T06:38:30.742781+0000","last_peered":"2024-05-09T06:38:30.742781+0000","last_clean":"2024-05-09T06:38:30.742781+0000","last_became_active":"2024-05-09T06:38:29.757527+0000","last_became_peered":"2024-05-09T06:38:29.757527+0000","last_unstale":"2024-05-09T06:38:30.742781+0000","last_undegraded":"2024-05-09T06:38:30.742781+0000","last_fullsized":"2024-05-09T06:38:30.742781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:33.601288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742178+0000","last_change":"2024-05-09T06:38:28.751772+0000","last_active":"2024-05-09T06:38:30.742178+0000","last_peered":"2024-05-09T06:38:30.742178+0000","last_clean":"2024-05-09T06:38:30.742178+0000","last_became_active":"2024-05-09T06:38:28.751455+0000","last_became_peered":"2024-05-09T06:38:28.751455+0000","last_unstale":"2024-05-09T06:38:30.742178+0000","last_undegraded":"2024-05-09T06:38:30.742178+0000","last_fullsized":"2024-05-09T06:38:30.742178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:27:20.878031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246703+0000","last_change":"2024-05-09T06:38:29.760060+0000","last_active":"2024-05-09T06:38:31.246703+0000","last_peered":"2024-05-09T06:38:31.246703+0000","last_clean":"2024-05-09T06:38:31.246703+0000","last_became_active":"2024-05-09T06:38:29.759799+0000","last_became_peered":"2024-05-09T06:38:29.759799+0000","last_unstale":"2024-05-09T06:38:31.246703+0000","last_undegraded":"2024-05-09T06:38:31.246703+0000","last_fullsized":"2024-05-09T06:38:31.246703+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:50.306342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757198+0000","last_change":"2024-05-09T06:38:28.739861+0000","last_active":"2024-05-09T06:38:30.757198+0000","last_peered":"2024-05-09T06:38:30.757198+0000","last_clean":"2024-05-09T06:38:30.757198+0000","last_became_active":"2024-05-09T06:38:28.739521+0000","last_became_peered":"2024-05-09T06:38:28.739521+0000","last_unstale":"2024-05-09T06:38:30.757198+0000","last_undegraded":"2024-05-09T06:38:30.757198+0000","last_fullsized":"2024-05-09T06:38:30.757198+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.473368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901590+0000","last_change":"2024-05-09T06:38:29.761001+0000","last_active":"2024-05-09T06:38:30.901590+0000","last_peered":"2024-05-09T06:38:30.901590+0000","last_clean":"2024-05-09T06:38:30.901590+0000","last_became_active":"2024-05-09T06:38:29.760697+0000","last_became_peered":"2024-05-09T06:38:29.760697+0000","last_unstale":"2024-05-09T06:38:30.901590+0000","last_undegraded":"2024-05-09T06:38:30.901590+0000","last_fullsized":"2024-05-09T06:38:30.901590+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:11.085019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.757372+0000","last_change":"2024-05-09T06:38:28.739194+0000","last_active":"2024-05-09T06:38:30.757372+0000","last_peered":"2024-05-09T06:38:30.757372+0000","last_clean":"2024-05-09T06:38:30.757372+0000","last_became_active":"2024-05-09T06:38:28.738875+0000","last_became_peered":"2024-05-09T06:38:28.738875+0000","last_unstale":"2024-05-09T06:38:30.757372+0000","last_undegraded":"2024-05-09T06:38:30.757372+0000","last_fullsized":"2024-05-09T06:38:30.757372+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:24.807360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.247137+0000","last_change":"2024-05-09T06:38:29.751188+0000","last_active":"2024-05-09T06:38:31.247137+0000","last_peered":"2024-05-09T06:38:31.247137+0000","last_clean":"2024-05-09T06:38:31.247137+0000","last_became_active":"2024-05-09T06:38:29.750897+0000","last_became_peered":"2024-05-09T06:38:29.750897+0000","last_unstale":"2024-05-09T06:38:31.247137+0000","last_undegraded":"2024-05-09T06:38:31.247137+0000","last_fullsized":"2024-05-09T06:38:31.247137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:06:43.881398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742686+0000","last_change":"2024-05-09T06:38:28.750379+0000","last_active":"2024-05-09T06:38:30.742686+0000","last_peered":"2024-05-09T06:38:30.742686+0000","last_clean":"2024-05-09T06:38:30.742686+0000","last_became_active":"2024-05-09T06:38:28.750040+0000","last_became_peered":"2024-05-09T06:38:28.750040+0000","last_unstale":"2024-05-09T06:38:30.742686+0000","last_undegraded":"2024-05-09T06:38:30.742686+0000","last_fullsized":"2024-05-09T06:38:30.742686+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:12.256422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742109+0000","last_change":"2024-05-09T06:38:29.757931+0000","last_active":"2024-05-09T06:38:30.742109+0000","last_peered":"2024-05-09T06:38:30.742109+0000","last_clean":"2024-05-09T06:38:30.742109+0000","last_became_active":"2024-05-09T06:38:29.756980+0000","last_became_peered":"2024-05-09T06:38:29.756980+0000","last_unstale":"2024-05-09T06:38:30.742109+0000","last_undegraded":"2024-05-09T06:38:30.742109+0000","last_fullsized":"2024-05-09T06:38:30.742109+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:34.117615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743796+0000","last_change":"2024-05-09T06:38:28.742817+0000","last_active":"2024-05-09T06:38:30.743796+0000","last_peered":"2024-05-09T06:38:30.743796+0000","last_clean":"2024-05-09T06:38:30.743796+0000","last_became_active":"2024-05-09T06:38:28.742480+0000","last_became_peered":"2024-05-09T06:38:28.742480+0000","last_unstale":"2024-05-09T06:38:30.743796+0000","last_undegraded":"2024-05-09T06:38:30.743796+0000","last_fullsized":"2024-05-09T06:38:30.743796+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:59.247908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246204+0000","last_change":"2024-05-09T06:38:29.755565+0000","last_active":"2024-05-09T06:38:31.246204+0000","last_peered":"2024-05-09T06:38:31.246204+0000","last_clean":"2024-05-09T06:38:31.246204+0000","last_became_active":"2024-05-09T06:38:29.755291+0000","last_became_peered":"2024-05-09T06:38:29.755291+0000","last_unstale":"2024-05-09T06:38:31.246204+0000","last_undegraded":"2024-05-09T06:38:31.246204+0000","last_fullsized":"2024-05-09T06:38:31.246204+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:26.346432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.743680+0000","last_change":"2024-05-09T06:38:28.745054+0000","last_active":"2024-05-09T06:38:30.743680+0000","last_peered":"2024-05-09T06:38:30.743680+0000","last_clean":"2024-05-09T06:38:30.743680+0000","last_became_active":"2024-05-09T06:38:28.744751+0000","last_became_peered":"2024-05-09T06:38:28.744751+0000","last_unstale":"2024-05-09T06:38:30.743680+0000","last_undegraded":"2024-05-09T06:38:30.743680+0000","last_fullsized":"2024-05-09T06:38:30.743680+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:01.824438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901112+0000","last_change":"2024-05-09T06:38:29.762323+0000","last_active":"2024-05-09T06:38:30.901112+0000","last_peered":"2024-05-09T06:38:30.901112+0000","last_clean":"2024-05-09T06:38:30.901112+0000","last_became_active":"2024-05-09T06:38:29.762012+0000","last_became_peered":"2024-05-09T06:38:29.762012+0000","last_unstale":"2024-05-09T06:38:30.901112+0000","last_undegraded":"2024-05-09T06:38:30.901112+0000","last_fullsized":"2024-05-09T06:38:30.901112+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:45.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.742076+0000","last_change":"2024-05-09T06:38:28.752996+0000","last_active":"2024-05-09T06:38:30.742076+0000","last_peered":"2024-05-09T06:38:30.742076+0000","last_clean":"2024-05-09T06:38:30.742076+0000","last_became_active":"2024-05-09T06:38:28.752716+0000","last_became_peered":"2024-05-09T06:38:28.752716+0000","last_unstale":"2024-05-09T06:38:30.742076+0000","last_undegraded":"2024-05-09T06:38:30.742076+0000","last_fullsized":"2024-05-09T06:38:30.742076+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:32.408558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.246443+0000","last_change":"2024-05-09T06:38:29.764208+0000","last_active":"2024-05-09T06:38:31.246443+0000","last_peered":"2024-05-09T06:38:31.246443+0000","last_clean":"2024-05-09T06:38:31.246443+0000","last_became_active":"2024-05-09T06:38:29.763914+0000","last_became_peered":"2024-05-09T06:38:29.763914+0000","last_unstale":"2024-05-09T06:38:31.246443+0000","last_undegraded":"2024-05-09T06:38:31.246443+0000","last_fullsized":"2024-05-09T06:38:31.246443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:47.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.900853+0000","last_change":"2024-05-09T06:38:28.744382+0000","last_active":"2024-05-09T06:38:30.900853+0000","last_peered":"2024-05-09T06:38:30.900853+0000","last_clean":"2024-05-09T06:38:30.900853+0000","last_became_active":"2024-05-09T06:38:28.743975+0000","last_became_peered":"2024-05-09T06:38:28.743975+0000","last_unstale":"2024-05-09T06:38:30.900853+0000","last_undegraded":"2024-05-09T06:38:30.900853+0000","last_fullsized":"2024-05-09T06:38:30.900853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:49:15.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901629+0000","last_change":"2024-05-09T06:38:29.763309+0000","last_active":"2024-05-09T06:38:30.901629+0000","last_peered":"2024-05-09T06:38:30.901629+0000","last_clean":"2024-05-09T06:38:30.901629+0000","last_became_active":"2024-05-09T06:38:29.763008+0000","last_became_peered":"2024-05-09T06:38:29.763008+0000","last_unstale":"2024-05-09T06:38:30.901629+0000","last_undegraded":"2024-05-09T06:38:30.901629+0000","last_fullsized":"2024-05-09T06:38:30.901629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:28.724535+0000","last_clean_scrub_stamp":"2024-05-09T06:38:28.724535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:43.709857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:31.245977+0000","last_change":"2024-05-09T06:38:28.743982+0000","last_active":"2024-05-09T06:38:31.245977+0000","last_peered":"2024-05-09T06:38:31.245977+0000","last_clean":"2024-05-09T06:38:31.245977+0000","last_became_active":"2024-05-09T06:38:28.743653+0000","last_became_peered":"2024-05-09T06:38:28.743653+0000","last_unstale":"2024-05-09T06:38:31.245977+0000","last_undegraded":"2024-05-09T06:38:31.245977+0000","last_fullsized":"2024-05-09T06:38:31.245977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:38:27.719928+0000","last_clean_scrub_stamp":"2024-05-09T06:38:27.719928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:14:46.631441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":357,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-09T06:38:30.901005+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:38:30.901005+0000","last_peered":"2024-05-09T06:38:30.901005+0000","last_clean":"2024-05-09T06:38:30.901005+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:38:30.901005+0000","last_undegraded":"2024-05-09T06:38:30.901005+0000","last_fullsized":"2024-05-09T06:38:30.901005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2117632,"data_stored":1296829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":706267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":705845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":707297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1523712,"data_stored":707331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":705685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":705707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2105344,"data_stored":1296075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:38:34.102 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:38:34.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:38:34.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.270+0000 7f2e42184640 1 Processor -- start 2024-05-09T06:38:34.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.270+0000 7f2e42184640 1 -- start start 2024-05-09T06:38:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e3c15c4c0 0x7f2e3c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2e3c0b5260 0x7f2e3c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3c156980 con 0x7f2e3c0b5260 2024-05-09T06:38:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3c06d870 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e42184640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c15d320 con 0x7f2e3c0b4de0 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2e3c0b5260 0x7f2e3c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2e3c0b5260 0x7f2e3c15cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48006/0 (socket says 172.21.15.1:48006) 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 -- 172.21.15.1:0/4269536214 learned_addr learned my addr 172.21.15.1:0/4269536214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3affd640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3b7fe640 1 --2- 172.21.15.1:0/4269536214 >> v2:172.21.15.6:3300/0 conn(0x7f2e3c15c4c0 0x7f2e3c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 -- 172.21.15.1:0/4269536214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 msgr2=0x7f2e3c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 -- 172.21.15.1:0/4269536214 >> v2:172.21.15.6:3300/0 conn(0x7f2e3c15c4c0 msgr2=0x7f2e3c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/4269536214 >> v2:172.21.15.6:3300/0 conn(0x7f2e3c15c4c0 0x7f2e3c06dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.271+0000 7f2e3bfff640 1 -- 172.21.15.1:0/4269536214 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c06cf40 con 0x7f2e3c0b5260 2024-05-09T06:38:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.272+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c15cde0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c04fb80 tx=0x7f2e3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3d4edc75160afe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.272+0000 7f2e3a7fc640 1 -- 172.21.15.1:0/4269536214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34036070 con 0x7f2e3c0b5260 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.272+0000 7f2e3a7fc640 1 -- 172.21.15.1:0/4269536214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34031600 con 0x7f2e3c0b5260 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.272+0000 7f2e3a7fc640 1 -- 172.21.15.1:0/4269536214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340319d0 con 0x7f2e3c0b5260 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 -- 172.21.15.1:0/4269536214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c0b5260 msgr2=0x7f2e3c15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c15cde0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c04fb80 tx=0x7f2e3402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 -- 172.21.15.1:0/4269536214 shutdown_connections 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c15cde0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 --2- 172.21.15.1:0/4269536214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 --2- 172.21.15.1:0/4269536214 >> v2:172.21.15.6:3300/0 conn(0x7f2e3c15c4c0 0x7f2e3c06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 -- 172.21.15.1:0/4269536214 >> 172.21.15.1:0/4269536214 conn(0x7f2e3c0cc4d0 msgr2=0x7f2e3c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 -- 172.21.15.1:0/4269536214 shutdown_connections 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.273+0000 7f2e42184640 1 -- 172.21.15.1:0/4269536214 wait complete. 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.274+0000 7f2e42184640 1 Processor -- start 2024-05-09T06:38:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.274+0000 7f2e42184640 1 -- start start 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c0ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b2c40 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c06d500 con 0x7f2e3c0b5260 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e42184640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b2dc0 con 0x7f2e3c0b4de0 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48020/0 (socket says 172.21.15.1:48020) 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 -- 172.21.15.1:0/3806379485 learned_addr learned my addr 172.21.15.1:0/3806379485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3affd640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3b7fe640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c0ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 -- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 msgr2=0x7f2e3c0ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c0ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 -- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e3c0b5260 msgr2=0x7f2e3c155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.275+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e3bfff640 1 -- 172.21.15.1:0/3806379485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c156100 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e3bfff640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009f40 tx=0x7f2e34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34036050 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34042070 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c0a9ac0 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34009510 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.276+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c0aa080 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.277+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c0752b0 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.277+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e34007bf0 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.278+0000 7f2e23fff640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 0x7f2e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.278+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f2e340abcb0 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.278+0000 7f2e3b7fe640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 0x7f2e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.278+0000 7f2e3b7fe640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 0x7f2e2806e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009970 tx=0x7f2e2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.282+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e34045360 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.537+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2e3c0b11d0 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.540+0000 7f2e23fff640 1 -- 172.21.15.1:0/3806379485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f2e3407a560 con 0x7f2e3c15c4c0 2024-05-09T06:38:34.520 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T06:38:34.520 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCMbDxm4IFBExAAjNAtPHIZGKbKkIMWOMflNQ== 2024-05-09T06:38:34.520 INFO:teuthology.orchestra.run.smithi001.stdout: caps mds = "allow" 2024-05-09T06:38:34.520 INFO:teuthology.orchestra.run.smithi001.stdout: caps mon = "allow r" 2024-05-09T06:38:34.521 INFO:teuthology.orchestra.run.smithi001.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:38:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:updated caps for client.0 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 msgr2=0x7f2e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 0x7f2e2806e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009970 tx=0x7f2e2c008040 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 msgr2=0x7f2e3c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009f40 tx=0x7f2e34004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 shutdown_connections 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e2806be30 0x7f2e2806e2f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e3c15c4c0 0x7f2e3c155bc0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e3c0b5260 0x7f2e3c155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 --2- 172.21.15.1:0/3806379485 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e3c0b4de0 0x7f2e3c0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 >> 172.21.15.1:0/3806379485 conn(0x7f2e3c0cc4d0 msgr2=0x7f2e3c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 shutdown_connections 2024-05-09T06:38:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.542+0000 7f2e42184640 1 -- 172.21.15.1:0/3806379485 wait complete. 2024-05-09T06:38:34.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:34.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.657+0000 7f3fa3328640 1 Processor -- start 2024-05-09T06:38:34.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 -- start start 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3f9c161950 0x7f3f9c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 0x7f3f9c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9c0b33c0 con 0x7f3f9c0d0f80 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0d4dd0 con 0x7f3f9c161950 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.658+0000 7f3fa3328640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b3660 con 0x7f3f9c0d4940 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3f9c161950 0x7f3f9c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3f9c161950 0x7f3f9c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48032/0 (socket says 172.21.15.1:48032) 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 -- 172.21.15.1:0/1788038951 learned_addr learned my addr 172.21.15.1:0/1788038951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa18c5640 1 --1- 172.21.15.1:0/1788038951 >> v1:172.21.15.1:6789/0 conn(0x7f3f9c0d0f80 0x7f3f9c0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60274/0 (socket says 172.21.15.1:60274) 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa08c3640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 0x7f3f9c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 -- 172.21.15.1:0/1788038951 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 msgr2=0x7f3f9c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 0x7f3f9c0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 -- 172.21.15.1:0/1788038951 >> v1:172.21.15.1:6789/0 conn(0x7f3f9c0d0f80 legacy=0x7f3f9c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3f9bfff640 1 -- 172.21.15.1:0/1788038951 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286828528 0 0) 0x7f3f9c0b33c0 con 0x7f3f9c0d0f80 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa10c4640 1 -- 172.21.15.1:0/1788038951 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0b3940 con 0x7f3f9c161950 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.659+0000 7f3fa08c3640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 0x7f3f9c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c161950 0x7f3f9c0d2400 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b3590 tx=0x7f3f9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30da131136cbeb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3f9bfff640 1 -- 172.21.15.1:0/1788038951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3f9c161950 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3f9bfff640 1 -- 172.21.15.1:0/1788038951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94032440 con 0x7f3f9c161950 2024-05-09T06:38:34.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3f9bfff640 1 -- 172.21.15.1:0/1788038951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94032810 con 0x7f3f9c161950 2024-05-09T06:38:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 -- 172.21.15.1:0/1788038951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c161950 msgr2=0x7f3f9c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c161950 0x7f3f9c0d2400 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b3590 tx=0x7f3f9402a250 comp rx=0 tx=0).stop 2024-05-09T06:38:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 -- 172.21.15.1:0/1788038951 shutdown_connections 2024-05-09T06:38:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c0d4940 0x7f3f9c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 --2- 172.21.15.1:0/1788038951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c161950 0x7f3f9c0d2400 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.660+0000 7f3fa3328640 1 -- 172.21.15.1:0/1788038951 >> 172.21.15.1:0/1788038951 conn(0x7f3f9c0d3e20 msgr2=0x7f3f9c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.661+0000 7f3fa3328640 1 -- 172.21.15.1:0/1788038951 shutdown_connections 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.661+0000 7f3fa3328640 1 -- 172.21.15.1:0/1788038951 wait complete. 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 Processor -- start 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 -- start start 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 0x7f3f9c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c161950 0x7f3f9c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.662+0000 7f3fa3328640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b52d0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa3328640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b5150 con 0x7f3f9c0d0f80 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa3328640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b5450 con 0x7f3f9c161950 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48046/0 (socket says 172.21.15.1:48046) 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 -- 172.21.15.1:0/764430501 learned_addr learned my addr 172.21.15.1:0/764430501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa08c3640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c161950 0x7f3f9c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 0x7f3f9c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 -- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c161950 msgr2=0x7f3f9c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c161950 0x7f3f9c153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 -- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 msgr2=0x7f3f9c0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 0x7f3f9c0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 -- 172.21.15.1:0/764430501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c15c840 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 0x7f3f9c0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa18c5640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00c780 tx=0x7f3f8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c018070 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c00a640 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c0cd690 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.663+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00aab0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.664+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c0cbd80 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.665+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c0752b0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.665+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f8c01b070 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.665+0000 7f3f99ffb640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:34.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.666+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:34.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.666+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f3f8c08a590 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.666+0000 7f3fa10c4640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3f94034000 tx=0x7f3f9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.670+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f8c058db0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.700 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:38:34.719+0000 7f955ae29640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:38:34.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.896+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f9c0d21e0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.897+0000 7f3f99ffb640 1 -- 172.21.15.1:0/764430501 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5171 (secure 0 0 0) 0x7f3f8c0586b0 con 0x7f3f9c0b86d0 2024-05-09T06:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:32.125327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9636":{"gid":9636,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.1:6837/1060593353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1060593353},{"type":"v1","addr":"172.21.15.1:6837","nonce":1060593353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9636,"qdb_cluster":[9636]},"id":8}]} 2024-05-09T06:38:34.878 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 99 2024-05-09T06:38:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 msgr2=0x7f3f8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3f94034000 tx=0x7f3f9402e000 comp rx=0 tx=0).stop 2024-05-09T06:38:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 msgr2=0x7f3f9c0b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00c780 tx=0x7f3f8c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:38:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 shutdown_connections 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3f8806be60 0x7f3f8806e320 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f9c0b86d0 0x7f3f9c0b8b50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.898+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3f9c161950 0x7f3f9c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.899+0000 7f3fa3328640 1 --2- 172.21.15.1:0/764430501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f9c0d0f80 0x7f3f9c0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.899+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 >> 172.21.15.1:0/764430501 conn(0x7f3f9c0d3e20 msgr2=0x7f3f9c15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.899+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 shutdown_connections 2024-05-09T06:38:34.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:34.899+0000 7f3fa3328640 1 -- 172.21.15.1:0/764430501 wait complete. 2024-05-09T06:38:35.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:38:32.125327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9636': {'gid': 9636, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.1:6837/1060593353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 1060593353}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 1060593353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9636, 'qdb_cluster': [9636]} 2024-05-09T06:38:35.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:38:35.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:38:35.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.421+0000 7fd07d693640 1 Processor -- start 2024-05-09T06:38:35.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.421+0000 7fd07d693640 1 -- start start 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0780d4940 0x7fd0780d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 0x7fd0780cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd0780d4dd0 con 0x7fd0780d4940 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd0780b33c0 con 0x7fd0780d0f80 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd07d693640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780b3660 con 0x7fd078161950 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd0767fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0780d4940 0x7fd0780d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd0767fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd0780d4940 0x7fd0780d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48056/0 (socket says 172.21.15.1:48056) 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.422+0000 7fd0767fc640 1 -- 172.21.15.1:0/968250193 learned_addr learned my addr 172.21.15.1:0/968250193 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0767fc640 1 -- 172.21.15.1:0/968250193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 msgr2=0x7fd0780d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd076ffd640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0777fe640 1 --2- 172.21.15.1:0/968250193 >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 0x7fd0780cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0767fc640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0767fc640 1 -- 172.21.15.1:0/968250193 >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 msgr2=0x7fd0780cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0767fc640 1 --2- 172.21.15.1:0/968250193 >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 0x7fd0780cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0767fc640 1 -- 172.21.15.1:0/968250193 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0780b3940 con 0x7fd0780d4940 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd076ffd640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.423+0000 7fd0777fe640 1 --2- 172.21.15.1:0/968250193 >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 0x7fd0780cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd0767fc640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780d4940 0x7fd0780d2940 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd07808c710 tx=0x7fd06c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e56947a1310580 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd075ffb640 1 -- 172.21.15.1:0/968250193 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c037070 con 0x7fd0780d4940 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd075ffb640 1 -- 172.21.15.1:0/968250193 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c032440 con 0x7fd0780d4940 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd075ffb640 1 -- 172.21.15.1:0/968250193 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c032810 con 0x7fd0780d4940 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 -- 172.21.15.1:0/968250193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780d4940 msgr2=0x7fd0780d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780d4940 0x7fd0780d2940 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fd07808c710 tx=0x7fd06c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 -- 172.21.15.1:0/968250193 shutdown_connections 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 --2- 172.21.15.1:0/968250193 >> v2:172.21.15.6:3300/0 conn(0x7fd0780d0f80 0x7fd0780cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780d4940 0x7fd0780d2940 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 --2- 172.21.15.1:0/968250193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 -- 172.21.15.1:0/968250193 >> 172.21.15.1:0/968250193 conn(0x7fd0780d3e20 msgr2=0x7fd0780d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 -- 172.21.15.1:0/968250193 shutdown_connections 2024-05-09T06:38:35.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.424+0000 7fd07d693640 1 -- 172.21.15.1:0/968250193 wait complete. 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 Processor -- start 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 -- start start 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0780d0f80 0x7fd0780cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780d1d80 con 0x7fd0780a1350 2024-05-09T06:38:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780d1c00 con 0x7fd0780d0f80 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.426+0000 7fd07d693640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780d1f00 con 0x7fd078161950 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48070/0 (socket says 172.21.15.1:48070) 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 -- 172.21.15.1:0/2677311258 learned_addr learned my addr 172.21.15.1:0/2677311258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd076ffd640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0780d0f80 0x7fd0780cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0767fc640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 -- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 msgr2=0x7fd0780b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 -- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0780d0f80 msgr2=0x7fd0780cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0780d0f80 0x7fd0780cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 -- 172.21.15.1:0/2677311258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0781f3960 con 0x7fd0780a1350 2024-05-09T06:38:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd0777fe640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fd07000cee0 tx=0x7fd07000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd05f7fe640 1 -- 172.21.15.1:0/2677311258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07001a070 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd05f7fe640 1 -- 172.21.15.1:0/2677311258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07000d840 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0780b97c0 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd05f7fe640 1 -- 172.21.15.1:0/2677311258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000dc30 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.427+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07815aee0 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.429+0000 7fd05f7fe640 1 -- 172.21.15.1:0/2677311258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd07001d070 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.429+0000 7fd05f7fe640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 0x7fd06406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.429+0000 7fd05f7fe640 1 -- 172.21.15.1:0/2677311258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fd07008b760 con 0x7fd0780a1350 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.429+0000 7fd076ffd640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 0x7fd06406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.430+0000 7fd076ffd640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 0x7fd06406e320 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd06800aa40 tx=0x7fd068008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.430+0000 7fd08039e740 1 Processor -- start 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.430+0000 7fd08039e740 1 -- start start 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a80757a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a8072d350 0x561a807735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561a806904d0 con 0x561a80774f10 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561a806383e0 con 0x561a80757670 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561a8068c490 con 0x561a8072d350 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48082/0 (socket says 172.21.15.1:48082) 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 -- 172.21.15.1:0/1001254226 learned_addr learned my addr 172.21.15.1:0/1001254226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd076ffd640 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a80757a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0777fe640 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a8072d350 0x561a807735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 -- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a8072d350 msgr2=0x561a807735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a8072d350 0x561a807735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 -- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 msgr2=0x561a80757a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.431+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a80757a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd0767fc640 1 -- 172.21.15.1:0/1001254226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a803a42d0 con 0x561a80774f10 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fd06c035040 tx=0x7fd06c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe16b9a3163280 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd05cff9640 1 -- 172.21.15.1:0/1001254226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c043070 con 0x561a80774f10 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd05cff9640 1 -- 172.21.15.1:0/1001254226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c0327e0 con 0x561a80774f10 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd05cff9640 1 -- 172.21.15.1:0/1001254226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c037040 con 0x561a80774f10 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd08039e740 1 -- 172.21.15.1:0/1001254226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 msgr2=0x561a80775460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.432+0000 7fd08039e740 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fd06c035040 tx=0x7fd06c02c000 comp rx=0 tx=0).stop 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 -- 172.21.15.1:0/1001254226 shutdown_connections 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a8072d350 0x561a807735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80774f10 0x561a80775460 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 --2- 172.21.15.1:0/1001254226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a80757a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 -- 172.21.15.1:0/1001254226 >> 172.21.15.1:0/1001254226 conn(0x561a80729c80 msgr2=0x561a80772480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 -- 172.21.15.1:0/1001254226 shutdown_connections 2024-05-09T06:38:35.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 -- 172.21.15.1:0/1001254226 wait complete. 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 -- start start 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.433+0000 7fd08039e740 1 Processor -- start 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80757670 0x561a807826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80774f10 0x561a8075aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561a8072e0f0 con 0x561a807460a0 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561a8064b8e0 con 0x561a80774f10 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561a80727bb0 con 0x561a80757670 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd08039e740 10 client.0 _ll_register_callbacks cb 0x561a806189c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48086/0 (socket says 172.21.15.1:48086) 2024-05-09T06:38:35.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd0777fe640 1 -- 172.21.15.1:0/3129730017 learned_addr learned my addr 172.21.15.1:0/3129730017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80757670 0x561a807826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.434+0000 7fd0767fc640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80774f10 0x561a8075aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 -- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80757670 msgr2=0x561a807826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80757670 0x561a807826a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 -- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80774f10 msgr2=0x561a8075aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80774f10 0x561a8075aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 -- 172.21.15.1:0/3129730017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8046e6b0 con 0x561a807460a0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd0777fe640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd070061f30 tx=0x7fd070059a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5590282b003afbcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070057c30 con 0x561a807460a0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fd070061fd0 con 0x561a807460a0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd06c0327e0 con 0x561a807460a0 2024-05-09T06:38:35.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd0700621d0 con 0x561a807460a0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 10 client.9771 fetch_fsmap learned FSMap version 99 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x561a807832a0 con 0x561a807460a0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c0327e0 con 0x561a807460a0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fd07005abb0 con 0x561a807460a0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 10 client.9771 fetch_fsmap finished waiting for FSMap version 99 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 10 client.9771 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 20 client.9771 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a80759ca0 0x561a8076a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 4 client.9771 mds_command: new command op to 9636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.435+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 --> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a8069dc00 con 0x561a80759ca0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a80759ca0 0x561a8076a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd055ffb640 10 client.9771 ms_handle_connect on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a80759ca0 0x561a8076a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ca831baba3504e server_cookie=ea1971b7b08f87ce in_seq=0 out_seq=0 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd055ffb640 1 -- 172.21.15.1:0/3129730017 <== mds.0 v2:172.21.15.1:6836/1060593353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a8069dc00 con 0x561a80759ca0 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd055ffb640 10 client.9771 handle_command_reply: tid=0 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.436+0000 7fd08039e740 1 client.9771 shutdown 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 msgr2=0x561a8075af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd070061f30 tx=0x7fd070059a70 comp rx=0 tx=0).stop 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 shutdown_connections 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a80759ca0 0x561a8076a370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a8075af60 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80774f10 0x561a8075aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd055ffb640 0 client.9771 ms_handle_reset on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:35.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 --2- 172.21.15.1:0/3129730017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80757670 0x561a807826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 >> 172.21.15.1:0/3129730017 conn(0x561a8072d350 msgr2=0x561a80775a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 shutdown_connections 2024-05-09T06:38:35.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 1 -- 172.21.15.1:0/3129730017 wait complete. 2024-05-09T06:38:35.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.437+0000 7fd08039e740 20 client.9771 trim_cache size 0 max 16384 2024-05-09T06:38:35.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.455+0000 7fd08039e740 1 Processor -- start 2024-05-09T06:38:35.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.455+0000 7fd08039e740 1 -- start start 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a807e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a807e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781ffab0 con 0x561a807460a0 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781fee40 con 0x561a80757670 2024-05-09T06:38:35.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd0781ff460 con 0x561a80774f10 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48096/0 (socket says 172.21.15.1:48096) 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 -- 172.21.15.1:0/3365307196 learned_addr learned my addr 172.21.15.1:0/3365307196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 -- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 msgr2=0x561a807e6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a807e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 -- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 msgr2=0x561a807e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a807e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 -- 172.21.15.1:0/3365307196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a80757bf0 con 0x561a807460a0 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd076ffd640 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd06800f610 tx=0x7fd068016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c76308516acf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd057fff640 1 -- 172.21.15.1:0/3365307196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068015c50 con 0x561a807460a0 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd057fff640 1 -- 172.21.15.1:0/3365307196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068030070 con 0x561a807460a0 2024-05-09T06:38:35.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.456+0000 7fd057fff640 1 -- 172.21.15.1:0/3365307196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06802a3f0 con 0x561a807460a0 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- 172.21.15.1:0/3365307196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 msgr2=0x561a80726e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd06800f610 tx=0x7fd068016000 comp rx=0 tx=0).stop 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- 172.21.15.1:0/3365307196 shutdown_connections 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a807e6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a80757670 0x561a807e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 --2- 172.21.15.1:0/3365307196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a807460a0 0x561a80726e40 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- 172.21.15.1:0/3365307196 >> 172.21.15.1:0/3365307196 conn(0x561a8072d350 msgr2=0x561a8075a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- 172.21.15.1:0/3365307196 shutdown_connections 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- 172.21.15.1:0/3365307196 wait complete. 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.457+0000 7fd08039e740 1 -- start start 2024-05-09T06:38:35.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 Processor -- start 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a807460a0 0x561a807e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a8077ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561a80759b20 con 0x561a80757670 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x561a807835e0 con 0x561a807460a0 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x561a8075a170 con 0x561a80774f10 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd08039e740 10 client.0 _ll_register_callbacks cb 0x7fd06c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.458+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48106/0 (socket says 172.21.15.1:48106) 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 -- 172.21.15.1:0/1160434298 learned_addr learned my addr 172.21.15.1:0/1160434298 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0777fe640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a8077ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd076ffd640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a807460a0 0x561a807e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 -- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 msgr2=0x561a8077ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a8077ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 -- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a807460a0 msgr2=0x561a807e33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a807460a0 0x561a807e33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 -- 172.21.15.1:0/1160434298 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a80745850 con 0x561a80757670 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd0767fc640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd06c02a7d0 tx=0x7fd06c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c459feb0cf742a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c04c070 con 0x561a80757670 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fd06c003a20 con 0x561a80757670 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd068030070 con 0x561a80757670 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd06c037040 con 0x561a80757670 2024-05-09T06:38:35.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd08039e740 10 client.9777 fetch_fsmap learned FSMap version 99 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x561a807742f0 con 0x561a80757670 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.459+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068030070 con 0x561a80757670 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fd06c003c20 con 0x561a80757670 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 10 client.9777 fetch_fsmap finished waiting for FSMap version 99 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 10 client.9777 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 20 client.9777 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a807ed030 0x561a80758330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 4 client.9777 mds_command: new command op to 9636 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 --> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] -- command(tid 0: {"prefix": "status"}) -- 0x561a80697f50 con 0x561a807ed030 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd076ffd640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a807ed030 0x561a80758330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd076ffd640 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a807ed030 0x561a80758330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa09fd8b43a426c9 server_cookie=e6fa66fc6718b98d in_seq=0 out_seq=0 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd055ffb640 10 client.9777 ms_handle_connect on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:35.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd055ffb640 1 -- 172.21.15.1:0/1160434298 <== mds.0 v2:172.21.15.1:6836/1060593353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x561a80697f50 con 0x561a807ed030 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd055ffb640 10 client.9777 handle_command_reply: tid=0 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 client.9777 shutdown 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 msgr2=0x561a8077e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd06c02a7d0 tx=0x7fd06c004730 comp rx=0 tx=0).stop 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 shutdown_connections 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x561a807ed030 0x561a80758330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x561a80774f10 0x561a8077ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd055ffb640 0 client.9777 ms_handle_reset on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a80757670 0x561a8077e9e0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 --2- 172.21.15.1:0/1160434298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561a807460a0 0x561a807e33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.460+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 >> 172.21.15.1:0/1160434298 conn(0x561a8072d350 msgr2=0x561a80756ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 shutdown_connections 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd08039e740 1 -- 172.21.15.1:0/1160434298 wait complete. 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd08039e740 20 client.9777 trim_cache size 0 max 16384 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9636, 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:38:35.449 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 5.7117543939999997, 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 99, 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 121, 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 121, 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 10.775542632000001 2024-05-09T06:38:35.450 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 msgr2=0x7fd06406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 0x7fd06406e320 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd06800aa40 tx=0x7fd068008040 comp rx=0 tx=0).stop 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 msgr2=0x7fd0780b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fd07000cee0 tx=0x7fd07000a710 comp rx=0 tx=0).stop 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 shutdown_connections 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd06406be60 0x7fd06406e320 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0780a1350 0x7fd0780b7030 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd078161950 0x7fd0780b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 --2- 172.21.15.1:0/2677311258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0780d0f80 0x7fd0780cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 >> 172.21.15.1:0/2677311258 conn(0x7fd0780d3e20 msgr2=0x7fd07815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 shutdown_connections 2024-05-09T06:38:35.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.461+0000 7fd07d693640 1 -- 172.21.15.1:0/2677311258 wait complete. 2024-05-09T06:38:35.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:38:35.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.576+0000 7f4ec47b0640 1 Processor -- start 2024-05-09T06:38:35.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.576+0000 7f4ec47b0640 1 -- start start 2024-05-09T06:38:35.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec47b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec47b0640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc099930 con 0x7f4ebc099b30 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec47b0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc162930 con 0x7f4ebc0c6860 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec47b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc0b1e20 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec1d4b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4ebc099b30 0x7f4ebc15f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60288/0 (socket says 172.21.15.1:60288) 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec1d4b640 1 -- 172.21.15.1:0/3919102562 learned_addr learned my addr 172.21.15.1:0/3919102562 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.577+0000 7f4ec2d4d640 1 --2- 172.21.15.1:0/3919102562 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353394042 0 0) 0x7f4eac001110 con 0x7f4ebc099b30 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ea8003520 con 0x7f4ebc099b30 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/3919102562 >> v1:172.21.15.6:6789/0 conn(0x7f4ebc0c6860 legacy=0x7f4ebc0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778114965 0 0) 0x7f4ebc162930 con 0x7f4ebc0c6860 2024-05-09T06:38:35.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/3919102562 >> v1:172.21.15.1:6789/0 conn(0x7f4ebc099b30 legacy=0x7f4ebc15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/3919102562 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc162c10 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.578+0000 7f4ec2d4d640 1 --2- 172.21.15.1:0/3919102562 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15e4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009dc0 tx=0x7f4eb402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7b27ee4d0a0a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb402ae60 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4007740 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec154a640 1 -- 172.21.15.1:0/3919102562 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007af0 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 -- 172.21.15.1:0/3919102562 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 msgr2=0x7f4ebc15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/3919102562 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15e4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009dc0 tx=0x7f4eb402a640 comp rx=0 tx=0).stop 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 -- 172.21.15.1:0/3919102562 shutdown_connections 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/3919102562 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15e4b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 -- 172.21.15.1:0/3919102562 >> 172.21.15.1:0/3919102562 conn(0x7f4ebc0b1fb0 msgr2=0x7f4ebc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 -- 172.21.15.1:0/3919102562 shutdown_connections 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.579+0000 7f4ec47b0640 1 -- 172.21.15.1:0/3919102562 wait complete. 2024-05-09T06:38:35.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.580+0000 7f4ec47b0640 1 Processor -- start 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 -- start start 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 0x7f4ebc0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc15fa30 con 0x7f4ebc0c6860 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc15ee90 con 0x7f4ebc099b30 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec47b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc0cdf50 con 0x7f4ebc0b2aa0 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec2d4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec2d4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48110/0 (socket says 172.21.15.1:48110) 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.581+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/113749850 learned_addr learned my addr 172.21.15.1:0/113749850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec1d4b640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec254c640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 0x7f4ebc0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 msgr2=0x7f4ebc15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 msgr2=0x7f4ebc0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 0x7f4ebc0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 -- 172.21.15.1:0/113749850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc150c30 con 0x7f4ebc0c6860 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec254c640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 0x7f4ebc0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec2d4d640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402ac30 tx=0x7f4eb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009450 con 0x7f4ebc0c6860 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4004250 con 0x7f4ebc0c6860 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc160480 con 0x7f4ebc0c6860 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4004420 con 0x7f4ebc0c6860 2024-05-09T06:38:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.582+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc160a40 con 0x7f4ebc0c6860 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.583+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc0752b0 con 0x7f4ebc0c6860 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.584+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4eb4039040 con 0x7f4ebc0c6860 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.584+0000 7f4eb2ffd640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 0x7f4ea006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.584+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f4eb40ac440 con 0x7f4ebc0c6860 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.584+0000 7f4ec254c640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 0x7f4ea006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:35.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.585+0000 7f4ec254c640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 0x7f4ea006e320 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc1f7640 tx=0x7f4eb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:35.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.588+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4eb4040030 con 0x7f4ebc0c6860 2024-05-09T06:38:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.810+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ebc06ce30 con 0x7f4ebc0c6860 2024-05-09T06:38:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.810+0000 7f4eb2ffd640 1 -- 172.21.15.1:0/113749850 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f4eb407acf0 con 0x7f4ebc0c6860 2024-05-09T06:38:35.791 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 msgr2=0x7f4ea006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 0x7f4ea006e320 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc1f7640 tx=0x7f4eb8009040 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 msgr2=0x7f4ebc1506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402ac30 tx=0x7f4eb4004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 shutdown_connections 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4ea006be60 0x7f4ea006e320 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ebc0c6860 0x7f4ebc1506f0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ebc0b2aa0 0x7f4ebc15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 --2- 172.21.15.1:0/113749850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ebc099b30 0x7f4ebc0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 >> 172.21.15.1:0/113749850 conn(0x7f4ebc0b1fb0 msgr2=0x7f4ebc15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:35.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 shutdown_connections 2024-05-09T06:38:35.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:35.812+0000 7f4ec47b0640 1 -- 172.21.15.1:0/113749850 wait complete. 2024-05-09T06:38:35.803 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:38:35.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:38:35.804 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:35.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:35.804 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.821 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:38:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42026sec preferred_lft 42026sec 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:35.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:35.824 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:38:35.886 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:38:35.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:35.887 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:38:35.943 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:38:35.944 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:38:35.944 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:38:35.944 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:38:35.944 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:38:35.944 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:38:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:38:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:36.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.074 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:38:36.090 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:38:36.090 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:38:36.090 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:36.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:38:36.156 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:38:36.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:36.156 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:36.213 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:36.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:36.214 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:36.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:36.336 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:36.393 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:36.394 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:36.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.396 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:36.449 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:38:36.449 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:38:36.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:36.517 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:36.574 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:36.575 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:36.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.577 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:36.578 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:38:36.596+0000 7fab3a2e7480 -1 init, newargv = 0x55e007471bc0 newargc=15 2024-05-09T06:38:36.578 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[80124]: starting ceph client 2024-05-09T06:38:36.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[80124]: starting fuse 2024-05-09T06:38:36.594 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:38:36.594 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:38:36.596 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:38:36.597 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:38:36.597 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:36.597 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:38:36.597 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:36.715 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.80124.asok 2024-05-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:36.725 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80124.asok 2024-05-09T06:38:36.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80124.asok status 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:38:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "80124", 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9789, 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 9789 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 937471477 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 937471477 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.9789 192.168.0.1:0/937471477", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/937471477", 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:38:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 99, 2024-05-09T06:38:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 121, 2024-05-09T06:38:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:38:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:38:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:38:36.868 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:36.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.874 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:38:36.874 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:36.890 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:38:36.890 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:36.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:36.890 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> parent = os.path.join(mount_point, "parent") 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> os.mkdir(parent) 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> flushed = os.path.join(parent, "flushed") 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> os.mkdir(flushed) 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> for f in ["alpha", "bravo", "charlie"]: 2024-05-09T06:38:36.957 DEBUG:teuthology.orchestra.run.smithi001:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-09T06:38:36.958 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:37.089 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./ 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./parent 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/charlie 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/alpha 2024-05-09T06:38:37.155 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/bravo 2024-05-09T06:38:37.155 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.155 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.155 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.156 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.156 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-09T06:38:37.156 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.263 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:38:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-09T06:38:37.272 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.380 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:38:37.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.389 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.389 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.390 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.390 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.390 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-09T06:38:37.390 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.497 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.506 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.506 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.506 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.506 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.507 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-09T06:38:37.507 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628280 2024-05-09T06:38:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.623 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.623 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.624 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.624 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.624 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-09T06:38:37.624 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.732 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627778 2024-05-09T06:38:37.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.742 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:37.743 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:37.743 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:37.743 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:37.743 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-09T06:38:37.743 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:37.851 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628279 2024-05-09T06:38:37.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:37.860 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:37.860 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:37.917 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:38.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.080+0000 7f278b6e7640 1 Processor -- start 2024-05-09T06:38:38.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.080+0000 7f278b6e7640 1 -- start start 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f278b6e7640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f278406e6d0 0x7f278406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f278b6e7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840ce0e0 con 0x7f27840b3300 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f278b6e7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27840ce3c0 con 0x7f27840cf180 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f278b6e7640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f27840ce6a0 con 0x7f278406e6d0 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789c84640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f27840b3300 0x7f27840b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60828/0 (socket says 172.21.15.1:60828) 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789c84640 1 -- 172.21.15.1:0/4281352793 learned_addr learned my addr 172.21.15.1:0/4281352793 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789483640 1 --2- 172.21.15.1:0/4281352793 >> v2:172.21.15.6:3300/0 conn(0x7f278406e6d0 0x7f278406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f277bfff640 1 -- 172.21.15.1:0/4281352793 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492579913 0 0) 0x7f27840ce0e0 con 0x7f27840b3300 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f277bfff640 1 -- 172.21.15.1:0/4281352793 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2770003520 con 0x7f27840b3300 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789483640 1 -- 172.21.15.1:0/4281352793 >> v1:172.21.15.6:6789/0 conn(0x7f27840cf180 legacy=0x7f27840cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789483640 1 -- 172.21.15.1:0/4281352793 >> v1:172.21.15.1:6789/0 conn(0x7f27840b3300 legacy=0x7f27840b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.081+0000 7f2789483640 1 -- 172.21.15.1:0/4281352793 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27840ce980 con 0x7f278406e6d0 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.082+0000 7f2789483640 1 --2- 172.21.15.1:0/4281352793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278406eab0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27800097a0 tx=0x7f278002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6ea290f52ce862 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.082+0000 7f277bfff640 1 -- 172.21.15.1:0/4281352793 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780037070 con 0x7f278406e6d0 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.082+0000 7f277bfff640 1 -- 172.21.15.1:0/4281352793 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2780032440 con 0x7f278406e6d0 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.082+0000 7f277bfff640 1 -- 172.21.15.1:0/4281352793 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780032810 con 0x7f278406e6d0 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 -- 172.21.15.1:0/4281352793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 msgr2=0x7f278406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 --2- 172.21.15.1:0/4281352793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278406eab0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27800097a0 tx=0x7f278002a230 comp rx=0 tx=0).stop 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 -- 172.21.15.1:0/4281352793 shutdown_connections 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 --2- 172.21.15.1:0/4281352793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278406eab0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 -- 172.21.15.1:0/4281352793 >> 172.21.15.1:0/4281352793 conn(0x7f27840cc4b0 msgr2=0x7f27840cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 -- 172.21.15.1:0/4281352793 shutdown_connections 2024-05-09T06:38:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.083+0000 7f278b6e7640 1 -- 172.21.15.1:0/4281352793 wait complete. 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.084+0000 7f278b6e7640 1 Processor -- start 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.084+0000 7f278b6e7640 1 -- start start 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27840cf180 0x7f278406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2784070ed0 con 0x7f27840b3300 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f278406eff0 con 0x7f278406e6d0 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f278b6e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f27841447c0 con 0x7f27840cf180 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f2788c82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f2788c82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41660/0 (socket says 172.21.15.1:41660) 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f2788c82640 1 -- 172.21.15.1:0/2783424731 learned_addr learned my addr 172.21.15.1:0/2783424731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f2789483640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.085+0000 7f2789c84640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27840cf180 0x7f278406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 -- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27840cf180 msgr2=0x7f278406d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27840cf180 0x7f278406d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 -- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 msgr2=0x7f278415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 -- 172.21.15.1:0/2783424731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278415bbb0 con 0x7f27840b3300 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2788c82640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f277400c470 tx=0x7f2774001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740052d0 con 0x7f27840b3300 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2774005470 con 0x7f27840b3300 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278415be90 con 0x7f27840b3300 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774007260 con 0x7f27840b3300 2024-05-09T06:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.086+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278415d510 con 0x7f27840b3300 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.087+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278415d710 con 0x7f27840b3300 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.088+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2774007400 con 0x7f27840b3300 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.088+0000 7f2779ffb640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 0x7f276806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.088+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f277408a510 con 0x7f27840b3300 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.088+0000 7f2789483640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 0x7f276806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.089+0000 7f2789483640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 0x7f276806e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2780034000 tx=0x7f278002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.093+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f278415d710 con 0x7f27840b3300 2024-05-09T06:38:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.277+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27840752b0 con 0x7f27840b3300 2024-05-09T06:38:38.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.278+0000 7f2779ffb640 1 -- 172.21.15.1:0/2783424731 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16820 (secure 0 0 0) 0x7f2774058640 con 0x7f27840b3300 2024-05-09T06:38:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:38.259 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":121,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:30.738835+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.280+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 msgr2=0x7f276806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.280+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 0x7f276806e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2780034000 tx=0x7f278002e000 comp rx=0 tx=0).stop 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.280+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 msgr2=0x7f278415b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.280+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f277400c470 tx=0x7f2774001e70 comp rx=0 tx=0).stop 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 shutdown_connections 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f276806be30 0x7f276806e2f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f27840cf180 0x7f278406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27840b3300 0x7f278415b360 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 --2- 172.21.15.1:0/2783424731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278406e6d0 0x7f278415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 >> 172.21.15.1:0/2783424731 conn(0x7f27840cc4b0 msgr2=0x7f27840b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 shutdown_connections 2024-05-09T06:38:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.281+0000 7f278b6e7640 1 -- 172.21.15.1:0/2783424731 wait complete. 2024-05-09T06:38:38.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:38.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.399+0000 7fb1de027640 1 Processor -- start 2024-05-09T06:38:38.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.399+0000 7fb1de027640 1 -- start start 2024-05-09T06:38:38.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.399+0000 7fb1de027640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1de027640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb1d80d45f0 0x7fb1d80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1de027640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d8161fe0 con 0x7fb1d80d1c70 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1de027640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d815f580 con 0x7fb1d80d45f0 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1de027640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d81622c0 con 0x7fb1d815e190 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb1d80d45f0 0x7fb1d80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d77fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb1d80d45f0 0x7fb1d80b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:58300/0 (socket says 172.21.15.1:58300) 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1ceffd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb1d80d1c70 0x7fb1d80d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:43690/0 (socket says 172.21.15.1:43690) 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d77fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33556/0 (socket says 172.21.15.1:33556) 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 -- 172.21.15.1:0/1364638451 learned_addr learned my addr 172.21.15.1:0/1364638451 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 -- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 msgr2=0x7fb1d815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 --2- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 -- 172.21.15.1:0/1364638451 >> v1:172.21.15.6:6789/0 conn(0x7fb1d80d1c70 legacy=0x7fb1d80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.400+0000 7fb1d7fff640 1 -- 172.21.15.1:0/1364638451 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d81625a0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.401+0000 7fb1d7fff640 1 --2- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b3de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009e50 tx=0x7fb1d002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb32a4f1a75b51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.401+0000 7fb1d6ffd640 1 -- 172.21.15.1:0/1364638451 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0036070 con 0x7fb1d80d45f0 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.401+0000 7fb1d6ffd640 1 -- 172.21.15.1:0/1364638451 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d0031600 con 0x7fb1d80d45f0 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.401+0000 7fb1d6ffd640 1 -- 172.21.15.1:0/1364638451 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00319d0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 -- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d45f0 msgr2=0x7fb1d80b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 --2- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b3de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009e50 tx=0x7fb1d002a710 comp rx=0 tx=0).stop 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 -- 172.21.15.1:0/1364638451 shutdown_connections 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 --2- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b3de0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 --2- 172.21.15.1:0/1364638451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 -- 172.21.15.1:0/1364638451 >> 172.21.15.1:0/1364638451 conn(0x7fb1d8073340 msgr2=0x7fb1d809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 -- 172.21.15.1:0/1364638451 shutdown_connections 2024-05-09T06:38:38.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.402+0000 7fb1de027640 1 -- 172.21.15.1:0/1364638451 wait complete. 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.403+0000 7fb1de027640 1 Processor -- start 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.403+0000 7fb1de027640 1 -- start start 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 0x7fb1d814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d80b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d80b56d0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8161c60 con 0x7fb1d80d1c70 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1de027640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d80b5850 con 0x7fb1d815e190 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41672/0 (socket says 172.21.15.1:41672) 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 -- 172.21.15.1:0/1866402861 learned_addr learned my addr 172.21.15.1:0/1866402861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1d7fff640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d80b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1d77fe640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 0x7fb1d814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 -- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 msgr2=0x7fb1d80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d80b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 -- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 msgr2=0x7fb1d814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 0x7fb1d814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1ceffd640 1 -- 172.21.15.1:0/1866402861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d80b32b0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.404+0000 7fb1d77fe640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 0x7fb1d814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1ceffd640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c440 tx=0x7fb1c0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00074e0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0007680 con 0x7fb1d80d45f0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d80c7bd0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004a20 con 0x7fb1d80d45f0 2024-05-09T06:38:38.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.405+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d80c8190 con 0x7fb1d80d45f0 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.406+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d80752b0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.406+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1c001a020 con 0x7fb1d80d45f0 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.407+0000 7fb1d4ff9640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.407+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fb1c008a780 con 0x7fb1d80d45f0 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.407+0000 7fb1d77fe640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.407+0000 7fb1d77fe640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80d4490 tx=0x7fb1c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.411+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb1c0010300 con 0x7fb1d80d45f0 2024-05-09T06:38:38.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.595+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1d808e1c0 con 0x7fb1d80d45f0 2024-05-09T06:38:38.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.595+0000 7fb1d4ff9640 1 -- 172.21.15.1:0/1866402861 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7fb1c0059030 con 0x7fb1d80d45f0 2024-05-09T06:38:38.576 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 msgr2=0x7fb1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80d4490 tx=0x7fb1c8008040 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 msgr2=0x7fb1d80b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c440 tx=0x7fb1c0001e70 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 shutdown_connections 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1d815e190 0x7fb1d80b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1d80d45f0 0x7fb1d80b2d70 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 --2- 172.21.15.1:0/1866402861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1d80d1c70 0x7fb1d814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 >> 172.21.15.1:0/1866402861 conn(0x7fb1d8073340 msgr2=0x7fb1d80b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 shutdown_connections 2024-05-09T06:38:38.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.597+0000 7fb1de027640 1 -- 172.21.15.1:0/1866402861 wait complete. 2024-05-09T06:38:38.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:38:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.715+0000 7f18be15a640 1 Processor -- start 2024-05-09T06:38:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.715+0000 7f18be15a640 1 -- start start 2024-05-09T06:38:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18be15a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18b806dd00 0x7f18b815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18be15a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18be15a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b806cf40 con 0x7f18b815c4c0 2024-05-09T06:38:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18be15a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f18b8156980 con 0x7f18b806dd00 2024-05-09T06:38:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18be15a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18b806e180 con 0x7f18b815cde0 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33570/0 (socket says 172.21.15.1:33570) 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18b6ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18b806dd00 0x7f18b815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.716+0000 7f18b7fff640 1 -- 172.21.15.1:0/2461534873 learned_addr learned my addr 172.21.15.1:0/2461534873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.717+0000 7f18b7fff640 1 -- 172.21.15.1:0/2461534873 >> v2:172.21.15.6:3300/0 conn(0x7f18b806dd00 msgr2=0x7f18b815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.717+0000 7f18b7fff640 1 --2- 172.21.15.1:0/2461534873 >> v2:172.21.15.6:3300/0 conn(0x7f18b806dd00 0x7f18b815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.717+0000 7f18b7fff640 1 -- 172.21.15.1:0/2461534873 >> v1:172.21.15.6:6789/0 conn(0x7f18b815c4c0 legacy=0x7f18b80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.717+0000 7f18b7fff640 1 -- 172.21.15.1:0/2461534873 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b806d220 con 0x7f18b815cde0 2024-05-09T06:38:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.717+0000 7f18b7fff640 1 --2- 172.21.15.1:0/2461534873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009e50 tx=0x7f18b002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc62270c6e5e21a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18b67fc640 1 -- 172.21.15.1:0/2461534873 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0036070 con 0x7f18b815cde0 2024-05-09T06:38:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18b67fc640 1 -- 172.21.15.1:0/2461534873 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b0031600 con 0x7f18b815cde0 2024-05-09T06:38:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18b67fc640 1 -- 172.21.15.1:0/2461534873 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00319d0 con 0x7f18b815cde0 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 -- 172.21.15.1:0/2461534873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 msgr2=0x7f18b8070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 --2- 172.21.15.1:0/2461534873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009e50 tx=0x7f18b002a710 comp rx=0 tx=0).stop 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 -- 172.21.15.1:0/2461534873 shutdown_connections 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 --2- 172.21.15.1:0/2461534873 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815cde0 0x7f18b8070c10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 --2- 172.21.15.1:0/2461534873 >> v2:172.21.15.6:3300/0 conn(0x7f18b806dd00 0x7f18b815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 -- 172.21.15.1:0/2461534873 >> 172.21.15.1:0/2461534873 conn(0x7f18b80cc4d0 msgr2=0x7f18b80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 -- 172.21.15.1:0/2461534873 shutdown_connections 2024-05-09T06:38:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.718+0000 7f18be15a640 1 -- 172.21.15.1:0/2461534873 wait complete. 2024-05-09T06:38:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.719+0000 7f18be15a640 1 Processor -- start 2024-05-09T06:38:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 -- start start 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b806dd00 0x7f18b8137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815c4c0 0x7f18b8135260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18b815d270 con 0x7f18b815cde0 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f18b806d870 con 0x7f18b806dd00 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.720+0000 7f18be15a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f18b80b54f0 con 0x7f18b815c4c0 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41682/0 (socket says 172.21.15.1:41682) 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 -- 172.21.15.1:0/3308540516 learned_addr learned my addr 172.21.15.1:0/3308540516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b6ffd640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815c4c0 0x7f18b8135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b77fe640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b806dd00 0x7f18b8137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 -- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815c4c0 msgr2=0x7f18b8135260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815c4c0 0x7f18b8135260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 -- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b806dd00 msgr2=0x7f18b8137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b806dd00 0x7f18b8137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 -- 172.21.15.1:0/3308540516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b815a860 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18b7fff640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f18b0034040 tx=0x7f18b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0041020 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b00077a0 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b815ab40 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0036040 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.721+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b815b100 con 0x7f18b815cde0 2024-05-09T06:38:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.722+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b80752b0 con 0x7f18b815cde0 2024-05-09T06:38:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.723+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18b0044020 con 0x7f18b815cde0 2024-05-09T06:38:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.723+0000 7f189ffff640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.723+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f18b00aba40 con 0x7f18b815cde0 2024-05-09T06:38:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.723+0000 7f18b77fe640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.724+0000 7f18b77fe640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 0x7f18a406e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f18a8001780 tx=0x7f18a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.728+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18b004b030 con 0x7f18b815cde0 2024-05-09T06:38:38.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.913+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18b80b19f0 con 0x7f18b815cde0 2024-05-09T06:38:38.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.913+0000 7f189ffff640 1 -- 172.21.15.1:0/3308540516 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f18b00abe20 con 0x7f18b815cde0 2024-05-09T06:38:38.894 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 msgr2=0x7f18a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 0x7f18a406e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f18a8001780 tx=0x7f18a8009040 comp rx=0 tx=0).stop 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 msgr2=0x7f18b80ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f18b0034040 tx=0x7f18b0004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 shutdown_connections 2024-05-09T06:38:38.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18b815cde0 0x7f18b80ac3c0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18b815c4c0 0x7f18b8135260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 --2- 172.21.15.1:0/3308540516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18b806dd00 0x7f18b8137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.915+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 >> 172.21.15.1:0/3308540516 conn(0x7f18b80cc4d0 msgr2=0x7f18b80afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.916+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 shutdown_connections 2024-05-09T06:38:38.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:38.916+0000 7f18be15a640 1 -- 172.21.15.1:0/3308540516 wait complete. 2024-05-09T06:38:38.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:38:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.034+0000 7ff4252f5640 1 Processor -- start 2024-05-09T06:38:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.034+0000 7ff4252f5640 1 -- start start 2024-05-09T06:38:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff4252f5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff42006dd00 0x7ff42015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff4252f5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42006cf40 con 0x7ff42015cde0 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff4252f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42006d220 con 0x7ff42015c4c0 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff4252f5640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff420156980 con 0x7ff42006dd00 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff417fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff42006dd00 0x7ff42015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff41f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff42015cde0 0x7ff420070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60832/0 (socket says 172.21.15.1:60832) 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff41f7fe640 1 -- 172.21.15.1:0/883481873 learned_addr learned my addr 172.21.15.1:0/883481873 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff417fff640 1 -- 172.21.15.1:0/883481873 >> v1:172.21.15.6:6789/0 conn(0x7ff42015c4c0 legacy=0x7ff4200b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff417fff640 1 -- 172.21.15.1:0/883481873 >> v1:172.21.15.1:6789/0 conn(0x7ff42015cde0 legacy=0x7ff420070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.035+0000 7ff417fff640 1 -- 172.21.15.1:0/883481873 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42006d500 con 0x7ff42006dd00 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.036+0000 7ff417fff640 1 --2- 172.21.15.1:0/883481873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42006dd00 0x7ff42015c8a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ff4080097a0 tx=0x7ff40802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5504f15c2616e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.036+0000 7ff41e7fc640 1 -- 172.21.15.1:0/883481873 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408037070 con 0x7ff42006dd00 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.036+0000 7ff41e7fc640 1 -- 172.21.15.1:0/883481873 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff408032440 con 0x7ff42006dd00 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.036+0000 7ff41e7fc640 1 -- 172.21.15.1:0/883481873 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408032810 con 0x7ff42006dd00 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 -- 172.21.15.1:0/883481873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42006dd00 msgr2=0x7ff42015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 --2- 172.21.15.1:0/883481873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42006dd00 0x7ff42015c8a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ff4080097a0 tx=0x7ff40802a230 comp rx=0 tx=0).stop 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 -- 172.21.15.1:0/883481873 shutdown_connections 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 --2- 172.21.15.1:0/883481873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42006dd00 0x7ff42015c8a0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 -- 172.21.15.1:0/883481873 >> 172.21.15.1:0/883481873 conn(0x7ff4200cc4d0 msgr2=0x7ff4200cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 -- 172.21.15.1:0/883481873 shutdown_connections 2024-05-09T06:38:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.037+0000 7ff4252f5640 1 -- 172.21.15.1:0/883481873 wait complete. 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.038+0000 7ff4252f5640 1 Processor -- start 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.038+0000 7ff4252f5640 1 -- start start 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42006dd00 0x7ff4201592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff42015c4c0 0x7ff4200aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff42006d870 con 0x7ff42015cde0 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff42006e180 con 0x7ff42006dd00 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff4252f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff4200b54f0 con 0x7ff42015c4c0 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41698/0 (socket says 172.21.15.1:41698) 2024-05-09T06:38:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 -- 172.21.15.1:0/1601127747 learned_addr learned my addr 172.21.15.1:0/1601127747 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff417fff640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff42015c4c0 0x7ff4200aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41effd640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42006dd00 0x7ff4201592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 -- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff42015c4c0 msgr2=0x7ff4200aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff42015c4c0 0x7ff4200aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 -- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42006dd00 msgr2=0x7ff4201592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42006dd00 0x7ff4201592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.039+0000 7ff41f7fe640 1 -- 172.21.15.1:0/1601127747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff420158250 con 0x7ff42015cde0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff41f7fe640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7ff41800cbc0 tx=0x7ff41800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418017070 con 0x7ff42015cde0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41800ad40 con 0x7ff42015cde0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418012600 con 0x7ff42015cde0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff420158530 con 0x7ff42015cde0 2024-05-09T06:38:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.040+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff420158af0 con 0x7ff42015cde0 2024-05-09T06:38:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.041+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4200752b0 con 0x7ff42015cde0 2024-05-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.041+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff41801a070 con 0x7ff42015cde0 2024-05-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.042+0000 7ff4177fe640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 0x7ff40006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.042+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7ff41808ac70 con 0x7ff42015cde0 2024-05-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.042+0000 7ff41effd640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 0x7ff40006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.042+0000 7ff41effd640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 0x7ff40006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4201599d0 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.046+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff418058e40 con 0x7ff42015cde0 2024-05-09T06:38:39.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.227+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff42015f410 con 0x7ff42015cde0 2024-05-09T06:38:39.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.228+0000 7ff4177fe640 1 -- 172.21.15.1:0/1601127747 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7ff41805d140 con 0x7ff42015cde0 2024-05-09T06:38:39.208 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:38:39.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 msgr2=0x7ff40006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 0x7ff40006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4201599d0 tx=0x7ff410009040 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 msgr2=0x7ff42015ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7ff41800cbc0 tx=0x7ff41800a430 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 shutdown_connections 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff40006be30 0x7ff40006e2f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff42015cde0 0x7ff42015ab60 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff42015c4c0 0x7ff4200aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 --2- 172.21.15.1:0/1601127747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff42006dd00 0x7ff4201592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 >> 172.21.15.1:0/1601127747 conn(0x7ff4200cc4d0 msgr2=0x7ff4200af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 shutdown_connections 2024-05-09T06:38:39.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.230+0000 7ff4252f5640 1 -- 172.21.15.1:0/1601127747 wait complete. 2024-05-09T06:38:39.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:38:39.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.346+0000 7fb5db5af640 1 Processor -- start 2024-05-09T06:38:39.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.346+0000 7fb5db5af640 1 -- start start 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5db5af640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5d4166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5db5af640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d40ce0e0 con 0x7fb5d406e6d0 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5db5af640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d40ce3c0 con 0x7fb5d40cf180 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5db5af640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d40ce6a0 con 0x7fb5d4163c70 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5d934b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb5d406e6d0 0x7fb5d406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60848/0 (socket says 172.21.15.1:60848) 2024-05-09T06:38:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5d934b640 1 -- 172.21.15.1:0/2100943112 learned_addr learned my addr 172.21.15.1:0/2100943112 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063933671 0 0) 0x7fb5d40ce0e0 con 0x7fb5d406e6d0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5d9b4c640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5d4166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.347+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5c0003520 con 0x7fb5d406e6d0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 631454645 0 0) 0x7fb5c0003520 con 0x7fb5d406e6d0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 msgr2=0x7fb5d4166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5d4166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 >> v1:172.21.15.6:6789/0 conn(0x7fb5d40cf180 legacy=0x7fb5d40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d40ce980 con 0x7fb5d406e6d0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb5d0001670 con 0x7fb5d406e6d0 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 >> v1:172.21.15.1:6789/0 conn(0x7fb5d406e6d0 legacy=0x7fb5d406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c00039e0 0x7fb5c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40ce6a0 con 0x7fb5c0004800 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c000ba70 con 0x7fb5c00039e0 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c000de80 con 0x7fb5d4163c70 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d934b640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c00039e0 0x7fb5c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d9b4c640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 msgr2=0x7fb5c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5d4163c70 0x7fb5c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c00039e0 msgr2=0x7fb5c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c00039e0 0x7fb5c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.348+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/2100943112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c0003520 con 0x7fb5c0004800 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40cf5f0 tx=0x7fb5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8300f9835df87ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8001370 con 0x7fb5c0004800 2024-05-09T06:38:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c80052d0 con 0x7fb5c0004800 2024-05-09T06:38:39.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5c7fff640 1 -- 172.21.15.1:0/2100943112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c802ab70 con 0x7fb5c0004800 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5db5af640 1 -- 172.21.15.1:0/2100943112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5c0004800 msgr2=0x7fb5c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5db5af640 1 --2- 172.21.15.1:0/2100943112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40cf5f0 tx=0x7fb5c0005830 comp rx=0 tx=0).stop 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.349+0000 7fb5d9b4c640 1 -- 172.21.15.1:0/2100943112 reap_dead start 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.350+0000 7fb5db5af640 1 -- 172.21.15.1:0/2100943112 shutdown_connections 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.350+0000 7fb5db5af640 1 -- 172.21.15.1:0/2100943112 >> 172.21.15.1:0/2100943112 conn(0x7fb5d40cc4b0 msgr2=0x7fb5d40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.350+0000 7fb5db5af640 1 -- 172.21.15.1:0/2100943112 shutdown_connections 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.350+0000 7fb5db5af640 1 -- 172.21.15.1:0/2100943112 wait complete. 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.351+0000 7fb5db5af640 1 Processor -- start 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.351+0000 7fb5db5af640 1 -- start start 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5c0004800 0x7fb5d40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 0x7fb5d40ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41447c0 con 0x7fb5d4158fe0 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4070ed0 con 0x7fb5d413d320 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d4144940 con 0x7fb5c0004800 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41720/0 (socket says 172.21.15.1:41720) 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/3842663897 learned_addr learned my addr 172.21.15.1:0/3842663897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d934b640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5c0004800 0x7fb5d40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d9b4c640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 0x7fb5d40ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5c0004800 msgr2=0x7fb5d40aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5c0004800 0x7fb5d40aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 msgr2=0x7fb5d40ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 0x7fb5d40ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 -- 172.21.15.1:0/3842663897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d413c2a0 con 0x7fb5d4158fe0 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d9b4c640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 0x7fb5d40ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:39.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5d8b4a640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80017e0 tx=0x7fb5c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8040020 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c8005480 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d413c560 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.352+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8001370 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.353+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d413cb20 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.354+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d4140f10 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.354+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5c8043020 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.355+0000 7fb5c5ffb640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 0x7fb5b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.355+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fb5c802be00 con 0x7fb5d4158fe0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.355+0000 7fb5d934b640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 0x7fb5b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.355+0000 7fb5d934b640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 0x7fb5b806e2f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0001cd0 tx=0x7fb5d0000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.359+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5c804a030 con 0x7fb5d4158fe0 2024-05-09T06:38:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.553+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5d40752b0 con 0x7fb5d4158fe0 2024-05-09T06:38:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.554+0000 7fb5c5ffb640 1 -- 172.21.15.1:0/3842663897 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7fb5c80798a0 con 0x7fb5d4158fe0 2024-05-09T06:38:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:38:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.555+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 msgr2=0x7fb5b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.555+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 0x7fb5b806e2f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0001cd0 tx=0x7fb5d0000ed0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 msgr2=0x7fb5d40ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80017e0 tx=0x7fb5c8001120 comp rx=0 tx=0).stop 2024-05-09T06:38:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 shutdown_connections 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb5b806be30 0x7fb5b806e2f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5d413d320 0x7fb5d40ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb5d4158fe0 0x7fb5d40ac240 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 --2- 172.21.15.1:0/3842663897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5c0004800 0x7fb5d40aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 >> 172.21.15.1:0/3842663897 conn(0x7fb5d40cc4b0 msgr2=0x7fb5d406d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 shutdown_connections 2024-05-09T06:38:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.556+0000 7fb5db5af640 1 -- 172.21.15.1:0/3842663897 wait complete. 2024-05-09T06:38:39.547 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:39.547 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:39.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:39.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.726+0000 7f0c882a5640 1 Processor -- start 2024-05-09T06:38:39.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.726+0000 7f0c882a5640 1 -- start start 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c882a5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0c80071ce0 0x7f0c800cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c882a5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800d04a0 con 0x7f0c800b0080 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c882a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800722c0 con 0x7f0c8015ece0 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c882a5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0c800725a0 con 0x7f0c80071ce0 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c86041640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0c800b0080 0x7f0c8015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60864/0 (socket says 172.21.15.1:60864) 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c86041640 1 -- 172.21.15.1:0/915193776 learned_addr learned my addr 172.21.15.1:0/915193776 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c85840640 1 --2- 172.21.15.1:0/915193776 >> v2:172.21.15.6:3300/0 conn(0x7f0c80071ce0 0x7f0c800cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583218048 0 0) 0x7f0c800d04a0 con 0x7f0c800b0080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c6c003520 con 0x7f0c800b0080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.727+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3446596247 0 0) 0x7f0c6c003520 con 0x7f0c800b0080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 >> v2:172.21.15.6:3300/0 conn(0x7f0c80071ce0 msgr2=0x7f0c800cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 --2- 172.21.15.1:0/915193776 >> v2:172.21.15.6:3300/0 conn(0x7f0c80071ce0 0x7f0c800cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 >> v1:172.21.15.6:6789/0 conn(0x7f0c8015ece0 legacy=0x7f0c8015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c80072880 con 0x7f0c800b0080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f0c74001670 con 0x7f0c800b0080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 >> v1:172.21.15.1:6789/0 conn(0x7f0c800b0080 legacy=0x7f0c8015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 0x7f0c6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0043b0 0x7f0c6c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c6c005080 0x7f0c6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800725a0 con 0x7f0c6c0039b0 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c00bfe0 con 0x7f0c6c005080 2024-05-09T06:38:39.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6c00c1e0 con 0x7f0c6c0043b0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 0x7f0c6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c85840640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0043b0 0x7f0c6c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86842640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c6c005080 0x7f0c6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 -- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0043b0 msgr2=0x7f0c6c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0043b0 0x7f0c6c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 -- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c6c005080 msgr2=0x7f0c6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c6c005080 0x7f0c6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86842640 1 -- 172.21.15.1:0/915193776 reap_dead start 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.728+0000 7f0c86041640 1 -- 172.21.15.1:0/915193776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c003520 con 0x7f0c6c0039b0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c86041640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 0x7f0c6c003d90 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c290 tx=0x7f0c6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486f044e3f4c7971 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74007d50 con 0x7f0c6c0039b0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c74005af0 con 0x7f0c6c0039b0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c8503f640 1 -- 172.21.15.1:0/915193776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74007070 con 0x7f0c6c0039b0 2024-05-09T06:38:39.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c882a5640 1 -- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 msgr2=0x7f0c6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.729+0000 7f0c882a5640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 0x7f0c6c003d90 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c290 tx=0x7f0c6c005830 comp rx=0 tx=0).stop 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.730+0000 7f0c882a5640 1 -- 172.21.15.1:0/915193776 shutdown_connections 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.730+0000 7f0c882a5640 1 --2- 172.21.15.1:0/915193776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c0039b0 0x7f0c6c003d90 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.730+0000 7f0c882a5640 1 -- 172.21.15.1:0/915193776 >> 172.21.15.1:0/915193776 conn(0x7f0c800d0b50 msgr2=0x7f0c800d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.730+0000 7f0c882a5640 1 -- 172.21.15.1:0/915193776 shutdown_connections 2024-05-09T06:38:39.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.730+0000 7f0c882a5640 1 -- 172.21.15.1:0/915193776 wait complete. 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.731+0000 7f0c882a5640 1 Processor -- start 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.731+0000 7f0c882a5640 1 -- start start 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 0x7f0c800b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c8015ce90 0x7f0c800c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800cbb60 con 0x7f0c800b35b0 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8015e7c0 con 0x7f0c8015ce90 2024-05-09T06:38:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800cbce0 con 0x7f0c6c0039b0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41744/0 (socket says 172.21.15.1:41744) 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 -- 172.21.15.1:0/3746978990 learned_addr learned my addr 172.21.15.1:0/3746978990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c86041640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 0x7f0c800b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c86842640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c8015ce90 0x7f0c800c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 -- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 msgr2=0x7f0c800b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 0x7f0c800b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 -- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c8015ce90 msgr2=0x7f0c800c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c8015ce90 0x7f0c800c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 -- 172.21.15.1:0/3746978990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8015cc50 con 0x7f0c800b35b0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c86041640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 0x7f0c800b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c85840640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0119c0 tx=0x7f0c7c011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00faf0 con 0x7f0c800b35b0 2024-05-09T06:38:39.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00fc90 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c800b3b20 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c014440 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.732+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c800b40e0 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.733+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c800b6a80 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.734+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c7c012040 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.734+0000 7f0c72ffd640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 0x7f0c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.734+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0c7c08b460 con 0x7f0c800b35b0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.734+0000 7f0c86041640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 0x7f0c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.735+0000 7f0c86041640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 0x7f0c6406e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0c74033000 tx=0x7f0c7402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.739+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c800b6a80 con 0x7f0c800b35b0 2024-05-09T06:38:39.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.926+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c800752b0 con 0x7f0c800b35b0 2024-05-09T06:38:39.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.927+0000 7f0c72ffd640 1 -- 172.21.15.1:0/3746978990 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16820 (secure 0 0 0) 0x7f0c7c019030 con 0x7f0c800b35b0 2024-05-09T06:38:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:39.908 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":121,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:30.738835+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:39.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 msgr2=0x7f0c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 0x7f0c6406e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0c74033000 tx=0x7f0c7402e000 comp rx=0 tx=0).stop 2024-05-09T06:38:39.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 msgr2=0x7f0c8015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:39.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0119c0 tx=0x7f0c7c011e80 comp rx=0 tx=0).stop 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 shutdown_connections 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0c6406be30 0x7f0c6406e2f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c8015ce90 0x7f0c800c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800b35b0 0x7f0c8015c710 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 --2- 172.21.15.1:0/3746978990 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c6c0039b0 0x7f0c800b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 >> 172.21.15.1:0/3746978990 conn(0x7f0c800d0b50 msgr2=0x7f0c80161100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 shutdown_connections 2024-05-09T06:38:39.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:39.930+0000 7f0c882a5640 1 -- 172.21.15.1:0/3746978990 wait complete. 2024-05-09T06:38:39.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:39.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.048+0000 7f8e55d96640 1 Processor -- start 2024-05-09T06:38:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.048+0000 7f8e55d96640 1 -- start start 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e55d96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e5015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e55d96640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e50161fe0 con 0x7f8e500d45f0 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e55d96640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e501622c0 con 0x7f8e500d1c70 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e55d96640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5015f580 con 0x7f8e5015e190 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e4ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8e500d45f0 0x7f8e500b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60874/0 (socket says 172.21.15.1:60874) 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e4ffff640 1 -- 172.21.15.1:0/1095587366 learned_addr learned my addr 172.21.15.1:0/1095587366 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e4f7fe640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e5015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680671634 0 0) 0x7f8e50161fe0 con 0x7f8e500d45f0 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.049+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e38003520 con 0x7f8e500d45f0 2024-05-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2657179500 0 0) 0x7f8e38003520 con 0x7f8e500d45f0 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 msgr2=0x7f8e5015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e5015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 >> v1:172.21.15.6:6789/0 conn(0x7f8e500d1c70 legacy=0x7f8e500d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e501625a0 con 0x7f8e500d45f0 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8e480019f0 con 0x7f8e500d45f0 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 >> v1:172.21.15.1:6789/0 conn(0x7f8e500d45f0 legacy=0x7f8e500b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e38003f20 0x7f8e38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e38004d40 0x7f8e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3800b9f0 con 0x7f8e38004d40 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3800bba0 con 0x7f8e38003f20 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3800bda0 con 0x7f8e5015e190 2024-05-09T06:38:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e38004d40 0x7f8e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4f7fe640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4effd640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e38003f20 0x7f8e38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 -- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 msgr2=0x7f8e380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e5015e190 0x7f8e380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 -- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e38003f20 msgr2=0x7f8e38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e38003f20 0x7f8e38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.050+0000 7f8e4ffff640 1 -- 172.21.15.1:0/1095587366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e38003520 con 0x7f8e38004d40 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.051+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e38004d40 0x7f8e380051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d4a80 tx=0x7f8e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3947f276609756 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.051+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800a450 con 0x7f8e38004d40 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.051+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4802f070 con 0x7f8e38004d40 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.051+0000 7f8e4e7fc640 1 -- 172.21.15.1:0/1095587366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4802bc60 con 0x7f8e38004d40 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 -- 172.21.15.1:0/1095587366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e38004d40 msgr2=0x7f8e380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 --2- 172.21.15.1:0/1095587366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e38004d40 0x7f8e380051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d4a80 tx=0x7f8e38005830 comp rx=0 tx=0).stop 2024-05-09T06:38:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e4ffff640 1 -- 172.21.15.1:0/1095587366 reap_dead start 2024-05-09T06:38:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 -- 172.21.15.1:0/1095587366 shutdown_connections 2024-05-09T06:38:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 -- 172.21.15.1:0/1095587366 >> 172.21.15.1:0/1095587366 conn(0x7f8e50073340 msgr2=0x7f8e5009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 -- 172.21.15.1:0/1095587366 shutdown_connections 2024-05-09T06:38:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.052+0000 7f8e55d96640 1 -- 172.21.15.1:0/1095587366 wait complete. 2024-05-09T06:38:40.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.053+0000 7f8e55d96640 1 Processor -- start 2024-05-09T06:38:40.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.053+0000 7f8e55d96640 1 -- start start 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e38004d40 0x7f8e500ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500b9130 0x7f8e501514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e500b56d0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50161c60 con 0x7f8e500b9130 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e55d96640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8e500b5850 con 0x7f8e38004d40 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41772/0 (socket says 172.21.15.1:41772) 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 -- 172.21.15.1:0/2985003428 learned_addr learned my addr 172.21.15.1:0/2985003428 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4f7fe640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e38004d40 0x7f8e500ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4effd640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500b9130 0x7f8e501514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 -- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e38004d40 msgr2=0x7f8e500ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e38004d40 0x7f8e500ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 -- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500b9130 msgr2=0x7f8e501514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500b9130 0x7f8e501514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.054+0000 7f8e4ffff640 1 -- 172.21.15.1:0/2985003428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e50151a30 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e4ffff640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802b470 tx=0x7f8e500d4dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48044070 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48001a60 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e500b98f0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800a450 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.055+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e500c96b0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.056+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e500752b0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.056+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e48047070 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.057+0000 7f8e37fff640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 0x7f8e3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.057+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f8e480ab050 con 0x7f8e500b6cc0 2024-05-09T06:38:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.057+0000 7f8e4f7fe640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 0x7f8e3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.057+0000 7f8e4f7fe640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 0x7f8e3c06e320 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d3300 tx=0x7f8e40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.061+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e480797d0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.243+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e5008e1c0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.243+0000 7f8e37fff640 1 -- 172.21.15.1:0/2985003428 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f8e480790d0 con 0x7f8e500b6cc0 2024-05-09T06:38:40.224 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 msgr2=0x7f8e3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 0x7f8e3c06e320 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d3300 tx=0x7f8e40008040 comp rx=0 tx=0).stop 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 msgr2=0x7f8e5015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802b470 tx=0x7f8e500d4dc0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 shutdown_connections 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8e3c06be60 0x7f8e3c06e320 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e500b6cc0 0x7f8e5015e570 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e500b9130 0x7f8e501514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 --2- 172.21.15.1:0/2985003428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e38004d40 0x7f8e500ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 >> 172.21.15.1:0/2985003428 conn(0x7f8e50073340 msgr2=0x7f8e5015f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 shutdown_connections 2024-05-09T06:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.245+0000 7f8e55d96640 1 -- 172.21.15.1:0/2985003428 wait complete. 2024-05-09T06:38:40.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:38:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.363+0000 7efe34a15640 1 Processor -- start 2024-05-09T06:38:40.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.363+0000 7efe34a15640 1 -- start start 2024-05-09T06:38:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.364+0000 7efe34a15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.364+0000 7efe34a15640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.364+0000 7efe34a15640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3015e900 con 0x7efe300cf990 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.364+0000 7efe34a15640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efe3015ebe0 con 0x7efe300b1900 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.364+0000 7efe34a15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe3015eec0 con 0x7efe3015e390 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:58422/0 (socket says 172.21.15.1:58422) 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:33646/0 (socket says 172.21.15.1:33646) 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2ed9d640 1 -- 172.21.15.1:0/17780991 learned_addr learned my addr 172.21.15.1:0/17780991 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 -- 172.21.15.1:0/17780991 >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 msgr2=0x7efe300cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 --2- 172.21.15.1:0/17780991 >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 -- 172.21.15.1:0/17780991 >> v1:172.21.15.6:6789/0 conn(0x7efe300cf990 legacy=0x7efe300b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2e59c640 1 -- 172.21.15.1:0/17780991 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3015f1a0 con 0x7efe3015e390 2024-05-09T06:38:40.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.365+0000 7efe2ed9d640 1 --2- 172.21.15.1:0/17780991 >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.366+0000 7efe2e59c640 1 --2- 172.21.15.1:0/17780991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe200097a0 tx=0x7efe2002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0e5229ee31fee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.366+0000 7efe2d59a640 1 -- 172.21.15.1:0/17780991 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20037070 con 0x7efe3015e390 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.366+0000 7efe2d59a640 1 -- 172.21.15.1:0/17780991 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2002ae50 con 0x7efe3015e390 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.366+0000 7efe2d59a640 1 -- 172.21.15.1:0/17780991 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20032660 con 0x7efe3015e390 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 -- 172.21.15.1:0/17780991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 msgr2=0x7efe300cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 --2- 172.21.15.1:0/17780991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe200097a0 tx=0x7efe2002a100 comp rx=0 tx=0).stop 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 -- 172.21.15.1:0/17780991 shutdown_connections 2024-05-09T06:38:40.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 --2- 172.21.15.1:0/17780991 >> v2:172.21.15.6:3300/0 conn(0x7efe300b1900 0x7efe300cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 --2- 172.21.15.1:0/17780991 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300cf450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 -- 172.21.15.1:0/17780991 >> 172.21.15.1:0/17780991 conn(0x7efe300b1da0 msgr2=0x7efe300b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 -- 172.21.15.1:0/17780991 shutdown_connections 2024-05-09T06:38:40.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.367+0000 7efe34a15640 1 -- 172.21.15.1:0/17780991 wait complete. 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.368+0000 7efe34a15640 1 Processor -- start 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.368+0000 7efe34a15640 1 -- start start 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe300b1900 0x7efe300b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe300cffb0 con 0x7efe300cf990 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efe30162ba0 con 0x7efe300b1900 2024-05-09T06:38:40.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe34a15640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe300cce20 con 0x7efe3015e390 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe2dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe2dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41780/0 (socket says 172.21.15.1:41780) 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.369+0000 7efe2dd9b640 1 -- 172.21.15.1:0/1455772002 learned_addr learned my addr 172.21.15.1:0/1455772002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2e59c640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe300b1900 0x7efe300b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2ed9d640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 -- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 msgr2=0x7efe300c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 -- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe300b1900 msgr2=0x7efe300b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe300b1900 0x7efe300b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 -- 172.21.15.1:0/1455772002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe300b49b0 con 0x7efe300cf990 2024-05-09T06:38:40.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe2dd9b640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c400 tx=0x7efe2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240052d0 con 0x7efe300cf990 2024-05-09T06:38:40.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24005490 con 0x7efe300cf990 2024-05-09T06:38:40.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe300b4c90 con 0x7efe300cf990 2024-05-09T06:38:40.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24007260 con 0x7efe300cf990 2024-05-09T06:38:40.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.370+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe300b5250 con 0x7efe300cf990 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.371+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300752b0 con 0x7efe300cf990 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.372+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe24007400 con 0x7efe300cf990 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.372+0000 7efe1effd640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 0x7efe1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.372+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7efe2408a4f0 con 0x7efe300cf990 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.372+0000 7efe2e59c640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 0x7efe1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.373+0000 7efe2e59c640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 0x7efe1006e320 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe20034000 tx=0x7efe2002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.377+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe24058da0 con 0x7efe300cf990 2024-05-09T06:38:40.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.559+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe300b2ae0 con 0x7efe300cf990 2024-05-09T06:38:40.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.559+0000 7efe1effd640 1 -- 172.21.15.1:0/1455772002 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7efe240586a0 con 0x7efe300cf990 2024-05-09T06:38:40.540 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 msgr2=0x7efe1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 0x7efe1006e320 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe20034000 tx=0x7efe2002e000 comp rx=0 tx=0).stop 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 msgr2=0x7efe300b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c400 tx=0x7efe2400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 shutdown_connections 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe1006be60 0x7efe1006e320 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe3015e390 0x7efe300c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe300cf990 0x7efe300b36b0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 --2- 172.21.15.1:0/1455772002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe300b1900 0x7efe300b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 >> 172.21.15.1:0/1455772002 conn(0x7efe300b1da0 msgr2=0x7efe300cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 shutdown_connections 2024-05-09T06:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.561+0000 7efe34a15640 1 -- 172.21.15.1:0/1455772002 wait complete. 2024-05-09T06:38:40.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:38:40.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.679+0000 7f492c73b640 1 Processor -- start 2024-05-09T06:38:40.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.679+0000 7f492c73b640 1 -- start start 2024-05-09T06:38:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.680+0000 7f492c73b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f49240d1c70 0x7f492415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.680+0000 7f492c73b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4924161fe0 con 0x7f49240d45f0 2024-05-09T06:38:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.680+0000 7f492c73b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49241622c0 con 0x7f492415e190 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.680+0000 7f492c73b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f492415f580 con 0x7f49240d1c70 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f4929cd6640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f49240d1c70 0x7f492415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f492acd8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f49240d45f0 0x7f49240b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60884/0 (socket says 172.21.15.1:60884) 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f492acd8640 1 -- 172.21.15.1:0/3901757656 learned_addr learned my addr 172.21.15.1:0/3901757656 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f4929cd6640 1 -- 172.21.15.1:0/3901757656 >> v1:172.21.15.6:6789/0 conn(0x7f492415e190 legacy=0x7f49240d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f4929cd6640 1 -- 172.21.15.1:0/3901757656 >> v1:172.21.15.1:6789/0 conn(0x7f49240d45f0 legacy=0x7f49240b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f49294d5640 1 -- 172.21.15.1:0/3901757656 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609773120 0 0) 0x7f4924161fe0 con 0x7f49240d45f0 2024-05-09T06:38:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.681+0000 7f4929cd6640 1 -- 172.21.15.1:0/3901757656 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241625a0 con 0x7f49240d1c70 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f4929cd6640 1 --2- 172.21.15.1:0/3901757656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49240d1c70 0x7f492415f780 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f49180097a0 tx=0x7f491802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fdcdc93baf04bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f49294d5640 1 -- 172.21.15.1:0/3901757656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918037070 con 0x7f49240d1c70 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f49294d5640 1 -- 172.21.15.1:0/3901757656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918032440 con 0x7f49240d1c70 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f49294d5640 1 -- 172.21.15.1:0/3901757656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918032810 con 0x7f49240d1c70 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 -- 172.21.15.1:0/3901757656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49240d1c70 msgr2=0x7f492415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 --2- 172.21.15.1:0/3901757656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49240d1c70 0x7f492415f780 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f49180097a0 tx=0x7f491802a230 comp rx=0 tx=0).stop 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 -- 172.21.15.1:0/3901757656 shutdown_connections 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 --2- 172.21.15.1:0/3901757656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49240d1c70 0x7f492415f780 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 -- 172.21.15.1:0/3901757656 >> 172.21.15.1:0/3901757656 conn(0x7f4924073340 msgr2=0x7f492409d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 -- 172.21.15.1:0/3901757656 shutdown_connections 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.682+0000 7f492c73b640 1 -- 172.21.15.1:0/3901757656 wait complete. 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 Processor -- start 2024-05-09T06:38:40.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 -- start start 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49240d1c70 0x7f4924159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49240d45f0 0x7f49240c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49240b56d0 con 0x7f492415e190 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4924161c60 con 0x7f49240d1c70 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492c73b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f49240b5850 con 0x7f49240d45f0 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.684+0000 7f492acd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41804/0 (socket says 172.21.15.1:41804) 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492a4d7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49240d1c70 0x7f4924159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 -- 172.21.15.1:0/3952656626 learned_addr learned my addr 172.21.15.1:0/3952656626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f4929cd6640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49240d45f0 0x7f49240c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 -- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49240d45f0 msgr2=0x7f49240c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49240d45f0 0x7f49240c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 -- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49240d1c70 msgr2=0x7f4924159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49240d1c70 0x7f4924159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 -- 172.21.15.1:0/3952656626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49240cc630 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492acd8640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f491400c7b0 tx=0x7f491400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914018070 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491400a640 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491400aa70 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49240cc910 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.685+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49240cced0 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.686+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49240752b0 con 0x7f492415e190 2024-05-09T06:38:40.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.687+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f491400ac10 con 0x7f492415e190 2024-05-09T06:38:40.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.687+0000 7f4922ffd640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 0x7f491006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.687+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f491408a540 con 0x7f492415e190 2024-05-09T06:38:40.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.687+0000 7f492a4d7640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 0x7f491006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.688+0000 7f492a4d7640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 0x7f491006e320 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f491c000f60 tx=0x7f491c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.692+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4914058690 con 0x7f492415e190 2024-05-09T06:38:40.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.877+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f492408e1c0 con 0x7f492415e190 2024-05-09T06:38:40.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.878+0000 7f4922ffd640 1 -- 172.21.15.1:0/3952656626 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f491405c990 con 0x7f492415e190 2024-05-09T06:38:40.859 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:38:40.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 msgr2=0x7f491006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 0x7f491006e320 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f491c000f60 tx=0x7f491c009040 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 msgr2=0x7f49240cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f491400c7b0 tx=0x7f491400cc70 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 shutdown_connections 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f491006be60 0x7f491006e320 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f492415e190 0x7f49240cd6d0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f49240d45f0 0x7f49240c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 --2- 172.21.15.1:0/3952656626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49240d1c70 0x7f4924159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 >> 172.21.15.1:0/3952656626 conn(0x7f4924073340 msgr2=0x7f49240d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 shutdown_connections 2024-05-09T06:38:40.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.880+0000 7f492c73b640 1 -- 172.21.15.1:0/3952656626 wait complete. 2024-05-09T06:38:40.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:38:40.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 Processor -- start 2024-05-09T06:38:40.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 -- start start 2024-05-09T06:38:40.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbeb41621e0 0x7fbeb40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb406ca70 con 0x7fbeb415bc20 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb4162650 con 0x7fbeb41621e0 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.996+0000 7fbebae88640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb40cc210 con 0x7fbeb406c500 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbeb41621e0 0x7fbeb40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb8c24640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fbeb415bc20 0x7fbeb40ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60898/0 (socket says 172.21.15.1:60898) 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb8c24640 1 -- 172.21.15.1:0/1031590005 learned_addr learned my addr 172.21.15.1:0/1031590005 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbeb41621e0 0x7fbeb40cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41812/0 (socket says 172.21.15.1:41812) 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb3fff640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 -- 172.21.15.1:0/1031590005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 msgr2=0x7fbeb4161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 -- 172.21.15.1:0/1031590005 >> v1:172.21.15.1:6789/0 conn(0x7fbeb415bc20 legacy=0x7fbeb40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb9425640 1 -- 172.21.15.1:0/1031590005 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb406cd50 con 0x7fbeb41621e0 2024-05-09T06:38:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.997+0000 7fbeb37fe640 1 -- 172.21.15.1:0/1031590005 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336163927 0 0) 0x7fbeb406ca70 con 0x7fbeb415bc20 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbeb9425640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40cd3b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c40fcdbcc3b913 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbeb37fe640 1 -- 172.21.15.1:0/1031590005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac037070 con 0x7fbeb41621e0 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbeb37fe640 1 -- 172.21.15.1:0/1031590005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac032440 con 0x7fbeb41621e0 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbeb37fe640 1 -- 172.21.15.1:0/1031590005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac032810 con 0x7fbeb41621e0 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 -- 172.21.15.1:0/1031590005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 msgr2=0x7fbeb40cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40cd3b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac02a180 comp rx=0 tx=0).stop 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 -- 172.21.15.1:0/1031590005 shutdown_connections 2024-05-09T06:38:40.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 --2- 172.21.15.1:0/1031590005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40cd3b0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 -- 172.21.15.1:0/1031590005 >> 172.21.15.1:0/1031590005 conn(0x7fbeb40d1bc0 msgr2=0x7fbeb40d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 -- 172.21.15.1:0/1031590005 shutdown_connections 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:40.998+0000 7fbebae88640 1 -- 172.21.15.1:0/1031590005 wait complete. 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.000+0000 7fbebae88640 1 Processor -- start 2024-05-09T06:38:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.000+0000 7fbebae88640 1 -- start start 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 0x7fbeb40b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb406d5f0 con 0x7fbeb41621e0 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb406d310 con 0x7fbeb415bc20 2024-05-09T06:38:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb415c000 con 0x7fbeb406c500 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41820/0 (socket says 172.21.15.1:41820) 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 -- 172.21.15.1:0/3270030112 learned_addr learned my addr 172.21.15.1:0/3270030112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb8c24640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb3fff640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 0x7fbeb40b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 -- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 msgr2=0x7fbeb4150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 -- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 msgr2=0x7fbeb40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 0x7fbeb40b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 -- 172.21.15.1:0/3270030112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb40cc950 con 0x7fbeb41621e0 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb3fff640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 0x7fbeb40b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb8c24640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb9425640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac037050 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac038040 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb40ccbf0 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004450 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.001+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb40cd1b0 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.003+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb40752b0 con 0x7fbeb41621e0 2024-05-09T06:38:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.003+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeac03c6d0 con 0x7fbeb41621e0 2024-05-09T06:38:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.003+0000 7fbeb17fa640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 0x7fbe9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.003+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fbeac0ab9a0 con 0x7fbeb41621e0 2024-05-09T06:38:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.003+0000 7fbeb8c24640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 0x7fbe9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.004+0000 7fbeb8c24640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 0x7fbe9406e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbea4000f60 tx=0x7fbea4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.008+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbeac07a250 con 0x7fbeb41621e0 2024-05-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.204+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbeb406dc60 con 0x7fbeb41621e0 2024-05-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.205+0000 7fbeb17fa640 1 -- 172.21.15.1:0/3270030112 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7fbeac079b50 con 0x7fbeb41621e0 2024-05-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 msgr2=0x7fbe9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 0x7fbe9406e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbea4000f60 tx=0x7fbea4009040 comp rx=0 tx=0).stop 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 msgr2=0x7fbeb40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac0046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 shutdown_connections 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fbe9406be30 0x7fbe9406e2f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb41621e0 0x7fbeb40b4f50 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbeb415bc20 0x7fbeb40b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 --2- 172.21.15.1:0/3270030112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbeb406c500 0x7fbeb4150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 >> 172.21.15.1:0/3270030112 conn(0x7fbeb40d1bc0 msgr2=0x7fbeb40d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 shutdown_connections 2024-05-09T06:38:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:41.207+0000 7fbebae88640 1 -- 172.21.15.1:0/3270030112 wait complete. 2024-05-09T06:38:41.198 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:38:41.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:41.199 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:41.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[80124]: fuse finished with error 0 and tester_r 0 2024-05-09T06:38:41.225 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:47.326 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:38:47.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:47.326 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:38:47.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-09T06:38:47.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.523+0000 7f6398e07640 1 Processor -- start 2024-05-09T06:38:47.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.523+0000 7f6398e07640 1 -- start start 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f6398e07640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f63940c41e0 0x7f63940ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f6398e07640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6394072050 con 0x7f63940d06f0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f6398e07640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6394072330 con 0x7f639406b1e0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f6398e07640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6394072610 con 0x7f63940c41e0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f639259c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f63940d06f0 0x7f63940cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60902/0 (socket says 172.21.15.1:60902) 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.524+0000 7f639259c640 1 -- 172.21.15.1:0/1658549478 learned_addr learned my addr 172.21.15.1:0/1658549478 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378681306 0 0) 0x7f6394072050 con 0x7f63940d06f0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> v2:172.21.15.6:3300/0 conn(0x7f63940c41e0 0x7f63940ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6380003520 con 0x7f63940d06f0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 998293706 0 0) 0x7f6380003520 con 0x7f63940d06f0 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 >> v2:172.21.15.6:3300/0 conn(0x7f63940c41e0 msgr2=0x7f63940ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 --2- 172.21.15.1:0/1658549478 >> v2:172.21.15.6:3300/0 conn(0x7f63940c41e0 0x7f63940ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 >> v1:172.21.15.6:6789/0 conn(0x7f639406b1e0 legacy=0x7f63940cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63940728f0 con 0x7f63940d06f0 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f6384001670 con 0x7f63940d06f0 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 >> v1:172.21.15.1:6789/0 conn(0x7f63940d06f0 legacy=0x7f63940cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63800039b0 0x7f6380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f6380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63800051c0 0x7f638000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6394072610 con 0x7f6380004800 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f638000c110 con 0x7f63800039b0 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f638000c310 con 0x7f63800051c0 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> v2:172.21.15.6:3300/0 conn(0x7f63940c41e0 0x7f63940ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f6380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6392d9d640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63800051c0 0x7f638000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f639259c640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63800039b0 0x7f6380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 -- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63800051c0 msgr2=0x7f638000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63800051c0 0x7f638000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 -- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63800039b0 msgr2=0x7f6380003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63800039b0 0x7f6380003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.525+0000 7f6391d9b640 1 -- 172.21.15.1:0/1658549478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6380003520 con 0x7f6380004800 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.526+0000 7f6392d9d640 1 -- 172.21.15.1:0/1658549478 reap_dead start 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.526+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f6380004c80 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098a0 tx=0x7f6380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da04d7b57fe614e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.526+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f6380004800 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.526+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c02aca0 con 0x7f6380004800 2024-05-09T06:38:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.526+0000 7f639159a640 1 -- 172.21.15.1:0/1658549478 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c00f040 con 0x7f6380004800 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 -- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 msgr2=0x7f6380004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f6380004c80 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098a0 tx=0x7f6380005830 comp rx=0 tx=0).stop 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 -- 172.21.15.1:0/1658549478 shutdown_connections 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 --2- 172.21.15.1:0/1658549478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f6380004c80 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 -- 172.21.15.1:0/1658549478 >> 172.21.15.1:0/1658549478 conn(0x7f63940d0b50 msgr2=0x7f63940d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 -- 172.21.15.1:0/1658549478 shutdown_connections 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.527+0000 7f6398e07640 1 -- 172.21.15.1:0/1658549478 wait complete. 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 Processor -- start 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 -- start start 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 0x7f639406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 0x7f63940c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63940cc860 con 0x7f6380004800 2024-05-09T06:38:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f63940cc6e0 con 0x7f63941f3510 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f6398e07640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f63940cc9e0 con 0x7f639406f200 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f639259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.529+0000 7f639259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41846/0 (socket says 172.21.15.1:41846) 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 -- 172.21.15.1:0/1948097576 learned_addr learned my addr 172.21.15.1:0/1948097576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 0x7f639406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6392d9d640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 0x7f63940c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 -- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 msgr2=0x7f63940c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 0x7f63940c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 -- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 msgr2=0x7f639406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 0x7f639406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 -- 172.21.15.1:0/1948097576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63941f3990 con 0x7f6380004800 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 0x7f639406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6392d9d640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 0x7f63940c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f639259c640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f63840072f0 tx=0x7f63840077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f638affd640 1 -- 172.21.15.1:0/1948097576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638401a070 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f638affd640 1 -- 172.21.15.1:0/1948097576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638400f860 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63940c47b0 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f638affd640 1 -- 172.21.15.1:0/1948097576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638400fc30 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.530+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63940c4d70 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.532+0000 7f638affd640 1 -- 172.21.15.1:0/1948097576 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f638401e3f0 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.532+0000 7f638affd640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.532+0000 7f638affd640 1 -- 172.21.15.1:0/1948097576 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f638408a6c0 con 0x7f6380004800 2024-05-09T06:38:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.532+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.533+0000 7f6391d9b640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.533+0000 7f639bbf2740 1 Processor -- start 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 -- start start 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5aff66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5afed0d00 con 0x55f5aff97030 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f5afe784c0 con 0x55f5affb4650 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f5afecccc0 con 0x55f5aff97770 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f6392d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f6392d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41854/0 (socket says 172.21.15.1:41854) 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f6392d9d640 1 -- 172.21.15.1:0/2719815751 learned_addr learned my addr 172.21.15.1:0/2719815751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.534+0000 7f639259c640 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5aff66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 -- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 msgr2=0x55f5aff96af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff96af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 -- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 msgr2=0x55f5aff66fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5aff66fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 -- 172.21.15.1:0/2719815751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5afcae7d0 con 0x55f5aff97030 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6392d9d640 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f638c003eb0 tx=0x7f638c001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2cdd74444cbbe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6373fff640 1 -- 172.21.15.1:0/2719815751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c005420 con 0x55f5aff97030 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6373fff640 1 -- 172.21.15.1:0/2719815751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c01a070 con 0x55f5aff97030 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.535+0000 7f6373fff640 1 -- 172.21.15.1:0/2719815751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0057c0 con 0x55f5aff97030 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- 172.21.15.1:0/2719815751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 msgr2=0x55f5aff6be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f638c003eb0 tx=0x7f638c001610 comp rx=0 tx=0).stop 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- 172.21.15.1:0/2719815751 shutdown_connections 2024-05-09T06:38:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97030 0x55f5aff6be40 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff96af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2719815751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5aff66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- 172.21.15.1:0/2719815751 >> 172.21.15.1:0/2719815751 conn(0x55f5aff69250 msgr2=0x55f5affb1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- 172.21.15.1:0/2719815751 shutdown_connections 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- 172.21.15.1:0/2719815751 wait complete. 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 -- start start 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.536+0000 7f639bbf2740 1 Processor -- start 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affb3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5aff6b400 con 0x55f5aff999f0 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55f5afe8bb90 con 0x55f5affb4650 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f5aff6a7f0 con 0x55f5aff97770 2024-05-09T06:38:47.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f639bbf2740 10 client.0 _ll_register_callbacks cb 0x55f5afec7da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.537+0000 7f6392d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41862/0 (socket says 172.21.15.1:41862) 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 -- 172.21.15.1:0/431652090 learned_addr learned my addr 172.21.15.1:0/431652090 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f639259c640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6391d9b640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affb3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 -- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 msgr2=0x55f5aff6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 -- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 msgr2=0x55f5affb3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affb3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 -- 172.21.15.1:0/431652090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5afbe43f0 con 0x55f5aff999f0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6391d9b640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affb3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f6392d9d640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f638c016000 tx=0x7f638c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70932213d52bd30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f63717fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0159c0 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f638c029c60 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638c01a070 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f638c029e60 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f639bbf2740 10 client.9861 fetch_fsmap learned FSMap version 99 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.538+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55f5afba40f0 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c01a070 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f638c015b60 con 0x55f5aff999f0 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 10 client.9861 fetch_fsmap finished waiting for FSMap version 99 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 10 client.9861 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 20 client.9861 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affb1f70 0x55f5affb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 4 client.9861 mds_command: new command op to 9636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:38:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 --> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5afede430 con 0x55f5affb1f70 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639259c640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affb1f70 0x55f5affb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f63717fa640 10 client.9861 ms_handle_connect on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.539+0000 7f639259c640 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affb1f70 0x55f5affb2380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dec4298ec9d26 server_cookie=3f752a0b4c6499d in_seq=0 out_seq=0 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f63717fa640 1 -- 172.21.15.1:0/431652090 <== mds.0 v2:172.21.15.1:6836/1060593353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5afede430 con 0x55f5affb1f70 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f63717fa640 10 client.9861 handle_command_reply: tid=0 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 client.9861 shutdown 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 msgr2=0x55f5aff99e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f638c016000 tx=0x7f638c001510 comp rx=0 tx=0).stop 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 shutdown_connections 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affb1f70 0x55f5affb2380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99e70 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affb3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 --2- 172.21.15.1:0/431652090 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f63717fa640 0 client.9861 ms_handle_reset on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 >> 172.21.15.1:0/431652090 conn(0x55f5aff97030 msgr2=0x55f5aff67d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 shutdown_connections 2024-05-09T06:38:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 1 -- 172.21.15.1:0/431652090 wait complete. 2024-05-09T06:38:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.540+0000 7f639bbf2740 20 client.9861 trim_cache size 0 max 16384 2024-05-09T06:38:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.555+0000 7f639bbf2740 1 Processor -- start 2024-05-09T06:38:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.555+0000 7f639bbf2740 1 -- start start 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff9ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affa8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6394201160 con 0x55f5aff999f0 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6394200140 con 0x55f5affb4650 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f63941ff490 con 0x55f5aff97770 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41878/0 (socket says 172.21.15.1:41878) 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 -- 172.21.15.1:0/2114318470 learned_addr learned my addr 172.21.15.1:0/2114318470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639259c640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff9ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6392d9d640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affa8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 -- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 msgr2=0x55f5aff9ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff9ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 -- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 msgr2=0x55f5affa8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affa8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6391d9b640 1 -- 172.21.15.1:0/2114318470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5aff67810 con 0x55f5aff999f0 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f639259c640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff9ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.556+0000 7f6392d9d640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affa8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f637c006af0 tx=0x7f637c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbafa199e079f659 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f63737fe640 1 -- 172.21.15.1:0/2114318470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037040 con 0x55f5aff999f0 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f63737fe640 1 -- 172.21.15.1:0/2114318470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c050070 con 0x55f5aff999f0 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f63737fe640 1 -- 172.21.15.1:0/2114318470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c03dca0 con 0x55f5aff999f0 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 -- 172.21.15.1:0/2114318470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 msgr2=0x55f5aff99dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f637c006af0 tx=0x7f637c02c000 comp rx=0 tx=0).stop 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 -- 172.21.15.1:0/2114318470 shutdown_connections 2024-05-09T06:38:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affa8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff999f0 0x55f5aff99dd0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2114318470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff97770 0x55f5aff9ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 -- 172.21.15.1:0/2114318470 >> 172.21.15.1:0/2114318470 conn(0x55f5aff97030 msgr2=0x55f5aff6c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 -- 172.21.15.1:0/2114318470 shutdown_connections 2024-05-09T06:38:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.557+0000 7f639bbf2740 1 -- 172.21.15.1:0/2114318470 wait complete. 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.558+0000 7f639bbf2740 1 -- start start 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.558+0000 7f639bbf2740 1 Processor -- start 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff999f0 0x55f5b002dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affd48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5afe50630 con 0x55f5aff97770 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f63941ffae0 con 0x55f5affb4650 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55f5aff853c0 con 0x55f5aff999f0 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639bbf2740 10 client.0 _ll_register_callbacks cb 0x7f638c003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41888/0 (socket says 172.21.15.1:41888) 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 -- 172.21.15.1:0/2606245786 learned_addr learned my addr 172.21.15.1:0/2606245786 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 -- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff999f0 msgr2=0x55f5b002dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff999f0 0x55f5b002dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 -- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 msgr2=0x55f5affd48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affd48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 -- 172.21.15.1:0/2606245786 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5aff87c10 con 0x55f5aff97770 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f639259c640 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f6384060e90 tx=0x7f6384009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698323499af88438 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.559+0000 7f63717fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384056ae0 con 0x55f5aff97770 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f6384054e50 con 0x55f5aff97770 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637c050070 con 0x55f5aff97770 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f6384077b80 con 0x55f5aff97770 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 10 client.9867 fetch_fsmap learned FSMap version 99 2024-05-09T06:38:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55f5affd5210 con 0x55f5aff97770 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c050070 con 0x55f5aff97770 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f6384057c30 con 0x55f5aff97770 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 10 client.9867 fetch_fsmap finished waiting for FSMap version 99 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 10 client.9867 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 20 client.9867 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affc1790 0x55f5affb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 4 client.9867 mds_command: new command op to 9636 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 --> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5afed8780 con 0x55f5affc1790 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affc1790 0x55f5affb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f6391d9b640 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affc1790 0x55f5affb2bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ecac0d0a76619 server_cookie=fc3f655d5b35fa68 in_seq=0 out_seq=0 2024-05-09T06:38:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.560+0000 7f63717fa640 10 client.9867 ms_handle_connect on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f63717fa640 1 -- 172.21.15.1:0/2606245786 <== mds.0 v2:172.21.15.1:6836/1060593353 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5afed8780 con 0x55f5affc1790 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f63717fa640 10 client.9867 handle_command_reply: tid=0 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 client.9867 shutdown 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 msgr2=0x55f5affb5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f6384060e90 tx=0x7f6384009040 comp rx=0 tx=0).stop 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 shutdown_connections 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:6836/1060593353,v1:172.21.15.1:6837/1060593353] conn(0x55f5affc1790 0x55f5affb2bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f5affb4650 0x55f5affd48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55f5aff999f0 0x55f5b002dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 --2- 172.21.15.1:0/2606245786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5aff97770 0x55f5affb5470 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f63717fa640 0 client.9867 ms_handle_reset on v2:172.21.15.1:6836/1060593353 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 >> 172.21.15.1:0/2606245786 conn(0x55f5aff97030 msgr2=0x55f5aff98e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 shutdown_connections 2024-05-09T06:38:47.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 1 -- 172.21.15.1:0/2606245786 wait complete. 2024-05-09T06:38:47.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.574+0000 7f639bbf2740 20 client.9867 trim_cache size 0 max 16384 2024-05-09T06:38:47.556 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "message": "", 2024-05-09T06:38:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0 2024-05-09T06:38:47.556 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 msgr2=0x7f637406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 msgr2=0x7f63941f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f63840072f0 tx=0x7f63840077b0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 shutdown_connections 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f639406f200 0x7f63940c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63941f3510 0x7f639406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 --2- 172.21.15.1:0/1948097576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380004800 0x7f63941f4cf0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 >> 172.21.15.1:0/1948097576 conn(0x7f63940d0b50 msgr2=0x7f639415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 shutdown_connections 2024-05-09T06:38:47.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.575+0000 7f6398e07640 1 -- 172.21.15.1:0/1948097576 wait complete. 2024-05-09T06:38:47.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:38:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.689+0000 7f1cd7653640 1 Processor -- start 2024-05-09T06:38:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.689+0000 7f1cd7653640 1 -- start start 2024-05-09T06:38:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd7653640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd7653640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1cd00b3300 0x7f1cd00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd7653640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd00ce0e0 con 0x7f1cd00cf180 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd7653640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd00ce3c0 con 0x7f1cd00b3300 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd7653640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd00ce6a0 con 0x7f1cd006e6d0 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd4bee640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1cd00cf180 0x7f1cd00cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51310/0 (socket says 172.21.15.1:51310) 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd4bee640 1 -- 172.21.15.1:0/3542183786 learned_addr learned my addr 172.21.15.1:0/3542183786 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/3542183786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.690+0000 7f1cd5bf0640 1 --2- 172.21.15.1:0/3542183786 >> v2:172.21.15.6:3300/0 conn(0x7f1cd00b3300 0x7f1cd00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716236943 0 0) 0x7f1cd00ce0e0 con 0x7f1cd00cf180 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cb8003520 con 0x7f1cd00cf180 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1cd53ef640 1 -- 172.21.15.1:0/3542183786 >> v2:172.21.15.6:3300/0 conn(0x7f1cd00b3300 msgr2=0x7f1cd00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/3542183786 >> v2:172.21.15.6:3300/0 conn(0x7f1cd00b3300 0x7f1cd00b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1cd53ef640 1 -- 172.21.15.1:0/3542183786 >> v1:172.21.15.1:6789/0 conn(0x7f1cd00cf180 legacy=0x7f1cd00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1077308201 0 0) 0x7f1cb8003520 con 0x7f1cd00cf180 2024-05-09T06:38:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.691+0000 7f1cd53ef640 1 -- 172.21.15.1:0/3542183786 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00ce980 con 0x7f1cd006e6d0 2024-05-09T06:38:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/3542183786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd006eab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009d90 tx=0x7f1cc002c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff209f32f0e8f593 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4001670 con 0x7f1cd006e6d0 2024-05-09T06:38:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc0007540 con 0x7f1cd006e6d0 2024-05-09T06:38:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1ccffff640 1 -- 172.21.15.1:0/3542183786 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00078f0 con 0x7f1cd006e6d0 2024-05-09T06:38:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 -- 172.21.15.1:0/3542183786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 msgr2=0x7f1cd006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 --2- 172.21.15.1:0/3542183786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd006eab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009d90 tx=0x7f1cc002c800 comp rx=0 tx=0).stop 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 -- 172.21.15.1:0/3542183786 shutdown_connections 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 --2- 172.21.15.1:0/3542183786 >> v2:172.21.15.6:3300/0 conn(0x7f1cd00b3300 0x7f1cd00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 --2- 172.21.15.1:0/3542183786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd006eab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.692+0000 7f1cd7653640 1 -- 172.21.15.1:0/3542183786 >> 172.21.15.1:0/3542183786 conn(0x7f1cd00cc4b0 msgr2=0x7f1cd00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.693+0000 7f1cd7653640 1 -- 172.21.15.1:0/3542183786 shutdown_connections 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.693+0000 7f1cd7653640 1 -- 172.21.15.1:0/3542183786 wait complete. 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.694+0000 7f1cd7653640 1 Processor -- start 2024-05-09T06:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.694+0000 7f1cd7653640 1 -- start start 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd00b3300 0x7f1cd01416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0070ed0 con 0x7f1cd00cf180 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd006eff0 con 0x7f1cd00b3300 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd01447c0 con 0x7f1cd006e6d0 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60196/0 (socket says 172.21.15.1:60196) 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 -- 172.21.15.1:0/1872887793 learned_addr learned my addr 172.21.15.1:0/1872887793 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd4bee640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd00b3300 0x7f1cd01416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 -- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 msgr2=0x7f1cd00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd00b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 -- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd00b3300 msgr2=0x7f1cd01416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd00b3300 0x7f1cd01416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 -- 172.21.15.1:0/1872887793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd015de10 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd5bf0640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e930 tx=0x7f1cc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8017070 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc800c980 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd014f9c0 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.695+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800cd70 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.696+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd00b0d30 con 0x7f1cd00cf180 2024-05-09T06:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.696+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00752b0 con 0x7f1cd00cf180 2024-05-09T06:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.697+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cc801a070 con 0x7f1cd00cf180 2024-05-09T06:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.697+0000 7f1ccdffb640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 0x7f1cbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.698+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f1cc808b330 con 0x7f1cd00cf180 2024-05-09T06:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.698+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 0x7f1cbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.698+0000 7f1cd53ef640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 0x7f1cbc06e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0036000 tx=0x7f1cc000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.702+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1cc8059c10 con 0x7f1cd00cf180 2024-05-09T06:38:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.925+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1cd006cf50 con 0x7f1cd00cf180 2024-05-09T06:38:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.925+0000 7f1ccdffb640 1 -- 172.21.15.1:0/1872887793 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f1cc8059510 con 0x7f1cd00cf180 2024-05-09T06:38:47.906 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:38:47.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 msgr2=0x7f1cbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 0x7f1cbc06e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0036000 tx=0x7f1cc000d040 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 msgr2=0x7f1cd0141c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e930 tx=0x7f1cc800edf0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 shutdown_connections 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1cbc06be30 0x7f1cbc06e2f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cd00cf180 0x7f1cd0141c30 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1cd00b3300 0x7f1cd01416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 --2- 172.21.15.1:0/1872887793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1cd006e6d0 0x7f1cd00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 >> 172.21.15.1:0/1872887793 conn(0x7f1cd00cc4b0 msgr2=0x7f1cd013f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 shutdown_connections 2024-05-09T06:38:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:47.927+0000 7f1cd7653640 1 -- 172.21.15.1:0/1872887793 wait complete. 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:47.918 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:38:47.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:38:47.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:38:47.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:47.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:47.919 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:38:47.935 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:38:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:38:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:38:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42014sec preferred_lft 42014sec 2024-05-09T06:38:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.937 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:38:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:47.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:47.938 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:38:48.000 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:38:48.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.001 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:38:48.059 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:38:48.059 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:38:48.059 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:38:48.060 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:38:48.060 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:38:48.060 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:38:48.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:38:48.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:48.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.189 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:38:48.205 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:38:48.205 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:38:48.206 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:48.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:38:48.272 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:38:48.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:48.272 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:48.328 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:48.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:48.328 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:48.384 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:48.450 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:48.509 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:48.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:48.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:48.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:48.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.511 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:48.564 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:38:48.564 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:38:48.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:48.633 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:48.691 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:48.692 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:48.693 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:48.694 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:48.694 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:48.694 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:48.694 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:48.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.694 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:48.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:38:48.712+0000 7f8f0991b480 -1 init, newargv = 0x55d511d1e1f0 newargc=15 2024-05-09T06:38:48.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[81233]: starting ceph client 2024-05-09T06:38:48.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[81233]: starting fuse 2024-05-09T06:38:48.711 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:38:48.711 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:38:48.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:38:48.711 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:38:48.712 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:38:48.713 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:48.833 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.81233.asok 2024-05-09T06:38:48.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:48.843 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81233.asok 2024-05-09T06:38:48.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81233.asok status 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "81233", 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:38:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9879, 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 9879 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1317236532 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1317236532 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.9879 192.168.0.1:0/1317236532", 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1317236532", 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 99, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 121, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:38:48.985 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:48.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:48.991 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:38:48.991 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:49.008 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:38:49.008 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:49.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:49.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:49.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:49.075 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-09T06:38:49.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:38:49.119 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:38:49.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:49.136 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-09T06:38:49.265 INFO:teuthology.orchestra.run.smithi001.stderr:8+0 records in 2024-05-09T06:38:49.265 INFO:teuthology.orchestra.run.smithi001.stderr:8+0 records out 2024-05-09T06:38:49.265 INFO:teuthology.orchestra.run.smithi001.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0346661 s, 242 MB/s 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-09T06:38:49.266 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:49.335 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627779 2024-05-09T06:38:49.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:49.344 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:38:49.344 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:38:49.344 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:38:49.345 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:38:49.345 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-09T06:38:49.345 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:38:49.452 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627780 2024-05-09T06:38:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:49.461 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:49.461 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:49.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:49.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.681+0000 7f9cb4df9640 1 Processor -- start 2024-05-09T06:38:49.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.681+0000 7f9cb4df9640 1 -- start start 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cb4df9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9cb006e6d0 0x7f9cb006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cb4df9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cb4df9640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb00ce0e0 con 0x7f9cb00cdab0 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cb4df9640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb00ce3c0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cb4df9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb00ce6a0 con 0x7f9cb00cf180 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cae59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9cb006e6d0 0x7f9cb006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9caed9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f9cb00cdab0 0x7f9cb015e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51322/0 (socket says 172.21.15.1:51322) 2024-05-09T06:38:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9caed9d640 1 -- 172.21.15.1:0/957510470 learned_addr learned my addr 172.21.15.1:0/957510470 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cae59c640 1 -- 172.21.15.1:0/957510470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 msgr2=0x7f9cb00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.682+0000 7f9cadd9b640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cae59c640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cad59a640 1 -- 172.21.15.1:0/957510470 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933089132 0 0) 0x7f9cb00ce0e0 con 0x7f9cb00cdab0 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cae59c640 1 -- 172.21.15.1:0/957510470 >> v1:172.21.15.1:6789/0 conn(0x7f9cb00cdab0 legacy=0x7f9cb015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cae59c640 1 -- 172.21.15.1:0/957510470 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00ce980 con 0x7f9cb006e6d0 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cae59c640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb006eab0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009d90 tx=0x7f9ca802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6956b54951d67665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cad59a640 1 -- 172.21.15.1:0/957510470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8007740 con 0x7f9cb006e6d0 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cad59a640 1 -- 172.21.15.1:0/957510470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca80078e0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.683+0000 7f9cad59a640 1 -- 172.21.15.1:0/957510470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8007cb0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 -- 172.21.15.1:0/957510470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 msgr2=0x7f9cb006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb006eab0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009d90 tx=0x7f9ca802a660 comp rx=0 tx=0).stop 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 -- 172.21.15.1:0/957510470 shutdown_connections 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/957510470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb006eab0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 -- 172.21.15.1:0/957510470 >> 172.21.15.1:0/957510470 conn(0x7f9cb00cc4b0 msgr2=0x7f9cb00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 -- 172.21.15.1:0/957510470 shutdown_connections 2024-05-09T06:38:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.684+0000 7f9cb4df9640 1 -- 172.21.15.1:0/957510470 wait complete. 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.685+0000 7f9cb4df9640 1 Processor -- start 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.685+0000 7f9cb4df9640 1 -- start start 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb00cdab0 0x7f9cb015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0070ed0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb006eff0 con 0x7f9cb00cdab0 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cb4df9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb01447c0 con 0x7f9cb00cf180 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cae59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cae59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60212/0 (socket says 172.21.15.1:60212) 2024-05-09T06:38:49.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9cae59c640 1 -- 172.21.15.1:0/1885080815 learned_addr learned my addr 172.21.15.1:0/1885080815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.686+0000 7f9caed9d640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cadd9b640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb00cdab0 0x7f9cb015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 -- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 msgr2=0x7f9cb00ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 -- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb00cdab0 msgr2=0x7f9cb015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb00cdab0 0x7f9cb015eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 -- 172.21.15.1:0/1885080815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb015efe0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cae59c640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009d90 tx=0x7f9ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca803f070 con 0x7f9cb006e6d0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca8031c40 con 0x7f9cb006e6d0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb015d480 con 0x7f9cb006e6d0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8043410 con 0x7f9cb006e6d0 2024-05-09T06:38:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.687+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb015da40 con 0x7f9cb006e6d0 2024-05-09T06:38:49.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.688+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb00752b0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.689+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ca80435b0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.689+0000 7f9ca6ffd640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 0x7f9c9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.689+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f9ca80ac620 con 0x7f9cb006e6d0 2024-05-09T06:38:49.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.689+0000 7f9cadd9b640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 0x7f9c9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.690+0000 7f9cadd9b640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 0x7f9c9006e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.694+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9ca807adc0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.881+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb006cf80 con 0x7f9cb006e6d0 2024-05-09T06:38:49.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.882+0000 7f9ca6ffd640 1 -- 172.21.15.1:0/1885080815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16820 (secure 0 0 0) 0x7f9ca807a6c0 con 0x7f9cb006e6d0 2024-05-09T06:38:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:49.862 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":121,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:30.738835+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:49.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 msgr2=0x7f9c9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 0x7f9c9006e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c98008040 comp rx=0 tx=0).stop 2024-05-09T06:38:49.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 msgr2=0x7f9cb015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009d90 tx=0x7f9ca8004a40 comp rx=0 tx=0).stop 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 shutdown_connections 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f9c9006be30 0x7f9c9006e2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9cb00cf180 0x7f9cb00ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9cb00cdab0 0x7f9cb015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 --2- 172.21.15.1:0/1885080815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9cb006e6d0 0x7f9cb015cdc0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 >> 172.21.15.1:0/1885080815 conn(0x7f9cb00cc4b0 msgr2=0x7f9cb00af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 shutdown_connections 2024-05-09T06:38:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:49.884+0000 7f9cb4df9640 1 -- 172.21.15.1:0/1885080815 wait complete. 2024-05-09T06:38:49.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:49.876 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:49.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.002+0000 7f5b7b8f5640 1 Processor -- start 2024-05-09T06:38:49.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.002+0000 7f5b7b8f5640 1 -- start start 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b7b8f5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5b74071fd0 0x7f5b7406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b7b8f5640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7415b420 con 0x7f5b7415c4a0 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b7b8f5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7415b6e0 con 0x7f5b74158a50 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b7b8f5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5b74158e30 con 0x7f5b74071fd0 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b79e92640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5b7415c4a0 0x7f5b7415ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51332/0 (socket says 172.21.15.1:51332) 2024-05-09T06:38:49.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b79e92640 1 -- 172.21.15.1:0/4148094976 learned_addr learned my addr 172.21.15.1:0/4148094976 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087736147 0 0) 0x7f5b7415b420 con 0x7f5b7415c4a0 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b60003520 con 0x7f5b7415c4a0 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b78e90640 1 --2- 172.21.15.1:0/4148094976 >> v2:172.21.15.6:3300/0 conn(0x7f5b74071fd0 0x7f5b7406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 22298392 0 0) 0x7f5b60003520 con 0x7f5b7415c4a0 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 >> v2:172.21.15.6:3300/0 conn(0x7f5b74071fd0 msgr2=0x7f5b7406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 --2- 172.21.15.1:0/4148094976 >> v2:172.21.15.6:3300/0 conn(0x7f5b74071fd0 0x7f5b7406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 >> v1:172.21.15.6:6789/0 conn(0x7f5b74158a50 legacy=0x7f5b740718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7415b9c0 con 0x7f5b7415c4a0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.003+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f5b6c0019f0 con 0x7f5b7415c4a0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 >> v1:172.21.15.1:6789/0 conn(0x7f5b7415c4a0 legacy=0x7f5b7415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b600039b0 0x7f5b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b60004800 0x7f5b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b74158e30 con 0x7f5b600051c0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000c060 con 0x7f5b600039b0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6000c260 con 0x7f5b60004800 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79691640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b600039b0 0x7f5b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b78e90640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b60004800 0x7f5b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 -- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b60004800 msgr2=0x7f5b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:49.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b60004800 0x7f5b60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 -- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b600039b0 msgr2=0x7f5b60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b600039b0 0x7f5b60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.004+0000 7f5b79e92640 1 -- 172.21.15.1:0/4148094976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b60003520 con 0x7f5b600051c0 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b79e92640 1 -- 172.21.15.1:0/4148094976 reap_dead start 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b79e92640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b6000b9c0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406c2c0 tx=0x7f5b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7ae8400c635318 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00a480 con 0x7f5b600051c0 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c02f070 con 0x7f5b600051c0 2024-05-09T06:38:49.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b6bfff640 1 -- 172.21.15.1:0/4148094976 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c02bc50 con 0x7f5b600051c0 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 msgr2=0x7f5b6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.005+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b6000b9c0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406c2c0 tx=0x7f5b60005830 comp rx=0 tx=0).stop 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.006+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/4148094976 shutdown_connections 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.006+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/4148094976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b6000b9c0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.006+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/4148094976 >> 172.21.15.1:0/4148094976 conn(0x7f5b740cc4d0 msgr2=0x7f5b740cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.006+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/4148094976 shutdown_connections 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.006+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/4148094976 wait complete. 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.007+0000 7f5b7b8f5640 1 Processor -- start 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.007+0000 7f5b7b8f5640 1 -- start start 2024-05-09T06:38:49.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 0x7f5b740ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 0x7f5b740af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7415bf80 con 0x7f5b600051c0 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b74072480 con 0x7f5b74154ca0 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b7b8f5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5b74070ea0 con 0x7f5b740ae540 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60234/0 (socket says 172.21.15.1:60234) 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 -- 172.21.15.1:0/1174332901 learned_addr learned my addr 172.21.15.1:0/1174332901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79e92640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 0x7f5b740af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b78e90640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 0x7f5b740ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 -- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 msgr2=0x7f5b740af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 0x7f5b740af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 -- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 msgr2=0x7f5b740ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 0x7f5b740ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:49.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 -- 172.21.15.1:0/1174332901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740b3da0 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79e92640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 0x7f5b740af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b78e90640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 0x7f5b740ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b79691640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000f5f0 tx=0x7f5b7000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.008+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700052e0 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.009+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70018070 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.009+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70003810 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.009+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b740b4080 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.009+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b741f43b0 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.010+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b740752b0 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.010+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b700039b0 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.011+0000 7f5b69ffb640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 0x7f5b5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.011+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f5b7008ae50 con 0x7f5b600051c0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.011+0000 7f5b78e90640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 0x7f5b5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:49.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.011+0000 7f5b78e90640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 0x7f5b5806e320 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b640093d0 tx=0x7f5b64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.015+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b700595d0 con 0x7f5b600051c0 2024-05-09T06:38:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.198+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b740b4250 con 0x7f5b600051c0 2024-05-09T06:38:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.199+0000 7f5b69ffb640 1 -- 172.21.15.1:0/1174332901 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f5b70058ed0 con 0x7f5b600051c0 2024-05-09T06:38:50.179 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:50.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 msgr2=0x7f5b5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 0x7f5b5806e320 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b640093d0 tx=0x7f5b64008040 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 msgr2=0x7f5b7414e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000f5f0 tx=0x7f5b7000fab0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 shutdown_connections 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be60 0x7f5b5806e320 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b740ae540 0x7f5b740af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b74154ca0 0x7f5b740ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 --2- 172.21.15.1:0/1174332901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b600051c0 0x7f5b7414e2a0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 >> 172.21.15.1:0/1174332901 conn(0x7f5b740cc4d0 msgr2=0x7f5b740b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 shutdown_connections 2024-05-09T06:38:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.201+0000 7f5b7b8f5640 1 -- 172.21.15.1:0/1174332901 wait complete. 2024-05-09T06:38:50.192 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:38:50.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.318+0000 7f381d502640 1 Processor -- start 2024-05-09T06:38:50.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 -- start start 2024-05-09T06:38:50.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f381815e190 0x7f381815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180d1c70 0x7f38180d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3818161fe0 con 0x7f38180b56d0 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f381815f580 con 0x7f381815e190 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.319+0000 7f381d502640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38181622c0 con 0x7f38180d1c70 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3816ffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f381815e190 0x7f381815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f38167fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180d1c70 0x7f38180d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f38177fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f38180b56d0 0x7f38180b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:54098/0 (socket says 172.21.15.1:54098) 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f38177fe640 1 -- 172.21.15.1:0/1884198744 learned_addr learned my addr 172.21.15.1:0/1884198744 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3816ffd640 1 -- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180d1c70 msgr2=0x7f38180d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3816ffd640 1 --2- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180d1c70 0x7f38180d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3816ffd640 1 -- 172.21.15.1:0/1884198744 >> v1:172.21.15.6:6789/0 conn(0x7f38180b56d0 legacy=0x7f38180b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3815ffb640 1 -- 172.21.15.1:0/1884198744 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819403092 0 0) 0x7f3818161fe0 con 0x7f38180b56d0 2024-05-09T06:38:50.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.320+0000 7f3816ffd640 1 -- 172.21.15.1:0/1884198744 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38181625a0 con 0x7f381815e190 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.321+0000 7f3816ffd640 1 --2- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f381815e190 0x7f381815f780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38100097a0 tx=0x7f381002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d38b9199959be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.321+0000 7f3815ffb640 1 -- 172.21.15.1:0/1884198744 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810037070 con 0x7f381815e190 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.321+0000 7f3815ffb640 1 -- 172.21.15.1:0/1884198744 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3810032440 con 0x7f381815e190 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.321+0000 7f3815ffb640 1 -- 172.21.15.1:0/1884198744 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810032810 con 0x7f381815e190 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 -- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f381815e190 msgr2=0x7f381815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 --2- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f381815e190 0x7f381815f780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38100097a0 tx=0x7f381002a180 comp rx=0 tx=0).stop 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 -- 172.21.15.1:0/1884198744 shutdown_connections 2024-05-09T06:38:50.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 --2- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180d1c70 0x7f38180d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 --2- 172.21.15.1:0/1884198744 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f381815e190 0x7f381815f780 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 -- 172.21.15.1:0/1884198744 >> 172.21.15.1:0/1884198744 conn(0x7f3818073340 msgr2=0x7f381809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 -- 172.21.15.1:0/1884198744 shutdown_connections 2024-05-09T06:38:50.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.322+0000 7f381d502640 1 -- 172.21.15.1:0/1884198744 wait complete. 2024-05-09T06:38:50.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.323+0000 7f381d502640 1 Processor -- start 2024-05-09T06:38:50.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.323+0000 7f381d502640 1 -- start start 2024-05-09T06:38:50.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180b56d0 0x7f38180c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 0x7f3818156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f38180d4420 con 0x7f381815e190 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3818161c60 con 0x7f38180d1c70 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f381d502640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f38180d45a0 con 0x7f38180b56d0 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f38177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f38167fc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 0x7f3818156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f38177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60250/0 (socket says 172.21.15.1:60250) 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f38177fe640 1 -- 172.21.15.1:0/3662570086 learned_addr learned my addr 172.21.15.1:0/3662570086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.324+0000 7f3816ffd640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180b56d0 0x7f38180c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 -- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180b56d0 msgr2=0x7f38180c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180b56d0 0x7f38180c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 -- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 msgr2=0x7f3818156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 0x7f3818156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 -- 172.21.15.1:0/3662570086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180caa50 con 0x7f381815e190 2024-05-09T06:38:50.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38167fc640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 0x7f3818156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f38177fe640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f380800c480 tx=0x7f380800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808018070 con 0x7f381815e190 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38080074e0 con 0x7f381815e190 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3818156cc0 con 0x7f381815e190 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808004a20 con 0x7f381815e190 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.325+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38180ca390 con 0x7f381815e190 2024-05-09T06:38:50.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.326+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38180752b0 con 0x7f381815e190 2024-05-09T06:38:50.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.327+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f380801b070 con 0x7f381815e190 2024-05-09T06:38:50.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.327+0000 7f380f7fe640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 0x7f37f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.327+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f380808a170 con 0x7f381815e190 2024-05-09T06:38:50.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.327+0000 7f3816ffd640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 0x7f37f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.328+0000 7f3816ffd640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 0x7f37f806e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3810034000 tx=0x7f381002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.331+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38080583d0 con 0x7f381815e190 2024-05-09T06:38:50.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.514+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f381808e1c0 con 0x7f381815e190 2024-05-09T06:38:50.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.514+0000 7f380f7fe640 1 -- 172.21.15.1:0/3662570086 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f380805c6d0 con 0x7f381815e190 2024-05-09T06:38:50.495 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:38:50.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 msgr2=0x7f37f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 0x7f37f806e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3810034000 tx=0x7f381002e000 comp rx=0 tx=0).stop 2024-05-09T06:38:50.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 msgr2=0x7f38180cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f380800c480 tx=0x7f380800c940 comp rx=0 tx=0).stop 2024-05-09T06:38:50.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 shutdown_connections 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f37f806be30 0x7f37f806e2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f381815e190 0x7f38180cbcb0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38180d1c70 0x7f3818156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.516+0000 7f381d502640 1 --2- 172.21.15.1:0/3662570086 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38180b56d0 0x7f38180c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.517+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 >> 172.21.15.1:0/3662570086 conn(0x7f3818073340 msgr2=0x7f38180b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.517+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 shutdown_connections 2024-05-09T06:38:50.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.517+0000 7f381d502640 1 -- 172.21.15.1:0/3662570086 wait complete. 2024-05-09T06:38:50.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:38:50.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.633+0000 7f3b390d4640 1 Processor -- start 2024-05-09T06:38:50.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.633+0000 7f3b390d4640 1 -- start start 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b390d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b340b11f0 0x7f3b340b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b390d4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3415e900 con 0x7f3b3415e390 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b390d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3415ebe0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b390d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3415eec0 con 0x7f3b340b11f0 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b3259c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b340b11f0 0x7f3b340b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b32d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3b3415e390 0x7f3b340cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51334/0 (socket says 172.21.15.1:51334) 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b32d9d640 1 -- 172.21.15.1:0/1938011927 learned_addr learned my addr 172.21.15.1:0/1938011927 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b3259c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b340b11f0 0x7f3b340b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60256/0 (socket says 172.21.15.1:60256) 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b3259c640 1 -- 172.21.15.1:0/1938011927 >> v1:172.21.15.6:6789/0 conn(0x7f3b340cc8e0 legacy=0x7f3b340ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:50.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b3259c640 1 -- 172.21.15.1:0/1938011927 >> v1:172.21.15.1:6789/0 conn(0x7f3b3415e390 legacy=0x7f3b340cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.634+0000 7f3b3259c640 1 -- 172.21.15.1:0/1938011927 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3415f1a0 con 0x7f3b340b11f0 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.635+0000 7f3b3259c640 1 --2- 172.21.15.1:0/1938011927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340b11f0 0x7f3b340b1680 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3b280097a0 tx=0x7f3b2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a3d130c366ee4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.635+0000 7f3b31d9b640 1 -- 172.21.15.1:0/1938011927 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28037070 con 0x7f3b340b11f0 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.635+0000 7f3b31d9b640 1 -- 172.21.15.1:0/1938011927 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28032440 con 0x7f3b340b11f0 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.635+0000 7f3b31d9b640 1 -- 172.21.15.1:0/1938011927 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28032810 con 0x7f3b340b11f0 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 -- 172.21.15.1:0/1938011927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340b11f0 msgr2=0x7f3b340b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 --2- 172.21.15.1:0/1938011927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340b11f0 0x7f3b340b1680 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3b280097a0 tx=0x7f3b2802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 -- 172.21.15.1:0/1938011927 shutdown_connections 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 --2- 172.21.15.1:0/1938011927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340b11f0 0x7f3b340b1680 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 -- 172.21.15.1:0/1938011927 >> 172.21.15.1:0/1938011927 conn(0x7f3b340b1da0 msgr2=0x7f3b340b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 -- 172.21.15.1:0/1938011927 shutdown_connections 2024-05-09T06:38:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.636+0000 7f3b390d4640 1 -- 172.21.15.1:0/1938011927 wait complete. 2024-05-09T06:38:50.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.637+0000 7f3b390d4640 1 Processor -- start 2024-05-09T06:38:50.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.637+0000 7f3b390d4640 1 -- start start 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b340b11f0 0x7f3b340b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b3415e390 0x7f3b340ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340cffb0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34162ba0 con 0x7f3b3415e390 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b390d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340cfb60 con 0x7f3b340b11f0 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b3259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b3259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60266/0 (socket says 172.21.15.1:60266) 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b3259c640 1 -- 172.21.15.1:0/919372130 learned_addr learned my addr 172.21.15.1:0/919372130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b32d9d640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b340b11f0 0x7f3b340b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.638+0000 7f3b3359e640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b3415e390 0x7f3b340ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 -- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b340b11f0 msgr2=0x7f3b340b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b340b11f0 0x7f3b340b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 -- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b3415e390 msgr2=0x7f3b340ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b3415e390 0x7f3b340ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 -- 172.21.15.1:0/919372130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340b49e0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b3259c640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f3b28000c00 tx=0x7f3b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28037040 con 0x7f3b340cc8e0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28042070 con 0x7f3b340cc8e0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b340b4cc0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28030a70 con 0x7f3b340cc8e0 2024-05-09T06:38:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.639+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3406e030 con 0x7f3b340cc8e0 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.640+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b340752b0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.641+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b28030c10 con 0x7f3b340cc8e0 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.641+0000 7f3b1b7fe640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.641+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f3b280abad0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.641+0000 7f3b32d9d640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.642+0000 7f3b32d9d640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 0x7f3b2006e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406f4a0 tx=0x7f3b24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.646+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b28045360 con 0x7f3b340cc8e0 2024-05-09T06:38:50.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.827+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b340b2ae0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.828+0000 7f3b1b7fe640 1 -- 172.21.15.1:0/919372130 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f3b2807a2f0 con 0x7f3b340cc8e0 2024-05-09T06:38:50.808 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 msgr2=0x7f3b2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 0x7f3b2006e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406f4a0 tx=0x7f3b24009040 comp rx=0 tx=0).stop 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 msgr2=0x7f3b341f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f3b28000c00 tx=0x7f3b280046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 shutdown_connections 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3b3415e390 0x7f3b340ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b340cc8e0 0x7f3b341f7510 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 --2- 172.21.15.1:0/919372130 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3b340b11f0 0x7f3b340b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.829+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 >> 172.21.15.1:0/919372130 conn(0x7f3b340b1da0 msgr2=0x7f3b3415b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.830+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 shutdown_connections 2024-05-09T06:38:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.830+0000 7f3b390d4640 1 -- 172.21.15.1:0/919372130 wait complete. 2024-05-09T06:38:50.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:38:50.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.946+0000 7fcda31c1640 1 Processor -- start 2024-05-09T06:38:50.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.946+0000 7fcda31c1640 1 -- start start 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda31c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 0x7fcd9c072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda31c1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 0x7fcd9c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda31c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9c15cdc0 con 0x7fcd9c06b1e0 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda31c1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c15d380 con 0x7fcd9c0703d0 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda31c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c072180 con 0x7fcd9c153150 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcd93fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 0x7fcd9c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda0f5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 0x7fcd9c072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda175e640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fcd9c06b1e0 0x7fcd9c0cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:54120/0 (socket says 172.21.15.1:54120) 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda175e640 1 -- 172.21.15.1:0/2202327522 learned_addr learned my addr 172.21.15.1:0/2202327522 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda0f5d640 1 -- 172.21.15.1:0/2202327522 >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 msgr2=0x7fcd9c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda0f5d640 1 --2- 172.21.15.1:0/2202327522 >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 0x7fcd9c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda0f5d640 1 -- 172.21.15.1:0/2202327522 >> v1:172.21.15.6:6789/0 conn(0x7fcd9c06b1e0 legacy=0x7fcd9c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcda0f5d640 1 -- 172.21.15.1:0/2202327522 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c15d0a0 con 0x7fcd9c153150 2024-05-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.947+0000 7fcd93fff640 1 --2- 172.21.15.1:0/2202327522 >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 0x7fcd9c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.948+0000 7fcda0f5d640 1 --2- 172.21.15.1:0/2202327522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 0x7fcd9c072610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097a0 tx=0x7fcd9802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153ef089d9487d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.948+0000 7fcd937fe640 1 -- 172.21.15.1:0/2202327522 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98037070 con 0x7fcd9c153150 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.948+0000 7fcd937fe640 1 -- 172.21.15.1:0/2202327522 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd9802ae50 con 0x7fcd9c153150 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.948+0000 7fcd937fe640 1 -- 172.21.15.1:0/2202327522 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98032660 con 0x7fcd9c153150 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 -- 172.21.15.1:0/2202327522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 msgr2=0x7fcd9c072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 --2- 172.21.15.1:0/2202327522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 0x7fcd9c072610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097a0 tx=0x7fcd9802a100 comp rx=0 tx=0).stop 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 -- 172.21.15.1:0/2202327522 shutdown_connections 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 --2- 172.21.15.1:0/2202327522 >> v2:172.21.15.6:3300/0 conn(0x7fcd9c0703d0 0x7fcd9c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 --2- 172.21.15.1:0/2202327522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c153150 0x7fcd9c072610 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 -- 172.21.15.1:0/2202327522 >> 172.21.15.1:0/2202327522 conn(0x7fcd9c0d0b50 msgr2=0x7fcd9c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 -- 172.21.15.1:0/2202327522 shutdown_connections 2024-05-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.949+0000 7fcda31c1640 1 -- 172.21.15.1:0/2202327522 wait complete. 2024-05-09T06:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.950+0000 7fcda31c1640 1 Processor -- start 2024-05-09T06:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.950+0000 7fcda31c1640 1 -- start start 2024-05-09T06:38:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c06b1e0 0x7fcd9c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd9c153150 0x7fcd9c152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c072cd0 con 0x7fcd9c0703d0 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c072b50 con 0x7fcd9c153150 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda31c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c15c9b0 con 0x7fcd9c06b1e0 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcd93fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60282/0 (socket says 172.21.15.1:60282) 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcd93fff640 1 -- 172.21.15.1:0/657489053 learned_addr learned my addr 172.21.15.1:0/657489053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda0f5d640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c06b1e0 0x7fcd9c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.951+0000 7fcda175e640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd9c153150 0x7fcd9c152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 -- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c06b1e0 msgr2=0x7fcd9c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c06b1e0 0x7fcd9c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 -- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd9c153150 msgr2=0x7fcd9c152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd9c153150 0x7fcd9c152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 -- 172.21.15.1:0/657489053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c142c70 con 0x7fcd9c0703d0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd93fff640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00ebe0 tx=0x7fcd8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c017070 con 0x7fcd9c0703d0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00cc30 con 0x7fcd9c0703d0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c142f50 con 0x7fcd9c0703d0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c013470 con 0x7fcd9c0703d0 2024-05-09T06:38:50.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.952+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c143510 con 0x7fcd9c0703d0 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.953+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c143950 con 0x7fcd9c0703d0 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.954+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd8c01a070 con 0x7fcd9c0703d0 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.954+0000 7fcd917fa640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 0x7fcd8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.954+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fcd8c08b270 con 0x7fcd9c0703d0 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.954+0000 7fcda0f5d640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 0x7fcd8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:50.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.955+0000 7fcda0f5d640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 0x7fcd8806e2f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd98034000 tx=0x7fcd9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:50.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:50.958+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd8c059430 con 0x7fcd9c0703d0 2024-05-09T06:38:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.153+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd9c1457c0 con 0x7fcd9c0703d0 2024-05-09T06:38:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.153+0000 7fcd917fa640 1 -- 172.21.15.1:0/657489053 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7fcd8c05d730 con 0x7fcd9c0703d0 2024-05-09T06:38:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 msgr2=0x7fcd8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 0x7fcd8806e2f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd98034000 tx=0x7fcd9802e000 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 msgr2=0x7fcd9c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00ebe0 tx=0x7fcd8c00c520 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 shutdown_connections 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd8806be30 0x7fcd8806e2f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd9c153150 0x7fcd9c152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd9c0703d0 0x7fcd9c0c6ac0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 --2- 172.21.15.1:0/657489053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd9c06b1e0 0x7fcd9c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 >> 172.21.15.1:0/657489053 conn(0x7fcd9c0d0b50 msgr2=0x7fcd9c0cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 shutdown_connections 2024-05-09T06:38:51.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.155+0000 7fcda31c1640 1 -- 172.21.15.1:0/657489053 wait complete. 2024-05-09T06:38:51.146 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:51.147 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:51.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:38:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.326+0000 7fb40ee5d640 1 Processor -- start 2024-05-09T06:38:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.326+0000 7fb40ee5d640 1 -- start start 2024-05-09T06:38:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4080b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4080b2a60 0x7fb4080b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb4080b4da0 0x7fb40815f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb4080b5230 con 0x7fb4080b4da0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb40806eb40 con 0x7fb4080b2a60 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40ee5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb40806ee20 con 0x7fb4081454b0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.327+0000 7fb40d3fa640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb4080b4da0 0x7fb40815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb4080b4da0 0x7fb40815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60288/0 (socket says 172.21.15.1:60288) 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 -- 172.21.15.1:0/2965905337 learned_addr learned my addr 172.21.15.1:0/2965905337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40cbf9640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4080b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb407fff640 1 --2- 172.21.15.1:0/2965905337 >> v2:172.21.15.6:3300/0 conn(0x7fb4080b2a60 0x7fb4080b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 -- 172.21.15.1:0/2965905337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 msgr2=0x7fb4080b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4080b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 -- 172.21.15.1:0/2965905337 >> v2:172.21.15.6:3300/0 conn(0x7fb4080b2a60 msgr2=0x7fb4080b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 --2- 172.21.15.1:0/2965905337 >> v2:172.21.15.6:3300/0 conn(0x7fb4080b2a60 0x7fb4080b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40d3fa640 1 -- 172.21.15.1:0/2965905337 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40806d8e0 con 0x7fb4080b4da0 2024-05-09T06:38:51.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.328+0000 7fb40cbf9640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4080b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40d3fa640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb40815f960 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fb4080731c0 tx=0x7fb3f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67cc50cf8e954b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb4077fe640 1 -- 172.21.15.1:0/2965905337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0037070 con 0x7fb4080b4da0 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb4077fe640 1 -- 172.21.15.1:0/2965905337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f0032440 con 0x7fb4080b4da0 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb4077fe640 1 -- 172.21.15.1:0/2965905337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0032810 con 0x7fb4080b4da0 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2965905337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 msgr2=0x7fb40815f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb40815f960 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fb4080731c0 tx=0x7fb3f002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2965905337 shutdown_connections 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb40815f960 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2965905337 >> v2:172.21.15.6:3300/0 conn(0x7fb4080b2a60 0x7fb4080b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2965905337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4080b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2965905337 >> 172.21.15.1:0/2965905337 conn(0x7fb4080cc4b0 msgr2=0x7fb4080cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2965905337 shutdown_connections 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.329+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2965905337 wait complete. 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.331+0000 7fb40ee5d640 1 Processor -- start 2024-05-09T06:38:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.331+0000 7fb40ee5d640 1 -- start start 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.331+0000 7fb40ee5d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4080b2a60 0x7fb40813f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4081f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb40806e0e0 con 0x7fb4080b4da0 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb408145890 con 0x7fb4080b2a60 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb40806e260 con 0x7fb4081454b0 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60292/0 (socket says 172.21.15.1:60292) 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 -- 172.21.15.1:0/2545582224 learned_addr learned my addr 172.21.15.1:0/2545582224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40d3fa640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4081f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40cbf9640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4080b2a60 0x7fb40813f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 -- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 msgr2=0x7fb4081f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4081f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 -- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4080b2a60 msgr2=0x7fb40813f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4080b2a60 0x7fb40813f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 -- 172.21.15.1:0/2545582224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40813ce30 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb407fff640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007e70 tx=0x7fb3f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8014620 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f8019070 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40813d110 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8014a00 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.332+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40813d6d0 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.333+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080752b0 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.334+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3f8014ba0 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.335+0000 7fb4057fa640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 0x7fb3e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.335+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fb3f808b680 con 0x7fb4080b4da0 2024-05-09T06:38:51.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.335+0000 7fb40cbf9640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 0x7fb3e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.335+0000 7fb40cbf9640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 0x7fb3e806e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb40813c6e0 tx=0x7fb400008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.339+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3f80598e0 con 0x7fb4080b4da0 2024-05-09T06:38:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.522+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb408072fa0 con 0x7fb4080b4da0 2024-05-09T06:38:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.524+0000 7fb4057fa640 1 -- 172.21.15.1:0/2545582224 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16820 (secure 0 0 0) 0x7fb3f805dbe0 con 0x7fb4080b4da0 2024-05-09T06:38:51.504 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:51.505 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":121,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:30.738835+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:38:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 msgr2=0x7fb3e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 0x7fb3e806e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb40813c6e0 tx=0x7fb400008040 comp rx=0 tx=0).stop 2024-05-09T06:38:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 msgr2=0x7fb4081f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007e70 tx=0x7fb3f800e4b0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 shutdown_connections 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb3e806be30 0x7fb3e806e2f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4081454b0 0x7fb4081f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4080b4da0 0x7fb4081f7270 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 --2- 172.21.15.1:0/2545582224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4080b2a60 0x7fb40813f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 >> 172.21.15.1:0/2545582224 conn(0x7fb4080cc4b0 msgr2=0x7fb4081438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 shutdown_connections 2024-05-09T06:38:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.526+0000 7fb40ee5d640 1 -- 172.21.15.1:0/2545582224 wait complete. 2024-05-09T06:38:51.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:38:51.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:38:51.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.644+0000 7f61080e1640 1 Processor -- start 2024-05-09T06:38:51.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.644+0000 7f61080e1640 1 -- start start 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f610006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f61000b4de0 0x7f610015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f61000b5260 0x7f610015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6100156980 con 0x7f61000b4de0 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f610006d870 con 0x7f61000b5260 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f61080e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f610015d320 con 0x7f610015c4c0 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f61000b4de0 0x7f610015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f61000b4de0 0x7f610015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60304/0 (socket says 172.21.15.1:60304) 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 -- 172.21.15.1:0/1002867822 learned_addr learned my addr 172.21.15.1:0/1002867822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f6105e7d640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f610006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610667e640 1 --2- 172.21.15.1:0/1002867822 >> v2:172.21.15.6:3300/0 conn(0x7f61000b5260 0x7f610015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 -- 172.21.15.1:0/1002867822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 msgr2=0x7f610006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f610006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 -- 172.21.15.1:0/1002867822 >> v2:172.21.15.6:3300/0 conn(0x7f61000b5260 msgr2=0x7f610015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 --2- 172.21.15.1:0/1002867822 >> v2:172.21.15.6:3300/0 conn(0x7f61000b5260 0x7f610015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.645+0000 7f610567c640 1 -- 172.21.15.1:0/1002867822 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610006cf40 con 0x7f61000b4de0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.646+0000 7f610567c640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f610015c8a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f61000b5830 tx=0x7f60f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e75dae5bd167063 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.646+0000 7f6104e7b640 1 -- 172.21.15.1:0/1002867822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0037070 con 0x7f61000b4de0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.646+0000 7f6104e7b640 1 -- 172.21.15.1:0/1002867822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f0032440 con 0x7f61000b4de0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.646+0000 7f6104e7b640 1 -- 172.21.15.1:0/1002867822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0032810 con 0x7f61000b4de0 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 -- 172.21.15.1:0/1002867822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 msgr2=0x7f610015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f610015c8a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f61000b5830 tx=0x7f60f002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 -- 172.21.15.1:0/1002867822 shutdown_connections 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 --2- 172.21.15.1:0/1002867822 >> v2:172.21.15.6:3300/0 conn(0x7f61000b5260 0x7f610015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f610015c8a0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 --2- 172.21.15.1:0/1002867822 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f610006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 -- 172.21.15.1:0/1002867822 >> 172.21.15.1:0/1002867822 conn(0x7f61000cc4d0 msgr2=0x7f61000cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 -- 172.21.15.1:0/1002867822 shutdown_connections 2024-05-09T06:38:51.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.647+0000 7f61080e1640 1 -- 172.21.15.1:0/1002867822 wait complete. 2024-05-09T06:38:51.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.648+0000 7f61080e1640 1 Processor -- start 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.648+0000 7f61080e1640 1 -- start start 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 0x7f61001f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f6100144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61000b2c40 con 0x7f61000b4de0 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f610006d500 con 0x7f61000b5260 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f61000b2dc0 con 0x7f610015c4c0 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60306/0 (socket says 172.21.15.1:60306) 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 -- 172.21.15.1:0/489001997 learned_addr learned my addr 172.21.15.1:0/489001997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f610667e640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f6100144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f610567c640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 0x7f61001f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 -- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 msgr2=0x7f6100144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f6100144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 -- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 msgr2=0x7f61001f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 0x7f61001f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 -- 172.21.15.1:0/489001997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61001f5080 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f610667e640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f6100144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f610567c640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 0x7f61001f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f6105e7d640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007e40 tx=0x7f60fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc014620 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc019070 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc014a00 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.649+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61000ade00 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.650+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61000ae3c0 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.651+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61000752b0 con 0x7f61000b4de0 2024-05-09T06:38:51.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.651+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60fc014ba0 con 0x7f61000b4de0 2024-05-09T06:38:51.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.651+0000 7f60f67fc640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.652+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f60fc08b880 con 0x7f61000b4de0 2024-05-09T06:38:51.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.652+0000 7f610567c640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.652+0000 7f610567c640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 0x7f60e406e2f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034000 tx=0x7f60f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.656+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61001f5080 con 0x7f61000b4de0 2024-05-09T06:38:51.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.841+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f610015f410 con 0x7f61000b4de0 2024-05-09T06:38:51.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.842+0000 7f60f67fc640 1 -- 172.21.15.1:0/489001997 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f60fc05a0a0 con 0x7f61000b4de0 2024-05-09T06:38:51.822 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:38:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 msgr2=0x7f60e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 0x7f60e406e2f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034000 tx=0x7f60f002e000 comp rx=0 tx=0).stop 2024-05-09T06:38:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 msgr2=0x7f61000aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007e40 tx=0x7f60fc00e4b0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 shutdown_connections 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f610015c4c0 0x7f6100144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61000b5260 0x7f61001f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 --2- 172.21.15.1:0/489001997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61000b4de0 0x7f61000aeea0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 >> 172.21.15.1:0/489001997 conn(0x7f61000cc4d0 msgr2=0x7f61000af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 shutdown_connections 2024-05-09T06:38:51.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.844+0000 7f61080e1640 1 -- 172.21.15.1:0/489001997 wait complete. 2024-05-09T06:38:51.835 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:38:51.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.962+0000 7f1413031640 1 Processor -- start 2024-05-09T06:38:51.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.962+0000 7f1413031640 1 -- start start 2024-05-09T06:38:51.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f1413031640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f1413031640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140c15b420 con 0x7f140c15c4a0 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f1413031640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140c15b6e0 con 0x7f140c158a50 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f1413031640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f140c158e30 con 0x7f140c071fd0 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f14115ce640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f140c15c4a0 0x7f140c15ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51350/0 (socket says 172.21.15.1:51350) 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f14115ce640 1 -- 172.21.15.1:0/1625867516 learned_addr learned my addr 172.21.15.1:0/1625867516 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.963+0000 7f140bfff640 1 --2- 172.21.15.1:0/1625867516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025753900 0 0) 0x7f140c15b420 con 0x7f140c15c4a0 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13f8003520 con 0x7f140c15c4a0 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897348105 0 0) 0x7f140c15b6e0 con 0x7f140c158a50 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f140c15b420 con 0x7f140c158a50 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140bfff640 1 -- 172.21.15.1:0/1625867516 >> v1:172.21.15.6:6789/0 conn(0x7f140c158a50 legacy=0x7f140c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140bfff640 1 -- 172.21.15.1:0/1625867516 >> v1:172.21.15.1:6789/0 conn(0x7f140c15c4a0 legacy=0x7f140c15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.964+0000 7f140bfff640 1 -- 172.21.15.1:0/1625867516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c15b9c0 con 0x7f140c071fd0 2024-05-09T06:38:51.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f140bfff640 1 --2- 172.21.15.1:0/1625867516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c06e6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009dc0 tx=0x7f13fc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1706c5fb1fa7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc007740 con 0x7f140c071fd0 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc0078e0 con 0x7f140c071fd0 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f140b7fe640 1 -- 172.21.15.1:0/1625867516 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc007cb0 con 0x7f140c071fd0 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 -- 172.21.15.1:0/1625867516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 msgr2=0x7f140c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 --2- 172.21.15.1:0/1625867516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c06e6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009dc0 tx=0x7f13fc02a6d0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 -- 172.21.15.1:0/1625867516 shutdown_connections 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 --2- 172.21.15.1:0/1625867516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c06e6d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 -- 172.21.15.1:0/1625867516 >> 172.21.15.1:0/1625867516 conn(0x7f140c0cc4d0 msgr2=0x7f140c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:51.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 -- 172.21.15.1:0/1625867516 shutdown_connections 2024-05-09T06:38:51.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.965+0000 7f1413031640 1 -- 172.21.15.1:0/1625867516 wait complete. 2024-05-09T06:38:51.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.966+0000 7f1413031640 1 Processor -- start 2024-05-09T06:38:51.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 -- start start 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f140c158a50 0x7f140c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f140c15bf80 con 0x7f140c15c4a0 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f140c072480 con 0x7f140c158a50 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.967+0000 7f1413031640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f140c070ea0 con 0x7f140c071fd0 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60310/0 (socket says 172.21.15.1:60310) 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 -- 172.21.15.1:0/14608001 learned_addr learned my addr 172.21.15.1:0/14608001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f140bfff640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f140c158a50 0x7f140c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f1410dcd640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 -- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 msgr2=0x7f140c1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c1480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 -- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f140c158a50 msgr2=0x7f140c14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f140c158a50 0x7f140c14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 -- 172.21.15.1:0/14608001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c14f730 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14115ce640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c390 tx=0x7f13f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4005d00 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c14bc60 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4007310 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.968+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c14c220 con 0x7f140c15c4a0 2024-05-09T06:38:51.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.969+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140c0752b0 con 0x7f140c15c4a0 2024-05-09T06:38:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.970+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13f401a070 con 0x7f140c15c4a0 2024-05-09T06:38:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.970+0000 7f14097fa640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 0x7f13ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.970+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f13f408a310 con 0x7f140c15c4a0 2024-05-09T06:38:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.970+0000 7f1410dcd640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 0x7f13ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.971+0000 7f1410dcd640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 0x7f13ec06e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1404007fd0 tx=0x7f1404009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:51.975+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13f40584e0 con 0x7f140c15c4a0 2024-05-09T06:38:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.155+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f140c15add0 con 0x7f140c15c4a0 2024-05-09T06:38:52.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.156+0000 7f14097fa640 1 -- 172.21.15.1:0/14608001 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f13f405c7e0 con 0x7f140c15c4a0 2024-05-09T06:38:52.136 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.157+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 msgr2=0x7f13ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.157+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 0x7f13ec06e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1404007fd0 tx=0x7f1404009040 comp rx=0 tx=0).stop 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.157+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 msgr2=0x7f140c14ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.157+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c390 tx=0x7f13f400c850 comp rx=0 tx=0).stop 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 shutdown_connections 2024-05-09T06:38:52.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f13ec06be30 0x7f13ec06e2f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f140c15c4a0 0x7f140c14ed60 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f140c158a50 0x7f140c14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 --2- 172.21.15.1:0/14608001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f140c071fd0 0x7f140c1480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 >> 172.21.15.1:0/14608001 conn(0x7f140c0cc4d0 msgr2=0x7f140c158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 shutdown_connections 2024-05-09T06:38:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.158+0000 7f1413031640 1 -- 172.21.15.1:0/14608001 wait complete. 2024-05-09T06:38:52.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:38:52.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.276+0000 7fd406e1c640 1 Processor -- start 2024-05-09T06:38:52.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.276+0000 7fd406e1c640 1 -- start start 2024-05-09T06:38:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd406e1c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd40015e190 0x7fd40015f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd406e1c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4000d45f0 0x7fd4000b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd406e1c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd400161fe0 con 0x7fd4000d1c70 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd406e1c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd40015f580 con 0x7fd4000d45f0 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd406e1c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd4001622c0 con 0x7fd40015e190 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4000d45f0 0x7fd4000b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd404bb8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd40015e190 0x7fd40015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4000d45f0 0x7fd4000b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60314/0 (socket says 172.21.15.1:60314) 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 -- 172.21.15.1:0/3730349003 learned_addr learned my addr 172.21.15.1:0/3730349003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd3fffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd4000d1c70 0x7fd4000d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51364/0 (socket says 172.21.15.1:51364) 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 -- 172.21.15.1:0/3730349003 >> v2:172.21.15.6:3300/0 conn(0x7fd40015e190 msgr2=0x7fd40015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 --2- 172.21.15.1:0/3730349003 >> v2:172.21.15.6:3300/0 conn(0x7fd40015e190 0x7fd40015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 -- 172.21.15.1:0/3730349003 >> v1:172.21.15.1:6789/0 conn(0x7fd4000d1c70 legacy=0x7fd4000d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd3ff7fe640 1 -- 172.21.15.1:0/3730349003 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524765608 0 0) 0x7fd400161fe0 con 0x7fd4000d1c70 2024-05-09T06:38:52.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.277+0000 7fd4053b9640 1 -- 172.21.15.1:0/3730349003 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4001625a0 con 0x7fd4000d45f0 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.278+0000 7fd4053b9640 1 --2- 172.21.15.1:0/3730349003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d45f0 0x7fd4000b3de0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd4000d1bf0 tx=0x7fd3f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154386e1b38afacb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.278+0000 7fd3ff7fe640 1 -- 172.21.15.1:0/3730349003 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0037070 con 0x7fd4000d45f0 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.278+0000 7fd3ff7fe640 1 -- 172.21.15.1:0/3730349003 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0032440 con 0x7fd4000d45f0 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.278+0000 7fd3ff7fe640 1 -- 172.21.15.1:0/3730349003 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0032810 con 0x7fd4000d45f0 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 -- 172.21.15.1:0/3730349003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d45f0 msgr2=0x7fd4000b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 --2- 172.21.15.1:0/3730349003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d45f0 0x7fd4000b3de0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd4000d1bf0 tx=0x7fd3f002a270 comp rx=0 tx=0).stop 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 -- 172.21.15.1:0/3730349003 shutdown_connections 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 --2- 172.21.15.1:0/3730349003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d45f0 0x7fd4000b3de0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 --2- 172.21.15.1:0/3730349003 >> v2:172.21.15.6:3300/0 conn(0x7fd40015e190 0x7fd40015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 -- 172.21.15.1:0/3730349003 >> 172.21.15.1:0/3730349003 conn(0x7fd400073340 msgr2=0x7fd40009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 -- 172.21.15.1:0/3730349003 shutdown_connections 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.279+0000 7fd406e1c640 1 -- 172.21.15.1:0/3730349003 wait complete. 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.280+0000 7fd406e1c640 1 Processor -- start 2024-05-09T06:38:52.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.280+0000 7fd406e1c640 1 -- start start 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4000d45f0 0x7fd4000b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd40015e190 0x7fd4000b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4000b56d0 con 0x7fd4000d1c70 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd400161c60 con 0x7fd4000d45f0 2024-05-09T06:38:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd406e1c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd4000b5850 con 0x7fd40015e190 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60330/0 (socket says 172.21.15.1:60330) 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 -- 172.21.15.1:0/166282 learned_addr learned my addr 172.21.15.1:0/166282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd3fffff640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4000d45f0 0x7fd4000b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 -- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd40015e190 msgr2=0x7fd4000b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd40015e190 0x7fd4000b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 -- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4000d45f0 msgr2=0x7fd4000b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4000d45f0 0x7fd4000b8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 -- 172.21.15.1:0/166282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4000cc130 con 0x7fd4000d1c70 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd404bb8640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800efc0 tx=0x7fd3f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.281+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8018020 con 0x7fd4000d1c70 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.282+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8014480 con 0x7fd4000d1c70 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.282+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8014850 con 0x7fd4000d1c70 2024-05-09T06:38:52.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.282+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4000cc410 con 0x7fd4000d1c70 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.282+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4000cc9d0 con 0x7fd4000d1c70 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.283+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4000cce10 con 0x7fd4000d1c70 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.283+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3f8005d10 con 0x7fd4000d1c70 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.283+0000 7fd3fd7fa640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.284+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fd3f808ba40 con 0x7fd4000d1c70 2024-05-09T06:38:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.284+0000 7fd3fffff640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.284+0000 7fd3fffff640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 0x7fd3e006e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8007fd0 tx=0x7fd3e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.288+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3f8022070 con 0x7fd4000d1c70 2024-05-09T06:38:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.469+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd4000752b0 con 0x7fd4000d1c70 2024-05-09T06:38:52.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.470+0000 7fd3fd7fa640 1 -- 172.21.15.1:0/166282 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7fd3f808be20 con 0x7fd4000d1c70 2024-05-09T06:38:52.450 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:38:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.471+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 msgr2=0x7fd3e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.471+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 0x7fd3e006e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8007fd0 tx=0x7fd3e8009040 comp rx=0 tx=0).stop 2024-05-09T06:38:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.471+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 msgr2=0x7fd4000ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.471+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800efc0 tx=0x7fd3f800c490 comp rx=0 tx=0).stop 2024-05-09T06:38:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 shutdown_connections 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd40015e190 0x7fd4000b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4000d45f0 0x7fd4000b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 --2- 172.21.15.1:0/166282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4000d1c70 0x7fd4000ca0d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 >> 172.21.15.1:0/166282 conn(0x7fd400073340 msgr2=0x7fd40006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 shutdown_connections 2024-05-09T06:38:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.472+0000 7fd406e1c640 1 -- 172.21.15.1:0/166282 wait complete. 2024-05-09T06:38:52.463 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:38:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.587+0000 7fe271722640 1 Processor -- start 2024-05-09T06:38:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.587+0000 7fe271722640 1 -- start start 2024-05-09T06:38:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe271722640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c15e5f0 0x7fe26c0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe271722640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe26c0b2f30 0x7fe26c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe271722640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c15e9d0 con 0x7fe26c0b53a0 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe271722640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c0b3480 con 0x7fe26c0b2f30 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe271722640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c0d12a0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe26b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe26c0b2f30 0x7fe26c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe26a7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe26c0b53a0 0x7fe26c0d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51370/0 (socket says 172.21.15.1:51370) 2024-05-09T06:38:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe26a7fc640 1 -- 172.21.15.1:0/3526115522 learned_addr learned my addr 172.21.15.1:0/3526115522 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe26b7fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe26c0b2f30 0x7fe26c0b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60340/0 (socket says 172.21.15.1:60340) 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.588+0000 7fe26affd640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c15e5f0 0x7fe26c0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe26b7fe640 1 -- 172.21.15.1:0/3526115522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c15e5f0 msgr2=0x7fe26c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe26b7fe640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c15e5f0 0x7fe26c0d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe26b7fe640 1 -- 172.21.15.1:0/3526115522 >> v1:172.21.15.1:6789/0 conn(0x7fe26c0b53a0 legacy=0x7fe26c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe26b7fe640 1 -- 172.21.15.1:0/3526115522 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c0d14f0 con 0x7fe26c0b2f30 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe269ffb640 1 -- 172.21.15.1:0/3526115522 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2153891019 0 0) 0x7fe26c15e9d0 con 0x7fe26c0b53a0 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe26b7fe640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c0b2f30 0x7fe26c0b3910 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7ffaec2e68b086 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.589+0000 7fe269ffb640 1 -- 172.21.15.1:0/3526115522 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c037070 con 0x7fe26c0b2f30 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe269ffb640 1 -- 172.21.15.1:0/3526115522 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c032440 con 0x7fe26c0b2f30 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe269ffb640 1 -- 172.21.15.1:0/3526115522 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c032810 con 0x7fe26c0b2f30 2024-05-09T06:38:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 -- 172.21.15.1:0/3526115522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c0b2f30 msgr2=0x7fe26c0b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c0b2f30 0x7fe26c0b3910 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097a0 tx=0x7fe25c02a180 comp rx=0 tx=0).stop 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 -- 172.21.15.1:0/3526115522 shutdown_connections 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c0b2f30 0x7fe26c0b3910 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 --2- 172.21.15.1:0/3526115522 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c15e5f0 0x7fe26c0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 -- 172.21.15.1:0/3526115522 >> 172.21.15.1:0/3526115522 conn(0x7fe26c0d3e90 msgr2=0x7fe26c0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 -- 172.21.15.1:0/3526115522 shutdown_connections 2024-05-09T06:38:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.590+0000 7fe271722640 1 -- 172.21.15.1:0/3526115522 wait complete. 2024-05-09T06:38:52.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.591+0000 7fe271722640 1 Processor -- start 2024-05-09T06:38:52.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.591+0000 7fe271722640 1 -- start start 2024-05-09T06:38:52.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c0b2f30 0x7fe26c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 0x7fe26c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0cfde0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0cfb00 con 0x7fe26c0b53a0 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe271722640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c0b3e50 con 0x7fe26c0b2f30 2024-05-09T06:38:52.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe26b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe26b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60346/0 (socket says 172.21.15.1:60346) 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe26b7fe640 1 -- 172.21.15.1:0/1688049347 learned_addr learned my addr 172.21.15.1:0/1688049347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe26affd640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c0b2f30 0x7fe26c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.592+0000 7fe26a7fc640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 0x7fe26c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 -- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c0b2f30 msgr2=0x7fe26c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c0b2f30 0x7fe26c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 -- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 msgr2=0x7fe26c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 0x7fe26c0cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 -- 172.21.15.1:0/1688049347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1f8060 con 0x7fe26c15e5f0 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26a7fc640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 0x7fe26c0cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe26b7fe640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fe25c035040 tx=0x7fe25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c037040 con 0x7fe26c15e5f0 2024-05-09T06:38:52.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c042070 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c0b6be0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0308a0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.593+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c0b71a0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.594+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0752b0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.594+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe25c030a40 con 0x7fe26c15e5f0 2024-05-09T06:38:52.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.595+0000 7fe2637fe640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 0x7fe24c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:52.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.595+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fe25c0ab7e0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.595+0000 7fe26affd640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 0x7fe24c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:52.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.595+0000 7fe26affd640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 0x7fe24c06e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe264011340 tx=0x7fe26400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:52.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.599+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe25c07a090 con 0x7fe26c15e5f0 2024-05-09T06:38:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.796+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe26c0d16f0 con 0x7fe26c15e5f0 2024-05-09T06:38:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.796+0000 7fe2637fe640 1 -- 172.21.15.1:0/1688049347 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7fe25c079990 con 0x7fe26c15e5f0 2024-05-09T06:38:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 msgr2=0x7fe24c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 0x7fe24c06e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe264011340 tx=0x7fe26400b040 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 msgr2=0x7fe26c0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fe25c035040 tx=0x7fe25c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 shutdown_connections 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe24c06be30 0x7fe24c06e2f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe26c15e5f0 0x7fe26c0cbfc0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c0b53a0 0x7fe26c0cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 --2- 172.21.15.1:0/1688049347 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c0b2f30 0x7fe26c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 >> 172.21.15.1:0/1688049347 conn(0x7fe26c0d3e90 msgr2=0x7fe26c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 shutdown_connections 2024-05-09T06:38:52.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:52.798+0000 7fe271722640 1 -- 172.21.15.1:0/1688049347 wait complete. 2024-05-09T06:38:52.790 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:38:52.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:52.790 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:52.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[81233]: fuse finished with error 0 and tester_r 0 2024-05-09T06:38:52.817 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:38:52.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:38:52.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:52.884 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:38:52.900 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:38:52.900 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:38:52.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:38:52.956 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi001.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:52.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:38:52.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:52.957 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:38:53.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:38:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T06:38:53.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:38:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.177+0000 7fdeee163640 1 Processor -- start 2024-05-09T06:38:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.177+0000 7fdeee163640 1 -- start start 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdeee163640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee8071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdeee163640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdee815ece0 0x7fdee815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdeee163640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee80d04a0 con 0x7fdee80cbb60 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdeee163640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdee815f690 con 0x7fdee815ece0 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdeee163640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80cbfe0 con 0x7fdee80b0080 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdee77fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee8071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdee7fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdee815ece0 0x7fdee815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdedffff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdee80cbb60 0x7fdee815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:54140/0 (socket says 172.21.15.1:54140) 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.178+0000 7fdedffff640 1 -- 172.21.15.1:0/796934009 learned_addr learned my addr 172.21.15.1:0/796934009 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee77fe640 1 -- 172.21.15.1:0/796934009 >> v2:172.21.15.6:3300/0 conn(0x7fdee815ece0 msgr2=0x7fdee815f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee77fe640 1 --2- 172.21.15.1:0/796934009 >> v2:172.21.15.6:3300/0 conn(0x7fdee815ece0 0x7fdee815f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee77fe640 1 -- 172.21.15.1:0/796934009 >> v1:172.21.15.6:6789/0 conn(0x7fdee80cbb60 legacy=0x7fdee815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee77fe640 1 -- 172.21.15.1:0/796934009 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80722c0 con 0x7fdee80b0080 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee77fe640 1 --2- 172.21.15.1:0/796934009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee8071ce0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fded80097a0 tx=0x7fded802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b05678093870a03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.179+0000 7fdee6ffd640 1 -- 172.21.15.1:0/796934009 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8037070 con 0x7fdee80b0080 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdee6ffd640 1 -- 172.21.15.1:0/796934009 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded8032440 con 0x7fdee80b0080 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdee6ffd640 1 -- 172.21.15.1:0/796934009 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8032810 con 0x7fdee80b0080 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 -- 172.21.15.1:0/796934009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 msgr2=0x7fdee8071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 --2- 172.21.15.1:0/796934009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee8071ce0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fded80097a0 tx=0x7fded802a180 comp rx=0 tx=0).stop 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 -- 172.21.15.1:0/796934009 shutdown_connections 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 --2- 172.21.15.1:0/796934009 >> v2:172.21.15.6:3300/0 conn(0x7fdee815ece0 0x7fdee815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 --2- 172.21.15.1:0/796934009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee8071ce0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 -- 172.21.15.1:0/796934009 >> 172.21.15.1:0/796934009 conn(0x7fdee80d0b50 msgr2=0x7fdee80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 -- 172.21.15.1:0/796934009 shutdown_connections 2024-05-09T06:38:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.180+0000 7fdeee163640 1 -- 172.21.15.1:0/796934009 wait complete. 2024-05-09T06:38:53.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.181+0000 7fdeee163640 1 Processor -- start 2024-05-09T06:38:53.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.181+0000 7fdeee163640 1 -- start start 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdee80cbb60 0x7fdee80b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8072b60 con 0x7fdee815ece0 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8072880 con 0x7fdee80cbb60 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdeee163640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdee815e7c0 con 0x7fdee80b0080 2024-05-09T06:38:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdee7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdee77fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdee7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60360/0 (socket says 172.21.15.1:60360) 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdee7fff640 1 -- 172.21.15.1:0/1751821222 learned_addr learned my addr 172.21.15.1:0/1751821222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.182+0000 7fdedffff640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdee80cbb60 0x7fdee80b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 -- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdee80cbb60 msgr2=0x7fdee80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdee80cbb60 0x7fdee80b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 -- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 msgr2=0x7fdee8152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 -- 172.21.15.1:0/1751821222 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80b4070 con 0x7fdee80b0080 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee7fff640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee77fe640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee806dee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdee8074a70 tx=0x7fded80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8041020 con 0x7fdee80b0080 2024-05-09T06:38:53.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded8030700 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8037040 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee80b4350 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.183+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee80b4910 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.184+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee80752b0 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.185+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fded8044070 con 0x7fdee80b0080 2024-05-09T06:38:53.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.185+0000 7fdee4ff9640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.185+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fded80aa860 con 0x7fdee80b0080 2024-05-09T06:38:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.185+0000 7fdedffff640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.186+0000 7fdedffff640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 0x7fdec806e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdee806fa10 tx=0x7fded0007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.191+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fded8078a30 con 0x7fdee80b0080 2024-05-09T06:38:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.410+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdee80c7250 con 0x7fdee80b0080 2024-05-09T06:38:53.392 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:38:53.412+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:38:53.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.419+0000 7fdee4ff9640 1 -- 172.21.15.1:0/1751821222 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7fded807cd30 con 0x7fdee80b0080 2024-05-09T06:38:53.399 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:38:53.400 INFO:tasks.ceph.mds.a.smithi001.stderr: -7703> 2024-05-09T06:38:34.719+0000 7f955ae29640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:38:53.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 msgr2=0x7fdec806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 0x7fdec806e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdee806fa10 tx=0x7fded0007e80 comp rx=0 tx=0).stop 2024-05-09T06:38:53.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 msgr2=0x7fdee806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee806dee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdee8074a70 tx=0x7fded80046c0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 shutdown_connections 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee815ece0 0x7fdee8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdee80cbb60 0x7fdee80b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 --2- 172.21.15.1:0/1751821222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdee80b0080 0x7fdee806dee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 >> 172.21.15.1:0/1751821222 conn(0x7fdee80d0b50 msgr2=0x7fdee80d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 shutdown_connections 2024-05-09T06:38:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.421+0000 7fdeee163640 1 -- 172.21.15.1:0/1751821222 wait complete. 2024-05-09T06:38:53.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:38:53.416 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:38:53.434 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.461 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:38:53.462 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.463 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.464 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:38:53.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T06:38:53.466 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.467 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.468 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.469 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = "" 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T06:38:53.470 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = True 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = False 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.471 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-09T06:38:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-09T06:38:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-09T06:38:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:38:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-09T06:38:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:38:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: mon down mkfs grace = 300 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:38:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2024-05-09T06:38:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = True 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = False 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:38:53.526 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.569 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T06:38:53.570 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T06:38:53.572 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.573 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T06:38:53.574 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.575 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:38:53.576 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T06:38:53.577 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = "" 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:38:53.578 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = False 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = False 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.579 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-09T06:38:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:38:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-09T06:38:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-09T06:38:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:38:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon down mkfs grace = 300 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-09T06:38:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2024-05-09T06:38:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = False 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = False 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:38:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:38:53.592 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 Processor -- start 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 -- start start 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f7fef10 0x56532f8a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f81b700 0x56532f810b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56532f817670 con 0x56532f8a4540 2024-05-09T06:38:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56532f776c80 con 0x56532f81b700 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.697+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56532f8165c0 con 0x56532f7fef10 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60404/0 (socket says 172.21.15.1:60404) 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 -- 172.21.15.1:0/3873644569 learned_addr learned my addr 172.21.15.1:0/3873644569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c60cd640 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f81b700 0x56532f810b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c68ce640 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f7fef10 0x56532f8a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 -- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f7fef10 msgr2=0x56532f8a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f7fef10 0x56532f8a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 -- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f81b700 msgr2=0x56532f810b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f81b700 0x56532f810b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 -- 172.21.15.1:0/3873644569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f6fbcb0 con 0x56532f8a4540 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0045f80 tx=0x7fc2c0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d351f5e60692ee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c50cb640 1 -- 172.21.15.1:0/3873644569 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0053890 con 0x56532f8a4540 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c50cb640 1 -- 172.21.15.1:0/3873644569 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0053a30 con 0x56532f8a4540 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.698+0000 7fc2c50cb640 1 -- 172.21.15.1:0/3873644569 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0053e20 con 0x56532f8a4540 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/3873644569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 msgr2=0x56532f811070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0045f80 tx=0x7fc2c0051000 comp rx=0 tx=0).stop 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/3873644569 shutdown_connections 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f811070 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f81b700 0x56532f810b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/3873644569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f7fef10 0x56532f8a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/3873644569 >> 172.21.15.1:0/3873644569 conn(0x56532f7cdb00 msgr2=0x56532f7cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/3873644569 shutdown_connections 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/3873644569 wait complete. 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 Processor -- start 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.699+0000 7fc2c8dcdf40 1 -- start start 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f7fef10 0x56532f89fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f81b700 0x56532f7cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f7f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56532f811c50 con 0x56532f8a4540 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56532f7cfa80 con 0x56532f7fef10 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c8dcdf40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56532f811f30 con 0x56532f81b700 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c70cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f7f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c70cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f7f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60410/0 (socket says 172.21.15.1:60410) 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c70cf640 1 -- 172.21.15.1:0/750708888 learned_addr learned my addr 172.21.15.1:0/750708888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c60cd640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f81b700 0x56532f7cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.700+0000 7fc2c68ce640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f7fef10 0x56532f89fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 -- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f81b700 msgr2=0x56532f7cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56532f81b700 0x56532f7cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 -- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f7fef10 msgr2=0x56532f89fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56532f7fef10 0x56532f89fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 -- 172.21.15.1:0/750708888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f810e40 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c70cf640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f7f6a40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0045a90 tx=0x7fc2c0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2affff640 1 -- 172.21.15.1:0/750708888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0053890 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2affff640 1 -- 172.21.15.1:0/750708888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0053be0 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56532f7fc580 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2affff640 1 -- 172.21.15.1:0/750708888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c006d400 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.701+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532f8110e0 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.702+0000 7fc2affff640 1 -- 172.21.15.1:0/750708888 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2c006d5a0 con 0x56532f8a4540 2024-05-09T06:38:53.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2affff640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2b406a840 0x7fc2b406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2affff640 1 -- 172.21.15.1:0/750708888 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fc2c00ceba0 con 0x56532f8a4540 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2c68ce640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2b406a840 0x7fc2b406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56532f7d03c0 0x56532f940260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x56532f8a0250 con 0x56532f7d03c0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2c60cd640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56532f7d03c0 0x56532f940260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.703+0000 7fc2c68ce640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2b406a840 0x7fc2b406cd00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80097e0 tx=0x7fc2b800e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.704+0000 7fc2c60cd640 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56532f7d03c0 0x56532f940260 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c60cd640 1 -- 172.21.15.1:0/750708888 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v122'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc2bc00f480 con 0x56532f7d03c0 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56532f7d03c0 msgr2=0x56532f940260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56532f7d03c0 0x56532f940260 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2b406a840 msgr2=0x7fc2b406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc2b406a840 0x7fc2b406cd00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80097e0 tx=0x7fc2b800e040 comp rx=0 tx=0).stop 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 msgr2=0x56532f7f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 --2- 172.21.15.1:0/750708888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56532f8a4540 0x56532f7f6a40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0045a90 tx=0x7fc2c0003330 comp rx=0 tx=0).stop 2024-05-09T06:38:53.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c70cf640 1 -- 172.21.15.1:0/750708888 reap_dead start 2024-05-09T06:38:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 shutdown_connections 2024-05-09T06:38:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 >> 172.21.15.1:0/750708888 conn(0x56532f7cdb00 msgr2=0x56532f7cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 shutdown_connections 2024-05-09T06:38:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.705+0000 7fc2c8dcdf40 1 -- 172.21.15.1:0/750708888 wait complete. 2024-05-09T06:38:53.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:38:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.825+0000 7ff56b54f640 1 Processor -- start 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.825+0000 7ff56b54f640 1 -- start start 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff564158a50 0x7ff5640718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 0x7ff564071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff56406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff564158e30 con 0x7ff564158a50 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff564072480 con 0x7ff56415c4a0 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff56b54f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff56415b420 con 0x7ff56406e6d0 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff564158a50 0x7ff5640718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff564158a50 0x7ff5640718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60418/0 (socket says 172.21.15.1:60418) 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 -- 172.21.15.1:0/1346585099 learned_addr learned my addr 172.21.15.1:0/1346585099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff569aec640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff56406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff568aea640 1 --2- 172.21.15.1:0/1346585099 >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 0x7ff564071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 -- 172.21.15.1:0/1346585099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 msgr2=0x7ff56406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff56406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 -- 172.21.15.1:0/1346585099 >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 msgr2=0x7ff564071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1346585099 >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 0x7ff564071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff5692eb640 1 -- 172.21.15.1:0/1346585099 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56415b6e0 con 0x7ff564158a50 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.826+0000 7ff568aea640 1 --2- 172.21.15.1:0/1346585099 >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 0x7ff564071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.827+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640718c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff564068a10 tx=0x7ff55c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8524d61dd6395124 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.827+0000 7ff563fff640 1 -- 172.21.15.1:0/1346585099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c032790 con 0x7ff564158a50 2024-05-09T06:38:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.827+0000 7ff563fff640 1 -- 172.21.15.1:0/1346585099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c038070 con 0x7ff564158a50 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.827+0000 7ff563fff640 1 -- 172.21.15.1:0/1346585099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c032b50 con 0x7ff564158a50 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 -- 172.21.15.1:0/1346585099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 msgr2=0x7ff5640718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640718c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff564068a10 tx=0x7ff55c02c890 comp rx=0 tx=0).stop 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 -- 172.21.15.1:0/1346585099 shutdown_connections 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff56406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1346585099 >> v2:172.21.15.6:3300/0 conn(0x7ff56415c4a0 0x7ff564071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1346585099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640718c0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 -- 172.21.15.1:0/1346585099 >> 172.21.15.1:0/1346585099 conn(0x7ff5640cc4d0 msgr2=0x7ff5640cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 -- 172.21.15.1:0/1346585099 shutdown_connections 2024-05-09T06:38:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.828+0000 7ff56b54f640 1 -- 172.21.15.1:0/1346585099 wait complete. 2024-05-09T06:38:53.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.829+0000 7ff56b54f640 1 Processor -- start 2024-05-09T06:38:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.829+0000 7ff56b54f640 1 -- start start 2024-05-09T06:38:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff564150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff56415c4a0 0x7ff5641577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff56415bf80 con 0x7ff564158a50 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff56415bca0 con 0x7ff56415c4a0 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff56b54f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff564070ea0 con 0x7ff56406e6d0 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60420/0 (socket says 172.21.15.1:60420) 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 -- 172.21.15.1:0/1749982508 learned_addr learned my addr 172.21.15.1:0/1749982508 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff569aec640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff56415c4a0 0x7ff5641577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff564150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 -- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 msgr2=0x7ff564150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff564150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 -- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff56415c4a0 msgr2=0x7ff5641577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff56415c4a0 0x7ff5641577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.830+0000 7ff568aea640 1 -- 172.21.15.1:0/1749982508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5641f0530 con 0x7ff564158a50 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff568aea640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff55800d7d0 tx=0x7ff55800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558004030 con 0x7ff564158a50 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5580041d0 con 0x7ff564158a50 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5641f2fa0 con 0x7ff564158a50 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5580096a0 con 0x7ff564158a50 2024-05-09T06:38:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.831+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5641f3560 con 0x7ff564158a50 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.832+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5640752b0 con 0x7ff564158a50 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.832+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff558009840 con 0x7ff564158a50 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.833+0000 7ff561ffb640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 0x7ff55006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.833+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7ff55808b090 con 0x7ff564158a50 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.833+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 0x7ff55006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.834+0000 7ff5692eb640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 0x7ff55006e320 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff55c037000 tx=0x7ff55c03e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:53.838+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5580592a0 con 0x7ff564158a50 2024-05-09T06:38:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.058+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff56406e1c0 con 0x7ff564158a50 2024-05-09T06:38:54.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.415+0000 7ff561ffb640 1 -- 172.21.15.1:0/1749982508 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) ==== 162+0+0 (secure 0 0 0) 0x7ff55805d5a0 con 0x7ff564158a50 2024-05-09T06:38:54.395 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 msgr2=0x7ff55006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 0x7ff55006e320 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff55c037000 tx=0x7ff55c03e040 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 msgr2=0x7ff5640b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff55800d7d0 tx=0x7ff55800dc90 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 shutdown_connections 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff55006be60 0x7ff55006e320 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff56415c4a0 0x7ff5641577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff564158a50 0x7ff5640b3ec0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 --2- 172.21.15.1:0/1749982508 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff56406e6d0 0x7ff564150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 >> 172.21.15.1:0/1749982508 conn(0x7ff5640cc4d0 msgr2=0x7ff5640cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 shutdown_connections 2024-05-09T06:38:54.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.418+0000 7ff56b54f640 1 -- 172.21.15.1:0/1749982508 wait complete. 2024-05-09T06:38:54.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.537+0000 7f963f415640 1 Processor -- start 2024-05-09T06:38:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.537+0000 7f963f415640 1 -- start start 2024-05-09T06:38:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963f415640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96380cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963f415640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96380ce0e0 con 0x7f963806e6d0 2024-05-09T06:38:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963f415640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96380ce3c0 con 0x7f96380cdab0 2024-05-09T06:38:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963f415640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f96380ce6a0 con 0x7f96380cf180 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963d1b1640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f963806e6d0 0x7f963806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:51400/0 (socket says 172.21.15.1:51400) 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963d1b1640 1 -- 172.21.15.1:0/461078869 learned_addr learned my addr 172.21.15.1:0/461078869 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f963c9b0640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96380cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933169982 0 0) 0x7f96380ce0e0 con 0x7f963806e6d0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.538+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9624003520 con 0x7f963806e6d0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4264259021 0 0) 0x7f9624003520 con 0x7f963806e6d0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 msgr2=0x7f96380cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96380cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 >> v1:172.21.15.6:6789/0 conn(0x7f96380cdab0 legacy=0x7f96380b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96380ce980 con 0x7f963806e6d0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f96340019f0 con 0x7f963806e6d0 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 >> v1:172.21.15.1:6789/0 conn(0x7f963806e6d0 legacy=0x7f963806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9624003f60 0x7f9624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9624004d40 0x7f96240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96380ce6a0 con 0x7f9624003f60 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f962400baa0 con 0x7f9624004d40 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f962400bca0 con 0x7f96380cf180 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9624003f60 0x7f9624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963d1b1640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963d9b2640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9624004d40 0x7f96240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 -- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 msgr2=0x7f96240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96380cf180 0x7f96240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 -- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9624004d40 msgr2=0x7f96240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9624004d40 0x7f96240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.539+0000 7f963c9b0640 1 -- 172.21.15.1:0/461078869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9624003520 con 0x7f9624003f60 2024-05-09T06:38:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.540+0000 7f963c9b0640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9624003f60 0x7f9624004800 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9628009ac0 tx=0x7f9624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8aa92c94a6ea76c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.540+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628036070 con 0x7f9624003f60 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.540+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628031400 con 0x7f9624003f60 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.540+0000 7f962ffff640 1 -- 172.21.15.1:0/461078869 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628009690 con 0x7f9624003f60 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 -- 172.21.15.1:0/461078869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9624003f60 msgr2=0x7f9624004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 --2- 172.21.15.1:0/461078869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9624003f60 0x7f9624004800 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9628009ac0 tx=0x7f9624005830 comp rx=0 tx=0).stop 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963d9b2640 1 -- 172.21.15.1:0/461078869 reap_dead start 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 -- 172.21.15.1:0/461078869 shutdown_connections 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 -- 172.21.15.1:0/461078869 >> 172.21.15.1:0/461078869 conn(0x7f96380cc4b0 msgr2=0x7f96380cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 -- 172.21.15.1:0/461078869 shutdown_connections 2024-05-09T06:38:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.541+0000 7f963f415640 1 -- 172.21.15.1:0/461078869 wait complete. 2024-05-09T06:38:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.542+0000 7f963f415640 1 Processor -- start 2024-05-09T06:38:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.542+0000 7f963f415640 1 -- start start 2024-05-09T06:38:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9624003f60 0x7f96380aa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 0x7f9638134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96381447c0 con 0x7f963815c3b0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9638070ed0 con 0x7f9638133de0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963f415640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f9638144940 con 0x7f9624003f60 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60428/0 (socket says 172.21.15.1:60428) 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 -- 172.21.15.1:0/3018970874 learned_addr learned my addr 172.21.15.1:0/3018970874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963d1b1640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9624003f60 0x7f96380aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 -- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9624003f60 msgr2=0x7f96380aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9624003f60 0x7f96380aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 -- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 msgr2=0x7f9638134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963d9b2640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 0x7f9638134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 0x7f9638134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 -- 172.21.15.1:0/3018970874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963815c830 con 0x7f963815c3b0 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963d9b2640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 0x7f9638134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.543+0000 7f963c9b0640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f9628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.544+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628040020 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.544+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628003c60 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.544+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963815cab0 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.544+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628036040 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.544+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963815d070 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.545+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96380752b0 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.546+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9628043020 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.546+0000 7f962dffb640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 0x7f961c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.546+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f96280ab220 con 0x7f963815c3b0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.546+0000 7f963d1b1640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 0x7f961c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.547+0000 7f963d1b1640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 0x7f961c06e290 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9634002230 tx=0x7f9634006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.551+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f962804a070 con 0x7f963815c3b0 2024-05-09T06:38:54.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.771+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963806cf50 con 0x7f963815c3b0 2024-05-09T06:38:54.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.772+0000 7f962dffb640 1 -- 172.21.15.1:0/3018970874 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5169 (secure 0 0 0) 0x7f963815c830 con 0x7f963815c3b0 2024-05-09T06:38:54.752 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:54.752 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:54.419298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9957":{"gid":9957,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6837/3597643567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":3597643567},{"type":"v1","addr":"172.21.15.1:6837","nonce":3597643567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-09T06:38:54.753 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 102 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 msgr2=0x7f961c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 0x7f961c06e290 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9634002230 tx=0x7f9634006040 comp rx=0 tx=0).stop 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 msgr2=0x7f96380aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f9628004730 comp rx=0 tx=0).stop 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 shutdown_connections 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.773+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f961c06bdd0 0x7f961c06e290 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9638133de0 0x7f9638134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963815c3b0 0x7f96380aaf10 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 --2- 172.21.15.1:0/3018970874 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9624003f60 0x7f96380aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:54.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 >> 172.21.15.1:0/3018970874 conn(0x7f96380cc4b0 msgr2=0x7f96380afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:54.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 shutdown_connections 2024-05-09T06:38:54.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:54.774+0000 7f963f415640 1 -- 172.21.15.1:0/3018970874 wait complete. 2024-05-09T06:38:55.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:38:54.419298+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9957}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9957': {'gid': 9957, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6837/3597643567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 3597643567}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 3597643567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:38:55.168 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9957:up:replay 2024-05-09T06:38:56.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:56.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.305+0000 7f45b2e37640 1 Processor -- start 2024-05-09T06:38:56.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.305+0000 7f45b2e37640 1 -- start start 2024-05-09T06:38:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b2e37640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f45ac15db10 0x7f45ac0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b2e37640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b2e37640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ac070f40 con 0x7f45ac15e500 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b2e37640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f45ac0cf170 con 0x7f45ac15db10 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b2e37640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac0cf450 con 0x7f45ac0715d0 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b0bd3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f45ac15db10 0x7f45ac0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45abfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b13d4640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f45ac15e500 0x7f45ac06e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:54172/0 (socket says 172.21.15.1:54172) 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.306+0000 7f45b13d4640 1 -- 172.21.15.1:0/36564639 learned_addr learned my addr 172.21.15.1:0/36564639 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.307+0000 7f45b0bd3640 1 -- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 msgr2=0x7f45ac15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.307+0000 7f45b0bd3640 1 --2- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.307+0000 7f45b0bd3640 1 -- 172.21.15.1:0/36564639 >> v1:172.21.15.6:6789/0 conn(0x7f45ac15e500 legacy=0x7f45ac06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.307+0000 7f45b0bd3640 1 -- 172.21.15.1:0/36564639 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac070c10 con 0x7f45ac15db10 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b0bd3640 1 --2- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15db10 0x7f45ac0b2a60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f459400aea0 tx=0x7f459402b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2c58d1f648a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45ab7fe640 1 -- 172.21.15.1:0/36564639 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594037070 con 0x7f45ac15db10 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45ab7fe640 1 -- 172.21.15.1:0/36564639 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4594008640 con 0x7f45ac15db10 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45ab7fe640 1 -- 172.21.15.1:0/36564639 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594008a10 con 0x7f45ac15db10 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 -- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15db10 msgr2=0x7f45ac0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 --2- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15db10 0x7f45ac0b2a60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f459400aea0 tx=0x7f459402b7b0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 -- 172.21.15.1:0/36564639 shutdown_connections 2024-05-09T06:38:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 --2- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 --2- 172.21.15.1:0/36564639 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15db10 0x7f45ac0b2a60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 -- 172.21.15.1:0/36564639 >> 172.21.15.1:0/36564639 conn(0x7f45ac0cc4b0 msgr2=0x7f45ac0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 -- 172.21.15.1:0/36564639 shutdown_connections 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.308+0000 7f45b2e37640 1 -- 172.21.15.1:0/36564639 wait complete. 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.310+0000 7f45b2e37640 1 Processor -- start 2024-05-09T06:38:56.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.310+0000 7f45b2e37640 1 -- start start 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15e500 0x7f45ac1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac15edc0 con 0x7f45ac15db10 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac15ec40 con 0x7f45ac15e500 2024-05-09T06:38:56.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b2e37640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac06ea50 con 0x7f45ac0715d0 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60434/0 (socket says 172.21.15.1:60434) 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 -- 172.21.15.1:0/2264550436 learned_addr learned my addr 172.21.15.1:0/2264550436 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b0bd3640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45b13d4640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15e500 0x7f45ac1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 -- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 msgr2=0x7f45ac0b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac0b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 -- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15e500 msgr2=0x7f45ac1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15e500 0x7f45ac1578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 -- 172.21.15.1:0/2264550436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac0b16f0 con 0x7f45ac15db10 2024-05-09T06:38:56.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45abfff640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9e0 tx=0x7f459c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.311+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c017070 con 0x7f45ac15db10 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.312+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f459c00c9d0 con 0x7f45ac15db10 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.312+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac0ad9c0 con 0x7f45ac15db10 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.312+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c00cdc0 con 0x7f45ac15db10 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.312+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac0adf80 con 0x7f45ac15db10 2024-05-09T06:38:56.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.313+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac0752b0 con 0x7f45ac15db10 2024-05-09T06:38:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.313+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f459c01a070 con 0x7f45ac15db10 2024-05-09T06:38:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.314+0000 7f45a97fa640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.314+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f459c08b280 con 0x7f45ac15db10 2024-05-09T06:38:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.314+0000 7f45b0bd3640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:56.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.314+0000 7f45b0bd3640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 0x7f458c06e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4594034000 tx=0x7f459402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:56.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.318+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f459c059490 con 0x7f45ac15db10 2024-05-09T06:38:56.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.544+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45ac0b21f0 con 0x7f45ac15db10 2024-05-09T06:38:56.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.545+0000 7f45a97fa640 1 -- 172.21.15.1:0/2264550436 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5169 (secure 0 0 0) 0x7f459c05d790 con 0x7f45ac15db10 2024-05-09T06:38:56.525 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:56.525 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:55.424653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9957":{"gid":9957,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.1:6837/3597643567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":3597643567},{"type":"v1","addr":"172.21.15.1:6837","nonce":3597643567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-09T06:38:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 104 2024-05-09T06:38:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 msgr2=0x7f458c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 0x7f458c06e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4594034000 tx=0x7f459402f000 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 msgr2=0x7f45ac1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9e0 tx=0x7f459c00eea0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 shutdown_connections 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45ac15e500 0x7f45ac1578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac15db10 0x7f45ac1539f0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 --2- 172.21.15.1:0/2264550436 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45ac0715d0 0x7f45ac0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 >> 172.21.15.1:0/2264550436 conn(0x7f45ac0cc4b0 msgr2=0x7f45ac06e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 shutdown_connections 2024-05-09T06:38:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:56.547+0000 7f45b2e37640 1 -- 172.21.15.1:0/2264550436 wait complete. 2024-05-09T06:38:56.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:38:55.424653+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9957}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9957': {'gid': 9957, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.1:6837/3597643567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 3597643567}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 3597643567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:38:56.942 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9957:up:rejoin 2024-05-09T06:38:57.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:38:58.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.079+0000 7f77ae641640 1 Processor -- start 2024-05-09T06:38:58.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.079+0000 7f77ae641640 1 -- start start 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77ae641640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a806e6d0 0x7f77a806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77ae641640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f77a80cf180 0x7f77a80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77ae641640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a80ce0e0 con 0x7f77a80cdab0 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77ae641640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f77a80cdec0 con 0x7f77a806e6d0 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77ae641640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f77a80ce3c0 con 0x7f77a80cf180 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a806e6d0 0x7f77a806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f77a806e6d0 0x7f77a806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50532/0 (socket says 172.21.15.1:50532) 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 -- 172.21.15.1:0/3007601553 learned_addr learned my addr 172.21.15.1:0/3007601553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a77fe640 1 --2- 172.21.15.1:0/3007601553 >> v2:172.21.15.6:3300/0 conn(0x7f77a80cf180 0x7f77a80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 -- 172.21.15.1:0/3007601553 >> v2:172.21.15.6:3300/0 conn(0x7f77a80cf180 msgr2=0x7f77a80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 --2- 172.21.15.1:0/3007601553 >> v2:172.21.15.6:3300/0 conn(0x7f77a80cf180 0x7f77a80cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 -- 172.21.15.1:0/3007601553 >> v1:172.21.15.6:6789/0 conn(0x7f77a80cdab0 legacy=0x7f77a815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.080+0000 7f77a7fff640 1 -- 172.21.15.1:0/3007601553 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a80ce6a0 con 0x7f77a806e6d0 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.081+0000 7f77a7fff640 1 --2- 172.21.15.1:0/3007601553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a806e6d0 0x7f77a806eab0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f77a80b27c0 tx=0x7f779802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac3fc5706cae709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.081+0000 7f77a6ffd640 1 -- 172.21.15.1:0/3007601553 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798037070 con 0x7f77a806e6d0 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.081+0000 7f77a6ffd640 1 -- 172.21.15.1:0/3007601553 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7798032440 con 0x7f77a806e6d0 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.081+0000 7f77a6ffd640 1 -- 172.21.15.1:0/3007601553 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798032810 con 0x7f77a806e6d0 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 -- 172.21.15.1:0/3007601553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a806e6d0 msgr2=0x7f77a806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 --2- 172.21.15.1:0/3007601553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a806e6d0 0x7f77a806eab0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f77a80b27c0 tx=0x7f779802a230 comp rx=0 tx=0).stop 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 -- 172.21.15.1:0/3007601553 shutdown_connections 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 --2- 172.21.15.1:0/3007601553 >> v2:172.21.15.6:3300/0 conn(0x7f77a80cf180 0x7f77a80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 --2- 172.21.15.1:0/3007601553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a806e6d0 0x7f77a806eab0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 -- 172.21.15.1:0/3007601553 >> 172.21.15.1:0/3007601553 conn(0x7f77a80cc4b0 msgr2=0x7f77a80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 -- 172.21.15.1:0/3007601553 shutdown_connections 2024-05-09T06:38:58.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.082+0000 7f77ae641640 1 -- 172.21.15.1:0/3007601553 wait complete. 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.083+0000 7f77ae641640 1 Processor -- start 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.083+0000 7f77ae641640 1 -- start start 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 0x7f77a815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 0x7f77a80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77a806eff0 con 0x7f77a80cf180 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77a80cec60 con 0x7f77a806e6d0 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77ae641640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f77a8070ed0 con 0x7f77a80cdab0 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50534/0 (socket says 172.21.15.1:50534) 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 -- 172.21.15.1:0/2423119245 learned_addr learned my addr 172.21.15.1:0/2423119245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77a7fff640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 0x7f77a815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77a77fe640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 0x7f77a80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 -- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 msgr2=0x7f77a80ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 0x7f77a80ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 -- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 msgr2=0x7f77a815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 0x7f77a815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77acbde640 1 -- 172.21.15.1:0/2423119245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a80ac6b0 con 0x7f77a80cf180 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77a7fff640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 0x7f77a815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.084+0000 7f77a77fe640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 0x7f77a80ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77acbde640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f77a000c780 tx=0x7f77a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0004b60 con 0x7f77a80cf180 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a0004d00 con 0x7f77a80cf180 2024-05-09T06:38:58.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a80ac990 con 0x7f77a80cf180 2024-05-09T06:38:58.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a000aa90 con 0x7f77a80cf180 2024-05-09T06:38:58.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.085+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a80acf50 con 0x7f77a80cf180 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.086+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a80752b0 con 0x7f77a80cf180 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.087+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77a000ac30 con 0x7f77a80cf180 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.087+0000 7f77a4ff9640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 0x7f779406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.087+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f77a008b540 con 0x7f77a80cf180 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.087+0000 7f77a7fff640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 0x7f779406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.088+0000 7f77a7fff640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 0x7f779406e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7798034000 tx=0x7f779802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.092+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77a0059da0 con 0x7f77a80cf180 2024-05-09T06:38:58.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.315+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77a806cf50 con 0x7f77a80cf180 2024-05-09T06:38:58.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.316+0000 7f77a4ff9640 1 -- 172.21.15.1:0/2423119245 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5176 (secure 0 0 0) 0x7f77a00596a0 con 0x7f77a80cf180 2024-05-09T06:38:58.296 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:38:58.296 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:57.427211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9957":{"gid":9957,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/3597643567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":3597643567},{"type":"v1","addr":"172.21.15.1:6837","nonce":3597643567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9957,"qdb_cluster":[9957]},"id":8}]} 2024-05-09T06:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 105 2024-05-09T06:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 msgr2=0x7f779406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 0x7f779406e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7798034000 tx=0x7f779802e000 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 msgr2=0x7f77a815fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f77a000c780 tx=0x7f77a000cc40 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 shutdown_connections 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f779406be30 0x7f779406e2f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a80cf180 0x7f77a815fa40 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f77a80cdab0 0x7f77a80ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 --2- 172.21.15.1:0/2423119245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77a806e6d0 0x7f77a815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 >> 172.21.15.1:0/2423119245 conn(0x7f77a80cc4b0 msgr2=0x7f77a813a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 shutdown_connections 2024-05-09T06:38:58.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.318+0000 7f77ae641640 1 -- 172.21.15.1:0/2423119245 wait complete. 2024-05-09T06:38:58.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:38:57.427211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9957}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9957': {'gid': 9957, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.1:6837/3597643567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 3597643567}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 3597643567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9957, 'qdb_cluster': [9957]} 2024-05-09T06:38:58.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:38:58.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:38:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.840+0000 7f6ca1693640 1 Processor -- start 2024-05-09T06:38:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.840+0000 7f6ca1693640 1 -- start start 2024-05-09T06:38:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6ca1693640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c9c15e1d0 0x7f6c9c06f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6ca1693640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6ca1693640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9c06c2f0 con 0x7f6c9c0d2890 2024-05-09T06:38:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6ca1693640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6c9c0cc3c0 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6ca1693640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c0ccc60 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6c9affd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c9c15e1d0 0x7f6c9c06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.841+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:34294/0 (socket says 172.21.15.1:34294) 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1096584925 learned_addr learned my addr 172.21.15.1:0/1096584925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9b7fe640 1 --1- 172.21.15.1:0/1096584925 >> v1:172.21.15.6:6789/0 conn(0x7f6c9c0d2890 0x7f6c9c0cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:46346/0 (socket says 172.21.15.1:46346) 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9affd640 1 -- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 msgr2=0x7f6c9c06fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9affd640 1 -- 172.21.15.1:0/1096584925 >> v1:172.21.15.6:6789/0 conn(0x7f6c9c0d2890 legacy=0x7f6c9c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c99ffb640 1 -- 172.21.15.1:0/1096584925 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472481 0 0) 0x7f6c94001110 con 0x7f6c9c0d2890 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9affd640 1 -- 172.21.15.1:0/1096584925 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c06f500 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.842+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c15e1d0 0x7f6c9c06f890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009dc0 tx=0x7f6c8c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4e9cbcbcf1dca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6c99ffb640 1 -- 172.21.15.1:0/1096584925 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c007740 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6c99ffb640 1 -- 172.21.15.1:0/1096584925 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0078e0 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6c99ffb640 1 -- 172.21.15.1:0/1096584925 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c007cb0 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 -- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c15e1d0 msgr2=0x7f6c9c06f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c15e1d0 0x7f6c9c06f890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009dc0 tx=0x7f6c8c02a5f0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 -- 172.21.15.1:0/1096584925 shutdown_connections 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e5b0 0x7f6c9c06fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 --2- 172.21.15.1:0/1096584925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c15e1d0 0x7f6c9c06f890 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 -- 172.21.15.1:0/1096584925 >> 172.21.15.1:0/1096584925 conn(0x7f6c9c0d1d70 msgr2=0x7f6c9c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 -- 172.21.15.1:0/1096584925 shutdown_connections 2024-05-09T06:38:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.843+0000 7f6ca1693640 1 -- 172.21.15.1:0/1096584925 wait complete. 2024-05-09T06:38:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.845+0000 7f6ca1693640 1 Processor -- start 2024-05-09T06:38:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.845+0000 7f6ca1693640 1 -- start start 2024-05-09T06:38:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 0x7f6c9c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e1d0 0x7f6c9c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c06f1b0 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c0cc8f0 con 0x7f6c9c0d2890 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c0d0fc0 con 0x7f6c9c15e1d0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50536/0 (socket says 172.21.15.1:50536) 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/4274280236 learned_addr learned my addr 172.21.15.1:0/4274280236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e1d0 0x7f6c9c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9affd640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 0x7f6c9c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e1d0 msgr2=0x7f6c9c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e1d0 0x7f6c9c1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 msgr2=0x7f6c9c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 0x7f6c9c0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/4274280236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c157b30 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9affd640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 0x7f6c9c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c750 tx=0x7f6c9400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c837fe640 1 -- 172.21.15.1:0/4274280236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94018070 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c157e10 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c837fe640 1 -- 172.21.15.1:0/4274280236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c940070d0 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6c837fe640 1 -- 172.21.15.1:0/4274280236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c940074c0 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.846+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c1583d0 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.848+0000 7f6c837fe640 1 -- 172.21.15.1:0/4274280236 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c940047e0 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.848+0000 7f6c837fe640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 0x7f6c8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.849+0000 7f6c837fe640 1 -- 172.21.15.1:0/4274280236 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f6c9408a360 con 0x7f6c9c15e5b0 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.849+0000 7f6c9affd640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 0x7f6c8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.849+0000 7f6c9affd640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 0x7f6c8806e2f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c034000 tx=0x7f6c8c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.849+0000 7f6ca439f740 1 Processor -- start 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 -- start start 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9912a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9930460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ac984b4d0 con 0x560ac98e8350 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ac97f33e0 con 0x560ac992ff10 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ac9847490 con 0x560ac9912670 2024-05-09T06:38:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6c9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50548/0 (socket says 172.21.15.1:50548) 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.850+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/877136018 learned_addr learned my addr 172.21.15.1:0/877136018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9affd640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9912a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9930460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 msgr2=0x560ac9912a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9912a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 msgr2=0x560ac9930460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9930460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 -- 172.21.15.1:0/877136018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac955f2d0 con 0x560ac98e8350 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9930460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c94058350 tx=0x7f6c9405c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fae99b9412320be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c80ff9640 1 -- 172.21.15.1:0/877136018 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94056460 con 0x560ac98e8350 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c80ff9640 1 -- 172.21.15.1:0/877136018 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c940566c0 con 0x560ac98e8350 2024-05-09T06:38:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.851+0000 7f6c80ff9640 1 -- 172.21.15.1:0/877136018 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94004ad0 con 0x560ac98e8350 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- 172.21.15.1:0/877136018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 msgr2=0x560ac992e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c94058350 tx=0x7f6c9405c6c0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- 172.21.15.1:0/877136018 shutdown_connections 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac98e8350 0x560ac992e5d0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9930460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 --2- 172.21.15.1:0/877136018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9912a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- 172.21.15.1:0/877136018 >> 172.21.15.1:0/877136018 conn(0x560ac98e4c80 msgr2=0x560ac992d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- 172.21.15.1:0/877136018 shutdown_connections 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- 172.21.15.1:0/877136018 wait complete. 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.852+0000 7f6ca439f740 1 -- start start 2024-05-09T06:38:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 Processor -- start 2024-05-09T06:38:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9913330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac99402b0 0x560ac9915ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ac98e90f0 con 0x560ac992ff10 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ac98068e0 con 0x560ac99402b0 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ac98e2bb0 con 0x560ac9912670 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6ca439f740 10 client.0 _ll_register_callbacks cb 0x560ac97d39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50564/0 (socket says 172.21.15.1:50564) 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.853+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1451887860 learned_addr learned my addr 172.21.15.1:0/1451887860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac99402b0 0x560ac9915ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9913330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 msgr2=0x560ac9913330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9913330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac99402b0 msgr2=0x560ac9915ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac99402b0 0x560ac9915ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1451887860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac96296b0 con 0x560ac992ff10 2024-05-09T06:38:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9913330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009e90 tx=0x7f6c900136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cee1cbbf5c16d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9001b070 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f6c90004020 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c940566c0 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f6c90025050 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6ca439f740 10 client.9999 fetch_fsmap learned FSMap version 105 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x560ac99147e0 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c940566c0 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.854+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f6c90028020 con 0x560ac992ff10 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 10 client.9999 fetch_fsmap finished waiting for FSMap version 105 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 10 client.9999 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 20 client.9999 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac9924a00 0x560ac993d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 4 client.9999 mds_command: new command op to 9957 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 --> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ac9858c00 con 0x560ac9924a00 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac9924a00 0x560ac993d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6c79ffb640 10 client.9999 ms_handle_connect on v2:172.21.15.1:6836/3597643567 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.855+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac9924a00 0x560ac993d900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2126f03d82318f95 server_cookie=e80da11d8d666b31 in_seq=0 out_seq=0 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1451887860 <== mds.0 v2:172.21.15.1:6836/3597643567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ac9858c00 con 0x560ac9924a00 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6c79ffb640 10 client.9999 handle_command_reply: tid=0 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 client.9999 shutdown 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 msgr2=0x560ac99159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009e90 tx=0x7f6c900136b0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 shutdown_connections 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac9924a00 0x560ac993d900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac99402b0 0x560ac9915ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac99159b0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6c79ffb640 0 client.9999 ms_handle_reset on v2:172.21.15.1:6836/3597643567 2024-05-09T06:38:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1451887860 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac9912670 0x560ac9913330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 >> 172.21.15.1:0/1451887860 conn(0x560ac98e8350 msgr2=0x560ac9931290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 shutdown_connections 2024-05-09T06:38:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 1 -- 172.21.15.1:0/1451887860 wait complete. 2024-05-09T06:38:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.856+0000 7f6ca439f740 20 client.9999 trim_cache size 0 max 16384 2024-05-09T06:38:58.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.874+0000 7f6ca439f740 1 Processor -- start 2024-05-09T06:38:58.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.874+0000 7f6ca439f740 1 -- start start 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9925950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9925e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1ffe70 con 0x560ac9912670 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c159a80 con 0x560ac992ff10 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c1ff7e0 con 0x560ac99402b0 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50576/0 (socket says 172.21.15.1:50576) 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 -- 172.21.15.1:0/16885423 learned_addr learned my addr 172.21.15.1:0/16885423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 -- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 msgr2=0x560ac9925e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9925e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 -- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 msgr2=0x560ac9925950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9925950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 -- 172.21.15.1:0/16885423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac98e4140 con 0x560ac9912670 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c9affd640 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02a630 tx=0x7f6c8c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dba34dcc0e17390 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.875+0000 7f6c7bfff640 1 -- 172.21.15.1:0/16885423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c052b50 con 0x560ac9912670 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6c7bfff640 1 -- 172.21.15.1:0/16885423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c052cf0 con 0x560ac9912670 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6c7bfff640 1 -- 172.21.15.1:0/16885423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0524d0 con 0x560ac9912670 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 -- 172.21.15.1:0/16885423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 msgr2=0x560ac9915940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02a630 tx=0x7f6c8c030000 comp rx=0 tx=0).stop 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 -- 172.21.15.1:0/16885423 shutdown_connections 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9925e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac992ff10 0x560ac9925950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 --2- 172.21.15.1:0/16885423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac9912670 0x560ac9915940 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 -- 172.21.15.1:0/16885423 >> 172.21.15.1:0/16885423 conn(0x560ac98e8350 msgr2=0x560ac992e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 -- 172.21.15.1:0/16885423 shutdown_connections 2024-05-09T06:38:58.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.876+0000 7f6ca439f740 1 -- 172.21.15.1:0/16885423 wait complete. 2024-05-09T06:38:58.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.877+0000 7f6ca439f740 1 -- start start 2024-05-09T06:38:58.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.877+0000 7f6ca439f740 1 Processor -- start 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.877+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 0x560ac99a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.877+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9915940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ac9913a00 con 0x560ac992ff10 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x560ac993e3d0 con 0x560ac9912670 2024-05-09T06:38:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x560ac9924e30 con 0x560ac99402b0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 10 client.0 _ll_register_callbacks cb 0x7f6c900093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50580/0 (socket says 172.21.15.1:50580) 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1997720727 learned_addr learned my addr 172.21.15.1:0/1997720727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 0x560ac99a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9b7fe640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9915940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 msgr2=0x560ac9915940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9915940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 msgr2=0x560ac99a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 0x560ac99a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 -- 172.21.15.1:0/1997720727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac99265b0 con 0x560ac992ff10 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 0x560ac99a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c9a7fc640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6c90008010 tx=0x7f6c90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaa8f08fa82b986 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9002d070 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f6c90016460 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8c052cf0 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f6c90038050 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 10 client.10005 fetch_fsmap learned FSMap version 105 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.878+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x560ac99a84f0 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c052cf0 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f6c9003b020 con 0x560ac992ff10 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 10 client.10005 fetch_fsmap finished waiting for FSMap version 105 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 10 client.10005 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 20 client.10005 populate_metadata read hostname 'smithi001' 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac98e6120 0x560ac98e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 4 client.10005 mds_command: new command op to 9957 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 --> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] -- command(tid 0: {"prefix": "status"}) -- 0x560ac9852f50 con 0x560ac98e6120 2024-05-09T06:38:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac98e6120 0x560ac98e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c9affd640 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac98e6120 0x560ac98e6500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3424ff6e0167d895 server_cookie=5b2d505c0b39b2b7 in_seq=0 out_seq=0 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c79ffb640 10 client.10005 ms_handle_connect on v2:172.21.15.1:6836/3597643567 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c79ffb640 1 -- 172.21.15.1:0/1997720727 <== mds.0 v2:172.21.15.1:6836/3597643567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x560ac9852f50 con 0x560ac98e6120 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6c79ffb640 10 client.10005 handle_command_reply: tid=0 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 1 client.10005 shutdown 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 msgr2=0x560ac990e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.879+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6c90008010 tx=0x7f6c90004730 comp rx=0 tx=0).stop 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 shutdown_connections 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x560ac98e6120 0x560ac98e6500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x560ac99402b0 0x560ac9915940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ac992ff10 0x560ac990e370 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 --2- 172.21.15.1:0/1997720727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560ac9912670 0x560ac99a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6c79ffb640 0 client.10005 ms_handle_reset on v2:172.21.15.1:6836/3597643567 2024-05-09T06:38:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 >> 172.21.15.1:0/1997720727 conn(0x560ac98e8350 msgr2=0x560ac98e7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 shutdown_connections 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 1 -- 172.21.15.1:0/1997720727 wait complete. 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca439f740 20 client.10005 trim_cache size 0 max 16384 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 9957, 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4572073750000003, 2024-05-09T06:38:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 105, 2024-05-09T06:38:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 122, 2024-05-09T06:38:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 122, 2024-05-09T06:38:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 5.4240440850000002 2024-05-09T06:38:58.868 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 msgr2=0x7f6c8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 0x7f6c8806e2f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c034000 tx=0x7f6c8c02f000 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 msgr2=0x7f6c9c09f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c750 tx=0x7f6c9400cc10 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 shutdown_connections 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f6c8806be30 0x7f6c8806e2f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c9c15e5b0 0x7f6c9c09f3d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c9c15e1d0 0x7f6c9c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 --2- 172.21.15.1:0/4274280236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9c0d2890 0x7f6c9c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 >> 172.21.15.1:0/4274280236 conn(0x7f6c9c0d1d70 msgr2=0x7f6c9c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 shutdown_connections 2024-05-09T06:38:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.880+0000 7f6ca1693640 1 -- 172.21.15.1:0/4274280236 wait complete. 2024-05-09T06:38:58.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:38:58.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.995+0000 7fee23e53640 1 Processor -- start 2024-05-09T06:38:58.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.995+0000 7fee23e53640 1 -- start start 2024-05-09T06:38:58.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee23e53640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fee1c15c680 0x7fee1c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee23e53640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1c15cc40 con 0x7fee1c0b0170 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee23e53640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1c15cf20 con 0x7fee1c1565e0 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee23e53640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fee1c15fd30 con 0x7fee1c15c680 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fee1c15c680 0x7fee1c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fee1c15c680 0x7fee1c0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50584/0 (socket says 172.21.15.1:50584) 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 -- 172.21.15.1:0/3459969712 learned_addr learned my addr 172.21.15.1:0/3459969712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 -- 172.21.15.1:0/3459969712 >> v1:172.21.15.6:6789/0 conn(0x7fee1c1565e0 legacy=0x7fee1c074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 -- 172.21.15.1:0/3459969712 >> v1:172.21.15.1:6789/0 conn(0x7fee1c0b0170 legacy=0x7fee1c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee20bed640 1 -- 172.21.15.1:0/3459969712 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330229853 0 0) 0x7fee1c15cc40 con 0x7fee1c0b0170 2024-05-09T06:38:58.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.996+0000 7fee213ee640 1 -- 172.21.15.1:0/3459969712 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c15d200 con 0x7fee1c15c680 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.997+0000 7fee213ee640 1 --2- 172.21.15.1:0/3459969712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0afc30 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009870 tx=0x7fee0c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe9924b7f68df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.997+0000 7fee20bed640 1 -- 172.21.15.1:0/3459969712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c037070 con 0x7fee1c15c680 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.997+0000 7fee20bed640 1 -- 172.21.15.1:0/3459969712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c02ae50 con 0x7fee1c15c680 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.997+0000 7fee20bed640 1 -- 172.21.15.1:0/3459969712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c032860 con 0x7fee1c15c680 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 -- 172.21.15.1:0/3459969712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 msgr2=0x7fee1c0afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 --2- 172.21.15.1:0/3459969712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0afc30 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009870 tx=0x7fee0c02a300 comp rx=0 tx=0).stop 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 -- 172.21.15.1:0/3459969712 shutdown_connections 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 --2- 172.21.15.1:0/3459969712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0afc30 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 -- 172.21.15.1:0/3459969712 >> 172.21.15.1:0/3459969712 conn(0x7fee1c0b1fb0 msgr2=0x7fee1c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 -- 172.21.15.1:0/3459969712 shutdown_connections 2024-05-09T06:38:58.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.998+0000 7fee23e53640 1 -- 172.21.15.1:0/3459969712 wait complete. 2024-05-09T06:38:58.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.999+0000 7fee23e53640 1 Processor -- start 2024-05-09T06:38:58.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:58.999+0000 7fee23e53640 1 -- start start 2024-05-09T06:38:58.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c0b0170 0x7fee1c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fee1c1565e0 0x7fee1c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c1608d0 con 0x7fee1c15c680 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c1569c0 con 0x7fee1c0b0170 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee23e53640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fee1c160a50 con 0x7fee1c1565e0 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50600/0 (socket says 172.21.15.1:50600) 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 -- 172.21.15.1:0/1567407227 learned_addr learned my addr 172.21.15.1:0/1567407227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee21bef640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c0b0170 0x7fee1c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 -- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fee1c1565e0 msgr2=0x7fee1c152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fee1c1565e0 0x7fee1c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 -- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c0b0170 msgr2=0x7fee1c0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c0b0170 0x7fee1c0ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 -- 172.21.15.1:0/1567407227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c159480 con 0x7fee1c15c680 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.000+0000 7fee223f0640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c760 tx=0x7fee1400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.001+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14003da0 con 0x7fee1c15c680 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.001+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee14005070 con 0x7fee1c15c680 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.001+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14005460 con 0x7fee1c15c680 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.001+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c159760 con 0x7fee1c15c680 2024-05-09T06:38:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.001+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c153000 con 0x7fee1c15c680 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.002+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c0752b0 con 0x7fee1c15c680 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.002+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee14005600 con 0x7fee1c15c680 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.003+0000 7fee127fc640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 0x7fee0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.003+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fee1408af50 con 0x7fee1c15c680 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.003+0000 7fee21bef640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 0x7fee0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.003+0000 7fee21bef640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 0x7fee0006e2f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fee18000f60 tx=0x7fee18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:58.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.007+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fee140597b0 con 0x7fee1c15c680 2024-05-09T06:38:59.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.229+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fee1c0cd7c0 con 0x7fee1c15c680 2024-05-09T06:38:59.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.229+0000 7fee127fc640 1 -- 172.21.15.1:0/1567407227 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7fee140590b0 con 0x7fee1c15c680 2024-05-09T06:38:59.209 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:38:59.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 msgr2=0x7fee0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 0x7fee0006e2f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fee18000f60 tx=0x7fee18009040 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 msgr2=0x7fee1c0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c760 tx=0x7fee1400cc20 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 shutdown_connections 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fee0006be30 0x7fee0006e2f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee1c15c680 0x7fee1c0b1250 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fee1c1565e0 0x7fee1c152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 --2- 172.21.15.1:0/1567407227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c0b0170 0x7fee1c0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 >> 172.21.15.1:0/1567407227 conn(0x7fee1c0b1fb0 msgr2=0x7fee1c1601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 shutdown_connections 2024-05-09T06:38:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59.231+0000 7fee23e53640 1 -- 172.21.15.1:0/1567407227 wait complete. 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:59.222 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:38:59.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:38:59.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:38:59.223 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:59.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.223 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:38:59.240 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:38:59.240 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:38:59.240 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:38:59.240 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.240 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42003sec preferred_lft 42003sec 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:38:59.241 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:38:59.242 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:59.243 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:38:59.243 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:38:59.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.243 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:38:59.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.307 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:38:59.364 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:38:59.364 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:38:59.364 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:38:59.364 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:38:59.364 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:38:59.365 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:38:59.404 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:38:59.424+0000 7f70f2c49640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:38:59.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:38:59.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:38:59.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.494 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:38:59.510 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:38:59.510 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:38:59.510 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:59.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:38:59.576 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:38:59.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:59.577 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:59.632 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:38:59.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:38:59.632 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:38:59.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:59.755 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:59.812 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:59.812 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:59.812 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:59.812 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:59.813 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:59.814 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:59.815 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:59.815 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:59.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.815 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:59.868 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:38:59.868 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:38:59.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:38:59.936 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:38:59.993 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:38:59.993 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:38:59.994 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:38:59.996 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:59.996 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:38:59.996 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:59.996 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:38:59.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:38:59.996 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:38:59.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:39:00.015+0000 7f560102f480 -1 init, newargv = 0x55ba774f3470 newargc=15 2024-05-09T06:38:59.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[82661]: starting ceph client 2024-05-09T06:39:00.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[82661]: starting fuse 2024-05-09T06:39:00.012 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:39:00.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:39:00.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:39:00.013 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:39:00.014 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:00.133 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.82661.asok 2024-05-09T06:39:00.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:00.145 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82661.asok 2024-05-09T06:39:00.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:00.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82661.asok status 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:39:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "82661", 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 10017, 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 10017 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:39:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3596972483 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3596972483 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.10017 192.168.0.1:0/3596972483", 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/3596972483", 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 105, 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 122, 2024-05-09T06:39:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:39:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:39:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:39:00.287 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:00.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:00.292 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:39:00.292 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:00.309 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:39:00.309 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:00.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:00.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:00.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:00.375 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-09T06:39:00.440 INFO:teuthology.orchestra.run.smithi001.stdout:./ 2024-05-09T06:39:00.440 INFO:teuthology.orchestra.run.smithi001.stdout:./parent 2024-05-09T06:39:00.441 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed 2024-05-09T06:39:00.441 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/charlie 2024-05-09T06:39:00.441 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/alpha 2024-05-09T06:39:00.441 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/unflushed 2024-05-09T06:39:00.441 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/unflushed/jfile 2024-05-09T06:39:00.441 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:00.442 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.442 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:00.442 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.442 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-09T06:39:00.442 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:00.549 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:39:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:00.558 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:00.558 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.558 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:00.558 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.559 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-09T06:39:00.559 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:00.666 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:39:00.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:00.676 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:00.676 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.676 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:00.676 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.677 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-09T06:39:00.677 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:00.785 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:39:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:00.793 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:00.794 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.794 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:00.794 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.794 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-09T06:39:00.794 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:00.901 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628280 2024-05-09T06:39:00.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:00.910 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:00.911 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:00.911 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:00.911 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:00.911 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-09T06:39:00.911 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:01.020 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627778 2024-05-09T06:39:01.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-09T06:39:01.029 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:01.137 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627779 2024-05-09T06:39:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-09T06:39:01.146 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:01.254 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627780 2024-05-09T06:39:01.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:01.263 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:01.263 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:01.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:01.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.484+0000 7fd6bc722640 1 Processor -- start 2024-05-09T06:39:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.484+0000 7fd6bc722640 1 -- start start 2024-05-09T06:39:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bc722640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd6b406e6d0 0x7fd6b406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bc722640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40ce0e0 con 0x7fd6b40b3300 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bc722640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40ce3c0 con 0x7fd6b40cf180 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bc722640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b40ce6a0 con 0x7fd6b406e6d0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6ba4be640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd6b406e6d0 0x7fd6b406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bacbf640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd6b40b3300 0x7fd6b40b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44608/0 (socket says 172.21.15.1:44608) 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.485+0000 7fd6bacbf640 1 -- 172.21.15.1:0/1853799880 learned_addr learned my addr 172.21.15.1:0/1853799880 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6ba4be640 1 -- 172.21.15.1:0/1853799880 >> v1:172.21.15.6:6789/0 conn(0x7fd6b40cf180 legacy=0x7fd6b40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6ba4be640 1 -- 172.21.15.1:0/1853799880 >> v1:172.21.15.1:6789/0 conn(0x7fd6b40b3300 legacy=0x7fd6b40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6ba4be640 1 -- 172.21.15.1:0/1853799880 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b40ce980 con 0x7fd6b406e6d0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6ba4be640 1 --2- 172.21.15.1:0/1853799880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b406e6d0 0x7fd6b406eab0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097a0 tx=0x7fd6b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9bb4e95e9624f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6b94bc640 1 -- 172.21.15.1:0/1853799880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0037070 con 0x7fd6b406e6d0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6b94bc640 1 -- 172.21.15.1:0/1853799880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0032440 con 0x7fd6b406e6d0 2024-05-09T06:39:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.486+0000 7fd6b94bc640 1 -- 172.21.15.1:0/1853799880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0032810 con 0x7fd6b406e6d0 2024-05-09T06:39:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 -- 172.21.15.1:0/1853799880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b406e6d0 msgr2=0x7fd6b406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 --2- 172.21.15.1:0/1853799880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b406e6d0 0x7fd6b406eab0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097a0 tx=0x7fd6b002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 -- 172.21.15.1:0/1853799880 shutdown_connections 2024-05-09T06:39:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 --2- 172.21.15.1:0/1853799880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b406e6d0 0x7fd6b406eab0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 -- 172.21.15.1:0/1853799880 >> 172.21.15.1:0/1853799880 conn(0x7fd6b40cc4b0 msgr2=0x7fd6b40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 -- 172.21.15.1:0/1853799880 shutdown_connections 2024-05-09T06:39:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.487+0000 7fd6bc722640 1 -- 172.21.15.1:0/1853799880 wait complete. 2024-05-09T06:39:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.488+0000 7fd6bc722640 1 Processor -- start 2024-05-09T06:39:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.488+0000 7fd6bc722640 1 -- start start 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6b406e6d0 0x7fd6b4133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6b40cf180 0x7fd6b41313b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4070ed0 con 0x7fd6b40b3300 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b406eff0 con 0x7fd6b40cf180 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6bc722640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b41447c0 con 0x7fd6b406e6d0 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50616/0 (socket says 172.21.15.1:50616) 2024-05-09T06:39:01.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 -- 172.21.15.1:0/2889864385 learned_addr learned my addr 172.21.15.1:0/2889864385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6ba4be640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6b406e6d0 0x7fd6b4133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 -- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6b406e6d0 msgr2=0x7fd6b4133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6b406e6d0 0x7fd6b4133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 -- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6b40cf180 msgr2=0x7fd6b41313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6b40cf180 0x7fd6b41313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.489+0000 7fd6b9cbd640 1 -- 172.21.15.1:0/2889864385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b4134560 con 0x7fd6b40b3300 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6b9cbd640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c400 tx=0x7fd6a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4017070 con 0x7fd6b40b3300 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a40035f0 con 0x7fd6b40b3300 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40039e0 con 0x7fd6b40b3300 2024-05-09T06:39:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b41419d0 con 0x7fd6b40b3300 2024-05-09T06:39:01.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.490+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b4141f90 con 0x7fd6b40b3300 2024-05-09T06:39:01.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.491+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b4142190 con 0x7fd6b40b3300 2024-05-09T06:39:01.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.491+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6a401a070 con 0x7fd6b40b3300 2024-05-09T06:39:01.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.492+0000 7fd6aaffd640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 0x7fd69806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.492+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fd6a408a2d0 con 0x7fd6b40b3300 2024-05-09T06:39:01.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.492+0000 7fd6ba4be640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 0x7fd69806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.492+0000 7fd6ba4be640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 0x7fd69806e2f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034000 tx=0x7fd6b002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.496+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6b4142190 con 0x7fd6b40b3300 2024-05-09T06:39:01.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.681+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6b41417c0 con 0x7fd6b40b3300 2024-05-09T06:39:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.682+0000 7fd6aaffd640 1 -- 172.21.15.1:0/2889864385 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16947 (secure 0 0 0) 0x7fd6a40584e0 con 0x7fd6b40b3300 2024-05-09T06:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":122,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:53.413234+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:01.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.683+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 msgr2=0x7fd69806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.683+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 0x7fd69806e2f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0034000 tx=0x7fd6b002e000 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.683+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 msgr2=0x7fd6b415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.683+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c400 tx=0x7fd6a400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 shutdown_connections 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd69806be30 0x7fd69806e2f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6b40cf180 0x7fd6b41313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b40b3300 0x7fd6b415a750 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 --2- 172.21.15.1:0/2889864385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6b406e6d0 0x7fd6b4133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 >> 172.21.15.1:0/2889864385 conn(0x7fd6b40cc4b0 msgr2=0x7fd6b40b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 shutdown_connections 2024-05-09T06:39:01.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.684+0000 7fd6bc722640 1 -- 172.21.15.1:0/2889864385 wait complete. 2024-05-09T06:39:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:39:01.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:39:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.805+0000 7fc5cf2e1640 1 Processor -- start 2024-05-09T06:39:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.805+0000 7fc5cf2e1640 1 -- start start 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cf2e1640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5c806ede0 0x7fc5c8165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cf2e1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c815b420 con 0x7fc5c8071fd0 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cf2e1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c815b6e0 con 0x7fc5c8158a50 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cf2e1640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c8158e30 con 0x7fc5c806ede0 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cd87e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc5c806ede0 0x7fc5c8165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cc87c640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc5c8071fd0 0x7fc5c806e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44610/0 (socket says 172.21.15.1:44610) 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cc87c640 1 -- 172.21.15.1:0/1160633858 learned_addr learned my addr 172.21.15.1:0/1160633858 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cd87e640 1 -- 172.21.15.1:0/1160633858 >> v1:172.21.15.6:6789/0 conn(0x7fc5c8158a50 legacy=0x7fc5c80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.806+0000 7fc5cd87e640 1 -- 172.21.15.1:0/1160633858 >> v1:172.21.15.1:6789/0 conn(0x7fc5c8071fd0 legacy=0x7fc5c806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.807+0000 7fc5cd87e640 1 -- 172.21.15.1:0/1160633858 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c815b9c0 con 0x7fc5c806ede0 2024-05-09T06:39:01.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.807+0000 7fc5cd87e640 1 --2- 172.21.15.1:0/1160633858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c8165b70 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d90 tx=0x7fc5c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240585f13d23b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.807+0000 7fc5bffff640 1 -- 172.21.15.1:0/1160633858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007740 con 0x7fc5c806ede0 2024-05-09T06:39:01.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.807+0000 7fc5bffff640 1 -- 172.21.15.1:0/1160633858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c00078e0 con 0x7fc5c806ede0 2024-05-09T06:39:01.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.807+0000 7fc5bffff640 1 -- 172.21.15.1:0/1160633858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007cb0 con 0x7fc5c806ede0 2024-05-09T06:39:01.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1160633858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 msgr2=0x7fc5c8165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1160633858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c8165b70 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d90 tx=0x7fc5c002a6f0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1160633858 shutdown_connections 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1160633858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c8165b70 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1160633858 >> 172.21.15.1:0/1160633858 conn(0x7fc5c80cc4d0 msgr2=0x7fc5c80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1160633858 shutdown_connections 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.808+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1160633858 wait complete. 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.809+0000 7fc5cf2e1640 1 Processor -- start 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.809+0000 7fc5cf2e1640 1 -- start start 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5c8071fd0 0x7fc5c814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c8158a50 0x7fc5c8156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c815bf80 con 0x7fc5c806ede0 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8072480 con 0x7fc5c8158a50 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cf2e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c8070ea0 con 0x7fc5c8071fd0 2024-05-09T06:39:01.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50628/0 (socket says 172.21.15.1:50628) 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 -- 172.21.15.1:0/1387446103 learned_addr learned my addr 172.21.15.1:0/1387446103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 -- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5c8071fd0 msgr2=0x7fc5c814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5c8071fd0 0x7fc5c814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 -- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c8158a50 msgr2=0x7fc5c8156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd87e640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c8158a50 0x7fc5c8156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c8158a50 0x7fc5c8156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.810+0000 7fc5cd07d640 1 -- 172.21.15.1:0/1387446103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81560e0 con 0x7fc5c806ede0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5cd07d640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c3c0 tx=0x7fc5c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40052d0 con 0x7fc5c806ede0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c4005470 con 0x7fc5c806ede0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c8157a10 con 0x7fc5c806ede0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4007260 con 0x7fc5c806ede0 2024-05-09T06:39:01.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.811+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c8156350 con 0x7fc5c806ede0 2024-05-09T06:39:01.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.812+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c80752b0 con 0x7fc5c806ede0 2024-05-09T06:39:01.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.812+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5c4007400 con 0x7fc5c806ede0 2024-05-09T06:39:01.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.813+0000 7fc5bdffb640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 0x7fc5ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:01.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.813+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fc5c408a600 con 0x7fc5c806ede0 2024-05-09T06:39:01.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.813+0000 7fc5cc87c640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 0x7fc5ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:01.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.814+0000 7fc5cc87c640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 0x7fc5ac06e320 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005880 tx=0x7fc5b8007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.817+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5c4058d40 con 0x7fc5c806ede0 2024-05-09T06:39:01.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.999+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5c8071a50 con 0x7fc5c806ede0 2024-05-09T06:39:01.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:01.999+0000 7fc5bdffb640 1 -- 172.21.15.1:0/1387446103 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7fc5c4058640 con 0x7fc5c806ede0 2024-05-09T06:39:01.979 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 msgr2=0x7fc5ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 0x7fc5ac06e320 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005880 tx=0x7fc5b8007a00 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 msgr2=0x7fc5c81e4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c3c0 tx=0x7fc5c400c880 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 shutdown_connections 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc5ac06be60 0x7fc5ac06e320 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c8158a50 0x7fc5c8156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc5c8071fd0 0x7fc5c814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 --2- 172.21.15.1:0/1387446103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5c806ede0 0x7fc5c81e4830 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 >> 172.21.15.1:0/1387446103 conn(0x7fc5c80cc4d0 msgr2=0x7fc5c806d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 shutdown_connections 2024-05-09T06:39:01.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.001+0000 7fc5cf2e1640 1 -- 172.21.15.1:0/1387446103 wait complete. 2024-05-09T06:39:01.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:39:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.121+0000 7fb81d9ce640 1 Processor -- start 2024-05-09T06:39:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.121+0000 7fb81d9ce640 1 -- start start 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb81d9ce640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb81806dd00 0x7fb81815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb81d9ce640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81806cf40 con 0x7fb81815cde0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb81d9ce640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81806d220 con 0x7fb81815c4c0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb81d9ce640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb818156980 con 0x7fb81806dd00 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb8177fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb81815cde0 0x7fb818070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44626/0 (socket says 172.21.15.1:44626) 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb8177fe640 1 -- 172.21.15.1:0/4267212442 learned_addr learned my addr 172.21.15.1:0/4267212442 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.122+0000 7fb8167fc640 1 --2- 172.21.15.1:0/4267212442 >> v2:172.21.15.6:3300/0 conn(0x7fb81806dd00 0x7fb81815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209653529 0 0) 0x7fb81806cf40 con 0x7fb81815cde0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb804003520 con 0x7fb81815cde0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148153274 0 0) 0x7fb81806d220 con 0x7fb81815c4c0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb81806cf40 con 0x7fb81815c4c0 2024-05-09T06:39:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3220959871 0 0) 0x7fb804003520 con 0x7fb81815cde0 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 >> v2:172.21.15.6:3300/0 conn(0x7fb81806dd00 msgr2=0x7fb81815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 --2- 172.21.15.1:0/4267212442 >> v2:172.21.15.6:3300/0 conn(0x7fb81806dd00 0x7fb81815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 >> v1:172.21.15.6:6789/0 conn(0x7fb81815c4c0 legacy=0x7fb8180b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81806d500 con 0x7fb81815cde0 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fb810001670 con 0x7fb81815cde0 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8167fc640 1 --2- 172.21.15.1:0/4267212442 >> v2:172.21.15.6:3300/0 conn(0x7fb81806dd00 0x7fb81815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 >> v1:172.21.15.1:6789/0 conn(0x7fb81815cde0 legacy=0x7fb818070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb804003b40 0x7fb804003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb804004800 0x7fb804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb80400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb80400c010 con 0x7fb8040051c0 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb80400c1c0 con 0x7fb804003b40 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb80400c3c0 con 0x7fb804004800 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb80400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8167fc640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb804004800 0x7fb804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb816ffd640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb804003b40 0x7fb804003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 -- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb804004800 msgr2=0x7fb804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb804004800 0x7fb804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 -- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb804003b40 msgr2=0x7fb804003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb804003b40 0x7fb804003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.123+0000 7fb8177fe640 1 -- 172.21.15.1:0/4267212442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb804003520 con 0x7fb8040051c0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.124+0000 7fb8177fe640 1 -- 172.21.15.1:0/4267212442 reap_dead start 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.124+0000 7fb8177fe640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb80400b9c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c160 tx=0x7fb804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a0eac318a5fd34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.124+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810007ca0 con 0x7fb8040051c0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.124+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb810005af0 con 0x7fb8040051c0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.124+0000 7fb815ffb640 1 -- 172.21.15.1:0/4267212442 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810007070 con 0x7fb8040051c0 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 -- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 msgr2=0x7fb80400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb80400b9c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c160 tx=0x7fb804005830 comp rx=0 tx=0).stop 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 -- 172.21.15.1:0/4267212442 shutdown_connections 2024-05-09T06:39:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/4267212442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb80400b9c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 -- 172.21.15.1:0/4267212442 >> 172.21.15.1:0/4267212442 conn(0x7fb8180cc4d0 msgr2=0x7fb8180cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 -- 172.21.15.1:0/4267212442 shutdown_connections 2024-05-09T06:39:02.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.125+0000 7fb81d9ce640 1 -- 172.21.15.1:0/4267212442 wait complete. 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.126+0000 7fb81d9ce640 1 Processor -- start 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.126+0000 7fb81d9ce640 1 -- start start 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8181599e0 0x7fb8181f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81815aff0 0x7fb8180ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb81806d870 con 0x7fb8040051c0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb81806e180 con 0x7fb81815aff0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb81d9ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180b2c40 con 0x7fb8181599e0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50640/0 (socket says 172.21.15.1:50640) 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 -- 172.21.15.1:0/192507424 learned_addr learned my addr 172.21.15.1:0/192507424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb8167fc640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8181599e0 0x7fb8181f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb8177fe640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81815aff0 0x7fb8180ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 -- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8181599e0 msgr2=0x7fb8181f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8181599e0 0x7fb8181f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 -- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81815aff0 msgr2=0x7fb8180ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81815aff0 0x7fb8180ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.127+0000 7fb816ffd640 1 -- 172.21.15.1:0/192507424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb818159f50 con 0x7fb8040051c0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb816ffd640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb80800fb10 tx=0x7fb808002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80801a070 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb808006d60 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81815a230 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808003b90 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.128+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81815a7f0 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.129+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8180752b0 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.129+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb80801d070 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.130+0000 7fb80ffff640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 0x7fb7f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.130+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fb80808aaa0 con 0x7fb8040051c0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.130+0000 7fb8167fc640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 0x7fb7f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.130+0000 7fb8167fc640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 0x7fb7f806e320 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb818144c20 tx=0x7fb800008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.134+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb808059300 con 0x7fb8040051c0 2024-05-09T06:39:02.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.316+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb818155550 con 0x7fb8040051c0 2024-05-09T06:39:02.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.317+0000 7fb80ffff640 1 -- 172.21.15.1:0/192507424 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7fb808058c00 con 0x7fb8040051c0 2024-05-09T06:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 msgr2=0x7fb7f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 0x7fb7f806e320 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb818144c20 tx=0x7fb800008040 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 msgr2=0x7fb8181f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb80800fb10 tx=0x7fb808002170 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 shutdown_connections 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb7f806be60 0x7fb7f806e320 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81815aff0 0x7fb8180ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8181599e0 0x7fb8181f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 --2- 172.21.15.1:0/192507424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8040051c0 0x7fb8181f4be0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 >> 172.21.15.1:0/192507424 conn(0x7fb8180cc4d0 msgr2=0x7fb8180af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 shutdown_connections 2024-05-09T06:39:02.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.319+0000 7fb81d9ce640 1 -- 172.21.15.1:0/192507424 wait complete. 2024-05-09T06:39:02.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:39:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.436+0000 7fa5ef37e640 1 Processor -- start 2024-05-09T06:39:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.437+0000 7fa5ef37e640 1 -- start start 2024-05-09T06:39:02.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.437+0000 7fa5ef37e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5e80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.437+0000 7fa5ef37e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e815e900 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.437+0000 7fa5ef37e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e815ebe0 con 0x7fa5e80cf990 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.437+0000 7fa5ef37e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e815eec0 con 0x7fa5e815e390 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ed91b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa5e80b1900 0x7fa5e80cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44630/0 (socket says 172.21.15.1:44630) 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ed91b640 1 -- 172.21.15.1:0/1944516472 learned_addr learned my addr 172.21.15.1:0/1944516472 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5e80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754556808 0 0) 0x7fa5e815e900 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d0003520 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2335816625 0 0) 0x7fa5d0003520 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 msgr2=0x7fa5e80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5e80cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 >> v1:172.21.15.6:6789/0 conn(0x7fa5e80cf990 legacy=0x7fa5e80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e815f1a0 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fa5d80019f0 con 0x7fa5e80b1900 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 >> v1:172.21.15.1:6789/0 conn(0x7fa5e80b1900 legacy=0x7fa5e80cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d00039e0 0x7fa5d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5d0004d40 0x7fa5d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e815eec0 con 0x7fa5d0004d40 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d000baf0 con 0x7fa5d00039e0 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d000bcf0 con 0x7fa5e815e390 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ed91b640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5d0004d40 0x7fa5d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d00039e0 0x7fa5d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.438+0000 7fa5ec919640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 -- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 msgr2=0x7fa5d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e815e390 0x7fa5d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 -- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d00039e0 msgr2=0x7fa5d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d00039e0 0x7fa5d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 -- 172.21.15.1:0/1944516472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d0003520 con 0x7fa5d0004d40 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5ed91b640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5d0004d40 0x7fa5d00051c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8161cc0 tx=0x7fa5d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65107b5725e2dc0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800a480 con 0x7fa5d0004d40 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.439+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d802f070 con 0x7fa5d0004d40 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5e7fff640 1 -- 172.21.15.1:0/1944516472 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d802bc90 con 0x7fa5d0004d40 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 -- 172.21.15.1:0/1944516472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5d0004d40 msgr2=0x7fa5d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/1944516472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5d0004d40 0x7fa5d00051c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8161cc0 tx=0x7fa5d0005830 comp rx=0 tx=0).stop 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ed91b640 1 -- 172.21.15.1:0/1944516472 reap_dead start 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 -- 172.21.15.1:0/1944516472 shutdown_connections 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 -- 172.21.15.1:0/1944516472 >> 172.21.15.1:0/1944516472 conn(0x7fa5e80b1da0 msgr2=0x7fa5e80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 -- 172.21.15.1:0/1944516472 shutdown_connections 2024-05-09T06:39:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.440+0000 7fa5ef37e640 1 -- 172.21.15.1:0/1944516472 wait complete. 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.441+0000 7fa5ef37e640 1 Processor -- start 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.441+0000 7fa5ef37e640 1 -- start start 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 0x7fa5e81573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e80b4950 0x7fa5e80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e80cffb0 con 0x7fa5e8156a30 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e8162ba0 con 0x7fa5d0004d40 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ef37e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e80cf450 con 0x7fa5e80b4950 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ec919640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ec919640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50662/0 (socket says 172.21.15.1:50662) 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ec919640 1 -- 172.21.15.1:0/2254583064 learned_addr learned my addr 172.21.15.1:0/2254583064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 0x7fa5e81573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ed91b640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e80b4950 0x7fa5e80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.442+0000 7fa5ec919640 1 -- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e80b4950 msgr2=0x7fa5e80b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ec919640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e80b4950 0x7fa5e80b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ec919640 1 -- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 msgr2=0x7fa5e81573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ec919640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 0x7fa5e81573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ec919640 1 -- 172.21.15.1:0/2254583064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e814aa90 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 0x7fa5e81573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ec919640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00f820 tx=0x7fa5dc00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0050d0 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc018070 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e814ad70 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc005490 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.443+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e8159bb0 con 0x7fa5e8156a30 2024-05-09T06:39:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.444+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e80752b0 con 0x7fa5e8156a30 2024-05-09T06:39:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.445+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5dc00d730 con 0x7fa5e8156a30 2024-05-09T06:39:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.445+0000 7fa5e5ffb640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 0x7fa5d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.445+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fa5dc08ae60 con 0x7fa5e8156a30 2024-05-09T06:39:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.445+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 0x7fa5d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.446+0000 7fa5ed11a640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 0x7fa5d406e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8161790 tx=0x7fa5e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.449+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa5dc0596c0 con 0x7fa5e8156a30 2024-05-09T06:39:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.635+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5e80b2ae0 con 0x7fa5e8156a30 2024-05-09T06:39:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.636+0000 7fa5e5ffb640 1 -- 172.21.15.1:0/2254583064 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7fa5dc058fc0 con 0x7fa5e8156a30 2024-05-09T06:39:02.616 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 msgr2=0x7fa5d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 0x7fa5d406e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8161790 tx=0x7fa5e0008040 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 msgr2=0x7fa5e8159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00f820 tx=0x7fa5dc00fce0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 shutdown_connections 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa5d406be30 0x7fa5d406e2f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5e80b4950 0x7fa5e80b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5e8156a30 0x7fa5e8159670 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 --2- 172.21.15.1:0/2254583064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5d0004d40 0x7fa5e81573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 >> 172.21.15.1:0/2254583064 conn(0x7fa5e80b1da0 msgr2=0x7fa5e80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 shutdown_connections 2024-05-09T06:39:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.638+0000 7fa5ef37e640 1 -- 172.21.15.1:0/2254583064 wait complete. 2024-05-09T06:39:02.629 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:39:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.754+0000 7f989de76640 1 Processor -- start 2024-05-09T06:39:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.754+0000 7f989de76640 1 -- start start 2024-05-09T06:39:02.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f989de76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f989806e6d0 0x7f989806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f989de76640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f98980b3300 0x7f98980b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f989de76640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98980ce0e0 con 0x7f98980cf180 2024-05-09T06:39:02.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f989de76640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f98980ce3c0 con 0x7f98980b3300 2024-05-09T06:39:02.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f989de76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f98980ce6a0 con 0x7f989806e6d0 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f9897fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f98980b3300 0x7f98980b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f9897fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f98980b3300 0x7f98980b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50676/0 (socket says 172.21.15.1:50676) 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.755+0000 7f9897fff640 1 -- 172.21.15.1:0/2681284105 learned_addr learned my addr 172.21.15.1:0/2681284105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f98977fe640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f989806e6d0 0x7f989806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f9897fff640 1 -- 172.21.15.1:0/2681284105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f989806e6d0 msgr2=0x7f989806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f9897fff640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f989806e6d0 0x7f989806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f9897fff640 1 -- 172.21.15.1:0/2681284105 >> v1:172.21.15.6:6789/0 conn(0x7f98980cf180 legacy=0x7f98980cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f9897fff640 1 -- 172.21.15.1:0/2681284105 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98980ce980 con 0x7f98980b3300 2024-05-09T06:39:02.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.756+0000 7f9897fff640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98980b3300 0x7f98980b3790 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f989000d150 tx=0x7f989002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26632dafd93575 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f98967fc640 1 -- 172.21.15.1:0/2681284105 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890009600 con 0x7f98980b3300 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f98967fc640 1 -- 172.21.15.1:0/2681284105 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890037070 con 0x7f98980b3300 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f98967fc640 1 -- 172.21.15.1:0/2681284105 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900099c0 con 0x7f98980b3300 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 -- 172.21.15.1:0/2681284105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98980b3300 msgr2=0x7f98980b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98980b3300 0x7f98980b3790 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f989000d150 tx=0x7f989002c930 comp rx=0 tx=0).stop 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 -- 172.21.15.1:0/2681284105 shutdown_connections 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98980b3300 0x7f98980b3790 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 --2- 172.21.15.1:0/2681284105 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f989806e6d0 0x7f989806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 -- 172.21.15.1:0/2681284105 >> 172.21.15.1:0/2681284105 conn(0x7f98980cc4b0 msgr2=0x7f98980cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 -- 172.21.15.1:0/2681284105 shutdown_connections 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.757+0000 7f989de76640 1 -- 172.21.15.1:0/2681284105 wait complete. 2024-05-09T06:39:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.758+0000 7f989de76640 1 Processor -- start 2024-05-09T06:39:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 -- start start 2024-05-09T06:39:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 0x7f9898137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98980cf180 0x7f98981e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9898070ed0 con 0x7f989806e6d0 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f989806eff0 con 0x7f98980b3300 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f989de76640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f98981447c0 con 0x7f98980cf180 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f98977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f98977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50690/0 (socket says 172.21.15.1:50690) 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.759+0000 7f98977fe640 1 -- 172.21.15.1:0/1692497016 learned_addr learned my addr 172.21.15.1:0/1692497016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f9896ffd640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 0x7f9898137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f9897fff640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98980cf180 0x7f98981e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 -- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98980cf180 msgr2=0x7f98981e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98980cf180 0x7f98981e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 -- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 msgr2=0x7f9898137b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 0x7f9898137b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 -- 172.21.15.1:0/1692497016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989813df10 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f9896ffd640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 0x7f9898137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f98977fe640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f988800d830 tx=0x7f988800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800b6c0 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888019070 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800bac0 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989813e1d0 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.760+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989813e790 con 0x7f989806e6d0 2024-05-09T06:39:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.761+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98980752b0 con 0x7f989806e6d0 2024-05-09T06:39:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.762+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f988801e460 con 0x7f989806e6d0 2024-05-09T06:39:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.762+0000 7f987ffff640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 0x7f988406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.762+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f988808b760 con 0x7f989806e6d0 2024-05-09T06:39:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.762+0000 7f9896ffd640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 0x7f988406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.763+0000 7f9896ffd640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 0x7f988406e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f98981441a0 tx=0x7f988c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.767+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9888059f30 con 0x7f989806e6d0 2024-05-09T06:39:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.961+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f989806cf50 con 0x7f989806e6d0 2024-05-09T06:39:02.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.962+0000 7f987ffff640 1 -- 172.21.15.1:0/1692497016 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) ==== 80+0+120 (secure 0 0 0) 0x7f9888059830 con 0x7f989806e6d0 2024-05-09T06:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6837/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6836/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:39:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 msgr2=0x7f988406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 0x7f988406e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f98981441a0 tx=0x7f988c009040 comp rx=0 tx=0).stop 2024-05-09T06:39:02.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 msgr2=0x7f989815a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f988800d830 tx=0x7f988800dcf0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 shutdown_connections 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f988406be30 0x7f988406e2f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98980cf180 0x7f98981e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98980b3300 0x7f9898137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 --2- 172.21.15.1:0/1692497016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f989806e6d0 0x7f989815a320 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 >> 172.21.15.1:0/1692497016 conn(0x7f98980cc4b0 msgr2=0x7f98980af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 shutdown_connections 2024-05-09T06:39:02.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:02.964+0000 7f989de76640 1 -- 172.21.15.1:0/1692497016 wait complete. 2024-05-09T06:39:02.955 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:02.955 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:02.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:03.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.136+0000 7f97b145d640 1 Processor -- start 2024-05-09T06:39:03.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 -- start start 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97ac161950 0x7f97ac0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97ac0d0f80 0x7f97ac0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ac0b33c0 con 0x7f97ac0d4940 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac0d4dd0 con 0x7f97ac161950 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.137+0000 7f97b145d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac0b3660 con 0x7f97ac0d0f80 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97ac161950 0x7f97ac0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aa7fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f97ac0d4940 0x7f97ac0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44634/0 (socket says 172.21.15.1:44634) 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aa7fc640 1 -- 172.21.15.1:0/3105102411 learned_addr learned my addr 172.21.15.1:0/3105102411 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:03.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f97ac161950 0x7f97ac0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50704/0 (socket says 172.21.15.1:50704) 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97ab7fe640 1 --2- 172.21.15.1:0/3105102411 >> v2:172.21.15.6:3300/0 conn(0x7f97ac0d0f80 0x7f97ac0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 -- 172.21.15.1:0/3105102411 >> v2:172.21.15.6:3300/0 conn(0x7f97ac0d0f80 msgr2=0x7f97ac0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 --2- 172.21.15.1:0/3105102411 >> v2:172.21.15.6:3300/0 conn(0x7f97ac0d0f80 0x7f97ac0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 -- 172.21.15.1:0/3105102411 >> v1:172.21.15.1:6789/0 conn(0x7f97ac0d4940 legacy=0x7f97ac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97a9ffb640 1 -- 172.21.15.1:0/3105102411 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941736162 0 0) 0x7f97ac0b33c0 con 0x7f97ac0d4940 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.138+0000 7f97aaffd640 1 -- 172.21.15.1:0/3105102411 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac0b3940 con 0x7f97ac161950 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97aaffd640 1 --2- 172.21.15.1:0/3105102411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac161950 0x7f97ac0d2400 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f97ac08e540 tx=0x7f979c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe4d0dfc70bcb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97a9ffb640 1 -- 172.21.15.1:0/3105102411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c037070 con 0x7f97ac161950 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97a9ffb640 1 -- 172.21.15.1:0/3105102411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c032440 con 0x7f97ac161950 2024-05-09T06:39:03.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97a9ffb640 1 -- 172.21.15.1:0/3105102411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c032810 con 0x7f97ac161950 2024-05-09T06:39:03.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 -- 172.21.15.1:0/3105102411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac161950 msgr2=0x7f97ac0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 --2- 172.21.15.1:0/3105102411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac161950 0x7f97ac0d2400 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f97ac08e540 tx=0x7f979c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 -- 172.21.15.1:0/3105102411 shutdown_connections 2024-05-09T06:39:03.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 --2- 172.21.15.1:0/3105102411 >> v2:172.21.15.6:3300/0 conn(0x7f97ac0d0f80 0x7f97ac0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 --2- 172.21.15.1:0/3105102411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac161950 0x7f97ac0d2400 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 -- 172.21.15.1:0/3105102411 >> 172.21.15.1:0/3105102411 conn(0x7f97ac0d3e20 msgr2=0x7f97ac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.139+0000 7f97b145d640 1 -- 172.21.15.1:0/3105102411 shutdown_connections 2024-05-09T06:39:03.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.140+0000 7f97b145d640 1 -- 172.21.15.1:0/3105102411 wait complete. 2024-05-09T06:39:03.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.141+0000 7f97b145d640 1 Processor -- start 2024-05-09T06:39:03.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.141+0000 7f97b145d640 1 -- start start 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.141+0000 7f97b145d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97ac0d0f80 0x7f97ac15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.141+0000 7f97b145d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97ac161950 0x7f97ac09eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac0b52d0 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac0b5150 con 0x7f97ac161950 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac0b5450 con 0x7f97ac0d0f80 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50706/0 (socket says 172.21.15.1:50706) 2024-05-09T06:39:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 -- 172.21.15.1:0/697506073 learned_addr learned my addr 172.21.15.1:0/697506073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97aaffd640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97ac0d0f80 0x7f97ac15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97aa7fc640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97ac161950 0x7f97ac09eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 -- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97ac0d0f80 msgr2=0x7f97ac15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97ac0d0f80 0x7f97ac15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 -- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97ac161950 msgr2=0x7f97ac09eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97ac161950 0x7f97ac09eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 -- 172.21.15.1:0/697506073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac15dee0 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97ab7fe640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f97a400cd70 tx=0x7f97a400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4003b90 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a4003d30 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac0ccc90 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4004240 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.142+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac0b6e20 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.143+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac0752b0 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.144+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97a40043e0 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.144+0000 7f97937fe640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 0x7f979806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.144+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f97a408b970 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.144+0000 7f97aaffd640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 0x7f979806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.145+0000 7f97aaffd640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 0x7f979806e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.149+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97a4059b80 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.334+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97ac0d21e0 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.335+0000 7f97937fe640 1 -- 172.21.15.1:0/697506073 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16947 (secure 0 0 0) 0x7f97a405de80 con 0x7f97ac0ce9f0 2024-05-09T06:39:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:03.316 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":122,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:38:53.413234+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 msgr2=0x7f979806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 0x7f979806e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 msgr2=0x7f97ac0ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f97a400cd70 tx=0x7f97a400a660 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 shutdown_connections 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f979806be30 0x7f979806e2f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ac0ce9f0 0x7f97ac0ceea0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97ac161950 0x7f97ac09eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 --2- 172.21.15.1:0/697506073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f97ac0d0f80 0x7f97ac15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 >> 172.21.15.1:0/697506073 conn(0x7f97ac0d3e20 msgr2=0x7f97ac0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 shutdown_connections 2024-05-09T06:39:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.337+0000 7f97b145d640 1 -- 172.21.15.1:0/697506073 wait complete. 2024-05-09T06:39:03.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:39:03.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:39:03.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.455+0000 7fa12fe8c640 1 Processor -- start 2024-05-09T06:39:03.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.455+0000 7fa12fe8c640 1 -- start start 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1280d06f0 0x7fa1280cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280c41e0 0x7fa1280ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 0x7fa128161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa1280cd160 con 0x7fa1280d06f0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa128072050 con 0x7fa12806b1e0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12fe8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa128072330 con 0x7fa1280c41e0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1280d06f0 0x7fa1280cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1280d06f0 0x7fa1280cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50712/0 (socket says 172.21.15.1:50712) 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 -- 172.21.15.1:0/3143362778 learned_addr learned my addr 172.21.15.1:0/3143362778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12e429640 1 --2- 172.21.15.1:0/3143362778 >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 0x7fa128161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12d427640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280c41e0 0x7fa1280ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 -- 172.21.15.1:0/3143362778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280c41e0 msgr2=0x7fa1280ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280c41e0 0x7fa1280ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 -- 172.21.15.1:0/3143362778 >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 msgr2=0x7fa128161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 --2- 172.21.15.1:0/3143362778 >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 0x7fa128161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12dc28640 1 -- 172.21.15.1:0/3143362778 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa128072610 con 0x7fa1280d06f0 2024-05-09T06:39:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.456+0000 7fa12e429640 1 --2- 172.21.15.1:0/3143362778 >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 0x7fa128161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.457+0000 7fa12dc28640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1280d06f0 0x7fa1280cc6e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa1280b2690 tx=0x7fa12402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c6de7aaf57b625 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.457+0000 7fa12cc26640 1 -- 172.21.15.1:0/3143362778 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124037070 con 0x7fa1280d06f0 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.457+0000 7fa12cc26640 1 -- 172.21.15.1:0/3143362778 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124032440 con 0x7fa1280d06f0 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.457+0000 7fa12cc26640 1 -- 172.21.15.1:0/3143362778 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124032810 con 0x7fa1280d06f0 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 -- 172.21.15.1:0/3143362778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1280d06f0 msgr2=0x7fa1280cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1280d06f0 0x7fa1280cc6e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa1280b2690 tx=0x7fa12402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 -- 172.21.15.1:0/3143362778 shutdown_connections 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/3143362778 >> v2:172.21.15.6:3300/0 conn(0x7fa12806b1e0 0x7fa128161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280c41e0 0x7fa1280ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/3143362778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1280d06f0 0x7fa1280cc6e0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 -- 172.21.15.1:0/3143362778 >> 172.21.15.1:0/3143362778 conn(0x7fa1280d0b50 msgr2=0x7fa1280d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 -- 172.21.15.1:0/3143362778 shutdown_connections 2024-05-09T06:39:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.458+0000 7fa12fe8c640 1 -- 172.21.15.1:0/3143362778 wait complete. 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.459+0000 7fa12fe8c640 1 Processor -- start 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.459+0000 7fa12fe8c640 1 -- start start 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1280c41e0 0x7fa12815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280d06f0 0x7fa1280c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281620c0 con 0x7fa12806b1e0 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa128071ce0 con 0x7fa1280c41e0 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12fe8c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa128162240 con 0x7fa1280d06f0 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50722/0 (socket says 172.21.15.1:50722) 2024-05-09T06:39:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 -- 172.21.15.1:0/1200147364 learned_addr learned my addr 172.21.15.1:0/1200147364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12d427640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1280c41e0 0x7fa12815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12e429640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280d06f0 0x7fa1280c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 -- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280d06f0 msgr2=0x7fa1280c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280d06f0 0x7fa1280c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 -- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1280c41e0 msgr2=0x7fa12815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1280c41e0 0x7fa12815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.460+0000 7fa12dc28640 1 -- 172.21.15.1:0/1200147364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281609c0 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa12dc28640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa12402a220 tx=0x7fa1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124037040 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124042070 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1280c7dc0 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124030880 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.461+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1280c8380 con 0x7fa12806b1e0 2024-05-09T06:39:03.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.462+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1280752b0 con 0x7fa12806b1e0 2024-05-09T06:39:03.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.462+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa124030a20 con 0x7fa12806b1e0 2024-05-09T06:39:03.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.463+0000 7fa11a7fc640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 0x7fa10c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.463+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fa1240ab890 con 0x7fa12806b1e0 2024-05-09T06:39:03.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.463+0000 7fa12d427640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 0x7fa10c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.463+0000 7fa12d427640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 0x7fa10c06e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa12815d970 tx=0x7fa11c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.467+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa124079980 con 0x7fa12806b1e0 2024-05-09T06:39:03.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.651+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa128072f30 con 0x7fa12806b1e0 2024-05-09T06:39:03.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.651+0000 7fa11a7fc640 1 -- 172.21.15.1:0/1200147364 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7fa12407dc80 con 0x7fa12806b1e0 2024-05-09T06:39:03.634 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 msgr2=0x7fa10c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 0x7fa10c06e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa12815d970 tx=0x7fa11c009190 comp rx=0 tx=0).stop 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 msgr2=0x7fa1280c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa12402a220 tx=0x7fa1240046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 shutdown_connections 2024-05-09T06:39:03.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa10c06be30 0x7fa10c06e2f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1280d06f0 0x7fa1280c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1280c41e0 0x7fa12815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 --2- 172.21.15.1:0/1200147364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa12806b1e0 0x7fa1280c7100 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.653+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 >> 172.21.15.1:0/1200147364 conn(0x7fa1280d0b50 msgr2=0x7fa1280b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.654+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 shutdown_connections 2024-05-09T06:39:03.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.654+0000 7fa12fe8c640 1 -- 172.21.15.1:0/1200147364 wait complete. 2024-05-09T06:39:03.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:39:03.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.770+0000 7fee0069a640 1 Processor -- start 2024-05-09T06:39:03.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.771+0000 7fee0069a640 1 -- start start 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fee0069a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fedf80d45f0 0x7fedf815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fee0069a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf8161fe0 con 0x7fedf80d1c70 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fee0069a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf81622c0 con 0x7fedf815e190 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fee0069a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fedf815f580 con 0x7fedf80d45f0 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfdc35640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fedf80d1c70 0x7fedf80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44650/0 (socket says 172.21.15.1:44650) 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfdc35640 1 -- 172.21.15.1:0/52719958 learned_addr learned my addr 172.21.15.1:0/52719958 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334886492 0 0) 0x7fedf8161fe0 con 0x7fedf80d1c70 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fede4003520 con 0x7fedf80d1c70 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfec37640 1 --2- 172.21.15.1:0/52719958 >> v2:172.21.15.6:3300/0 conn(0x7fedf80d45f0 0x7fedf815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1142385254 0 0) 0x7fede4003520 con 0x7fedf80d1c70 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 >> v2:172.21.15.6:3300/0 conn(0x7fedf80d45f0 msgr2=0x7fedf815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 --2- 172.21.15.1:0/52719958 >> v2:172.21.15.6:3300/0 conn(0x7fedf80d45f0 0x7fedf815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 >> v1:172.21.15.6:6789/0 conn(0x7fedf815e190 legacy=0x7fedf80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf81625a0 con 0x7fedf80d1c70 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fede8001670 con 0x7fedf80d1c70 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 >> v1:172.21.15.1:6789/0 conn(0x7fedf80d1c70 legacy=0x7fedf80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 0x7fede4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4004800 0x7fede4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede4005210 0x7fede400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede400c110 con 0x7fede4003a00 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fede400c2c0 con 0x7fede4005210 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fede400c4c0 con 0x7fede4004800 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfe436640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 0x7fede4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.772+0000 7fedfdc35640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4004800 0x7fede4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfec37640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede4005210 0x7fede400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 -- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4004800 msgr2=0x7fede4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4004800 0x7fede4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 -- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede4005210 msgr2=0x7fede400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede4005210 0x7fede400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 -- 172.21.15.1:0/52719958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede4003520 con 0x7fede4003a00 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfdc35640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4004800 0x7fede4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfec37640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede4005210 0x7fede400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfec37640 1 -- 172.21.15.1:0/52719958 reap_dead start 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfe436640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 0x7fede4003e00 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fedf4003d80 tx=0x7fede4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e90d259718201f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.773+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4037070 con 0x7fede4003a00 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf402a9c0 con 0x7fede4003a00 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fedfd434640 1 -- 172.21.15.1:0/52719958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4003980 con 0x7fede4003a00 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 -- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 msgr2=0x7fede4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 0x7fede4003e00 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fedf4003d80 tx=0x7fede4005830 comp rx=0 tx=0).stop 2024-05-09T06:39:03.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 -- 172.21.15.1:0/52719958 shutdown_connections 2024-05-09T06:39:03.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 --2- 172.21.15.1:0/52719958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede4003a00 0x7fede4003e00 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 -- 172.21.15.1:0/52719958 >> 172.21.15.1:0/52719958 conn(0x7fedf8073340 msgr2=0x7fedf809d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 -- 172.21.15.1:0/52719958 shutdown_connections 2024-05-09T06:39:03.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.774+0000 7fee0069a640 1 -- 172.21.15.1:0/52719958 wait complete. 2024-05-09T06:39:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.775+0000 7fee0069a640 1 Processor -- start 2024-05-09T06:39:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.775+0000 7fee0069a640 1 -- start start 2024-05-09T06:39:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4003a00 0x7fedf80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf80b8680 0x7fedf80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fedf80b56d0 con 0x7fedf80c9890 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8161c60 con 0x7fedf80b8680 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fee0069a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fedf80b5850 con 0x7fede4003a00 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfdc35640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfdc35640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50744/0 (socket says 172.21.15.1:50744) 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfdc35640 1 -- 172.21.15.1:0/2780229823 learned_addr learned my addr 172.21.15.1:0/2780229823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfe436640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4003a00 0x7fedf80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfec37640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf80b8680 0x7fedf80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfdc35640 1 -- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4003a00 msgr2=0x7fedf80cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.776+0000 7fedfdc35640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4003a00 0x7fedf80cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedfdc35640 1 -- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf80b8680 msgr2=0x7fedf80b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedfdc35640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf80b8680 0x7fedf80b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedfdc35640 1 -- 172.21.15.1:0/2780229823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf80cdfa0 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedfdc35640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fede800e8a0 tx=0x7fede800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede800c870 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede8018070 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf80b95d0 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede800cc50 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.777+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf80cb310 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.778+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf80b9af0 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.778+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fede801d460 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.779+0000 7fedeeffd640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 0x7feddc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.779+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fede808af80 con 0x7fedf80c9890 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.779+0000 7fedfe436640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 0x7feddc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:03.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.779+0000 7fedfe436640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 0x7feddc06e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fedf4034000 tx=0x7fedf402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.783+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fede80596c0 con 0x7fedf80c9890 2024-05-09T06:39:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.967+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fedf80752b0 con 0x7fedf80c9890 2024-05-09T06:39:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.967+0000 7fedeeffd640 1 -- 172.21.15.1:0/2780229823 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) ==== 104+0+11 (secure 0 0 0) 0x7fede8058fc0 con 0x7fedf80c9890 2024-05-09T06:39:03.948 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 msgr2=0x7feddc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 0x7feddc06e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fedf4034000 tx=0x7fedf402e000 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 msgr2=0x7fedf80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fede800e8a0 tx=0x7fede800ed60 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 shutdown_connections 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7feddc06be30 0x7feddc06e2f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf80b8680 0x7fedf80b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf80c9890 0x7fedf80c9cf0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 --2- 172.21.15.1:0/2780229823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fede4003a00 0x7fedf80cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 >> 172.21.15.1:0/2780229823 conn(0x7fedf8073340 msgr2=0x7fedf80b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:03.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 shutdown_connections 2024-05-09T06:39:03.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:03.969+0000 7fee0069a640 1 -- 172.21.15.1:0/2780229823 wait complete. 2024-05-09T06:39:03.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:39:04.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.087+0000 7fc974288640 1 Processor -- start 2024-05-09T06:39:04.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.087+0000 7fc974288640 1 -- start start 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc974288640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc96c158a50 0x7fc96c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc974288640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c15b420 con 0x7fc96c06ede0 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc974288640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c15b6e0 con 0x7fc96c15c4a0 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc974288640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c158e30 con 0x7fc96c158a50 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc972024640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc96c158a50 0x7fc96c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc972825640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc96c06ede0 0x7fc96c15ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44662/0 (socket says 172.21.15.1:44662) 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc972024640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc96c158a50 0x7fc96c0718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50758/0 (socket says 172.21.15.1:50758) 2024-05-09T06:39:04.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.088+0000 7fc972825640 1 -- 172.21.15.1:0/474984962 learned_addr learned my addr 172.21.15.1:0/474984962 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:04.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc972024640 1 -- 172.21.15.1:0/474984962 >> v1:172.21.15.6:6789/0 conn(0x7fc96c15c4a0 legacy=0x7fc96c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:04.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc972024640 1 -- 172.21.15.1:0/474984962 >> v1:172.21.15.1:6789/0 conn(0x7fc96c06ede0 legacy=0x7fc96c15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc972024640 1 -- 172.21.15.1:0/474984962 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c15b9c0 con 0x7fc96c158a50 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc972024640 1 --2- 172.21.15.1:0/474984962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c158a50 0x7fc96c0718c0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009d90 tx=0x7fc95c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904a0a3418a1ee94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc971022640 1 -- 172.21.15.1:0/474984962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c007740 con 0x7fc96c158a50 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc971022640 1 -- 172.21.15.1:0/474984962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c0078e0 con 0x7fc96c158a50 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.089+0000 7fc971022640 1 -- 172.21.15.1:0/474984962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c007cb0 con 0x7fc96c158a50 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 -- 172.21.15.1:0/474984962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c158a50 msgr2=0x7fc96c0718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 --2- 172.21.15.1:0/474984962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c158a50 0x7fc96c0718c0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009d90 tx=0x7fc95c02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 -- 172.21.15.1:0/474984962 shutdown_connections 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 --2- 172.21.15.1:0/474984962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c158a50 0x7fc96c0718c0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 -- 172.21.15.1:0/474984962 >> 172.21.15.1:0/474984962 conn(0x7fc96c0cc4d0 msgr2=0x7fc96c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:04.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 -- 172.21.15.1:0/474984962 shutdown_connections 2024-05-09T06:39:04.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.090+0000 7fc974288640 1 -- 172.21.15.1:0/474984962 wait complete. 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.091+0000 7fc974288640 1 Processor -- start 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.091+0000 7fc974288640 1 -- start start 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc96c158a50 0x7fc96c15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc96c15c4a0 0x7fc96c151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c15bf80 con 0x7fc96c06ede0 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c072480 con 0x7fc96c15c4a0 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc974288640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c070ea0 con 0x7fc96c158a50 2024-05-09T06:39:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc972024640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc972024640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50760/0 (socket says 172.21.15.1:50760) 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc972024640 1 -- 172.21.15.1:0/4258437855 learned_addr learned my addr 172.21.15.1:0/4258437855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc972825640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc96c15c4a0 0x7fc96c151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.092+0000 7fc971823640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc96c158a50 0x7fc96c15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 -- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc96c158a50 msgr2=0x7fc96c15de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc96c158a50 0x7fc96c15de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 -- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc96c15c4a0 msgr2=0x7fc96c151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc96c15c4a0 0x7fc96c151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 -- 172.21.15.1:0/4258437855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c15cd70 con 0x7fc96c06ede0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc972024640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009eb0 tx=0x7fc95c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c040070 con 0x7fc96c06ede0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c031c40 con 0x7fc96c06ede0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c045460 con 0x7fc96c06ede0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c15d050 con 0x7fc96c06ede0 2024-05-09T06:39:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.093+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c15d610 con 0x7fc96c06ede0 2024-05-09T06:39:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.094+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c0752b0 con 0x7fc96c06ede0 2024-05-09T06:39:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.094+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc95c039040 con 0x7fc96c06ede0 2024-05-09T06:39:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.095+0000 7fc96affd640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 0x7fc95806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.095+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fc95c0acac0 con 0x7fc96c06ede0 2024-05-09T06:39:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.095+0000 7fc971823640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 0x7fc95806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.096+0000 7fc971823640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 0x7fc95806e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc96c154a90 tx=0x7fc960009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.100+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc95c07b290 con 0x7fc96c06ede0 2024-05-09T06:39:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.284+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc96c14fed0 con 0x7fc96c06ede0 2024-05-09T06:39:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.284+0000 7fc96affd640 1 -- 172.21.15.1:0/4258437855 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) ==== 100+0+11 (secure 0 0 0) 0x7fc95c07ab90 con 0x7fc96c06ede0 2024-05-09T06:39:04.265 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:39:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 msgr2=0x7fc95806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 0x7fc95806e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc96c154a90 tx=0x7fc960009040 comp rx=0 tx=0).stop 2024-05-09T06:39:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 msgr2=0x7fc96c14f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009eb0 tx=0x7fc95c004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 shutdown_connections 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc95806be30 0x7fc95806e2f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc96c15c4a0 0x7fc96c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc96c158a50 0x7fc96c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 --2- 172.21.15.1:0/4258437855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc96c06ede0 0x7fc96c14f990 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 >> 172.21.15.1:0/4258437855 conn(0x7fc96c0cc4d0 msgr2=0x7fc96c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 shutdown_connections 2024-05-09T06:39:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.287+0000 7fc974288640 1 -- 172.21.15.1:0/4258437855 wait complete. 2024-05-09T06:39:04.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:39:04.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.403+0000 7f328c06d640 1 Processor -- start 2024-05-09T06:39:04.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.403+0000 7f328c06d640 1 -- start start 2024-05-09T06:39:04.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f328c06d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f328415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f328c06d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3284099930 con 0x7f32840c6860 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f328c06d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3284162930 con 0x7f3284099b30 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f328c06d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f32840b1e20 con 0x7f32840b2aa0 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3289e09640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f32840c6860 0x7f32840c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44676/0 (socket says 172.21.15.1:44676) 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3289e09640 1 -- 172.21.15.1:0/651199940 learned_addr learned my addr 172.21.15.1:0/651199940 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f328a60a640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f328415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268542900 0 0) 0x7f3280001110 con 0x7f32840c6860 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270003520 con 0x7f32840c6860 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3125785005 0 0) 0x7f3270003520 con 0x7f32840c6860 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 msgr2=0x7f328415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f328415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 >> v1:172.21.15.6:6789/0 conn(0x7f3284099b30 legacy=0x7f328415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284162c10 con 0x7f32840c6860 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f3280001840 con 0x7f32840c6860 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 >> v1:172.21.15.1:6789/0 conn(0x7f32840c6860 legacy=0x7f32840c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f32700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270003f60 0x7f3270004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f327000bb50 con 0x7f3270003f60 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f327000bcd0 con 0x7f3270004d40 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.404+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f327000bed0 con 0x7f32840b2aa0 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270003f60 0x7f3270004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289e09640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f32700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f328a60a640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 -- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 msgr2=0x7f32700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32840b2aa0 0x7f32700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 -- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270004d40 msgr2=0x7f32700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.405+0000 7f3289608640 1 -- 172.21.15.1:0/651199940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270003520 con 0x7f3270003f60 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f3289608640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270003f60 0x7f3270004800 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f3274001750 tx=0x7f3270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8c56a59aeacd9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274001370 con 0x7f3270003f60 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32740052d0 con 0x7f3270003f60 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f3288e07640 1 -- 172.21.15.1:0/651199940 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327402abe0 con 0x7f3270003f60 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 -- 172.21.15.1:0/651199940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270003f60 msgr2=0x7f3270004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 --2- 172.21.15.1:0/651199940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270003f60 0x7f3270004800 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f3274001750 tx=0x7f3270005830 comp rx=0 tx=0).stop 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328a60a640 1 -- 172.21.15.1:0/651199940 reap_dead start 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 -- 172.21.15.1:0/651199940 shutdown_connections 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 -- 172.21.15.1:0/651199940 >> 172.21.15.1:0/651199940 conn(0x7f32840b1fb0 msgr2=0x7f32840b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 -- 172.21.15.1:0/651199940 shutdown_connections 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.406+0000 7f328c06d640 1 -- 172.21.15.1:0/651199940 wait complete. 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 Processor -- start 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 -- start start 2024-05-09T06:39:04.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270003f60 0x7f32840cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32841ea060 0x7f32841f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f328415fa30 con 0x7f32841ea4c0 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f328415ee90 con 0x7f3270003f60 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328c06d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f328415e4b0 con 0x7f32841ea060 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328a60a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328a60a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50774/0 (socket says 172.21.15.1:50774) 2024-05-09T06:39:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.408+0000 7f328a60a640 1 -- 172.21.15.1:0/2244682612 learned_addr learned my addr 172.21.15.1:0/2244682612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f3289608640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32841ea060 0x7f32841f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f3289e09640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270003f60 0x7f32840cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 -- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32841ea060 msgr2=0x7f32841f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32841ea060 0x7f32841f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 -- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270003f60 msgr2=0x7f32840cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270003f60 0x7f32840cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 -- 172.21.15.1:0/2244682612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32841e7ca0 con 0x7f32841ea4c0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328a60a640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f327c011bf0 tx=0x7f327c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c00fdb0 con 0x7f32841ea4c0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c015430 con 0x7f32841ea4c0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32841e7f80 con 0x7f32841ea4c0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c015820 con 0x7f32841ea4c0 2024-05-09T06:39:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.409+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841e8540 con 0x7f32841ea4c0 2024-05-09T06:39:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.410+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284074860 con 0x7f32841ea4c0 2024-05-09T06:39:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.411+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f327c0159c0 con 0x7f32841ea4c0 2024-05-09T06:39:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.411+0000 7f327a7fc640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 0x7f326806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.411+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f327c08bcd0 con 0x7f32841ea4c0 2024-05-09T06:39:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.411+0000 7f3289e09640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 0x7f326806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:04.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.412+0000 7f3289e09640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 0x7f326806e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3280001eb0 tx=0x7f3280007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:04.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.415+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3284074860 con 0x7f32841ea4c0 2024-05-09T06:39:04.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.611+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f328406ce30 con 0x7f32841ea4c0 2024-05-09T06:39:04.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.612+0000 7f327a7fc640 1 -- 172.21.15.1:0/2244682612 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) ==== 80+0+120 (secure 0 0 0) 0x7f327c019030 con 0x7f32841ea4c0 2024-05-09T06:39:04.593 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6837/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:04.594 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6836/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:listed 2 entries 2024-05-09T06:39:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 msgr2=0x7f326806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 0x7f326806e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3280001eb0 tx=0x7f3280007dc0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 msgr2=0x7f32841f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f327c011bf0 tx=0x7f327c00f580 comp rx=0 tx=0).stop 2024-05-09T06:39:04.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 shutdown_connections 2024-05-09T06:39:04.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f326806be30 0x7f326806e2f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32841ea4c0 0x7f32841f83e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f32841ea060 0x7f32841f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 --2- 172.21.15.1:0/2244682612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3270003f60 0x7f32840cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:04.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 >> 172.21.15.1:0/2244682612 conn(0x7f32840b1fb0 msgr2=0x7f328406e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:04.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 shutdown_connections 2024-05-09T06:39:04.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:04.614+0000 7f328c06d640 1 -- 172.21.15.1:0/2244682612 wait complete. 2024-05-09T06:39:04.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:39:04.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:04.606 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:04.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[82661]: fuse finished with error 0 and tester_r 0 2024-05-09T06:39:04.632 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:10.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:39:10.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:10.734 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:39:10.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:10.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.933+0000 7fe0921d5640 1 Processor -- start 2024-05-09T06:39:10.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 -- start start 2024-05-09T06:39:10.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08c0b33c0 con 0x7fe08c0d0f80 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe08c0d4dd0 con 0x7fe08c161950 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.934+0000 7fe0921d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c0b3660 con 0x7fe08c0d4940 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08b7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:51618/0 (socket says 172.21.15.1:51618) 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 -- 172.21.15.1:0/347773794 learned_addr learned my addr 172.21.15.1:0/347773794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08b7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:58674/0 (socket says 172.21.15.1:58674) 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08bfff640 1 --1- 172.21.15.1:0/347773794 >> v1:172.21.15.6:6789/0 conn(0x7fe08c0d0f80 0x7fe08c0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:48830/0 (socket says 172.21.15.1:48830) 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 -- 172.21.15.1:0/347773794 >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 msgr2=0x7fe08c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 --2- 172.21.15.1:0/347773794 >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 -- 172.21.15.1:0/347773794 >> v1:172.21.15.6:6789/0 conn(0x7fe08c0d0f80 legacy=0x7fe08c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08affd640 1 -- 172.21.15.1:0/347773794 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c0b3940 con 0x7fe08c0d4940 2024-05-09T06:39:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.935+0000 7fe08b7fe640 1 --2- 172.21.15.1:0/347773794 >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.936+0000 7fe08affd640 1 --2- 172.21.15.1:0/347773794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b3590 tx=0x7fe08002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d733a2be2137b227 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.936+0000 7fe08a7fc640 1 -- 172.21.15.1:0/347773794 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080037070 con 0x7fe08c0d4940 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.936+0000 7fe08a7fc640 1 -- 172.21.15.1:0/347773794 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe080032440 con 0x7fe08c0d4940 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.936+0000 7fe08a7fc640 1 -- 172.21.15.1:0/347773794 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080032810 con 0x7fe08c0d4940 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 -- 172.21.15.1:0/347773794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 msgr2=0x7fe08c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 --2- 172.21.15.1:0/347773794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b3590 tx=0x7fe08002a250 comp rx=0 tx=0).stop 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 -- 172.21.15.1:0/347773794 shutdown_connections 2024-05-09T06:39:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 --2- 172.21.15.1:0/347773794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0d4940 0x7fe08c0d2940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:10.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 --2- 172.21.15.1:0/347773794 >> v2:172.21.15.6:3300/0 conn(0x7fe08c161950 0x7fe08c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:10.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 -- 172.21.15.1:0/347773794 >> 172.21.15.1:0/347773794 conn(0x7fe08c0d3e20 msgr2=0x7fe08c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:10.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 -- 172.21.15.1:0/347773794 shutdown_connections 2024-05-09T06:39:10.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.937+0000 7fe0921d5640 1 -- 172.21.15.1:0/347773794 wait complete. 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.938+0000 7fe0921d5640 1 Processor -- start 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 -- start start 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe08c161950 0x7fe08c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0cce90 0x7fe08c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0b52d0 con 0x7fe08c0d0f80 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0b5150 con 0x7fe08c161950 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe0921d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c0b5450 con 0x7fe08c0cce90 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe08b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe08b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50558/0 (socket says 172.21.15.1:50558) 2024-05-09T06:39:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe08b7fe640 1 -- 172.21.15.1:0/4007013830 learned_addr learned my addr 172.21.15.1:0/4007013830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe08affd640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe08c161950 0x7fe08c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.939+0000 7fe08bfff640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0cce90 0x7fe08c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 -- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0cce90 msgr2=0x7fe08c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0cce90 0x7fe08c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 -- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe08c161950 msgr2=0x7fe08c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe08c161950 0x7fe08c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 -- 172.21.15.1:0/4007013830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c0cd560 con 0x7fe08c0d0f80 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe08b7fe640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e960 tx=0x7fe07c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017070 con 0x7fe08c0d0f80 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07c00ca00 con 0x7fe08c0d0f80 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c0cd840 con 0x7fe08c0d0f80 2024-05-09T06:39:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00cdd0 con 0x7fe08c0d0f80 2024-05-09T06:39:10.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.940+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c14bc40 con 0x7fe08c0d0f80 2024-05-09T06:39:10.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.941+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c0752b0 con 0x7fe08c0d0f80 2024-05-09T06:39:10.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.942+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe07c01a070 con 0x7fe08c0d0f80 2024-05-09T06:39:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.942+0000 7fe073fff640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 0x7fe07806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.942+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fe07c08b170 con 0x7fe08c0d0f80 2024-05-09T06:39:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.942+0000 7fe08affd640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 0x7fe07806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.943+0000 7fe08affd640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 0x7fe07806e320 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe080034000 tx=0x7fe08002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:10.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:10.947+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe07c059380 con 0x7fe08c0d0f80 2024-05-09T06:39:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.167+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe08c06d930 con 0x7fe08c0d0f80 2024-05-09T06:39:11.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.168+0000 7fe073fff640 1 -- 172.21.15.1:0/4007013830 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5176 (secure 0 0 0) 0x7fe07c05d680 con 0x7fe08c0d0f80 2024-05-09T06:39:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:11.149 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:38:57.427211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9957":{"gid":9957,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/3597643567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":3597643567},{"type":"v1","addr":"172.21.15.1:6837","nonce":3597643567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9957,"qdb_cluster":[9957]},"id":8}]} 2024-05-09T06:39:11.149 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 105 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 msgr2=0x7fe07806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 0x7fe07806e320 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe080034000 tx=0x7fe08002e000 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 msgr2=0x7fe08c1e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e960 tx=0x7fe07c00ee20 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 shutdown_connections 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe07806be60 0x7fe07806e320 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe08c0cce90 0x7fe08c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe08c161950 0x7fe08c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 --2- 172.21.15.1:0/4007013830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe08c0d0f80 0x7fe08c1e6cb0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 >> 172.21.15.1:0/4007013830 conn(0x7fe08c0d3e20 msgr2=0x7fe08c0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 shutdown_connections 2024-05-09T06:39:11.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.170+0000 7fe0921d5640 1 -- 172.21.15.1:0/4007013830 wait complete. 2024-05-09T06:39:11.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-05-09T06:39:11.287 INFO:teuthology.orchestra.run.smithi001.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-09T06:39:11.287 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-09T06:39:11.287 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-09T06:39:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 Processor -- start 2024-05-09T06:39:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 -- start start 2024-05-09T06:39:11.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc900030f0 0x5577c13d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 0x5577c13da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5577c1352c80 con 0x5577c13a9a20 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5577c1356cc0 con 0x7fcc900030f0 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca8150f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5577c13d4ba0 con 0x5577c1461c50 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca6452640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca6452640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50586/0 (socket says 172.21.15.1:50586) 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca6452640 1 -- 172.21.15.1:0/441922981 learned_addr learned my addr 172.21.15.1:0/441922981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca5450640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 0x5577c13da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.353+0000 7fcca5c51640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc900030f0 0x5577c13d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 -- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 msgr2=0x5577c13da0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 0x5577c13da0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 -- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc900030f0 msgr2=0x5577c13d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc900030f0 0x5577c13d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 -- 172.21.15.1:0/441922981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577c147b320 con 0x5577c13a9a20 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca5450640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 0x5577c13da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca6452640 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x5577c13db570 tx=0x7fcca0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d615011ea44982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcc97fff640 1 -- 172.21.15.1:0/441922981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca001a070 con 0x5577c13a9a20 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcc97fff640 1 -- 172.21.15.1:0/441922981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca0015660 con 0x5577c13a9a20 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcc97fff640 1 -- 172.21.15.1:0/441922981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0015a30 con 0x5577c13a9a20 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 -- 172.21.15.1:0/441922981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 msgr2=0x5577c13a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x5577c13db570 tx=0x7fcca0005a00 comp rx=0 tx=0).stop 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 -- 172.21.15.1:0/441922981 shutdown_connections 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c13a9a20 0x5577c13a9ea0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5577c1461c50 0x5577c13da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 --2- 172.21.15.1:0/441922981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc900030f0 0x5577c13d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.354+0000 7fcca8150f40 1 -- 172.21.15.1:0/441922981 >> 172.21.15.1:0/441922981 conn(0x5577c13abd10 msgr2=0x5577c1469b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.355+0000 7fcca8150f40 1 -- 172.21.15.1:0/441922981 shutdown_connections 2024-05-09T06:39:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.355+0000 7fcca8150f40 1 -- 172.21.15.1:0/441922981 wait complete. 2024-05-09T06:39:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.355+0000 7fcca8150f40 1 Processor -- start 2024-05-09T06:39:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.355+0000 7fcca8150f40 1 -- start start 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5577c13a9a20 0x5577c145e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 0x5577c145eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcc900030f0 0x5577c13ed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5577c13d54d0 con 0x5577c1461c50 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5577c13da5f0 con 0x5577c13a9a20 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca8150f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5577c13ed2d0 con 0x7fcc900030f0 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 0x5577c145eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 0x5577c145eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50592/0 (socket says 172.21.15.1:50592) 2024-05-09T06:39:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 -- 172.21.15.1:0/1333641808 learned_addr learned my addr 172.21.15.1:0/1333641808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca6452640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcc900030f0 0x5577c13ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5c51640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5577c13a9a20 0x5577c145e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 -- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcc900030f0 msgr2=0x5577c13ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcc900030f0 0x5577c13ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 -- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5577c13a9a20 msgr2=0x5577c145e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5577c13a9a20 0x5577c145e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 -- 172.21.15.1:0/1333641808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577c147db70 con 0x5577c1461c50 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5c51640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5577c13a9a20 0x5577c145e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.356+0000 7fcca5450640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 0x5577c145eaf0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b5c0 tx=0x7fcc9c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.357+0000 7fcc96ffd640 1 -- 172.21.15.1:0/1333641808 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c058040 con 0x5577c1461c50 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.357+0000 7fcc96ffd640 1 -- 172.21.15.1:0/1333641808 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c053a00 con 0x5577c1461c50 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.357+0000 7fcc96ffd640 1 -- 172.21.15.1:0/1333641808 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c053db0 con 0x5577c1461c50 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.357+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577c147de50 con 0x5577c1461c50 2024-05-09T06:39:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.357+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577c147e130 con 0x5577c1461c50 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.358+0000 7fcc96ffd640 1 -- 172.21.15.1:0/1333641808 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc9c060470 con 0x5577c1461c50 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcc96ffd640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcc8806a810 0x7fcc8806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcc96ffd640 1 -- 172.21.15.1:0/1333641808 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7fcc9c0cdda0 con 0x5577c1461c50 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca5c51640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcc8806a810 0x7fcc8806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca8150f40 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5577c147a080 0x5577c147ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x5577c13edcc0 con 0x5577c147a080 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca6452640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5577c147a080 0x5577c147ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca5c51640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcc8806a810 0x7fcc8806ccd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc98002250 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.359+0000 7fcca6452640 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5577c147a080 0x5577c147ca10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca6452640 1 -- 172.21.15.1:0/1333641808 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fcca0015810 con 0x5577c147a080 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5577c147a080 msgr2=0x5577c147ca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x5577c147a080 0x5577c147ca10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcc8806a810 msgr2=0x7fcc8806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcc8806a810 0x7fcc8806ccd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc98002250 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 msgr2=0x5577c145eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 --2- 172.21.15.1:0/1333641808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5577c1461c50 0x5577c145eaf0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b5c0 tx=0x7fcc9c0012d0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca6452640 1 -- 172.21.15.1:0/1333641808 reap_dead start 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 shutdown_connections 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 >> 172.21.15.1:0/1333641808 conn(0x5577c13abd10 msgr2=0x5577c1482a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 shutdown_connections 2024-05-09T06:39:11.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.360+0000 7fcca8150f40 1 -- 172.21.15.1:0/1333641808 wait complete. 2024-05-09T06:39:11.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:39:11.347 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-09T06:39:11.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.411+0000 7f50f0940f40 1 Processor -- start 2024-05-09T06:39:11.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.411+0000 7f50f0940f40 1 -- start start 2024-05-09T06:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16b48ff0 0x55ca16b3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16bd2540 0x55ca16b40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ca16aa4c80 con 0x55ca16b001d0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ca16aa8cc0 con 0x55ca16bd2540 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50f0940f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ca16b40940 con 0x55ca16b48ff0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50606/0 (socket says 172.21.15.1:50606) 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 -- 172.21.15.1:0/1057276056 learned_addr learned my addr 172.21.15.1:0/1057276056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50edc42640 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16b48ff0 0x55ca16b3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50eec44640 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16bd2540 0x55ca16b40400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 -- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16b48ff0 msgr2=0x55ca16b3fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16b48ff0 0x55ca16b3fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 -- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16bd2540 msgr2=0x55ca16b40400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16bd2540 0x55ca16b40400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 -- 172.21.15.1:0/1057276056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca16bceb20 con 0x55ca16b001d0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ee443640 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f50e0001620 tx=0x7f50e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af9cb1d5f26956a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ecc40640 1 -- 172.21.15.1:0/1057276056 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0004520 con 0x55ca16b001d0 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ecc40640 1 -- 172.21.15.1:0/1057276056 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e00046c0 con 0x55ca16b001d0 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.412+0000 7f50ecc40640 1 -- 172.21.15.1:0/1057276056 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0004a70 con 0x55ca16b001d0 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 -- 172.21.15.1:0/1057276056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 msgr2=0x55ca16afcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f50e0001620 tx=0x7f50e0001ae0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 -- 172.21.15.1:0/1057276056 shutdown_connections 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16bd2540 0x55ca16b40400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16b48ff0 0x55ca16b3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 --2- 172.21.15.1:0/1057276056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b001d0 0x55ca16afcfc0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 -- 172.21.15.1:0/1057276056 >> 172.21.15.1:0/1057276056 conn(0x55ca16afbb00 msgr2=0x55ca16aff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 -- 172.21.15.1:0/1057276056 shutdown_connections 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.413+0000 7f50f0940f40 1 -- 172.21.15.1:0/1057276056 wait complete. 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 Processor -- start 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 -- start start 2024-05-09T06:39:11.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16b001d0 0x55ca16c5e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 0x55ca16c6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16bd2540 0x55ca16bca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ca16c5ee30 con 0x55ca16b48ff0 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ca16bce840 con 0x55ca16b001d0 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50f0940f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ca16c6af00 con 0x55ca16bd2540 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50edc42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 0x55ca16c6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50edc42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 0x55ca16c6a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50620/0 (socket says 172.21.15.1:50620) 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50edc42640 1 -- 172.21.15.1:0/2930240327 learned_addr learned my addr 172.21.15.1:0/2930240327 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50ee443640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16b001d0 0x55ca16c5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.414+0000 7f50eec44640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16bd2540 0x55ca16bca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 -- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16bd2540 msgr2=0x55ca16bca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ca16bd2540 0x55ca16bca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 -- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16b001d0 msgr2=0x55ca16c5e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16b001d0 0x55ca16c5e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 -- 172.21.15.1:0/2930240327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca16bcac70 con 0x55ca16b48ff0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50ee443640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca16b001d0 0x55ca16c5e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50edc42640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 0x55ca16c6a9c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x55ca16afe560 tx=0x7f50d800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50e77fe640 1 -- 172.21.15.1:0/2930240327 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8015070 con 0x55ca16b48ff0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50e77fe640 1 -- 172.21.15.1:0/2930240327 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d8011640 con 0x55ca16b48ff0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ca16c5bae0 con 0x55ca16b48ff0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50e77fe640 1 -- 172.21.15.1:0/2930240327 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800be60 con 0x55ca16b48ff0 2024-05-09T06:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.415+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca16bcaf50 con 0x55ca16b48ff0 2024-05-09T06:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50e77fe640 1 -- 172.21.15.1:0/2930240327 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50d8018070 con 0x55ca16b48ff0 2024-05-09T06:39:11.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50e77fe640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50d006a810 0x7f50d006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50e77fe640 1 -- 172.21.15.1:0/2930240327 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f50d8089bf0 con 0x55ca16b48ff0 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50ee443640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50d006a810 0x7f50d006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50f0940f40 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55ca16c5c5c0 0x55ca16c5c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.417+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x55ca16c5cf00 con 0x55ca16c5c5c0 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50eec44640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55ca16c5c5c0 0x55ca16c5c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50ee443640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50d006a810 0x7f50d006ccd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f50e0001b20 tx=0x7f50e00021f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50eec44640 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55ca16c5c5c0 0x55ca16c5c9c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50eec44640 1 -- 172.21.15.1:0/2930240327 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f50e8054690 con 0x55ca16c5c5c0 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55ca16c5c5c0 msgr2=0x55ca16c5c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55ca16c5c5c0 0x55ca16c5c9c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50d006a810 msgr2=0x7f50d006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f50d006a810 0x7f50d006ccd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f50e0001b20 tx=0x7f50e00021f0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 msgr2=0x55ca16c6a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 --2- 172.21.15.1:0/2930240327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca16b48ff0 0x55ca16c6a9c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x55ca16afe560 tx=0x7f50d800df90 comp rx=0 tx=0).stop 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50eec44640 1 -- 172.21.15.1:0/2930240327 reap_dead start 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 shutdown_connections 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.418+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 >> 172.21.15.1:0/2930240327 conn(0x55ca16afbb00 msgr2=0x55ca16aff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.419+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 shutdown_connections 2024-05-09T06:39:11.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.419+0000 7f50f0940f40 1 -- 172.21.15.1:0/2930240327 wait complete. 2024-05-09T06:39:11.410 INFO:teuthology.orchestra.run.smithi001.stdout:mytag123 2024-05-09T06:39:11.410 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-09T06:39:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.476+0000 7f87ddb66f40 1 Processor -- start 2024-05-09T06:39:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.476+0000 7f87ddb66f40 1 -- start start 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c95ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c960850 0x55bc7c8d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc7c963dd0 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55bc7c84dc80 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55bc7c97de90 con 0x55bc7c960850 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50634/0 (socket says 172.21.15.1:50634) 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 -- 172.21.15.1:0/3888492733 learned_addr learned my addr 172.21.15.1:0/3888492733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 -- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c960850 msgr2=0x55bc7c8d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c960850 0x55bc7c8d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 -- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c8a4dd0 msgr2=0x55bc7c95ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87db667640 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c95ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c95ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 -- 172.21.15.1:0/3888492733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7c7d2cb0 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.477+0000 7f87dae66640 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f87d0001db0 tx=0x7f87d000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1e90ae9ae1daa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87d9e64640 1 -- 172.21.15.1:0/3888492733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d000bde0 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87d9e64640 1 -- 172.21.15.1:0/3888492733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d000f400 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87d9e64640 1 -- 172.21.15.1:0/3888492733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d000f7f0 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 -- 172.21.15.1:0/3888492733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 msgr2=0x55bc7c97d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f87d0001db0 tx=0x7f87d000b5b0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 -- 172.21.15.1:0/3888492733 shutdown_connections 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c960850 0x55bc7c8d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8d3bc0 0x55bc7c97d310 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/3888492733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c95ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 -- 172.21.15.1:0/3888492733 >> 172.21.15.1:0/3888492733 conn(0x55bc7c8d27c0 msgr2=0x55bc7c8a88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 -- 172.21.15.1:0/3888492733 shutdown_connections 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.478+0000 7f87ddb66f40 1 -- 172.21.15.1:0/3888492733 wait complete. 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 Processor -- start 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 -- start start 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c960cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c8d3bc0 0x55bc7c8d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c960850 0x55bc7c8d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc7c9634c0 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55bc7c8f07f0 con 0x55bc7c960850 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.479+0000 7f87ddb66f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55bc7c8f0510 con 0x55bc7c8d3bc0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c960cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c960cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50636/0 (socket says 172.21.15.1:50636) 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 -- 172.21.15.1:0/4137421004 learned_addr learned my addr 172.21.15.1:0/4137421004 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87dae66640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c8d3bc0 0x55bc7c8d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87dbe68640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c960850 0x55bc7c8d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 -- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c8d3bc0 msgr2=0x55bc7c8d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55bc7c8d3bc0 0x55bc7c8d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 -- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c960850 msgr2=0x55bc7c8d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c960850 0x55bc7c8d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 -- 172.21.15.1:0/4137421004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7c9777f0 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87dbe68640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc7c960850 0x55bc7c8d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87db667640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c960cc0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87d8e62640 1 -- 172.21.15.1:0/4137421004 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc06a070 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87d8e62640 1 -- 172.21.15.1:0/4137421004 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc051910 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc7c8efaf0 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87d8e62640 1 -- 172.21.15.1:0/4137421004 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc055040 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.480+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7c8f00b0 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87d8e62640 1 -- 172.21.15.1:0/4137421004 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87cc06d070 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87d8e62640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f87c806a810 0x7f87c806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87d8e62640 1 -- 172.21.15.1:0/4137421004 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f87cc0cd5f0 con 0x55bc7c8a4dd0 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87dae66640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f87c806a810 0x7f87c806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55bc7c977fb0 0x55bc7c959180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.482+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x55bc7c8cfad0 con 0x55bc7c977fb0 2024-05-09T06:39:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87dbe68640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55bc7c977fb0 0x55bc7c959180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87dae66640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f87c806a810 0x7f87c806ccd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x55bc7c95ff30 tx=0x7f87d0009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87dbe68640 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55bc7c977fb0 0x55bc7c959180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87dbe68640 1 -- 172.21.15.1:0/4137421004 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f87d400b710 con 0x55bc7c977fb0 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55bc7c977fb0 msgr2=0x55bc7c959180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.483+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55bc7c977fb0 0x55bc7c959180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f87c806a810 msgr2=0x7f87c806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f87c806a810 0x7f87c806ccd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x55bc7c95ff30 tx=0x7f87d0009000 comp rx=0 tx=0).stop 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 msgr2=0x55bc7c960cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 --2- 172.21.15.1:0/4137421004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc7c8a4dd0 0x55bc7c960cc0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc002fb0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87dbe68640 1 -- 172.21.15.1:0/4137421004 reap_dead start 2024-05-09T06:39:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 shutdown_connections 2024-05-09T06:39:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 >> 172.21.15.1:0/4137421004 conn(0x55bc7c8d27c0 msgr2=0x55bc7c8a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 shutdown_connections 2024-05-09T06:39:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.484+0000 7f87ddb66f40 1 -- 172.21.15.1:0/4137421004 wait complete. 2024-05-09T06:39:11.470 INFO:teuthology.orchestra.run.smithi001.stdout:mytag123 2024-05-09T06:39:11.470 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.535+0000 7f9a189b7f40 1 Processor -- start 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.535+0000 7f9a189b7f40 1 -- start start 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39cddaff0 0x55d39cdd3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 0x55d39cdd4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d39cd36c80 con 0x55d39cd911c0 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d39cdbf450 con 0x55d39ce64540 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d39cdd18a0 con 0x55d39cddaff0 2024-05-09T06:39:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50642/0 (socket says 172.21.15.1:50642) 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 -- 172.21.15.1:0/1401876484 learned_addr learned my addr 172.21.15.1:0/1401876484 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a15cb9640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39cddaff0 0x55d39cdd3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a16cbb640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 0x55d39cdd4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 -- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39cddaff0 msgr2=0x55d39cdd3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39cddaff0 0x55d39cdd3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 -- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 msgr2=0x55d39cdd4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 0x55d39cdd4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 -- 172.21.15.1:0/1401876484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39ccbbcb0 con 0x55d39cd911c0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a16cbb640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 0x55d39cdd4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a164ba640 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800b3c0 tx=0x7f9a0800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85e829148254eee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a14cb7640 1 -- 172.21.15.1:0/1401876484 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08012020 con 0x55d39cd911c0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a14cb7640 1 -- 172.21.15.1:0/1401876484 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0800d6b0 con 0x55d39cd911c0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.536+0000 7f9a14cb7640 1 -- 172.21.15.1:0/1401876484 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800daa0 con 0x55d39cd911c0 2024-05-09T06:39:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 -- 172.21.15.1:0/1401876484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 msgr2=0x55d39cdd5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800b3c0 tx=0x7f9a0800b880 comp rx=0 tx=0).stop 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 -- 172.21.15.1:0/1401876484 shutdown_connections 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39ce64540 0x55d39cdd4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39cddaff0 0x55d39cdd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/1401876484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 -- 172.21.15.1:0/1401876484 >> 172.21.15.1:0/1401876484 conn(0x55d39cd8db00 msgr2=0x55d39cdd9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 -- 172.21.15.1:0/1401876484 shutdown_connections 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.537+0000 7f9a189b7f40 1 -- 172.21.15.1:0/1401876484 wait complete. 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 Processor -- start 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 -- start start 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 0x55d39ce56820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39ce64540 0x55d39cefabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d39cdcfd30 con 0x55d39cddaff0 2024-05-09T06:39:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d39ce63220 con 0x55d39cd911c0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a189b7f40 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d39cdd6030 con 0x55d39ce64540 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a15cb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 0x55d39ce56820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a15cb9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 0x55d39ce56820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50652/0 (socket says 172.21.15.1:50652) 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a15cb9640 1 -- 172.21.15.1:0/2225610208 learned_addr learned my addr 172.21.15.1:0/2225610208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a16cbb640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39ce64540 0x55d39cefabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.538+0000 7f9a164ba640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 -- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39ce64540 msgr2=0x55d39cefabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d39ce64540 0x55d39cefabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 -- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39cd911c0 msgr2=0x55d39cdd5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d39cd911c0 0x55d39cdd5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 -- 172.21.15.1:0/2225610208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39ce56d60 con 0x55d39cddaff0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a15cb9640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 0x55d39ce56820 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a077fe640 1 -- 172.21.15.1:0/2225610208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c064030 con 0x55d39cddaff0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a077fe640 1 -- 172.21.15.1:0/2225610208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c0547b0 con 0x55d39cddaff0 2024-05-09T06:39:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d39ce62d80 con 0x55d39cddaff0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a077fe640 1 -- 172.21.15.1:0/2225610208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c059040 con 0x55d39cddaff0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.539+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d39ce60ae0 con 0x55d39cddaff0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a077fe640 1 -- 172.21.15.1:0/2225610208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a0c067030 con 0x55d39cddaff0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a077fe640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f99f806a810 0x7f99f806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a077fe640 1 -- 172.21.15.1:0/2225610208 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f9a0c0cd6a0 con 0x55d39cddaff0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a164ba640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f99f806a810 0x7f99f806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d39ce5f7f0 0x55d39cdd30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x55d39cdd3630 con 0x55d39ce5f7f0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.541+0000 7f9a16cbb640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d39ce5f7f0 0x55d39cdd30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a164ba640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f99f806a810 0x7f99f806ccd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000c00 tx=0x7f9a08009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a16cbb640 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d39ce5f7f0 0x55d39cdd30f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a16cbb640 1 -- 172.21.15.1:0/2225610208 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f9a10002a80 con 0x55d39ce5f7f0 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d39ce5f7f0 msgr2=0x55d39cdd30f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d39ce5f7f0 0x55d39cdd30f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f99f806a810 msgr2=0x7f99f806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f99f806a810 0x7f99f806ccd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000c00 tx=0x7f9a08009000 comp rx=0 tx=0).stop 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 msgr2=0x55d39ce56820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 --2- 172.21.15.1:0/2225610208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d39cddaff0 0x55d39ce56820 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c002fb0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a16cbb640 1 -- 172.21.15.1:0/2225610208 reap_dead start 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 shutdown_connections 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 >> 172.21.15.1:0/2225610208 conn(0x55d39cd8db00 msgr2=0x55d39ce60f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 shutdown_connections 2024-05-09T06:39:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.542+0000 7f9a189b7f40 1 -- 172.21.15.1:0/2225610208 wait complete. 2024-05-09T06:39:11.529 INFO:teuthology.orchestra.run.smithi001.stdout:mytag123 2024-05-09T06:39:11.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-09T06:39:11.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.657+0000 7f428ad42640 1 Processor -- start 2024-05-09T06:39:11.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.657+0000 7f428ad42640 1 -- start start 2024-05-09T06:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4284158a50 0x7f42840718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f428415c4a0 0x7f4284071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 0x7f428406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4284158e30 con 0x7f4284158a50 2024-05-09T06:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4284072480 con 0x7f428406e6d0 2024-05-09T06:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f428ad42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f428415b420 con 0x7f428415c4a0 2024-05-09T06:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4284158a50 0x7f42840718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4284158a50 0x7f42840718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50656/0 (socket says 172.21.15.1:50656) 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 -- 172.21.15.1:0/525542393 learned_addr learned my addr 172.21.15.1:0/525542393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4283fff640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f428415c4a0 0x7f4284071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f42892df640 1 --2- 172.21.15.1:0/525542393 >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 0x7f428406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 -- 172.21.15.1:0/525542393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f428415c4a0 msgr2=0x7f4284071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f428415c4a0 0x7f4284071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 -- 172.21.15.1:0/525542393 >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 msgr2=0x7f428406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 --2- 172.21.15.1:0/525542393 >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 0x7f428406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.658+0000 7f4288ade640 1 -- 172.21.15.1:0/525542393 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428415b6e0 con 0x7f4284158a50 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.659+0000 7f42892df640 1 --2- 172.21.15.1:0/525542393 >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 0x7f428406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.659+0000 7f4288ade640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4284158a50 0x7f42840718c0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f428406c2a0 tx=0x7f427c02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c4eb43c7799c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.659+0000 7f42837fe640 1 -- 172.21.15.1:0/525542393 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c038070 con 0x7f4284158a50 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.659+0000 7f42837fe640 1 -- 172.21.15.1:0/525542393 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c031710 con 0x7f4284158a50 2024-05-09T06:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.659+0000 7f42837fe640 1 -- 172.21.15.1:0/525542393 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c031ae0 con 0x7f4284158a50 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 -- 172.21.15.1:0/525542393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4284158a50 msgr2=0x7f42840718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4284158a50 0x7f42840718c0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f428406c2a0 tx=0x7f427c02c3f0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 -- 172.21.15.1:0/525542393 shutdown_connections 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 --2- 172.21.15.1:0/525542393 >> v2:172.21.15.6:3300/0 conn(0x7f428406e6d0 0x7f428406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f428415c4a0 0x7f4284071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 --2- 172.21.15.1:0/525542393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4284158a50 0x7f42840718c0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 -- 172.21.15.1:0/525542393 >> 172.21.15.1:0/525542393 conn(0x7f42840cc4d0 msgr2=0x7f42840cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 -- 172.21.15.1:0/525542393 shutdown_connections 2024-05-09T06:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.660+0000 7f428ad42640 1 -- 172.21.15.1:0/525542393 wait complete. 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.661+0000 7f428ad42640 1 Processor -- start 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.661+0000 7f428ad42640 1 -- start start 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 0x7f42841504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 0x7f4284154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f428415bf80 con 0x7f428406e6d0 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f428415bca0 con 0x7f428415c4a0 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f428ad42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4284070ea0 con 0x7f4284158a50 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f4288ade640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f4288ade640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50660/0 (socket says 172.21.15.1:50660) 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f4288ade640 1 -- 172.21.15.1:0/1361205182 learned_addr learned my addr 172.21.15.1:0/1361205182 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f42892df640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 0x7f4284154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f4283fff640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 0x7f42841504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.662+0000 7f4288ade640 1 -- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 msgr2=0x7f42841504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4288ade640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 0x7f42841504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4288ade640 1 -- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 msgr2=0x7f4284154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4288ade640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 0x7f4284154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4288ade640 1 -- 172.21.15.1:0/1361205182 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42840ae660 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4283fff640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 0x7f42841504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f42892df640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 0x7f4284154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f4288ade640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f42817fa640 1 -- 172.21.15.1:0/1361205182 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c042020 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f42817fa640 1 -- 172.21.15.1:0/1361205182 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c033830 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42840ae940 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f42817fa640 1 -- 172.21.15.1:0/1361205182 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c038040 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.663+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42840aef00 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.665+0000 7f42817fa640 1 -- 172.21.15.1:0/1361205182 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f427c045020 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.665+0000 7f42817fa640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 0x7f426406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.666+0000 7f42817fa640 1 -- 172.21.15.1:0/1361205182 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f427c0abb80 con 0x7f428406e6d0 2024-05-09T06:39:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.666+0000 7f4283fff640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 0x7f426406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.666+0000 7f4283fff640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 0x7f426406e320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42841567f0 tx=0x7f427400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.666+0000 7f428da52740 1 Processor -- start 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.666+0000 7f428da52740 1 -- start start 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c42770 0x562110c41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562110b7bd00 con 0x562110c42030 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562110b234c0 con 0x562110c5f650 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562110b77cc0 con 0x562110c42770 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50668/0 (socket says 172.21.15.1:50668) 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f42892df640 1 -- 172.21.15.1:0/1080739222 learned_addr learned my addr 172.21.15.1:0/1080739222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f4288ade640 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f4283fff640 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c42770 0x562110c41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f42892df640 1 -- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c42770 msgr2=0x562110c41af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.667+0000 7f42892df640 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c42770 0x562110c41af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f42892df640 1 -- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 msgr2=0x562110c11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f42892df640 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f42892df640 1 -- 172.21.15.1:0/1080739222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621109597d0 con 0x562110c42030 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f42892df640 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b970 tx=0x7f426c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd0db15d27c5b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f427a7fc640 1 -- 172.21.15.1:0/1080739222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c016070 con 0x562110c42030 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f427a7fc640 1 -- 172.21.15.1:0/1080739222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00dc70 con 0x562110c42030 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f427a7fc640 1 -- 172.21.15.1:0/1080739222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c012460 con 0x562110c42030 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 -- 172.21.15.1:0/1080739222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 msgr2=0x562110c16e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b970 tx=0x7f426c00be30 comp rx=0 tx=0).stop 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 -- 172.21.15.1:0/1080739222 shutdown_connections 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42030 0x562110c16e40 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c42770 0x562110c41af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 --2- 172.21.15.1:0/1080739222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.668+0000 7f428da52740 1 -- 172.21.15.1:0/1080739222 >> 172.21.15.1:0/1080739222 conn(0x562110c14250 msgr2=0x562110c5caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.669+0000 7f428da52740 1 -- 172.21.15.1:0/1080739222 shutdown_connections 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.669+0000 7f428da52740 1 -- 172.21.15.1:0/1080739222 wait complete. 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.669+0000 7f428da52740 1 -- start start 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.669+0000 7f428da52740 1 Processor -- start 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c3d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c60390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562110c16400 con 0x562110c42770 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562110b36b90 con 0x562110c5f650 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562110c157f0 con 0x562110c5a250 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f428da52740 10 client.0 _ll_register_callbacks cb 0x562110b72da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50680/0 (socket says 172.21.15.1:50680) 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 -- 172.21.15.1:0/2775653167 learned_addr learned my addr 172.21.15.1:0/2775653167 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f42892df640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c60390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4283fff640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c3d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 -- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 msgr2=0x562110c60390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c60390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 -- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 msgr2=0x562110c3d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.670+0000 7f4288ade640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c3d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f4288ade640 1 -- 172.21.15.1:0/2775653167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211088f3f0 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f4283fff640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c3d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f4288ade640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f427c0760f0 tx=0x7f427c079da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c51216b33901776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0a7020 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f427c03bdd0 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c00dc70 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f427c08f030 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 10 client.10143 fetch_fsmap learned FSMap version 105 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x56211084f0f0 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00dc70 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f427c0ab040 con 0x562110c42770 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 10 client.10143 fetch_fsmap finished waiting for FSMap version 105 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 10 client.10143 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 20 client.10143 populate_metadata read hostname 'smithi001' 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5ab70 0x562110c5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 4 client.10143 mds_command: new command op to 9957 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 --> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562110b89430 con 0x562110c5ab70 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.671+0000 7f4283fff640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5ab70 0x562110c5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.672+0000 7f42637fe640 10 client.10143 ms_handle_connect on v2:172.21.15.1:6836/3597643567 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.672+0000 7f4283fff640 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5ab70 0x562110c5af50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5405e1dccc48bdab server_cookie=84c4a36f007b5ed1 in_seq=0 out_seq=0 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.672+0000 7f42637fe640 1 -- 172.21.15.1:0/2775653167 <== mds.0 v2:172.21.15.1:6836/3597643567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562110b89430 con 0x562110c5ab70 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.672+0000 7f42637fe640 10 client.10143 handle_command_reply: tid=0 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.672+0000 7f428da52740 1 client.10143 shutdown 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 msgr2=0x562110c6d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f427c0760f0 tx=0x7f427c079da0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 shutdown_connections 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5ab70 0x562110c5af50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c60390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c5f650 0x562110c3d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 --2- 172.21.15.1:0/2775653167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c42770 0x562110c6d8a0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f42637fe640 0 client.10143 ms_handle_reset on v2:172.21.15.1:6836/3597643567 2024-05-09T06:39:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 >> 172.21.15.1:0/2775653167 conn(0x562110c42030 msgr2=0x562110c5caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 shutdown_connections 2024-05-09T06:39:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 1 -- 172.21.15.1:0/2775653167 wait complete. 2024-05-09T06:39:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.673+0000 7f428da52740 20 client.10143 trim_cache size 0 max 16384 2024-05-09T06:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 Processor -- start 2024-05-09T06:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 -- start start 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c5c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c15950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4284200df0 con 0x562110c5f650 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f42841ffdf0 con 0x562110c42770 2024-05-09T06:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.688+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f428415dba0 con 0x562110c5a250 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50692/0 (socket says 172.21.15.1:50692) 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 -- 172.21.15.1:0/3141799109 learned_addr learned my addr 172.21.15.1:0/3141799109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4283fff640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c15950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4288ade640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c5c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 -- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 msgr2=0x562110c15950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c15950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 -- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 msgr2=0x562110c5c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c5c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 -- 172.21.15.1:0/3141799109 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562110c5b2c0 con 0x562110c5f650 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4288ade640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c5c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f42892df640 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b940 tx=0x7f426c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28112147d8fbe190 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4279ffb640 1 -- 172.21.15.1:0/3141799109 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c02d070 con 0x562110c5f650 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4279ffb640 1 -- 172.21.15.1:0/3141799109 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00b040 con 0x562110c5f650 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.689+0000 7f4279ffb640 1 -- 172.21.15.1:0/3141799109 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c016040 con 0x562110c5f650 2024-05-09T06:39:11.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- 172.21.15.1:0/3141799109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 msgr2=0x562110c15e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b940 tx=0x7f426c013000 comp rx=0 tx=0).stop 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- 172.21.15.1:0/3141799109 shutdown_connections 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c15e90 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110c15950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 --2- 172.21.15.1:0/3141799109 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c5c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- 172.21.15.1:0/3141799109 >> 172.21.15.1:0/3141799109 conn(0x562110c42030 msgr2=0x562110c3d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- 172.21.15.1:0/3141799109 shutdown_connections 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- 172.21.15.1:0/3141799109 wait complete. 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 -- start start 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.690+0000 7f428da52740 1 Processor -- start 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110cd4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562110c12ae0 con 0x562110c5f650 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f42841ff7a0 con 0x562110c42770 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x562110c136a0 con 0x562110c5a250 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f428da52740 10 client.0 _ll_register_callbacks cb 0x7f427c07a3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f42892df640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50694/0 (socket says 172.21.15.1:50694) 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f42892df640 1 -- 172.21.15.1:0/1027082062 learned_addr learned my addr 172.21.15.1:0/1027082062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f4283fff640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110cd4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.691+0000 7f4288ade640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 -- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 msgr2=0x562110cd4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110cd4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 -- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 msgr2=0x562110c6c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c6c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 -- 172.21.15.1:0/1027082062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562110c15390 con 0x562110c5f650 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42892df640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d738468b82613413 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c02d030 con 0x562110c5f650 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f426c006c30 con 0x562110c5f650 2024-05-09T06:39:11.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c00b040 con 0x562110c5f650 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f426c03d050 con 0x562110c5f650 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 10 client.10149 fetch_fsmap learned FSMap version 105 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x562110c15ad0 con 0x562110c5f650 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00b040 con 0x562110c5f650 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f426c040020 con 0x562110c5f650 2024-05-09T06:39:11.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 10 client.10149 fetch_fsmap finished waiting for FSMap version 105 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 10 client.10149 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 20 client.10149 populate_metadata read hostname 'smithi001' 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5c180 0x562110c602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 4 client.10149 mds_command: new command op to 9957 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.692+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 --> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562110b83780 con 0x562110c5c180 2024-05-09T06:39:11.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.693+0000 7f4288ade640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5c180 0x562110c602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.693+0000 7f42637fe640 10 client.10149 ms_handle_connect on v2:172.21.15.1:6836/3597643567 2024-05-09T06:39:11.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.693+0000 7f4288ade640 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5c180 0x562110c602b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aec302484fb1a0 server_cookie=93aaf90bacdf42d6 in_seq=0 out_seq=0 2024-05-09T06:39:11.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f42637fe640 1 -- 172.21.15.1:0/1027082062 <== mds.0 v2:172.21.15.1:6836/3597643567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562110b83780 con 0x562110c5c180 2024-05-09T06:39:11.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f42637fe640 10 client.10149 handle_command_reply: tid=0 2024-05-09T06:39:11.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 client.10149 shutdown 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 msgr2=0x562110c5fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c004730 comp rx=0 tx=0).stop 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 shutdown_connections 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:6836/3597643567,v1:172.21.15.1:6837/3597643567] conn(0x562110c5c180 0x562110c602b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562110c5f650 0x562110c5fd70 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x562110c5a250 0x562110cd4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f42637fe640 0 client.10149 ms_handle_reset on v2:172.21.15.1:6836/3597643567 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 --2- 172.21.15.1:0/1027082062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562110c42770 0x562110c6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 >> 172.21.15.1:0/1027082062 conn(0x562110c42030 msgr2=0x562110c596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 shutdown_connections 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 1 -- 172.21.15.1:0/1027082062 wait complete. 2024-05-09T06:39:11.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.708+0000 7f428da52740 20 client.10149 trim_cache size 0 max 16384 2024-05-09T06:39:11.691 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "message": "", 2024-05-09T06:39:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0 2024-05-09T06:39:11.691 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 msgr2=0x7f426406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 0x7f426406e320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42841567f0 tx=0x7f427400b040 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 msgr2=0x7f4284153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 shutdown_connections 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f426406be60 0x7f426406e320 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f428415c4a0 0x7f4284154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4284158a50 0x7f42841504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 --2- 172.21.15.1:0/1361205182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428406e6d0 0x7f4284153b90 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 >> 172.21.15.1:0/1361205182 conn(0x7f42840cc4d0 msgr2=0x7f4284156820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 shutdown_connections 2024-05-09T06:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.709+0000 7f428ad42640 1 -- 172.21.15.1:0/1361205182 wait complete. 2024-05-09T06:39:11.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:39:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 Processor -- start 2024-05-09T06:39:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 -- start start 2024-05-09T06:39:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4690161950 0x7f46900d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46900d4940 0x7f46900d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46900b33c0 con 0x7f46900d0f80 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f46900d4dd0 con 0x7f46900d4940 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.826+0000 7f4695bfb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f46900b3660 con 0x7f4690161950 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46900d4940 0x7f46900d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f46900d4940 0x7f46900d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50708/0 (socket says 172.21.15.1:50708) 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 -- 172.21.15.1:0/1878775539 learned_addr learned my addr 172.21.15.1:0/1878775539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468f7fe640 1 --2- 172.21.15.1:0/1878775539 >> v2:172.21.15.6:3300/0 conn(0x7f4690161950 0x7f46900d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 -- 172.21.15.1:0/1878775539 >> v2:172.21.15.6:3300/0 conn(0x7f4690161950 msgr2=0x7f46900d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 --2- 172.21.15.1:0/1878775539 >> v2:172.21.15.6:3300/0 conn(0x7f4690161950 0x7f46900d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 -- 172.21.15.1:0/1878775539 >> v1:172.21.15.6:6789/0 conn(0x7f46900d0f80 legacy=0x7f46900cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.827+0000 7f468effd640 1 -- 172.21.15.1:0/1878775539 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46900b3940 con 0x7f46900d4940 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f468effd640 1 --2- 172.21.15.1:0/1878775539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46900d4940 0x7f46900d2940 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f469006b7f0 tx=0x7f468402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4e0c6b28ed68f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f468e7fc640 1 -- 172.21.15.1:0/1878775539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684037070 con 0x7f46900d4940 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f468e7fc640 1 -- 172.21.15.1:0/1878775539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4684032440 con 0x7f46900d4940 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f468e7fc640 1 -- 172.21.15.1:0/1878775539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684032810 con 0x7f46900d4940 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 -- 172.21.15.1:0/1878775539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46900d4940 msgr2=0x7f46900d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 --2- 172.21.15.1:0/1878775539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46900d4940 0x7f46900d2940 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f469006b7f0 tx=0x7f468402a270 comp rx=0 tx=0).stop 2024-05-09T06:39:11.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 -- 172.21.15.1:0/1878775539 shutdown_connections 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 --2- 172.21.15.1:0/1878775539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46900d4940 0x7f46900d2940 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 --2- 172.21.15.1:0/1878775539 >> v2:172.21.15.6:3300/0 conn(0x7f4690161950 0x7f46900d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 -- 172.21.15.1:0/1878775539 >> 172.21.15.1:0/1878775539 conn(0x7f46900d3e20 msgr2=0x7f46900d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 -- 172.21.15.1:0/1878775539 shutdown_connections 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.828+0000 7f4695bfb640 1 -- 172.21.15.1:0/1878775539 wait complete. 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 Processor -- start 2024-05-09T06:39:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 -- start start 2024-05-09T06:39:11.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46900d0f80 0x7f469015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46901f4260 0x7f46900b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46900b52d0 con 0x7f4690161950 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46900b5150 con 0x7f46900d0f80 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.830+0000 7f4695bfb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f46900b5450 con 0x7f46901f4260 2024-05-09T06:39:11.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50714/0 (socket says 172.21.15.1:50714) 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 -- 172.21.15.1:0/2076684486 learned_addr learned my addr 172.21.15.1:0/2076684486 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468f7fe640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46900d0f80 0x7f469015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468ffff640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46901f4260 0x7f46900b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 -- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46901f4260 msgr2=0x7f46900b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46901f4260 0x7f46900b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 -- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46900d0f80 msgr2=0x7f469015b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:11.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46900d0f80 0x7f469015b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 -- 172.21.15.1:0/2076684486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46900b6de0 con 0x7f4690161950 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f468effd640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f468402a2b0 tx=0x7f46840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684041020 con 0x7f4690161950 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46840306c0 con 0x7f4690161950 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46900b70c0 con 0x7f4690161950 2024-05-09T06:39:11.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684037040 con 0x7f4690161950 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.831+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46900b7680 con 0x7f4690161950 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.833+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46900752b0 con 0x7f4690161950 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.833+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4684044020 con 0x7f4690161950 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.833+0000 7f4677fff640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 0x7f467c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.834+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6187+0+0 (secure 0 0 0) 0x7f46840ab630 con 0x7f4690161950 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.834+0000 7f468f7fe640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 0x7f467c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:11.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.834+0000 7f468f7fe640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 0x7f467c06e320 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f46800079a0 tx=0x7f4680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:11.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:11.838+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f468404b030 con 0x7f4690161950 2024-05-09T06:39:12.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.061+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46900d21e0 con 0x7f4690161950 2024-05-09T06:39:12.042 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:39:12.062+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:39:12.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.063+0000 7f4677fff640 1 -- 172.21.15.1:0/2076684486 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f4684079840 con 0x7f4690161950 2024-05-09T06:39:12.043 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 msgr2=0x7f467c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 0x7f467c06e320 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f46800079a0 tx=0x7f4680008040 comp rx=0 tx=0).stop 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 msgr2=0x7f46900a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f468402a2b0 tx=0x7f46840046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 shutdown_connections 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f467c06be60 0x7f467c06e320 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f46901f4260 0x7f46900b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4690161950 0x7f46900a1570 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 --2- 172.21.15.1:0/2076684486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46900d0f80 0x7f469015b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.066+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 >> 172.21.15.1:0/2076684486 conn(0x7f46900d3e20 msgr2=0x7f46900b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.067+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 shutdown_connections 2024-05-09T06:39:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.067+0000 7f4695bfb640 1 -- 172.21.15.1:0/2076684486 wait complete. 2024-05-09T06:39:12.048 INFO:tasks.ceph.mds.a.smithi001.stderr: -2296> 2024-05-09T06:38:59.424+0000 7f70f2c49640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:39:12.058 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-09T06:39:12.077 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:39:12.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.122+0000 7fd7a240e280 1 -- start start 2024-05-09T06:39:12.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.122+0000 7fd7a240e280 1 Processor -- start 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565269234c00 0x5652692a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565269235c00 0x5652692a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526a008600 con 0x565269235800 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56526a008480 con 0x565269235c00 2024-05-09T06:39:12.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56526a008300 con 0x565269234c00 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50748/0 (socket says 172.21.15.1:50748) 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 -- 172.21.15.1:0/955327378 learned_addr learned my addr 172.21.15.1:0/955327378 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565269235c00 0x5652692a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79feaa640 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565269234c00 0x5652692a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 -- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565269234c00 msgr2=0x5652692a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565269234c00 0x5652692a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 -- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565269235c00 msgr2=0x5652692a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565269235c00 0x5652692a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 -- 172.21.15.1:0/955327378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526926a780 con 0x565269235800 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x56526a04ea80 tx=0x565269fc9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d40878ada0b919 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79d6a5640 1 -- 172.21.15.1:0/955327378 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a9c0 con 0x565269235800 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79d6a5640 1 -- 172.21.15.1:0/955327378 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565269260e00 con 0x565269235800 2024-05-09T06:39:12.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd79d6a5640 1 -- 172.21.15.1:0/955327378 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a680 con 0x565269235800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526926a3c0 con 0x565269235800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56526926b680 con 0x565269235800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.123+0000 7fd7a240e280 4 waiting for MDS map... 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd79d6a5640 1 -- 172.21.15.1:0/955327378 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x56526a10c600 con 0x565269235800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd79d6a5640 1 -- 172.21.15.1:0/955327378 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x56526928f180 con 0x565269235800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd7a240e280 4 Got MDS map 106 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd7a240e280 10 main: JournalTool::main 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd7a240e280 4 main: JournalTool: connecting to RADOS... 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd7a240e280 1 Processor -- start 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.124+0000 7fd7a240e280 1 -- start start 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158000 0x5652692a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526a008180 con 0x56526a158800 2024-05-09T06:39:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x565269296480 con 0x56526a158000 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x565269296900 con 0x56526a158400 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50764/0 (socket says 172.21.15.1:50764) 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79f6a9640 1 -- 172.21.15.1:0/443167599 learned_addr learned my addr 172.21.15.1:0/443167599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158000 0x5652692a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 -- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158000 msgr2=0x5652692a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158000 0x5652692a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 -- 172.21.15.1:0/443167599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 msgr2=0x5652692a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79feaa640 1 -- 172.21.15.1:0/443167599 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526926b4a0 con 0x56526a158400 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.125+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd79feaa640 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a5b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x56526a0247e0 tx=0x56526a026c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da00cf76ee694d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd79b6a1640 1 -- 172.21.15.1:0/443167599 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a9c0 con 0x56526a158400 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd79b6a1640 1 -- 172.21.15.1:0/443167599 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565269260a80 con 0x56526a158400 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd79b6a1640 1 -- 172.21.15.1:0/443167599 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a680 con 0x56526a158400 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 -- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 msgr2=0x5652692a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a5b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x56526a0247e0 tx=0x56526a026c00 comp rx=0 tx=0).stop 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 -- 172.21.15.1:0/443167599 shutdown_connections 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158800 0x5652692a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a5b80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 --2- 172.21.15.1:0/443167599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158000 0x5652692a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 -- 172.21.15.1:0/443167599 >> 172.21.15.1:0/443167599 conn(0x56526a00bc00 msgr2=0x56526931e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 -- 172.21.15.1:0/443167599 shutdown_connections 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.126+0000 7fd7a240e280 1 -- 172.21.15.1:0/443167599 wait complete. 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.127+0000 7fd7a240e280 1 Processor -- start 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.127+0000 7fd7a240e280 1 -- start start 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.127+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 0x5652692a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526a009800 con 0x56526a158000 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56526a009680 con 0x56526a158800 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56526a009500 con 0x56526a158400 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50778/0 (socket says 172.21.15.1:50778) 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1874197168 learned_addr learned my addr 172.21.15.1:0/1874197168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 0x5652692a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 msgr2=0x5652692a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 msgr2=0x5652692a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 0x5652692a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1874197168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526926a000 con 0x56526a158000 2024-05-09T06:39:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 0x5652692a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x56526a025560 tx=0x56526a026d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd799e9e640 1 -- 172.21.15.1:0/1874197168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a9c0 con 0x56526a158000 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd799e9e640 1 -- 172.21.15.1:0/1874197168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565269260c40 con 0x56526a158000 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56526926a960 con 0x56526a158000 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd799e9e640 1 -- 172.21.15.1:0/1874197168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a680 con 0x56526a158000 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526926b860 con 0x56526a158000 2024-05-09T06:39:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.128+0000 7fd7a240e280 4 main: JournalTool: resolving pool 17 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd799e9e640 1 -- 172.21.15.1:0/1874197168 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x56526a170c00 con 0x56526a158000 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd799e9e640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 0x5652692a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd799e9e640 1 -- 172.21.15.1:0/1874197168 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x56526a15fa00 con 0x56526a158000 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd7a240e280 4 main: JournalTool: creating IoCtx.. 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd7a240e280 4 main: Executing for rank 0 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 0x5652692a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.130+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 0x5652692a7700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x56526a0247e0 tx=0x565269fc85a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 -- start start 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 Processor -- start 2024-05-09T06:39:12.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a159000 0x5652692a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a177000 0x5652692a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56526a009380 con 0x56526a177400 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56526a009200 con 0x56526a159000 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd7a240e280 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x56526a008c00 con 0x56526a177000 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd79f6a9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50794/0 (socket says 172.21.15.1:50794) 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.131+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 learned_addr learned my addr 172.21.15.1:0/1786063320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a177000 0x5652692a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a159000 0x5652692a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a177000 msgr2=0x5652692a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a177000 0x5652692a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a159000 msgr2=0x5652692a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a159000 0x5652692a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526926ba40 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x56526a04f980 tx=0x5652692ead80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3ea7ae25ec271c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd796e98640 1 -- 172.21.15.1:0/1786063320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925ab60 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd796e98640 1 -- 172.21.15.1:0/1786063320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5652692601c0 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd796e98640 1 -- 172.21.15.1:0/1786063320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56526925a820 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526926a000 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56526926a960 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 4 waiting for MDS map... 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd796e98640 1 -- 172.21.15.1:0/1786063320 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x56526a15e800 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd796e98640 1 -- 172.21.15.1:0/1786063320 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x56526928f500 con 0x56526a177400 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 4 Got MDS map 106 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56526a177800 0x56526961e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.132+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526a177c00 con 0x56526a177800 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.133+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56526a177800 0x56526961e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.133+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56526a177800 0x56526961e000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=c792e9ca5e1d4f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.136+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1786063320 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v123'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565269336000 con 0x56526a177800 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.136+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x565269603800 0x56526961e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.136+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x565269603c00 con 0x565269603800 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.136+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x565269603800 0x56526961e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.136+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x565269603800 0x56526961e580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=b0b0718a5d1d73f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd79feaa640 1 -- 172.21.15.1:0/1786063320 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v123'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565269336500 con 0x565269603800 2024-05-09T06:39:12.122 INFO:teuthology.orchestra.run.smithi001.stdout:writing EResetJournal entry 2024-05-09T06:39:12.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526962a000 con 0x565269603800 2024-05-09T06:39:12.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56526962a400 0x56526961eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526962a800 con 0x56526962a400 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56526962ac00 0x56526961f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526962b000 con 0x56526962ac00 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 0x56526961f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526962b800 con 0x56526962b400 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526962bc00 con 0x56526962ac00 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x565269630000 con 0x56526a177800 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56526962a400 0x56526961eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56526962ac00 0x56526961f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 0x56526961f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.140+0000 7fd79feaa640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56526962ac00 0x56526961f080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=8f2f65082468aaba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.141+0000 7fd79f6a9640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 0x56526961f600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=eb22b7cb3c19c501 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.141+0000 7fd79feaa640 1 -- 172.21.15.1:0/1786063320 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(3 200.00000001 [delete] v123'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565269336500 con 0x565269603800 2024-05-09T06:39:12.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.141+0000 7fd7a06ab640 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56526962a400 0x56526961eb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=fa64d555fa547fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.141+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1786063320 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(8 200.00000006 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565269336000 con 0x56526a177800 2024-05-09T06:39:12.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.142+0000 7fd79feaa640 1 -- 172.21.15.1:0/1786063320 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(7 200.00000005 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565269336500 con 0x56526962ac00 2024-05-09T06:39:12.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.142+0000 7fd79feaa640 1 -- 172.21.15.1:0/1786063320 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(5 200.00000003 [delete] v123'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565269336500 con 0x56526962ac00 2024-05-09T06:39:12.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.143+0000 7fd7a06ab640 1 -- 172.21.15.1:0/1786063320 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(4 200.00000002 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565269336000 con 0x56526962a400 2024-05-09T06:39:12.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.143+0000 7fd795695640 1 -- 172.21.15.1:0/1786063320 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56526a159c00 con 0x565269603800 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd79feaa640 1 -- 172.21.15.1:0/1786063320 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v123'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565269336500 con 0x565269603800 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(6 200.00000004 [delete] v123'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565269336780 con 0x56526962b400 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56526962ac00 msgr2=0x56526961f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x56526962ac00 0x56526961f080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56526a177800 msgr2=0x56526961e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x56526a177800 0x56526961e000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x565269603800 msgr2=0x56526961e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x565269603800 0x56526961e580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56526962a400 msgr2=0x56526961eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x56526962a400 0x56526961eb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 reap_dead start 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 msgr2=0x56526961f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 0x56526961f600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd79f6a9640 1 -- 172.21.15.1:0/1786063320 reap_dead start 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 msgr2=0x5652692a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x56526a04f980 tx=0x5652692ead80 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 shutdown_connections 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x56526962b400 0x56526961f600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1786063320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a177400 0x5652692a5600 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 >> 172.21.15.1:0/1786063320 conn(0x56526a158c00 msgr2=0x56526931e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 shutdown_connections 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.145+0000 7fd7a240e280 1 -- 172.21.15.1:0/1786063320 wait complete. 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 msgr2=0x5652692a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 0x5652692a7700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x56526a0247e0 tx=0x565269fc85a0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 msgr2=0x5652692a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x56526a025560 tx=0x56526a026d20 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 shutdown_connections 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x56526a176c00 0x5652692a7700 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56526a158800 0x5652692a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x56526a158400 0x5652692a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/1874197168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56526a158000 0x5652692a7180 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 >> 172.21.15.1:0/1874197168 conn(0x56526a00bc00 msgr2=0x56526931e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 shutdown_connections 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/1874197168 wait complete. 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 msgr2=0x5652692a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x56526a04ea80 tx=0x565269fc9f20 comp rx=0 tx=0).stop 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 shutdown_connections 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x565269235c00 0x5652692a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565269235800 0x5652692a4b00 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 --2- 172.21.15.1:0/955327378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565269234c00 0x5652692a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 >> 172.21.15.1:0/955327378 conn(0x565269235000 msgr2=0x56526931f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 shutdown_connections 2024-05-09T06:39:12.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.146+0000 7fd7a240e280 1 -- 172.21.15.1:0/955327378 wait complete. 2024-05-09T06:39:12.132 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.074017, result: writing EResetJournal entry 2024-05-09T06:39:12.133 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-09T06:39:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.230+0000 7f6286221740 1 -- start start 2024-05-09T06:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.230+0000 7f6286221740 1 Processor -- start 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 0x55a95f9db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960742800 0x55a95f9db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a96075a480 con 0x55a95f967c00 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a96075a600 con 0x55a960742400 2024-05-09T06:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a96075a780 con 0x55a960742800 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50808/0 (socket says 172.21.15.1:50808) 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 -- 172.21.15.1:0/2320330621 learned_addr learned my addr 172.21.15.1:0/2320330621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6284646640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960742800 0x55a95f9db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283e45640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 0x55a95f9db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 -- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960742800 msgr2=0x55a95f9db080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960742800 0x55a95f9db080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 -- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 msgr2=0x55a95f9db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 0x55a95f9db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 -- 172.21.15.1:0/2320330621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95f993680 con 0x55a95f967c00 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283e45640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 0x55a95f9db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.231+0000 7f6283644640 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x55a96079b5f0 tx=0x55a9607009c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac999ee880f4d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6281640640 1 -- 172.21.15.1:0/2320330621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cb60 con 0x55a95f967c00 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6281640640 1 -- 172.21.15.1:0/2320330621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a95f9b01c0 con 0x55a95f967c00 2024-05-09T06:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6281640640 1 -- 172.21.15.1:0/2320330621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cd00 con 0x55a95f967c00 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95f9932c0 con 0x55a95f967c00 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a95f992b40 con 0x55a95f967c00 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 4 waiting for MDS map... 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6281640640 1 -- 172.21.15.1:0/2320330621 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55a960784400 con 0x55a95f967c00 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6281640640 1 -- 172.21.15.1:0/2320330621 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x55a95f9bf180 con 0x55a95f967c00 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 4 Got MDS map 106 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 4 datascan.main: Using metadata pool output 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 4 datascan.main: connecting to RADOS... 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.232+0000 7f6286221740 1 Processor -- start 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 -- start start 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a95f9c6180 con 0x55a960857800 2024-05-09T06:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a96075af00 con 0x55a960857400 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a96075b080 con 0x55a960857c00 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50818/0 (socket says 172.21.15.1:50818) 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 -- 172.21.15.1:0/360880306 learned_addr learned my addr 172.21.15.1:0/360880306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6284646640 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283644640 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 -- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857c00 msgr2=0x55a95f9dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 -- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857400 msgr2=0x55a95f9dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.233+0000 7f6283e45640 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6283e45640 1 -- 172.21.15.1:0/360880306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95f9923c0 con 0x55a960857800 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6283e45640 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x55a96085aba0 tx=0x55a96085ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a4747f072bc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f627f63c640 1 -- 172.21.15.1:0/360880306 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cb60 con 0x55a960857800 2024-05-09T06:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f627f63c640 1 -- 172.21.15.1:0/360880306 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a95f9b0c40 con 0x55a960857800 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f627f63c640 1 -- 172.21.15.1:0/360880306 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cd00 con 0x55a960857800 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 -- 172.21.15.1:0/360880306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 msgr2=0x55a95f9dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x55a96085aba0 tx=0x55a96085ec00 comp rx=0 tx=0).stop 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 -- 172.21.15.1:0/360880306 shutdown_connections 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 --2- 172.21.15.1:0/360880306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 -- 172.21.15.1:0/360880306 >> 172.21.15.1:0/360880306 conn(0x55a960742000 msgr2=0x55a9606eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 -- 172.21.15.1:0/360880306 shutdown_connections 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.234+0000 7f6286221740 1 -- 172.21.15.1:0/360880306 wait complete. 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 Processor -- start 2024-05-09T06:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 -- start start 2024-05-09T06:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a96075b380 con 0x55a960857800 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a96075b200 con 0x55a960857c00 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.235+0000 7f6286221740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55a96075b500 con 0x55a960857400 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50826/0 (socket says 172.21.15.1:50826) 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 learned_addr learned my addr 172.21.15.1:0/2958720384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857400 msgr2=0x55a95f9dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960857400 0x55a95f9dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857c00 msgr2=0x55a95f9dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960857c00 0x55a95f9dc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95f993a40 con 0x55a960857800 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x55a96085b860 tx=0x55a960701f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f627de39640 1 -- 172.21.15.1:0/2958720384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cb60 con 0x55a960857800 2024-05-09T06:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f627de39640 1 -- 172.21.15.1:0/2958720384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a95f9b0a80 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a95f9934a0 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f627de39640 1 -- 172.21.15.1:0/2958720384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a95f98cd00 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95f993c20 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.236+0000 7f6286221740 4 datascan.init: resolving metadata pool 17 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.237+0000 7f627de39640 1 -- 172.21.15.1:0/2958720384 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55a9608bac00 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f627de39640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 0x55a95f9dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f627de39640 1 -- 172.21.15.1:0/2958720384 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55a960740000 con 0x55a960857800 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 0x55a95f9dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6286221740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6286221740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6286221740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a9608b7000 0x55a95f9dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7400 con 0x55a9608b7000 2024-05-09T06:39:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a9608b7000 0x55a95f9dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 0x55a95f9dd700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55a95f9a7470 tx=0x55a95fcf4000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.238+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a9608b7000 0x55a95f9dcc00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.239+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.239+0000 7f627ee3b640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55a95fcf8000 0x55a95fcfe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.239+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf8400 con 0x55a95fcf8000 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.239+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55a95fcf8000 0x55a95fcfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.239+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55a95fcf8000 0x55a95fcfe000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.242+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.242+0000 7f627f63c640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55a95fcf8c00 0x55a95fcfe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.242+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9000 con 0x55a95fcf8c00 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.242+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55a95fcf8c00 0x55a95fcfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.242+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55a95fcf8c00 0x55a95fcfe580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a95fcf8c00 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf8800 con 0x55a9608b7000 2024-05-09T06:39:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9400 con 0x55a9608b7000 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a960856c00 con 0x55a95fcf8000 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.243+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f627f63c640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55a95fcf8800 0x55a95fcfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9400 con 0x55a95fcf8800 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55a95fcf8800 0x55a95fcfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55a95fcf8800 0x55a95fcfeb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f627ee3b640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55a960856800 0x55a95fcff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a960856400 con 0x55a960856800 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.244+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55a960856800 0x55a95fcff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55a960856800 0x55a95fcff600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7400 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf8400 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.245+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9c00 con 0x55a95fcf8c00 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a95fcf8c00 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9800 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14000 con 0x55a95fcf8000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(13 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f627ee3b640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55a960856c00 0x55a95fcffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a960856400 con 0x55a960856c00 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.246+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55a960856c00 0x55a95fcffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f6283e45640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55a960856c00 0x55a95fcffb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14400 con 0x55a95fcf8800 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(15 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f627ee3b640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a95fd15400 0x55a95fd00100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15800 con 0x55a95fd15400 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.247+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a95fd15400 0x55a95fd00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.248+0000 7f6283644640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a95fd15400 0x55a95fd00100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.248+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.248+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14800 con 0x55a95fd15400 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.248+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15c00 con 0x55a960856800 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14c00 con 0x55a960856c00 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(19 [pgnls start_epoch 123 out=81b] v122'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7800 con 0x55a960856c00 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6286221740 10 datascan.operator(): handling object 1099511627778.0 2024-05-09T06:39:12.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.249+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7c00 con 0x55a960856c00 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(21 10000000002.00000000 [call] v123'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9000 con 0x55a960856800 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15000 con 0x55a960856800 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(23 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f627ee3b640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55a95fd14000 0x55a95fd00680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15c00 con 0x55a95fd14000 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.251+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55a95fd14000 0x55a95fd00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f6284646640 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55a95fd14000 0x55a95fd00680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(24 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15800 con 0x55a960856800 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 5 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15000 con 0x55a9608b7000 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(26 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.252+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14800 con 0x55a95fd15400 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(27 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14c00 con 0x55a960856c00 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 5 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a000 con 0x55a960856800 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.253+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14400 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.254+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=81b] v121'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.254+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9c00 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.254+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.254+0000 7f6286221740 10 datascan.operator(): handling object 1099511628279.0 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.254+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9400 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.255+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v123'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.255+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf8400 con 0x55a9608b7000 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.255+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(33 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.255+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9000 con 0x55a960856800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 7 ==== osd_op_reply(34 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a400 con 0x55a95fcf8000 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(35 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a000 con 0x55a960856800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 8 ==== osd_op_reply(36 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a800 con 0x55a960856800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 9 ==== osd_op_reply(37 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.256+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a400 con 0x55a95fcf8000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(38 [pgnls start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2ac00 con 0x55a95fcf8800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(39 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2bc00 con 0x55a95fd14000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(40 [pgnls start_epoch 123 out=81b] v121'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.257+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9800 con 0x55a95fd14000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.258+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.258+0000 7f6286221740 10 datascan.operator(): handling object 1099511627780.1 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.258+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7400 con 0x55a95fcf8800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.259+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(42 10000000004.00000000 [call] v123'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.259+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a960856400 con 0x55a960856800 2024-05-09T06:39:12.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.260+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 10 ==== osd_op_reply(43 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.260+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9000 con 0x55a95fd15400 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.260+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(44 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.260+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2b000 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.261+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(45 [pgnls start_epoch 123 out=81b] v122'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.261+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9800 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.261+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.261+0000 7f6286221740 10 datascan.operator(): handling object 1099511628280.0 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.261+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9c00 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.262+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v123'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.262+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf9400 con 0x55a9608b7000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.262+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 9 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.262+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2b400 con 0x55a95fcf8800 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.263+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(49 [pgnls start_epoch 123 out=81b] v123'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.263+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55a95fcf8400 con 0x55a95fcf8800 2024-05-09T06:39:12.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.263+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.263+0000 7f6286221740 10 datascan.operator(): handling object 1099511627780.0 2024-05-09T06:39:12.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.263+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15000 con 0x55a95fcf8800 2024-05-09T06:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.264+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 10 ==== osd_op_reply(51 10000000004.00000000 [call] v123'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.264+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14800 con 0x55a95fcf8c00 2024-05-09T06:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.264+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a95fcf8c00 2024-05-09T06:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.264+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2b800 con 0x55a95fcf8000 2024-05-09T06:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.265+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(53 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.265+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15c00 con 0x55a95fd14000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.265+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 7 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.265+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7400 con 0x55a95fd14000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 8 ==== osd_op_reply(55 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15800 con 0x55a95fcf8c00 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a95fcf8c00 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a000 con 0x55a9608b7000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.0 v2:172.21.15.1:6819/3367065226 10 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a9608b7000 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.266+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14c00 con 0x55a960856800 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.3 v2:172.21.15.1:6800/2131549113 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a960856800 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a800 con 0x55a95fcf8800 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14800 con 0x55a960856c00 2024-05-09T06:39:12.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a400 con 0x55a95fd15400 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.267+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15800 con 0x55a95fcf8800 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 12 ==== osd_op_reply(62 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2bc00 con 0x55a95fcf8000 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7800 con 0x55a95fd15400 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.268+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2ac00 con 0x55a95fcf8800 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 13 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a9608b7c00 con 0x55a95fd14000 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 9 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2b400 con 0x55a960856c00 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 <== osd.2 v2:172.21.15.1:6808/1632480843 7 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada780 con 0x55a960856c00 2024-05-09T06:39:12.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.269+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15000 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.270+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.270+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a000 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.270+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 9 ==== osd_op_reply(69 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.270+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15800 con 0x55a95fd15400 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 7 ==== osd_op_reply(70 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a800 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.5 v2:172.21.15.6:6801/2308051940 10 ==== osd_op_reply(71 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fcf8000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd15c00 con 0x55a95fcf8800 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.1 v2:172.21.15.1:6816/1858675679 14 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fcf8800 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.271+0000 7f627f63c640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd2a400 con 0x55a95fd15400 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6283644640 1 -- 172.21.15.1:0/2958720384 <== osd.7 v2:172.21.15.6:6816/3240730649 8 ==== osd_op_reply(73 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada000 con 0x55a95fd15400 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f627ee3b640 1 -- 172.21.15.1:0/2958720384 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55a95fd14c00 con 0x55a95fd14000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6284646640 1 -- 172.21.15.1:0/2958720384 <== osd.4 v2:172.21.15.6:6800/3928343737 10 ==== osd_op_reply(74 [pgnls start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a95fada500 con 0x55a95fd14000 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a9608b7000 msgr2=0x55a95f9dcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55a9608b7000 0x55a95f9dcc00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55a95fcf8800 msgr2=0x55a95fcfeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55a95fcf8800 0x55a95fcfeb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55a960856c00 msgr2=0x55a95fcffb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55a960856c00 0x55a95fcffb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55a960856800 msgr2=0x55a95fcff600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55a960856800 0x55a95fcff600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 reap_dead start 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55a95fd14000 msgr2=0x55a95fd00680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55a95fd14000 0x55a95fd00680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55a95fcf8000 msgr2=0x55a95fcfe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55a95fcf8000 0x55a95fcfe000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55a95fcf8c00 msgr2=0x55a95fcfe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55a95fcf8c00 0x55a95fcfe580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a95fd15400 msgr2=0x55a95fd00100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55a95fd15400 0x55a95fd00100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6283e45640 1 -- 172.21.15.1:0/2958720384 reap_dead start 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 msgr2=0x55a95f9dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 0x55a95f9dd700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x55a95f9a7470 tx=0x55a95fcf4000 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 msgr2=0x55a95f9dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x55a96085b860 tx=0x55a960701f80 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 shutdown_connections 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55a9608b6c00 0x55a95f9dd700 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 --2- 172.21.15.1:0/2958720384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a960857800 0x55a95f9dc100 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 >> 172.21.15.1:0/2958720384 conn(0x55a960742000 msgr2=0x55a9606eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 shutdown_connections 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.272+0000 7f6286221740 1 -- 172.21.15.1:0/2958720384 wait complete. 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 msgr2=0x55a95f9dbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x55a96079b5f0 tx=0x55a9607009c0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 shutdown_connections 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55a960742800 0x55a95f9db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a95f967c00 0x55a95f9dbb80 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 --2- 172.21.15.1:0/2320330621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a960742400 0x55a95f9db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 >> 172.21.15.1:0/2320330621 conn(0x55a95f967000 msgr2=0x55a9606eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 shutdown_connections 2024-05-09T06:39:12.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.273+0000 7f6286221740 1 -- 172.21.15.1:0/2320330621 wait complete. 2024-05-09T06:39:12.261 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.128254, result: 2024-05-09T06:39:12.261 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-09T06:39:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.364+0000 7fba6dc16740 1 -- start start 2024-05-09T06:39:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 Processor -- start 2024-05-09T06:39:12.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 0x55d744172b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7440ffc00 0x55d744173080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d744e96180 con 0x55d7440fec00 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d744e96300 con 0x55d7440ffc00 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d744e96480 con 0x55d7440ff800 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6b8e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6b8e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50848/0 (socket says 172.21.15.1:50848) 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.365+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1224320626 learned_addr learned my addr 172.21.15.1:0/1224320626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7440ffc00 0x55d744173080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 0x55d744172b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 msgr2=0x55d744172b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 0x55d744172b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7440ffc00 msgr2=0x55d744173080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7440ffc00 0x55d744173080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1224320626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74412b680 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 0x55d744172b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x55d744f1b1d0 tx=0x55d744e99ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423d372cf8ef448d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba690de640 1 -- 172.21.15.1:0/1224320626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d7441249c0 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba690de640 1 -- 172.21.15.1:0/1224320626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d744148e00 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba690de640 1 -- 172.21.15.1:0/1224320626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d744124680 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74412b860 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d74412b2c0 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 4 waiting for MDS map... 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba690de640 1 -- 172.21.15.1:0/1224320626 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55d744fd8200 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba690de640 1 -- 172.21.15.1:0/1224320626 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x55d744157180 con 0x55d7440fec00 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 4 Got MDS map 106 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 4 datascan.main: Using metadata pool output 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.366+0000 7fba6dc16740 4 datascan.main: connecting to RADOS... 2024-05-09T06:39:12.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 Processor -- start 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 -- start start 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744173600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030400 0x55d744173b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.367+0000 7fba6dc16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d744e96600 con 0x55d745030800 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d744e96780 con 0x55d745030000 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d744e96900 con 0x55d745030400 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50858/0 (socket says 172.21.15.1:50858) 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1122493047 learned_addr learned my addr 172.21.15.1:0/1122493047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030400 0x55d744173b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744173600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030400 msgr2=0x55d744173b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030400 0x55d744173b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 msgr2=0x55d744173600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744173600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1122493047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74412b4a0 con 0x55d745030800 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x55d744fcc7e0 tx=0x55d744fe4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba96bbe311f1260 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba670da640 1 -- 172.21.15.1:0/1122493047 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d7441249c0 con 0x55d745030800 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba670da640 1 -- 172.21.15.1:0/1122493047 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d744148a80 con 0x55d745030800 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.368+0000 7fba670da640 1 -- 172.21.15.1:0/1122493047 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d744124680 con 0x55d745030800 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- 172.21.15.1:0/1122493047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 msgr2=0x55d744174100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x55d744fcc7e0 tx=0x55d744fe4c00 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- 172.21.15.1:0/1122493047 shutdown_connections 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030800 0x55d744174100 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030400 0x55d744173b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1122493047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744173600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- 172.21.15.1:0/1122493047 >> 172.21.15.1:0/1122493047 conn(0x55d744ee1c00 msgr2=0x55d744207000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- 172.21.15.1:0/1122493047 shutdown_connections 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- 172.21.15.1:0/1122493047 wait complete. 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 Processor -- start 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.369+0000 7fba6dc16740 1 -- start start 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744175180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 0x55d744174c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030800 0x55d744174680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d744e97c80 con 0x55d745030400 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55d744e97b00 con 0x55d745030000 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55d744e97980 con 0x55d745030800 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 0x55d744174c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 0x55d744174c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50870/0 (socket says 172.21.15.1:50870) 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 learned_addr learned my addr 172.21.15.1:0/1898518052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030800 0x55d744174680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744175180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030800 msgr2=0x55d744174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d745030800 0x55d744174680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 msgr2=0x55d744175180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d745030000 0x55d744175180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74412b0e0 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 0x55d744174c00 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x55d744f1be30 tx=0x55d744e986c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba658d7640 1 -- 172.21.15.1:0/1898518052 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d744124b60 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba658d7640 1 -- 172.21.15.1:0/1898518052 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d744148c40 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d74412ba40 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.370+0000 7fba658d7640 1 -- 172.21.15.1:0/1898518052 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d744124820 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.371+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74412a3c0 con 0x55d745030400 2024-05-09T06:39:12.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.371+0000 7fba6dc16740 4 datascan.init: resolving metadata pool 17 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba658d7640 1 -- 172.21.15.1:0/1898518052 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55d745044c00 con 0x55d745030400 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba658d7640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55d74504f000 0x55d744175700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba658d7640 1 -- 172.21.15.1:0/1898518052 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x55d745037200 con 0x55d745030400 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6dc16740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6dc16740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6dc16740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55d74504f000 0x55d744175700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55d745030c00 0x55d744174100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d745031000 con 0x55d745030c00 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.372+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55d745030c00 0x55d744174100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55d74504f000 0x55d744175700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55d744e9af90 tx=0x55d74447c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55d745030c00 0x55d744174100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55d745031c00 0x55d744173b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d745031400 con 0x55d745031c00 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.373+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55d745031c00 0x55d744173b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55d745031c00 0x55d744173b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55d745031800 0x55d744173600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d745031000 con 0x55d745031800 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55d745031800 0x55d744173600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.374+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55d745031800 0x55d744173600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.375+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.375+0000 7fba668d9640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55d74448e000 0x55d744494000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.375+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e400 con 0x55d74448e000 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.375+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55d74448e000 0x55d744494000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.375+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55d74448e000 0x55d744494000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448ec00 con 0x55d745031c00 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e800 con 0x55d745031800 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.376+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f000 con 0x55d745031800 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f800 con 0x55d74448e000 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba670da640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d74448f400 0x55d744494580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d74448f400 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d74448f400 0x55d744494580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.377+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d74448f400 0x55d744494580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448fc00 con 0x55d745030c00 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2400 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.378+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2800 con 0x55d745031c00 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448ec00 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e800 con 0x55d74448e000 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.379+0000 7fba668d9640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55d745031400 0x55d744494b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74504fc00 con 0x55d745031400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55d745031400 0x55d744494b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba6b8e3640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55d745031400 0x55d744494b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f000 con 0x55d74448f400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba668d9640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55d7444a2400 0x55d744495080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.380+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d7444a2400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.381+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55d7444a2400 0x55d744495080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.381+0000 7fba6b0e2640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55d7444a2400 0x55d744495080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.381+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.381+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d745031000 con 0x55d7444a2400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2c00 con 0x55d745030c00 2024-05-09T06:39:12.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74504ec00 con 0x55d745031400 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 123 out=81b] v123'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.382+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6000 con 0x55d74448e000 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3000 con 0x55d745031c00 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.383+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6400 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3400 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6800 con 0x55d74448e000 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.384+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7400 con 0x55d74448f400 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6c00 con 0x55d745030c00 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7800 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7000 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.385+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7c00 con 0x55d745031c00 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2800 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7400 con 0x55d74448e000 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.386+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3000 con 0x55d745031400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7c00 con 0x55d74448f400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74504fc00 con 0x55d7444a2400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.387+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6400 con 0x55d7444a2400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74504ec00 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7800 con 0x55d745031400 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 123 out=49b] v123'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.388+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f000 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba670da640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55d74448ec00 0x55d744495600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6000 con 0x55d74448ec00 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.389+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55d74448ec00 0x55d744495600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba6c0e4640 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55d74448ec00 0x55d744495600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3400 con 0x55d745030c00 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6800 con 0x55d745031800 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.390+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4000 con 0x55d7444a2400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6c00 con 0x55d745031400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4400 con 0x55d745030c00 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.391+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7400 con 0x55d74448f400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 123 out=81b] v123'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-09T06:39:12.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 10 datascan.operator(): handling object 100000001f7.0 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3c00 con 0x55d74448f400 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e400 con 0x55d745031800 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.392+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f800 con 0x55d745030c00 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.get_frag_of: frag is * 2024-05-09T06:39:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d74448fc00 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e800 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 20 datascan.read_dentry: bravo_head not found in result 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.393+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55d74448e400 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.394+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v123'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.394+0000 7fba6dc16740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.394+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74504fc00 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2c00 con 0x55d745030c00 2024-05-09T06:39:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4800 con 0x55d74448e000 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d745031000 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.395+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4c00 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4000 con 0x55d74448e000 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5000 con 0x55d74448f400 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.396+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2800 con 0x55d74448ec00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.397+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 123 out=49b] v121'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.397+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5400 con 0x55d745030c00 2024-05-09T06:39:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.397+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.397+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3000 con 0x55d7444a2400 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5800 con 0x55d74448ec00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 123 out=49b] v123'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2000 con 0x55d745031800 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.398+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5c00 con 0x55d74448f400 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 123 out=49b] v123'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3400 con 0x55d745031c00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4400 con 0x55d74448e000 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.399+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448fc00 con 0x55d74448ec00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2c00 con 0x55d74448ec00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f000 con 0x55d745031c00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.6 v2:172.21.15.6:6817/1315796905 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d745031c00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.400+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3c00 con 0x55d745031800 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.0 v2:172.21.15.1:6819/3367065226 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031800 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5800 con 0x55d745030c00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.3 v2:172.21.15.1:6800/2131549113 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d745030c00 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3800 con 0x55d74448f400 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5c00 con 0x55d745031400 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.401+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7c00 con 0x55d7444a2400 2024-05-09T06:39:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.402+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.402+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4800 con 0x55d74448f400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.402+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.402+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b6000 con 0x55d74448e000 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.403+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.403+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4c00 con 0x55d7444a2400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.403+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.403+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444b7400 con 0x55d74448f400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448fc00 con 0x55d74448ec00 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d745031000 con 0x55d745031400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba6b8e3640 1 -- 172.21.15.1:0/1898518052 <== osd.2 v2:172.21.15.1:6808/1632480843 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272780 con 0x55d745031400 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.404+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f000 con 0x55d74448e000 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d74448f800 con 0x55d74448e000 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2800 con 0x55d7444a2400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.405+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c4400 con 0x55d74448e000 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.5 v2:172.21.15.6:6801/2308051940 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d74448e000 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a3400 con 0x55d74448f400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.1 v2:172.21.15.1:6816/1858675679 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448f400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba668d9640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444c5800 con 0x55d7444a2400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 <== osd.7 v2:172.21.15.6:6816/3240730649 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272000 con 0x55d7444a2400 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.406+0000 7fba670da640 1 -- 172.21.15.1:0/1898518052 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x55d7444a2c00 con 0x55d74448ec00 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6c0e4640 1 -- 172.21.15.1:0/1898518052 <== osd.4 v2:172.21.15.6:6800/3928343737 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d744272500 con 0x55d74448ec00 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55d745031800 msgr2=0x55d744173600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x55d745031800 0x55d744173600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d74448f400 msgr2=0x55d744494580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x55d74448f400 0x55d744494580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55d745031400 msgr2=0x55d744494b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x55d745031400 0x55d744494b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55d745030c00 msgr2=0x55d744174100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x55d745030c00 0x55d744174100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55d74448ec00 msgr2=0x55d744495600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x55d74448ec00 0x55d744495600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55d74448e000 msgr2=0x55d744494000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 reap_dead start 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x55d74448e000 0x55d744494000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 reap_dead start 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 reap_dead start 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55d745031c00 msgr2=0x55d744173b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x55d745031c00 0x55d744173b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55d7444a2400 msgr2=0x55d744495080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x55d7444a2400 0x55d744495080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55d74504f000 msgr2=0x55d744175700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x55d74504f000 0x55d744175700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55d744e9af90 tx=0x55d74447c000 comp rx=0 tx=0).stop 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 msgr2=0x55d744174c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1898518052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d745030400 0x55d744174c00 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x55d744f1be30 tx=0x55d744e986c0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6b0e2640 1 -- 172.21.15.1:0/1898518052 reap_dead start 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 shutdown_connections 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 >> 172.21.15.1:0/1898518052 conn(0x55d744ee1c00 msgr2=0x55d744207000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 shutdown_connections 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.407+0000 7fba6dc16740 1 -- 172.21.15.1:0/1898518052 wait complete. 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 msgr2=0x55d744172580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x55d744f1b1d0 tx=0x55d744e99ec0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 shutdown_connections 2024-05-09T06:39:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d7440ffc00 0x55d744173080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55d7440ff800 0x55d744172b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 --2- 172.21.15.1:0/1224320626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7440fec00 0x55d744172580 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 >> 172.21.15.1:0/1224320626 conn(0x55d7440ff000 msgr2=0x55d744206800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 shutdown_connections 2024-05-09T06:39:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.408+0000 7fba6dc16740 1 -- 172.21.15.1:0/1224320626 wait complete. 2024-05-09T06:39:12.401 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.139547, result: 2024-05-09T06:39:12.401 DEBUG:teuthology.orchestra.run.smithi001:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-09T06:39:12.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.501+0000 7f18b2411740 1 -- start start 2024-05-09T06:39:12.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.501+0000 7f18b2411740 1 Processor -- start 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635498ae400 0x563548b3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af400 0x563548b3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5635498ce000 con 0x5635498af000 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5635498ce180 con 0x5635498af400 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5635498ce300 con 0x5635498ae400 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50880/0 (socket says 172.21.15.1:50880) 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 -- 172.21.15.1:0/1663660146 learned_addr learned my addr 172.21.15.1:0/1663660146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b004e640 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635498ae400 0x563548b3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18b084f640 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af400 0x563548b3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 -- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635498ae400 msgr2=0x563548b3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635498ae400 0x563548b3d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 -- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af400 msgr2=0x563548b3d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af400 0x563548b3d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.502+0000 7f18af84d640 1 -- 172.21.15.1:0/1663660146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563548af5680 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18af84d640 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x5635498eeff0 tx=0x5635498629c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b205f1168154a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18ad849640 1 -- 172.21.15.1:0/1663660146 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeeb60 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18ad849640 1 -- 172.21.15.1:0/1663660146 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563548b12c40 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18ad849640 1 -- 172.21.15.1:0/1663660146 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeed00 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563548af52c0 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563548af4b40 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 4 waiting for MDS map... 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18ad849640 1 -- 172.21.15.1:0/1663660146 <== mon.0 v2:172.21.15.1:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x5635498d2400 con 0x5635498af000 2024-05-09T06:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18ad849640 1 -- 172.21.15.1:0/1663660146 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x563548b21180 con 0x5635498af000 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 4 Got MDS map 106 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 4 datascan.main: Using metadata pool output 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.503+0000 7f18b2411740 4 datascan.main: connecting to RADOS... 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 Processor -- start 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 -- start start 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635499b2400 0x563548b3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b2800 0x563548b3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5635498ce480 con 0x5635498afc00 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5635498ce600 con 0x5635499b2400 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5635498ce780 con 0x5635499b2800 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b084f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b084f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50886/0 (socket says 172.21.15.1:50886) 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b084f640 1 -- 172.21.15.1:0/2951150878 learned_addr learned my addr 172.21.15.1:0/2951150878 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18af84d640 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b2800 0x563548b3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.504+0000 7f18b004e640 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635499b2400 0x563548b3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 -- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b2800 msgr2=0x563548b3e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b2800 0x563548b3e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 -- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635499b2400 msgr2=0x563548b3e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635499b2400 0x563548b3e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 -- 172.21.15.1:0/2951150878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563548af43c0 con 0x5635498afc00 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b084f640 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x5635499a8d50 tx=0x5635499b4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fcfcdd92881e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18ab845640 1 -- 172.21.15.1:0/2951150878 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeeb60 con 0x5635498afc00 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18ab845640 1 -- 172.21.15.1:0/2951150878 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563548b13340 con 0x5635498afc00 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18ab845640 1 -- 172.21.15.1:0/2951150878 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeed00 con 0x5635498afc00 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 -- 172.21.15.1:0/2951150878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 msgr2=0x563548b3c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x5635499a8d50 tx=0x5635499b4c00 comp rx=0 tx=0).stop 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 -- 172.21.15.1:0/2951150878 shutdown_connections 2024-05-09T06:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b2800 0x563548b3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635499b2400 0x563548b3e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 --2- 172.21.15.1:0/2951150878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498afc00 0x563548b3c580 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 -- 172.21.15.1:0/2951150878 >> 172.21.15.1:0/2951150878 conn(0x5635498af800 msgr2=0x56354984d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 -- 172.21.15.1:0/2951150878 shutdown_connections 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.505+0000 7f18b2411740 1 -- 172.21.15.1:0/2951150878 wait complete. 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 Processor -- start 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 -- start start 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af800 0x563548b3f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 0x563548b3f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b3000 0x563548b3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5635498cfe00 con 0x5635499b3400 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5635498cfc80 con 0x5635498af800 2024-05-09T06:39:12.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.506+0000 7f18b2411740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x5635498cfb00 con 0x5635499b3000 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 0x563548b3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 0x563548b3f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50892/0 (socket says 172.21.15.1:50892) 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 learned_addr learned my addr 172.21.15.1:0/3662934198 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b3000 0x563548b3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af800 0x563548b3f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b3000 msgr2=0x563548b3ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635499b3000 0x563548b3ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af800 msgr2=0x563548b3f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af800 0x563548b3f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563548af5a40 con 0x5635499b3400 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 0x563548b3f180 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x5635499a9b90 tx=0x5635499b4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18aa042640 1 -- 172.21.15.1:0/3662934198 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeeb60 con 0x5635499b3400 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18aa042640 1 -- 172.21.15.1:0/3662934198 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563548b121c0 con 0x5635499b3400 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18aa042640 1 -- 172.21.15.1:0/3662934198 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563548aeed00 con 0x5635499b3400 2024-05-09T06:39:12.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563548af54a0 con 0x5635499b3400 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563548af5c20 con 0x5635499b3400 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.507+0000 7f18b2411740 4 datascan.init: resolving metadata pool 17 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.508+0000 7f18aa042640 1 -- 172.21.15.1:0/3662934198 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x563549a1ac00 con 0x5635499b3400 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18aa042640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563549a1ec00 0x563548e56000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18aa042640 1 -- 172.21.15.1:0/3662934198 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x563549a18c00 con 0x5635499b3400 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b2411740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b2411740 4 datascan.main: resolving metadata pool 17 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563549a1ec00 0x563548e56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x5635499b2c00 0x563548b3e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x5635499b2800 con 0x5635499b2c00 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x5635499b2c00 0x563548b3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.509+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563549a1ec00 0x563548e56000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5635499a8d50 tx=0x563549862a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x5635499b2c00 0x563548b3e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x5635499b2800 0x563548e56580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f000 con 0x5635499b2800 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x5635499b2800 0x563548e56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.510+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x5635499b2800 0x563548e56580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f000 con 0x5635499b2c00 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x563549a1f000 0x563548e56b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f400 con 0x563549a1f000 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x563549a1f000 0x563548e56b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.511+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x563549a1f000 0x563548e56b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f400 con 0x563549a1f000 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f400 con 0x5635499b2800 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=149b] v123'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.512+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f400 con 0x5635499b2c00 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 3 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x563549a1f400 0x563548e57080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1f800 con 0x563549a1f400 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x563549a1f400 0x563548e57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x563549a1f400 0x563548e57080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563549a1f800 0x563548e57600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.513+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1fc00 con 0x563549a1f800 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563549a1f800 0x563548e57600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563549a1f800 0x563548e57600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1fc00 con 0x5635499b2800 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 3 ==== osd_op_reply(10 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.514+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1fc00 con 0x5635499b2c00 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 4 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563549a1fc00 con 0x563549a1f000 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 3 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563549a1fc00 0x563548e58680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6a000 con 0x563549a1fc00 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.515+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563549a1fc00 0x563548e58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.516+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563549a1fc00 0x563548e58680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.516+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== osd_op_reply(13 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.516+0000 7f18b004e640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563548e6a000 0x563548e58c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.516+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6a400 con 0x563548e6a000 2024-05-09T06:39:12.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.516+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563548e6a000 0x563548e58c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563548e6a000 0x563548e58c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18af84d640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563548e6a400 0x563548e57b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563548e6a400 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563548e6a400 0x563548e57b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18b084f640 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563548e6a400 0x563548e57b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== osd_op_reply(15 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c500 con 0x563548e6a400 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.517+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f400 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f000 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 4 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x5635499b2800 2024-05-09T06:39:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 4 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x5635499b2800 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 5 ==== osd_op_reply(19 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.518+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563548e6a000 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.519+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.519+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1fc00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.519+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== osd_op_reply(21 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.519+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563548e6a000 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.519+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x5635499b2c00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 5 ==== osd_op_reply(23 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x5635499b2c00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 6 ==== osd_op_reply(24 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f800 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.520+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.521+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x5635499b2800 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.521+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 6 ==== osd_op_reply(26 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.521+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f000 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.521+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 5 ==== osd_op_reply(27 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.521+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f800 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 3 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f000 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1fc00 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f400 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 3 ==== osd_op_reply(31 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.522+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1fc00 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 4 ==== osd_op_reply(32 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-09T06:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5635499b2400 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.523+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a400 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a800 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7ac00 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-09T06:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b000 con 0x5635499b2800 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.524+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b400 con 0x563549a1f400 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b800 con 0x5635499b2800 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-09T06:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7bc00 con 0x563548e6a400 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x563548e6a400 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.525+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563548e6a000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a400 con 0x5635499b2c00 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-09T06:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.526+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a800 con 0x5635499b2800 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7ac00 con 0x563549a1f000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5635499b2400 con 0x563549a1f000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 20 datascan.scan_links: injected ino detected 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-09T06:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.527+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b400 con 0x563549a1fc00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 8 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 15 ==== osd_op_reply(49 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.528+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 9 ==== osd_op_reply(50 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 9 ==== osd_op_reply(51 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 10 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 16 ==== osd_op_reply(53 [pgnls start_epoch 123 out=149b] v123'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.529+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 10 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f400 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 5 ==== osd_op_reply(55 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f800 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.530+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 17 ==== osd_op_reply(57 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.531+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1fc00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563548e6a000 2024-05-09T06:39:12.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563548e6a400 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.2 v2:172.21.15.1:6808/1632480843 3 ==== osd_op_reply(62 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c500 con 0x563548e6a400 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f400 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 6 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.532+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 12 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 18 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 19 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.533+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563548e6a000 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 6 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1fc00 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 7 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563548e6a000 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 7 ==== osd_op_reply(69 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.534+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 12 ==== osd_op_reply(70 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2c00 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 13 ==== osd_op_reply(71 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 5 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x5635499b2800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 20 ==== osd_op_reply(73 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.535+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.536+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 13 ==== osd_op_reply(74 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.536+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.536+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.4 v2:172.21.15.6:6800/3928343737 6 ==== osd_op_reply(75 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c500 con 0x563549a1f800 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.536+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f000 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 14 ==== osd_op_reply(76 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1fc00 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 8 ==== osd_op_reply(77 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1f400 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 7 ==== osd_op_reply(78 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.537+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a000 con 0x563549a1fc00 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 9 ==== osd_op_reply(79 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a400 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6bc00 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.538+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6b800 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6b400 con 0x5635499b2800 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6b000 con 0x5635499b2800 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b000 con 0x5635499b2800 2024-05-09T06:39:12.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.539+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b800 con 0x563549a1f400 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.1 v2:172.21.15.1:6816/1858675679 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x563549a1f400 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7bc00 con 0x5635499b2800 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a800 con 0x563548e6a400 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.2 v2:172.21.15.1:6808/1632480843 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c500 con 0x563548e6a400 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.540+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a400 con 0x563548e6a000 2024-05-09T06:39:12.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.5 v2:172.21.15.6:6801/2308051940 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x563548e6a000 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7ac00 con 0x5635499b2c00 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.541+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b400 con 0x5635499b2800 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-09T06:39:12.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x5635499b2400 con 0x563549a1f000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6bc00 con 0x563549a1f000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 20 datascan.scan_links: injected ino detected 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-09T06:39:12.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.542+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6a800 con 0x563549a1fc00 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.6 v2:172.21.15.6:6817/1315796905 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x56354983c780 con 0x563549a1fc00 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 10 datascan.scan_links: processing 1 injected_inos 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e6ac00 con 0x563549a1f000 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 20 datascan.scan_links: first is now 2 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.543+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b000 con 0x563549a1f000 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 <== osd.0 v2:172.21.15.1:6819/3367065226 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v123'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56354983c780 con 0x563549a1f000 2024-05-09T06:39:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 10 datascan.scan_links: updating inotable 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7bc00 con 0x5635499b2800 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.544+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563548e7a800 con 0x5635499b2800 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b084f640 1 -- 172.21.15.1:0/3662934198 <== osd.3 v2:172.21.15.1:6800/2131549113 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v123'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x56354983c500 con 0x5635499b2800 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 10 datascan.scan_links: updating snaptable 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563548e7b400 con 0x5635499b2c00 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18af84d640 1 -- 172.21.15.1:0/3662934198 <== osd.7 v2:172.21.15.6:6816/3240730649 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x56354983c000 con 0x5635499b2c00 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x563549a1f000 msgr2=0x563548e56b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x563549a1f000 0x563548e56b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x563549a1f400 msgr2=0x563548e57080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x563549a1f400 0x563548e57080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563548e6a400 msgr2=0x563548e57b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x563548e6a400 0x563548e57b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x5635499b2800 msgr2=0x563548e56580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x5635499b2800 0x563548e56580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563549a1f800 msgr2=0x563548e57600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x563549a1f800 0x563548e57600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563548e6a000 msgr2=0x563548e58c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x563548e6a000 0x563548e58c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563549a1fc00 msgr2=0x563548e58680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x563549a1fc00 0x563548e58680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x5635499b2c00 msgr2=0x563548b3e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x5635499b2c00 0x563548b3e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563549a1ec00 msgr2=0x563548e56000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x563549a1ec00 0x563548e56000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5635499a8d50 tx=0x563549862a20 comp rx=0 tx=0).stop 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 msgr2=0x563548b3f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 --2- 172.21.15.1:0/3662934198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635499b3400 0x563548b3f180 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x5635499a9b90 tx=0x5635499b4d20 comp rx=0 tx=0).stop 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 reap_dead start 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 reap_dead start 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 reap_dead start 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 reap_dead start 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b004e640 1 -- 172.21.15.1:0/3662934198 reap_dead start 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 shutdown_connections 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 >> 172.21.15.1:0/3662934198 conn(0x5635498afc00 msgr2=0x56354984d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 shutdown_connections 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.546+0000 7f18b2411740 1 -- 172.21.15.1:0/3662934198 wait complete. 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 msgr2=0x563548b3cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x5635498eeff0 tx=0x5635498629c0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 shutdown_connections 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5635498af400 0x563548b3d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635498af000 0x563548b3cb00 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 --2- 172.21.15.1:0/1663660146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x5635498ae400 0x563548b3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 >> 172.21.15.1:0/1663660146 conn(0x563548ac9000 msgr2=0x56354984c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 shutdown_connections 2024-05-09T06:39:12.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.547+0000 7f18b2411740 1 -- 172.21.15.1:0/1663660146 wait complete. 2024-05-09T06:39:12.547 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.146205, result: 2024-05-09T06:39:12.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:39:12.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:39:12.622 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T06:39:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T06:39:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:39:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T06:39:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.628 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T06:39:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:39:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:39:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T06:39:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:39:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.635 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = "" 2024-05-09T06:39:12.635 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.635 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.635 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = False 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = True 2024-05-09T06:39:12.636 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:39:12.637 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:39:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:39:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.640 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.641 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-09T06:39:12.642 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.643 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-09T06:39:12.644 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:39:12.645 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: mon down mkfs grace = 300 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-09T06:39:12.646 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:39:12.647 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = False 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = True 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.648 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:39:12.692 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T06:39:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:39:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:39:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T06:39:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T06:39:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T06:39:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T06:39:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T06:39:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = "" 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T06:39:12.743 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = True 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = True 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T06:39:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.745 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-09T06:39:12.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-09T06:39:12.747 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.1,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 91cbcc67-ea38-4460-ad67-94260dad3bad 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:39:12.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.749 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-09T06:39:12.750 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.751 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-09T06:39:12.752 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon down mkfs grace = 300 2024-05-09T06:39:12.753 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-09T06:39:12.754 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = True 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = True 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2024-05-09T06:39:12.755 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2024-05-09T06:39:12.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-09T06:39:12.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.923+0000 7efe543c8640 1 Processor -- start 2024-05-09T06:39:12.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.923+0000 7efe543c8640 1 -- start start 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe543c8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 0x7efe4c0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe543c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe543c8640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4c0d2e20 con 0x7efe4c0d0ce0 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe543c8640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efe4c0d04a0 con 0x7efe4c15e3d0 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe543c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c0cfc00 con 0x7efe4c0cf690 2024-05-09T06:39:12.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe52965640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7efe4c0d0ce0 0x7efe4c0d1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42402/0 (socket says 172.21.15.1:42402) 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe52965640 1 -- 172.21.15.1:0/3584574079 learned_addr learned my addr 172.21.15.1:0/3584574079 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe51963640 1 --2- 172.21.15.1:0/3584574079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe52164640 1 --2- 172.21.15.1:0/3584574079 >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 0x7efe4c0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.924+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137537003 0 0) 0x7efe4c0d2e20 con 0x7efe4c0d0ce0 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe38003520 con 0x7efe4c0d0ce0 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51963640 1 -- 172.21.15.1:0/3584574079 >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 msgr2=0x7efe4c0d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4083793085 0 0) 0x7efe38003520 con 0x7efe4c0d0ce0 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51963640 1 --2- 172.21.15.1:0/3584574079 >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 0x7efe4c0d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51963640 1 -- 172.21.15.1:0/3584574079 >> v1:172.21.15.1:6789/0 conn(0x7efe4c0d0ce0 legacy=0x7efe4c0d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51963640 1 -- 172.21.15.1:0/3584574079 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c0cfee0 con 0x7efe4c0cf690 2024-05-09T06:39:12.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe52164640 1 --2- 172.21.15.1:0/3584574079 >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 0x7efe4c0d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.925+0000 7efe51963640 1 --2- 172.21.15.1:0/3584574079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0d07a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097d0 tx=0x7efe3c02a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207ce49acd15c8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c037070 con 0x7efe4c0cf690 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3c02abd0 con 0x7efe4c0cf690 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe51162640 1 -- 172.21.15.1:0/3584574079 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44001670 con 0x7efe4c0cf690 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 -- 172.21.15.1:0/3584574079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 msgr2=0x7efe4c0d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 --2- 172.21.15.1:0/3584574079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0d07a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097d0 tx=0x7efe3c02a140 comp rx=0 tx=0).stop 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 -- 172.21.15.1:0/3584574079 shutdown_connections 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 --2- 172.21.15.1:0/3584574079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0d07a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 --2- 172.21.15.1:0/3584574079 >> v2:172.21.15.6:3300/0 conn(0x7efe4c15e3d0 0x7efe4c0d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 -- 172.21.15.1:0/3584574079 >> 172.21.15.1:0/3584574079 conn(0x7efe4c0d3d90 msgr2=0x7efe4c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:12.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 -- 172.21.15.1:0/3584574079 shutdown_connections 2024-05-09T06:39:12.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.926+0000 7efe543c8640 1 -- 172.21.15.1:0/3584574079 wait complete. 2024-05-09T06:39:12.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 Processor -- start 2024-05-09T06:39:12.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 -- start start 2024-05-09T06:39:12.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe4c0d0ce0 0x7efe4c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c0b49d0 con 0x7efe4c15e3d0 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c0b34a0 con 0x7efe4c0d0ce0 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe543c8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c0b4b50 con 0x7efe4c0cf690 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe52965640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe52164640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe52965640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50902/0 (socket says 172.21.15.1:50902) 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe52965640 1 -- 172.21.15.1:0/1252199752 learned_addr learned my addr 172.21.15.1:0/1252199752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.928+0000 7efe51963640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe4c0d0ce0 0x7efe4c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 -- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 msgr2=0x7efe4c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 -- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe4c0d0ce0 msgr2=0x7efe4c0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:12.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe4c0d0ce0 0x7efe4c0b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 -- 172.21.15.1:0/1252199752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c0cb390 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe52965640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7efe44007fd0 tx=0x7efe4400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44017070 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe44005ce0 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c0cb670 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44007290 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.929+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c0cbc30 con 0x7efe4c15e3d0 2024-05-09T06:39:12.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.930+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c0ce3c0 con 0x7efe4c15e3d0 2024-05-09T06:39:12.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.931+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe4401a070 con 0x7efe4c15e3d0 2024-05-09T06:39:12.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.931+0000 7efe42ffd640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 0x7efe3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:12.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.931+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7efe4408a410 con 0x7efe4c15e3d0 2024-05-09T06:39:12.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.931+0000 7efe52164640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 0x7efe3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:12.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.932+0000 7efe52164640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 0x7efe3006e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efe48009bd0 tx=0x7efe4800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:12.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:12.936+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe4c0ce3c0 con 0x7efe4c15e3d0 2024-05-09T06:39:13.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:13.158+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efe4c0d49f0 con 0x7efe4c15e3d0 2024-05-09T06:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.069+0000 7efe42ffd640 1 -- 172.21.15.1:0/1252199752 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7efe44058530 con 0x7efe4c15e3d0 2024-05-09T06:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-09T06:39:14.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 msgr2=0x7efe3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 0x7efe3006e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efe48009bd0 tx=0x7efe4800b040 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 msgr2=0x7efe4c0b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7efe44007fd0 tx=0x7efe4400c930 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 shutdown_connections 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efe3006be30 0x7efe3006e2f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe4c15e3d0 0x7efe4c0b1ae0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe4c0d0ce0 0x7efe4c0b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 --2- 172.21.15.1:0/1252199752 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe4c0cf690 0x7efe4c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 >> 172.21.15.1:0/1252199752 conn(0x7efe4c0d3d90 msgr2=0x7efe4c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 shutdown_connections 2024-05-09T06:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.072+0000 7efe543c8640 1 -- 172.21.15.1:0/1252199752 wait complete. 2024-05-09T06:39:14.068 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.198+0000 7f1da284d640 1 Processor -- start 2024-05-09T06:39:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.198+0000 7f1da284d640 1 -- start start 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da284d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c0c6860 0x7f1d9c099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da284d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d9c15e4b0 0x7f1d9c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da284d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c099930 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da284d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1d9c0b1e20 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da284d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c15ee90 con 0x7f1d9c0c6860 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d9c15e4b0 0x7f1d9c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1d9c15e4b0 0x7f1d9c15f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50906/0 (socket says 172.21.15.1:50906) 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 -- 172.21.15.1:0/2560516277 learned_addr learned my addr 172.21.15.1:0/2560516277 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1d9bfff640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c0c6860 0x7f1d9c099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 -- 172.21.15.1:0/2560516277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c0c6860 msgr2=0x7f1d9c099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c0c6860 0x7f1d9c099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 -- 172.21.15.1:0/2560516277 >> v1:172.21.15.6:6789/0 conn(0x7f1d9c0b2aa0 legacy=0x7f1d9c0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:14.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.199+0000 7f1da0dea640 1 -- 172.21.15.1:0/2560516277 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c162930 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.200+0000 7f1da0dea640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c15e4b0 0x7f1d9c15f320 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009e50 tx=0x7f1d9402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d83dade4133ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.200+0000 7f1d9affd640 1 -- 172.21.15.1:0/2560516277 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94036070 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.200+0000 7f1d9affd640 1 -- 172.21.15.1:0/2560516277 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d94031600 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.200+0000 7f1d9affd640 1 -- 172.21.15.1:0/2560516277 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940319d0 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 -- 172.21.15.1:0/2560516277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c15e4b0 msgr2=0x7f1d9c15f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c15e4b0 0x7f1d9c15f320 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009e50 tx=0x7f1d9402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 -- 172.21.15.1:0/2560516277 shutdown_connections 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c15e4b0 0x7f1d9c15f320 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 --2- 172.21.15.1:0/2560516277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c0c6860 0x7f1d9c099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 -- 172.21.15.1:0/2560516277 >> 172.21.15.1:0/2560516277 conn(0x7f1d9c0b1fb0 msgr2=0x7f1d9c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 -- 172.21.15.1:0/2560516277 shutdown_connections 2024-05-09T06:39:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.201+0000 7f1da284d640 1 -- 172.21.15.1:0/2560516277 wait complete. 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.202+0000 7f1da284d640 1 Processor -- start 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.202+0000 7f1da284d640 1 -- start start 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d9c0c6860 0x7f1d9c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c15e4b0 0x7f1d9c1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c15f860 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c15eb20 con 0x7f1d9c0c6860 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da284d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c15f9e0 con 0x7f1d9c15e4b0 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50918/0 (socket says 172.21.15.1:50918) 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1d9bfff640 1 -- 172.21.15.1:0/4154478955 learned_addr learned my addr 172.21.15.1:0/4154478955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1d9b7fe640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d9c0c6860 0x7f1d9c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.203+0000 7f1da0dea640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c15e4b0 0x7f1d9c1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 -- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c15e4b0 msgr2=0x7f1d9c1f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c15e4b0 0x7f1d9c1f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 -- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d9c0c6860 msgr2=0x7f1d9c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d9c0c6860 0x7f1d9c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 -- 172.21.15.1:0/4154478955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c1f8480 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d9bfff640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c7f0 tx=0x7f1d8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c015070 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c004240 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c1f8760 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c007620 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.204+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c0b35d0 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.205+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c0b3100 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.206+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d8c0079d0 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.206+0000 7f1d98ff9640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 0x7f1d8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.206+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f1d8c08b400 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.206+0000 7f1d9b7fe640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 0x7f1d8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.206+0000 7f1d9b7fe640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 0x7f1d8806e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0cb150 tx=0x7f1d90007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.210+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d8c0595b0 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.430+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9c0752b0 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.431+0000 7f1d98ff9640 1 -- 172.21.15.1:0/4154478955 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5169 (secure 0 0 0) 0x7f1d8c05d8b0 con 0x7f1d9c0b2aa0 2024-05-09T06:39:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:14.411 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:14.073784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-09T06:39:14.412 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 109 2024-05-09T06:39:14.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 msgr2=0x7f1d8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 0x7f1d8806e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0cb150 tx=0x7f1d90007e30 comp rx=0 tx=0).stop 2024-05-09T06:39:14.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 msgr2=0x7f1d9c161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:14.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c7f0 tx=0x7f1d8c00ccb0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 shutdown_connections 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1d8806be30 0x7f1d8806e2f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1d9c15e4b0 0x7f1d9c1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d9c0c6860 0x7f1d9c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.432+0000 7f1da284d640 1 --2- 172.21.15.1:0/4154478955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c0b2aa0 0x7f1d9c161d60 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.433+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 >> 172.21.15.1:0/4154478955 conn(0x7f1d9c0b1fb0 msgr2=0x7f1d9c0c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.433+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 shutdown_connections 2024-05-09T06:39:14.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:14.433+0000 7f1da284d640 1 -- 172.21.15.1:0/4154478955 wait complete. 2024-05-09T06:39:14.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:39:14.073784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10161}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10161': {'gid': 10161, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.1:6837/520088510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 520088510}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 520088510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:39:14.827 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10161:up:replay 2024-05-09T06:39:15.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:15.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.965+0000 7f7be6631640 1 Processor -- start 2024-05-09T06:39:15.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.965+0000 7f7be6631640 1 -- start start 2024-05-09T06:39:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7be6631640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7be015e6d0 0x7f7be006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7be6631640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be0070f40 con 0x7f7be00b2a60 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7be6631640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be00cf170 con 0x7f7be015db10 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7be6631640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7be00cf450 con 0x7f7be015e6d0 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7bdf7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7be00b2a60 0x7f7be015dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42410/0 (socket says 172.21.15.1:42410) 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7bdf7fe640 1 -- 172.21.15.1:0/2829096610 learned_addr learned my addr 172.21.15.1:0/2829096610 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443838700 0 0) 0x7f7be0070f40 con 0x7f7be00b2a60 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7be4bce640 1 --2- 172.21.15.1:0/2829096610 >> v2:172.21.15.6:3300/0 conn(0x7f7be015e6d0 0x7f7be006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.966+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bc8003440 con 0x7f7be00b2a60 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1445609261 0 0) 0x7f7bc8003440 con 0x7f7be00b2a60 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 >> v2:172.21.15.6:3300/0 conn(0x7f7be015e6d0 msgr2=0x7f7be006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 --2- 172.21.15.1:0/2829096610 >> v2:172.21.15.6:3300/0 conn(0x7f7be015e6d0 0x7f7be006e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 >> v1:172.21.15.6:6789/0 conn(0x7f7be015db10 legacy=0x7f7be00715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be0070c10 con 0x7f7be00b2a60 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f7bd4001670 con 0x7f7be00b2a60 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 >> v1:172.21.15.1:6789/0 conn(0x7f7be00b2a60 legacy=0x7f7be015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bc8003aa0 0x7f7bc8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8008850 0x7f7bc800acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7bc800da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00cf450 con 0x7f7bc800b560 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc800e300 con 0x7f7bc8008850 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc800e500 con 0x7f7bc8003aa0 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7bc800da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdf7fe640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8008850 0x7f7bc800acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7bdffff640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bc8003aa0 0x7f7bc8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 -- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bc8003aa0 msgr2=0x7f7bc8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7bc8003aa0 0x7f7bc8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 -- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8008850 msgr2=0x7f7bc800acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bc8008850 0x7f7bc800acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.967+0000 7f7be4bce640 1 -- 172.21.15.1:0/2829096610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc8003440 con 0x7f7bc800b560 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.968+0000 7f7be4bce640 1 -- 172.21.15.1:0/2829096610 reap_dead start 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.968+0000 7f7be4bce640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7bc800da20 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009f90 tx=0x7f7bc8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae6264bec214965 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.968+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8035020 con 0x7f7bc800b560 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.968+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8003b90 con 0x7f7bc800b560 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.968+0000 7f7bdeffd640 1 -- 172.21.15.1:0/2829096610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8009c10 con 0x7f7bc800b560 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 -- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 msgr2=0x7f7bc800da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7bc800da20 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009f90 tx=0x7f7bc8003730 comp rx=0 tx=0).stop 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 -- 172.21.15.1:0/2829096610 shutdown_connections 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 --2- 172.21.15.1:0/2829096610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7bc800da20 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 -- 172.21.15.1:0/2829096610 >> 172.21.15.1:0/2829096610 conn(0x7f7be00cc4b0 msgr2=0x7f7be00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 -- 172.21.15.1:0/2829096610 shutdown_connections 2024-05-09T06:39:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.969+0000 7f7be6631640 1 -- 172.21.15.1:0/2829096610 wait complete. 2024-05-09T06:39:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.970+0000 7f7be6631640 1 Processor -- start 2024-05-09T06:39:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 -- start start 2024-05-09T06:39:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7be0157150 0x7f7be01575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7be00b0230 0x7f7be00b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7be015dfc0 con 0x7f7bc800b560 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7be015ee50 con 0x7f7be00b0230 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7be6631640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7be015e140 con 0x7f7be0157150 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7bdffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7bdffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50944/0 (socket says 172.21.15.1:50944) 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.971+0000 7f7bdffff640 1 -- 172.21.15.1:0/1467872511 learned_addr learned my addr 172.21.15.1:0/1467872511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7be4bce640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7be00b0230 0x7f7be00b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdf7fe640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7be0157150 0x7f7be01575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 -- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7be0157150 msgr2=0x7f7be01575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7be0157150 0x7f7be01575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 -- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7be00b0230 msgr2=0x7f7be00b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7be00b0230 0x7f7be00b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:15.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 -- 172.21.15.1:0/1467872511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be0157e10 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdffff640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000f620 tx=0x7f7bd000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00052e0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd0018070 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01580f0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0003810 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.972+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be0157af0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.973+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be00752b0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.974+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bd00039b0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.974+0000 7f7bdcff9640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 0x7f7bcc06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.974+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f7bd008aea0 con 0x7f7bc800b560 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.974+0000 7f7bdf7fe640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 0x7f7bcc06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:15.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.975+0000 7f7bdf7fe640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 0x7f7bcc06e290 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7be01f3290 tx=0x7f7bd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:15.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:15.979+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bd0059610 con 0x7f7bc800b560 2024-05-09T06:39:16.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.201+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7be00b5d40 con 0x7f7bc800b560 2024-05-09T06:39:16.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.201+0000 7f7bdcff9640 1 -- 172.21.15.1:0/1467872511 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5169 (secure 0 0 0) 0x7f7bd0058f10 con 0x7f7bc800b560 2024-05-09T06:39:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:16.182 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:15.078593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-09T06:39:16.182 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 111 2024-05-09T06:39:16.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.203+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 msgr2=0x7f7bcc06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:16.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.203+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 0x7f7bcc06e290 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7be01f3290 tx=0x7f7bd4009040 comp rx=0 tx=0).stop 2024-05-09T06:39:16.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.203+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 msgr2=0x7f7be0154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:16.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.203+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000f620 tx=0x7f7bd000fae0 comp rx=0 tx=0).stop 2024-05-09T06:39:16.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 shutdown_connections 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7bcc06bdd0 0x7f7bcc06e290 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7be00b0230 0x7f7be00b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7be0157150 0x7f7be01575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 --2- 172.21.15.1:0/1467872511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bc800b560 0x7f7be0154870 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 >> 172.21.15.1:0/1467872511 conn(0x7f7be00cc4b0 msgr2=0x7f7be0069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 shutdown_connections 2024-05-09T06:39:16.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:16.204+0000 7f7be6631640 1 -- 172.21.15.1:0/1467872511 wait complete. 2024-05-09T06:39:16.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:39:15.078593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10161}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10161': {'gid': 10161, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.1:6837/520088510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 520088510}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 520088510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T06:39:16.598 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10161:up:rejoin 2024-05-09T06:39:17.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:17.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.736+0000 7f2e7fd09640 1 Processor -- start 2024-05-09T06:39:17.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.737+0000 7f2e7fd09640 1 -- start start 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7fd09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e780d4940 0x7f2e780b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7fd09640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e780b33c0 con 0x7f2e780d2b10 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7fd09640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e780b3660 con 0x7f2e78161950 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7fd09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2e780d4dd0 con 0x7f2e780d4940 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7e2a6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2e780d2b10 0x7f2e780b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49846/0 (socket says 172.21.15.1:49846) 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7e2a6640 1 -- 172.21.15.1:0/3830314363 learned_addr learned my addr 172.21.15.1:0/3830314363 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7d2a4640 1 --2- 172.21.15.1:0/3830314363 >> v2:172.21.15.6:3300/0 conn(0x7f2e780d4940 0x7f2e780b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858940530 0 0) 0x7f2e780b33c0 con 0x7f2e780d2b10 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e64003520 con 0x7f2e780d2b10 2024-05-09T06:39:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2366127555 0 0) 0x7f2e64003520 con 0x7f2e780d2b10 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 >> v2:172.21.15.6:3300/0 conn(0x7f2e780d4940 msgr2=0x7f2e780b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 --2- 172.21.15.1:0/3830314363 >> v2:172.21.15.6:3300/0 conn(0x7f2e780d4940 0x7f2e780b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 >> v1:172.21.15.6:6789/0 conn(0x7f2e78161950 legacy=0x7f2e780d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780b3940 con 0x7f2e780d2b10 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f2e700019f0 con 0x7f2e780d2b10 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 >> v1:172.21.15.1:6789/0 conn(0x7f2e780d2b10 legacy=0x7f2e780b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e64004800 0x7f2e64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e640051c0 0x7f2e6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780d4dd0 con 0x7f2e640039b0 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6400c060 con 0x7f2e640051c0 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.738+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6400c260 con 0x7f2e64004800 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7d2a4640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e64004800 0x7f2e64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7e2a6640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e640051c0 0x7f2e6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 -- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e64004800 msgr2=0x7f2e64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e64004800 0x7f2e64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 -- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e640051c0 msgr2=0x7f2e6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e640051c0 0x7f2e6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7e2a6640 1 -- 172.21.15.1:0/3830314363 reap_dead start 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.739+0000 7f2e7daa5640 1 -- 172.21.15.1:0/3830314363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e64003520 con 0x7f2e640039b0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e64003dd0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2e74001720 tx=0x7f2e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5501bf78f77da1c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74001370 con 0x7f2e640039b0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e740052d0 con 0x7f2e640039b0 2024-05-09T06:39:17.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7caa3640 1 -- 172.21.15.1:0/3830314363 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402ab90 con 0x7f2e640039b0 2024-05-09T06:39:17.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7fd09640 1 -- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 msgr2=0x7f2e64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e64003dd0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2e74001720 tx=0x7f2e64005830 comp rx=0 tx=0).stop 2024-05-09T06:39:17.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7fd09640 1 -- 172.21.15.1:0/3830314363 shutdown_connections 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/3830314363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e64003dd0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.740+0000 7f2e7fd09640 1 -- 172.21.15.1:0/3830314363 >> 172.21.15.1:0/3830314363 conn(0x7f2e780d3e20 msgr2=0x7f2e780d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.741+0000 7f2e7fd09640 1 -- 172.21.15.1:0/3830314363 shutdown_connections 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.741+0000 7f2e7fd09640 1 -- 172.21.15.1:0/3830314363 wait complete. 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.743+0000 7f2e7fd09640 1 Processor -- start 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.743+0000 7f2e7fd09640 1 -- start start 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e780b7450 0x7f2e7806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e780b78d0 0x7f2e7806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780d1100 con 0x7f2e640039b0 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780d0f80 con 0x7f2e780b7450 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2e780d1c00 con 0x7f2e780b78d0 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57176/0 (socket says 172.21.15.1:57176) 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 -- 172.21.15.1:0/109993936 learned_addr learned my addr 172.21.15.1:0/109993936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7d2a4640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e780b7450 0x7f2e7806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7e2a6640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e780b78d0 0x7f2e7806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 -- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e780b78d0 msgr2=0x7f2e7806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e780b78d0 0x7f2e7806d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 -- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e780b7450 msgr2=0x7f2e7806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e780b7450 0x7f2e7806cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 -- 172.21.15.1:0/109993936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780b7e30 con 0x7f2e640039b0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7daa5640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e74001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74007a50 con 0x7f2e640039b0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74041070 con 0x7f2e640039b0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e780b8110 con 0x7f2e640039b0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74005480 con 0x7f2e640039b0 2024-05-09T06:39:17.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e780b86d0 con 0x7f2e640039b0 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.744+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e780752b0 con 0x7f2e640039b0 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.745+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e740067d0 con 0x7f2e640039b0 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.746+0000 7f2e6e7fc640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 0x7f2e5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.746+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f2e740ab3c0 con 0x7f2e640039b0 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.746+0000 7f2e7d2a4640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 0x7f2e5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:17.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.746+0000 7f2e7d2a4640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 0x7f2e5c06e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2e7806be70 tx=0x7f2e68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:17.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.751+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e74007680 con 0x7f2e640039b0 2024-05-09T06:39:17.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.974+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e780d21e0 con 0x7f2e640039b0 2024-05-09T06:39:17.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.974+0000 7f2e6e7fc640 1 -- 172.21.15.1:0/109993936 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5178 (secure 0 0 0) 0x7f2e74079a90 con 0x7f2e640039b0 2024-05-09T06:39:17.955 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:17.955 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:17.081146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10161,"qdb_cluster":[10161]},"id":8}]} 2024-05-09T06:39:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 112 2024-05-09T06:39:17.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.977+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 msgr2=0x7f2e5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.977+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 0x7f2e5c06e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2e7806be70 tx=0x7f2e68008040 comp rx=0 tx=0).stop 2024-05-09T06:39:17.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 msgr2=0x7f2e78153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:17.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e74001120 comp rx=0 tx=0).stop 2024-05-09T06:39:17.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 shutdown_connections 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2e5c06be30 0x7f2e5c06e2f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e780b78d0 0x7f2e7806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e780b7450 0x7f2e7806cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 --2- 172.21.15.1:0/109993936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e640039b0 0x7f2e78153110 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 >> 172.21.15.1:0/109993936 conn(0x7f2e780d3e20 msgr2=0x7f2e780d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 shutdown_connections 2024-05-09T06:39:17.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:17.978+0000 7f2e7fd09640 1 -- 172.21.15.1:0/109993936 wait complete. 2024-05-09T06:39:18.050 INFO:tasks.ceph.mds.a.smithi001.stderr:2024-05-09T06:39:18.071+0000 7f01e0ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:39:18.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:38:29.105728+0000', 'modified': '2024-05-09T06:39:17.081146+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10161}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10161': {'gid': 10161, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.1:6837/520088510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6836', 'nonce': 520088510}, {'type': 'v1', 'addr': '172.21.15.1:6837', 'nonce': 520088510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10161, 'qdb_cluster': [10161]} 2024-05-09T06:39:18.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:39:18.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T06:39:18.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.501+0000 7fd24a688640 1 Processor -- start 2024-05-09T06:39:18.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.501+0000 7fd24a688640 1 -- start start 2024-05-09T06:39:18.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd24a688640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2440c6860 0x7fd244099b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd24a688640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 0x7fd24415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd24a688640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd244099930 con 0x7fd2440b2aa0 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd24a688640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440b1e20 con 0x7fd2440c6860 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd24a688640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd24415ee90 con 0x7fd24415e4b0 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd243fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2440c6860 0x7fd244099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd243fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2440c6860 0x7fd244099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57178/0 (socket says 172.21.15.1:57178) 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.502+0000 7fd243fff640 1 -- 172.21.15.1:0/276807607 learned_addr learned my addr 172.21.15.1:0/276807607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd248c25640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 0x7fd24415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd243fff640 1 -- 172.21.15.1:0/276807607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 msgr2=0x7fd24415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd243fff640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 0x7fd24415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd243fff640 1 -- 172.21.15.1:0/276807607 >> v1:172.21.15.6:6789/0 conn(0x7fd2440b2aa0 legacy=0x7fd2440c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd243fff640 1 -- 172.21.15.1:0/276807607 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244162930 con 0x7fd2440c6860 2024-05-09T06:39:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd248c25640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 0x7fd24415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.503+0000 7fd243fff640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2440c6860 0x7fd244099b30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fd234009870 tx=0x7fd23402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b9c2d66069e4b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd2437fe640 1 -- 172.21.15.1:0/276807607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234037070 con 0x7fd2440c6860 2024-05-09T06:39:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd2437fe640 1 -- 172.21.15.1:0/276807607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234032440 con 0x7fd2440c6860 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd2437fe640 1 -- 172.21.15.1:0/276807607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234032810 con 0x7fd2440c6860 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 -- 172.21.15.1:0/276807607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2440c6860 msgr2=0x7fd244099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2440c6860 0x7fd244099b30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fd234009870 tx=0x7fd23402a270 comp rx=0 tx=0).stop 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 -- 172.21.15.1:0/276807607 shutdown_connections 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd24415e4b0 0x7fd24415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 --2- 172.21.15.1:0/276807607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2440c6860 0x7fd244099b30 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 -- 172.21.15.1:0/276807607 >> 172.21.15.1:0/276807607 conn(0x7fd2440b1fb0 msgr2=0x7fd2440b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 -- 172.21.15.1:0/276807607 shutdown_connections 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.504+0000 7fd24a688640 1 -- 172.21.15.1:0/276807607 wait complete. 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.505+0000 7fd24a688640 1 Processor -- start 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 -- start start 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 0x7fd2441466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 0x7fd2440b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd24415f860 con 0x7fd24415e4b0 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd24415eb20 con 0x7fd2440c6860 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd24a688640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd24415f9e0 con 0x7fd2440b2aa0 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd248c25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd248c25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57180/0 (socket says 172.21.15.1:57180) 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.506+0000 7fd248c25640 1 -- 172.21.15.1:0/2751266504 learned_addr learned my addr 172.21.15.1:0/2751266504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd243fff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 0x7fd2441466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd23bfff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 0x7fd2440b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 -- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 msgr2=0x7fd2441466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 0x7fd2441466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 -- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 msgr2=0x7fd2440b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 0x7fd2440b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 -- 172.21.15.1:0/2751266504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440b5ad0 con 0x7fd24415e4b0 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd243fff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 0x7fd2441466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd23bfff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 0x7fd2440b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd248c25640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007df0 tx=0x7fd23c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd2417fa640 1 -- 172.21.15.1:0/2751266504 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c01a070 con 0x7fd24415e4b0 2024-05-09T06:39:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd2417fa640 1 -- 172.21.15.1:0/2751266504 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c003990 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2440cacb0 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd2417fa640 1 -- 172.21.15.1:0/2751266504 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003d80 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.507+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd244146c20 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.509+0000 7fd2417fa640 1 -- 172.21.15.1:0/2751266504 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd23c01d070 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.509+0000 7fd2417fa640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 0x7fd22406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.509+0000 7fd2417fa640 1 -- 172.21.15.1:0/2751266504 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fd23c08b330 con 0x7fd24415e4b0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.509+0000 7fd243fff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 0x7fd22406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.510+0000 7fd243fff640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 0x7fd22406e2f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd234034000 tx=0x7fd23402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.510+0000 7fd24d392740 1 Processor -- start 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.510+0000 7fd24d392740 1 -- start start 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccc0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccc78350 0x55eccccbe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ecccbdb4d0 con 0x55ecccca2670 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ecccb833e0 con 0x55eccccbff10 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ecccbd7490 con 0x55ecccc78350 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57192/0 (socket says 172.21.15.1:57192) 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 -- 172.21.15.1:0/2608235721 learned_addr learned my addr 172.21.15.1:0/2608235721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 -- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccc78350 msgr2=0x55eccccbe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccc78350 0x55eccccbe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 -- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 msgr2=0x55eccccc0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccc0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.511+0000 7fd243fff640 1 -- 172.21.15.1:0/2608235721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccc8ef2d0 con 0x55ecccca2670 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd243fff640 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fd234035040 tx=0x7fd23402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d6ce9507e070c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd239ffb640 1 -- 172.21.15.1:0/2608235721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23404e070 con 0x55ecccca2670 2024-05-09T06:39:18.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd239ffb640 1 -- 172.21.15.1:0/2608235721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23403e460 con 0x55ecccca2670 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd239ffb640 1 -- 172.21.15.1:0/2608235721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234037040 con 0x55ecccca2670 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 -- 172.21.15.1:0/2608235721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 msgr2=0x55ecccca2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fd234035040 tx=0x7fd23402c000 comp rx=0 tx=0).stop 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 -- 172.21.15.1:0/2608235721 shutdown_connections 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccc78350 0x55eccccbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccc0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 --2- 172.21.15.1:0/2608235721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca2a50 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 -- 172.21.15.1:0/2608235721 >> 172.21.15.1:0/2608235721 conn(0x55ecccc74c80 msgr2=0x55eccccbd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 -- 172.21.15.1:0/2608235721 shutdown_connections 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.512+0000 7fd24d392740 1 -- 172.21.15.1:0/2608235721 wait complete. 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.513+0000 7fd24d392740 1 -- start start 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.513+0000 7fd24d392740 1 Processor -- start 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccca2670 0x55ecccca5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55ecccca5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ecccc790f0 con 0x55ecccccd560 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ecccb968e0 con 0x55eccccbff10 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55ecccc72bb0 con 0x55ecccca2670 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd24d392740 10 client.0 _ll_register_callbacks cb 0x55ecccb639c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:39:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57196/0 (socket says 172.21.15.1:57196) 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 -- 172.21.15.1:0/2131849884 learned_addr learned my addr 172.21.15.1:0/2131849884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd243fff640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccca2670 0x55ecccca5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd23bfff640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55ecccca5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 -- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccca2670 msgr2=0x55ecccca5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccca2670 0x55ecccca5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 -- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 msgr2=0x55ecccca5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55ecccca5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd248c25640 1 -- 172.21.15.1:0/2131849884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccc9b96b0 con 0x55ecccccd560 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.514+0000 7fd23bfff640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55ecccca5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd248c25640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fd23c061a50 tx=0x7fd23c05d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e1e60c4a085412 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c059600 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fd23c0597a0 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd23403e460 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fd23c071170 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 10 client.10251 fetch_fsmap learned FSMap version 112 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55eccccb5d20 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23403e460 con 0x55ecccccd560 2024-05-09T06:39:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7fd23c089240 con 0x55ecccccd560 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 10 client.10251 fetch_fsmap finished waiting for FSMap version 112 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 10 client.10251 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 20 client.10251 populate_metadata read hostname 'smithi001' 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55eccccd15c0 0x55eccccb5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 4 client.10251 mds_command: new command op to 10161 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 --> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecccbe8c00 con 0x55eccccd15c0 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.515+0000 7fd243fff640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55eccccd15c0 0x55eccccb5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd222ffd640 10 client.10251 ms_handle_connect on v2:172.21.15.1:6836/520088510 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd243fff640 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55eccccd15c0 0x55eccccb5380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101083c5983479b8 server_cookie=a9233d396c86d9ce in_seq=0 out_seq=0 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd222ffd640 1 -- 172.21.15.1:0/2131849884 <== mds.0 v2:172.21.15.1:6836/520088510 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecccbe8c00 con 0x55eccccd15c0 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd222ffd640 10 client.10251 handle_command_reply: tid=0 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 client.10251 shutdown 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 msgr2=0x55ecccca3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fd23c061a50 tx=0x7fd23c05d750 comp rx=0 tx=0).stop 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 shutdown_connections 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55eccccd15c0 0x55eccccb5380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccccd560 0x55ecccca3d40 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd222ffd640 0 client.10251 ms_handle_reset on v2:172.21.15.1:6836/520088510 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55ecccca5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 --2- 172.21.15.1:0/2131849884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccca2670 0x55ecccca5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.516+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 >> 172.21.15.1:0/2131849884 conn(0x55ecccc78350 msgr2=0x55eccccc1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.517+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 shutdown_connections 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.517+0000 7fd24d392740 1 -- 172.21.15.1:0/2131849884 wait complete. 2024-05-09T06:39:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.517+0000 7fd24d392740 20 client.10251 trim_cache size 0 max 16384 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.535+0000 7fd24d392740 1 Processor -- start 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.535+0000 7fd24d392740 1 -- start start 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.535+0000 7fd24d392740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.535+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccbe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd24d392740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2441ffd20 con 0x55ecccca2670 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440cd7e0 con 0x55eccccbff10 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd244143be0 con 0x55ecccccd560 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57198/0 (socket says 172.21.15.1:57198) 2024-05-09T06:39:18.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 -- 172.21.15.1:0/2013549989 learned_addr learned my addr 172.21.15.1:0/2013549989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd23bfff640 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccbe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd248c25640 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 -- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 msgr2=0x55eccccb2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 -- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 msgr2=0x55eccccbe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccbe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 -- 172.21.15.1:0/2013549989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecccc78770 con 0x55ecccca2670 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd243fff640 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd2340090b0 tx=0x7fd23403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4607bbe034d8ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd2397fa640 1 -- 172.21.15.1:0/2013549989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23404e040 con 0x55ecccca2670 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd2397fa640 1 -- 172.21.15.1:0/2013549989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23403e460 con 0x55ecccca2670 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.536+0000 7fd2397fa640 1 -- 172.21.15.1:0/2013549989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234037040 con 0x55ecccca2670 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- 172.21.15.1:0/2013549989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 msgr2=0x55ecccca3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd2340090b0 tx=0x7fd23403b000 comp rx=0 tx=0).stop 2024-05-09T06:39:18.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- 172.21.15.1:0/2013549989 shutdown_connections 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55eccccbff10 0x55eccccbe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 --2- 172.21.15.1:0/2013549989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- 172.21.15.1:0/2013549989 >> 172.21.15.1:0/2013549989 conn(0x55ecccc78350 msgr2=0x55ecccccc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- 172.21.15.1:0/2013549989 shutdown_connections 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- 172.21.15.1:0/2013549989 wait complete. 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.537+0000 7fd24d392740 1 -- start start 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 Processor -- start 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eccccd1440 con 0x55eccccbff10 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55eccccd0f20 con 0x55ecccca2670 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x55eccccd1a90 con 0x55ecccccd560 2024-05-09T06:39:18.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 10 client.0 _ll_register_callbacks cb 0x7fd23c0556c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57200/0 (socket says 172.21.15.1:57200) 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 -- 172.21.15.1:0/3565936720 learned_addr learned my addr 172.21.15.1:0/3565936720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd248c25640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd243fff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 -- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 msgr2=0x55eccccb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 -- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ecccca2670 msgr2=0x55ecccca3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 -- 172.21.15.1:0/3565936720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecccc74730 con 0x55eccccbff10 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd23bfff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd24415a0c0 tx=0x7fd22c0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f744616cf7a2c047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T06:39:18.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c012c00 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fd22c012da0 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd23403e460 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fd22c022050 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 10 client.10257 fetch_fsmap learned FSMap version 112 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55ecccd37270 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23403e460 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.539+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7fd22c025020 con 0x55eccccbff10 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 10 client.10257 fetch_fsmap finished waiting for FSMap version 112 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 10 client.10257 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 20 client.10257 populate_metadata read hostname 'smithi001' 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55ecccca62c0 0x55eccccbbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 4 client.10257 mds_command: new command op to 10161 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 --> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] -- command(tid 0: {"prefix": "status"}) -- 0x55ecccc73db0 con 0x55ecccca62c0 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd243fff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55ecccca62c0 0x55eccccbbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd222ffd640 10 client.10257 ms_handle_connect on v2:172.21.15.1:6836/520088510 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd243fff640 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55ecccca62c0 0x55eccccbbeb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9738f5fa5c58b092 server_cookie=62c7ddd5544bcc23 in_seq=0 out_seq=0 2024-05-09T06:39:18.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd222ffd640 1 -- 172.21.15.1:0/3565936720 <== mds.0 v2:172.21.15.1:6836/520088510 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55ecccc73db0 con 0x55ecccca62c0 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd222ffd640 10 client.10257 handle_command_reply: tid=0 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 client.10257 shutdown 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 msgr2=0x55eccccb5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd24415a0c0 tx=0x7fd22c0055b0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 shutdown_connections 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:6836/520088510,v1:172.21.15.1:6837/520088510] conn(0x55ecccca62c0 0x55eccccbbeb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55ecccccd560 0x55eccccb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eccccbff10 0x55eccccb5080 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 --2- 172.21.15.1:0/3565936720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ecccca2670 0x55ecccca3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd222ffd640 0 client.10257 ms_handle_reset on v2:172.21.15.1:6836/520088510 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 >> 172.21.15.1:0/3565936720 conn(0x55ecccc78350 msgr2=0x55ecccccc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 shutdown_connections 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.541+0000 7fd24d392740 1 -- 172.21.15.1:0/3565936720 wait complete. 2024-05-09T06:39:18.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24d392740 20 client.10257 trim_cache size 0 max 16384 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "91cbcc67-ea38-4460-ad67-94260dad3bad", 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 10161, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 4.4662623909999999, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 112, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 123, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 123, 2024-05-09T06:39:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 6.4421567890000002 2024-05-09T06:39:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 msgr2=0x7fd22406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 0x7fd22406e2f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd234034000 tx=0x7fd23402e000 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 msgr2=0x7fd2441610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007df0 tx=0x7fd23c00d4a0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 shutdown_connections 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd22406be30 0x7fd22406e2f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd24415e4b0 0x7fd2441610d0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2440c6860 0x7fd2440b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 --2- 172.21.15.1:0/2751266504 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2440b2aa0 0x7fd2441466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 >> 172.21.15.1:0/2751266504 conn(0x7fd2440b1fb0 msgr2=0x7fd2440b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 shutdown_connections 2024-05-09T06:39:18.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.542+0000 7fd24a688640 1 -- 172.21.15.1:0/2751266504 wait complete. 2024-05-09T06:39:18.541 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:39:18.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.665+0000 7f5b7cc4d640 1 Processor -- start 2024-05-09T06:39:18.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.665+0000 7f5b7cc4d640 1 -- start start 2024-05-09T06:39:18.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.666+0000 7f5b7cc4d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b78161950 0x7f5b780d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.666+0000 7f5b7cc4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b780d4940 0x7f5b780d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.666+0000 7f5b7cc4d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b780b33c0 con 0x7f5b780d0f80 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.666+0000 7f5b7cc4d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5b780d4dd0 con 0x7f5b78161950 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.666+0000 7f5b7cc4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5b780b3660 con 0x7f5b780d4940 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b78161950 0x7f5b780d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b76d9d640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5b780d0f80 0x7f5b780cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49860/0 (socket says 172.21.15.1:49860) 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b76d9d640 1 -- 172.21.15.1:0/4213133242 learned_addr learned my addr 172.21.15.1:0/4213133242 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b75d9b640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b780d4940 0x7f5b780d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7659c640 1 -- 172.21.15.1:0/4213133242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b780d4940 msgr2=0x7f5b780d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7659c640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b780d4940 0x7f5b780d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7659c640 1 -- 172.21.15.1:0/4213133242 >> v1:172.21.15.1:6789/0 conn(0x7f5b780d0f80 legacy=0x7f5b780cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7559a640 1 -- 172.21.15.1:0/4213133242 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529812223 0 0) 0x7f5b780b33c0 con 0x7f5b780d0f80 2024-05-09T06:39:18.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.667+0000 7f5b7659c640 1 -- 172.21.15.1:0/4213133242 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b780b3940 con 0x7f5b78161950 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7659c640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78161950 0x7f5b780d2400 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5b780b3590 tx=0x7f5b7002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bd8c7767ac2968 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7559a640 1 -- 172.21.15.1:0/4213133242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7002ae60 con 0x7f5b78161950 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7559a640 1 -- 172.21.15.1:0/4213133242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70007740 con 0x7f5b78161950 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7559a640 1 -- 172.21.15.1:0/4213133242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70007b10 con 0x7f5b78161950 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/4213133242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78161950 msgr2=0x7f5b780d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.668+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78161950 0x7f5b780d2400 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5b780b3590 tx=0x7f5b7002a640 comp rx=0 tx=0).stop 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/4213133242 shutdown_connections 2024-05-09T06:39:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b780d4940 0x7f5b780d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/4213133242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b78161950 0x7f5b780d2400 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/4213133242 >> 172.21.15.1:0/4213133242 conn(0x7f5b780d3e20 msgr2=0x7f5b780d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/4213133242 shutdown_connections 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.669+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/4213133242 wait complete. 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.670+0000 7f5b7cc4d640 1 Processor -- start 2024-05-09T06:39:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.670+0000 7f5b7cc4d640 1 -- start start 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78161950 0x7f5b781f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b781f7630 0x7f5b780cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b780b52d0 con 0x7f5b780d0f80 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b780b5150 con 0x7f5b781f7630 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7cc4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5b780b5450 con 0x7f5b78161950 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57216/0 (socket says 172.21.15.1:57216) 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 -- 172.21.15.1:0/1036757099 learned_addr learned my addr 172.21.15.1:0/1036757099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b75d9b640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78161950 0x7f5b781f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b76d9d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b781f7630 0x7f5b780cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 -- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78161950 msgr2=0x7f5b781f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78161950 0x7f5b781f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 -- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b781f7630 msgr2=0x7f5b780cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b781f7630 0x7f5b780cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 -- 172.21.15.1:0/1036757099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7809f390 con 0x7f5b780d0f80 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.671+0000 7f5b7659c640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e80 tx=0x7f5b70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.672+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7003f070 con 0x7f5b780d0f80 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.672+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7002ae60 con 0x7f5b780d0f80 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.672+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7814e350 con 0x7f5b780d0f80 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.672+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70009450 con 0x7f5b780d0f80 2024-05-09T06:39:18.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.672+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7814e7b0 con 0x7f5b780d0f80 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.673+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b781f7a40 con 0x7f5b780d0f80 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.673+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b70044460 con 0x7f5b780d0f80 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.673+0000 7f5b6effd640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.674+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f5b700ac750 con 0x7f5b780d0f80 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.674+0000 7f5b75d9b640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.674+0000 7f5b75d9b640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 0x7f5b5806e2f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806caf0 tx=0x7f5b60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.678+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b781f7a40 con 0x7f5b780d0f80 2024-05-09T06:39:18.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.898+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5b780752b0 con 0x7f5b780d0f80 2024-05-09T06:39:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.898+0000 7f5b6effd640 1 -- 172.21.15.1:0/1036757099 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7f5b7814e350 con 0x7f5b780d0f80 2024-05-09T06:39:18.878 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:39:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 msgr2=0x7f5b5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 0x7f5b5806e2f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806caf0 tx=0x7f5b60008040 comp rx=0 tx=0).stop 2024-05-09T06:39:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 msgr2=0x7f5b7814de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e80 tx=0x7f5b70004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 shutdown_connections 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5b5806be30 0x7f5b5806e2f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b781f7630 0x7f5b780cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5b78161950 0x7f5b781f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 --2- 172.21.15.1:0/1036757099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b780d0f80 0x7f5b7814de10 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 >> 172.21.15.1:0/1036757099 conn(0x7f5b780d3e20 msgr2=0x7f5b780d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 shutdown_connections 2024-05-09T06:39:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:18.900+0000 7f5b7cc4d640 1 -- 172.21.15.1:0/1036757099 wait complete. 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:39:18.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:39:18.892 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:39:18.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:18.892 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T06:39:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41983sec preferred_lft 41983sec 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.910 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::bcae:7aff:fe76:dd08/64 scope link 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether a2:15:8a:d8:82:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::a015:8aff:fed8:8260/64 scope link 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:39:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether ae:d6:61:e3:12:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::acd6:61ff:fee3:1216/64 scope link 2024-05-09T06:39:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:39:18.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:18.912 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:39:18.974 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:39:18.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:18.976 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T06:39:19.033 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T06:39:19.033 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:39:19.033 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:39:19.033 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:39:19.033 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:39:19.034 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T06:39:19.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:39:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:19.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:19.163 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:39:19.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:39:19.179 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:39:19.179 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:39:19.180 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:39:19.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:19.180 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T06:39:19.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:39:19.248 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:39:19.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:39:19.248 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:19.327 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:39:19.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:39:19.328 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:19.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:39:19.452 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:39:19.509 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:39:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:39:19.511 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:19.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:19.512 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:39:19.566 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:39:19.567 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:39:19.610 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T06:39:19.637 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T06:39:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:39:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:19.696 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:19.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:2024-05-09T06:39:19.715+0000 7fe030f3d480 -1 init, newargv = 0x5576500b4020 newargc=15 2024-05-09T06:39:19.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:19.698 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T06:39:19.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[84537]: starting ceph client 2024-05-09T06:39:19.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[84537]: starting fuse 2024-05-09T06:39:19.715 INFO:teuthology.orchestra.run.smithi001.stdout:42 2024-05-09T06:39:19.715 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-09T06:39:19.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:39:19.715 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T06:39:19.715 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T06:39:19.715 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:39:19.716 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.0")) 2024-05-09T06:39:19.717 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:19.836 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.0.84537.asok 2024-05-09T06:39:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:19.847 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84537.asok 2024-05-09T06:39:19.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:19.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84537.asok status 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "0", 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:39:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "84537", 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 10275, 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 10275 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1929731438 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1929731438 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.10275 192.168.0.1:0/1929731438", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/1929731438", 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 112, 2024-05-09T06:39:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 123, 2024-05-09T06:39:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:39:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T06:39:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T06:39:19.990 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:19.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:20.000 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T06:39:20.000 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:20.016 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T06:39:20.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:20.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:20.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:20.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:20.084 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-09T06:39:20.150 INFO:teuthology.orchestra.run.smithi001.stdout:./ 2024-05-09T06:39:20.150 INFO:teuthology.orchestra.run.smithi001.stdout:./parent 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/charlie 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/alpha 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/flushed/bravo 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/unflushed 2024-05-09T06:39:20.151 INFO:teuthology.orchestra.run.smithi001.stdout:./parent/unflushed/jfile 2024-05-09T06:39:20.151 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.152 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.152 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.152 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.152 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-09T06:39:20.152 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.259 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T06:39:20.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.268 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.268 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.268 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.268 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.268 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-09T06:39:20.269 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.376 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627776 2024-05-09T06:39:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.385 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.385 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.385 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.385 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.386 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-09T06:39:20.386 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.493 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627777 2024-05-09T06:39:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.503 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.504 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.504 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.504 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.504 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-09T06:39:20.505 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.612 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628280 2024-05-09T06:39:20.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.621 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.622 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.622 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.622 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.622 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-09T06:39:20.622 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.730 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627778 2024-05-09T06:39:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.738 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.739 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.739 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.739 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.739 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-09T06:39:20.739 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.847 INFO:teuthology.orchestra.run.smithi001.stdout:1099511628279 2024-05-09T06:39:20.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.856 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.856 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.857 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.857 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.857 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-09T06:39:20.857 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:20.964 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627779 2024-05-09T06:39:20.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> stdin-killer -- python3 -c ' 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> import stat 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-09T06:39:20.974 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T06:39:21.082 INFO:teuthology.orchestra.run.smithi001.stdout:1099511627780 2024-05-09T06:39:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:39:21.092 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:39:21.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:39:21.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:39:21.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:39:21.215 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:39:21.215 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:21.215 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:21.232 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:21.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.397+0000 7fd8b5c2a640 1 Processor -- start 2024-05-09T06:39:21.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.397+0000 7fd8b5c2a640 1 -- start start 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8b5c2a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd8b01454b0 0x7fd8b00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8b5c2a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8b0070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8b5c2a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b006eb40 con 0x7fd8b00b2a60 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8b5c2a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b006ee20 con 0x7fd8b01454b0 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8b5c2a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b006d8e0 con 0x7fd8b00b4f70 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8aeffd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd8b00b2a60 0x7fd8b00b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49864/0 (socket says 172.21.15.1:49864) 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8aeffd640 1 -- 172.21.15.1:0/3736135517 learned_addr learned my addr 172.21.15.1:0/3736135517 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3736135517 >> v2:172.21.15.6:3300/0 conn(0x7fd8b01454b0 0x7fd8b00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8affff640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8b0070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911345757 0 0) 0x7fd8b006eb40 con 0x7fd8b00b2a60 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd898003520 con 0x7fd8b00b2a60 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.398+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2358938949 0 0) 0x7fd898003520 con 0x7fd8b00b2a60 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 msgr2=0x7fd8b0070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8b0070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 >> v2:172.21.15.6:3300/0 conn(0x7fd8b01454b0 msgr2=0x7fd8b00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 --2- 172.21.15.1:0/3736135517 >> v2:172.21.15.6:3300/0 conn(0x7fd8b01454b0 0x7fd8b00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b006dbc0 con 0x7fd8b00b2a60 2024-05-09T06:39:21.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7fd8a4001670 con 0x7fd8b00b2a60 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 >> v1:172.21.15.1:6789/0 conn(0x7fd8b00b2a60 legacy=0x7fd8b00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd898003f30 0x7fd898004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898004d40 0x7fd8980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b006ee20 con 0x7fd898003f30 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b006d8e0 con 0x7fd898004d40 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd89800b9c0 con 0x7fd8b00b4f70 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd898003f30 0x7fd898004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8affff640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898004d40 0x7fd8980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 -- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 msgr2=0x7fd8980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b00b4f70 0x7fd8980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 -- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898004d40 msgr2=0x7fd8980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898004d40 0x7fd8980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.399+0000 7fd8aeffd640 1 -- 172.21.15.1:0/3736135517 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd898003520 con 0x7fd898003f30 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8aeffd640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd898003f30 0x7fd898004800 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c290 tx=0x7fd898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5747bb9b6fbe2d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4007d50 con 0x7fd898003f30 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a402e070 con 0x7fd898003f30 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8ae7fc640 1 -- 172.21.15.1:0/3736135517 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4005d70 con 0x7fd898003f30 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3736135517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd898003f30 msgr2=0x7fd898004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3736135517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd898003f30 0x7fd898004800 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c290 tx=0x7fd898005830 comp rx=0 tx=0).stop 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.400+0000 7fd8affff640 1 -- 172.21.15.1:0/3736135517 reap_dead start 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.401+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3736135517 shutdown_connections 2024-05-09T06:39:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.401+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3736135517 >> 172.21.15.1:0/3736135517 conn(0x7fd8b00cc4b0 msgr2=0x7fd8b00cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.401+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3736135517 shutdown_connections 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.401+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3736135517 wait complete. 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.402+0000 7fd8b5c2a640 1 Processor -- start 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.402+0000 7fd8b5c2a640 1 -- start start 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 0x7fd8b015ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b015b200 0x7fd8b01e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b006e260 con 0x7fd8b015b730 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b006e0e0 con 0x7fd898003f30 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b0070c10 con 0x7fd8b015b200 2024-05-09T06:39:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57232/0 (socket says 172.21.15.1:57232) 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 -- 172.21.15.1:0/3146709502 learned_addr learned my addr 172.21.15.1:0/3146709502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8aeffd640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b015b200 0x7fd8b01e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 0x7fd8b015ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 -- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b015b200 msgr2=0x7fd8b01e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b015b200 0x7fd8b01e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 -- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 msgr2=0x7fd8b015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 0x7fd8b015ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 -- 172.21.15.1:0/3146709502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b01e1340 con 0x7fd8b015b730 2024-05-09T06:39:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 0x7fd8b015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8affff640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00b5400 tx=0x7fd8a800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8019070 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a8011e90 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003bd0 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.403+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b01e1620 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.404+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b01e3e10 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.405+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b01e4010 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.405+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8a801c070 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.406+0000 7fd897fff640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 0x7fd89c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.406+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 0x7fd89c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.406+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fd8a808b950 con 0x7fd8b015b730 2024-05-09T06:39:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.406+0000 7fd8af7fe640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 0x7fd89c06e320 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009bc0 tx=0x7fd8a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.410+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8b01e4010 con 0x7fd8b015b730 2024-05-09T06:39:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.593+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8b00752b0 con 0x7fd8b015b730 2024-05-09T06:39:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.595+0000 7fd897fff640 1 -- 172.21.15.1:0/3146709502 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17075 (secure 0 0 0) 0x7fd8a8010040 con 0x7fd8b015b730 2024-05-09T06:39:21.574 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:21.574 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":123,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:39:12.063134+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.596+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 msgr2=0x7fd89c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.596+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 0x7fd89c06e320 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009bc0 tx=0x7fd8a0008040 comp rx=0 tx=0).stop 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.596+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 msgr2=0x7fd8b01e3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.596+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00b5400 tx=0x7fd8a800f580 comp rx=0 tx=0).stop 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 shutdown_connections 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd89c06be60 0x7fd89c06e320 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b015b730 0x7fd8b01e3230 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd8b015b200 0x7fd8b01e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 --2- 172.21.15.1:0/3146709502 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd898003f30 0x7fd8b015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 >> 172.21.15.1:0/3146709502 conn(0x7fd8b00cc4b0 msgr2=0x7fd8b00b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 shutdown_connections 2024-05-09T06:39:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.597+0000 7fd8b5c2a640 1 -- 172.21.15.1:0/3146709502 wait complete. 2024-05-09T06:39:21.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:39:21.588 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:39:21.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.716+0000 7fd2cc363640 1 Processor -- start 2024-05-09T06:39:21.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.716+0000 7fd2cc363640 1 -- start start 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2cc363640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c40cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2cc363640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2c40ceba0 0x7fd2c40ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2cc363640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c40b1340 con 0x7fd2c40d08b0 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2cc363640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c40cc260 con 0x7fd2c40ceba0 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2cc363640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c40d1020 con 0x7fd2c415e270 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca900640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd2c40ceba0 0x7fd2c40ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2c98fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd2c40d08b0 0x7fd2c40ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49878/0 (socket says 172.21.15.1:49878) 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2c98fe640 1 -- 172.21.15.1:0/1523345604 learned_addr learned my addr 172.21.15.1:0/1523345604 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca0ff640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c40cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca900640 1 -- 172.21.15.1:0/1523345604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 msgr2=0x7fd2c40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca900640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c40cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca900640 1 -- 172.21.15.1:0/1523345604 >> v1:172.21.15.1:6789/0 conn(0x7fd2c40d08b0 legacy=0x7fd2c40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2ca900640 1 -- 172.21.15.1:0/1523345604 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c40cc4b0 con 0x7fd2c40ceba0 2024-05-09T06:39:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.717+0000 7fd2c90fd640 1 -- 172.21.15.1:0/1523345604 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906816612 0 0) 0x7fd2b4001110 con 0x7fd2c40d08b0 2024-05-09T06:39:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.718+0000 7fd2ca900640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c40ccf90 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097a0 tx=0x7fd2bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2398460536952b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.718+0000 7fd2c90fd640 1 -- 172.21.15.1:0/1523345604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc037070 con 0x7fd2c40ceba0 2024-05-09T06:39:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.718+0000 7fd2c90fd640 1 -- 172.21.15.1:0/1523345604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc032440 con 0x7fd2c40ceba0 2024-05-09T06:39:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.718+0000 7fd2c90fd640 1 -- 172.21.15.1:0/1523345604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc032810 con 0x7fd2c40ceba0 2024-05-09T06:39:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 -- 172.21.15.1:0/1523345604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 msgr2=0x7fd2c40ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c40ccf90 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097a0 tx=0x7fd2bc02a210 comp rx=0 tx=0).stop 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 -- 172.21.15.1:0/1523345604 shutdown_connections 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c40ccf90 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 --2- 172.21.15.1:0/1523345604 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 -- 172.21.15.1:0/1523345604 >> 172.21.15.1:0/1523345604 conn(0x7fd2c40d1c10 msgr2=0x7fd2c40d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 -- 172.21.15.1:0/1523345604 shutdown_connections 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.719+0000 7fd2cc363640 1 -- 172.21.15.1:0/1523345604 wait complete. 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.720+0000 7fd2cc363640 1 Processor -- start 2024-05-09T06:39:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.720+0000 7fd2cc363640 1 -- start start 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40d08b0 0x7fd2c4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c41609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c415e650 con 0x7fd2c40ceba0 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c40cca70 con 0x7fd2c40d08b0 2024-05-09T06:39:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2cc363640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c40d0cf0 con 0x7fd2c415e270 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57246/0 (socket says 172.21.15.1:57246) 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 -- 172.21.15.1:0/3512360354 learned_addr learned my addr 172.21.15.1:0/3512360354 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca900640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c41609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 -- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 msgr2=0x7fd2c41609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c41609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 -- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40d08b0 msgr2=0x7fd2c4154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40d08b0 0x7fd2c4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 -- 172.21.15.1:0/3512360354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c4156890 con 0x7fd2c40ceba0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2ca0ff640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0007ef0 tx=0x7fd2c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.721+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0014620 con 0x7fd2c40ceba0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.722+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0019070 con 0x7fd2c40ceba0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.722+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0014a00 con 0x7fd2c40ceba0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.722+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c4156b70 con 0x7fd2c40ceba0 2024-05-09T06:39:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.722+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c4157130 con 0x7fd2c40ceba0 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.723+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40752b0 con 0x7fd2c40ceba0 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.723+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c0014ba0 con 0x7fd2c40ceba0 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.724+0000 7fd2baffd640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 0x7fd2a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.724+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fd2c008ba30 con 0x7fd2c40ceba0 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.724+0000 7fd2c98fe640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 0x7fd2a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.724+0000 7fd2c98fe640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 0x7fd2a806e320 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cffc0 tx=0x7fd2b4001310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:21.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.728+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2c005a1a0 con 0x7fd2c40ceba0 2024-05-09T06:39:21.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.911+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2c4161800 con 0x7fd2c40ceba0 2024-05-09T06:39:21.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.911+0000 7fd2baffd640 1 -- 172.21.15.1:0/3512360354 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7fd2c0059aa0 con 0x7fd2c40ceba0 2024-05-09T06:39:21.891 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:39:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 msgr2=0x7fd2a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 0x7fd2a806e320 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cffc0 tx=0x7fd2b4001310 comp rx=0 tx=0).stop 2024-05-09T06:39:21.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 msgr2=0x7fd2c4152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0007ef0 tx=0x7fd2c000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 shutdown_connections 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fd2a806be60 0x7fd2a806e320 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd2c415e270 0x7fd2c41609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2c40d08b0 0x7fd2c4154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.913+0000 7fd2cc363640 1 --2- 172.21.15.1:0/3512360354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2c40ceba0 0x7fd2c4152c50 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.914+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 >> 172.21.15.1:0/3512360354 conn(0x7fd2c40d1c10 msgr2=0x7fd2c40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.914+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 shutdown_connections 2024-05-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:21.914+0000 7fd2cc363640 1 -- 172.21.15.1:0/3512360354 wait complete. 2024-05-09T06:39:21.905 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:39:22.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.032+0000 7f4a97c4d640 1 Processor -- start 2024-05-09T06:39:22.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.032+0000 7f4a97c4d640 1 -- start start 2024-05-09T06:39:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a97c4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a9006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a97c4d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9015b420 con 0x7f4a90158a50 2024-05-09T06:39:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a97c4d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9015b6e0 con 0x7f4a9015c4a0 2024-05-09T06:39:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a97c4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4a90158e30 con 0x7f4a90071fd0 2024-05-09T06:39:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a959e9640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4a90158a50 0x7f4a900718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49886/0 (socket says 172.21.15.1:49886) 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a959e9640 1 -- 172.21.15.1:0/2257257857 learned_addr learned my addr 172.21.15.1:0/2257257857 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.033+0000 7f4a951e8640 1 --2- 172.21.15.1:0/2257257857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a9006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a949e7640 1 -- 172.21.15.1:0/2257257857 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732778795 0 0) 0x7f4a9015b420 con 0x7f4a90158a50 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a949e7640 1 -- 172.21.15.1:0/2257257857 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a78003520 con 0x7f4a90158a50 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a951e8640 1 -- 172.21.15.1:0/2257257857 >> v1:172.21.15.6:6789/0 conn(0x7f4a9015c4a0 legacy=0x7f4a9015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a951e8640 1 -- 172.21.15.1:0/2257257857 >> v1:172.21.15.1:6789/0 conn(0x7f4a90158a50 legacy=0x7f4a900718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a951e8640 1 -- 172.21.15.1:0/2257257857 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9015b9c0 con 0x7f4a90071fd0 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.034+0000 7f4a951e8640 1 --2- 172.21.15.1:0/2257257857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a9006e6d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097d0 tx=0x7f4a8402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d5ca3faa4c4523 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a949e7640 1 -- 172.21.15.1:0/2257257857 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84037070 con 0x7f4a90071fd0 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a949e7640 1 -- 172.21.15.1:0/2257257857 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84032440 con 0x7f4a90071fd0 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a949e7640 1 -- 172.21.15.1:0/2257257857 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840327f0 con 0x7f4a90071fd0 2024-05-09T06:39:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 -- 172.21.15.1:0/2257257857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 msgr2=0x7f4a9006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/2257257857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a9006e6d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097d0 tx=0x7f4a8402a210 comp rx=0 tx=0).stop 2024-05-09T06:39:22.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 -- 172.21.15.1:0/2257257857 shutdown_connections 2024-05-09T06:39:22.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/2257257857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a9006e6d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 -- 172.21.15.1:0/2257257857 >> 172.21.15.1:0/2257257857 conn(0x7f4a900cc4d0 msgr2=0x7f4a900cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 -- 172.21.15.1:0/2257257857 shutdown_connections 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.035+0000 7f4a97c4d640 1 -- 172.21.15.1:0/2257257857 wait complete. 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 Processor -- start 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 -- start start 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a90154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 0x7f4a9006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9015bf80 con 0x7f4a90158a50 2024-05-09T06:39:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90072480 con 0x7f4a9015c4a0 2024-05-09T06:39:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a97c4d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4a90070ea0 con 0x7f4a90071fd0 2024-05-09T06:39:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a951e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a951e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57252/0 (socket says 172.21.15.1:57252) 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.037+0000 7f4a951e8640 1 -- 172.21.15.1:0/1679597011 learned_addr learned my addr 172.21.15.1:0/1679597011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a959e9640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a90154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a961ea640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 0x7f4a9006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 -- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 msgr2=0x7f4a90154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a90154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 -- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 msgr2=0x7f4a9006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 0x7f4a9006dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 -- 172.21.15.1:0/1679597011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a900ae910 con 0x7f4a90158a50 2024-05-09T06:39:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a959e9640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a90154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a961ea640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 0x7f4a9006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a951e8640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097d0 tx=0x7f4a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84037050 con 0x7f4a90158a50 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84043070 con 0x7f4a90158a50 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a90157110 con 0x7f4a90158a50 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84032810 con 0x7f4a90158a50 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.038+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a900aebb0 con 0x7f4a90158a50 2024-05-09T06:39:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.039+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a90073dc0 con 0x7f4a90158a50 2024-05-09T06:39:22.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.040+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a840329b0 con 0x7f4a90158a50 2024-05-09T06:39:22.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.040+0000 7f4a8e7fc640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 0x7f4a7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.040+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f4a840abb80 con 0x7f4a90158a50 2024-05-09T06:39:22.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.040+0000 7f4a959e9640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 0x7f4a7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.041+0000 7f4a959e9640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 0x7f4a7c06e2f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a80001560 tx=0x7f4a80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.044+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a90073dc0 con 0x7f4a90158a50 2024-05-09T06:39:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.228+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a900b2b20 con 0x7f4a90158a50 2024-05-09T06:39:22.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.229+0000 7f4a8e7fc640 1 -- 172.21.15.1:0/1679597011 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f4a8407a2f0 con 0x7f4a90158a50 2024-05-09T06:39:22.208 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 msgr2=0x7f4a7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 0x7f4a7c06e2f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a80001560 tx=0x7f4a80009040 comp rx=0 tx=0).stop 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 msgr2=0x7f4a9006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f4a840097d0 tx=0x7f4a840046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 shutdown_connections 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4a7c06be30 0x7f4a7c06e2f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a9015c4a0 0x7f4a9006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a90158a50 0x7f4a9006da60 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 --2- 172.21.15.1:0/1679597011 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4a90071fd0 0x7f4a90154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 >> 172.21.15.1:0/1679597011 conn(0x7f4a900cc4d0 msgr2=0x7f4a90159040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 shutdown_connections 2024-05-09T06:39:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.231+0000 7f4a97c4d640 1 -- 172.21.15.1:0/1679597011 wait complete. 2024-05-09T06:39:22.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:39:22.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.349+0000 7f682384d640 1 Processor -- start 2024-05-09T06:39:22.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 -- start start 2024-05-09T06:39:22.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f681c15e190 0x7f681c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 0x7f681c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681c161fe0 con 0x7f681c0d1c70 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f681c15f580 con 0x7f681c15e190 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.350+0000 7f682384d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f681c1622c0 con 0x7f681c0d45f0 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6820de8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f681c0d1c70 0x7f681c0d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49888/0 (socket says 172.21.15.1:49888) 2024-05-09T06:39:22.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6820de8640 1 -- 172.21.15.1:0/2934316118 learned_addr learned my addr 172.21.15.1:0/2934316118 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6821dea640 1 --2- 172.21.15.1:0/2934316118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 0x7f681c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f68215e9640 1 --2- 172.21.15.1:0/2934316118 >> v2:172.21.15.6:3300/0 conn(0x7f681c15e190 0x7f681c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3427791883 0 0) 0x7f681c161fe0 con 0x7f681c0d1c70 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6804003520 con 0x7f681c0d1c70 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2525084433 0 0) 0x7f6804003520 con 0x7f681c0d1c70 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6821dea640 1 -- 172.21.15.1:0/2934316118 >> v2:172.21.15.6:3300/0 conn(0x7f681c15e190 msgr2=0x7f681c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6821dea640 1 --2- 172.21.15.1:0/2934316118 >> v2:172.21.15.6:3300/0 conn(0x7f681c15e190 0x7f681c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6821dea640 1 -- 172.21.15.1:0/2934316118 >> v1:172.21.15.1:6789/0 conn(0x7f681c0d1c70 legacy=0x7f681c0d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.351+0000 7f6821dea640 1 -- 172.21.15.1:0/2934316118 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c1625a0 con 0x7f681c0d45f0 2024-05-09T06:39:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.352+0000 7f6821dea640 1 --2- 172.21.15.1:0/2934316118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 0x7f681c0b3de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fd9b1063aab700 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.352+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810001670 con 0x7f681c0d45f0 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.352+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680c007540 con 0x7f681c0d45f0 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.352+0000 7f681bfff640 1 -- 172.21.15.1:0/2934316118 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c007910 con 0x7f681c0d45f0 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 -- 172.21.15.1:0/2934316118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 msgr2=0x7f681c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 --2- 172.21.15.1:0/2934316118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 0x7f681c0b3de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c02a5f0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 -- 172.21.15.1:0/2934316118 shutdown_connections 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 --2- 172.21.15.1:0/2934316118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d45f0 0x7f681c0b3de0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 --2- 172.21.15.1:0/2934316118 >> v2:172.21.15.6:3300/0 conn(0x7f681c15e190 0x7f681c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 -- 172.21.15.1:0/2934316118 >> 172.21.15.1:0/2934316118 conn(0x7f681c073340 msgr2=0x7f681c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 -- 172.21.15.1:0/2934316118 shutdown_connections 2024-05-09T06:39:22.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.353+0000 7f682384d640 1 -- 172.21.15.1:0/2934316118 wait complete. 2024-05-09T06:39:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.354+0000 7f682384d640 1 Processor -- start 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.354+0000 7f682384d640 1 -- start start 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d1c70 0x7f681c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 0x7f681c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f681c0b56d0 con 0x7f681c15e190 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f681c161c60 con 0x7f681c0d45f0 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f682384d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f681c0b5850 con 0x7f681c0d1c70 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57262/0 (socket says 172.21.15.1:57262) 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 -- 172.21.15.1:0/541721000 learned_addr learned my addr 172.21.15.1:0/541721000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f68215e9640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d1c70 0x7f681c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6820de8640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 0x7f681c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 -- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d1c70 msgr2=0x7f681c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d1c70 0x7f681c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 -- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 msgr2=0x7f681c0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 0x7f681c0cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 -- 172.21.15.1:0/541721000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c0cc8f0 con 0x7f681c15e190 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6820de8640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 0x7f681c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.355+0000 7f6821dea640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.356+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c040070 con 0x7f681c15e190 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.356+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680c031b40 con 0x7f681c15e190 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.356+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c0ccbd0 con 0x7f681c15e190 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.356+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c007e90 con 0x7f681c15e190 2024-05-09T06:39:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.356+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c0cd190 con 0x7f681c15e190 2024-05-09T06:39:22.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.357+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c0752b0 con 0x7f681c15e190 2024-05-09T06:39:22.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.357+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f680c039040 con 0x7f681c15e190 2024-05-09T06:39:22.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.358+0000 7f6819ffb640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 0x7f680806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.358+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f680c0ac660 con 0x7f681c15e190 2024-05-09T06:39:22.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.358+0000 7f68215e9640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 0x7f680806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.358+0000 7f68215e9640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 0x7f680806e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6814009a30 tx=0x7f6814008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.362+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f680c04b340 con 0x7f681c15e190 2024-05-09T06:39:22.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.545+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f681c08e1c0 con 0x7f681c15e190 2024-05-09T06:39:22.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.545+0000 7f6819ffb640 1 -- 172.21.15.1:0/541721000 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7f680c07ad30 con 0x7f681c15e190 2024-05-09T06:39:22.525 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:39:22.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 msgr2=0x7f680806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 0x7f680806e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6814009a30 tx=0x7f6814008040 comp rx=0 tx=0).stop 2024-05-09T06:39:22.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 msgr2=0x7f681c0b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:22.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 shutdown_connections 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f680806be30 0x7f680806e2f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f681c15e190 0x7f681c0b2060 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f681c0d45f0 0x7f681c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 --2- 172.21.15.1:0/541721000 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f681c0d1c70 0x7f681c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 >> 172.21.15.1:0/541721000 conn(0x7f681c073340 msgr2=0x7f681c0d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 shutdown_connections 2024-05-09T06:39:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.548+0000 7f682384d640 1 -- 172.21.15.1:0/541721000 wait complete. 2024-05-09T06:39:22.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:39:22.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.664+0000 7fc9366d4640 1 Processor -- start 2024-05-09T06:39:22.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.664+0000 7fc9366d4640 1 -- start start 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc9366d4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9300d2b10 0x7fc9300b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc9366d4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9300b33c0 con 0x7fc9300d4940 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc9366d4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9300b3660 con 0x7fc930161950 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc9366d4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc9300d4dd0 con 0x7fc9300d2b10 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc934c71640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9300d2b10 0x7fc9300b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc927fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc9300d4940 0x7fc9300b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49892/0 (socket says 172.21.15.1:49892) 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc927fff640 1 -- 172.21.15.1:0/2148924085 learned_addr learned my addr 172.21.15.1:0/2148924085 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc934c71640 1 -- 172.21.15.1:0/2148924085 >> v1:172.21.15.6:6789/0 conn(0x7fc930161950 legacy=0x7fc9300d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc934c71640 1 -- 172.21.15.1:0/2148924085 >> v1:172.21.15.1:6789/0 conn(0x7fc9300d4940 legacy=0x7fc9300b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc92f7fe640 1 -- 172.21.15.1:0/2148924085 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786040416 0 0) 0x7fc9300b3660 con 0x7fc930161950 2024-05-09T06:39:22.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.665+0000 7fc934c71640 1 -- 172.21.15.1:0/2148924085 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9300b3940 con 0x7fc9300d2b10 2024-05-09T06:39:22.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.666+0000 7fc934c71640 1 --2- 172.21.15.1:0/2148924085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300b4850 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917b86dc43ba3203 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.666+0000 7fc92f7fe640 1 -- 172.21.15.1:0/2148924085 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928007740 con 0x7fc9300d2b10 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.666+0000 7fc92f7fe640 1 -- 172.21.15.1:0/2148924085 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9280078e0 con 0x7fc9300d2b10 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.666+0000 7fc92f7fe640 1 -- 172.21.15.1:0/2148924085 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928007cb0 con 0x7fc9300d2b10 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 -- 172.21.15.1:0/2148924085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 msgr2=0x7fc9300b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2148924085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300b4850 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802a5f0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 -- 172.21.15.1:0/2148924085 shutdown_connections 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2148924085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300b4850 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 -- 172.21.15.1:0/2148924085 >> 172.21.15.1:0/2148924085 conn(0x7fc9300d3e20 msgr2=0x7fc9300d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 -- 172.21.15.1:0/2148924085 shutdown_connections 2024-05-09T06:39:22.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.667+0000 7fc9366d4640 1 -- 172.21.15.1:0/2148924085 wait complete. 2024-05-09T06:39:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.668+0000 7fc9366d4640 1 Processor -- start 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.668+0000 7fc9366d4640 1 -- start start 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 0x7fc930153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9300b8550 0x7fc9300b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300d1100 con 0x7fc9300d2b10 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300d0f80 con 0x7fc9300b8550 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc9366d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc9300d1c00 con 0x7fc930161950 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57284/0 (socket says 172.21.15.1:57284) 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 -- 172.21.15.1:0/2807691903 learned_addr learned my addr 172.21.15.1:0/2807691903 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc927fff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 0x7fc930153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc934c71640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9300b8550 0x7fc9300b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 -- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 msgr2=0x7fc930153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 0x7fc930153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 -- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9300b8550 msgr2=0x7fc9300b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9300b8550 0x7fc9300b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc92ffff640 1 -- 172.21.15.1:0/2807691903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9300cbaa0 con 0x7fc9300d2b10 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.669+0000 7fc927fff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 0x7fc930153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc92ffff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c410 tx=0x7fc92000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920017070 con 0x7fc9300d2b10 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9200052d0 con 0x7fc9300d2b10 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9300cbd80 con 0x7fc9300d2b10 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920003810 con 0x7fc9300d2b10 2024-05-09T06:39:22.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.670+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9300cc340 con 0x7fc9300d2b10 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.671+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9300cc540 con 0x7fc9300d2b10 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.671+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc92001a070 con 0x7fc9300d2b10 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.672+0000 7fc92d7fa640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.672+0000 7fc927fff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.672+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fc92008a4a0 con 0x7fc9300d2b10 2024-05-09T06:39:22.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.672+0000 7fc927fff640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 0x7fc91006e2f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc918007fd0 tx=0x7fc918007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:22.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.676+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc920058c10 con 0x7fc9300d2b10 2024-05-09T06:39:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.871+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9300752b0 con 0x7fc9300d2b10 2024-05-09T06:39:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.871+0000 7fc92d7fa640 1 -- 172.21.15.1:0/2807691903 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v123) ==== 80+0+240 (secure 0 0 0) 0x7fc920058510 con 0x7fc9300d2b10 2024-05-09T06:39:22.851 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6837/3597643567 2024-05-10T06:39:12.063103+0000 2024-05-09T06:39:22.851 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6836/3597643567 2024-05-10T06:39:12.063103+0000 2024-05-09T06:39:22.851 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6837/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:22.851 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6836/1060593353 2024-05-10T06:38:53.413211+0000 2024-05-09T06:39:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:listed 4 entries 2024-05-09T06:39:22.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 msgr2=0x7fc91006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 0x7fc91006e2f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc918007fd0 tx=0x7fc918007d40 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 msgr2=0x7fc9300cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c410 tx=0x7fc92000c8d0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 shutdown_connections 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9300b8550 0x7fc9300b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc930161950 0x7fc930153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 --2- 172.21.15.1:0/2807691903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9300d2b10 0x7fc9300cd120 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 >> 172.21.15.1:0/2807691903 conn(0x7fc9300d3e20 msgr2=0x7fc9300b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 shutdown_connections 2024-05-09T06:39:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:22.873+0000 7fc9366d4640 1 -- 172.21.15.1:0/2807691903 wait complete. 2024-05-09T06:39:22.863 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T06:39:22.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:22.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:39:22.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi001.stderr:ceph-fuse[84537]: fuse finished with error 0 and tester_r 0 2024-05-09T06:39:22.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:39:22.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:22.890 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:39:23.056 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:39:23.057 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:39:23.057 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:23.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:39:23.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:39:23.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:23.077 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:39:23.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:39:23.136 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-09T06:39:23.137 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:39:23.137 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:23.137 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:23.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:39:23.155 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:39:23.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:23.156 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:39:23.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:39:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-09T06:39:23.212 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:39:23.212 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:39:23.212 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:23.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:39:23.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:39:23.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:23.232 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:39:23.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:39:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-09T06:39:23.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.416+0000 7f768a56a640 1 Processor -- start 2024-05-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 -- start start 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7684161950 0x7f76840d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d4940 0x7f76840d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76840b33c0 con 0x7f76840d0f80 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f76840d4dd0 con 0x7f7684161950 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.417+0000 7f768a56a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76840b3660 con 0x7f76840d4940 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7684161950 0x7f76840d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7684161950 0x7f76840d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57298/0 (socket says 172.21.15.1:57298) 2024-05-09T06:39:23.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 -- 172.21.15.1:0/1212628236 learned_addr learned my addr 172.21.15.1:0/1212628236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7688b07640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f76840d0f80 0x7f76840cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49906/0 (socket says 172.21.15.1:49906) 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f76837fe640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d4940 0x7f76840d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 -- 172.21.15.1:0/1212628236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d4940 msgr2=0x7f76840d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d4940 0x7f76840d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 -- 172.21.15.1:0/1212628236 >> v1:172.21.15.1:6789/0 conn(0x7f76840d0f80 legacy=0x7f76840cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.418+0000 7f7683fff640 1 -- 172.21.15.1:0/1212628236 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840b3940 con 0x7f7684161950 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f7683fff640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7684161950 0x7f76840d2400 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f76840b3590 tx=0x7f767402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c04051a8800cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f7682ffd640 1 -- 172.21.15.1:0/1212628236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674037070 con 0x7f7684161950 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f7682ffd640 1 -- 172.21.15.1:0/1212628236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7674032440 con 0x7f7684161950 2024-05-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f7682ffd640 1 -- 172.21.15.1:0/1212628236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674032810 con 0x7f7684161950 2024-05-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 -- 172.21.15.1:0/1212628236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7684161950 msgr2=0x7f76840d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7684161950 0x7f76840d2400 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f76840b3590 tx=0x7f767402a250 comp rx=0 tx=0).stop 2024-05-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 -- 172.21.15.1:0/1212628236 shutdown_connections 2024-05-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d4940 0x7f76840d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 --2- 172.21.15.1:0/1212628236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7684161950 0x7f76840d2400 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 -- 172.21.15.1:0/1212628236 >> 172.21.15.1:0/1212628236 conn(0x7f76840d3e20 msgr2=0x7f76840d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 -- 172.21.15.1:0/1212628236 shutdown_connections 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.419+0000 7f768a56a640 1 -- 172.21.15.1:0/1212628236 wait complete. 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 Processor -- start 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 -- start start 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d0f80 0x7f768414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7684161950 0x7f76841e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b52d0 con 0x7f76840b8880 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b5150 con 0x7f7684161950 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.421+0000 7f768a56a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f76840b5450 con 0x7f76840d0f80 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57312/0 (socket says 172.21.15.1:57312) 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 -- 172.21.15.1:0/831291529 learned_addr learned my addr 172.21.15.1:0/831291529 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7683fff640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d0f80 0x7f768414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f76837fe640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7684161950 0x7f76841e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 -- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d0f80 msgr2=0x7f768414e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d0f80 0x7f768414e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 -- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7684161950 msgr2=0x7f76841e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7684161950 0x7f76841e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 -- 172.21.15.1:0/831291529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840b77e0 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7688b07640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c7a0 tx=0x7f767c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c017070 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c00a640 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76840b7ac0 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c00a9f0 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.422+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76840b8080 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.423+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76840752b0 con 0x7f76840b8880 2024-05-09T06:39:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.424+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f767c01a070 con 0x7f76840b8880 2024-05-09T06:39:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.424+0000 7f7680ff9640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 0x7f767006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.424+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f767c08a690 con 0x7f76840b8880 2024-05-09T06:39:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.424+0000 7f7683fff640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 0x7f767006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.425+0000 7f7683fff640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 0x7f767006e2f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7674034000 tx=0x7f767402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.429+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f767c058710 con 0x7f76840b8880 2024-05-09T06:39:23.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.652+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76840b31a0 con 0x7f76840b8880 2024-05-09T06:39:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.652+0000 7f7680ff9640 1 -- 172.21.15.1:0/831291529 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5178 (secure 0 0 0) 0x7f767c05ca10 con 0x7f76840b8880 2024-05-09T06:39:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:23.632 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:17.081146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10161,"qdb_cluster":[10161]},"id":8}]} 2024-05-09T06:39:23.632 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 112 2024-05-09T06:39:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 msgr2=0x7f767006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 0x7f767006e2f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7674034000 tx=0x7f767402e000 comp rx=0 tx=0).stop 2024-05-09T06:39:23.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 msgr2=0x7f76840b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c7a0 tx=0x7f767c00cc60 comp rx=0 tx=0).stop 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 shutdown_connections 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f767006be30 0x7f767006e2f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76840b8880 0x7f76840b8d10 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7684161950 0x7f76841e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 --2- 172.21.15.1:0/831291529 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76840d0f80 0x7f768414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 >> 172.21.15.1:0/831291529 conn(0x7f76840d3e20 msgr2=0x7f768415ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 shutdown_connections 2024-05-09T06:39:23.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.655+0000 7f768a56a640 1 -- 172.21.15.1:0/831291529 wait complete. 2024-05-09T06:39:23.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.770+0000 7f011bb89640 1 Processor -- start 2024-05-09T06:39:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.770+0000 7f011bb89640 1 -- start start 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011bb89640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f01140b2a60 0x7f011415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011bb89640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0114070f40 con 0x7f011415e500 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011bb89640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01140cf170 con 0x7f011415db10 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011bb89640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f011415e950 con 0x7f01140b2a60 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011a126640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f011415e500 0x7f011406e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49914/0 (socket says 172.21.15.1:49914) 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f011a126640 1 -- 172.21.15.1:0/4158557583 learned_addr learned my addr 172.21.15.1:0/4158557583 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.771+0000 7f0119124640 1 --2- 172.21.15.1:0/4158557583 >> v2:172.21.15.6:3300/0 conn(0x7f01140b2a60 0x7f011415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0118923640 1 -- 172.21.15.1:0/4158557583 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804478835 0 0) 0x7f0114070f40 con 0x7f011415e500 2024-05-09T06:39:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0118923640 1 -- 172.21.15.1:0/4158557583 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00fc003520 con 0x7f011415e500 2024-05-09T06:39:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0119124640 1 -- 172.21.15.1:0/4158557583 >> v1:172.21.15.6:6789/0 conn(0x7f011415db10 legacy=0x7f01140715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0119124640 1 -- 172.21.15.1:0/4158557583 >> v1:172.21.15.1:6789/0 conn(0x7f011415e500 legacy=0x7f011406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0119124640 1 -- 172.21.15.1:0/4158557583 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01140cf450 con 0x7f01140b2a60 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.772+0000 7f0119124640 1 --2- 172.21.15.1:0/4158557583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01140b2a60 0x7f011415dfc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f01080097a0 tx=0x7f010802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a0f6b7acc890d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f0118923640 1 -- 172.21.15.1:0/4158557583 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108037070 con 0x7f01140b2a60 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f0118923640 1 -- 172.21.15.1:0/4158557583 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0108032440 con 0x7f01140b2a60 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f0118923640 1 -- 172.21.15.1:0/4158557583 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01080327f0 con 0x7f01140b2a60 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 -- 172.21.15.1:0/4158557583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01140b2a60 msgr2=0x7f011415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 --2- 172.21.15.1:0/4158557583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01140b2a60 0x7f011415dfc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f01080097a0 tx=0x7f010802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 -- 172.21.15.1:0/4158557583 shutdown_connections 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 --2- 172.21.15.1:0/4158557583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01140b2a60 0x7f011415dfc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 -- 172.21.15.1:0/4158557583 >> 172.21.15.1:0/4158557583 conn(0x7f01140cc4b0 msgr2=0x7f01140cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 -- 172.21.15.1:0/4158557583 shutdown_connections 2024-05-09T06:39:23.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.773+0000 7f011bb89640 1 -- 172.21.15.1:0/4158557583 wait complete. 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.774+0000 7f011bb89640 1 Processor -- start 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 -- start start 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01140b2a60 0x7f0114153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f011415db10 0x7f0114153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f011415edc0 con 0x7f011415e500 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f011415ec40 con 0x7f011415db10 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011bb89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f011406ea50 con 0x7f01140b2a60 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011a126640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011a126640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57316/0 (socket says 172.21.15.1:57316) 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f011a126640 1 -- 172.21.15.1:0/696796292 learned_addr learned my addr 172.21.15.1:0/696796292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f0119925640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01140b2a60 0x7f0114153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.775+0000 7f0119124640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f011415db10 0x7f0114153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 -- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01140b2a60 msgr2=0x7f0114153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01140b2a60 0x7f0114153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 -- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f011415db10 msgr2=0x7f0114153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f011415db10 0x7f0114153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 -- 172.21.15.1:0/696796292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141513c0 con 0x7f011415e500 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011a126640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f010400caa0 tx=0x7f010400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104017070 con 0x7f011415e500 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010400ac00 con 0x7f011415e500 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141516a0 con 0x7f011415e500 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104012400 con 0x7f011415e500 2024-05-09T06:39:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.776+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0114151c60 con 0x7f011415e500 2024-05-09T06:39:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.777+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01140752b0 con 0x7f011415e500 2024-05-09T06:39:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.778+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f010401a070 con 0x7f011415e500 2024-05-09T06:39:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.778+0000 7f01127fc640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 0x7f010006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.778+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f010408ad90 con 0x7f011415e500 2024-05-09T06:39:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.778+0000 7f0119925640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 0x7f010006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.779+0000 7f0119925640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 0x7f010006e2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0114153110 tx=0x7f010c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.783+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0104058f40 con 0x7f011415e500 2024-05-09T06:39:23.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.965+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01140696e0 con 0x7f011415e500 2024-05-09T06:39:23.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.966+0000 7f01127fc640 1 -- 172.21.15.1:0/696796292 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17075 (secure 0 0 0) 0x7f010405d240 con 0x7f011415e500 2024-05-09T06:39:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:23.946 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":123,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:39:12.063134+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 msgr2=0x7f010006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 0x7f010006e2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0114153110 tx=0x7f010c009040 comp rx=0 tx=0).stop 2024-05-09T06:39:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 msgr2=0x7f0114148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f010400caa0 tx=0x7f010400cf60 comp rx=0 tx=0).stop 2024-05-09T06:39:23.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 shutdown_connections 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f010006be30 0x7f010006e2f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f011415e500 0x7f0114148d50 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f011415db10 0x7f0114153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 --2- 172.21.15.1:0/696796292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01140b2a60 0x7f0114153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 >> 172.21.15.1:0/696796292 conn(0x7f01140cc4b0 msgr2=0x7f01140b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 shutdown_connections 2024-05-09T06:39:23.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:23.968+0000 7f011bb89640 1 -- 172.21.15.1:0/696796292 wait complete. 2024-05-09T06:39:23.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:27:09.112330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:38:27.404781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:38:28.082194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:39:23.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:39:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.087+0000 7f5f12770640 1 Processor -- start 2024-05-09T06:39:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.087+0000 7f5f12770640 1 -- start start 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f12770640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f12770640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 0x7f5f0c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f12770640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0c0cf1b0 con 0x7f5f0c0cdb90 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f12770640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5f0c15a8b0 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f12770640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c06dc10 con 0x7f5f0c071610 2024-05-09T06:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f0bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57332/0 (socket says 172.21.15.1:57332) 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f0bfff640 1 -- 172.21.15.1:0/2844578534 learned_addr learned my addr 172.21.15.1:0/2844578534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.088+0000 7f5f10d0d640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 0x7f5f0c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0bfff640 1 -- 172.21.15.1:0/2844578534 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 msgr2=0x7f5f0c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0bfff640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 0x7f5f0c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0bfff640 1 -- 172.21.15.1:0/2844578534 >> v1:172.21.15.6:6789/0 conn(0x7f5f0c0cdb90 legacy=0x7f5f0c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0bfff640 1 -- 172.21.15.1:0/2844578534 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0cf490 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f10d0d640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 0x7f5f0c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:24.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0bfff640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea642dab0493ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.089+0000 7f5f0b7fe640 1 -- 172.21.15.1:0/2844578534 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc037070 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f0b7fe640 1 -- 172.21.15.1:0/2844578534 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc032440 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f0b7fe640 1 -- 172.21.15.1:0/2844578534 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc032810 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 -- 172.21.15.1:0/2844578534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 msgr2=0x7f5f0c0cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02a210 comp rx=0 tx=0).stop 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 -- 172.21.15.1:0/2844578534 shutdown_connections 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c071610 0x7f5f0c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 --2- 172.21.15.1:0/2844578534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c0cea80 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 -- 172.21.15.1:0/2844578534 >> 172.21.15.1:0/2844578534 conn(0x7f5f0c0cc4d0 msgr2=0x7f5f0c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 -- 172.21.15.1:0/2844578534 shutdown_connections 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.090+0000 7f5f12770640 1 -- 172.21.15.1:0/2844578534 wait complete. 2024-05-09T06:39:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.091+0000 7f5f12770640 1 Processor -- start 2024-05-09T06:39:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.091+0000 7f5f12770640 1 -- start start 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f0c071610 0x7f5f0c1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c0cdb90 0x7f5f0c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0b2be0 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0b2a60 con 0x7f5f0c071610 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f12770640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c06cd30 con 0x7f5f0c0cdb90 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f10d0d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f10d0d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57346/0 (socket says 172.21.15.1:57346) 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f10d0d640 1 -- 172.21.15.1:0/1930588608 learned_addr learned my addr 172.21.15.1:0/1930588608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f0bfff640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f0c071610 0x7f5f0c1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f03fff640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c0cdb90 0x7f5f0c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.092+0000 7f5f10d0d640 1 -- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c0cdb90 msgr2=0x7f5f0c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f10d0d640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c0cdb90 0x7f5f0c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f10d0d640 1 -- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f0c071610 msgr2=0x7f5f0c1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f10d0d640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f0c071610 0x7f5f0c1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f10d0d640 1 -- 172.21.15.1:0/1930588608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c142ae0 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f10d0d640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400ce80 tx=0x7f5f0400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0401a070 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0400d880 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400dc70 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c14a070 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.093+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c14a630 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.094+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c072f80 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.095+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f0401d070 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.095+0000 7f5f097fa640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 0x7f5eec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.095+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f5f0408b740 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.095+0000 7f5f0bfff640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 0x7f5eec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.096+0000 7f5f0bfff640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 0x7f5eec06e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5efc034000 tx=0x7f5efc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.100+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f0c072f80 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.284+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f0c0752b0 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.284+0000 7f5f097fa640 1 -- 172.21.15.1:0/1930588608 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f5f0c142ae0 con 0x7f5f0c15a4d0 2024-05-09T06:39:24.263 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T06:39:24.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 msgr2=0x7f5eec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 0x7f5eec06e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5efc034000 tx=0x7f5efc02e000 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 msgr2=0x7f5f0c166370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400ce80 tx=0x7f5f0400a710 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 shutdown_connections 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5eec06be30 0x7f5eec06e2f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f0c15a4d0 0x7f5f0c166370 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f0c0cdb90 0x7f5f0c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 --2- 172.21.15.1:0/1930588608 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f0c071610 0x7f5f0c1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 >> 172.21.15.1:0/1930588608 conn(0x7f5f0c0cc4d0 msgr2=0x7f5f0c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 shutdown_connections 2024-05-09T06:39:24.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.286+0000 7f5f12770640 1 -- 172.21.15.1:0/1930588608 wait complete. 2024-05-09T06:39:24.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:39:24.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.405+0000 7f2c7f761640 1 Processor -- start 2024-05-09T06:39:24.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.405+0000 7f2c7f761640 1 -- start start 2024-05-09T06:39:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7f761640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2c7815e300 0x7f2c780d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7f761640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7f761640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7806c420 con 0x7f2c780ae9c0 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7f761640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2c7806f3f0 con 0x7f2c7815e300 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7f761640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7806c700 con 0x7f2c7806ffb0 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:45056/0 (socket says 172.21.15.1:45056) 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 -- 172.21.15.1:0/1372188556 learned_addr learned my addr 172.21.15.1:0/1372188556 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7d4fd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2c7815e300 0x7f2c780d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7ccfc640 1 --1- 172.21.15.1:0/1372188556 >> v1:172.21.15.6:6789/0 conn(0x7f2c780ae9c0 0x7f2c7806f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:47110/0 (socket says 172.21.15.1:47110) 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 -- 172.21.15.1:0/1372188556 >> v2:172.21.15.6:3300/0 conn(0x7f2c7815e300 msgr2=0x7f2c780d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 --2- 172.21.15.1:0/1372188556 >> v2:172.21.15.6:3300/0 conn(0x7f2c7815e300 0x7f2c780d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 -- 172.21.15.1:0/1372188556 >> v1:172.21.15.6:6789/0 conn(0x7f2c780ae9c0 legacy=0x7f2c7806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.406+0000 7f2c7dcfe640 1 -- 172.21.15.1:0/1372188556 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7806c9e0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.407+0000 7f2c7dcfe640 1 --2- 172.21.15.1:0/1372188556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009e50 tx=0x7f2c7002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d40f3b7517e0b43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.407+0000 7f2c6ffff640 1 -- 172.21.15.1:0/1372188556 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70036070 con 0x7f2c7806ffb0 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.407+0000 7f2c6ffff640 1 -- 172.21.15.1:0/1372188556 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c70031600 con 0x7f2c7806ffb0 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.407+0000 7f2c6ffff640 1 -- 172.21.15.1:0/1372188556 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700319d0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 -- 172.21.15.1:0/1372188556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 msgr2=0x7f2c780d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 --2- 172.21.15.1:0/1372188556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009e50 tx=0x7f2c7002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 -- 172.21.15.1:0/1372188556 shutdown_connections 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 --2- 172.21.15.1:0/1372188556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c7806ffb0 0x7f2c780d0e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 --2- 172.21.15.1:0/1372188556 >> v2:172.21.15.6:3300/0 conn(0x7f2c7815e300 0x7f2c780d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 -- 172.21.15.1:0/1372188556 >> 172.21.15.1:0/1372188556 conn(0x7f2c780d1c50 msgr2=0x7f2c780d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 -- 172.21.15.1:0/1372188556 shutdown_connections 2024-05-09T06:39:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.408+0000 7f2c7f761640 1 -- 172.21.15.1:0/1372188556 wait complete. 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.409+0000 7f2c7f761640 1 Processor -- start 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.409+0000 7f2c7f761640 1 -- start start 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 0x7f2c781f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c7815e300 0x7f2c78158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7806d4a0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7806cfa0 con 0x7f2c7815e300 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7f761640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7806d620 con 0x7f2c780ae9c0 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57352/0 (socket says 172.21.15.1:57352) 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 -- 172.21.15.1:0/3129971914 learned_addr learned my addr 172.21.15.1:0/3129971914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7ccfc640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 0x7f2c781f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 -- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 msgr2=0x7f2c781f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 0x7f2c781f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7dcfe640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c7815e300 0x7f2c78158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 -- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c7815e300 msgr2=0x7f2c78158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c7815e300 0x7f2c78158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 -- 172.21.15.1:0/3129971914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7809c930 con 0x7f2c7806ffb0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7ccfc640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 0x7f2c781f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.410+0000 7f2c7d4fd640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400e870 tx=0x7f2c7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.411+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74017070 con 0x7f2c7806ffb0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.411+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7400c940 con 0x7f2c7806ffb0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.411+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7809cc10 con 0x7f2c7806ffb0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.411+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400cd30 con 0x7f2c7806ffb0 2024-05-09T06:39:24.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.411+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7809d1d0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.412+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c780b6410 con 0x7f2c7806ffb0 2024-05-09T06:39:24.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.413+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c7401a070 con 0x7f2c7806ffb0 2024-05-09T06:39:24.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.413+0000 7f2c6dffb640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 0x7f2c5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.413+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f2c7408b2c0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.413+0000 7f2c7ccfc640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 0x7f2c5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.414+0000 7f2c7ccfc640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 0x7f2c5c06e320 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2c781f5640 tx=0x7f2c68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.417+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c74059a30 con 0x7f2c7806ffb0 2024-05-09T06:39:24.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.599+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c780752b0 con 0x7f2c7806ffb0 2024-05-09T06:39:24.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.600+0000 7f2c6dffb640 1 -- 172.21.15.1:0/3129971914 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f2c74059330 con 0x7f2c7806ffb0 2024-05-09T06:39:24.580 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.601+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 msgr2=0x7f2c5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.601+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 0x7f2c5c06e320 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2c781f5640 tx=0x7f2c68009040 comp rx=0 tx=0).stop 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 msgr2=0x7f2c7809bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400e870 tx=0x7f2c7400ed30 comp rx=0 tx=0).stop 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 shutdown_connections 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f2c5c06be60 0x7f2c5c06e320 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c7815e300 0x7f2c78158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2c780ae9c0 0x7f2c781f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 --2- 172.21.15.1:0/3129971914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c7806ffb0 0x7f2c7809bfa0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 >> 172.21.15.1:0/3129971914 conn(0x7f2c780d1c50 msgr2=0x7f2c7806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 shutdown_connections 2024-05-09T06:39:24.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.602+0000 7f2c7f761640 1 -- 172.21.15.1:0/3129971914 wait complete. 2024-05-09T06:39:24.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:39:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.719+0000 7f9546815640 1 Processor -- start 2024-05-09T06:39:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.719+0000 7f9546815640 1 -- start start 2024-05-09T06:39:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f9546815640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f95400ceba0 0x7f95400cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f9546815640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95400b1340 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f9546815640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95400d1020 con 0x7f95400d08b0 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f9546815640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f95400cc260 con 0x7f95400ceba0 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f953ffff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f954015e270 0x7f95400ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49928/0 (socket says 172.21.15.1:49928) 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.720+0000 7f953ffff640 1 -- 172.21.15.1:0/3560608756 learned_addr learned my addr 172.21.15.1:0/3560608756 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3560608756 >> v2:172.21.15.6:3300/0 conn(0x7f95400ceba0 0x7f95400cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019507113 0 0) 0x7f9530001110 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9528003520 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2125981879 0 0) 0x7f9528003520 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 >> v2:172.21.15.6:3300/0 conn(0x7f95400ceba0 msgr2=0x7f95400cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 --2- 172.21.15.1:0/3560608756 >> v2:172.21.15.6:3300/0 conn(0x7f95400ceba0 0x7f95400cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 >> v1:172.21.15.6:6789/0 conn(0x7f95400d08b0 legacy=0x7f95400ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95400cc4b0 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f9530001840 con 0x7f954015e270 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 >> v1:172.21.15.1:6789/0 conn(0x7f954015e270 legacy=0x7f95400ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f9528003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9528004800 0x7f9528004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95280051c0 0x7f952800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95400cc260 con 0x7f95280039b0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f952800c070 con 0x7f95280051c0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f952800c270 con 0x7f9528004800 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f9528003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9528004800 0x7f9528004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f9544db2640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95280051c0 0x7f952800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 -- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9528004800 msgr2=0x7f9528004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9528004800 0x7f9528004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 -- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95280051c0 msgr2=0x7f952800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95280051c0 0x7f952800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f9544db2640 1 -- 172.21.15.1:0/3560608756 reap_dead start 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953ffff640 1 -- 172.21.15.1:0/3560608756 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9528003520 con 0x7f95280039b0 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.721+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9528004800 0x7f9528004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:24.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.722+0000 7f953ffff640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f9528003dd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f953000c4c0 tx=0x7f9528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5249c5c00985c481 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.722+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953000c0e0 con 0x7f95280039b0 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.722+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530034070 con 0x7f95280039b0 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.722+0000 7f953effd640 1 -- 172.21.15.1:0/3560608756 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530005d30 con 0x7f95280039b0 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 -- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 msgr2=0x7f9528003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f9528003dd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f953000c4c0 tx=0x7f9528005830 comp rx=0 tx=0).stop 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 -- 172.21.15.1:0/3560608756 shutdown_connections 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 --2- 172.21.15.1:0/3560608756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f9528003dd0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 -- 172.21.15.1:0/3560608756 >> 172.21.15.1:0/3560608756 conn(0x7f95400d1c10 msgr2=0x7f95400d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 -- 172.21.15.1:0/3560608756 shutdown_connections 2024-05-09T06:39:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.723+0000 7f9546815640 1 -- 172.21.15.1:0/3560608756 wait complete. 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.724+0000 7f9546815640 1 Processor -- start 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.724+0000 7f9546815640 1 -- start start 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9540150890 0x7f954014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 0x7f95400ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f954015e650 con 0x7f95280039b0 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f95400cca70 con 0x7f95400cdee0 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9546815640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f95400d0cc0 con 0x7f9540150890 2024-05-09T06:39:24.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57372/0 (socket says 172.21.15.1:57372) 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 -- 172.21.15.1:0/3670826815 learned_addr learned my addr 172.21.15.1:0/3670826815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9540150890 0x7f954014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9544db2640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 0x7f95400ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 -- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9540150890 msgr2=0x7f954014ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9540150890 0x7f954014ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 -- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 msgr2=0x7f95400ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 0x7f95400ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f953ffff640 1 -- 172.21.15.1:0/3670826815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954014de70 con 0x7f95280039b0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.725+0000 7f9544db2640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 0x7f95400ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f953ffff640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f953000c4c0 tx=0x7f9530001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530004e60 con 0x7f95280039b0 2024-05-09T06:39:24.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953003f030 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954014d260 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530007680 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.726+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954014d820 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.727+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95400752b0 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.727+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f953002f460 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.728+0000 7f953cff9640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 0x7f952c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.728+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f95300ab970 con 0x7f95280039b0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.728+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 0x7f952c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:24.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.728+0000 7f953f7fe640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 0x7f952c06e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9540150d90 tx=0x7f9534008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:24.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.732+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9530049030 con 0x7f95280039b0 2024-05-09T06:39:24.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.915+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95400d29e0 con 0x7f95280039b0 2024-05-09T06:39:24.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.915+0000 7f953cff9640 1 -- 172.21.15.1:0/3670826815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7f95400b1340 con 0x7f95280039b0 2024-05-09T06:39:24.895 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T06:39:24.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 msgr2=0x7f952c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 0x7f952c06e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9540150d90 tx=0x7f9534008040 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 msgr2=0x7f954009f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f953000c4c0 tx=0x7f9530001c10 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 shutdown_connections 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f952c06be30 0x7f952c06e2f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95400cdee0 0x7f95400ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9540150890 0x7f954014ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 --2- 172.21.15.1:0/3670826815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95280039b0 0x7f954009f340 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 >> 172.21.15.1:0/3670826815 conn(0x7f95400d1c10 msgr2=0x7f9540156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 shutdown_connections 2024-05-09T06:39:24.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:24.917+0000 7f9546815640 1 -- 172.21.15.1:0/3670826815 wait complete. 2024-05-09T06:39:24.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:25.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.032+0000 7f1a1562a640 1 Processor -- start 2024-05-09T06:39:25.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.032+0000 7f1a1562a640 1 -- start start 2024-05-09T06:39:25.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.032+0000 7f1a1562a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a10099b30 0x7f1a100b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.032+0000 7f1a1562a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 0x7f1a1015f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a1562a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a10099930 con 0x7f1a100c6860 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a1562a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1a100b1e20 con 0x7f1a10099b30 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a1562a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1a1015ee90 con 0x7f1a1015e4b0 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a10099b30 0x7f1a100b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1a10099b30 0x7f1a100b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57382/0 (socket says 172.21.15.1:57382) 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 -- 172.21.15.1:0/1309088925 learned_addr learned my addr 172.21.15.1:0/1309088925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0f7fe640 1 --2- 172.21.15.1:0/1309088925 >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 0x7f1a1015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 -- 172.21.15.1:0/1309088925 >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 msgr2=0x7f1a1015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 --2- 172.21.15.1:0/1309088925 >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 0x7f1a1015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 -- 172.21.15.1:0/1309088925 >> v1:172.21.15.6:6789/0 conn(0x7f1a100c6860 legacy=0x7f1a100c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0e7fc640 1 -- 172.21.15.1:0/1309088925 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a10162930 con 0x7f1a10099b30 2024-05-09T06:39:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.033+0000 7f1a0f7fe640 1 --2- 172.21.15.1:0/1309088925 >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 0x7f1a1015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a0e7fc640 1 --2- 172.21.15.1:0/1309088925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a10099b30 0x7f1a100b2aa0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1a100ca960 tx=0x7f1a0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e4573c76a68cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a0dffb640 1 -- 172.21.15.1:0/1309088925 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04037070 con 0x7f1a10099b30 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a0dffb640 1 -- 172.21.15.1:0/1309088925 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04032440 con 0x7f1a10099b30 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a0dffb640 1 -- 172.21.15.1:0/1309088925 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04032810 con 0x7f1a10099b30 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 -- 172.21.15.1:0/1309088925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a10099b30 msgr2=0x7f1a100b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 --2- 172.21.15.1:0/1309088925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a10099b30 0x7f1a100b2aa0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1a100ca960 tx=0x7f1a0402a270 comp rx=0 tx=0).stop 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 -- 172.21.15.1:0/1309088925 shutdown_connections 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 --2- 172.21.15.1:0/1309088925 >> v2:172.21.15.6:3300/0 conn(0x7f1a1015e4b0 0x7f1a1015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 --2- 172.21.15.1:0/1309088925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a10099b30 0x7f1a100b2aa0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 -- 172.21.15.1:0/1309088925 >> 172.21.15.1:0/1309088925 conn(0x7f1a100b1fb0 msgr2=0x7f1a100b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.034+0000 7f1a1562a640 1 -- 172.21.15.1:0/1309088925 shutdown_connections 2024-05-09T06:39:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.035+0000 7f1a1562a640 1 -- 172.21.15.1:0/1309088925 wait complete. 2024-05-09T06:39:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.036+0000 7f1a1562a640 1 Processor -- start 2024-05-09T06:39:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.036+0000 7f1a1562a640 1 -- start start 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a10099b30 0x7f1a100cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 0x7f1a1014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1015f860 con 0x7f1a1015e4b0 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1015eb20 con 0x7f1a100c6860 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1015f9e0 con 0x7f1a10099b30 2024-05-09T06:39:25.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57394/0 (socket says 172.21.15.1:57394) 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 -- 172.21.15.1:0/3804147061 learned_addr learned my addr 172.21.15.1:0/3804147061 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0e7fc640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 0x7f1a1014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0effd640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a10099b30 0x7f1a100cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 -- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a10099b30 msgr2=0x7f1a100cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a10099b30 0x7f1a100cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 -- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 msgr2=0x7f1a1014c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 0x7f1a1014c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 -- 172.21.15.1:0/3804147061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a100c85d0 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0e7fc640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 0x7f1a1014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a0f7fe640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f1a08007df0 tx=0x7f1a0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0801a070 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a08003990 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a100c88b0 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08003d80 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.037+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a100c8e70 con 0x7f1a1015e4b0 2024-05-09T06:39:25.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.038+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a100752b0 con 0x7f1a1015e4b0 2024-05-09T06:39:25.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.039+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a0801d070 con 0x7f1a1015e4b0 2024-05-09T06:39:25.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.039+0000 7f19f77fe640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 0x7f19fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.040+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f1a0808b290 con 0x7f1a1015e4b0 2024-05-09T06:39:25.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.040+0000 7f1a0effd640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 0x7f19fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.040+0000 7f1a0effd640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 0x7f19fc06e320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001640 tx=0x7f1a00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.044+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a08010070 con 0x7f1a1015e4b0 2024-05-09T06:39:25.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.267+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a1006ce30 con 0x7f1a1015e4b0 2024-05-09T06:39:25.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.267+0000 7f19f77fe640 1 -- 172.21.15.1:0/3804147061 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5178 (secure 0 0 0) 0x7f1a08059a90 con 0x7f1a1015e4b0 2024-05-09T06:39:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:25.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:17.081146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10161,"qdb_cluster":[10161]},"id":8}]} 2024-05-09T06:39:25.247 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 112 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 msgr2=0x7f19fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 0x7f19fc06e320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001640 tx=0x7f1a00009040 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 msgr2=0x7f1a100b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f1a08007df0 tx=0x7f1a0800d4a0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 shutdown_connections 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f19fc06be60 0x7f19fc06e320 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a1015e4b0 0x7f1a100b3a80 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a100c6860 0x7f1a1014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 --2- 172.21.15.1:0/3804147061 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a10099b30 0x7f1a100cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 >> 172.21.15.1:0/3804147061 conn(0x7f1a100b1fb0 msgr2=0x7f1a100b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 shutdown_connections 2024-05-09T06:39:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.269+0000 7f1a1562a640 1 -- 172.21.15.1:0/3804147061 wait complete. 2024-05-09T06:39:25.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:25.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.386+0000 7fb4bbb70640 1 Processor -- start 2024-05-09T06:39:25.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.386+0000 7fb4bbb70640 1 -- start start 2024-05-09T06:39:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4bbb70640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 0x7fb4b415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4bbb70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 0x7fb4b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4bbb70640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b4070f40 con 0x7fb4b415db10 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4bbb70640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b40cf170 con 0x7fb4b40b2a60 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4bbb70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b40cf450 con 0x7fb4b415e500 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4b910b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 0x7fb4b415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4ba10d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 0x7fb4b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4b990c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb4b415db10 0x7fb4b40715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:47134/0 (socket says 172.21.15.1:47134) 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4b990c640 1 -- 172.21.15.1:0/2898484211 learned_addr learned my addr 172.21.15.1:0/2898484211 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4b890a640 1 -- 172.21.15.1:0/2898484211 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127255780 0 0) 0x7fb4b4070f40 con 0x7fb4b415db10 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.387+0000 7fb4b890a640 1 -- 172.21.15.1:0/2898484211 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a0003520 con 0x7fb4b415db10 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4ba10d640 1 -- 172.21.15.1:0/2898484211 >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 msgr2=0x7fb4b415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4ba10d640 1 --2- 172.21.15.1:0/2898484211 >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 0x7fb4b415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4ba10d640 1 -- 172.21.15.1:0/2898484211 >> v1:172.21.15.6:6789/0 conn(0x7fb4b415db10 legacy=0x7fb4b40715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4ba10d640 1 -- 172.21.15.1:0/2898484211 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b4070c10 con 0x7fb4b415e500 2024-05-09T06:39:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4b910b640 1 --2- 172.21.15.1:0/2898484211 >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 0x7fb4b415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4ba10d640 1 --2- 172.21.15.1:0/2898484211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 0x7fb4b406e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009d90 tx=0x7fb4ac02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ed03a23f62b0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.388+0000 7fb4b890a640 1 -- 172.21.15.1:0/2898484211 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac031600 con 0x7fb4b415e500 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4b890a640 1 -- 172.21.15.1:0/2898484211 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ac0317a0 con 0x7fb4b415e500 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4b890a640 1 -- 172.21.15.1:0/2898484211 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac031b70 con 0x7fb4b415e500 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 -- 172.21.15.1:0/2898484211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 msgr2=0x7fb4b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/2898484211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 0x7fb4b406e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009d90 tx=0x7fb4ac02a5d0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 -- 172.21.15.1:0/2898484211 shutdown_connections 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/2898484211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415e500 0x7fb4b406e340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/2898484211 >> v2:172.21.15.6:3300/0 conn(0x7fb4b40b2a60 0x7fb4b415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 -- 172.21.15.1:0/2898484211 >> 172.21.15.1:0/2898484211 conn(0x7fb4b40cc4b0 msgr2=0x7fb4b40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 -- 172.21.15.1:0/2898484211 shutdown_connections 2024-05-09T06:39:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.389+0000 7fb4bbb70640 1 -- 172.21.15.1:0/2898484211 wait complete. 2024-05-09T06:39:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.390+0000 7fb4bbb70640 1 Processor -- start 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.390+0000 7fb4bbb70640 1 -- start start 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415db10 0x7fb4b4156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4b415e500 0x7fb4b40a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b406ceb0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b406cd30 con 0x7fb4b415e500 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4bbb70640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b406d030 con 0x7fb4b415db10 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4b990c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4b990c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57408/0 (socket says 172.21.15.1:57408) 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4b990c640 1 -- 172.21.15.1:0/1805248672 learned_addr learned my addr 172.21.15.1:0/1805248672 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4ba10d640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4b415e500 0x7fb4b40a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.391+0000 7fb4b910b640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415db10 0x7fb4b4156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 -- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415db10 msgr2=0x7fb4b4156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415db10 0x7fb4b4156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 -- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4b415e500 msgr2=0x7fb4b40a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4b415e500 0x7fb4b40a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 -- 172.21.15.1:0/1805248672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b40a9cf0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4b990c640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c460 tx=0x7fb4b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0017070 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0005ce0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b41567e0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0007290 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.392+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b4156da0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.393+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b40752b0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.394+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4b001a070 con 0x7fb4b40b2a60 2024-05-09T06:39:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.394+0000 7fb4aa7fc640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 0x7fb49806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.394+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fb4b008a350 con 0x7fb4b40b2a60 2024-05-09T06:39:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.394+0000 7fb4b910b640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 0x7fb49806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.395+0000 7fb4b910b640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 0x7fb49806e320 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400a700 tx=0x7fb4a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.399+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4b0058b50 con 0x7fb4b40b2a60 2024-05-09T06:39:25.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.583+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4b415aed0 con 0x7fb4b40b2a60 2024-05-09T06:39:25.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.584+0000 7fb4aa7fc640 1 -- 172.21.15.1:0/1805248672 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17075 (secure 0 0 0) 0x7fb4b0058450 con 0x7fb4b40b2a60 2024-05-09T06:39:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:25.563 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":123,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:39:12.063134+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 msgr2=0x7fb49806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 0x7fb49806e320 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400a700 tx=0x7fb4a4008040 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 msgr2=0x7fb4b40ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c460 tx=0x7fb4b000c920 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 shutdown_connections 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb49806be60 0x7fb49806e320 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4b415e500 0x7fb4b40a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4b415db10 0x7fb4b4156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 --2- 172.21.15.1:0/1805248672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b40b2a60 0x7fb4b40ae9e0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 >> 172.21.15.1:0/1805248672 conn(0x7fb4b40cc4b0 msgr2=0x7fb4b406d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 shutdown_connections 2024-05-09T06:39:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.586+0000 7fb4bbb70640 1 -- 172.21.15.1:0/1805248672 wait complete. 2024-05-09T06:39:25.576 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:39:25.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.701+0000 7f1bccabc640 1 Processor -- start 2024-05-09T06:39:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.701+0000 7f1bccabc640 1 -- start start 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bccabc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bc815c680 0x7f1bc80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bccabc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc81601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bccabc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc815cc40 con 0x7f1bc81565e0 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bccabc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc815fd30 con 0x7f1bc815c680 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bccabc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc815cf20 con 0x7f1bc80b0170 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc5d9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bc815c680 0x7f1bc80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc6d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc81601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc659c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1bc81565e0 0x7f1bc8074530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:47148/0 (socket says 172.21.15.1:47148) 2024-05-09T06:39:25.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc659c640 1 -- 172.21.15.1:0/1726706084 learned_addr learned my addr 172.21.15.1:0/1726706084 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 msgr2=0x7f1bc81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc5d9b640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc81601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/1726706084 >> v1:172.21.15.6:6789/0 conn(0x7f1bc81565e0 legacy=0x7f1bc8074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/1726706084 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc815d200 con 0x7f1bc815c680 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.702+0000 7f1bc6d9d640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc81601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.703+0000 7f1bc5d9b640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80afc30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363e6ea2a0a9dd69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.703+0000 7f1bc559a640 1 -- 172.21.15.1:0/1726706084 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037070 con 0x7f1bc815c680 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.703+0000 7f1bc559a640 1 -- 172.21.15.1:0/1726706084 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc032440 con 0x7f1bc815c680 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.703+0000 7f1bc559a640 1 -- 172.21.15.1:0/1726706084 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc032810 con 0x7f1bc815c680 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 -- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 msgr2=0x7f1bc80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80afc30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a180 comp rx=0 tx=0).stop 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 -- 172.21.15.1:0/1726706084 shutdown_connections 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc81601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 --2- 172.21.15.1:0/1726706084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80afc30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 -- 172.21.15.1:0/1726706084 >> 172.21.15.1:0/1726706084 conn(0x7f1bc80b1fb0 msgr2=0x7f1bc80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 -- 172.21.15.1:0/1726706084 shutdown_connections 2024-05-09T06:39:25.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.704+0000 7f1bccabc640 1 -- 172.21.15.1:0/1726706084 wait complete. 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.705+0000 7f1bccabc640 1 Processor -- start 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.705+0000 7f1bccabc640 1 -- start start 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc80b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc8160700 con 0x7f1bc81565e0 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81569c0 con 0x7f1bc815c680 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bccabc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc8160880 con 0x7f1bc80b0170 2024-05-09T06:39:25.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bc5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bc5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57412/0 (socket says 172.21.15.1:57412) 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/3013589958 learned_addr learned my addr 172.21.15.1:0/3013589958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bc6d9d640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.706+0000 7f1bc659c640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc80b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 msgr2=0x7f1bc80b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc80b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 msgr2=0x7f1bc80ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 -- 172.21.15.1:0/3013589958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80b49b0 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bc5d9b640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc035040 tx=0x7f1bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037040 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc030440 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc80b4c90 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc030850 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.707+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc80b5250 con 0x7f1bc81565e0 2024-05-09T06:39:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.708+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc80cee90 con 0x7f1bc81565e0 2024-05-09T06:39:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.709+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bbc044020 con 0x7f1bc81565e0 2024-05-09T06:39:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.709+0000 7f1bb6ffd640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 0x7f1ba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.709+0000 7f1bc659c640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 0x7f1ba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.709+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7f1bbc0ab240 con 0x7f1bc81565e0 2024-05-09T06:39:25.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.710+0000 7f1bc659c640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 0x7f1ba806e2f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8007fd0 tx=0x7f1bb8007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:25.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.713+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1bc80cee90 con 0x7f1bc81565e0 2024-05-09T06:39:25.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.937+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bc80cda50 con 0x7f1bc81565e0 2024-05-09T06:39:25.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.938+0000 7f1bb6ffd640 1 -- 172.21.15.1:0/3013589958 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5178 (secure 0 0 0) 0x7f1bbc0799b0 con 0x7f1bc81565e0 2024-05-09T06:39:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:25.917 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:17.081146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10161,"qdb_cluster":[10161]},"id":8}]} 2024-05-09T06:39:25.918 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 112 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.939+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 msgr2=0x7f1ba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.939+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 0x7f1ba806e2f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8007fd0 tx=0x7f1bb8007e80 comp rx=0 tx=0).stop 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.939+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 msgr2=0x7f1bc814a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.939+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc035040 tx=0x7f1bbc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 shutdown_connections 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1ba806be30 0x7f1ba806e2f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bc815c680 0x7f1bc80ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bc81565e0 0x7f1bc814a670 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 --2- 172.21.15.1:0/3013589958 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1bc80b0170 0x7f1bc80b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:25.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 >> 172.21.15.1:0/3013589958 conn(0x7f1bc80b1fb0 msgr2=0x7f1bc80cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:25.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 shutdown_connections 2024-05-09T06:39:25.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:25.940+0000 7f1bccabc640 1 -- 172.21.15.1:0/3013589958 wait complete. 2024-05-09T06:39:25.930 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.055+0000 7fa1c57b0640 1 Processor -- start 2024-05-09T06:39:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.055+0000 7fa1c57b0640 1 -- start start 2024-05-09T06:39:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 0x7fa1c015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1c006e4f0 0x7fa1c006e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 0x7fa1c00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c00b3b80 con 0x7fa1c006e4f0 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c00aef90 con 0x7fa1c015e050 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1c57b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c00b3530 con 0x7fa1c00ae2a0 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1be7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1c006e4f0 0x7fa1c006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1be7fc640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa1c006e4f0 0x7fa1c006e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57426/0 (socket says 172.21.15.1:57426) 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1be7fc640 1 -- 172.21.15.1:0/3811793406 learned_addr learned my addr 172.21.15.1:0/3811793406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1beffd640 1 --2- 172.21.15.1:0/3811793406 >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 0x7fa1c015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.056+0000 7fa1bf7fe640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 0x7fa1c00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 -- 172.21.15.1:0/3811793406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 msgr2=0x7fa1c00ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 0x7fa1c00ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 -- 172.21.15.1:0/3811793406 >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 msgr2=0x7fa1c015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 --2- 172.21.15.1:0/3811793406 >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 0x7fa1c015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 -- 172.21.15.1:0/3811793406 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c00b3810 con 0x7fa1c006e4f0 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1bf7fe640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 0x7fa1c00ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1beffd640 1 --2- 172.21.15.1:0/3811793406 >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 0x7fa1c015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1be7fc640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c006e4f0 0x7fa1c006e980 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0067ef0 tx=0x7fa1a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c41c2ba31a8cc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1bdffb640 1 -- 172.21.15.1:0/3811793406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8037070 con 0x7fa1c006e4f0 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1bdffb640 1 -- 172.21.15.1:0/3811793406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a8032440 con 0x7fa1c006e4f0 2024-05-09T06:39:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.057+0000 7fa1bdffb640 1 -- 172.21.15.1:0/3811793406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8032810 con 0x7fa1c006e4f0 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 -- 172.21.15.1:0/3811793406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c006e4f0 msgr2=0x7fa1c006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c006e4f0 0x7fa1c006e980 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0067ef0 tx=0x7fa1a802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 -- 172.21.15.1:0/3811793406 shutdown_connections 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c00ae2a0 0x7fa1c00ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/3811793406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c006e4f0 0x7fa1c006e980 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/3811793406 >> v2:172.21.15.6:3300/0 conn(0x7fa1c015e050 0x7fa1c015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 -- 172.21.15.1:0/3811793406 >> 172.21.15.1:0/3811793406 conn(0x7fa1c006d270 msgr2=0x7fa1c006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 -- 172.21.15.1:0/3811793406 shutdown_connections 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.058+0000 7fa1c57b0640 1 -- 172.21.15.1:0/3811793406 wait complete. 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.059+0000 7fa1c57b0640 1 Processor -- start 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.059+0000 7fa1c57b0640 1 -- start start 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c006e4f0 0x7fa1c0149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c00ae2a0 0x7fa1c0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c015eaf0 con 0x7fa1c015e050 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c015e970 con 0x7fa1c00ae2a0 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1c57b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c00b4010 con 0x7fa1c006e4f0 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1bf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57442/0 (socket says 172.21.15.1:57442) 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1bf7fe640 1 -- 172.21.15.1:0/2096518163 learned_addr learned my addr 172.21.15.1:0/2096518163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1be7fc640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c00ae2a0 0x7fa1c0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.060+0000 7fa1beffd640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c006e4f0 0x7fa1c0149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 -- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c006e4f0 msgr2=0x7fa1c0149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c006e4f0 0x7fa1c0149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 -- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c00ae2a0 msgr2=0x7fa1c0154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c00ae2a0 0x7fa1c0154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 -- 172.21.15.1:0/2096518163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c0153600 con 0x7fa1c015e050 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1bf7fe640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800cef0 tx=0x7fa1b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b801a070 con 0x7fa1c015e050 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b800d860 con 0x7fa1c015e050 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c01538e0 con 0x7fa1c015e050 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b800dc50 con 0x7fa1c015e050 2024-05-09T06:39:26.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.061+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c0153ea0 con 0x7fa1c015e050 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.062+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c00752b0 con 0x7fa1c015e050 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.062+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1b801d070 con 0x7fa1c015e050 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.063+0000 7fa1b77fe640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 0x7fa1a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.063+0000 7fa1beffd640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 0x7fa1a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.063+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fa1b808b8d0 con 0x7fa1c015e050 2024-05-09T06:39:26.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.064+0000 7fa1beffd640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 0x7fa1a006e2f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000aa40 tx=0x7fa1b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.068+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1b8059ab0 con 0x7fa1c015e050 2024-05-09T06:39:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.291+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1c0072260 con 0x7fa1c015e050 2024-05-09T06:39:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.291+0000 7fa1b77fe640 1 -- 172.21.15.1:0/2096518163 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5178 (secure 0 0 0) 0x7fa1b805ddb0 con 0x7fa1c015e050 2024-05-09T06:39:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:26.271 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:38:29.105728+0000","modified":"2024-05-09T06:39:17.081146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10161":{"gid":10161,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.1:6837/520088510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":520088510},{"type":"v1","addr":"172.21.15.1:6837","nonce":520088510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10161,"qdb_cluster":[10161]},"id":8}]} 2024-05-09T06:39:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 112 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 msgr2=0x7fa1a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 0x7fa1a006e2f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000aa40 tx=0x7fa1b0008040 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 msgr2=0x7fa1c01551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800cef0 tx=0x7fa1b800a710 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 shutdown_connections 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa1a006be30 0x7fa1a006e2f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c015e050 0x7fa1c01551f0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c00ae2a0 0x7fa1c0154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 --2- 172.21.15.1:0/2096518163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c006e4f0 0x7fa1c0149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 >> 172.21.15.1:0/2096518163 conn(0x7fa1c006d270 msgr2=0x7fa1c00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 shutdown_connections 2024-05-09T06:39:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.293+0000 7fa1c57b0640 1 -- 172.21.15.1:0/2096518163 wait complete. 2024-05-09T06:39:26.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.408+0000 7fdd4408c640 1 Processor -- start 2024-05-09T06:39:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.408+0000 7fdd4408c640 1 -- start start 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd3c0cbb60 0x7fdd3c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 0x7fdd3c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0cbfe0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d04a0 con 0x7fdd3c15fd70 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd4408c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c0722c0 con 0x7fdd3c0b0080 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd41627640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd3c0cbb60 0x7fdd3c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd41627640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdd3c0cbb60 0x7fdd3c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57450/0 (socket says 172.21.15.1:57450) 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd41627640 1 -- 172.21.15.1:0/4141009770 learned_addr learned my addr 172.21.15.1:0/4141009770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd41627640 1 -- 172.21.15.1:0/4141009770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 msgr2=0x7fdd3c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd42629640 1 --2- 172.21.15.1:0/4141009770 >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 0x7fdd3c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.409+0000 7fdd41e28640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41627640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41627640 1 -- 172.21.15.1:0/4141009770 >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 msgr2=0x7fdd3c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41627640 1 --2- 172.21.15.1:0/4141009770 >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 0x7fdd3c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41627640 1 -- 172.21.15.1:0/4141009770 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0725a0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd42629640 1 --2- 172.21.15.1:0/4141009770 >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 0x7fdd3c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41e28640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd41627640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c15f830 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0b2690 tx=0x7fdd2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc4f97edca2400 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd40e26640 1 -- 172.21.15.1:0/4141009770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c037070 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd40e26640 1 -- 172.21.15.1:0/4141009770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c032440 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.410+0000 7fdd40e26640 1 -- 172.21.15.1:0/4141009770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c032810 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 -- 172.21.15.1:0/4141009770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 msgr2=0x7fdd3c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c15f830 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0b2690 tx=0x7fdd2c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 -- 172.21.15.1:0/4141009770 shutdown_connections 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 --2- 172.21.15.1:0/4141009770 >> v2:172.21.15.6:3300/0 conn(0x7fdd3c15fd70 0x7fdd3c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c15f830 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 --2- 172.21.15.1:0/4141009770 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 -- 172.21.15.1:0/4141009770 >> 172.21.15.1:0/4141009770 conn(0x7fdd3c0d0b50 msgr2=0x7fdd3c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 -- 172.21.15.1:0/4141009770 shutdown_connections 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.411+0000 7fdd4408c640 1 -- 172.21.15.1:0/4141009770 wait complete. 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.412+0000 7fdd4408c640 1 Processor -- start 2024-05-09T06:39:26.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.412+0000 7fdd4408c640 1 -- start start 2024-05-09T06:39:26.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c1f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 0x7fdd3c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c15e7c0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c072b60 con 0x7fdd3c15fd70 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd4408c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c15f220 con 0x7fdd3c0b0080 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57456/0 (socket says 172.21.15.1:57456) 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 -- 172.21.15.1:0/299682647 learned_addr learned my addr 172.21.15.1:0/299682647 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 -- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 msgr2=0x7fdd3c1f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c1f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 -- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 msgr2=0x7fdd3c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd42629640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 0x7fdd3c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.413+0000 7fdd41627640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 0x7fdd3c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd41627640 1 -- 172.21.15.1:0/299682647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c1f57f0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd42629640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 0x7fdd3c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd41627640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02a220 tx=0x7fdd2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c037040 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c042070 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c030bb0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c1f5ad0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.414+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c1f6090 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.415+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c0752b0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.415+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd2c047460 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.416+0000 7fdd327fc640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 0x7fdd2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.416+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fdd2c0abcd0 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.416+0000 7fdd41e28640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 0x7fdd2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.416+0000 7fdd41e28640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 0x7fdd2006e2f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0cb050 tx=0x7fdd38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.420+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd2c045360 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.606+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd3c1f6430 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.607+0000 7fdd327fc640 1 -- 172.21.15.1:0/299682647 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17075 (secure 0 0 0) 0x7fdd2c07a440 con 0x7fdd3c0cbb60 2024-05-09T06:39:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:26.587 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":123,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:39:12.063134+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:38:27.404781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-09T06:38:28.082194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 msgr2=0x7fdd2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 0x7fdd2006e2f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0cb050 tx=0x7fdd38008040 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 msgr2=0x7fdd3c1f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02a220 tx=0x7fdd2c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 shutdown_connections 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fdd2006be30 0x7fdd2006e2f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd3c15fd70 0x7fdd3c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd3c0cbb60 0x7fdd3c1f4e80 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 --2- 172.21.15.1:0/299682647 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdd3c0b0080 0x7fdd3c1f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 >> 172.21.15.1:0/299682647 conn(0x7fdd3c0d0b50 msgr2=0x7fdd3c15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 shutdown_connections 2024-05-09T06:39:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.609+0000 7fdd4408c640 1 -- 172.21.15.1:0/299682647 wait complete. 2024-05-09T06:39:26.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:39:26.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.725+0000 7fa62d864640 1 Processor -- start 2024-05-09T06:39:26.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.725+0000 7fa62d864640 1 -- start start 2024-05-09T06:39:26.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa62d864640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa6281617a0 0x7fa6280b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa62d864640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa6280b2dd0 0x7fa6280d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa62d864640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6280d2b10 con 0x7fa6280d2710 2024-05-09T06:39:26.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa62d864640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa6280d1480 con 0x7fa6281617a0 2024-05-09T06:39:26.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa62d864640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa6280d0870 con 0x7fa6280b2dd0 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa626ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa6281617a0 0x7fa6280b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa626ffd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa6281617a0 0x7fa6280b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57460/0 (socket says 172.21.15.1:57460) 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.726+0000 7fa626ffd640 1 -- 172.21.15.1:0/2144915289 learned_addr learned my addr 172.21.15.1:0/2144915289 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa6267fc640 1 --2- 172.21.15.1:0/2144915289 >> v2:172.21.15.6:3300/0 conn(0x7fa6280b2dd0 0x7fa6280d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa626ffd640 1 -- 172.21.15.1:0/2144915289 >> v2:172.21.15.6:3300/0 conn(0x7fa6280b2dd0 msgr2=0x7fa6280d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa626ffd640 1 --2- 172.21.15.1:0/2144915289 >> v2:172.21.15.6:3300/0 conn(0x7fa6280b2dd0 0x7fa6280d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa626ffd640 1 -- 172.21.15.1:0/2144915289 >> v1:172.21.15.6:6789/0 conn(0x7fa6280d2710 legacy=0x7fa628161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa626ffd640 1 -- 172.21.15.1:0/2144915289 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6280d0b50 con 0x7fa6281617a0 2024-05-09T06:39:26.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa626ffd640 1 --2- 172.21.15.1:0/2144915289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280b5010 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fa6280b6a90 tx=0x7fa62002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab114836caf57fea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.727+0000 7fa625ffb640 1 -- 172.21.15.1:0/2144915289 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620037070 con 0x7fa6281617a0 2024-05-09T06:39:26.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa625ffb640 1 -- 172.21.15.1:0/2144915289 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa620032440 con 0x7fa6281617a0 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa625ffb640 1 -- 172.21.15.1:0/2144915289 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620032810 con 0x7fa6281617a0 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 -- 172.21.15.1:0/2144915289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 msgr2=0x7fa6280b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 --2- 172.21.15.1:0/2144915289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280b5010 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fa6280b6a90 tx=0x7fa62002a230 comp rx=0 tx=0).stop 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 -- 172.21.15.1:0/2144915289 shutdown_connections 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 --2- 172.21.15.1:0/2144915289 >> v2:172.21.15.6:3300/0 conn(0x7fa6280b2dd0 0x7fa6280d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 --2- 172.21.15.1:0/2144915289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280b5010 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 -- 172.21.15.1:0/2144915289 >> 172.21.15.1:0/2144915289 conn(0x7fa62809d180 msgr2=0x7fa6280d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 -- 172.21.15.1:0/2144915289 shutdown_connections 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.728+0000 7fa62d864640 1 -- 172.21.15.1:0/2144915289 wait complete. 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.729+0000 7fa62d864640 1 Processor -- start 2024-05-09T06:39:26.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 -- start start 2024-05-09T06:39:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6280b2dd0 0x7fa6280ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 0x7fa6280caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa628161b80 con 0x7fa6281617a0 2024-05-09T06:39:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280d1110 con 0x7fa6280d2710 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa62d864640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa628161d00 con 0x7fa6280b2dd0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57472/0 (socket says 172.21.15.1:57472) 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.730+0000 7fa6277fe640 1 -- 172.21.15.1:0/1451660691 learned_addr learned my addr 172.21.15.1:0/1451660691 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 -- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6280b2dd0 msgr2=0x7fa6280ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6280b2dd0 0x7fa6280ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6267fc640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 0x7fa6280caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 -- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 msgr2=0x7fa6280caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 0x7fa6280caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 -- 172.21.15.1:0/1451660691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6280b6ab0 con 0x7fa6281617a0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6267fc640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 0x7fa6280caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa6277fe640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c780 tx=0x7fa61800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618004b60 con 0x7fa6281617a0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6280b6d90 con 0x7fa6281617a0 2024-05-09T06:39:26.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa618004d00 con 0x7fa6281617a0 2024-05-09T06:39:26.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800aa90 con 0x7fa6281617a0 2024-05-09T06:39:26.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.731+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6280b7630 con 0x7fa6281617a0 2024-05-09T06:39:26.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.732+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6280b7830 con 0x7fa6281617a0 2024-05-09T06:39:26.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.733+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa61800ac30 con 0x7fa6281617a0 2024-05-09T06:39:26.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.733+0000 7fa61ffff640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 0x7fa60806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.733+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6273+0+0 (secure 0 0 0) 0x7fa61808b650 con 0x7fa6281617a0 2024-05-09T06:39:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.733+0000 7fa626ffd640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 0x7fa60806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.734+0000 7fa626ffd640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 0x7fa60806e2f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa620034000 tx=0x7fa62002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.738+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6280b7830 con 0x7fa6281617a0 2024-05-09T06:39:26.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.962+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6280752b0 con 0x7fa6281617a0 2024-05-09T06:39:26.941 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-05-09T06:39:26.962+0000 7f5fe81b8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:39:26.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.964+0000 7fa61ffff640 1 -- 172.21.15.1:0/1451660691 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7fa618059dd0 con 0x7fa6281617a0 2024-05-09T06:39:26.943 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:39:26.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 msgr2=0x7fa60806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 0x7fa60806e2f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa620034000 tx=0x7fa62002c000 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 msgr2=0x7fa6280ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c780 tx=0x7fa61800cc40 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 shutdown_connections 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa60806be30 0x7fa60806e2f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6281617a0 0x7fa6280ccb70 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6280d2710 0x7fa6280caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 --2- 172.21.15.1:0/1451660691 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa6280b2dd0 0x7fa6280ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 >> 172.21.15.1:0/1451660691 conn(0x7fa62809d180 msgr2=0x7fa6280b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 shutdown_connections 2024-05-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:26.967+0000 7fa62d864640 1 -- 172.21.15.1:0/1451660691 wait complete. 2024-05-09T06:39:26.948 INFO:tasks.ceph.mds.a.smithi001.stderr: -1439> 2024-05-09T06:39:18.071+0000 7f01e0ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:39:26.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:39:26.976 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T06:39:27.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.086+0000 7fde51431640 1 Processor -- start 2024-05-09T06:39:27.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.086+0000 7fde51431640 1 -- start start 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde51431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c06b8f0 0x7fde4c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde51431640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fde4c0d1640 0x7fde4c06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde51431640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c072030 con 0x7fde4c159b90 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde51431640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fde4c072cb0 con 0x7fde4c0d1640 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde51431640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c06c150 con 0x7fde4c06b8f0 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4affd640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fde4c159b90 0x7fde4c06b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49944/0 (socket says 172.21.15.1:49944) 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4affd640 1 -- 172.21.15.1:0/529343036 learned_addr learned my addr 172.21.15.1:0/529343036 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4b7fe640 1 --2- 172.21.15.1:0/529343036 >> v2:172.21.15.1:3300/0 conn(0x7fde4c0d1640 0x7fde4c06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4a7fc640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c06b8f0 0x7fde4c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4b7fe640 1 -- 172.21.15.1:0/529343036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c06b8f0 msgr2=0x7fde4c15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4b7fe640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c06b8f0 0x7fde4c15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4b7fe640 1 -- 172.21.15.1:0/529343036 >> v1:172.21.15.1:6789/0 conn(0x7fde4c159b90 legacy=0x7fde4c06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde49ffb640 1 -- 172.21.15.1:0/529343036 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416685105 0 0) 0x7fde4c072030 con 0x7fde4c159b90 2024-05-09T06:39:27.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.087+0000 7fde4b7fe640 1 -- 172.21.15.1:0/529343036 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c072310 con 0x7fde4c0d1640 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.088+0000 7fde4b7fe640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c0d1640 0x7fde4c06f480 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013318eb7ec0715 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.088+0000 7fde49ffb640 1 -- 172.21.15.1:0/529343036 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c037070 con 0x7fde4c0d1640 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.088+0000 7fde49ffb640 1 -- 172.21.15.1:0/529343036 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c02ae50 con 0x7fde4c0d1640 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.088+0000 7fde49ffb640 1 -- 172.21.15.1:0/529343036 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c032660 con 0x7fde4c0d1640 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 -- 172.21.15.1:0/529343036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c0d1640 msgr2=0x7fde4c06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c0d1640 0x7fde4c06f480 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02a100 comp rx=0 tx=0).stop 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 -- 172.21.15.1:0/529343036 shutdown_connections 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c06b8f0 0x7fde4c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 --2- 172.21.15.1:0/529343036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c0d1640 0x7fde4c06f480 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 -- 172.21.15.1:0/529343036 >> 172.21.15.1:0/529343036 conn(0x7fde4c0d0b50 msgr2=0x7fde4c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:27.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 -- 172.21.15.1:0/529343036 shutdown_connections 2024-05-09T06:39:27.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.089+0000 7fde51431640 1 -- 172.21.15.1:0/529343036 wait complete. 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.090+0000 7fde51431640 1 Processor -- start 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.090+0000 7fde51431640 1 -- start start 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 0x7fde4c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 0x7fde4c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c15d580 con 0x7fde4c06b8f0 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0728d0 con 0x7fde4c0d1640 2024-05-09T06:39:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde51431640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c071ce0 con 0x7fde4c159b90 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57518/0 (socket says 172.21.15.1:57518) 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 -- 172.21.15.1:0/1529918583 learned_addr learned my addr 172.21.15.1:0/1529918583 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4b7fe640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 0x7fde4c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4a7fc640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 0x7fde4c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 -- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 msgr2=0x7fde4c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 0x7fde4c0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 -- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 msgr2=0x7fde4c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 0x7fde4c0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4affd640 1 -- 172.21.15.1:0/1529918583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c0cdc90 con 0x7fde4c06b8f0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4b7fe640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 0x7fde4c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.091+0000 7fde4a7fc640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 0x7fde4c0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde4affd640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fde4400c7f0 tx=0x7fde4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44018070 con 0x7fde4c06b8f0 2024-05-09T06:39:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4400a640 con 0x7fde4c06b8f0 2024-05-09T06:39:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c0cdf70 con 0x7fde4c06b8f0 2024-05-09T06:39:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4400aa30 con 0x7fde4c06b8f0 2024-05-09T06:39:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.092+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c0c84b0 con 0x7fde4c06b8f0 2024-05-09T06:39:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.093+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c0752b0 con 0x7fde4c06b8f0 2024-05-09T06:39:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.093+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde4401b070 con 0x7fde4c06b8f0 2024-05-09T06:39:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.094+0000 7fde437fe640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 0x7fde2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.094+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6359+0+0 (secure 0 0 0) 0x7fde4408a7d0 con 0x7fde4c06b8f0 2024-05-09T06:39:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.094+0000 7fde4a7fc640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 0x7fde2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.094+0000 7fde4a7fc640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 0x7fde2c06e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1f5460 tx=0x7fde34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.098+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde44058e60 con 0x7fde4c06b8f0 2024-05-09T06:39:27.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.319+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fde4c072e30 con 0x7fde4c06b8f0 2024-05-09T06:39:27.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.971+0000 7fde437fe640 1 -- 172.21.15.1:0/1529918583 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7fde44058760 con 0x7fde4c06b8f0 2024-05-09T06:39:27.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 msgr2=0x7fde2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 0x7fde2c06e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1f5460 tx=0x7fde34008040 comp rx=0 tx=0).stop 2024-05-09T06:39:27.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 msgr2=0x7fde4c1569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:27.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fde4400c7f0 tx=0x7fde4400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 shutdown_connections 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fde2c06be30 0x7fde2c06e2f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fde4c159b90 0x7fde4c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde4c0d1640 0x7fde4c0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 --2- 172.21.15.1:0/1529918583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c06b8f0 0x7fde4c1569d0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 >> 172.21.15.1:0/1529918583 conn(0x7fde4c0d0b50 msgr2=0x7fde4c06b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 shutdown_connections 2024-05-09T06:39:27.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:27.974+0000 7fde51431640 1 -- 172.21.15.1:0/1529918583 wait complete. 2024-05-09T06:39:27.969 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:39:28.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.145+0000 7fe391a7d640 1 Processor -- start 2024-05-09T06:39:28.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.145+0000 7fe391a7d640 1 -- start start 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe391a7d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe38c0b3820 0x7fe38c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe391a7d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe38c0ced00 0x7fe38c070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe391a7d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe38c0b3cc0 con 0x7fe38c145880 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe391a7d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe38c0b3fa0 con 0x7fe38c0ced00 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe391a7d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe38c0b4280 con 0x7fe38c0b3820 2024-05-09T06:39:28.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe38bfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe38c0ced00 0x7fe38c070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe38b7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe38c145880 0x7fe38c15cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57548/0 (socket says 172.21.15.1:57548) 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.146+0000 7fe38b7fe640 1 -- 172.21.15.1:0/2567628798 learned_addr learned my addr 172.21.15.1:0/2567628798 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38affd640 1 --2- 172.21.15.1:0/2567628798 >> v2:172.21.15.6:3300/0 conn(0x7fe38c0b3820 0x7fe38c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38bfff640 1 -- 172.21.15.1:0/2567628798 >> v2:172.21.15.6:3300/0 conn(0x7fe38c0b3820 msgr2=0x7fe38c0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38bfff640 1 --2- 172.21.15.1:0/2567628798 >> v2:172.21.15.6:3300/0 conn(0x7fe38c0b3820 0x7fe38c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38bfff640 1 -- 172.21.15.1:0/2567628798 >> v1:172.21.15.1:6789/0 conn(0x7fe38c145880 legacy=0x7fe38c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38a7fc640 1 -- 172.21.15.1:0/2567628798 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467329671 0 0) 0x7fe38c0b3cc0 con 0x7fe38c145880 2024-05-09T06:39:28.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38bfff640 1 -- 172.21.15.1:0/2567628798 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c0b4560 con 0x7fe38c0ced00 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.147+0000 7fe38bfff640 1 --2- 172.21.15.1:0/2567628798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ced00 0x7fe38c070d30 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fe38c06c1f0 tx=0x7fe38402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265237ef266422b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe38a7fc640 1 -- 172.21.15.1:0/2567628798 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384007740 con 0x7fe38c0ced00 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe38a7fc640 1 -- 172.21.15.1:0/2567628798 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3840078e0 con 0x7fe38c0ced00 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe38a7fc640 1 -- 172.21.15.1:0/2567628798 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384007cb0 con 0x7fe38c0ced00 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 -- 172.21.15.1:0/2567628798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ced00 msgr2=0x7fe38c070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 --2- 172.21.15.1:0/2567628798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ced00 0x7fe38c070d30 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fe38c06c1f0 tx=0x7fe38402a680 comp rx=0 tx=0).stop 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 -- 172.21.15.1:0/2567628798 shutdown_connections 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 --2- 172.21.15.1:0/2567628798 >> v2:172.21.15.6:3300/0 conn(0x7fe38c0b3820 0x7fe38c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 --2- 172.21.15.1:0/2567628798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ced00 0x7fe38c070d30 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 -- 172.21.15.1:0/2567628798 >> 172.21.15.1:0/2567628798 conn(0x7fe38c06b3c0 msgr2=0x7fe38c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:28.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 -- 172.21.15.1:0/2567628798 shutdown_connections 2024-05-09T06:39:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.148+0000 7fe391a7d640 1 -- 172.21.15.1:0/2567628798 wait complete. 2024-05-09T06:39:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.149+0000 7fe391a7d640 1 Processor -- start 2024-05-09T06:39:28.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 -- start start 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0b3820 0x7fe38c0aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0ced00 0x7fe38c0af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c071270 con 0x7fe38c145880 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c0cfee0 con 0x7fe38c0ced00 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.150+0000 7fe391a7d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe38c0713f0 con 0x7fe38c0b3820 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59776/0 (socket says 172.21.15.1:59776) 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 -- 172.21.15.1:0/1361467408 learned_addr learned my addr 172.21.15.1:0/1361467408 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38affd640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0ced00 0x7fe38c0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38b7fe640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0b3820 0x7fe38c0aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 -- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0b3820 msgr2=0x7fe38c0aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0b3820 0x7fe38c0aa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 -- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0ced00 msgr2=0x7fe38c0af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0ced00 0x7fe38c0af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 -- 172.21.15.1:0/1361467408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c15f890 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe38bfff640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fe38402ac70 tx=0x7fe384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384040070 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe384031c90 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c15fb70 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384045680 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.151+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c160130 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.152+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38c13fa90 con 0x7fe38c145880 2024-05-09T06:39:28.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.153+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe384039040 con 0x7fe38c145880 2024-05-09T06:39:28.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.153+0000 7fe373fff640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 0x7fe37806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:28.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.153+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6359+0+0 (secure 0 0 0) 0x7fe3840aca50 con 0x7fe38c145880 2024-05-09T06:39:28.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.153+0000 7fe38b7fe640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 0x7fe37806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:28.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.154+0000 7fe38b7fe640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 0x7fe37806e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe37c007fd0 tx=0x7fe37c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:28.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.158+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe38407ab40 con 0x7fe38c145880 2024-05-09T06:39:28.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:28.341+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe38c075300 con 0x7fe38c145880 2024-05-09T06:39:29.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.307+0000 7fe373fff640 1 -- 172.21.15.1:0/1361467408 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) ==== 181+0+0 (secure 0 0 0) 0x7fe38407ee40 con 0x7fe38c145880 2024-05-09T06:39:29.288 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:39:29.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 msgr2=0x7fe37806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 0x7fe37806e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe37c007fd0 tx=0x7fe37c007de0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 msgr2=0x7fe38c1f49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fe38402ac70 tx=0x7fe384004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 shutdown_connections 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe37806be30 0x7fe37806e2f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c145880 0x7fe38c1f49e0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0ced00 0x7fe38c0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 --2- 172.21.15.1:0/1361467408 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0b3820 0x7fe38c0aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.311+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 >> 172.21.15.1:0/1361467408 conn(0x7fe38c06b3c0 msgr2=0x7fe38c0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.312+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 shutdown_connections 2024-05-09T06:39:29.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.312+0000 7fe391a7d640 1 -- 172.21.15.1:0/1361467408 wait complete. 2024-05-09T06:39:29.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:39:29.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.444+0000 7fe38089a640 1 Processor -- start 2024-05-09T06:39:29.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.444+0000 7fe38089a640 1 -- start start 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe38089a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe37c15ec10 0x7fe37c0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe38089a640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0ccf20 con 0x7fe37c072580 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe38089a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0ccc40 con 0x7fe37c157fa0 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe38089a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c15e130 con 0x7fe37c15ec10 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe37c15ec10 0x7fe37c0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe37c15ec10 0x7fe37c0b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59778/0 (socket says 172.21.15.1:59778) 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 -- 172.21.15.1:0/2427243888 learned_addr learned my addr 172.21.15.1:0/2427243888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe37ad9d640 1 --1- 172.21.15.1:0/2427243888 >> v1:172.21.15.1:6789/0 conn(0x7fe37c072580 0x7fe37c072a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57550/0 (socket says 172.21.15.1:57550) 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 -- 172.21.15.1:0/2427243888 >> v1:172.21.15.6:6789/0 conn(0x7fe37c157fa0 legacy=0x7fe37c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 -- 172.21.15.1:0/2427243888 >> v1:172.21.15.1:6789/0 conn(0x7fe37c072580 legacy=0x7fe37c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.445+0000 7fe373fff640 1 -- 172.21.15.1:0/2427243888 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c15e410 con 0x7fe37c15ec10 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.446+0000 7fe373fff640 1 --2- 172.21.15.1:0/2427243888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c15ec10 0x7fe37c0b4bd0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fe36400bb00 tx=0x7fe36402c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc342aa2d8e92e7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.446+0000 7fe379d9b640 1 -- 172.21.15.1:0/2427243888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36403a070 con 0x7fe37c15ec10 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.446+0000 7fe379d9b640 1 -- 172.21.15.1:0/2427243888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364032480 con 0x7fe37c15ec10 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.446+0000 7fe379d9b640 1 -- 172.21.15.1:0/2427243888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364032850 con 0x7fe37c15ec10 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 -- 172.21.15.1:0/2427243888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c15ec10 msgr2=0x7fe37c0b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 --2- 172.21.15.1:0/2427243888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c15ec10 0x7fe37c0b4bd0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fe36400bb00 tx=0x7fe36402c790 comp rx=0 tx=0).stop 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 -- 172.21.15.1:0/2427243888 shutdown_connections 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 --2- 172.21.15.1:0/2427243888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c15ec10 0x7fe37c0b4bd0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 -- 172.21.15.1:0/2427243888 >> 172.21.15.1:0/2427243888 conn(0x7fe37c06b3c0 msgr2=0x7fe37c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 -- 172.21.15.1:0/2427243888 shutdown_connections 2024-05-09T06:39:29.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.447+0000 7fe38089a640 1 -- 172.21.15.1:0/2427243888 wait complete. 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.448+0000 7fe38089a640 1 Processor -- start 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.448+0000 7fe38089a640 1 -- start start 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 0x7fe37c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe37c15ec10 0x7fe37c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0cfcb0 con 0x7fe37c072580 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c158380 con 0x7fe37c157fa0 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe38089a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c0cfe30 con 0x7fe37c15ec10 2024-05-09T06:39:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe37a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe37a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59792/0 (socket says 172.21.15.1:59792) 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe37a59c640 1 -- 172.21.15.1:0/1608310397 learned_addr learned my addr 172.21.15.1:0/1608310397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe373fff640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 0x7fe37c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe37ad9d640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe37c15ec10 0x7fe37c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.449+0000 7fe37a59c640 1 -- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe37c15ec10 msgr2=0x7fe37c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe37a59c640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe37c15ec10 0x7fe37c0b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe37a59c640 1 -- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 msgr2=0x7fe37c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe37a59c640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 0x7fe37c0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe37a59c640 1 -- 172.21.15.1:0/1608310397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c151180 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe373fff640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 0x7fe37c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe37a59c640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c480 tx=0x7fe36c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0052d0 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c005470 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c151460 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c012870 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.450+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c151a20 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.451+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c075320 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.452+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe36c012a10 con 0x7fe37c072580 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.452+0000 7fe372ffd640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 0x7fe35c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.452+0000 7fe373fff640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 0x7fe35c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:29.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.452+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5921+0+0 (secure 0 0 0) 0x7fe36c08ab70 con 0x7fe37c072580 2024-05-09T06:39:29.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.453+0000 7fe373fff640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 0x7fe35c06e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe364037000 tx=0x7fe364030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:29.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.457+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe36c0594d0 con 0x7fe37c072580 2024-05-09T06:39:29.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:29.645+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe37c0cee60 con 0x7fe37c072580 2024-05-09T06:39:30.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.309+0000 7fe372ffd640 1 -- 172.21.15.1:0/1608310397 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) ==== 169+0+0 (secure 0 0 0) 0x7fe36c058dd0 con 0x7fe37c072580 2024-05-09T06:39:30.290 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T06:39:30.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.312+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 msgr2=0x7fe35c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.312+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 0x7fe35c06e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe364037000 tx=0x7fe364030000 comp rx=0 tx=0).stop 2024-05-09T06:39:30.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.312+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 msgr2=0x7fe37c14cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.312+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c480 tx=0x7fe36c00c940 comp rx=0 tx=0).stop 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 shutdown_connections 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe35c06be30 0x7fe35c06e2f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe37c15ec10 0x7fe37c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe37c157fa0 0x7fe37c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 --2- 172.21.15.1:0/1608310397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe37c072580 0x7fe37c14cd10 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 >> 172.21.15.1:0/1608310397 conn(0x7fe37c06b3c0 msgr2=0x7fe37c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 shutdown_connections 2024-05-09T06:39:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.313+0000 7fe38089a640 1 -- 172.21.15.1:0/1608310397 wait complete. 2024-05-09T06:39:30.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:39:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.442+0000 7f1f50e83640 1 Processor -- start 2024-05-09T06:39:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.442+0000 7f1f50e83640 1 -- start start 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f50e83640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f4c147960 0x7f1f4c071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f50e83640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f50e83640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4c147d40 con 0x7f1f4c0720a0 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f50e83640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4c06d2f0 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f50e83640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4c072820 con 0x7f1f4c147960 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59794/0 (socket says 172.21.15.1:59794) 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f49d9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1f4c0720a0 0x7f1f4c06d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57558/0 (socket says 172.21.15.1:57558) 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/3952899410 learned_addr learned my addr 172.21.15.1:0/3952899410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4a59c640 1 --2- 172.21.15.1:0/3952899410 >> v2:172.21.15.6:3300/0 conn(0x7f1f4c147960 0x7f1f4c071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/3952899410 >> v2:172.21.15.6:3300/0 conn(0x7f1f4c147960 msgr2=0x7f1f4c071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 --2- 172.21.15.1:0/3952899410 >> v2:172.21.15.6:3300/0 conn(0x7f1f4c147960 0x7f1f4c071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/3952899410 >> v1:172.21.15.1:6789/0 conn(0x7f1f4c0720a0 legacy=0x7f1f4c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/3952899410 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c072b00 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.443+0000 7f1f4959a640 1 -- 172.21.15.1:0/3952899410 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936470664 0 0) 0x7f1f4c147d40 con 0x7f1f4c0720a0 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.444+0000 7f1f4ad9d640 1 --2- 172.21.15.1:0/3952899410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009870 tx=0x7f1f3c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7e0c8a90dd91c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.444+0000 7f1f4959a640 1 -- 172.21.15.1:0/3952899410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c037070 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.444+0000 7f1f4959a640 1 -- 172.21.15.1:0/3952899410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3c032440 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.444+0000 7f1f4959a640 1 -- 172.21.15.1:0/3952899410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c032810 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 -- 172.21.15.1:0/3952899410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c0ae8a0 msgr2=0x7f1f4c0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 --2- 172.21.15.1:0/3952899410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009870 tx=0x7f1f3c02a2a0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 -- 172.21.15.1:0/3952899410 shutdown_connections 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 --2- 172.21.15.1:0/3952899410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c0ae8a0 0x7f1f4c0aed30 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 --2- 172.21.15.1:0/3952899410 >> v2:172.21.15.6:3300/0 conn(0x7f1f4c147960 0x7f1f4c071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 -- 172.21.15.1:0/3952899410 >> 172.21.15.1:0/3952899410 conn(0x7f1f4c0b1ba0 msgr2=0x7f1f4c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 -- 172.21.15.1:0/3952899410 shutdown_connections 2024-05-09T06:39:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.445+0000 7f1f50e83640 1 -- 172.21.15.1:0/3952899410 wait complete. 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.446+0000 7f1f50e83640 1 Processor -- start 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.446+0000 7f1f50e83640 1 -- start start 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 0x7f1f4c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 0x7f1f4c141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c072530 con 0x7f1f4c147960 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0730c0 con 0x7f1f4c0720a0 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f50e83640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c06dee0 con 0x7f1f4c0ae8a0 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59804/0 (socket says 172.21.15.1:59804) 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/4241607313 learned_addr learned my addr 172.21.15.1:0/4241607313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f49d9b640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 0x7f1f4c141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4a59c640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 0x7f1f4c13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 msgr2=0x7f1f4c141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 0x7f1f4c141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 msgr2=0x7f1f4c13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 0x7f1f4c13f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.447+0000 7f1f4ad9d640 1 -- 172.21.15.1:0/4241607313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c1360d0 con 0x7f1f4c147960 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f49d9b640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 0x7f1f4c141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f4a59c640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 0x7f1f4c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f4ad9d640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009870 tx=0x7f1f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c041020 con 0x7f1f4c147960 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3c03d640 con 0x7f1f4c147960 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c13ff70 con 0x7f1f4c147960 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c037040 con 0x7f1f4c147960 2024-05-09T06:39:30.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.448+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c0cd900 con 0x7f1f4c147960 2024-05-09T06:39:30.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.449+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c074610 con 0x7f1f4c147960 2024-05-09T06:39:30.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.450+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f3c044020 con 0x7f1f4c147960 2024-05-09T06:39:30.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.450+0000 7f1f42ffd640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 0x7f1f2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.450+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f1f3c0ab4f0 con 0x7f1f4c147960 2024-05-09T06:39:30.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.450+0000 7f1f4a59c640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 0x7f1f2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.451+0000 7f1f4a59c640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 0x7f1f2c06e2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007440 tx=0x7f1f4400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.454+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f3c04b030 con 0x7f1f4c147960 2024-05-09T06:39:30.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.636+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1f4c0b2390 con 0x7f1f4c147960 2024-05-09T06:39:30.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.639+0000 7f1f42ffd640 1 -- 172.21.15.1:0/4241607313 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f1f4c1360d0 con 0x7f1f4c147960 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 msgr2=0x7f1f2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 0x7f1f2c06e2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007440 tx=0x7f1f4400b040 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 msgr2=0x7f1f4c1351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009870 tx=0x7f1f3c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 shutdown_connections 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f1f2c06be30 0x7f1f2c06e2f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f4c147960 0x7f1f4c1351f0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f4c0ae8a0 0x7f1f4c141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 --2- 172.21.15.1:0/4241607313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f4c0720a0 0x7f1f4c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 >> 172.21.15.1:0/4241607313 conn(0x7f1f4c0b1ba0 msgr2=0x7f1f4c0b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:30.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 shutdown_connections 2024-05-09T06:39:30.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.641+0000 7f1f50e83640 1 -- 172.21.15.1:0/4241607313 wait complete. 2024-05-09T06:39:30.633 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-09T06:39:30.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.757+0000 7efedb838640 1 Processor -- start 2024-05-09T06:39:30.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.757+0000 7efedb838640 1 -- start start 2024-05-09T06:39:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efedb838640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efed40b2aa0 0x7efed415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efedb838640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed4099930 con 0x7efed40c6860 2024-05-09T06:39:30.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efedb838640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed4162930 con 0x7efed4099b30 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efedb838640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7efed40b1e20 con 0x7efed40b2aa0 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efed95d4640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7efed40c6860 0x7efed40c6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57574/0 (socket says 172.21.15.1:57574) 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efed95d4640 1 -- 172.21.15.1:0/1897822705 learned_addr learned my addr 172.21.15.1:0/1897822705 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.758+0000 7efed9dd5640 1 --2- 172.21.15.1:0/1897822705 >> v2:172.21.15.6:3300/0 conn(0x7efed40b2aa0 0x7efed415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163680728 0 0) 0x7efec4001490 con 0x7efed40c6860 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efebc003520 con 0x7efed40c6860 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1150126834 0 0) 0x7efebc003520 con 0x7efed40c6860 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 >> v2:172.21.15.6:3300/0 conn(0x7efed40b2aa0 msgr2=0x7efed415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 --2- 172.21.15.1:0/1897822705 >> v2:172.21.15.6:3300/0 conn(0x7efed40b2aa0 0x7efed415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 >> v1:172.21.15.6:6789/0 conn(0x7efed4099b30 legacy=0x7efed415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed4162c10 con 0x7efed40c6860 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7efec4001bc0 con 0x7efed40c6860 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 >> v1:172.21.15.1:6789/0 conn(0x7efed40c6860 legacy=0x7efed40c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0039b0 0x7efebc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efebc004800 0x7efebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 0x7efebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efebc00c130 con 0x7efebc0051c0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efebc00c2e0 con 0x7efebc004800 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efebc00c4e0 con 0x7efebc0039b0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 0x7efebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed8dd3640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efebc004800 0x7efebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed95d4640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0039b0 0x7efebc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 -- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0039b0 msgr2=0x7efebc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0039b0 0x7efebc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 -- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efebc004800 msgr2=0x7efebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efebc004800 0x7efebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed9dd5640 1 -- 172.21.15.1:0/1897822705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc003520 con 0x7efebc0051c0 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.759+0000 7efed95d4640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0039b0 0x7efebc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.760+0000 7efed9dd5640 1 -- 172.21.15.1:0/1897822705 reap_dead start 2024-05-09T06:39:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.760+0000 7efed9dd5640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 0x7efebc00b9c0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7efecc009ac0 tx=0x7efebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae785bac05eca16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.760+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc009690 con 0x7efebc0051c0 2024-05-09T06:39:30.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.760+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc004020 con 0x7efebc0051c0 2024-05-09T06:39:30.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.760+0000 7efed3fff640 1 -- 172.21.15.1:0/1897822705 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc004410 con 0x7efebc0051c0 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 -- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 msgr2=0x7efebc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 0x7efebc00b9c0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7efecc009ac0 tx=0x7efebc005830 comp rx=0 tx=0).stop 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 -- 172.21.15.1:0/1897822705 shutdown_connections 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 --2- 172.21.15.1:0/1897822705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efebc0051c0 0x7efebc00b9c0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 -- 172.21.15.1:0/1897822705 >> 172.21.15.1:0/1897822705 conn(0x7efed40b1fb0 msgr2=0x7efed40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:30.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 -- 172.21.15.1:0/1897822705 shutdown_connections 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.761+0000 7efedb838640 1 -- 172.21.15.1:0/1897822705 wait complete. 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.762+0000 7efedb838640 1 Processor -- start 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.762+0000 7efedb838640 1 -- start start 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0051c0 0x7efed4150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efed40c7450 0x7efed40cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efed415e4b0 con 0x7efed415a8d0 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efed415ee90 con 0x7efed40c7450 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efedb838640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7efed415e630 con 0x7efebc0051c0 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59814/0 (socket says 172.21.15.1:59814) 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 -- 172.21.15.1:0/954205952 learned_addr learned my addr 172.21.15.1:0/954205952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed9dd5640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efed40c7450 0x7efed40cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed95d4640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0051c0 0x7efed4150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 -- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0051c0 msgr2=0x7efed4150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0051c0 0x7efed4150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 -- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efed40c7450 msgr2=0x7efed40cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efed40c7450 0x7efed40cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 -- 172.21.15.1:0/954205952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40cd3d0 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.763+0000 7efed8dd3640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7efec800f5f0 tx=0x7efec800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.764+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80052e0 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.764+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec8018070 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.764+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed40cce10 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.764+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8003810 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.764+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed40ca100 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.765+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed40752b0 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.765+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efec80039b0 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.766+0000 7efed1ffb640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 0x7efec006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.766+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7efec808ab80 con 0x7efed415a8d0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.766+0000 7efed95d4640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 0x7efec006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:30.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.766+0000 7efed95d4640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 0x7efec006e320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7efec4001750 tx=0x7efec40020f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:30.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:30.770+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efec80595f0 con 0x7efed415a8d0 2024-05-09T06:39:31.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.029+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7efed406ce30 con 0x7efed415a8d0 2024-05-09T06:39:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.319+0000 7efed1ffb640 1 -- 172.21.15.1:0/954205952 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v572) ==== 144+0+0 (secure 0 0 0) 0x7efec8058ef0 con 0x7efed415a8d0 2024-05-09T06:39:31.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 msgr2=0x7efec006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:31.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 0x7efec006e320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7efec4001750 tx=0x7efec40020f0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 msgr2=0x7efed40cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:31.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7efec800f5f0 tx=0x7efec800fab0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 shutdown_connections 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7efec006be60 0x7efec006e320 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efed40c7450 0x7efed40cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efed415a8d0 0x7efed40cbfa0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 --2- 172.21.15.1:0/954205952 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efebc0051c0 0x7efed4150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 >> 172.21.15.1:0/954205952 conn(0x7efed40b1fb0 msgr2=0x7efed40c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 shutdown_connections 2024-05-09T06:39:31.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:31.322+0000 7efedb838640 1 -- 172.21.15.1:0/954205952 wait complete. 2024-05-09T06:39:31.314 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-09T06:39:31.314 INFO:tasks.cephfs_test_runner: 2024-05-09T06:39:31.314 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T06:39:31.314 INFO:tasks.cephfs_test_runner:Ran 7 tests in 681.245s 2024-05-09T06:39:31.315 INFO:tasks.cephfs_test_runner: 2024-05-09T06:39:31.315 INFO:tasks.cephfs_test_runner:OK 2024-05-09T06:39:31.315 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-09T06:39:31.325 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T06:39:31.344 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T06:39:31.345 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:31.345 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:31.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:39:31.362 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T06:39:31.362 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:31.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3300 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:39:31.419 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:39:31.419 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:31.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:39:31.439 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T06:39:31.439 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:39:31.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1273 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:39:31.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.499 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T06:39:31.558 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T06:39:31.558 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T06:39:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.558 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:39:31.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:39:31.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.699 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-05-09T06:39:31.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.796 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T06:39:31.814 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:39:31.814 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:39:31.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:39:31.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.884 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:39:31.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:39:31.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-09T06:39:32.051 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T06:39:32.063 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T06:39:32.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:39:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.190+0000 7f5d5f722640 1 Processor -- start 2024-05-09T06:39:32.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.191+0000 7f5d5f722640 1 -- start start 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.191+0000 7f5d5f722640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58164ce0 0x7f5d581670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.191+0000 7f5d5f722640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d58071c20 con 0x7f5d581454b0 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.191+0000 7f5d5f722640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5806eb40 con 0x7f5d580b2a60 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.191+0000 7f5d5f722640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5806ee20 con 0x7f5d58164ce0 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58164ce0 0x7f5d581670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d58164ce0 0x7f5d581670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59828/0 (socket says 172.21.15.1:59828) 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5d4be640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5d581454b0 0x7f5d580b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57576/0 (socket says 172.21.15.1:57576) 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 -- 172.21.15.1:0/1191017786 learned_addr learned my addr 172.21.15.1:0/1191017786 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 -- 172.21.15.1:0/1191017786 >> v1:172.21.15.6:6789/0 conn(0x7f5d580b2a60 legacy=0x7f5d580b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 -- 172.21.15.1:0/1191017786 >> v1:172.21.15.1:6789/0 conn(0x7f5d581454b0 legacy=0x7f5d580b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.192+0000 7f5d5dcbf640 1 -- 172.21.15.1:0/1191017786 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5806d8e0 con 0x7f5d58164ce0 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5dcbf640 1 --2- 172.21.15.1:0/1191017786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58164ce0 0x7f5d581670d0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009e50 tx=0x7f5d4802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee71129d4939c0bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d57fff640 1 -- 172.21.15.1:0/1191017786 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48036070 con 0x7f5d58164ce0 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d57fff640 1 -- 172.21.15.1:0/1191017786 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d48031600 con 0x7f5d58164ce0 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d57fff640 1 -- 172.21.15.1:0/1191017786 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480319f0 con 0x7f5d58164ce0 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 -- 172.21.15.1:0/1191017786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58164ce0 msgr2=0x7f5d581670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 --2- 172.21.15.1:0/1191017786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58164ce0 0x7f5d581670d0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009e50 tx=0x7f5d4802a7c0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 -- 172.21.15.1:0/1191017786 shutdown_connections 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 --2- 172.21.15.1:0/1191017786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d58164ce0 0x7f5d581670d0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 -- 172.21.15.1:0/1191017786 >> 172.21.15.1:0/1191017786 conn(0x7f5d580cc4b0 msgr2=0x7f5d580cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 -- 172.21.15.1:0/1191017786 shutdown_connections 2024-05-09T06:39:32.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.193+0000 7f5d5f722640 1 -- 172.21.15.1:0/1191017786 wait complete. 2024-05-09T06:39:32.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 Processor -- start 2024-05-09T06:39:32.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 -- start start 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d580b2a60 0x7f5d581435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58164ce0 0x7f5d580ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580b5140 con 0x7f5d581454b0 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58145890 con 0x7f5d58164ce0 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.195+0000 7f5d5f722640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5806e0e0 con 0x7f5d580b2a60 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59836/0 (socket says 172.21.15.1:59836) 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 -- 172.21.15.1:0/804215267 learned_addr learned my addr 172.21.15.1:0/804215267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5d4be640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d580b2a60 0x7f5d581435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5dcbf640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58164ce0 0x7f5d580ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 -- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d580b2a60 msgr2=0x7f5d581435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d580b2a60 0x7f5d581435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 -- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58164ce0 msgr2=0x7f5d580ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58164ce0 0x7f5d580ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 -- 172.21.15.1:0/804215267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580ab8a0 con 0x7f5d581454b0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5ccbd640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c430 tx=0x7f5d4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0052d0 con 0x7f5d581454b0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c005490 con 0x7f5d581454b0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d580abb80 con 0x7f5d581454b0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.196+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007260 con 0x7f5d581454b0 2024-05-09T06:39:32.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.197+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d580ac140 con 0x7f5d581454b0 2024-05-09T06:39:32.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.198+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d580752b0 con 0x7f5d581454b0 2024-05-09T06:39:32.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.198+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d4c007400 con 0x7f5d581454b0 2024-05-09T06:39:32.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.199+0000 7f5d55ffb640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.199+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f5d4c08a2a0 con 0x7f5d581454b0 2024-05-09T06:39:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.199+0000 7f5d5d4be640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.199+0000 7f5d5d4be640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000f60 tx=0x7f5d50000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.203+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d4c058620 con 0x7f5d581454b0 2024-05-09T06:39:32.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.386+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d58072fa0 con 0x7f5d4406be30 2024-05-09T06:39:32.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.387+0000 7f5d55ffb640 1 -- 172.21.15.1:0/804215267 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7f5d58072fa0 con 0x7f5d4406be30 2024-05-09T06:39:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:32.368 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 msgr2=0x7f5d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000f60 tx=0x7f5d50000ed0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 msgr2=0x7f5d5812ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c430 tx=0x7f5d4c00c8f0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 shutdown_connections 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f5d4406be30 0x7f5d4406e2f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d58164ce0 0x7f5d580ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d581454b0 0x7f5d5812ce90 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 --2- 172.21.15.1:0/804215267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d580b2a60 0x7f5d581435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 >> 172.21.15.1:0/804215267 conn(0x7f5d580cc4b0 msgr2=0x7f5d580b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 shutdown_connections 2024-05-09T06:39:32.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.389+0000 7f5d5f722640 1 -- 172.21.15.1:0/804215267 wait complete. 2024-05-09T06:39:32.380 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-05-09T06:39:31.235376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291540,"kb_used_data":29636,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754683180,"statfs":{"total":773094113280,"available":772795576320,"internally_reserved":0,"allocated":30347264,"data_stored":23735159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001239"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-09T06:39:12.144493+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:39:12.144493+0000","last_peered":"2024-05-09T06:39:12.144493+0000","last_clean":"2024-05-09T06:39:12.144493+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:39:12.144493+0000","last_undegraded":"2024-05-09T06:39:12.144493+0000","last_fullsized":"2024-05-09T06:39:12.144493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1548288,"data_stored":721704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":725367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2134016,"data_stored":1309112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:39:32.381 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:39:32.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.504+0000 7f73abe2c640 1 Processor -- start 2024-05-09T06:39:32.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 -- start start 2024-05-09T06:39:32.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 0x7f73a40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a40b33c0 con 0x7f73a40d4940 2024-05-09T06:39:32.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f73a40d4dd0 con 0x7f73a40d0f80 2024-05-09T06:39:32.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.505+0000 7f73abe2c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f73a40b3660 con 0x7f73a4161950 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a9bc8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 0x7f73a40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73aa3c9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73aa3c9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.1:52334/0 (socket says 172.21.15.1:52334) 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a93c7640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f73a40d4940 0x7f73a40cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:45478/0 (socket says 172.21.15.1:45478) 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73aa3c9640 1 -- 172.21.15.1:0/2810948138 learned_addr learned my addr 172.21.15.1:0/2810948138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a9bc8640 1 -- 172.21.15.1:0/2810948138 >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 msgr2=0x7f73a40d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/2810948138 >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a9bc8640 1 -- 172.21.15.1:0/2810948138 >> v1:172.21.15.6:6789/0 conn(0x7f73a40d4940 legacy=0x7f73a40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a8bc6640 1 -- 172.21.15.1:0/2810948138 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581447158 0 0) 0x7f73a40b33c0 con 0x7f73a40d4940 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73a9bc8640 1 -- 172.21.15.1:0/2810948138 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a40b3940 con 0x7f73a4161950 2024-05-09T06:39:32.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.506+0000 7f73aa3c9640 1 --2- 172.21.15.1:0/2810948138 >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.507+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/2810948138 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 0x7f73a40d2400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73a40b3590 tx=0x7f73a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8cdc1dff5d5769 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.507+0000 7f73a8bc6640 1 -- 172.21.15.1:0/2810948138 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0037070 con 0x7f73a4161950 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.507+0000 7f73a8bc6640 1 -- 172.21.15.1:0/2810948138 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0032440 con 0x7f73a4161950 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.507+0000 7f73a8bc6640 1 -- 172.21.15.1:0/2810948138 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0032810 con 0x7f73a4161950 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 -- 172.21.15.1:0/2810948138 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 msgr2=0x7f73a40d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 --2- 172.21.15.1:0/2810948138 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 0x7f73a40d2400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73a40b3590 tx=0x7f73a002a180 comp rx=0 tx=0).stop 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 -- 172.21.15.1:0/2810948138 shutdown_connections 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 --2- 172.21.15.1:0/2810948138 >> v2:172.21.15.6:3300/0 conn(0x7f73a40d0f80 0x7f73a40d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 --2- 172.21.15.1:0/2810948138 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a4161950 0x7f73a40d2400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 -- 172.21.15.1:0/2810948138 >> 172.21.15.1:0/2810948138 conn(0x7f73a40d3e20 msgr2=0x7f73a40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 -- 172.21.15.1:0/2810948138 shutdown_connections 2024-05-09T06:39:32.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.508+0000 7f73abe2c640 1 -- 172.21.15.1:0/2810948138 wait complete. 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.509+0000 7f73abe2c640 1 Processor -- start 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.509+0000 7f73abe2c640 1 -- start start 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 0x7f73a4155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a40cbb50 0x7f73a40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40b52d0 con 0x7f73a4161950 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40b5150 con 0x7f73a40cbb50 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73abe2c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f73a40b5450 con 0x7f73a40d0f80 2024-05-09T06:39:32.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59846/0 (socket says 172.21.15.1:59846) 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 -- 172.21.15.1:0/376583122 learned_addr learned my addr 172.21.15.1:0/376583122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73aa3c9640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a40cbb50 0x7f73a40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 0x7f73a4155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 -- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 msgr2=0x7f73a4155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 0x7f73a4155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 -- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a40cbb50 msgr2=0x7f73a40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a40cbb50 0x7f73a40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 -- 172.21.15.1:0/376583122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41f4810 con 0x7f73a4161950 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 0x7f73a4155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.510+0000 7f73a93c7640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f739400c4d0 tx=0x7f739400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.511+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394018070 con 0x7f73a4161950 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.511+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394007500 con 0x7f73a4161950 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.511+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a40cb070 con 0x7f73a4161950 2024-05-09T06:39:32.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.511+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394004a20 con 0x7f73a4161950 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.511+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a40cb630 con 0x7f73a4161950 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.512+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a40cb830 con 0x7f73a4161950 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.513+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f739401b070 con 0x7f73a4161950 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.513+0000 7f739a7fc640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 0x7f738806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.513+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f739408a000 con 0x7f73a4161950 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.513+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 0x7f738806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.514+0000 7f73a9bc8640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 0x7f738806e2f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f73a0034000 tx=0x7f73a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.517+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73a40cb830 con 0x7f73a4161950 2024-05-09T06:39:32.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.702+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73a40752b0 con 0x7f738806be30 2024-05-09T06:39:32.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.703+0000 7f739a7fc640 1 -- 172.21.15.1:0/376583122 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7f73a40752b0 con 0x7f738806be30 2024-05-09T06:39:32.708 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 msgr2=0x7f738806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 0x7f738806e2f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f73a0034000 tx=0x7f73a002e000 comp rx=0 tx=0).stop 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 msgr2=0x7f73a40cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f739400c4d0 tx=0x7f739400c990 comp rx=0 tx=0).stop 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 shutdown_connections 2024-05-09T06:39:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f738806be30 0x7f738806e2f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a40cbb50 0x7f73a40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73a4161950 0x7f73a40cd3d0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 --2- 172.21.15.1:0/376583122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f73a40d0f80 0x7f73a4155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 >> 172.21.15.1:0/376583122 conn(0x7f73a40d3e20 msgr2=0x7f73a40d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 shutdown_connections 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.705+0000 7f73abe2c640 1 -- 172.21.15.1:0/376583122 wait complete. 2024-05-09T06:39:32.711 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-05-09T06:39:31.235376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291540,"kb_used_data":29636,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754683180,"statfs":{"total":773094113280,"available":772795576320,"internally_reserved":0,"allocated":30347264,"data_stored":23735159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001239"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-09T06:39:12.144493+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:39:12.144493+0000","last_peered":"2024-05-09T06:39:12.144493+0000","last_clean":"2024-05-09T06:39:12.144493+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:39:12.144493+0000","last_undegraded":"2024-05-09T06:39:12.144493+0000","last_fullsized":"2024-05-09T06:39:12.144493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1548288,"data_stored":721704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":725367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2134016,"data_stored":1309112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:39:32.712 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T06:39:32.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.836+0000 7effe2838640 1 Processor -- start 2024-05-09T06:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.836+0000 7effe2838640 1 -- start start 2024-05-09T06:39:32.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe2838640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7effdc1565e0 0x7effdc074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe2838640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 0x7effdc160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe2838640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effdc074a70 con 0x7effdc15c680 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe2838640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7effdc15fd30 con 0x7effdc1565e0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe2838640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7effdc15cc40 con 0x7effdc1601c0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effdbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7effdc1565e0 0x7effdc074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effd3fff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7effdc15c680 0x7effdc0afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57586/0 (socket says 172.21.15.1:57586) 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effdbfff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7effdc1565e0 0x7effdc074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59848/0 (socket says 172.21.15.1:59848) 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effd3fff640 1 -- 172.21.15.1:0/2279996118 learned_addr learned my addr 172.21.15.1:0/2279996118 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effe0dd5640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 0x7effdc160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effdbfff640 1 -- 172.21.15.1:0/2279996118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 msgr2=0x7effdc160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effdbfff640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 0x7effdc160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.837+0000 7effdbfff640 1 -- 172.21.15.1:0/2279996118 >> v1:172.21.15.1:6789/0 conn(0x7effdc15c680 legacy=0x7effdc0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effdbfff640 1 -- 172.21.15.1:0/2279996118 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc15cf20 con 0x7effdc1565e0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effe0dd5640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 0x7effdc160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effdbfff640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc074530 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097a0 tx=0x7effcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874307fb6678577d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effdb7fe640 1 -- 172.21.15.1:0/2279996118 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc037070 con 0x7effdc1565e0 2024-05-09T06:39:32.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effdb7fe640 1 -- 172.21.15.1:0/2279996118 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc032440 con 0x7effdc1565e0 2024-05-09T06:39:32.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.838+0000 7effdb7fe640 1 -- 172.21.15.1:0/2279996118 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc032810 con 0x7effdc1565e0 2024-05-09T06:39:32.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 -- 172.21.15.1:0/2279996118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 msgr2=0x7effdc074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc074530 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097a0 tx=0x7effcc02a180 comp rx=0 tx=0).stop 2024-05-09T06:39:32.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 -- 172.21.15.1:0/2279996118 shutdown_connections 2024-05-09T06:39:32.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc1601c0 0x7effdc160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 --2- 172.21.15.1:0/2279996118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc074530 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 -- 172.21.15.1:0/2279996118 >> 172.21.15.1:0/2279996118 conn(0x7effdc0b1fb0 msgr2=0x7effdc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 -- 172.21.15.1:0/2279996118 shutdown_connections 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.839+0000 7effe2838640 1 -- 172.21.15.1:0/2279996118 wait complete. 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.840+0000 7effe2838640 1 Processor -- start 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.840+0000 7effe2838640 1 -- start start 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc15c680 0x7effdc1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effdc1601c0 0x7effdc09d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7effdc1569c0 con 0x7effdc1565e0 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7effdc15d4e0 con 0x7effdc1601c0 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effe2838640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7effdc0b0340 con 0x7effdc15c680 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59860/0 (socket says 172.21.15.1:59860) 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 -- 172.21.15.1:0/3841336667 learned_addr learned my addr 172.21.15.1:0/3841336667 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 -- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc15c680 msgr2=0x7effdc1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc15c680 0x7effdc1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 -- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effdc1601c0 msgr2=0x7effdc09d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effdc1601c0 0x7effdc09d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.841+0000 7effdbfff640 1 -- 172.21.15.1:0/3841336667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc159000 con 0x7effdc1565e0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effdbfff640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7effcc035040 tx=0x7effcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc037040 con 0x7effdc1565e0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc030770 con 0x7effdc1565e0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc0b31d0 con 0x7effdc1565e0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc030b40 con 0x7effdc1565e0 2024-05-09T06:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.842+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc0b3470 con 0x7effdc1565e0 2024-05-09T06:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.843+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effdc0752b0 con 0x7effdc1565e0 2024-05-09T06:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.843+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effcc044020 con 0x7effdc1565e0 2024-05-09T06:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.844+0000 7effd97fa640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 0x7effbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.844+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7effcc0ab320 con 0x7effdc1565e0 2024-05-09T06:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.844+0000 7effd3fff640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 0x7effbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.844+0000 7effd3fff640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 0x7effbc06e2f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7effc4000f60 tx=0x7effc4007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:32.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:32.848+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7effcc032e20 con 0x7effdc1565e0 2024-05-09T06:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.028+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effdc0b3890 con 0x7effbc06be30 2024-05-09T06:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.028+0000 7effd97fa640 1 -- 172.21.15.1:0/3841336667 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7effdc0b3890 con 0x7effbc06be30 2024-05-09T06:39:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 msgr2=0x7effbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 0x7effbc06e2f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7effc4000f60 tx=0x7effc4007a80 comp rx=0 tx=0).stop 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 msgr2=0x7effdc0ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7effcc035040 tx=0x7effcc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 shutdown_connections 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7effbc06be30 0x7effbc06e2f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effdc1601c0 0x7effdc09d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effdc15c680 0x7effdc1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 --2- 172.21.15.1:0/3841336667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effdc1565e0 0x7effdc0ae740 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 >> 172.21.15.1:0/3841336667 conn(0x7effdc0b1fb0 msgr2=0x7effdc1556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.030+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 shutdown_connections 2024-05-09T06:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.031+0000 7effe2838640 1 -- 172.21.15.1:0/3841336667 wait complete. 2024-05-09T06:39:33.022 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-05-09T06:39:31.235376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291540,"kb_used_data":29636,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754683180,"statfs":{"total":773094113280,"available":772795576320,"internally_reserved":0,"allocated":30347264,"data_stored":23735159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001239"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-09T06:39:12.144493+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:39:12.144493+0000","last_peered":"2024-05-09T06:39:12.144493+0000","last_clean":"2024-05-09T06:39:12.144493+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:39:12.144493+0000","last_undegraded":"2024-05-09T06:39:12.144493+0000","last_fullsized":"2024-05-09T06:39:12.144493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1309868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":5730304,"data_stored":4913603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5746688,"data_stored":4915986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1548288,"data_stored":721704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":725367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2134016,"data_stored":1309112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:39:33.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:39:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.147+0000 7fa22cd42640 1 Processor -- start 2024-05-09T06:39:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.147+0000 7fa22cd42640 1 -- start start 2024-05-09T06:39:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22cd42640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa2281454b0 0x7fa2280b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22cd42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa228070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22cd42640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa22806eb40 con 0x7fa2280b2a60 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22cd42640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa22806ee20 con 0x7fa2281454b0 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22cd42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa22806d8e0 con 0x7fa2280b4f70 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.148+0000 7fa22659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa2281454b0 0x7fa2280b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa2281454b0 0x7fa2280b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59866/0 (socket says 172.21.15.1:59866) 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 -- 172.21.15.1:0/568570961 learned_addr learned my addr 172.21.15.1:0/568570961 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa226d9d640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa228070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 -- 172.21.15.1:0/568570961 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 msgr2=0x7fa228070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa228070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 -- 172.21.15.1:0/568570961 >> v1:172.21.15.6:6789/0 conn(0x7fa2280b2a60 legacy=0x7fa2280b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.149+0000 7fa22659c640 1 -- 172.21.15.1:0/568570961 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22806dbc0 con 0x7fa2281454b0 2024-05-09T06:39:33.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa22659c640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280b4320 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fa218009870 tx=0x7fa21802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dd9373be84cae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:33.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa225d9b640 1 -- 172.21.15.1:0/568570961 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218037070 con 0x7fa2281454b0 2024-05-09T06:39:33.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa225d9b640 1 -- 172.21.15.1:0/568570961 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa218032440 con 0x7fa2281454b0 2024-05-09T06:39:33.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa225d9b640 1 -- 172.21.15.1:0/568570961 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218032810 con 0x7fa2281454b0 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa22cd42640 1 -- 172.21.15.1:0/568570961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 msgr2=0x7fa2280b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa22cd42640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280b4320 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fa218009870 tx=0x7fa21802a210 comp rx=0 tx=0).stop 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa22cd42640 1 -- 172.21.15.1:0/568570961 shutdown_connections 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.150+0000 7fa22cd42640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa228070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.151+0000 7fa22cd42640 1 --2- 172.21.15.1:0/568570961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280b4320 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.151+0000 7fa22cd42640 1 -- 172.21.15.1:0/568570961 >> 172.21.15.1:0/568570961 conn(0x7fa2280cc4b0 msgr2=0x7fa2280cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.151+0000 7fa22cd42640 1 -- 172.21.15.1:0/568570961 shutdown_connections 2024-05-09T06:39:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.151+0000 7fa22cd42640 1 -- 172.21.15.1:0/568570961 wait complete. 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.152+0000 7fa22cd42640 1 Processor -- start 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.152+0000 7fa22cd42640 1 -- start start 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2280b2a60 0x7fa22815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa2280ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa22806e260 con 0x7fa2281454b0 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa22806e0e0 con 0x7fa2280b2a60 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22cd42640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa228071150 con 0x7fa2280b4f70 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59870/0 (socket says 172.21.15.1:59870) 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 -- 172.21.15.1:0/1381933093 learned_addr learned my addr 172.21.15.1:0/1381933093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa22659c640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2280b2a60 0x7fa22815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa21ffff640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa2280ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 -- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 msgr2=0x7fa2280ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa2280ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 -- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2280b2a60 msgr2=0x7fa22815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2280b2a60 0x7fa22815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.153+0000 7fa226d9d640 1 -- 172.21.15.1:0/1381933093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2280c5870 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa226d9d640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa220007e30 tx=0x7fa22000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220014620 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa220019070 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220014a00 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2280c5b50 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.154+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2280c6110 con 0x7fa2281454b0 2024-05-09T06:39:33.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.155+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2280752b0 con 0x7fa2281454b0 2024-05-09T06:39:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.156+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa220014ba0 con 0x7fa2281454b0 2024-05-09T06:39:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.156+0000 7fa21effd640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 0x7fa20806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.156+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fa22008b6c0 con 0x7fa2281454b0 2024-05-09T06:39:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.157+0000 7fa22659c640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 0x7fa20806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.157+0000 7fa22659c640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 0x7fa20806e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa218034000 tx=0x7fa21802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.161+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa22005a130 con 0x7fa2281454b0 2024-05-09T06:39:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.346+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa228072fa0 con 0x7fa2281454b0 2024-05-09T06:39:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.347+0000 7fa21effd640 1 -- 172.21.15.1:0/1381933093 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+13423 (secure 0 0 0) 0x7fa220059a30 con 0x7fa2281454b0 2024-05-09T06:39:33.328 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:33.328 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":126,"fsid":"91cbcc67-ea38-4460-ad67-94260dad3bad","created":"2024-05-09T06:26:56.717142+0000","modified":"2024-05-09T06:39:30.308748+0000","last_up_change":"2024-05-09T06:27:07.516422+0000","last_in_change":"2024-05-09T06:26:59.030580+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:27:09.112330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936044ae-137f-4baa-92c2-b88931dba8a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6821","nonce":3367065226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6825","nonce":3367065226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6831","nonce":3367065226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":3367065226},{"type":"v1","addr":"172.21.15.1:6829","nonce":3367065226}]},"public_addr":"172.21.15.1:6821/3367065226","cluster_addr":"172.21.15.1:6825/3367065226","heartbeat_back_addr":"172.21.15.1:6831/3367065226","heartbeat_front_addr":"172.21.15.1:6829/3367065226","state":["exists","up"]},{"osd":1,"uuid":"4d973a54-6715-4cb9-9841-1bd1cb6da31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6817","nonce":1858675679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6820","nonce":1858675679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6828","nonce":1858675679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1858675679},{"type":"v1","addr":"172.21.15.1:6824","nonce":1858675679}]},"public_addr":"172.21.15.1:6817/1858675679","cluster_addr":"172.21.15.1:6820/1858675679","heartbeat_back_addr":"172.21.15.1:6828/1858675679","heartbeat_front_addr":"172.21.15.1:6824/1858675679","state":["exists","up"]},{"osd":2,"uuid":"75cf529c-4960-4054-8bb5-c4a1cf3b3a45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6809","nonce":1632480843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6811","nonce":1632480843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6815","nonce":1632480843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1632480843},{"type":"v1","addr":"172.21.15.1:6813","nonce":1632480843}]},"public_addr":"172.21.15.1:6809/1632480843","cluster_addr":"172.21.15.1:6811/1632480843","heartbeat_back_addr":"172.21.15.1:6815/1632480843","heartbeat_front_addr":"172.21.15.1:6813/1632480843","state":["exists","up"]},{"osd":3,"uuid":"37bd81bc-61f1-4708-8a99-57fff41b8bd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6801","nonce":2131549113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6803","nonce":2131549113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6807","nonce":2131549113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2131549113},{"type":"v1","addr":"172.21.15.1:6805","nonce":2131549113}]},"public_addr":"172.21.15.1:6801/2131549113","cluster_addr":"172.21.15.1:6803/2131549113","heartbeat_back_addr":"172.21.15.1:6807/2131549113","heartbeat_front_addr":"172.21.15.1:6805/2131549113","state":["exists","up"]},{"osd":4,"uuid":"62d462ba-310a-43bf-8f74-53e0341b086f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6802","nonce":3928343737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6806","nonce":3928343737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6814","nonce":3928343737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3928343737},{"type":"v1","addr":"172.21.15.6:6810","nonce":3928343737}]},"public_addr":"172.21.15.6:6802/3928343737","cluster_addr":"172.21.15.6:6806/3928343737","heartbeat_back_addr":"172.21.15.6:6814/3928343737","heartbeat_front_addr":"172.21.15.6:6810/3928343737","state":["exists","up"]},{"osd":5,"uuid":"1d54b45c-4b3c-4a15-9fac-c84f3d8d235d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6801","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6803","nonce":2308051940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6805","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6807","nonce":2308051940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6813","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6815","nonce":2308051940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":2308051940},{"type":"v1","addr":"172.21.15.6:6811","nonce":2308051940}]},"public_addr":"172.21.15.6:6803/2308051940","cluster_addr":"172.21.15.6:6807/2308051940","heartbeat_back_addr":"172.21.15.6:6815/2308051940","heartbeat_front_addr":"172.21.15.6:6811/2308051940","state":["exists","up"]},{"osd":6,"uuid":"09b68656-4d19-45d1-853b-7a886409ed0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6817","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6819","nonce":1315796905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6821","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6823","nonce":1315796905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6831","nonce":1315796905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":1315796905},{"type":"v1","addr":"172.21.15.6:6827","nonce":1315796905}]},"public_addr":"172.21.15.6:6819/1315796905","cluster_addr":"172.21.15.6:6823/1315796905","heartbeat_back_addr":"172.21.15.6:6831/1315796905","heartbeat_front_addr":"172.21.15.6:6827/1315796905","state":["exists","up"]},{"osd":7,"uuid":"86cb3022-da70-45da-8eeb-4186d6bc51f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6818","nonce":3240730649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6822","nonce":3240730649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6829","nonce":3240730649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3240730649},{"type":"v1","addr":"172.21.15.6:6826","nonce":3240730649}]},"public_addr":"172.21.15.6:6818/3240730649","cluster_addr":"172.21.15.6:6822/3240730649","heartbeat_back_addr":"172.21.15.6:6829/3240730649","heartbeat_front_addr":"172.21.15.6:6826/3240730649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.122502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.213915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.153740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.170519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.151577+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.140464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.205123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:27:04.157391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6837/520088510":"2024-05-10T06:39:26.962487+0000","172.21.15.1:6837/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/3597643567":"2024-05-10T06:39:12.063103+0000","172.21.15.1:6836/520088510":"2024-05-10T06:39:26.962487+0000","172.21.15.1:6837/1060593353":"2024-05-10T06:38:53.413211+0000","172.21.15.1:6836/1060593353":"2024-05-10T06:38:53.413211+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 msgr2=0x7fa20806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 0x7fa20806e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa218034000 tx=0x7fa21802e000 comp rx=0 tx=0).stop 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 msgr2=0x7fa2280ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa220007e30 tx=0x7fa22000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 shutdown_connections 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa20806be30 0x7fa20806e2f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2281454b0 0x7fa2280ae8f0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa2280b4f70 0x7fa2280ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 --2- 172.21.15.1:0/1381933093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2280b2a60 0x7fa22815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:33.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 >> 172.21.15.1:0/1381933093 conn(0x7fa2280cc4b0 msgr2=0x7fa2280691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:33.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 shutdown_connections 2024-05-09T06:39:33.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:33.349+0000 7fa22cd42640 1 -- 172.21.15.1:0/1381933093 wait complete. 2024-05-09T06:39:34.342 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T06:39:34.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:34.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.473+0000 7f809f73b640 1 Processor -- start 2024-05-09T06:39:34.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.473+0000 7f809f73b640 1 -- start start 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809f73b640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f809815e310 0x7f80980afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809f73b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f8098071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809f73b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f809806e050 con 0x7f80980b3360 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809f73b640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f80980ce340 con 0x7f809815e310 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809f73b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80980b0190 con 0x7f809806b8f0 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809d4d7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f809815e310 0x7f80980afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809ccd6640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f80980b3360 0x7f809806b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57588/0 (socket says 172.21.15.1:57588) 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809ccd6640 1 -- 172.21.15.1:0/3360959594 learned_addr learned my addr 172.21.15.1:0/3360959594 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.474+0000 7f809dcd8640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f8098071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f809d4d7640 1 -- 172.21.15.1:0/3360959594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 msgr2=0x7f8098071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:34.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f809d4d7640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f8098071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f809d4d7640 1 -- 172.21.15.1:0/3360959594 >> v1:172.21.15.1:6789/0 conn(0x7f80980b3360 legacy=0x7f809806b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f8097fff640 1 -- 172.21.15.1:0/3360959594 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111131604 0 0) 0x7f809806e050 con 0x7f80980b3360 2024-05-09T06:39:34.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f809d4d7640 1 -- 172.21.15.1:0/3360959594 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980cd730 con 0x7f809815e310 2024-05-09T06:39:34.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f809d4d7640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f80980afc50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f8088009d90 tx=0x7f808802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2c1c00031777de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.475+0000 7f8097fff640 1 -- 172.21.15.1:0/3360959594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088007740 con 0x7f809815e310 2024-05-09T06:39:34.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f8097fff640 1 -- 172.21.15.1:0/3360959594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80880078e0 con 0x7f809815e310 2024-05-09T06:39:34.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f8097fff640 1 -- 172.21.15.1:0/3360959594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088007cb0 con 0x7f809815e310 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 -- 172.21.15.1:0/3360959594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 msgr2=0x7f80980afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f80980afc50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f8088009d90 tx=0x7f808802a660 comp rx=0 tx=0).stop 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 -- 172.21.15.1:0/3360959594 shutdown_connections 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f8098071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 --2- 172.21.15.1:0/3360959594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f80980afc50 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 -- 172.21.15.1:0/3360959594 >> 172.21.15.1:0/3360959594 conn(0x7f80980b2870 msgr2=0x7f80980b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 -- 172.21.15.1:0/3360959594 shutdown_connections 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.476+0000 7f809f73b640 1 -- 172.21.15.1:0/3360959594 wait complete. 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.477+0000 7f809f73b640 1 Processor -- start 2024-05-09T06:39:34.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.477+0000 7f809f73b640 1 -- start start 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f809814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b3360 0x7f80980b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f8098156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80980cdfd0 con 0x7f809815e310 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f80980cdcf0 con 0x7f80980b3360 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809f73b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f80980cf600 con 0x7f809806b8f0 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809dcd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f8098156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809dcd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f8098156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59886/0 (socket says 172.21.15.1:59886) 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809dcd8640 1 -- 172.21.15.1:0/3381810507 learned_addr learned my addr 172.21.15.1:0/3381810507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809d4d7640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f809814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.478+0000 7f809ccd6640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b3360 0x7f80980b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 -- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 msgr2=0x7f809814c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f809806b8f0 0x7f809814c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 -- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b3360 msgr2=0x7f80980b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b3360 0x7f80980b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980b50b0 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809dcd8640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f8098156300 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f809000d4e0 tx=0x7f809000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900125f0 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8090019070 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80980b5390 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900129f0 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.479+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80980b4c30 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.480+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8098067ac0 con 0x7f809815e310 2024-05-09T06:39:34.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8090012b90 con 0x7f809815e310 2024-05-09T06:39:34.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f808406be60 0x7f808406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f809008af00 con 0x7f809815e310 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f809d4d7640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f808406be60 0x7f808406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7f8084072440 0x7f8084074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8084074f50 con 0x7f8084072440 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.481+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f809008d600 con 0x7f809815e310 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.482+0000 7f809ccd6640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7f8084072440 0x7f8084074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.482+0000 7f809d4d7640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f808406be60 0x7f808406e320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.482+0000 7f809ccd6640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7f8084072440 0x7f8084074880 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.483+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== osd.0 v2:172.21.15.1:6819/3367065226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8084074f50 con 0x7f8084072440 2024-05-09T06:39:34.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.496+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 --> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8098075320 con 0x7f8084072440 2024-05-09T06:39:34.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.496+0000 7f8095ffb640 1 -- 172.21.15.1:0/3381810507 <== osd.0 v2:172.21.15.1:6819/3367065226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8098075320 con 0x7f8084072440 2024-05-09T06:39:34.477 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:34.477 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:34.477 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.496+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7f8084072440 msgr2=0x7f8084074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.496+0000 7f809f73b640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6819/3367065226,v1:172.21.15.1:6821/3367065226] conn(0x7f8084072440 0x7f8084074880 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f808406be60 msgr2=0x7f808406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f808406be60 0x7f808406e320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802f000 comp rx=0 tx=0).stop 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 msgr2=0x7f8098156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 --2- 172.21.15.1:0/3381810507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f809815e310 0x7f8098156300 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f809000d4e0 tx=0x7f809000d9a0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809dcd8640 1 -- 172.21.15.1:0/3381810507 reap_dead start 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 shutdown_connections 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 >> 172.21.15.1:0/3381810507 conn(0x7f80980b2870 msgr2=0x7f80980b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 shutdown_connections 2024-05-09T06:39:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.497+0000 7f809f73b640 1 -- 172.21.15.1:0/3381810507 wait complete. 2024-05-09T06:39:34.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T06:39:34.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.611+0000 7fce2d95c640 1 Processor -- start 2024-05-09T06:39:34.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.611+0000 7fce2d95c640 1 -- start start 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce2d95c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce2d95c640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce280b33c0 con 0x7fce280d4940 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce2d95c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce280b3660 con 0x7fce280d0f80 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce2d95c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce280d4dd0 con 0x7fce28161950 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce267fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fce280d4940 0x7fce280cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57602/0 (socket says 172.21.15.1:57602) 2024-05-09T06:39:34.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.612+0000 7fce267fc640 1 -- 172.21.15.1:0/3291165546 learned_addr learned my addr 172.21.15.1:0/3291165546 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce26ffd640 1 --2- 172.21.15.1:0/3291165546 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084797094 0 0) 0x7fce280b33c0 con 0x7fce280d4940 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce14003520 con 0x7fce280d4940 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce26ffd640 1 -- 172.21.15.1:0/3291165546 >> v1:172.21.15.6:6789/0 conn(0x7fce280d0f80 legacy=0x7fce280cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce26ffd640 1 -- 172.21.15.1:0/3291165546 >> v1:172.21.15.1:6789/0 conn(0x7fce280d4940 legacy=0x7fce280cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478610786 0 0) 0x7fce280b3660 con 0x7fce280d0f80 2024-05-09T06:39:34.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.613+0000 7fce26ffd640 1 -- 172.21.15.1:0/3291165546 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce280b3940 con 0x7fce28161950 2024-05-09T06:39:34.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce26ffd640 1 --2- 172.21.15.1:0/3291165546 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280d2400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce20009dc0 tx=0x7fce2002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0433f937cfbef31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20007740 con 0x7fce28161950 2024-05-09T06:39:34.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce200078e0 con 0x7fce28161950 2024-05-09T06:39:34.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce25ffb640 1 -- 172.21.15.1:0/3291165546 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20007c90 con 0x7fce28161950 2024-05-09T06:39:34.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce2d95c640 1 -- 172.21.15.1:0/3291165546 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 msgr2=0x7fce280d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce2d95c640 1 --2- 172.21.15.1:0/3291165546 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280d2400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce20009dc0 tx=0x7fce2002a6d0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce2d95c640 1 -- 172.21.15.1:0/3291165546 shutdown_connections 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce2d95c640 1 --2- 172.21.15.1:0/3291165546 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280d2400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.614+0000 7fce2d95c640 1 -- 172.21.15.1:0/3291165546 >> 172.21.15.1:0/3291165546 conn(0x7fce280d3e20 msgr2=0x7fce280d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.615+0000 7fce2d95c640 1 -- 172.21.15.1:0/3291165546 shutdown_connections 2024-05-09T06:39:34.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.615+0000 7fce2d95c640 1 -- 172.21.15.1:0/3291165546 wait complete. 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.616+0000 7fce2d95c640 1 Processor -- start 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.616+0000 7fce2d95c640 1 -- start start 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce280d0f80 0x7fce281532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fce280b5150 con 0x7fce280cd090 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fce280cf850 con 0x7fce280d0f80 2024-05-09T06:39:34.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fce280b52d0 con 0x7fce28161950 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59902/0 (socket says 172.21.15.1:59902) 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 -- 172.21.15.1:0/170000988 learned_addr learned my addr 172.21.15.1:0/170000988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce267fc640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce26ffd640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce280d0f80 0x7fce281532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 -- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 msgr2=0x7fce280ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 -- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce280d0f80 msgr2=0x7fce281532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce280d0f80 0x7fce281532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 -- 172.21.15.1:0/170000988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce280cc8d0 con 0x7fce280cd090 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce277fe640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c3e0 tx=0x7fce1800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18017070 con 0x7fce280cd090 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce180052f0 con 0x7fce280cd090 2024-05-09T06:39:34.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18003850 con 0x7fce280cd090 2024-05-09T06:39:34.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.617+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce280b6ab0 con 0x7fce280cd090 2024-05-09T06:39:34.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.618+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce280b7070 con 0x7fce280cd090 2024-05-09T06:39:34.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.619+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce280752b0 con 0x7fce280cd090 2024-05-09T06:39:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.619+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce1801a070 con 0x7fce280cd090 2024-05-09T06:39:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.620+0000 7fce1ffff640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 0x7fce0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.620+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fce1808a120 con 0x7fce280cd090 2024-05-09T06:39:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.620+0000 7fce26ffd640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 0x7fce0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.620+0000 7fce26ffd640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 0x7fce0806e2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fce20034000 tx=0x7fce2002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.624+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce180584a0 con 0x7fce280cd090 2024-05-09T06:39:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.807+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fce2815f070 con 0x7fce0806be30 2024-05-09T06:39:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.807+0000 7fce1ffff640 1 -- 172.21.15.1:0/170000988 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fce2815f070 con 0x7fce0806be30 2024-05-09T06:39:34.787 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 msgr2=0x7fce0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 0x7fce0806e2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fce20034000 tx=0x7fce2002f000 comp rx=0 tx=0).stop 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 msgr2=0x7fce280cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c3e0 tx=0x7fce1800c8a0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 shutdown_connections 2024-05-09T06:39:34.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.809+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fce0806be30 0x7fce0806e2f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce280cd090 0x7fce280cbba0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fce28161950 0x7fce280ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 --2- 172.21.15.1:0/170000988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce280d0f80 0x7fce281532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 >> 172.21.15.1:0/170000988 conn(0x7fce280d3e20 msgr2=0x7fce280d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 shutdown_connections 2024-05-09T06:39:34.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.810+0000 7fce2d95c640 1 -- 172.21.15.1:0/170000988 wait complete. 2024-05-09T06:39:34.800 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T06:39:34.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:34.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.928+0000 7f797c9f4640 1 Processor -- start 2024-05-09T06:39:34.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.928+0000 7f797c9f4640 1 -- start start 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978143bb0 0x7f79780cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f79781446d0 0x7f79780d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7978144110 0x7f797815e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f79780ce8d0 con 0x7f7978144110 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f79780cdd50 con 0x7f79781446d0 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.929+0000 7f797c9f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f79780ce030 con 0x7f7978143bb0 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7978144110 0x7f797815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7978144110 0x7f797815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59918/0 (socket says 172.21.15.1:59918) 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 -- 172.21.15.1:0/2562761123 learned_addr learned my addr 172.21.15.1:0/2562761123 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:34.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f797659c640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978143bb0 0x7f79780cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7975d9b640 1 --2- 172.21.15.1:0/2562761123 >> v2:172.21.15.6:3300/0 conn(0x7f79781446d0 0x7f79780d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 -- 172.21.15.1:0/2562761123 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978143bb0 msgr2=0x7f79780cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978143bb0 0x7f79780cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 -- 172.21.15.1:0/2562761123 >> v2:172.21.15.6:3300/0 conn(0x7f79781446d0 msgr2=0x7f79780d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 --2- 172.21.15.1:0/2562761123 >> v2:172.21.15.6:3300/0 conn(0x7f79781446d0 0x7f79780d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.930+0000 7f7976d9d640 1 -- 172.21.15.1:0/2562761123 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79780ce310 con 0x7f7978144110 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f7976d9d640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978144110 0x7f797815e960 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f79780cd710 tx=0x7f797002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edd2b3ef9121b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797559a640 1 -- 172.21.15.1:0/2562761123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970032700 con 0x7f7978144110 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797559a640 1 -- 172.21.15.1:0/2562761123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79700328a0 con 0x7f7978144110 2024-05-09T06:39:34.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797559a640 1 -- 172.21.15.1:0/2562761123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970032c70 con 0x7f7978144110 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 -- 172.21.15.1:0/2562761123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978144110 msgr2=0x7f797815e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978144110 0x7f797815e960 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f79780cd710 tx=0x7f797002c800 comp rx=0 tx=0).stop 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 -- 172.21.15.1:0/2562761123 shutdown_connections 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978144110 0x7f797815e960 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 --2- 172.21.15.1:0/2562761123 >> v2:172.21.15.6:3300/0 conn(0x7f79781446d0 0x7f79780d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 --2- 172.21.15.1:0/2562761123 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978143bb0 0x7f79780cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.931+0000 7f797c9f4640 1 -- 172.21.15.1:0/2562761123 >> 172.21.15.1:0/2562761123 conn(0x7f797806b3c0 msgr2=0x7f797806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.932+0000 7f797c9f4640 1 -- 172.21.15.1:0/2562761123 shutdown_connections 2024-05-09T06:39:34.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.932+0000 7f797c9f4640 1 -- 172.21.15.1:0/2562761123 wait complete. 2024-05-09T06:39:34.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.933+0000 7f797c9f4640 1 Processor -- start 2024-05-09T06:39:34.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.933+0000 7f797c9f4640 1 -- start start 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 0x7f79780b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978144110 0x7f79780cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79781446d0 0x7f79780b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f79780cdaf0 con 0x7f7978143bb0 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f79780cd970 con 0x7f79781446d0 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f797815eea0 con 0x7f7978144110 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 0x7f79780b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 0x7f79780b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59926/0 (socket says 172.21.15.1:59926) 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 -- 172.21.15.1:0/3788339233 learned_addr learned my addr 172.21.15.1:0/3788339233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 -- 172.21.15.1:0/3788339233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978144110 msgr2=0x7f79780cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f7975d9b640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978144110 0x7f79780cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7978144110 0x7f79780cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 -- 172.21.15.1:0/3788339233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79781446d0 msgr2=0x7f79780b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79781446d0 0x7f79780b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79780b4a80 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797659c640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 0x7f79780b4540 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f7968009db0 tx=0x7f796800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968018020 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968014480 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79780b34a0 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968014850 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.934+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79780b3a60 con 0x7f7978143bb0 2024-05-09T06:39:34.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.935+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7978075320 con 0x7f7978143bb0 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.936+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7968005d10 con 0x7f7978143bb0 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.936+0000 7f796effd640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f795806be30 0x7f795806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f7975d9b640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f795806be30 0x7f795806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f796808bfc0 con 0x7f7978143bb0 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f796effd640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f7958072410 0x7f7958074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:34.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79680020b0 con 0x7f7958072410 2024-05-09T06:39:34.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f796808c450 con 0x7f7978143bb0 2024-05-09T06:39:34.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f7976d9d640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f7958072410 0x7f7958074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:34.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f7975d9b640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f795806be30 0x7f795806e2f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f79781f3260 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.937+0000 7f7976d9d640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f7958072410 0x7f7958074850 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:34.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.938+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== osd.1 v2:172.21.15.1:6816/1858675679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79680020b0 con 0x7f7958072410 2024-05-09T06:39:34.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.951+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 --> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f79780cf6b0 con 0x7f7958072410 2024-05-09T06:39:34.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.951+0000 7f796effd640 1 -- 172.21.15.1:0/3788339233 <== osd.1 v2:172.21.15.1:6816/1858675679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f79780cf6b0 con 0x7f7958072410 2024-05-09T06:39:34.932 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:34.932 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f7958072410 msgr2=0x7f7958074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6816/1858675679,v1:172.21.15.1:6817/1858675679] conn(0x7f7958072410 0x7f7958074850 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f795806be30 msgr2=0x7f795806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f795806be30 0x7f795806e2f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f79781f3260 tx=0x7f7960008040 comp rx=0 tx=0).stop 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 msgr2=0x7f79780b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 --2- 172.21.15.1:0/3788339233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7978143bb0 0x7f79780b4540 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f7968009db0 tx=0x7f796800c490 comp rx=0 tx=0).stop 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f7976d9d640 1 -- 172.21.15.1:0/3788339233 reap_dead start 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 shutdown_connections 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 >> 172.21.15.1:0/3788339233 conn(0x7f797806b3c0 msgr2=0x7f797815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 shutdown_connections 2024-05-09T06:39:34.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:34.952+0000 7f797c9f4640 1 -- 172.21.15.1:0/3788339233 wait complete. 2024-05-09T06:39:34.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T06:39:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.067+0000 7f0da199e640 1 Processor -- start 2024-05-09T06:39:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.067+0000 7f0da199e640 1 -- start start 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 0x7f0d9c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c0ce0e0 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c0ce3c0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0da199e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c0ce6a0 con 0x7f0d9c0cdab0 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9affd640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59928/0 (socket says 172.21.15.1:59928) 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9affd640 1 -- 172.21.15.1:0/2998804716 learned_addr learned my addr 172.21.15.1:0/2998804716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9a7fc640 1 --2- 172.21.15.1:0/2998804716 >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 0x7f0d9c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9b7fe640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9affd640 1 -- 172.21.15.1:0/2998804716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 msgr2=0x7f0d9c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.068+0000 7f0d9affd640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c15e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d9affd640 1 -- 172.21.15.1:0/2998804716 >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 msgr2=0x7f0d9c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d9affd640 1 --2- 172.21.15.1:0/2998804716 >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 0x7f0d9c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d9affd640 1 -- 172.21.15.1:0/2998804716 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c0ce980 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d9a7fc640 1 --2- 172.21.15.1:0/2998804716 >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 0x7f0d9c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d9affd640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0b27c0 tx=0x7f0d8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b277f77efa564bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d99ffb640 1 -- 172.21.15.1:0/2998804716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c036070 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d99ffb640 1 -- 172.21.15.1:0/2998804716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c031600 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.069+0000 7f0d99ffb640 1 -- 172.21.15.1:0/2998804716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0319d0 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 -- 172.21.15.1:0/2998804716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c06e6d0 msgr2=0x7f0d9c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0b27c0 tx=0x7f0d8c02a6a0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 -- 172.21.15.1:0/2998804716 shutdown_connections 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c15e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 --2- 172.21.15.1:0/2998804716 >> v2:172.21.15.6:3300/0 conn(0x7f0d9c0cf180 0x7f0d9c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 --2- 172.21.15.1:0/2998804716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c06eab0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 -- 172.21.15.1:0/2998804716 >> 172.21.15.1:0/2998804716 conn(0x7f0d9c0cc4b0 msgr2=0x7f0d9c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 -- 172.21.15.1:0/2998804716 shutdown_connections 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.070+0000 7f0da199e640 1 -- 172.21.15.1:0/2998804716 wait complete. 2024-05-09T06:39:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.071+0000 7f0da199e640 1 Processor -- start 2024-05-09T06:39:35.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.071+0000 7f0da199e640 1 -- start start 2024-05-09T06:39:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c070ed0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c06eff0 con 0x7f0d9c06e6d0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0da199e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c1447c0 con 0x7f0d9c0cdab0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59936/0 (socket says 172.21.15.1:59936) 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0d9b7fe640 1 -- 172.21.15.1:0/1311562377 learned_addr learned my addr 172.21.15.1:0/1311562377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0d9affd640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.072+0000 7f0d9a7fc640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 -- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 msgr2=0x7f0d9c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 -- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d9c06e6d0 msgr2=0x7f0d9c1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 -- 172.21.15.1:0/1311562377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c13cbc0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d9b7fe640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d830 tx=0x7f0d9400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94018020 con 0x7f0d9c0cf180 2024-05-09T06:39:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d940096c0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c13cea0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94009a90 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.073+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1f4130 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.074+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c0752b0 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.075+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d9401b020 con 0x7f0d9c0cf180 2024-05-09T06:39:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.075+0000 7f0d93fff640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 0x7f0d7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.075+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f0d9408b040 con 0x7f0d9c0cf180 2024-05-09T06:39:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.075+0000 7f0d9affd640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 0x7f0d7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.076+0000 7f0d9affd640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 0x7f0d7c06e2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c033000 tx=0x7f0d8c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.079+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d94022030 con 0x7f0d9c0cf180 2024-05-09T06:39:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.262+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f0d9c06cf50 con 0x7f0d7c06be30 2024-05-09T06:39:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.263+0000 7f0d93fff640 1 -- 172.21.15.1:0/1311562377 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0d9c06cf50 con 0x7f0d7c06be30 2024-05-09T06:39:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T06:39:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 msgr2=0x7f0d7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 0x7f0d7c06e2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c033000 tx=0x7f0d8c009000 comp rx=0 tx=0).stop 2024-05-09T06:39:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 msgr2=0x7f0d9c13c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d830 tx=0x7f0d9400dcf0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 shutdown_connections 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0d7c06be30 0x7f0d7c06e2f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d9c0cf180 0x7f0d9c13c050 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d9c0cdab0 0x7f0d9c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 --2- 172.21.15.1:0/1311562377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d9c06e6d0 0x7f0d9c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 >> 172.21.15.1:0/1311562377 conn(0x7f0d9c0cc4b0 msgr2=0x7f0d9c0b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 shutdown_connections 2024-05-09T06:39:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.265+0000 7f0da199e640 1 -- 172.21.15.1:0/1311562377 wait complete. 2024-05-09T06:39:35.256 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T06:39:35.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:35.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.385+0000 7f824db32640 1 Processor -- start 2024-05-09T06:39:35.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.385+0000 7f824db32640 1 -- start start 2024-05-09T06:39:35.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82481620b0 0x7f8248162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f824815ef10 0x7f824806e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f824806eac0 0x7f82480cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f824806eef0 con 0x7f824806eac0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f82480b3e30 con 0x7f824815ef10 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f824db32640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82480b40b0 con 0x7f82481620b0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f824806eac0 0x7f82480cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f824806eac0 0x7f82480cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59952/0 (socket says 172.21.15.1:59952) 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 -- 172.21.15.1:0/2265555078 learned_addr learned my addr 172.21.15.1:0/2265555078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8246ffd640 1 --2- 172.21.15.1:0/2265555078 >> v2:172.21.15.6:3300/0 conn(0x7f824815ef10 0x7f824806e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f82477fe640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82481620b0 0x7f8248162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 -- 172.21.15.1:0/2265555078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82481620b0 msgr2=0x7f8248162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82481620b0 0x7f8248162490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 -- 172.21.15.1:0/2265555078 >> v2:172.21.15.6:3300/0 conn(0x7f824815ef10 msgr2=0x7f824806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 --2- 172.21.15.1:0/2265555078 >> v2:172.21.15.6:3300/0 conn(0x7f824815ef10 0x7f824806e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.386+0000 7f8247fff640 1 -- 172.21.15.1:0/2265555078 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82480b4370 con 0x7f824806eac0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.387+0000 7f8247fff640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f824806eac0 0x7f82480cf020 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f8248068b20 tx=0x7f824002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df342742352dd62a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.387+0000 7f82467fc640 1 -- 172.21.15.1:0/2265555078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240036070 con 0x7f824806eac0 2024-05-09T06:39:35.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.387+0000 7f82467fc640 1 -- 172.21.15.1:0/2265555078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240031600 con 0x7f824806eac0 2024-05-09T06:39:35.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.387+0000 7f82467fc640 1 -- 172.21.15.1:0/2265555078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400319d0 con 0x7f824806eac0 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 -- 172.21.15.1:0/2265555078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f824806eac0 msgr2=0x7f82480cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f824806eac0 0x7f82480cf020 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f8248068b20 tx=0x7f824002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 -- 172.21.15.1:0/2265555078 shutdown_connections 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f824806eac0 0x7f82480cf020 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 --2- 172.21.15.1:0/2265555078 >> v2:172.21.15.6:3300/0 conn(0x7f824815ef10 0x7f824806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 --2- 172.21.15.1:0/2265555078 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82481620b0 0x7f8248162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 -- 172.21.15.1:0/2265555078 >> 172.21.15.1:0/2265555078 conn(0x7f824806bd00 msgr2=0x7f82480b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 -- 172.21.15.1:0/2265555078 shutdown_connections 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.388+0000 7f824db32640 1 -- 172.21.15.1:0/2265555078 wait complete. 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.389+0000 7f824db32640 1 Processor -- start 2024-05-09T06:39:35.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.389+0000 7f824db32640 1 -- start start 2024-05-09T06:39:35.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f824806eac0 0x7f8248141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f824815ef10 0x7f82480afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 0x7f8248143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82480b4c10 con 0x7f82481620b0 2024-05-09T06:39:35.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82480b4930 con 0x7f824806eac0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f824db32640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f82481629d0 con 0x7f824815ef10 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f8247fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 0x7f8248143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f8247fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 0x7f8248143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59964/0 (socket says 172.21.15.1:59964) 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f8247fff640 1 -- 172.21.15.1:0/2328536113 learned_addr learned my addr 172.21.15.1:0/2328536113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f82477fe640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f824806eac0 0x7f8248141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.390+0000 7f8246ffd640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f824815ef10 0x7f82480afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 -- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f824815ef10 msgr2=0x7f82480afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f824815ef10 0x7f82480afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 -- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f824806eac0 msgr2=0x7f8248141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f824806eac0 0x7f8248141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481447f0 con 0x7f82481620b0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f8247fff640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 0x7f8248143da0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f8240009f40 tx=0x7f8240004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240036050 con 0x7f82481620b0 2024-05-09T06:39:35.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240042070 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481461b0 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240009510 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.391+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8248146770 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.392+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82480b5130 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8240007bd0 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f823406be60 0x7f823406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f82400ac4a0 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f82477fe640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f823406be60 0x7f823406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f8234072440 0x7f8234074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8240006b90 con 0x7f8234072440 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f82400ac930 con 0x7f82481620b0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.393+0000 7f8246ffd640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f8234072440 0x7f8234074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.394+0000 7f82477fe640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f823406be60 0x7f823406e320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f82480b1ea0 tx=0x7f8238008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.394+0000 7f8246ffd640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f8234072440 0x7f8234074880 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.394+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== osd.2 v2:172.21.15.1:6808/1632480843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8240006b90 con 0x7f8234072440 2024-05-09T06:39:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 --> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8248075320 con 0x7f8234072440 2024-05-09T06:39:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f822ffff640 1 -- 172.21.15.1:0/2328536113 <== osd.2 v2:172.21.15.1:6808/1632480843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8248075320 con 0x7f8234072440 2024-05-09T06:39:35.388 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:35.388 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f8234072440 msgr2=0x7f8234074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6808/1632480843,v1:172.21.15.1:6809/1632480843] conn(0x7f8234072440 0x7f8234074880 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f823406be60 msgr2=0x7f823406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f823406be60 0x7f823406e320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f82480b1ea0 tx=0x7f8238008040 comp rx=0 tx=0).stop 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 msgr2=0x7f8248143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 --2- 172.21.15.1:0/2328536113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481620b0 0x7f8248143da0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f8240009f40 tx=0x7f8240004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f8247fff640 1 -- 172.21.15.1:0/2328536113 reap_dead start 2024-05-09T06:39:35.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 shutdown_connections 2024-05-09T06:39:35.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 >> 172.21.15.1:0/2328536113 conn(0x7f824806bd00 msgr2=0x7f8248158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 shutdown_connections 2024-05-09T06:39:35.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.408+0000 7f824db32640 1 -- 172.21.15.1:0/2328536113 wait complete. 2024-05-09T06:39:35.399 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T06:39:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.523+0000 7f758814c640 1 Processor -- start 2024-05-09T06:39:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.523+0000 7f758814c640 1 -- start start 2024-05-09T06:39:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f758814c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 0x7f758006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f758814c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 0x7f75800b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f758814c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75800ce0e0 con 0x7f75800cf180 2024-05-09T06:39:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f758814c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f75800ce3c0 con 0x7f75800b3300 2024-05-09T06:39:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f758814c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75800ce6a0 con 0x7f758006e6d0 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f7585ee8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 0x7f758006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f75866e9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 0x7f75800b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f75856e7640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f75800cf180 0x7f75800cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:45498/0 (socket says 172.21.15.1:45498) 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.524+0000 7f75856e7640 1 -- 172.21.15.1:0/652116454 learned_addr learned my addr 172.21.15.1:0/652116454 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7584ee6640 1 -- 172.21.15.1:0/652116454 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015106025 0 0) 0x7f75800ce0e0 con 0x7f75800cf180 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7584ee6640 1 -- 172.21.15.1:0/652116454 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f756c003520 con 0x7f75800cf180 2024-05-09T06:39:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7585ee8640 1 -- 172.21.15.1:0/652116454 >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 msgr2=0x7f75800b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7585ee8640 1 --2- 172.21.15.1:0/652116454 >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 0x7f75800b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7585ee8640 1 -- 172.21.15.1:0/652116454 >> v1:172.21.15.6:6789/0 conn(0x7f75800cf180 legacy=0x7f75800cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f7585ee8640 1 -- 172.21.15.1:0/652116454 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75800ce980 con 0x7f758006e6d0 2024-05-09T06:39:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.525+0000 7f75866e9640 1 --2- 172.21.15.1:0/652116454 >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 0x7f75800b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f7585ee8640 1 --2- 172.21.15.1:0/652116454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 0x7f758006eab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83842d08a77d284 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f7584ee6640 1 -- 172.21.15.1:0/652116454 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c037070 con 0x7f758006e6d0 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f7584ee6640 1 -- 172.21.15.1:0/652116454 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c032440 con 0x7f758006e6d0 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f7584ee6640 1 -- 172.21.15.1:0/652116454 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0327f0 con 0x7f758006e6d0 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 -- 172.21.15.1:0/652116454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 msgr2=0x7f758006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 --2- 172.21.15.1:0/652116454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 0x7f758006eab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c02a180 comp rx=0 tx=0).stop 2024-05-09T06:39:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 -- 172.21.15.1:0/652116454 shutdown_connections 2024-05-09T06:39:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 --2- 172.21.15.1:0/652116454 >> v2:172.21.15.6:3300/0 conn(0x7f75800b3300 0x7f75800b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 --2- 172.21.15.1:0/652116454 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f758006e6d0 0x7f758006eab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 -- 172.21.15.1:0/652116454 >> 172.21.15.1:0/652116454 conn(0x7f75800cc4b0 msgr2=0x7f75800cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 -- 172.21.15.1:0/652116454 shutdown_connections 2024-05-09T06:39:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.526+0000 7f758814c640 1 -- 172.21.15.1:0/652116454 wait complete. 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 Processor -- start 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 -- start start 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 0x7f75800b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75800cf180 0x7f758015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7580070ed0 con 0x7f758006e6d0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f758006eff0 con 0x7f75800cf180 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f758814c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f75801447c0 con 0x7f75800b3300 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f7585ee8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.528+0000 7f7585ee8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59968/0 (socket says 172.21.15.1:59968) 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 -- 172.21.15.1:0/3864870641 learned_addr learned my addr 172.21.15.1:0/3864870641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75856e7640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 0x7f75800b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75866e9640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75800cf180 0x7f758015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 -- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 msgr2=0x7f75800b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 0x7f75800b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 -- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75800cf180 msgr2=0x7f758015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75800cf180 0x7f758015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 -- 172.21.15.1:0/3864870641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580140650 con 0x7f758006e6d0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75856e7640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 0x7f75800b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f7585ee8640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f757c02a1c0 tx=0x7f757c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c037040 con 0x7f758006e6d0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c042070 con 0x7f758006e6d0 2024-05-09T06:39:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7580140930 con 0x7f758006e6d0 2024-05-09T06:39:35.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c03b870 con 0x7f758006e6d0 2024-05-09T06:39:35.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.529+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7580140ef0 con 0x7f758006e6d0 2024-05-09T06:39:35.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.530+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75800752b0 con 0x7f758006e6d0 2024-05-09T06:39:35.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.531+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f757c03ba10 con 0x7f758006e6d0 2024-05-09T06:39:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.531+0000 7f75767fc640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.531+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f757c0ab690 con 0x7f758006e6d0 2024-05-09T06:39:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.531+0000 7f75856e7640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.532+0000 7f75856e7640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7570007fd0 tx=0x7f7570009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.536+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f757c07a110 con 0x7f758006e6d0 2024-05-09T06:39:35.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.719+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f758006cf50 con 0x7f756406be30 2024-05-09T06:39:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.719+0000 7f75767fc640 1 -- 172.21.15.1:0/3864870641 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f758006cf50 con 0x7f756406be30 2024-05-09T06:39:35.700 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 msgr2=0x7f756406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7570007fd0 tx=0x7f7570009040 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 msgr2=0x7f758015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f757c02a1c0 tx=0x7f757c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 shutdown_connections 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75800cf180 0x7f758015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75800b3300 0x7f75800b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 --2- 172.21.15.1:0/3864870641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f758006e6d0 0x7f758015a650 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 >> 172.21.15.1:0/3864870641 conn(0x7f75800cc4b0 msgr2=0x7f758013f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 shutdown_connections 2024-05-09T06:39:35.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.721+0000 7f758814c640 1 -- 172.21.15.1:0/3864870641 wait complete. 2024-05-09T06:39:35.712 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T06:39:35.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:35.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.840+0000 7f7b2c48c640 1 Processor -- start 2024-05-09T06:39:35.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.840+0000 7f7b2c48c640 1 -- start start 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2c48c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 0x7f7b2415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2c48c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b240b3820 0x7f7b240ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2c48c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2415d470 con 0x7f7b240ced00 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2c48c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7b240cf160 con 0x7f7b240b3820 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2c48c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b240b3cc0 con 0x7f7b24145880 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2a228640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 0x7f7b2415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b29a27640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b240b3820 0x7f7b240ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2aa29640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7b240ced00 0x7f7b2415fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.1:45500/0 (socket says 172.21.15.1:45500) 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.841+0000 7f7b2aa29640 1 -- 172.21.15.1:0/4100581168 learned_addr learned my addr 172.21.15.1:0/4100581168 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b29226640 1 -- 172.21.15.1:0/4100581168 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347318558 0 0) 0x7f7b2415d470 con 0x7f7b240ced00 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b2a228640 1 -- 172.21.15.1:0/4100581168 >> v2:172.21.15.6:3300/0 conn(0x7f7b240b3820 msgr2=0x7f7b240ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b2a228640 1 --2- 172.21.15.1:0/4100581168 >> v2:172.21.15.6:3300/0 conn(0x7f7b240b3820 0x7f7b240ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b2a228640 1 -- 172.21.15.1:0/4100581168 >> v1:172.21.15.6:6789/0 conn(0x7f7b240ced00 legacy=0x7f7b2415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b2a228640 1 -- 172.21.15.1:0/4100581168 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b240b3fa0 con 0x7f7b24145880 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b2a228640 1 --2- 172.21.15.1:0/4100581168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 0x7f7b2415cf30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0097a0 tx=0x7f7b1c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c44ecb6d14605e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.842+0000 7f7b29226640 1 -- 172.21.15.1:0/4100581168 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c037070 con 0x7f7b24145880 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b29226640 1 -- 172.21.15.1:0/4100581168 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c032440 con 0x7f7b24145880 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b29226640 1 -- 172.21.15.1:0/4100581168 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c032810 con 0x7f7b24145880 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 -- 172.21.15.1:0/4100581168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 msgr2=0x7f7b2415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/4100581168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 0x7f7b2415cf30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0097a0 tx=0x7f7b1c02a180 comp rx=0 tx=0).stop 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 -- 172.21.15.1:0/4100581168 shutdown_connections 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/4100581168 >> v2:172.21.15.6:3300/0 conn(0x7f7b240b3820 0x7f7b240ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/4100581168 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b24145880 0x7f7b2415cf30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 -- 172.21.15.1:0/4100581168 >> 172.21.15.1:0/4100581168 conn(0x7f7b2406b3c0 msgr2=0x7f7b2406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 -- 172.21.15.1:0/4100581168 shutdown_connections 2024-05-09T06:39:35.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.843+0000 7f7b2c48c640 1 -- 172.21.15.1:0/4100581168 wait complete. 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.844+0000 7f7b2c48c640 1 Processor -- start 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.844+0000 7f7b2c48c640 1 -- start start 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 0x7f7b2415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b240ced00 0x7f7b2415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b24145880 0x7f7b2415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b240b4840 con 0x7f7b240b3820 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7b240b4560 con 0x7f7b24145880 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2c48c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f7b240cfee0 con 0x7f7b240ced00 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2a228640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 0x7f7b2415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2a228640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 0x7f7b2415bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59982/0 (socket says 172.21.15.1:59982) 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.845+0000 7f7b2a228640 1 -- 172.21.15.1:0/2838688057 learned_addr learned my addr 172.21.15.1:0/2838688057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2aa29640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b24145880 0x7f7b2415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b29a27640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b240ced00 0x7f7b2415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 -- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b240ced00 msgr2=0x7f7b2415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b240ced00 0x7f7b2415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 -- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b24145880 msgr2=0x7f7b2415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b24145880 0x7f7b2415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2415c4c0 con 0x7f7b240b3820 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2aa29640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b24145880 0x7f7b2415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2a228640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 0x7f7b2415bf80 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c035040 tx=0x7f7b1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c037040 con 0x7f7b240b3820 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c030500 con 0x7f7b240b3820 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b24131b20 con 0x7f7b240b3820 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c030910 con 0x7f7b240b3820 2024-05-09T06:39:35.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.846+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2412f1e0 con 0x7f7b240b3820 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.847+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b24131e00 con 0x7f7b240b3820 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b1c044020 con 0x7f7b240b3820 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b0806be60 0x7f7b0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f7b1c0aafa0 con 0x7f7b240b3820 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b29a27640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b0806be60 0x7f7b0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7f7b08072440 0x7f7b08074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b1c032440 con 0x7f7b08072440 2024-05-09T06:39:35.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f7b1c0b2050 con 0x7f7b240b3820 2024-05-09T06:39:35.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.848+0000 7f7b2aa29640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7f7b08072440 0x7f7b08074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.849+0000 7f7b29a27640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b0806be60 0x7f7b0806e320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7b2406bc20 tx=0x7f7b14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.849+0000 7f7b2aa29640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7f7b08072440 0x7f7b08074880 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.849+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== osd.3 v2:172.21.15.1:6800/2131549113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b1c032440 con 0x7f7b08072440 2024-05-09T06:39:35.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 --> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7b24075320 con 0x7f7b08072440 2024-05-09T06:39:35.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b1affd640 1 -- 172.21.15.1:0/2838688057 <== osd.3 v2:172.21.15.1:6800/2131549113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7b24075320 con 0x7f7b08072440 2024-05-09T06:39:35.843 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:35.843 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:35.843 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:35.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7f7b08072440 msgr2=0x7f7b08074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6800/2131549113,v1:172.21.15.1:6801/2131549113] conn(0x7f7b08072440 0x7f7b08074880 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b0806be60 msgr2=0x7f7b0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f7b0806be60 0x7f7b0806e320 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7b2406bc20 tx=0x7f7b14008040 comp rx=0 tx=0).stop 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 msgr2=0x7f7b2415bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 --2- 172.21.15.1:0/2838688057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b240b3820 0x7f7b2415bf80 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c035040 tx=0x7f7b1c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2aa29640 1 -- 172.21.15.1:0/2838688057 reap_dead start 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 shutdown_connections 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 >> 172.21.15.1:0/2838688057 conn(0x7f7b2406b3c0 msgr2=0x7f7b24070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 shutdown_connections 2024-05-09T06:39:35.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.863+0000 7f7b2c48c640 1 -- 172.21.15.1:0/2838688057 wait complete. 2024-05-09T06:39:35.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T06:39:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.978+0000 7f050a9e7640 1 Processor -- start 2024-05-09T06:39:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.978+0000 7f050a9e7640 1 -- start start 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f050a9e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0504161950 0x7f05040d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f050a9e7640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05040d4940 0x7f05040d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f050a9e7640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05040b33c0 con 0x7f05040d0f80 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f050a9e7640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f05040d4dd0 con 0x7f05040d4940 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f050a9e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05040b3660 con 0x7f0504161950 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.979+0000 7f05037fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05040d4940 0x7f05040d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f05040d4940 0x7f05040d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59996/0 (socket says 172.21.15.1:59996) 2024-05-09T06:39:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f0508f84640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f05040d0f80 0x7f05040cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57614/0 (socket says 172.21.15.1:57614) 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 -- 172.21.15.1:0/104176511 learned_addr learned my addr 172.21.15.1:0/104176511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f0503fff640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0504161950 0x7f05040d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 -- 172.21.15.1:0/104176511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0504161950 msgr2=0x7f05040d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0504161950 0x7f05040d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 -- 172.21.15.1:0/104176511 >> v1:172.21.15.1:6789/0 conn(0x7f05040d0f80 legacy=0x7f05040cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f0502ffd640 1 -- 172.21.15.1:0/104176511 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262468209 0 0) 0x7f05040b33c0 con 0x7f05040d0f80 2024-05-09T06:39:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.980+0000 7f05037fe640 1 -- 172.21.15.1:0/104176511 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05040b3940 con 0x7f05040d4940 2024-05-09T06:39:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f05037fe640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05040d4940 0x7f05040d2940 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f05040b3590 tx=0x7f04f802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d2ea9087022585 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f0502ffd640 1 -- 172.21.15.1:0/104176511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8037070 con 0x7f05040d4940 2024-05-09T06:39:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f0502ffd640 1 -- 172.21.15.1:0/104176511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8032440 con 0x7f05040d4940 2024-05-09T06:39:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f0502ffd640 1 -- 172.21.15.1:0/104176511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8032810 con 0x7f05040d4940 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 -- 172.21.15.1:0/104176511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05040d4940 msgr2=0x7f05040d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05040d4940 0x7f05040d2940 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f05040b3590 tx=0x7f04f802a250 comp rx=0 tx=0).stop 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 -- 172.21.15.1:0/104176511 shutdown_connections 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05040d4940 0x7f05040d2940 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 --2- 172.21.15.1:0/104176511 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0504161950 0x7f05040d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 -- 172.21.15.1:0/104176511 >> 172.21.15.1:0/104176511 conn(0x7f05040d3e20 msgr2=0x7f05040d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 -- 172.21.15.1:0/104176511 shutdown_connections 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.981+0000 7f050a9e7640 1 -- 172.21.15.1:0/104176511 wait complete. 2024-05-09T06:39:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.982+0000 7f050a9e7640 1 Processor -- start 2024-05-09T06:39:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 -- start start 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05040d0f80 0x7f05041594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05040cb900 0x7f05040cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05040b52d0 con 0x7f0504161950 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f05040b5150 con 0x7f05040d0f80 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.983+0000 7f050a9e7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f05040b5450 con 0x7f05040cb900 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60010/0 (socket says 172.21.15.1:60010) 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 -- 172.21.15.1:0/1295482840 learned_addr learned my addr 172.21.15.1:0/1295482840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f0508f84640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05040cb900 0x7f05040cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f0503fff640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05040d0f80 0x7f05041594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 -- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05040cb900 msgr2=0x7f05040cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05040cb900 0x7f05040cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 -- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05040d0f80 msgr2=0x7f05041594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05040d0f80 0x7f05041594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 -- 172.21.15.1:0/1295482840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05040cc910 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f05037fe640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f04f8000c00 tx=0x7f04f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8041020 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8030640 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05040cc070 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8037040 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.984+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05040cc630 con 0x7f0504161950 2024-05-09T06:39:35.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.985+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05041599e0 con 0x7f0504161950 2024-05-09T06:39:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.986+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04f8044020 con 0x7f0504161950 2024-05-09T06:39:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.986+0000 7f0500ff9640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 0x7f04f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.986+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f04f80ab3b0 con 0x7f0504161950 2024-05-09T06:39:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.986+0000 7f0503fff640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 0x7f04f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:35.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.987+0000 7f0503fff640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 0x7f04f006e320 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009aa0 tx=0x7f04f4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:35.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:35.991+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05041599e0 con 0x7f0504161950 2024-05-09T06:39:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.179+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f05040752b0 con 0x7f04f006be60 2024-05-09T06:39:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.179+0000 7f0500ff9640 1 -- 172.21.15.1:0/1295482840 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05040752b0 con 0x7f04f006be60 2024-05-09T06:39:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 msgr2=0x7f04f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 0x7f04f006e320 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009aa0 tx=0x7f04f4009190 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 msgr2=0x7f05040b8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f04f8000c00 tx=0x7f04f80046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 shutdown_connections 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f04f006be60 0x7f04f006e320 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05040cb900 0x7f05040cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504161950 0x7f05040b8d80 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 --2- 172.21.15.1:0/1295482840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05040d0f80 0x7f05041594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 >> 172.21.15.1:0/1295482840 conn(0x7f05040d3e20 msgr2=0x7f05040b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 shutdown_connections 2024-05-09T06:39:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.181+0000 7f050a9e7640 1 -- 172.21.15.1:0/1295482840 wait complete. 2024-05-09T06:39:36.174 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T06:39:36.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:36.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.301+0000 7f88e7a2a640 1 Processor -- start 2024-05-09T06:39:36.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 -- start start 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88e00b2ed0 0x7f88e00ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f88e00b2110 0x7f88e00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e00d1cc0 con 0x7f88e00d2630 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f88e00d4d10 con 0x7f88e00b2ed0 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.302+0000 7f88e7a2a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f88e00d1f80 con 0x7f88e00b2110 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88e00b2ed0 0x7f88e00ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88e00b2ed0 0x7f88e00ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60020/0 (socket says 172.21.15.1:60020) 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 -- 172.21.15.1:0/2886482054 learned_addr learned my addr 172.21.15.1:0/2886482054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e5fc7640 1 --2- 172.21.15.1:0/2886482054 >> v2:172.21.15.6:3300/0 conn(0x7f88e00b2110 0x7f88e00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 -- 172.21.15.1:0/2886482054 >> v2:172.21.15.6:3300/0 conn(0x7f88e00b2110 msgr2=0x7f88e00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 --2- 172.21.15.1:0/2886482054 >> v2:172.21.15.6:3300/0 conn(0x7f88e00b2110 0x7f88e00cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 -- 172.21.15.1:0/2886482054 >> v1:172.21.15.6:6789/0 conn(0x7f88e00d2630 legacy=0x7f88e015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.303+0000 7f88e4fc5640 1 -- 172.21.15.1:0/2886482054 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e00b0c00 con 0x7f88e00b2ed0 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e4fc5640 1 --2- 172.21.15.1:0/2886482054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e00ceba0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f88e008e0e0 tx=0x7f88d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b7df167e4fa9fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88d7fff640 1 -- 172.21.15.1:0/2886482054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0037070 con 0x7f88e00b2ed0 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88d7fff640 1 -- 172.21.15.1:0/2886482054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d0032440 con 0x7f88e00b2ed0 2024-05-09T06:39:36.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88d7fff640 1 -- 172.21.15.1:0/2886482054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0032810 con 0x7f88e00b2ed0 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 -- 172.21.15.1:0/2886482054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 msgr2=0x7f88e00ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/2886482054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e00ceba0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f88e008e0e0 tx=0x7f88d002a270 comp rx=0 tx=0).stop 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 -- 172.21.15.1:0/2886482054 shutdown_connections 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/2886482054 >> v2:172.21.15.6:3300/0 conn(0x7f88e00b2110 0x7f88e00cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/2886482054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e00ceba0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.304+0000 7f88e7a2a640 1 -- 172.21.15.1:0/2886482054 >> 172.21.15.1:0/2886482054 conn(0x7f88e00d3ee0 msgr2=0x7f88e00d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.305+0000 7f88e7a2a640 1 -- 172.21.15.1:0/2886482054 shutdown_connections 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.305+0000 7f88e7a2a640 1 -- 172.21.15.1:0/2886482054 wait complete. 2024-05-09T06:39:36.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.306+0000 7f88e7a2a640 1 Processor -- start 2024-05-09T06:39:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.306+0000 7f88e7a2a640 1 -- start start 2024-05-09T06:39:36.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.306+0000 7f88e7a2a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88e00b2110 0x7f88e014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.306+0000 7f88e7a2a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e7a2a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88e00d2630 0x7f88e015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e7a2a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88e00d2b90 con 0x7f88e00b2ed0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e7a2a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88e00d2a10 con 0x7f88e00b2110 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e7a2a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88e00d2d10 con 0x7f88e00d2630 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e014dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60026/0 (socket says 172.21.15.1:60026) 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 -- 172.21.15.1:0/335083337 learned_addr learned my addr 172.21.15.1:0/335083337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e57c6640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88e00b2110 0x7f88e014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e5fc7640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88e00d2630 0x7f88e015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 -- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88e00d2630 msgr2=0x7f88e015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88e00d2630 0x7f88e015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 -- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88e00b2110 msgr2=0x7f88e014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88e00b2110 0x7f88e014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e015e310 con 0x7f88e00b2ed0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e4fc5640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e014dbf0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f88d002a2b0 tx=0x7f88d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0041020 con 0x7f88e00b2ed0 2024-05-09T06:39:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d00306c0 con 0x7f88e00b2ed0 2024-05-09T06:39:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e015fce0 con 0x7f88e00b2ed0 2024-05-09T06:39:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.307+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0037040 con 0x7f88e00b2ed0 2024-05-09T06:39:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.308+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e01602a0 con 0x7f88e00b2ed0 2024-05-09T06:39:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.308+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88e015e5b0 con 0x7f88e00b2ed0 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.309+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88d0044020 con 0x7f88e00b2ed0 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88d5ffb640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88c406be30 0x7f88c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f88d00ab550 con 0x7f88e00b2ed0 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88e57c6640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88c406be30 0x7f88c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88d5ffb640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f88c4072410 0x7f88c4074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88d002cea0 con 0x7f88c4072410 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f88d00b2050 con 0x7f88e00b2ed0 2024-05-09T06:39:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88e57c6640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88c406be30 0x7f88c406e2f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007fd0 tx=0x7f88dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.310+0000 7f88e5fc7640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f88c4072410 0x7f88c4074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.311+0000 7f88e5fc7640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f88c4072410 0x7f88c4074850 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.311+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== osd.4 v2:172.21.15.6:6800/3928343737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88d002cea0 con 0x7f88c4072410 2024-05-09T06:39:36.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 --> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f88e00d3a90 con 0x7f88c4072410 2024-05-09T06:39:36.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88d5ffb640 1 -- 172.21.15.1:0/335083337 <== osd.4 v2:172.21.15.6:6800/3928343737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f88e00d3a90 con 0x7f88c4072410 2024-05-09T06:39:36.305 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:36.306 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f88c4072410 msgr2=0x7f88c4074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.6:6800/3928343737,v1:172.21.15.6:6802/3928343737] conn(0x7f88c4072410 0x7f88c4074850 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88c406be30 msgr2=0x7f88c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f88c406be30 0x7f88c406e2f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007fd0 tx=0x7f88dc009040 comp rx=0 tx=0).stop 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 msgr2=0x7f88e014dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 --2- 172.21.15.1:0/335083337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88e00b2ed0 0x7f88e014dbf0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f88d002a2b0 tx=0x7f88d00046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e5fc7640 1 -- 172.21.15.1:0/335083337 reap_dead start 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.325+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 shutdown_connections 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.326+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 >> 172.21.15.1:0/335083337 conn(0x7f88e00d3ee0 msgr2=0x7f88e00b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.326+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 shutdown_connections 2024-05-09T06:39:36.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.326+0000 7f88e7a2a640 1 -- 172.21.15.1:0/335083337 wait complete. 2024-05-09T06:39:36.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T06:39:36.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.441+0000 7f4d692b3640 1 Processor -- start 2024-05-09T06:39:36.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 -- start start 2024-05-09T06:39:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4d64161950 0x7f4d640d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d640d4940 0x7f4d640d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4d640d0f80 0x7f4d640cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4d640d4dd0 con 0x7f4d640d0f80 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4d640b33c0 con 0x7f4d64161950 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d692b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640b3660 con 0x7f4d640d4940 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d637fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4d640d0f80 0x7f4d640cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.442+0000 7f4d637fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4d640d0f80 0x7f4d640cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60030/0 (socket says 172.21.15.1:60030) 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 -- 172.21.15.1:0/2992516214 learned_addr learned my addr 172.21.15.1:0/2992516214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d627fc640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d640d4940 0x7f4d640d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d62ffd640 1 --2- 172.21.15.1:0/2992516214 >> v2:172.21.15.6:3300/0 conn(0x7f4d64161950 0x7f4d640d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 -- 172.21.15.1:0/2992516214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d640d4940 msgr2=0x7f4d640d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d640d4940 0x7f4d640d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 -- 172.21.15.1:0/2992516214 >> v2:172.21.15.6:3300/0 conn(0x7f4d64161950 msgr2=0x7f4d640d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 --2- 172.21.15.1:0/2992516214 >> v2:172.21.15.6:3300/0 conn(0x7f4d64161950 0x7f4d640d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.443+0000 7f4d637fe640 1 -- 172.21.15.1:0/2992516214 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d640b3940 con 0x7f4d640d0f80 2024-05-09T06:39:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d637fe640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d640d0f80 0x7f4d640cea30 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406b7f0 tx=0x7f4d5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b975bd15dbcdb5c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d61ffb640 1 -- 172.21.15.1:0/2992516214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54037070 con 0x7f4d640d0f80 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d61ffb640 1 -- 172.21.15.1:0/2992516214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d54032440 con 0x7f4d640d0f80 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d61ffb640 1 -- 172.21.15.1:0/2992516214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54032810 con 0x7f4d640d0f80 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 -- 172.21.15.1:0/2992516214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d640d0f80 msgr2=0x7f4d640cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d640d0f80 0x7f4d640cea30 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406b7f0 tx=0x7f4d5402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 -- 172.21.15.1:0/2992516214 shutdown_connections 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d640d0f80 0x7f4d640cea30 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 --2- 172.21.15.1:0/2992516214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d640d4940 0x7f4d640d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 --2- 172.21.15.1:0/2992516214 >> v2:172.21.15.6:3300/0 conn(0x7f4d64161950 0x7f4d640d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 -- 172.21.15.1:0/2992516214 >> 172.21.15.1:0/2992516214 conn(0x7f4d640d3e20 msgr2=0x7f4d640d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 -- 172.21.15.1:0/2992516214 shutdown_connections 2024-05-09T06:39:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.444+0000 7f4d692b3640 1 -- 172.21.15.1:0/2992516214 wait complete. 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 Processor -- start 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 -- start start 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d640d0f80 0x7f4d6415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d64161950 0x7f4d6409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640d1d80 con 0x7f4d6409dfe0 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640d1c00 con 0x7f4d640d0f80 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.446+0000 7f4d692b3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640d1f00 con 0x7f4d64161950 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60044/0 (socket says 172.21.15.1:60044) 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 -- 172.21.15.1:0/1592729483 learned_addr learned my addr 172.21.15.1:0/1592729483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d62ffd640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d640d0f80 0x7f4d6415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d627fc640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d64161950 0x7f4d6409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 -- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d64161950 msgr2=0x7f4d6409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d64161950 0x7f4d6409daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 -- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d640d0f80 msgr2=0x7f4d6415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d640d0f80 0x7f4d6415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 -- 172.21.15.1:0/1592729483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d640cbd40 con 0x7f4d6409dfe0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d637fe640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f4d540090b0 tx=0x7f4d540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54037040 con 0x7f4d6409dfe0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d54042070 con 0x7f4d6409dfe0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d640cc8b0 con 0x7f4d6409dfe0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54030920 con 0x7f4d6409dfe0 2024-05-09T06:39:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.447+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d640cce70 con 0x7f4d6409dfe0 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.448+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d640cd070 con 0x7f4d6409dfe0 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.449+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d54030ac0 con 0x7f4d6409dfe0 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.449+0000 7f4d5b7fe640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 0x7f4d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.449+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f4d540ab470 con 0x7f4d6409dfe0 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.449+0000 7f4d62ffd640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 0x7f4d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.450+0000 7f4d62ffd640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 0x7f4d4406e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0039e0 tx=0x7f4d5c009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.454+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4d640cd070 con 0x7f4d6409dfe0 2024-05-09T06:39:36.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.635+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f4d640752b0 con 0x7f4d4406be30 2024-05-09T06:39:36.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.635+0000 7f4d5b7fe640 1 -- 172.21.15.1:0/1592729483 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4d640752b0 con 0x7f4d4406be30 2024-05-09T06:39:36.616 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T06:39:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 msgr2=0x7f4d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 0x7f4d4406e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0039e0 tx=0x7f4d5c009830 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 msgr2=0x7f4d640cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f4d540090b0 tx=0x7f4d540046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 shutdown_connections 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4d4406be30 0x7f4d4406e2f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d6409dfe0 0x7f4d640cef90 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d64161950 0x7f4d6409daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 --2- 172.21.15.1:0/1592729483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d640d0f80 0x7f4d6415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 >> 172.21.15.1:0/1592729483 conn(0x7f4d640d3e20 msgr2=0x7f4d640b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 shutdown_connections 2024-05-09T06:39:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.637+0000 7f4d692b3640 1 -- 172.21.15.1:0/1592729483 wait complete. 2024-05-09T06:39:36.628 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T06:39:36.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.755+0000 7f900861b640 1 Processor -- start 2024-05-09T06:39:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.755+0000 7f900861b640 1 -- start start 2024-05-09T06:39:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f900861b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f90000cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f900861b640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900015f780 con 0x7f900006e320 2024-05-09T06:39:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f900861b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90000cdd50 con 0x7f90001446d0 2024-05-09T06:39:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f900861b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90000ce8d0 con 0x7f9000143bb0 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f9006bb8640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f900006e320 0x7f900015f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57630/0 (socket says 172.21.15.1:57630) 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.756+0000 7f9006bb8640 1 -- 172.21.15.1:0/2872093609 learned_addr learned my addr 172.21.15.1:0/2872093609 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90063b7640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f90000cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274038014 0 0) 0x7f900015f780 con 0x7f900006e320 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fe8003520 con 0x7f900006e320 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761618262 0 0) 0x7f8fe8003520 con 0x7f900006e320 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 msgr2=0x7f90000cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f90000cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 >> v1:172.21.15.6:6789/0 conn(0x7f90001446d0 legacy=0x7f900015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90000ce030 con 0x7f900006e320 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8ff00019f0 con 0x7f900006e320 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 >> v1:172.21.15.1:6789/0 conn(0x7f900006e320 legacy=0x7f900015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fe80039e0 0x7f8fe8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f8fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f8fe8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90000ce8d0 con 0x7f8fe8004800 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe800bb80 con 0x7f8fe80039e0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe800bd80 con 0x7f9000143bb0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f8fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f90063b7640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fe80039e0 0x7f8fe8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9006bb8640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f8fe8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 -- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 msgr2=0x7f8fe8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f8fe8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 -- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fe80039e0 msgr2=0x7f8fe8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8fe80039e0 0x7f8fe8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9005bb6640 1 -- 172.21.15.1:0/2872093609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe8003520 con 0x7f8fe8004800 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.757+0000 7f9006bb8640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9000143bb0 0x7f8fe8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.758+0000 7f9005bb6640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f8fe8004c80 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4001750 tx=0x7f8fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354d1adf5d0add14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.758+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4001370 con 0x7f8fe8004800 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.758+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff40052d0 con 0x7f8fe8004800 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.758+0000 7f90053b5640 1 -- 172.21.15.1:0/2872093609 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff402abe0 con 0x7f8fe8004800 2024-05-09T06:39:36.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 -- 172.21.15.1:0/2872093609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 msgr2=0x7f8fe8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 --2- 172.21.15.1:0/2872093609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f8fe8004c80 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4001750 tx=0x7f8fe8005830 comp rx=0 tx=0).stop 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f9006bb8640 1 -- 172.21.15.1:0/2872093609 reap_dead start 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 -- 172.21.15.1:0/2872093609 shutdown_connections 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 -- 172.21.15.1:0/2872093609 >> 172.21.15.1:0/2872093609 conn(0x7f900006b3c0 msgr2=0x7f900006b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 -- 172.21.15.1:0/2872093609 shutdown_connections 2024-05-09T06:39:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.759+0000 7f900861b640 1 -- 172.21.15.1:0/2872093609 wait complete. 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.760+0000 7f900861b640 1 Processor -- start 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.760+0000 7f900861b640 1 -- start start 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f9000151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90001425b0 0x7f900013cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9000142a10 0x7f900013d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90000cd970 con 0x7f8fe8004800 2024-05-09T06:39:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f90000ce5f0 con 0x7f9000142a10 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f900861b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f90000cdaf0 con 0x7f90001425b0 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f9000151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f9000151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60066/0 (socket says 172.21.15.1:60066) 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 -- 172.21.15.1:0/3153925385 learned_addr learned my addr 172.21.15.1:0/3153925385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f9005bb6640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90001425b0 0x7f900013cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f9006bb8640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9000142a10 0x7f900013d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 -- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90001425b0 msgr2=0x7f900013cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90001425b0 0x7f900013cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 -- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9000142a10 msgr2=0x7f900013d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9000142a10 0x7f900013d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f90063b7640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90001407f0 con 0x7f8fe8004800 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f9005bb6640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f90001425b0 0x7f900013cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:36.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.761+0000 7f9006bb8640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9000142a10 0x7f900013d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f90063b7640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f9000151310 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80119a0 tx=0x7f8ff8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800faf0 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff800fc90 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8014440 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9000140ad0 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.762+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9000141090 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.763+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9000075320 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.763+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ff8012040 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f8ffeffd640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8fec06be60 0x7f8fec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f8ff808b3d0 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f9005bb6640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8fec06be60 0x7f8fec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f8ffeffd640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7f8fec072440 0x7f8fec074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fec074f50 con 0x7f8fec072440 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f8ff8092050 con 0x7f8fe8004800 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f9006bb8640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7f8fec072440 0x7f8fec074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.764+0000 7f9005bb6640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8fec06be60 0x7f8fec06e320 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4032000 tx=0x7f8ff402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.765+0000 7f9006bb8640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7f8fec072440 0x7f8fec074880 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.766+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== osd.5 v2:172.21.15.6:6801/2308051940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fec074f50 con 0x7f8fec072440 2024-05-09T06:39:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.779+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 --> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f90000cf6b0 con 0x7f8fec072440 2024-05-09T06:39:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.779+0000 7f8ffeffd640 1 -- 172.21.15.1:0/3153925385 <== osd.5 v2:172.21.15.6:6801/2308051940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f90000cf6b0 con 0x7f8fec072440 2024-05-09T06:39:36.760 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:36.760 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:36.760 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7f8fec072440 msgr2=0x7f8fec074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.6:6801/2308051940,v1:172.21.15.6:6803/2308051940] conn(0x7f8fec072440 0x7f8fec074880 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8fec06be60 msgr2=0x7f8fec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f8fec06be60 0x7f8fec06e320 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4032000 tx=0x7f8ff402e000 comp rx=0 tx=0).stop 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 msgr2=0x7f9000151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 --2- 172.21.15.1:0/3153925385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe8004800 0x7f9000151310 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80119a0 tx=0x7f8ff8011e60 comp rx=0 tx=0).stop 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f9006bb8640 1 -- 172.21.15.1:0/3153925385 reap_dead start 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 shutdown_connections 2024-05-09T06:39:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 >> 172.21.15.1:0/3153925385 conn(0x7f900006b3c0 msgr2=0x7f90000b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 shutdown_connections 2024-05-09T06:39:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.780+0000 7f900861b640 1 -- 172.21.15.1:0/3153925385 wait complete. 2024-05-09T06:39:36.770 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T06:39:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.895+0000 7fe1a9a64640 1 Processor -- start 2024-05-09T06:39:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.895+0000 7fe1a9a64640 1 -- start start 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a9a64640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 0x7fe1a40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a9a64640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1a40d0f80 0x7fe1a40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a9a64640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a40b33c0 con 0x7fe1a40d4940 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a9a64640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a40d4dd0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a9a64640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a40b3660 con 0x7fe1a4161950 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1a40d0f80 0x7fe1a40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1a40d0f80 0x7fe1a40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60072/0 (socket says 172.21.15.1:60072) 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.896+0000 7fe1a3fff640 1 -- 172.21.15.1:0/352730177 learned_addr learned my addr 172.21.15.1:0/352730177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a37fe640 1 --2- 172.21.15.1:0/352730177 >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 0x7fe1a40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a3fff640 1 -- 172.21.15.1:0/352730177 >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 msgr2=0x7fe1a40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a3fff640 1 --2- 172.21.15.1:0/352730177 >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 0x7fe1a40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a3fff640 1 -- 172.21.15.1:0/352730177 >> v1:172.21.15.1:6789/0 conn(0x7fe1a40d4940 legacy=0x7fe1a40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a27fc640 1 -- 172.21.15.1:0/352730177 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636337301 0 0) 0x7fe1a40b33c0 con 0x7fe1a40d4940 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a3fff640 1 -- 172.21.15.1:0/352730177 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a40b3940 con 0x7fe1a40d0f80 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.897+0000 7fe1a37fe640 1 --2- 172.21.15.1:0/352730177 >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 0x7fe1a40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a3fff640 1 --2- 172.21.15.1:0/352730177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40d2940 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406b7f0 tx=0x7fe19c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06cbb86a366ca89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a27fc640 1 -- 172.21.15.1:0/352730177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c036070 con 0x7fe1a40d0f80 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a27fc640 1 -- 172.21.15.1:0/352730177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19c031600 con 0x7fe1a40d0f80 2024-05-09T06:39:36.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a27fc640 1 -- 172.21.15.1:0/352730177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c0319d0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 -- 172.21.15.1:0/352730177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 msgr2=0x7fe1a40d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/352730177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40d2940 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406b7f0 tx=0x7fe19c02a730 comp rx=0 tx=0).stop 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 -- 172.21.15.1:0/352730177 shutdown_connections 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/352730177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40d2940 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/352730177 >> v2:172.21.15.6:3300/0 conn(0x7fe1a4161950 0x7fe1a40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 -- 172.21.15.1:0/352730177 >> 172.21.15.1:0/352730177 conn(0x7fe1a40d3e20 msgr2=0x7fe1a40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:36.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 -- 172.21.15.1:0/352730177 shutdown_connections 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.898+0000 7fe1a9a64640 1 -- 172.21.15.1:0/352730177 wait complete. 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.899+0000 7fe1a9a64640 1 Processor -- start 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 -- start start 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1a4161950 0x7fe1a4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1a4153310 0x7fe1a41f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a40b52d0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a40b5150 con 0x7fe1a4161950 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.900+0000 7fe1a9a64640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a40b5450 con 0x7fe1a4153310 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60074/0 (socket says 172.21.15.1:60074) 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 -- 172.21.15.1:0/312691097 learned_addr learned my addr 172.21.15.1:0/312691097 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a3fff640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1a4153310 0x7fe1a41f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a2ffd640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1a4161950 0x7fe1a4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 -- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1a4153310 msgr2=0x7fe1a41f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1a4153310 0x7fe1a41f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 -- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1a4161950 msgr2=0x7fe1a4152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1a4161950 0x7fe1a4152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 -- 172.21.15.1:0/312691097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a41537a0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a37fe640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c820 tx=0x7fe19400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194015070 con 0x7fe1a40d0f80 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe194007a30 con 0x7fe1a40d0f80 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a40b6bb0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194007de0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.901+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a40b7170 con 0x7fe1a40d0f80 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.902+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a40752b0 con 0x7fe1a40d0f80 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.903+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe194018070 con 0x7fe1a40d0f80 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.903+0000 7fe18bfff640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 0x7fe19006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.903+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fe19408ac70 con 0x7fe1a40d0f80 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.903+0000 7fe1a2ffd640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 0x7fe19006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:36.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.904+0000 7fe1a2ffd640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 0x7fe19006e2f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406d480 tx=0x7fe198009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:36.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:36.908+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1940596e0 con 0x7fe1a40d0f80 2024-05-09T06:39:37.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.090+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe1a415f930 con 0x7fe19006be30 2024-05-09T06:39:37.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.090+0000 7fe18bfff640 1 -- 172.21.15.1:0/312691097 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1a415f930 con 0x7fe19006be30 2024-05-09T06:39:37.070 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T06:39:37.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 msgr2=0x7fe19006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 0x7fe19006e2f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406d480 tx=0x7fe198009040 comp rx=0 tx=0).stop 2024-05-09T06:39:37.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 msgr2=0x7fe1a40cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c820 tx=0x7fe19400cce0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 shutdown_connections 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fe19006be30 0x7fe19006e2f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1a4153310 0x7fe1a41f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1a4161950 0x7fe1a4152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 --2- 172.21.15.1:0/312691097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1a40d0f80 0x7fe1a40cd220 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 >> 172.21.15.1:0/312691097 conn(0x7fe1a40d3e20 msgr2=0x7fe1a415ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 shutdown_connections 2024-05-09T06:39:37.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.092+0000 7fe1a9a64640 1 -- 172.21.15.1:0/312691097 wait complete. 2024-05-09T06:39:37.083 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T06:39:37.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.211+0000 7f4e40410640 1 Processor -- start 2024-05-09T06:39:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.211+0000 7f4e40410640 1 -- start start 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e40410640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4e3815e5f0 0x7f4e380b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e40410640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e380ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e40410640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e380cb0e0 con 0x7f4e3815e210 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e40410640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4e380cca50 con 0x7f4e3815e5f0 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e40410640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e380b17c0 con 0x7f4e381628f0 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4e3815e5f0 0x7f4e380b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4e3815e5f0 0x7f4e380b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60084/0 (socket says 172.21.15.1:60084) 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 -- 172.21.15.1:0/185565948 learned_addr learned my addr 172.21.15.1:0/185565948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3e9ad640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e380ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 -- 172.21.15.1:0/185565948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 msgr2=0x7f4e380ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e380ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 -- 172.21.15.1:0/185565948 >> v1:172.21.15.6:6789/0 conn(0x7f4e3815e210 legacy=0x7f4e380ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.212+0000 7f4e3d9ab640 1 -- 172.21.15.1:0/185565948 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e380b1aa0 con 0x7f4e3815e5f0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.213+0000 7f4e3d9ab640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e3815e5f0 0x7f4e380b1170 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009870 tx=0x7f4e2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed35ccfb197b5cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.213+0000 7f4e3d1aa640 1 -- 172.21.15.1:0/185565948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28037070 con 0x7f4e3815e5f0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.213+0000 7f4e3d1aa640 1 -- 172.21.15.1:0/185565948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e28032440 con 0x7f4e3815e5f0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.213+0000 7f4e3d1aa640 1 -- 172.21.15.1:0/185565948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28032810 con 0x7f4e3815e5f0 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 -- 172.21.15.1:0/185565948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e3815e5f0 msgr2=0x7f4e380b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e3815e5f0 0x7f4e380b1170 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009870 tx=0x7f4e2802a210 comp rx=0 tx=0).stop 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 -- 172.21.15.1:0/185565948 shutdown_connections 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e380ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 --2- 172.21.15.1:0/185565948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e3815e5f0 0x7f4e380b1170 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 -- 172.21.15.1:0/185565948 >> 172.21.15.1:0/185565948 conn(0x7f4e38071a50 msgr2=0x7f4e380b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 -- 172.21.15.1:0/185565948 shutdown_connections 2024-05-09T06:39:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.214+0000 7f4e40410640 1 -- 172.21.15.1:0/185565948 wait complete. 2024-05-09T06:39:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.215+0000 7f4e40410640 1 Processor -- start 2024-05-09T06:39:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.215+0000 7f4e40410640 1 -- start start 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e3815e210 0x7f4e380afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e38157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 0x7f4e380c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e380cd420 con 0x7f4e38161720 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38162d70 con 0x7f4e3815e210 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f4e380cd5a0 con 0x7f4e381628f0 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 0x7f4e380c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 0x7f4e380c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60092/0 (socket says 172.21.15.1:60092) 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 -- 172.21.15.1:0/271697236 learned_addr learned my addr 172.21.15.1:0/271697236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3d9ab640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e38157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e1ac640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e3815e210 0x7f4e380afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 -- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 msgr2=0x7f4e38157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e381628f0 0x7f4e38157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 -- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e3815e210 msgr2=0x7f4e380afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e3815e210 0x7f4e380afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e380ae310 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e3e9ad640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 0x7f4e380c3de0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007e60 tx=0x7f4e3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30014420 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e380af2e0 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.216+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30014820 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.217+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e380cc130 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.217+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e38075320 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.218+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e300149c0 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e2effd640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e1c06be30 0x7f4e1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e3e1ac640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e1c06be30 0x7f4e1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f4e3008bfc0 con 0x7f4e38161720 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e2effd640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f4e1c072410 0x7f4e1c074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e30015ec0 con 0x7f4e1c072410 2024-05-09T06:39:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f4e3008c370 con 0x7f4e38161720 2024-05-09T06:39:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e3e1ac640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e1c06be30 0x7f4e1c06e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004f60 tx=0x7f4e34007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.219+0000 7f4e3d9ab640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f4e1c072410 0x7f4e1c074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.220+0000 7f4e3d9ab640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f4e1c072410 0x7f4e1c074850 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.220+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== osd.6 v2:172.21.15.6:6817/1315796905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e30015ec0 con 0x7f4e1c072410 2024-05-09T06:39:37.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.234+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 --> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4e3815d7e0 con 0x7f4e1c072410 2024-05-09T06:39:37.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.234+0000 7f4e2effd640 1 -- 172.21.15.1:0/271697236 <== osd.6 v2:172.21.15.6:6817/1315796905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4e3815d7e0 con 0x7f4e1c072410 2024-05-09T06:39:37.215 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:37.215 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f4e1c072410 msgr2=0x7f4e1c074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.6:6817/1315796905,v1:172.21.15.6:6819/1315796905] conn(0x7f4e1c072410 0x7f4e1c074850 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e1c06be30 msgr2=0x7f4e1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f4e1c06be30 0x7f4e1c06e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004f60 tx=0x7f4e34007e30 comp rx=0 tx=0).stop 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 msgr2=0x7f4e380c3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 --2- 172.21.15.1:0/271697236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e38161720 0x7f4e380c3de0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007e60 tx=0x7f4e3000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e3e9ad640 1 -- 172.21.15.1:0/271697236 reap_dead start 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 shutdown_connections 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 >> 172.21.15.1:0/271697236 conn(0x7f4e38071a50 msgr2=0x7f4e380cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 shutdown_connections 2024-05-09T06:39:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.235+0000 7f4e40410640 1 -- 172.21.15.1:0/271697236 wait complete. 2024-05-09T06:39:37.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T06:39:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.351+0000 7fc784989640 1 Processor -- start 2024-05-09T06:39:37.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.351+0000 7fc784989640 1 -- start start 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc784989640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc7801454b0 0x7fc7800b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc784989640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc780070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc784989640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78006eb40 con 0x7fc7800b2a60 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc784989640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc78006ee20 con 0x7fc7801454b0 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc784989640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc78006d8e0 con 0x7fc7800b4f70 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77e59c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc7801454b0 0x7fc7800b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77dd9b640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc7800b2a60 0x7fc7800b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57650/0 (socket says 172.21.15.1:57650) 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77dd9b640 1 -- 172.21.15.1:0/1028130100 learned_addr learned my addr 172.21.15.1:0/1028130100 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77ed9d640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc780070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77e59c640 1 -- 172.21.15.1:0/1028130100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 msgr2=0x7fc780070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77e59c640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc780070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77e59c640 1 -- 172.21.15.1:0/1028130100 >> v1:172.21.15.1:6789/0 conn(0x7fc7800b2a60 legacy=0x7fc7800b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77d59a640 1 -- 172.21.15.1:0/1028130100 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192762847 0 0) 0x7fc78006eb40 con 0x7fc7800b2a60 2024-05-09T06:39:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.352+0000 7fc77e59c640 1 -- 172.21.15.1:0/1028130100 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78006dbc0 con 0x7fc7801454b0 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.353+0000 7fc77e59c640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7801454b0 0x7fc7800b4320 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097a0 tx=0x7fc77002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8921dbad7735a0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.353+0000 7fc77d59a640 1 -- 172.21.15.1:0/1028130100 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770037070 con 0x7fc7801454b0 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.353+0000 7fc77d59a640 1 -- 172.21.15.1:0/1028130100 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770032440 con 0x7fc7801454b0 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.353+0000 7fc77d59a640 1 -- 172.21.15.1:0/1028130100 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770032810 con 0x7fc7801454b0 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 -- 172.21.15.1:0/1028130100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7801454b0 msgr2=0x7fc7800b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7801454b0 0x7fc7800b4320 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fc7700097a0 tx=0x7fc77002a180 comp rx=0 tx=0).stop 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 -- 172.21.15.1:0/1028130100 shutdown_connections 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc780070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 --2- 172.21.15.1:0/1028130100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7801454b0 0x7fc7800b4320 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 -- 172.21.15.1:0/1028130100 >> 172.21.15.1:0/1028130100 conn(0x7fc7800cc4b0 msgr2=0x7fc7800cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 -- 172.21.15.1:0/1028130100 shutdown_connections 2024-05-09T06:39:37.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.354+0000 7fc784989640 1 -- 172.21.15.1:0/1028130100 wait complete. 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.355+0000 7fc784989640 1 Processor -- start 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 -- start start 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc78013ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7801454b0 0x7fc78013f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc78006e260 con 0x7fc7800b2a60 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc78006e0e0 con 0x7fc7801454b0 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc784989640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fc780071150 con 0x7fc7800b4f70 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc77e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc77e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60106/0 (socket says 172.21.15.1:60106) 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.356+0000 7fc77e59c640 1 -- 172.21.15.1:0/3582998797 learned_addr learned my addr 172.21.15.1:0/3582998797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77dd9b640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc78013ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77ed9d640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7801454b0 0x7fc78013f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 -- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 msgr2=0x7fc78013ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc78013ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 -- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7801454b0 msgr2=0x7fc78013f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7801454b0 0x7fc78013f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 -- 172.21.15.1:0/3582998797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800ab220 con 0x7fc7800b2a60 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc77e59c640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fc770035040 tx=0x7fc7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770037040 con 0x7fc7800b2a60 2024-05-09T06:39:37.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770030500 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770030910 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7800ab500 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.357+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7800abac0 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.358+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7800752b0 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.359+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc770044020 con 0x7fc7800b2a60 2024-05-09T06:39:37.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.359+0000 7fc76effd640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.359+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fc7700aafa0 con 0x7fc7800b2a60 2024-05-09T06:39:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.360+0000 7fc77dd9b640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.360+0000 7fc77dd9b640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 0x7fc76006e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc7800ad7e0 tx=0x7fc774007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.364+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc770079aa0 con 0x7fc7800b2a60 2024-05-09T06:39:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.547+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fc7800aafd0 con 0x7fc76006be30 2024-05-09T06:39:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.547+0000 7fc76effd640 1 -- 172.21.15.1:0/3582998797 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc7800aafd0 con 0x7fc76006be30 2024-05-09T06:39:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 msgr2=0x7fc76006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 0x7fc76006e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc7800ad7e0 tx=0x7fc774007b90 comp rx=0 tx=0).stop 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 msgr2=0x7fc7800ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fc770035040 tx=0x7fc7700046c0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 shutdown_connections 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fc76006be30 0x7fc76006e2f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7801454b0 0x7fc78013f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc7800b4f70 0x7fc78013ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 --2- 172.21.15.1:0/3582998797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7800b2a60 0x7fc7800ae060 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 >> 172.21.15.1:0/3582998797 conn(0x7fc7800cc4b0 msgr2=0x7fc7800b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 shutdown_connections 2024-05-09T06:39:37.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.549+0000 7fc784989640 1 -- 172.21.15.1:0/3582998797 wait complete. 2024-05-09T06:39:37.540 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T06:39:37.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:39:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.668+0000 7f0dfa577640 1 Processor -- start 2024-05-09T06:39:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.668+0000 7f0dfa577640 1 -- start start 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0df415e210 0x7f0df40b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0df415e5f0 0x7f0df41628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df40ca9d0 0x7f0df40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0df4162e30 con 0x7f0df415e210 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0df40cca50 con 0x7f0df415e5f0 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0dfa577640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0df40b17c0 con 0x7f0df40ca9d0 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0df3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0df415e210 0x7f0df40b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0df3fff640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0df415e210 0x7f0df40b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60114/0 (socket says 172.21.15.1:60114) 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0df3fff640 1 -- 172.21.15.1:0/4206198367 learned_addr learned my addr 172.21.15.1:0/4206198367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0df8b14640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df40ca9d0 0x7f0df40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.669+0000 7f0df37fe640 1 --2- 172.21.15.1:0/4206198367 >> v2:172.21.15.6:3300/0 conn(0x7f0df415e5f0 0x7f0df41628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 -- 172.21.15.1:0/4206198367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df40ca9d0 msgr2=0x7f0df40ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df40ca9d0 0x7f0df40ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 -- 172.21.15.1:0/4206198367 >> v2:172.21.15.6:3300/0 conn(0x7f0df415e5f0 msgr2=0x7f0df41628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4206198367 >> v2:172.21.15.6:3300/0 conn(0x7f0df415e5f0 0x7f0df41628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 -- 172.21.15.1:0/4206198367 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df40b1aa0 con 0x7f0df415e210 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.670+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df40b1170 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0df4071910 tx=0x7f0de402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f24b39d88879056 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0df2ffd640 1 -- 172.21.15.1:0/4206198367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4037070 con 0x7f0df415e210 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0df2ffd640 1 -- 172.21.15.1:0/4206198367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de4032440 con 0x7f0df415e210 2024-05-09T06:39:37.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0df2ffd640 1 -- 172.21.15.1:0/4206198367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4032810 con 0x7f0df415e210 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 -- 172.21.15.1:0/4206198367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 msgr2=0x7f0df40b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df40b1170 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0df4071910 tx=0x7f0de402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 -- 172.21.15.1:0/4206198367 shutdown_connections 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df40ca9d0 0x7f0df40ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4206198367 >> v2:172.21.15.6:3300/0 conn(0x7f0df415e5f0 0x7f0df41628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4206198367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df40b1170 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 -- 172.21.15.1:0/4206198367 >> 172.21.15.1:0/4206198367 conn(0x7f0df4071a50 msgr2=0x7f0df40b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 -- 172.21.15.1:0/4206198367 shutdown_connections 2024-05-09T06:39:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.671+0000 7f0dfa577640 1 -- 172.21.15.1:0/4206198367 wait complete. 2024-05-09T06:39:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.672+0000 7f0dfa577640 1 Processor -- start 2024-05-09T06:39:37.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 -- start start 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df40ca9d0 0x7f0df415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df41f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df41e7cc0 0x7f0df40c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0df40cafd0 con 0x7f0df415e210 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0df40cae50 con 0x7f0df40ca9d0 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0dfa577640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f0df40cb150 con 0x7f0df41e7cc0 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.673+0000 7f0df37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df41f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df41f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60122/0 (socket says 172.21.15.1:60122) 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 -- 172.21.15.1:0/4250251935 learned_addr learned my addr 172.21.15.1:0/4250251935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df8b14640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df41e7cc0 0x7f0df40c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df40ca9d0 0x7f0df415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 -- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df41e7cc0 msgr2=0x7f0df40c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0df41e7cc0 0x7f0df40c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 -- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df40ca9d0 msgr2=0x7f0df415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df40ca9d0 0x7f0df415c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df415cf20 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df40ca9d0 0x7f0df415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df37fe640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df41f4260 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e870 tx=0x7f0de800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8017070 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de800c940 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df41f47a0 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de800cd30 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.674+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df40ad1c0 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.675+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0df4073730 con 0x7f0df415e210 2024-05-09T06:39:37.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0de801a070 con 0x7f0df415e210 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0de006be30 0x7f0de006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f0de808ac50 con 0x7f0df415e210 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0de006be30 0x7f0de006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7f0de0072410 0x7f0de0074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0de0074f20 con 0x7f0de0072410 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.676+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f0de808d350 con 0x7f0df415e210 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.677+0000 7f0df8b14640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7f0de0072410 0x7f0de0074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.677+0000 7f0df3fff640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0de006be30 0x7f0de006e2f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.677+0000 7f0df8b14640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7f0de0072410 0x7f0de0074850 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.678+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== osd.7 v2:172.21.15.6:6816/3240730649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0de0074f20 con 0x7f0de0072410 2024-05-09T06:39:37.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.691+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 --> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0df4075320 con 0x7f0de0072410 2024-05-09T06:39:37.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0df0ff9640 1 -- 172.21.15.1:0/4250251935 <== osd.7 v2:172.21.15.6:6816/3240730649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0df4075320 con 0x7f0de0072410 2024-05-09T06:39:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T06:39:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:39:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T06:39:37.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7f0de0072410 msgr2=0x7f0de0074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.6:6816/3240730649,v1:172.21.15.6:6818/3240730649] conn(0x7f0de0072410 0x7f0de0074850 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0de006be30 msgr2=0x7f0de006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f0de006be30 0x7f0de006e2f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).stop 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 msgr2=0x7f0df41f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 --2- 172.21.15.1:0/4250251935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df415e210 0x7f0df41f4260 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e870 tx=0x7f0de800ed30 comp rx=0 tx=0).stop 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0df8b14640 1 -- 172.21.15.1:0/4250251935 reap_dead start 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 shutdown_connections 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 >> 172.21.15.1:0/4250251935 conn(0x7f0df4071a50 msgr2=0x7f0df40cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 shutdown_connections 2024-05-09T06:39:37.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.692+0000 7f0dfa577640 1 -- 172.21.15.1:0/4250251935 wait complete. 2024-05-09T06:39:37.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T06:39:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.807+0000 7fcd4b122640 1 Processor -- start 2024-05-09T06:39:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.807+0000 7fcd4b122640 1 -- start start 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd4b122640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcd44071ce0 0x7fcd440cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd4b122640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd4415f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd4b122640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd440d04a0 con 0x7fcd440b0080 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd4b122640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4415f670 con 0x7fcd44071ce0 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd4b122640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd440722c0 con 0x7fcd4415ece0 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd48ebe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcd440b0080 0x7fcd4415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48930/0 (socket says 172.21.15.1:48930) 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd48ebe640 1 -- 172.21.15.1:0/873265438 learned_addr learned my addr 172.21.15.1:0/873265438 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd3bfff640 1 --2- 172.21.15.1:0/873265438 >> v2:172.21.15.6:3300/0 conn(0x7fcd44071ce0 0x7fcd440cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd496bf640 1 --2- 172.21.15.1:0/873265438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd4415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959592408 0 0) 0x7fcd440d04a0 con 0x7fcd440b0080 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.808+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd2c003520 con 0x7fcd440b0080 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2560437335 0 0) 0x7fcd2c003520 con 0x7fcd440b0080 2024-05-09T06:39:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd496bf640 1 -- 172.21.15.1:0/873265438 >> v2:172.21.15.6:3300/0 conn(0x7fcd44071ce0 msgr2=0x7fcd440cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:37.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd496bf640 1 --2- 172.21.15.1:0/873265438 >> v2:172.21.15.6:3300/0 conn(0x7fcd44071ce0 0x7fcd440cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd496bf640 1 -- 172.21.15.1:0/873265438 >> v1:172.21.15.1:6789/0 conn(0x7fcd440b0080 legacy=0x7fcd4415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd496bf640 1 -- 172.21.15.1:0/873265438 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd440725a0 con 0x7fcd4415ece0 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.809+0000 7fcd496bf640 1 --2- 172.21.15.1:0/873265438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd4415f130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009d90 tx=0x7fcd3c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ca6e4e36f8d7e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40001670 con 0x7fcd4415ece0 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c007540 con 0x7fcd4415ece0 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd3b7fe640 1 -- 172.21.15.1:0/873265438 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c007910 con 0x7fcd4415ece0 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 -- 172.21.15.1:0/873265438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 msgr2=0x7fcd4415f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 --2- 172.21.15.1:0/873265438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd4415f130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009d90 tx=0x7fcd3c02a5f0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 -- 172.21.15.1:0/873265438 shutdown_connections 2024-05-09T06:39:37.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 --2- 172.21.15.1:0/873265438 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd4415f130 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 --2- 172.21.15.1:0/873265438 >> v2:172.21.15.6:3300/0 conn(0x7fcd44071ce0 0x7fcd440cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 -- 172.21.15.1:0/873265438 >> 172.21.15.1:0/873265438 conn(0x7fcd440d0b50 msgr2=0x7fcd440d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 -- 172.21.15.1:0/873265438 shutdown_connections 2024-05-09T06:39:37.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.810+0000 7fcd4b122640 1 -- 172.21.15.1:0/873265438 wait complete. 2024-05-09T06:39:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.811+0000 7fcd4b122640 1 Processor -- start 2024-05-09T06:39:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.811+0000 7fcd4b122640 1 -- start start 2024-05-09T06:39:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 0x7fcd4415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd440b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4415e7c0 con 0x7fcd440b0080 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44072b60 con 0x7fcd44071ce0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd4b122640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4415ff40 con 0x7fcd4415ece0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35350/0 (socket says 172.21.15.1:35350) 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.812+0000 7fcd3bfff640 1 -- 172.21.15.1:0/708923586 learned_addr learned my addr 172.21.15.1:0/708923586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd496bf640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd440b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd48ebe640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 0x7fcd4415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 -- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 msgr2=0x7fcd440b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd440b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 -- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 msgr2=0x7fcd4415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 0x7fcd4415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 -- 172.21.15.1:0/708923586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd440c89f0 con 0x7fcd440b0080 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd48ebe640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 0x7fcd4415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd3bfff640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400e9b0 tx=0x7fcd3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34017070 con 0x7fcd440b0080 2024-05-09T06:39:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3400c9a0 con 0x7fcd440b0080 2024-05-09T06:39:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd440c8cd0 con 0x7fcd440b0080 2024-05-09T06:39:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400cd90 con 0x7fcd440b0080 2024-05-09T06:39:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.813+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd440c9290 con 0x7fcd440b0080 2024-05-09T06:39:37.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.814+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4406f540 con 0x7fcd440b0080 2024-05-09T06:39:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.815+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd3401a070 con 0x7fcd440b0080 2024-05-09T06:39:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.815+0000 7fcd397fa640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.815+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fcd3408b2d0 con 0x7fcd440b0080 2024-05-09T06:39:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.815+0000 7fcd48ebe640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:37.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.816+0000 7fcd48ebe640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005880 tx=0x7fcd40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:37.820+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd4406f540 con 0x7fcd440b0080 2024-05-09T06:39:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.003+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fcd440d0f30 con 0x7fcd3006be30 2024-05-09T06:39:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.003+0000 7fcd397fa640 1 -- 172.21.15.1:0/708923586 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcd440d0f30 con 0x7fcd3006be30 2024-05-09T06:39:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 msgr2=0x7fcd3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005880 tx=0x7fcd40009040 comp rx=0 tx=0).stop 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 msgr2=0x7fcd440b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400e9b0 tx=0x7fcd3400ee70 comp rx=0 tx=0).stop 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 shutdown_connections 2024-05-09T06:39:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcd4415ece0 0x7fcd440b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd440b0080 0x7fcd440b3b00 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 --2- 172.21.15.1:0/708923586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd44071ce0 0x7fcd4415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 >> 172.21.15.1:0/708923586 conn(0x7fcd440d0b50 msgr2=0x7fcd44160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 shutdown_connections 2024-05-09T06:39:37.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.006+0000 7fcd4b122640 1 -- 172.21.15.1:0/708923586 wait complete. 2024-05-09T06:39:37.997 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:39:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.121+0000 7fb71997e640 1 Processor -- start 2024-05-09T06:39:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.121+0000 7fb71997e640 1 -- start start 2024-05-09T06:39:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb71997e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb71997e640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb7140d0f80 0x7fb7140d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb71997e640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7140b33c0 con 0x7fb7140d4940 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb71997e640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb7140d4dd0 con 0x7fb7140d0f80 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb71997e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb7140b3660 con 0x7fb714161950 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb7137fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb7140d0f80 0x7fb7140d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb7137fe640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb7140d0f80 0x7fb7140d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35366/0 (socket says 172.21.15.1:35366) 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb7137fe640 1 -- 172.21.15.1:0/2146369900 learned_addr learned my addr 172.21.15.1:0/2146369900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb7127fc640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb7140d4940 0x7fb7140cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:48940/0 (socket says 172.21.15.1:48940) 2024-05-09T06:39:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.122+0000 7fb712ffd640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb7137fe640 1 -- 172.21.15.1:0/2146369900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 msgr2=0x7fb7140d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb7137fe640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb7137fe640 1 -- 172.21.15.1:0/2146369900 >> v1:172.21.15.1:6789/0 conn(0x7fb7140d4940 legacy=0x7fb7140cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb7137fe640 1 -- 172.21.15.1:0/2146369900 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7140b3940 con 0x7fb7140d0f80 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb711ffb640 1 -- 172.21.15.1:0/2146369900 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725984567 0 0) 0x7fb7140b33c0 con 0x7fb7140d4940 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.123+0000 7fb7137fe640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140d0f80 0x7fb7140d2940 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b3590 tx=0x7fb70402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4dc2286cd63020 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb711ffb640 1 -- 172.21.15.1:0/2146369900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704036070 con 0x7fb7140d0f80 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb711ffb640 1 -- 172.21.15.1:0/2146369900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704031600 con 0x7fb7140d0f80 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb711ffb640 1 -- 172.21.15.1:0/2146369900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040319d0 con 0x7fb7140d0f80 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 -- 172.21.15.1:0/2146369900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140d0f80 msgr2=0x7fb7140d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140d0f80 0x7fb7140d2940 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b3590 tx=0x7fb70402a710 comp rx=0 tx=0).stop 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 -- 172.21.15.1:0/2146369900 shutdown_connections 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140d0f80 0x7fb7140d2940 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 --2- 172.21.15.1:0/2146369900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 -- 172.21.15.1:0/2146369900 >> 172.21.15.1:0/2146369900 conn(0x7fb7140d3e20 msgr2=0x7fb7140d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 -- 172.21.15.1:0/2146369900 shutdown_connections 2024-05-09T06:39:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.124+0000 7fb71997e640 1 -- 172.21.15.1:0/2146369900 wait complete. 2024-05-09T06:39:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.125+0000 7fb71997e640 1 Processor -- start 2024-05-09T06:39:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 -- start start 2024-05-09T06:39:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7140d0f80 0x7fb7140c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb7140b52d0 con 0x7fb7140b6f50 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb7140b5150 con 0x7fb7140d0f80 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb71997e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fb7140b5450 con 0x7fb714161950 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35370/0 (socket says 172.21.15.1:35370) 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb7137fe640 1 -- 172.21.15.1:0/334446989 learned_addr learned my addr 172.21.15.1:0/334446989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb712ffd640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7140d0f80 0x7fb7140c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.126+0000 7fb7127fc640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 -- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 msgr2=0x7fb7140b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 -- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7140d0f80 msgr2=0x7fb7140c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7140d0f80 0x7fb7140c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 -- 172.21.15.1:0/334446989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7140cb500 con 0x7fb7140b6f50 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb7137fe640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb704009f40 tx=0x7fb704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704036050 con 0x7fb7140b6f50 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704042070 con 0x7fb7140b6f50 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7140cb7e0 con 0x7fb7140b6f50 2024-05-09T06:39:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704009510 con 0x7fb7140b6f50 2024-05-09T06:39:38.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.127+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7140cbda0 con 0x7fb7140b6f50 2024-05-09T06:39:38.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.128+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7140752b0 con 0x7fb7140b6f50 2024-05-09T06:39:38.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.129+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb704007bf0 con 0x7fb7140b6f50 2024-05-09T06:39:38.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.129+0000 7fb70bfff640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:38.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.129+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fb7040aba20 con 0x7fb7140b6f50 2024-05-09T06:39:38.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.130+0000 7fb712ffd640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:38.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.130+0000 7fb712ffd640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 0x7fb6f406e2f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009aa0 tx=0x7fb70c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:38.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.134+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb704079e40 con 0x7fb7140b6f50 2024-05-09T06:39:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.314+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7140d21e0 con 0x7fb6f406be30 2024-05-09T06:39:38.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.315+0000 7fb70bfff640 1 -- 172.21.15.1:0/334446989 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13488 (secure 0 0 0) 0x7fb7140d21e0 con 0x7fb6f406be30 2024-05-09T06:39:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:38.295 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 msgr2=0x7fb6f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 0x7fb6f406e2f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009aa0 tx=0x7fb70c009190 comp rx=0 tx=0).stop 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 msgr2=0x7fb7140b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb704009f40 tx=0x7fb704004a40 comp rx=0 tx=0).stop 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 shutdown_connections 2024-05-09T06:39:38.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7140b6f50 0x7fb7140b86f0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb714161950 0x7fb7140b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.316+0000 7fb71997e640 1 --2- 172.21.15.1:0/334446989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7140d0f80 0x7fb7140c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.317+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 >> 172.21.15.1:0/334446989 conn(0x7fb7140d3e20 msgr2=0x7fb71415e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.317+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 shutdown_connections 2024-05-09T06:39:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:38.317+0000 7fb71997e640 1 -- 172.21.15.1:0/334446989 wait complete. 2024-05-09T06:39:38.308 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-05-09T06:39:37.236266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275432,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754699288,"statfs":{"total":773094113280,"available":772812070912,"internally_reserved":0,"allocated":13803520,"data_stored":7085200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001534"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":368,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-09T06:39:30.896942+0000","last_change":"2024-05-09T06:27:10.275815+0000","last_active":"2024-05-09T06:39:30.896942+0000","last_peered":"2024-05-09T06:39:30.896942+0000","last_clean":"2024-05-09T06:39:30.896942+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:39:30.896942+0000","last_undegraded":"2024-05-09T06:39:30.896942+0000","last_fullsized":"2024-05-09T06:39:30.896942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:27:09.112410+0000","last_clean_scrub_stamp":"2024-05-09T06:27:09.112410+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.337215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1328426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2170880,"data_stored":1328426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:39:38.311 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T06:27:09.112410+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=6, tm_min=27, tm_sec=9, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=6, tm_min=39, tm_sec=33, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T06:39:38.311 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T06:39:49.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:39:49.283 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:39:58.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:39:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.450+0000 7f883565d640 1 Processor -- start 2024-05-09T06:39:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.450+0000 7f883565d640 1 -- start start 2024-05-09T06:39:58.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.450+0000 7f883565d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f883006e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f883565d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883015eb40 con 0x7f88300ae2a0 2024-05-09T06:39:58.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f883565d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88300aef90 con 0x7f883015e050 2024-05-09T06:39:58.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f883565d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f88300b3b80 con 0x7f883006e4f0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882f7fe640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f88300ae2a0 0x7f88300ae710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55316/0 (socket says 172.21.15.1:55316) 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882f7fe640 1 -- 172.21.15.1:0/338058964 learned_addr learned my addr 172.21.15.1:0/338058964 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882e7fc640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f883006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165328332 0 0) 0x7f883015eb40 con 0x7f88300ae2a0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8818003520 con 0x7f88300ae2a0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3892292273 0 0) 0x7f8818003520 con 0x7f88300ae2a0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 msgr2=0x7f883006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f883006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 >> v1:172.21.15.6:6789/0 conn(0x7f883015e050 legacy=0x7f883015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300b3530 con 0x7f88300ae2a0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 814331372 0 0) 0x7f8828001670 con 0x7f88300ae2a0 2024-05-09T06:39:58.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 >> v1:172.21.15.1:6789/0 conn(0x7f88300ae2a0 legacy=0x7f88300ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f8818003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8818004800 0x7f8818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f88180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.451+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88300b3b80 con 0x7f88180039e0 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f881800baa0 con 0x7f8818004800 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f881800bca0 con 0x7f883006e4f0 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f8818003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882f7fe640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f88180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882e7fc640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8818004800 0x7f8818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 -- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 msgr2=0x7f88180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f883006e4f0 0x7f88180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 -- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8818004800 msgr2=0x7f8818004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8818004800 0x7f8818004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.452+0000 7f882effd640 1 -- 172.21.15.1:0/338058964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8818003520 con 0x7f88180039e0 2024-05-09T06:39:58.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f882effd640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f8818003e00 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f8820001e20 tx=0x7f8818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fa43a978fab258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820001a70 con 0x7f88180039e0 2024-05-09T06:39:58.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882002ae10 con 0x7f88180039e0 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f882dffb640 1 -- 172.21.15.1:0/338058964 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820003bb0 con 0x7f88180039e0 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f883565d640 1 -- 172.21.15.1:0/338058964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 msgr2=0x7f8818003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f883565d640 1 --2- 172.21.15.1:0/338058964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f8818003e00 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f8820001e20 tx=0x7f8818005830 comp rx=0 tx=0).stop 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f882f7fe640 1 -- 172.21.15.1:0/338058964 reap_dead start 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f883565d640 1 -- 172.21.15.1:0/338058964 shutdown_connections 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f883565d640 1 -- 172.21.15.1:0/338058964 >> 172.21.15.1:0/338058964 conn(0x7f883006d270 msgr2=0x7f883006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.453+0000 7f883565d640 1 -- 172.21.15.1:0/338058964 shutdown_connections 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.454+0000 7f883565d640 1 -- 172.21.15.1:0/338058964 wait complete. 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.455+0000 7f883565d640 1 Processor -- start 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.455+0000 7f883565d640 1 -- start start 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88300c4f60 0x7f88301e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88300c53e0 0x7f88301e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f883006eb00 con 0x7f88180039e0 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f883006e980 con 0x7f88300c53e0 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f883565d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7f883006ec80 con 0x7f88300c4f60 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44812/0 (socket says 172.21.15.1:44812) 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 -- 172.21.15.1:0/1317891902 learned_addr learned my addr 172.21.15.1:0/1317891902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882f7fe640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88300c53e0 0x7f88301e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882e7fc640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88300c4f60 0x7f88301e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 -- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88300c4f60 msgr2=0x7f88301e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88300c4f60 0x7f88301e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 -- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88300c53e0 msgr2=0x7f88301e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88300c53e0 0x7f88301e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 -- 172.21.15.1:0/1317891902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883014ce10 con 0x7f88180039e0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.456+0000 7f882effd640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f8820005040 tx=0x7f88200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.457+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882003f030 con 0x7f88180039e0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.457+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8820003e10 con 0x7f88180039e0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.457+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88300c59b0 con 0x7f88180039e0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.457+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820037040 con 0x7f88180039e0 2024-05-09T06:39:58.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.457+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88300c5f70 con 0x7f88180039e0 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.458+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300752b0 con 0x7f88180039e0 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.458+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8820042070 con 0x7f88180039e0 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.459+0000 7f88177fe640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 0x7f881c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.459+0000 7f882e7fc640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 0x7f881c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.459+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7f88200aaa80 con 0x7f88180039e0 2024-05-09T06:39:58.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.460+0000 7f882e7fc640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 0x7f881c06e320 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f88301f7500 tx=0x7f8824009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.464+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f88200ace20 con 0x7f88180039e0 2024-05-09T06:39:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.642+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 --> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88300b1c30 con 0x7f881c06be60 2024-05-09T06:39:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.643+0000 7f88177fe640 1 -- 172.21.15.1:0/1317891902 <== mgr.4107 v2:172.21.15.1:6832/2620481298 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f88300b1c30 con 0x7f881c06be60 2024-05-09T06:39:58.623 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:58.624 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 msgr2=0x7f881c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 0x7f881c06e320 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f88301f7500 tx=0x7f8824009190 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 msgr2=0x7f883014c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f8820005040 tx=0x7f88200014a0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 shutdown_connections 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7f881c06be60 0x7f881c06e320 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88300c53e0 0x7f88301e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88300c4f60 0x7f88301e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 --2- 172.21.15.1:0/1317891902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88180039e0 0x7f883014c8d0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 >> 172.21.15.1:0/1317891902 conn(0x7f883006d270 msgr2=0x7f88300b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 shutdown_connections 2024-05-09T06:39:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.645+0000 7f883565d640 1 -- 172.21.15.1:0/1317891902 wait complete. 2024-05-09T06:39:58.636 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-05-09T06:39:57.240760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":275416,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754699304,"statfs":{"total":773094113280,"available":772812087296,"internally_reserved":0,"allocated":13803520,"data_stored":7085200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002705"},"pg_stats":[{"pgid":"1.0","version":"126'119","reported_seq":379,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-05-09T06:39:38.948033+0000","last_change":"2024-05-09T06:39:38.948072+0000","last_active":"2024-05-09T06:39:38.948033+0000","last_peered":"2024-05-09T06:39:38.948033+0000","last_clean":"2024-05-09T06:39:38.948033+0000","last_became_active":"2024-05-09T06:27:10.274753+0000","last_became_peered":"2024-05-09T06:27:10.274753+0000","last_unstale":"2024-05-09T06:39:38.948033+0000","last_undegraded":"2024-05-09T06:39:38.948033+0000","last_fullsized":"2024-05-09T06:39:38.948033+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'119","last_scrub_stamp":"2024-05-09T06:39:38.947944+0000","last_deep_scrub":"126'119","last_deep_scrub_stamp":"2024-05-09T06:39:38.947944+0000","last_clean_scrub_stamp":"2024-05-09T06:39:38.947944+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:22.831359+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1328426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":738058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2170880,"data_stored":1328426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:39:58.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T06:39:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.762+0000 7ff05c334640 1 Processor -- start 2024-05-09T06:39:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.762+0000 7ff05c334640 1 -- start start 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05c334640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05c334640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 0x7ff0540ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05c334640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff054072050 con 0x7ff0540c41e0 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05c334640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff0540cd160 con 0x7ff0540cf8e0 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05c334640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff054072330 con 0x7ff0540d06f0 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05a0d0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05a8d1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 0x7ff0540ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05a0d0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.1:49846/0 (socket says 172.21.15.1:49846) 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.763+0000 7ff05a0d0640 1 -- 172.21.15.1:0/3970130012 learned_addr learned my addr 172.21.15.1:0/3970130012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a0d0640 1 -- 172.21.15.1:0/3970130012 >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 msgr2=0x7ff0540ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a0d0640 1 --2- 172.21.15.1:0/3970130012 >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 0x7ff0540ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a0d0640 1 -- 172.21.15.1:0/3970130012 >> v1:172.21.15.6:6789/0 conn(0x7ff0540c41e0 legacy=0x7ff054161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a0d0640 1 -- 172.21.15.1:0/3970130012 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff054072610 con 0x7ff0540d06f0 2024-05-09T06:39:58.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a8d1640 1 --2- 172.21.15.1:0/3970130012 >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 0x7ff0540ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff05a0d0640 1 --2- 172.21.15.1:0/3970130012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff050009870 tx=0x7ff05002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47755b85b5c314fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.764+0000 7ff0590ce640 1 -- 172.21.15.1:0/3970130012 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050037070 con 0x7ff0540d06f0 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff0590ce640 1 -- 172.21.15.1:0/3970130012 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff050032440 con 0x7ff0540d06f0 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff0590ce640 1 -- 172.21.15.1:0/3970130012 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050032810 con 0x7ff0540d06f0 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 -- 172.21.15.1:0/3970130012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 msgr2=0x7ff0540cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 --2- 172.21.15.1:0/3970130012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff050009870 tx=0x7ff05002a270 comp rx=0 tx=0).stop 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 -- 172.21.15.1:0/3970130012 shutdown_connections 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 --2- 172.21.15.1:0/3970130012 >> v2:172.21.15.6:3300/0 conn(0x7ff0540cf8e0 0x7ff0540ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 --2- 172.21.15.1:0/3970130012 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540d06f0 0x7ff0540cc6e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 -- 172.21.15.1:0/3970130012 >> 172.21.15.1:0/3970130012 conn(0x7ff0540d0b50 msgr2=0x7ff0540d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:58.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 -- 172.21.15.1:0/3970130012 shutdown_connections 2024-05-09T06:39:58.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.765+0000 7ff05c334640 1 -- 172.21.15.1:0/3970130012 wait complete. 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.766+0000 7ff05c334640 1 Processor -- start 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.766+0000 7ff05c334640 1 -- start start 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540c41e0 0x7ff0541f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0540cf8e0 0x7ff0540c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff05406b1e0 con 0x7ff0540d06f0 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff054071ce0 con 0x7ff0540cf8e0 2024-05-09T06:39:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05c334640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7ff05406b360 con 0x7ff0540c41e0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44826/0 (socket says 172.21.15.1:44826) 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 -- 172.21.15.1:0/3353881096 learned_addr learned my addr 172.21.15.1:0/3353881096 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a0d0640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540c41e0 0x7ff0541f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff0598cf640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0540cf8e0 0x7ff0540c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 -- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540c41e0 msgr2=0x7ff0541f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540c41e0 0x7ff0541f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 -- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0540cf8e0 msgr2=0x7ff0540c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0540cf8e0 0x7ff0540c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.767+0000 7ff05a8d1640 1 -- 172.21.15.1:0/3353881096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0540c4780 con 0x7ff0540d06f0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff05a8d1640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009f00 tx=0x7ff04c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c018030 con 0x7ff0540d06f0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0540c4a60 con 0x7ff0540d06f0 2024-05-09T06:39:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff04c00ce50 con 0x7ff0540d06f0 2024-05-09T06:39:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c003bb0 con 0x7ff0540d06f0 2024-05-09T06:39:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.768+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0540c5020 con 0x7ff0540d06f0 2024-05-09T06:39:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.769+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0540752b0 con 0x7ff0540d06f0 2024-05-09T06:39:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.770+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff04c01b070 con 0x7ff0540d06f0 2024-05-09T06:39:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.770+0000 7ff04affd640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 0x7ff03806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.770+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7ff04c08b6f0 con 0x7ff0540d06f0 2024-05-09T06:39:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.770+0000 7ff05a0d0640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 0x7ff03806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.771+0000 7ff05a0d0640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 0x7ff03806e2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff050034000 tx=0x7ff05002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:58.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.775+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff04c010040 con 0x7ff0540d06f0 2024-05-09T06:39:58.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.957+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7ff0540c51f0 con 0x7ff0540d06f0 2024-05-09T06:39:58.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.960+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7ff04c059b20 con 0x7ff0540d06f0 2024-05-09T06:39:58.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.962+0000 7ff04affd640 1 -- 172.21.15.1:0/3353881096 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff04c05de20 con 0x7ff0540d06f0 2024-05-09T06:39:58.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 msgr2=0x7ff03806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 0x7ff03806e2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff050034000 tx=0x7ff05002e000 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 msgr2=0x7ff05406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009f00 tx=0x7ff04c00c490 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 shutdown_connections 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7ff03806be30 0x7ff03806e2f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0540d06f0 0x7ff05406dd00 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff0540cf8e0 0x7ff0540c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 --2- 172.21.15.1:0/3353881096 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff0540c41e0 0x7ff0541f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 >> 172.21.15.1:0/3353881096 conn(0x7ff0540d0b50 msgr2=0x7ff05415d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 shutdown_connections 2024-05-09T06:39:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:58.963+0000 7ff05c334640 1 -- 172.21.15.1:0/3353881096 wait complete. 2024-05-09T06:39:58.955 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:39:58.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:39:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.079+0000 7fa882bf4640 1 Processor -- start 2024-05-09T06:39:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.079+0000 7fa882bf4640 1 -- start start 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa882bf4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 0x7fa87c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa882bf4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 0x7fa87c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa882bf4640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c06eb40 con 0x7fa87c0b2a60 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa882bf4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c06ee20 con 0x7fa87c0b4f70 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa882bf4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c06d8e0 con 0x7fa87c1454b0 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa87bfff640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa87c0b2a60 0x7fa87c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55322/0 (socket says 172.21.15.1:55322) 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa87bfff640 1 -- 172.21.15.1:0/3928513747 learned_addr learned my addr 172.21.15.1:0/3928513747 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa880990640 1 --2- 172.21.15.1:0/3928513747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 0x7fa87c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa881191640 1 --2- 172.21.15.1:0/3928513747 >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 0x7fa87c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa87b7fe640 1 -- 172.21.15.1:0/3928513747 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302358367 0 0) 0x7fa87c06eb40 con 0x7fa87c0b2a60 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.080+0000 7fa87b7fe640 1 -- 172.21.15.1:0/3928513747 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa868003520 con 0x7fa87c0b2a60 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa880990640 1 -- 172.21.15.1:0/3928513747 >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 msgr2=0x7fa87c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa880990640 1 --2- 172.21.15.1:0/3928513747 >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 0x7fa87c070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa880990640 1 -- 172.21.15.1:0/3928513747 >> v1:172.21.15.1:6789/0 conn(0x7fa87c0b2a60 legacy=0x7fa87c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa880990640 1 -- 172.21.15.1:0/3928513747 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c06dbc0 con 0x7fa87c1454b0 2024-05-09T06:39:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa881191640 1 --2- 172.21.15.1:0/3928513747 >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 0x7fa87c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.081+0000 7fa880990640 1 --2- 172.21.15.1:0/3928513747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 0x7fa87c0b4320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009d90 tx=0x7fa86c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c081dc52195be88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa87b7fe640 1 -- 172.21.15.1:0/3928513747 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c007740 con 0x7fa87c1454b0 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa87b7fe640 1 -- 172.21.15.1:0/3928513747 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa86c0078e0 con 0x7fa87c1454b0 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa87b7fe640 1 -- 172.21.15.1:0/3928513747 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c007cd0 con 0x7fa87c1454b0 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 -- 172.21.15.1:0/3928513747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 msgr2=0x7fa87c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 --2- 172.21.15.1:0/3928513747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 0x7fa87c0b4320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009d90 tx=0x7fa86c02a660 comp rx=0 tx=0).stop 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 -- 172.21.15.1:0/3928513747 shutdown_connections 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 --2- 172.21.15.1:0/3928513747 >> v2:172.21.15.6:3300/0 conn(0x7fa87c0b4f70 0x7fa87c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 --2- 172.21.15.1:0/3928513747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c1454b0 0x7fa87c0b4320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 -- 172.21.15.1:0/3928513747 >> 172.21.15.1:0/3928513747 conn(0x7fa87c0cc4b0 msgr2=0x7fa87c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 -- 172.21.15.1:0/3928513747 shutdown_connections 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.082+0000 7fa882bf4640 1 -- 172.21.15.1:0/3928513747 wait complete. 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.083+0000 7fa882bf4640 1 Processor -- start 2024-05-09T06:39:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 -- start start 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87c0b2a60 0x7fa87c0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c0b4f70 0x7fa87c1e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c06e260 con 0x7fa87c1454b0 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c06e0e0 con 0x7fa87c0b2a60 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.084+0000 7fa882bf4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c071150 con 0x7fa87c0b4f70 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44838/0 (socket says 172.21.15.1:44838) 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 -- 172.21.15.1:0/1132005122 learned_addr learned my addr 172.21.15.1:0/1132005122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa880990640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87c0b2a60 0x7fa87c0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa87bfff640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c0b4f70 0x7fa87c1e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 -- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c0b4f70 msgr2=0x7fa87c1e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c0b4f70 0x7fa87c1e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 -- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87c0b2a60 msgr2=0x7fa87c0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87c0b2a60 0x7fa87c0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 -- 172.21.15.1:0/1132005122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c1e1cb0 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa881191640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e930 tx=0x7fa87400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874017070 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa87400ca00 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c1e1f90 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87400ce10 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.085+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1e2550 con 0x7fa87c1454b0 2024-05-09T06:39:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.086+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c1e2b10 con 0x7fa87c1454b0 2024-05-09T06:39:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.087+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa87401a070 con 0x7fa87c1454b0 2024-05-09T06:39:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.087+0000 7fa8797fa640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 0x7fa85c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.088+0000 7fa880990640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 0x7fa85c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.088+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5507+0+0 (secure 0 0 0) 0x7fa87408b030 con 0x7fa87c1454b0 2024-05-09T06:39:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.088+0000 7fa880990640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 0x7fa85c06e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:59.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.092+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8740593c0 con 0x7fa87c1454b0 2024-05-09T06:39:59.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.316+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa87c0752b0 con 0x7fa87c1454b0 2024-05-09T06:39:59.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.316+0000 7fa8797fa640 1 -- 172.21.15.1:0/1132005122 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3860 (secure 0 0 0) 0x7fa87405d6c0 con 0x7fa87c1454b0 2024-05-09T06:39:59.297 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T06:39:59.297 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2920200247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2920200247},{"type":"v1","addr":"172.21.15.1:6835","nonce":2920200247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6834/1391736482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1391736482},{"type":"v1","addr":"172.21.15.6:6834","nonce":1391736482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/2626906845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6833","nonce":2626906845},{"type":"v1","addr":"172.21.15.6:6835","nonce":2626906845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":10380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2580481038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2580481038},{"type":"v1","addr":"172.21.15.1:6837","nonce":2580481038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-05-09T06:39:59.297 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 114 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 msgr2=0x7fa85c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 0x7fa85c06e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02f000 comp rx=0 tx=0).stop 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 msgr2=0x7fa87c140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e930 tx=0x7fa87400edf0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 shutdown_connections 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:6832/2620481298,v1:172.21.15.1:6833/2620481298] conn(0x7fa85c06be30 0x7fa85c06e2f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa87c1454b0 0x7fa87c140e20 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa87c0b4f70 0x7fa87c1e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.318+0000 7fa882bf4640 1 --2- 172.21.15.1:0/1132005122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87c0b2a60 0x7fa87c0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.319+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 >> 172.21.15.1:0/1132005122 conn(0x7fa87c0cc4b0 msgr2=0x7fa87c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.319+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 shutdown_connections 2024-05-09T06:39:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T06:39:59.319+0000 7fa882bf4640 1 -- 172.21.15.1:0/1132005122 wait complete. 2024-05-09T06:39:59.310 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-09T06:39:59.310 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T06:39:59.310 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:39:59.310 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.319 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:39:59.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:39:59.320 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.369 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:39:59.370 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:39:59.370 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.382 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:39:59.382 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:39:59.382 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.434 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:39:59.434 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T06:39:59.434 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T06:39:59.434 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.459 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T06:39:59.459 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T06:39:59.459 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.530 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T06:39:59.530 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T06:39:59.531 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.598 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T06:39:59.598 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T06:39:59.598 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.671 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T06:39:59.672 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T06:39:59.672 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.700 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T06:39:59.700 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T06:39:59.700 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.771 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T06:39:59.771 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T06:39:59.771 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.849 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T06:39:59.850 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T06:39:59.850 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.920 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T06:39:59.921 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T06:39:59.921 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T06:39:59.921 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.948 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T06:39:59.948 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T06:39:59.948 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:39:59.989 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T06:39:59.989 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T06:39:59.989 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T06:39:59.989 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:40:00.001 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T06:40:00.001 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T06:40:00.001 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:40:00.042 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T06:40:00.042 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T06:40:00.042 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:40:00.095 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T06:40:00.095 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T06:40:00.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-09T06:40:00.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:40:00.148 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T06:40:00.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:40:00.334 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T06:40:00.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:40:00.423 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T06:40:00.482 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:40:00.482 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T06:40:00.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:40:00.531 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T06:40:00.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:40:00.720 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T06:40:00.792 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:40:00.792 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T06:40:00.857 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:40:00.857 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T06:40:00.924 INFO:tasks.ceph:Archiving mon data... 2024-05-09T06:40:00.926 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/data/mon.a.tgz 2024-05-09T06:40:00.926 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-05-09T06:40:00.943 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.B3AxvylFoG 2024-05-09T06:40:00.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.B3AxvylFoG 2024-05-09T06:40:01.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.B3AxvylFoG 2024-05-09T06:40:01.211 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.B3AxvylFoG is 449KB 2024-05-09T06:40:01.310 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.B3AxvylFoG 2024-05-09T06:40:01.326 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/data/mon.b.tgz 2024-05-09T06:40:01.326 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-09T06:40:01.343 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.YVYqtZfAla 2024-05-09T06:40:01.344 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YVYqtZfAla 2024-05-09T06:40:01.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.YVYqtZfAla 2024-05-09T06:40:01.604 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.YVYqtZfAla is 466KB 2024-05-09T06:40:01.674 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.YVYqtZfAla 2024-05-09T06:40:01.690 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/data/mon.c.tgz 2024-05-09T06:40:01.690 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-09T06:40:01.746 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.ahJlge9b44 2024-05-09T06:40:01.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ahJlge9b44 2024-05-09T06:40:01.902 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.ahJlge9b44 2024-05-09T06:40:01.987 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.ahJlge9b44 is 466KB 2024-05-09T06:40:02.058 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.ahJlge9b44 2024-05-09T06:40:02.074 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T06:40:02.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T06:40:02.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T06:40:02.165 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T06:40:02.195 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T06:40:02.196 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T06:40:02.198 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi001/crash 2024-05-09T06:40:02.199 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T06:40:02.234 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi006/crash 2024-05-09T06:40:02.235 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T06:40:02.260 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T06:40:02.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:40:02.280 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:40:02.353 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T06:40:02.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T06:40:02.408 INFO:tasks.ceph:Compressing logs... 2024-05-09T06:40:02.408 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:40:02.425 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:40:02.450 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79756.log 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56833.log 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78666.log 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79756.log.gz 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56833.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65886.log 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78666.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56833.log.gz 2024-05-09T06:40:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65032.log 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78666.log.gz 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65886.log.gz 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59357.log 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61155.log 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr: 91.2% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65032.log.gz 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68601.log 2024-05-09T06:40:02.454 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68643.log 2024-05-09T06:40:02.486 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41812.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.1.41812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35778.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5/var/log/ceph/ceph.log: --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.35778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.41812.log.gz 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35778.log.gz 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.35698.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gz 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.35658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35567.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.audit.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.35567.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi006.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35567.log.gz 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42258.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.3.42258.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose 91.3% -- /var/log/ceph/ceph-osd.6.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.3.42258.log.gz 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35527.log 2024-05-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.6.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35487.log 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.35487.log: /var/log/ceph/ceph-client.admin.35527.log: 88.3% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35527.log.gz 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35487.log.gz 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.35618.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-05-09T06:40:02.502 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35738.log 2024-05-09T06:40:02.514 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.35738.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35738.log.gz 2024-05-09T06:40:02.565 INFO:teuthology.orchestra.run.smithi006.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59357.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68601.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61155.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75344.log 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68601.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47027.log 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81887.log 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68643.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75344.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53948.log 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68685.log 2024-05-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53948.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81887.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75651.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53948.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68685.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77481.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75651.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51588.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77481.log: /var/log/ceph/ceph-client.admin.49640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81998.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51588.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 91.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.77481.log.gz -- 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.86361.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51588.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61921.log 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81998.log.gz 2024-05-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65844.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61921.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86361.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61921.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76878.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65844.log.gz/var/log/ceph/ceph-client.admin.49976.log: 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83139.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76878.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67048.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63404.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76878.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6%/var/log/ceph/ceph-client.admin.67048.log: -- replaced with /var/log/ceph/ceph-client.admin.83139.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71513.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63404.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67048.log.gz 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-05-09T06:40:02.618 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63404.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66248.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71513.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:gzip/var/log/ceph/ceph-client.admin.50522.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76335.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 88.2% --verbose -- /var/log/ceph/ceph-client.admin.56318.log -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66248.log: 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66248.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76335.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49000.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56146.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72363.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.56318.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49000.log.gz 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79193.log 2024-05-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi001.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.56146.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77116.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72363.log: /var/log/ceph/ceph-client.admin.79193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53516.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72363.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79193.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83560.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77116.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53643.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83560.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53516.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57211.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53643.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83560.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71003.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53643.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57211.log.gz 2024-05-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75470.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71003.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74204.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44036.log: /var/log/ceph/ceph-client.admin.75470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75470.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72526.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74204.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85291.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67638.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72526.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41098.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85291.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55975.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67638.log.gz 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58119.log 2024-05-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41098.log: /var/log/ceph/ceph-client.admin.55975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54624.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41098.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58119.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55975.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58119.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80661.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54624.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80561.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80661.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80477.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80661.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58457.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80561.log.gz 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2024-05-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80477.log: /var/log/ceph/ceph-client.admin.58457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80477.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58457.log.gz/var/log/ceph/ceph-client.admin.52712.log: 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76377.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59159.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43534.log: /var/log/ceph/ceph-client.admin.76377.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45444.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76377.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55398.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45444.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59159.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45444.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73981.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55398.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59873.log 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-05-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85207.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73981.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41369.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59873.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71208.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85207.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73346.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41369.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76765.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73346.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60144.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.71208.log.gz 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78624.log 2024-05-09T06:40:02.624 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60144.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76765.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60144.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75566.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78624.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67512.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79067.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75566.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42994.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67512.log.gz 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58033.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79067.log: /var/log/ceph/ceph-client.admin.42994.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.79067.log.gz --verbose 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.82095.log 2024-05-09T06:40:02.625 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42994.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49394.log 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82095.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58033.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41753.log 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49394.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82095.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54242.log 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49394.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54242.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41753.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47405.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54242.log.gz 2024-05-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45957.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66536.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45957.log: /var/log/ceph/ceph-client.admin.48536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57650.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45957.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68082.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66536.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68082.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57650.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49556.log 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68082.log.gz 2024-05-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83443.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49556.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83523.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83443.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62924.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83523.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61538.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62924.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83523.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57043.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62924.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61538.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61538.log.gz 2024-05-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57043.log: /var/log/ceph/ceph.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57043.log.gz -5 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72628.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68995.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72628.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61794.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72628.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84928.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68995.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61794.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84928.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-05-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51146.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56536.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78582.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58951.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56536.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51528.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78582.log: /var/log/ceph/ceph-client.0.58951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75428.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78582.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57524.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51528.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78078.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75428.log.gz 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-05-09T06:40:02.630 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78078.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57524.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74460.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78078.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 89.1% 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42558.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74460.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.58951.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61454.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50851.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67974.log 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-05-09T06:40:02.631 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61454.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66434.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50851.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82335.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66434.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67974.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56384.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66434.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48755.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67256.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.82335.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79323.log 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56384.log.gz 2024-05-09T06:40:02.632 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48755.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58247.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67256.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79323.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58247.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84886.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44049.log: /var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86925.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84886.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59915.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86403.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86925.log.gz 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66621.log 2024-05-09T06:40:02.633 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57566.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59915.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86403.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.66621.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58289.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57566.log: /var/log/ceph/ceph-client.admin.44051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75386.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57566.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86655.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58289.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53906.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75386.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86655.log.gz 2024-05-09T06:40:02.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76293.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42684.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53906.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85688.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58499.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76293.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62698.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85688.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78540.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58499.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61330.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62698.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71045.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61330.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78540.log.gz 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82661.log 2024-05-09T06:40:02.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61330.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56071.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71045.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.82661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67386.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79235.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67386.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56071.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74544.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67386.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80703.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79235.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81661.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49084.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80703.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74544.log.gz/var/log/ceph/ceph-client.admin.81661.log: 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44938.log 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80703.log.gz 2024-05-09T06:40:02.636 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81661.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73939.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49084.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77566.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44938.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80519.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73939.log: 93.5% -- replaced with /var/log/ceph/ceph-client.0.82661.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83742.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73939.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77566.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80519.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63253.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56427.log 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63253.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.83742.log.gz 2024-05-09T06:40:02.637 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66705.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56427.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63253.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50354.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56427.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66392.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50354.log: 87.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.66705.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50354.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64774.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66392.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.60567.log: /var/log/ceph/ceph-client.admin.41285.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86823.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73514.log 2024-05-09T06:40:02.638 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64774.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41285.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76461.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73514.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86823.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53239.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73514.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70161.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76461.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48350.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64353.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53239.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70161.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57253.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48350.log.gz 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62824.log 2024-05-09T06:40:02.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57253.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64353.log.gz -5 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55230.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57253.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62131.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62824.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:gzip/var/log/ceph/ceph-client.admin.55230.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62131.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.55230.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68769.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.62131.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71371.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78330.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68769.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74334.log 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71371.log.gz 2024-05-09T06:40:02.640 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81619.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78330.log: /var/log/ceph/ceph-client.admin.74334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68953.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78330.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74334.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63309.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81619.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73651.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68953.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70877.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63309.log: /var/log/ceph/ceph-client.admin.73651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69769.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63309.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73651.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.69769.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70877.log.gz 2024-05-09T06:40:02.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86529.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51188.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77737.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78288.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86529.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61250.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77737.log: /var/log/ceph/ceph-client.admin.78288.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.69769.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80829.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61250.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78288.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77737.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61250.log.gz 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85375.log 2024-05-09T06:40:02.642 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80829.log: /var/log/ceph/ceph-client.admin.41795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55272.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80829.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70819.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85375.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48708.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55272.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83097.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70819.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78456.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48708.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83097.log.gz 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55440.log 2024-05-09T06:40:02.643 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78456.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85054.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73075.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75202.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55440.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85054.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73075.log.gz/var/log/ceph/ceph-client.admin.75202.log: 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57001.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75202.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61580.log 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-05-09T06:40:02.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54848.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57001.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70269.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61580.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54848.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76603.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70269.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86885.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52896.log: /var/log/ceph/ceph-client.admin.76603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82401.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76603.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74870.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86885.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46775.log 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.74870.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82401.log.gz 2024-05-09T06:40:02.645 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86487.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60525.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46775.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46733.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86487.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53558.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60525.log: /var/log/ceph/ceph-client.admin.46733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60525.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53558.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46733.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48916.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53558.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73117.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83486.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2%/var/log/ceph/ceph-client.admin.48916.log: -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62089.log 2024-05-09T06:40:02.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48916.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73117.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56596.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83486.log: /var/log/ceph/ceph-client.admin.62089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.83486.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56596.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62089.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64437.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.74870.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56596.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41879.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64437.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.84105.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64437.log.gz 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43408.log 2024-05-09T06:40:02.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84105.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84105.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43408.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41879.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79005.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43408.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.46607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73812.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79005.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78792.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79005.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73812.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49352.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78792.log: gzip/var/log/ceph/ceph-client.admin.44400.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78792.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49352.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-05-09T06:40:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80603.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.49352.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.70311.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50978.log: /var/log/ceph/ceph-client.admin.80603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76419.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80603.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67130.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51020.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76419.log: 91.0%/var/log/ceph/ceph-client.admin.67130.log: -- replaced with /var/log/ceph/ceph-client.admin.70311.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53990.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76419.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67130.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70569.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77826.log 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53990.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51020.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53990.log.gz 2024-05-09T06:40:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61752.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77826.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.70569.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76687.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77826.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58415.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61752.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76687.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61752.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73159.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76687.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72484.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58415.log: /var/log/ceph/ceph-client.admin.73159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50606.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58415.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73159.log.gz 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72442.log 2024-05-09T06:40:02.650 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72484.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58163.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59507.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50606.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72442.log: /var/log/ceph/ceph-client.admin.58163.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72442.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59615.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58163.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78162.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59507.log: /var/log/ceph/ceph-client.admin.59615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59507.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78162.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59615.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48434.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78162.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69355.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79374.log 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-05-09T06:40:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48434.log: /var/log/ceph/ceph-client.admin.69355.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48434.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46649.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69355.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59182.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79374.log: /var/log/ceph/ceph-client.admin.46649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41499.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79374.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59182.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46649.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53100.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59182.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44286.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41499.log: /var/log/ceph/ceph-client.admin.53100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80907.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41499.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53100.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67596.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86013.log 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80907.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44286.log.gz 2024-05-09T06:40:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35739.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67596.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.80907.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76561.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35739.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86013.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35739.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79864.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75628.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44047.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76561.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79864.log: /var/log/ceph/ceph-client.admin.75628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79864.log.gz 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83808.log 2024-05-09T06:40:02.653 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75628.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84537.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44047.log: /var/log/ceph/ceph-client.admin.83808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44047.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.84537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59807.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78708.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.83808.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72297.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78708.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59807.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-05-09T06:40:02.654 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40921.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72297.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73033.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40921.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40921.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46985.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.84537.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78372.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75302.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73033.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78372.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46985.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66747.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78372.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53780.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.47447.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75302.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66747.log: /var/log/ceph/ceph-client.admin.53780.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66747.log.gz 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57825.log 2024-05-09T06:40:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78204.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53780.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47447.log: /var/log/ceph/ceph-client.admin.57825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75967.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57825.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78204.log: /var/log/ceph/ceph-client.0.75967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64043.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78204.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42868.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57356.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66873.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64043.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86055.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42868.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73731.log 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57356.log.gz 2024-05-09T06:40:02.656 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86193.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66873.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69313.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86055.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73731.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78246.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86193.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69313.log.gz/var/log/ceph/ceph-client.admin.78246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46455.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78246.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.46455.log: /var/log/ceph/ceph-client.admin.76645.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46455.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.84151.log 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-05-09T06:40:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73304.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76645.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.84151.log: --verbose -- /var/log/ceph/ceph-client.admin.58691.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73304.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84151.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-osd.1.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0%/var/log/ceph/ceph-client.admin.58691.log: -- replaced with /var/log/ceph/ceph-client.admin.73304.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46859.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58691.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.75967.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46859.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46859.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-05-09T06:40:02.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.53022.log: --verbose -- /var/log/ceph/ceph-client.admin.44046.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49934.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85165.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44046.log: -- /var/log/ceph/ceph-client.admin.72907.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85165.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44046.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85971.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85165.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.72907.log: --verbose -- /var/log/ceph/ceph-client.admin.72038.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85971.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72907.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60228.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85971.log.gz 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79416.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72038.log: /var/log/ceph/ceph-client.admin.60228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83013.log 2024-05-09T06:40:02.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79416.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60228.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% 90.7% -- replaced with /var/log/ceph/ceph-client.admin.72038.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79416.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65206.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83013.log.gz/var/log/ceph/ceph-client.admin.51486.log: 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64840.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65206.log: /var/log/ceph/ceph-client.admin.64840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54667.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58331.log: 88.4% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64840.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65206.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62966.log 2024-05-09T06:40:02.660 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54667.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73472.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71621.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62966.log: /var/log/ceph/ceph-client.admin.73472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4%/var/log/ceph/ceph-client.admin.71621.log: -- replaced with /var/log/ceph/ceph-client.admin.62966.log.gz 87.8% 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73472.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71621.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45835.log/var/log/ceph/ceph-client.admin.60102.log: 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.0.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78750.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55020.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78750.log: /var/log/ceph/ceph-client.0.45835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78750.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.45835.log.gz 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68827.log 2024-05-09T06:40:02.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51924.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68827.log: /var/log/ceph/ceph-client.admin.55020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55020.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68827.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52938.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61668.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52938.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.51924.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52938.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84063.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74120.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61668.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84063.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84063.log.gz 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63362.log 2024-05-09T06:40:02.662 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77439.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74120.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77439.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63362.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68869.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77439.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43450.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53693.log: /var/log/ceph/ceph-client.admin.68869.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86277.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68869.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43450.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43450.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86697.log 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86277.log: /var/log/ceph/ceph-client.admin.43715.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86277.log.gz 2024-05-09T06:40:02.663 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86697.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86697.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64395.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42826.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64948.log: /var/log/ceph/ceph-client.admin.64395.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64395.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67848.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50060.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42826.log: /var/log/ceph/ceph-client.admin.67848.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42826.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67848.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86571.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50060.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50060.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55062.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76879.log 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86571.log: /var/log/ceph/ceph-client.admin.55062.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86571.log.gz 2024-05-09T06:40:02.664 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55062.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67680.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76879.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.76879.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79109.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67680.log: /var/log/ceph/ceph-client.admin.79109.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67680.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79109.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60693.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82293.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60693.log: /var/log/ceph/ceph-client.admin.48392.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60693.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65664.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82293.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82293.log.gz 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85501.log 2024-05-09T06:40:02.665 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65508.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65664.log: /var/log/ceph/ceph-client.admin.85501.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.65664.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85501.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76519.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.65508.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.65508.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60819.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76519.log: /var/log/ceph/ceph-client.admin.60819.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76519.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66308.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41837.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79458.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66308.log: /var/log/ceph/ceph-client.admin.79458.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66308.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79458.log.gz 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2024-05-09T06:40:02.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43240.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60441.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69443.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50480.log: /var/log/ceph/ceph-client.admin.60441.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60441.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69443.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.69443.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85543.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60399.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43711.log: /var/log/ceph/ceph-client.admin.85543.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85543.log.gzgzip 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60399.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60399.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43712.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86781.log 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51318.log: /var/log/ceph/ceph-client.admin.43712.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-05-09T06:40:02.667 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43712.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48104.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86781.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86781.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64127.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45190.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48104.log: /var/log/ceph/ceph-client.admin.64127.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48104.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64127.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82151.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45190.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79546.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44770.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82151.log: /var/log/ceph/ceph-client.admin.79546.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82151.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79546.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44770.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44770.log.gz 2024-05-09T06:40:02.668 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53474.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50312.log: /var/log/ceph/ceph-client.admin.45022.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53474.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53474.log.gz 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50438.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55146.log: /var/log/ceph/ceph-client.admin.50438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62005.log 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50438.log.gz 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49126.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-05-09T06:40:02.669 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85645.log 2024-05-09T06:40:02.670 INFO:teuthology.orchestra.run.smithi006.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T06:40:02.809 INFO:teuthology.orchestra.run.smithi006.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T06:40:02.886 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46817.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62005.log: /var/log/ceph/ceph-client.admin.85645.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62005.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59723.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85645.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46817.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46817.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44043.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41457.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59723.log: /var/log/ceph/ceph-client.admin.44043.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44043.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59723.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74250.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41457.log: /var/log/ceph/ceph-client.admin.74250.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41457.log.gz 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74250.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56233.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56018.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52796.log: /var/log/ceph/ceph-client.admin.42366.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68016.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56018.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.56018.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43156.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68016.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68016.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43716.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54890.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43156.log: /var/log/ceph/ceph-client.admin.43716.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57085.log 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54890.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54890.log.gz 2024-05-09T06:40:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43713.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69509.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57085.log: /var/log/ceph/ceph-client.admin.43713.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57085.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43713.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52670.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58625.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69509.log: /var/log/ceph/ceph-client.admin.52670.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69509.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52670.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44403.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58625.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58625.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46691.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66038.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44403.log: /var/log/ceph/ceph-client.admin.46691.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44403.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66038.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66038.log.gz 2024-05-09T06:40:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83597.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68342.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49226.log: /var/log/ceph/ceph-client.admin.83597.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83597.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.68342.log: 91.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.68342.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41627.log: /var/log/ceph/ceph-client.admin.47279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56959.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44686.log 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2%/var/log/ceph/ceph-client.admin.56959.log: -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-05-09T06:40:02.890 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56959.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46901.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44686.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79504.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46901.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46901.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83281.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79504.log: /var/log/ceph/ceph-client.admin.83281.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79504.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83281.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63092.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78414.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61963.log 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78414.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63092.log.gz 2024-05-09T06:40:02.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72949.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78414.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61836.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72949.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61963.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72949.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52754.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61836.log: /var/log/ceph/ceph-client.admin.67428.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61836.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49310.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52754.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35372.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67428.log.gz 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2024-05-09T06:40:02.892 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35372.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49310.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.35372.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73262.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44406.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64269.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58373.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64269.log: /var/log/ceph/ceph-client.admin.73262.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73262.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64269.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65140.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58373.log: /var/log/ceph/ceph-client.admin.48494.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58373.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-05-09T06:40:02.893 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64085.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65140.log: /var/log/ceph/ceph-client.admin.51232.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65140.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43324.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85585.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64085.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70777.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85585.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43324.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85585.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70010.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57169.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70777.log: /var/log/ceph/ceph-client.admin.70010.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70777.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70010.log.gz 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.42600.log 2024-05-09T06:40:02.894 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69260.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57169.log: /var/log/ceph/ceph-client.admin.42600.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57169.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41711.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72189.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69260.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73219.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72189.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41711.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79630.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72189.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67764.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73219.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79630.log.gz 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86613.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74502.log 2024-05-09T06:40:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67764.log: /var/log/ceph/ceph-client.admin.86613.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67764.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86613.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81845.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74502.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74502.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64666.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81845.log: /var/log/ceph/ceph-client.admin.64666.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81845.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64666.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49850.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63450.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85249.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45106.log: /var/log/ceph/ceph-client.admin.63450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63450.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55314.log 2024-05-09T06:40:02.896 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85249.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59441.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50228.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55314.log: /var/log/ceph/ceph-client.admin.59441.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55314.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71310.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59441.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67932.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50228.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68727.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67932.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67932.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68727.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.71310.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68727.log.gz 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86739.log 2024-05-09T06:40:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41585.log 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60651.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86739.log.gz 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71185.log 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62782.log 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41585.log: /var/log/ceph/ceph-client.admin.71185.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41585.log.gz 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71185.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44404.log 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62782.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62782.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71467.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44404.log: /var/log/ceph/ceph-client.admin.44052.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44404.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47069.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71467.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29344.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47069.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71467.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47069.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph.tmp-client.admin.29344.log: -- /var/log/ceph/ceph-client.admin.69091.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29344.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61710.log 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41945.log: /var/log/ceph/ceph-client.admin.69091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69091.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-05-09T06:40:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61496.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83055.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61710.log.gz 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64211.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61496.log.gz 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49766.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64211.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83055.log.gz 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64211.log.gz 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72739.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83892.log 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72739.log: /var/log/ceph/ceph-client.admin.49766.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49766.log.gz 2024-05-09T06:40:02.900 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72739.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48188.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.83892.log: -- /var/log/ceph/ceph-client.admin.54978.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48188.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48188.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42768.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.83892.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86235.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42768.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54978.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42768.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82247.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84009.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82247.log: /var/log/ceph/ceph-client.admin.86235.log: 88.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86235.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82247.log.gz 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73388.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56917.log 2024-05-09T06:40:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84009.log: /var/log/ceph/ceph-client.admin.73388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84009.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73388.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52250.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50144.log: /var/log/ceph/ceph-client.admin.56917.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56917.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77803.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56875.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.52250.log: /var/log/ceph/ceph-client.admin.77803.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77803.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54932.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56875.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56875.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66164.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54932.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54932.log.gz 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53313.log 2024-05-09T06:40:02.902 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66164.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66164.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60186.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53313.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53313.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54797.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60186.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60186.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74610.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54797.log: 93.1% -- replaced with /var/log/ceph/ceph-client.0.52250.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54797.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84259.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.74610.log: /var/log/ceph/ceph-client.admin.67806.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74610.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44285.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84259.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67806.log.gz 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2024-05-09T06:40:02.903 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40829.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44285.log: /var/log/ceph/ceph-client.admin.43492.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44285.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85417.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67722.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40829.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81577.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67722.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85417.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67722.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75160.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.81577.log: /var/log/ceph/ceph-client.admin.41201.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81577.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75160.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-05-09T06:40:02.904 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80124.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.80124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56680.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47237.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44728.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56680.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56680.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71268.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44728.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71268.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.71268.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64311.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47615.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66476.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64311.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64311.log.gz 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56638.log 2024-05-09T06:40:02.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66476.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66476.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.80124.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48620.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70961.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56638.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86319.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2%/var/log/ceph/ceph-client.admin.70961.log: -- replaced with /var/log/ceph/ceph-client.admin.48620.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51672.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70961.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75118.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86319.log: /var/log/ceph/ceph-client.admin.51672.log: 88.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86319.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51672.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80787.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47657.log 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75118.log: /var/log/ceph/ceph-client.admin.80787.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75118.log.gz 2024-05-09T06:40:02.906 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80787.log.gz 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58583.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43366.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50396.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43366.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58583.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54709.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43366.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63008.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54709.log: /var/log/ceph/ceph-client.admin.63008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44854.log 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50396.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47962.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54709.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-05-09T06:40:02.907 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.54158.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75707.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44854.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78035.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54158.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63188.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75707.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77297.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78035.log: /var/log/ceph/ceph-client.admin.63188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77653.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77297.log: 88.1% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78035.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63188.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77297.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57608.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77653.log: gzip/var/log/ceph/ceph-client.admin.57608.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77653.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57608.log.gz 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74418.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54459.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74162.log 2024-05-09T06:40:02.908 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74418.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50767.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54459.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69193.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74162.log: /var/log/ceph/ceph-client.admin.50767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62740.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69193.log: 87.8% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74162.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50767.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73430.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: 91.2%/var/log/ceph/ceph-client.admin.62740.log: -- replaced with /var/log/ceph/ceph-client.admin.69193.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85459.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62740.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73430.log: /var/log/ceph/ceph-client.admin.85459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77565.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42642.log: 88.6% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85459.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73430.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 89.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.60060.log 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-05-09T06:40:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83239.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59235.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60060.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77565.log.gz/var/log/ceph/ceph-client.admin.83239.log: 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60060.log.gz 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83239.log.gz 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49184.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66206.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49184.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.59235.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44812.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49184.log.gz 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57127.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66206.log: /var/log/ceph/ceph-client.admin.44812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79714.log 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57127.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66206.log.gz 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2024-05-09T06:40:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57127.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79714.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67298.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49514.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67298.log: /var/log/ceph/ceph-client.admin.49514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72781.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67298.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49514.log.gz/var/log/ceph/ceph-client.admin.48799.log: 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48958.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49042.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72781.log: /var/log/ceph/ceph-client.admin.48958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69156.log 2024-05-09T06:40:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49042.log: 88.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72781.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48958.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49042.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69944.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52628.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69156.log: gzip/var/log/ceph/ceph-client.admin.69944.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5%/var/log/ceph/ceph-client.admin.52628.log: -- replaced with /var/log/ceph/ceph-client.admin.69156.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52628.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69944.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66968.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51714.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79279.log 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66968.log: /var/log/ceph/ceph-client.admin.51714.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66968.log.gz 2024-05-09T06:40:02.912 INFO:teuthology.orchestra.run.smithi001.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51714.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44644.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74292.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79279.log: /var/log/ceph/ceph-client.admin.44644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60357.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74292.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79279.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44644.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74292.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55188.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41669.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60357.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82971.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55188.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62197.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1%/var/log/ceph/ceph-client.admin.82971.log: -- replaced with /var/log/ceph/ceph-client.admin.41669.log.gz 2024-05-09T06:40:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82971.log.gzgzip 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62197.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72586.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57906.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67470.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72586.log: /var/log/ceph/ceph-client.admin.57906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68911.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67470.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57906.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72586.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67470.log.gz 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53737.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.68911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70203.log 2024-05-09T06:40:02.914 INFO:teuthology.orchestra.run.smithi001.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68911.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66789.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53737.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72823.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50894.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66789.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70203.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77199.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66789.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50894.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72823.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55356.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50894.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77199.log: /var/log/ceph/ceph-client.admin.55356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55356.log.gz/var/log/ceph/ceph-client.admin.50186.log: 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.77199.log.gz 2024-05-09T06:40:02.915 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72104.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54284.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67342.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54032.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54284.log: /var/log/ceph/ceph-client.admin.67342.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.72104.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54284.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67342.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50102.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59291.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50102.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54032.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0%/var/log/ceph/ceph-client.admin.59291.log: -- replaced with /var/log/ceph/ceph-client.admin.50102.log.gz 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77393.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86098.log 2024-05-09T06:40:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77393.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59291.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77393.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57990.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86098.log: /var/log/ceph/ceph-client.admin.51104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43709.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86098.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80973.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57990.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83653.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43709.log: /var/log/ceph/ceph-client.admin.80973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83653.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43709.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80973.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57398.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85772.log 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45148.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.83653.log.gz 2024-05-09T06:40:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57398.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77139.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57398.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77139.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85772.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64169.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83323.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.77139.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61113.log: /var/log/ceph/ceph-client.admin.64169.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85730.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64169.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86445.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83323.log: /var/log/ceph/ceph-client.admin.85730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78120.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.86445.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83323.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85730.log.gz 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51630.log 2024-05-09T06:40:02.918 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86445.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60735.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61878.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.51630.log: 89.1% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51630.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78120.log.gz/var/log/ceph/ceph-client.admin.60735.log: 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60735.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77050.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71123.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61878.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61412.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62866.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.77050.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71123.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75244.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61412.log.gz 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.62866.log: /var/log/ceph/ceph-client.admin.75244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48146.log 2024-05-09T06:40:02.919 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6%/var/log/ceph/ceph-client.admin.51360.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62866.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75244.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74376.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65098.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85814.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74376.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48146.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74376.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65098.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65098.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.2.46333.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77241.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85814.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.2.46333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83197.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73896.log 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77241.log: 91.1% -- replaced with /var/log/ceph/ceph-client.2.46333.log.gz/var/log/ceph/ceph-client.admin.83197.log: 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77241.log.gz 2024-05-09T06:40:02.920 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83197.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67554.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73896.log: /var/log/ceph/ceph-client.admin.50686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59981.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67554.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73896.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86967.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67554.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85012.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66579.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7%/var/log/ceph/ceph-client.admin.86967.log: -- replaced with /var/log/ceph/ceph-client.admin.59981.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85012.log: 87.9%gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.61029.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85012.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86967.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70919.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66579.log.gz 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.61029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-05-09T06:40:02.921 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49808.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70919.log: /var/log/ceph/ceph-client.admin.60861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49808.log: 89.2% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70919.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49808.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.53276.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74025.log 88.0% 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70735.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: 89.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.79672.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74025.log: /var/log/ceph/ceph-client.admin.70735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-05-09T06:40:02.922 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7%/var/log/ceph/ceph-client.admin.79672.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74025.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70735.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79672.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73854.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57440.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47153.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.73854.log: 87.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57440.log: -- replaced with /var/log/ceph/ceph-client.admin.73854.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59657.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57440.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62457.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59657.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.47153.log.gz --verbose 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.52980.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.62457.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59657.log.gz 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65928.log 2024-05-09T06:40:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79798.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52980.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65928.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65928.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72991.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70527.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.62457.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55104.log 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79798.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71555.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72991.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70527.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.55104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45316.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55104.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71555.log.gz 2024-05-09T06:40:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47573.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45316.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58075.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50809.log: /var/log/ceph/ceph-client.admin.58075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mds.a.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58075.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81719.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48272.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49892.log: /var/log/ceph/ceph-client.admin.81719.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81719.log.gz 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79151.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43036.log 2024-05-09T06:40:02.925 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48272.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.79151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43036.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79151.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43036.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62047.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84193.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47111.log: /var/log/ceph/ceph-client.admin.62047.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62047.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81493.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80435.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81493.log: /var/log/ceph/ceph-client.admin.84193.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81493.log.gz 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53864.log 2024-05-09T06:40:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.80435.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.84193.log.gz 87.6% 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80435.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66350.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64515.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53864.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81803.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64515.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66350.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64515.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48230.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81803.log: /var/log/ceph/ceph-client.admin.43714.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81803.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70693.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7%/var/log/ceph/ceph-client.admin.48230.log: -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65800.log 2024-05-09T06:40:02.927 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65800.log: /var/log/ceph/ceph-client.admin.70693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70693.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65800.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63050.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56145.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43705.log: /var/log/ceph/ceph-client.admin.63050.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63050.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83719.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81761.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56145.log: /var/log/ceph/ceph-client.admin.83719.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83719.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56145.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63492.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66831.log 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.81761.log: /var/log/ceph/ceph-client.admin.63492.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81761.log.gz 2024-05-09T06:40:02.928 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72255.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 91.2%/var/log/ceph/ceph-client.admin.66831.log: -- replaced with /var/log/ceph/ceph-client.admin.63492.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66831.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83365.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.72255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66080.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72255.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.83365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67214.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66080.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83365.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66080.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77993.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57482.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67214.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.77993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78887.log 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57482.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77993.log.gz 2024-05-09T06:40:02.929 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57482.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51756.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65757.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78887.log: /var/log/ceph/ceph-client.admin.51756.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78887.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70377.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51756.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70461.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65757.log: /var/log/ceph/ceph-client.admin.70377.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65757.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50564.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70377.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50564.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50564.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70461.log.gz 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60483.log 2024-05-09T06:40:02.930 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60777.log: /var/log/ceph/ceph-client.admin.43114.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz/var/log/ceph/ceph-client.admin.60483.log: 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60483.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64581.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49724.log: /var/log/ceph/ceph-client.admin.45232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85333.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.85333.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85333.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64581.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58205.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70095.log 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-05-09T06:40:02.931 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67172.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70095.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58205.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64732.log 91.3% -- replaced with /var/log/ceph/ceph-client.admin.70095.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73556.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67172.log: /var/log/ceph/ceph-client.admin.64732.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67172.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76966.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0%/var/log/ceph/ceph-client.admin.73556.log: -- replaced with /var/log/ceph/ceph-client.admin.64732.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63558.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73556.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.76966.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.76966.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61624.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63558.log: /var/log/ceph/ceph-client.admin.61624.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63558.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61624.log.gz 2024-05-09T06:40:02.932 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81535.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48842.log: /var/log/ceph/ceph-client.admin.81535.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56722.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81535.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44602.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.56722.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56722.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83850.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54200.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44602.log: /var/log/ceph/ceph-client.admin.83850.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44602.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67890.log 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54200.log.gz 2024-05-09T06:40:02.933 INFO:teuthology.orchestra.run.smithi001.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.83850.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69133.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.67890.log: /var/log/ceph/ceph-client.admin.69133.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67890.log.gz 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78498.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69133.log.gz 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49472.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.78498.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78498.log.gz 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52854.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48062.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49472.log.gz 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.52854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80745.log 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48062.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52854.log.gz 2024-05-09T06:40:02.934 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48062.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.80745.log: -- /var/log/ceph/ceph-client.admin.75076.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80745.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47699.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.75076.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75076.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71881.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55766.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47699.log: /var/log/ceph/ceph-client.0.71881.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47699.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71425.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.55766.log: 92.3% -- replaced with /var/log/ceph/ceph-client.0.71881.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69401.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.71425.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71425.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64882.log 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.69401.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69401.log.gz 2024-05-09T06:40:02.935 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84970.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.64882.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.55766.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 90.7% -- replaced with /var/log/ceph/ceph-client.admin.64882.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72865.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58541.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.84970.log: /var/log/ceph/ceph-client.admin.72865.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84970.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57948.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72865.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.58541.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86151.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58541.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57948.log: /var/log/ceph/ceph-client.admin.86151.log: 87.3% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86151.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57948.log.gz 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57745.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60609.log 2024-05-09T06:40:02.936 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.57745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41327.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60609.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57745.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60609.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77351.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53202.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41327.log: /var/log/ceph/ceph-client.admin.77351.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41327.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77351.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65248.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66122.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.65248.log: /var/log/ceph/ceph-client.admin.53202.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65248.log.gz 89.6% 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53202.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54540.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74069.log 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.66122.log: /var/log/ceph/ceph-client.admin.54540.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66122.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54540.log.gz 2024-05-09T06:40:02.937 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79588.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48663.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.74069.log: /var/log/ceph/ceph-client.admin.79588.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74069.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81233.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79588.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48663.log: /var/log/ceph/ceph-client.0.81233.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48663.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82205.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.60945.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53822.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.82205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82205.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63818.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.53822.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53822.log.gz 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63230.log 2024-05-09T06:40:02.938 INFO:teuthology.orchestra.run.smithi001.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.81233.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.63818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54753.log 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.63230.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63230.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70651.log 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.54753.log: 91.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54753.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.0.63818.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66663.log 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59549.log 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.70651.log: /var/log/ceph/ceph-client.admin.66663.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70651.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66663.log.gz 2024-05-09T06:40:02.939 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.59549.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59549.log.gz 2024-05-09T06:40:02.940 INFO:teuthology.orchestra.run.smithi006.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T06:40:03.003 INFO:teuthology.orchestra.run.smithi001.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T06:40:03.707 INFO:teuthology.orchestra.run.smithi001.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T06:40:03.770 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T06:40:03.794 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T06:40:03.846 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T06:40:03.865 INFO:teuthology.orchestra.run.smithi001.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T06:40:03.938 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T06:40:03.940 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-09T06:40:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m1.477s 2024-05-09T06:40:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m6.207s 2024-05-09T06:40:03.940 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m0.232s 2024-05-09T06:40:03.979 INFO:teuthology.orchestra.run.smithi001.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T06:40:04.083 INFO:teuthology.orchestra.run.smithi001.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T06:40:04.289 INFO:teuthology.orchestra.run.smithi001.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T06:40:04.292 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T06:40:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m1.852s 2024-05-09T06:40:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m7.785s 2024-05-09T06:40:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m0.775s 2024-05-09T06:40:04.292 INFO:tasks.ceph:Archiving logs... 2024-05-09T06:40:04.292 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi001/log 2024-05-09T06:40:04.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:40:05.913 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi006/log 2024-05-09T06:40:05.914 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:40:06.785 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T06:40:06.795 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T06:40:06.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:40:06.798 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:40:06.885 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T06:40:06.894 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T06:40:06.895 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:40:06.898 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:40:06.912 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:40:06.915 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +1629us[+1625us] +/- 65ms 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -836us[ -840us] +/- 56ms 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -770us[ -770us] +/- 75ms 2024-05-09T06:40:06.917 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +1632us[+1641us] +/- 65ms 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -853us[ -853us] +/- 56ms 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -776us[ -768us] +/- 75ms 2024-05-09T06:40:06.920 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:40:06.921 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T06:40:06.930 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T06:40:06.931 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T06:40:06.942 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:40:07.056 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:40:07.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:40:07.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:40:07.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:40:07.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:40:07.192 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T06:40:07.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T06:40:07.250 INFO:teuthology.task.internal:Duration was 1215.368063 seconds 2024-05-09T06:40:07.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T06:40:07.260 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T06:40:07.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:40:07.262 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:40:07.301 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:40:07.307 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:40:07.525 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T06:40:07.526 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T06:40:07.526 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:40:07.588 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T06:40:07.588 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:40:07.618 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T06:40:07.618 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:40:07.630 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:40:07.687 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T06:40:07.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:40:07.698 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:40:07.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T06:40:07.848 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T06:40:07.849 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:40:07.875 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:40:07.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T06:40:07.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:40:07.917 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:40:07.939 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-05-09T06:40:07.965 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-05-09T06:40:07.979 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:40:08.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:40:08.009 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:40:08.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:40:08.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T06:40:08.050 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T06:40:08.051 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi001 2024-05-09T06:40:08.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:40:08.096 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698532/remote/smithi006 2024-05-09T06:40:08.096 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:40:08.130 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T06:40:08.130 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:40:08.133 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:40:08.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T06:40:08.192 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T06:40:08.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T06:40:08.206 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T06:40:08.206 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:40:08.208 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:40:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:40 /home/ubuntu/cephtest 2024-05-09T06:40:08.237 INFO:teuthology.orchestra.run.smithi006.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:40 /home/ubuntu/cephtest 2024-05-09T06:40:08.238 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T06:40:08.250 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T06:40:08.280 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1215.3680627346039 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T06:40:08.280 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:40:08.370 INFO:teuthology.run:pass